A "recipe" is a codified procedure for one kind of work: steps, tools, data sources and verification checks written by a specialist and loaded into an agent harness. This brief summarizes what current published evidence does and does not support about recipe-driven harnesses.
Supported: harness and interface design change outcomes on the same model
SWE-agent resolved 12.47% of SWE-bench tasks with GPT-4 Turbo versus 3.8% for a prior non-interactive retrieval system; ablations attribute the gain to interface design (https://arxiv.org/abs/2405.15793). Voyager's verified, reusable skill library produced 3.3x more discoveries and zero-shot generalization where recipe-less baselines made little progress - evidence from a game environment, adjacent to business settings (https://arxiv.org/abs/2305.16291).
Supported with a correction: predictable and verifiable, not deterministic
Predefined code paths make orchestration predictable (Anthropic, Building Effective Agents, https://www.anthropic.com/engineering/building-effective-agents). Output format can be pinned: OpenAI reports 100% schema adherence for Structured Outputs on its own evaluations, which is format compliance, not semantic correctness (https://openai.com/index/introducing-structured-outputs-in-the-api/). Served model output can vary even at temperature zero (Thinking Machines, https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/). The robust pattern is deterministic checks wrapped around a stochastic model (Martin Fowler, https://martinfowler.com/articles/harness-engineering.html). Recipes alone do not guarantee deterministic semantic answers, and format constraints guarantee compliance, not correctness.
Unproven: net token savings
Recipes remove repeated explanation and retries but add their own injected context. Anthropic reports agents use roughly 4x the tokens of chat and multi-agent systems about 15x; token usage, tool-call count and model choice jointly explained 95% of BrowseComp performance variance (https://www.anthropic.com/engineering/built-multi-agent-research-system). TheAgentCompany benchmarked the best system at 24% of 175 workplace tasks at roughly $6.3 per task (https://arxiv.org/abs/2412.14161). AI Agents That Matter shows cost-controlled evaluation and accuracy-cost trade-offs, and warns about benchmark overfitting (https://arxiv.org/abs/2407.01502). We found no published controlled with/without-recipes comparison on matched tasks in this research pass.
Practical notes
"Generic" does not mean recipe-less: general harnesses can load specialist skills. Recipes cover known cases and degrade as environments change; maintenance cost is under-measured. General harnesses remain better for novel or one-off tasks. Vertical software investors argue the same direction from the market side (Bessemer Venture Partners, https://www.bvp.com/atlas/part-i-the-future-of-ai-is-vertical).
Sources
Retrieved October 27, 2026.
- https://arxiv.org/abs/2405.15793
- https://arxiv.org/abs/2305.16291
- https://arxiv.org/abs/2412.14161
- https://arxiv.org/abs/2407.01502
- https://www.anthropic.com/engineering/built-multi-agent-research-system
- https://www.anthropic.com/engineering/building-effective-agents
- https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/
- https://martinfowler.com/articles/harness-engineering.html
- https://www.bvp.com/atlas/part-i-the-future-of-ai-is-vertical
- https://openai.com/index/introducing-structured-outputs-in-the-api/