Repository instructions reduce agent runtime and output tokens
Source → Observed → Interpretation → Model implication
On the Impact of AGENTS.md Files on the Efficiency of AI Coding Agents
View source →The authors studied 124 merged pull-request tasks from 10 repositories using OpenAI Codex with gpt-5.2-codex. Each task was run against the same pre-merge repository snapshot under paired conditions that differed only in whether the root AGENTS.md file was present.
With AGENTS.md present, median wall-clock completion time decreased from 98.57 seconds to 70.34 seconds, a 28.64% reduction. Mean completion time decreased from 162.94 seconds to 129.91 seconds, a 20.27% reduction. The wall-clock difference was statistically significant under a Wilcoxon signed-rank test at p<0.05.
Median output-token consumption decreased from 2,925 to 2,440 tokens, a 16.58% reduction. Mean output-token consumption decreased from 5,744.81 to 4,591.46 tokens, a 20.08% reduction. The output-token difference was also statistically significant at p<0.05.
Median input tokens, cached input tokens, and total tokens were slightly higher with AGENTS.md, and the paper does not mark differences in those metrics as statistically significant. The authors interpret the output-token result as being influenced by a small number of very high-cost runs rather than a uniform reduction across all tasks.
The experiments ran in fresh, isolated Docker environments without reused task state. The study did not comprehensively evaluate semantic correctness or functional equivalence; a manual sanity check of 50 randomly sampled tasks established only that outputs were non-empty, non-trivial, and consistent with the intended task rather than aborted or random changes.
When a pull request lacked a usable natural-language description, the authors generated a GitHub-issue-style task statement with a local language model given the human-authored pull-request diff and the pre-merge repository tree. The same generated task statement was used in both paired conditions.
A version-controlled repository instruction file is an environmental condition rather than a model change. Holding the task, repository state and agent configuration constant while adding that context changed two operational costs: completion time and generated output. This makes Context an experimentally varied part of agent fitness and connects it to Economics, while the mixed token results caution against treating additional context as a uniform reduction in resource use.
SUPPORTS. The paired results support the Selection proposition that engineering environments can measurably affect agent operation. In this bounded study, repository-level instructions were associated with lower runtime and output generation cost even though the model and task were held constant. The evidence supports an operational effect on the fitness denominator; because correctness was not evaluated, it does not establish improved useful outcome or overall agentic-system fitness.
What this does not establish
- The study does not establish that AGENTS.md improves correctness, maintainability, alignment with developer intent, or equivalence to the human-authored pull requests.
- It does not establish that every token metric improves: median input, cached-input, and total-token consumption were slightly higher with AGENTS.md, and their differences were not reported as statistically significant.
- The experiment used one agent configuration and model. The authors identify agent stochasticity, other models and agent systems, and broader repository samples as threats to generality and directions for replication.
- Tasks were restricted to small, merged, code-only pull requests. The study does not establish that the effect extends to larger refactorings, multi-module changes, documentation work, or other development tasks.
- The paired design holds task input constant between conditions, but the paper does not establish that the results would be the same with naturally authored issue descriptions; task statements were generated from the target pull-request diff when descriptions were insufficient.
- The binary comparison does not identify which AGENTS.md content, structure, specificity, or workflow guidance caused the observed differences.
- It does not report organizational adoption, concurrent agent operation, or population-scale results and establishes nothing about 1K-agent scale.
When correctness and repeated stochastic runs are measured alongside cost, which properties of repository-level instructions improve useful outcomes across models and larger engineering tasks rather than only reducing runtime or output generation?