Multi-agent cooperation rarely reached the accuracy-energy-latency frontier
Source → Observed → Interpretation → Model implication
Engineering Sustainable Agents: A Systematic Comparison of Agentic LLMs for Developer Workflows
View source →The study evaluates code generation, technical-debt identification, code- vulnerability detection, log parsing, and log analysis using HumanEval, MLCQ, PrimeVul, and LogHub HDFS datasets.
The experimental ladder compares a non-agentic single query, a single-agent setup, a dual-agent configuration, and a multi-agent workflow using an AG2/AutoGen-style orchestration framework.
The core factorial design contains 1,800 runs across five tasks, four agent configurations, two prompt strategies, five models, three hardware platforms, and three repetitions. Forty additional GPT-OSS runs bring the reported total to 1,840.
The evaluated models are instruction-tuned open-weight Gemma, CodeGemma, Qwen, Qwen-Coder, DeepSeek-Coder, and GPT-OSS variants. The hardware ranges from a developer workstation to two GPU servers.
The protocol fixes temperature at zero, restarts Ollama for each run, disables orchestration caching, attributes process-level energy with CodeCarbon, and applies Friedman and Wilcoxon tests with matched-pairs effect sizes.
Averaged across the study, multi-agent configurations consumed 6.36 times as much energy and took 6.07 times as long as the non-agentic baseline. The largest reported slowdown for a task-hardware pair was 160 times.
The non-agentic configuration achieved the highest average accuracy in four of five tasks. Multi-agent configuration achieved the highest average result for vulnerability detection.
Of 66 configurations on the three-dimensional accuracy, energy, and latency Pareto front, 36 were non-agentic, 23 single-agent, six dual-agent, and one multi-agent. Non-agentic and single-agent configurations therefore accounted for 59 of 66 Pareto-optimal results.
Model and prompt effects varied by task rather than producing one global winner. The paper reports that prompt strategy alone changed log-parsing accuracy from 3.4% to 54.3% in one comparison.
Hardware differences became more important for heavier configurations: a high-end server was up to 3.4 times faster than a workstation for multi-agent work. Energy and latency were strongly correlated in log-log space, with a reported Pearson correlation of 0.987.
The study makes agentic configuration an explicit variant exposed to a task-specific fitness function. Task, model, prompt, hardware, accuracy, latency, and energy jointly determine which configuration is viable; no single architecture dominates across environments. Its strongest implication is for Cooperation. Adding coordinating agents imposed a systematic resource cost, while measurable accuracy benefit was limited to particular work such as vulnerability detection. Cooperation is therefore not a monotonic consequence of increasing agent count. It remains fit only where task-specific outcome gains exceed its coordination, execution, latency, and energy costs.
REFINES. Supports Selection with a controlled factorial comparison showing that materially different agent configurations occupy different accuracy, latency, and energy positions as the task, model, prompt, and hardware environment changes. Refines Cooperation by providing counterevidence to a general cooperation advantage: multi-agent systems can improve a selected task, but lightweight configurations dominate most measured Pareto-optimal choices. C03 should therefore be read as conditional on demonstrated net fitness relative to useful isolated operation.
What this does not establish
- The paper is an arXiv preprint and does not establish completed peer review or independent replication of the reported findings.
- The study evaluates benchmark tasks with local open-weight models and one orchestration framework; results may differ with frontier hosted models, other agent architectures, tool environments, or production repositories.
- The evaluated task configurations differ in task-specific workflow and role design, so agent count is not isolated from every other orchestration detail.
- Accuracy uses task-specific metrics that are not interchangeable across code generation, debt detection, vulnerability detection, and log work.
- CodeCarbon attributes process energy and estimates emissions using a fixed Canadian carbon-intensity assumption; the measurements do not establish the lifecycle environmental footprint of deployed agent services.
- Most core experiments were repeated three times, while the additional GPT-OSS configurations were run once because of resource constraints.
- Benchmark latency and energy do not directly establish monetary cost, developer attention, maintainability, production reliability, or business value.
- The results do not establish that multi-agent cooperation is generally harmful: the vulnerability-detection result shows that benefits can be task-specific and may justify additional cost under different priorities.
- The study does not examine persistent differentiated cooperation that forms a higher-order engineering object under C04.
Which observable task characteristics predict when the outcome improvement from multi-agent cooperation will exceed its additional coordination, latency, energy, verification, and human-attention costs in production engineering?