Practice Observations
Observations of specific engineering use cases, encountered problems, and reported practices extracted from the evidence corpus.
18Evidence
18Assessed
0Pending
39Observations
100%Coverage
Assessment coverage measures evidence records explicitly assessed for practice observations. Assessed evidence may legitimately contain zero observations.
8 practice observations · economics
| Company | Specific use case being solved | Problem encountered | Reported practice | Selection condition |
|---|---|---|---|---|
| Cursor | Choose whether a task should use multi-agent orchestration or a plain agent. | Coordination ceremony can cost more than it saves on work small enough for one agent. | The Orchestrate playbook says not to orchestrate when one agent can finish inside the task budget and uses a task-scale threshold for orchestration. | coordinationeconomics |
| Teresa Torres | Run judgment-based evaluation without paying its cost for every case. | LLM-as-a-Judge evaluations are more expensive than deterministic assertions. | Use a cheap code assertion first and send only threshold-crossing cases to an LLM judge. | economicsverification |
| Uber | Reduce the cost of high-volume agentic engineering work. | Using an expensive primary model for every bounded subtask makes scaled execution more costly. | A primary model decomposes and evaluates work while lower-cost subagents execute bounded tasks. | coordinationeconomics |
| Uber | Determine whether agent automation is economically efficient. | Token or request spend alone does not show how much useful engineering work was delivered. | Uber tracks outcome-denominated cost together with workload-specific quality signals and delivered volume. | economicsverification |
| Uber | Find expensive behavioral patterns in live agent sessions. | Repeated unnecessary tool use and other poor session behaviors can silently consume compute. | Uber uses live cost counters and a session-analysis dashboard that identifies 16 anti-pattern classes with financial impact and targeted remediation. | economicslearningobservability |
| Uber | Choose models for specific managed software-engineering agent workloads. | General model benchmarks do not establish the best quality, reliability, and economics for a particular production workload. | Uber builds benchmarks from each managed agent's real work and evaluates cost per completed task, output quality, and reliability. | economicsverification |
| Gloaguen et al. | Provide coding agents repository-specific instructions, workflow guidance, and tool information. | Agents need repository-specific knowledge, but extra context can be redundant and increase exploration and inference cost without improving task resolution. | The evaluated CTXbench repositories use developer-committed repository context files that agents read and follow during coding tasks. | contexteconomics |
| Lulla et al. | Provide coding agents repository-specific guidance while they execute pull-request tasks. | Agents otherwise need to infer repository-specific instructions during execution, which can increase completion time and generated output. | The evaluated repositories provide a version-controlled root AGENTS.md file as repository-level context for the coding agent. | contexteconomics |