Evidence-backed practices
Emerging Practices
Evidence-backed engineering practices emerging across public engineering organizations, shown with the use case, problem encountered, reported response, and Selection conditions behind each observation.
47Evidence
47Assessed
0Pending
165Observations
100%Coverage
Assessment coverage measures evidence records explicitly assessed for emerging practices. Assessed evidence may legitimately contain zero practice observations.
67 evidence-backed practice observations · context
| Company | Specific use case being solved | Problem encountered | Reported practice | Selection condition |
|---|---|---|---|---|
| Anthropic | Serve ephemeral agents and CI jobs operating continuously against a very large monorepo. | Complete local repositories and offline-capable clients transfer and maintain data that online automated clients rarely need. | Anthropic moves history, diff, blame, commit, merge, and rebase operations to a server while clients retain bounded caches and fetch only requested data. | contexteconomicsexecution |
| Anthropic | Let CI, agents, and internal tools query repository state without checking out or maintaining a repository. | Every consumer independently cloning and maintaining repository data multiplies compute, storage, transfer, and operational work. | Scion exposes diff, ancestry, blame, merge-base, blob, and related operations through horizontally scalable APIs. | contexteconomicsexecution |
| Atlassian | Expand an agent tool surface without allowing tool and context metadata to consume the task budget. | Adding products and tools can increase prompt overhead before an agent begins the requested work. | Atlassian benchmarks matched Jira and Confluence work across MCP versions and redesigns context delivery to reduce token use while expanding the available tool surface. | contexteconomicsobservability |
| Cursor | Distinguish a production regression from expected effects after a deployment. | Operators must connect a changed signal to one of several deployed changes and decide whether the movement is intentional. | Follow the change through deployment and compare its planned signals with a pre-deploy baseline to identify localized or unintended regressions. | contextobservabilityverification |
| Cursor | Detect exploit paths introduced by a pull-request change. | Local pattern matching can flag superficial risks while missing vulnerabilities that depend on data flow, authorization, dependencies, or broader codebase context. | Analyze every pull request against whole-codebase context and return severity, attack path, explanation, and a proposed fix for each finding. | contextverification |
| Cursor | Define how a particular code change will be verified after deployment. | Generic dashboards and alerts may not cover the risks, intended effects, or instrumentation gaps introduced by a specific diff. | Read the pull-request diff before merge, draft an editable plan of expected effects and risk signals, and flag where telemetry cannot determine success. | contextobservabilityverification |
| Cursor | Preserve source-location guidance while reducing context added by large file reads. | Numbering every source line adds substantial repeated context when agents read tens of thousands of lines. | Include line numbers at a sparser interval that remains sufficient for model citations, then verify quality before rollout. | contexteconomics |
| Cursor | Delegate work when fresh subagent context saves more than coordination costs. | Isolated subagents can reduce inherited context but may duplicate work or pursue tasks invalidated by the parent's progress. | Avoid blanket instructions to delegate exploration and constrain alternate-model selection to cases directed by the user or harness. | contextcoordinationeconomics |
| Cursor | Keep a broad tool set available without placing every definition in every model request. | Most specialized tools are used in a minority of conversations, but static definitions consume context throughout every turn. | Keep high-frequency and flow-critical tools in static context and load lower-frequency MCP and built-in tool definitions only when needed. | contexteconomicsexecution |
| Cursor | Optimize context construction for the real distribution of agent tasks. | Offline evals can overrepresent hard tasks and miss the token, latency, and behavior effects seen in ordinary production traffic. | A/B test context configurations on production traffic while tracking token use, cost, latency, tool-call errors, agent usage, and quality. | contexteconomics |
| Cursor | Avoid repeatedly sending harness instructions that current models no longer need. | Legacy behavioral guidance remains in every turn after models learn to follow simpler tool definitions. | Re-test system instructions across model families and remove directions whose absence does not degrade production behavior. | contexteconomics |
| Cursor | Reuse unchanged prompt prefixes as an agent conversation grows. | Variable setup mixed with stable instructions and tools prevents effective prefix-cache reuse across turns. | Place explicit cache breakpoints after stable request layers and move user- and request-specific setup behind those boundaries. | contexteconomics |
| Will Larson | Prevent an agent from executing project work without a defined direction or observable outcome. | A task backlog can appear actionable even when the project lacks an explicit goal or a way to measure progress. | Start each project loop by auditing the goal definition and measurement tools, and iterate with the operator to create missing prerequisites. | contextobservability |
| Will Larson | Give an agent enough project state to reason beyond one assigned task. | Goals, measures, work state, and implementation context held in private human knowledge cannot regulate an automated project loop. | Represent goals and approach in a Notion RFC, measures in Datadog or Snowflake, and shared work state in a Linear project that the agent can audit. | contextcoordinationobservability |
| Will Larson | Keep agent execution aligned when project understanding or operating results change over time. | Continuing to drain an old task list can optimize against a stale description after the project or its environment has changed. | Continue to the next task only when the project description is fresh; otherwise restart the loop from goal and measurement assessment. | contextlearningobservability |
| Will Larson | Implement one project across frontend, backend, infrastructure, and data repositories. | Repository-scoped checkouts make coordinated cross-repository agent changes difficult to manage. | Maintain about ten independent local workspaces, each with a checkout of every repository, and operate agents at workspace rather than repository scope. | contextcoordinationexecution |
| Atlassian | Retrieve several related components, tokens, or guidance records during one agent task. | Agents made repeated context round trips, accumulating avoidable lookup latency. | Add a batch command and improve fuzzy search so agents can consolidate related context queries. | contexteconomicsexecution |
| Atlassian | Choose how a coding agent should retrieve structured engineering context. | Equivalent knowledge can produce different end-to-end outcomes when invocation, instructions, and output formats differ. | Hold the agent, model, tasks, and questions constant; vary only the context-delivery path; then record pass rate, task time, tokens, tool calls, and transcripts. | contexteconomicsobservabilityverification |
| Atlassian | Expose one governed knowledge source to agents with different tool access. | Some agent environments provide a shell but no MCP client, while some sandboxes cannot access a terminal. | Generate multiple interfaces from shared handlers and data, prefer the benchmarked CLI path, and fall back to MCP or bundled references when required by the environment. | contextexecution |
| Cursor | Make discoveries from one agent execution available to later agents working on the same body of work. | Session-local agents repeatedly rediscover codebase procedures, architectural context, and user preferences. | Synchronize project files containing research, artifacts, codebase knowledge, and preferences across every cloud and local machine used by the Project. | contextlearning |
| OpenAI | Keep agents operating reliably across sessions that span hours, days, and multiple context windows. | Long-running work requires context continuity, recoverable execution, files, code execution, and preserved intermediate results. | Use a managed, versioned agent harness with automatic context compaction and a durable execution environment. | contextexecution |
| OpenAI | Let an agent use a large tool environment without loading every tool definition or raw result into context. | Large tool catalogs and result sets consume tokens and dilute the context needed for the task. | Load relevant tools on demand, run related calls programmatically, and return only relevant combined results to model context. | contextexecution |
| Cursor | Coordinate multi-day programs containing many stacked pull requests and subagents. | Parallel work needs explicit ownership, bounded context transfer, durable state, and independent verification. | pstack defines coordinator, optional sub-coordinator, worker, and verifier roles; structured state files; queue events; and worker briefs. | contextcoordinationverification |
| Meta | Execute expert reasoning procedures without loading every instruction and source into each agent turn. | Flat instructions and broad retrieval introduce mixed-relevance context that consumes tokens and degrades attention. | Separate knowledge from composable reasoning recipes and load only the instructions and files required for each analytical phase. | context |
| Meta | Give an agent durable organizational positions, terminology, and routing rules without encoding them in model weights. | Retrieving raw documents forces the agent to re-derive institutional reasoning on every run, producing slow and inconsistent analysis. | Distill frequently used organizational knowledge into versioned files with explicit boundaries, dependencies, consumers, and deterministic routing. | contextlearning |
| Infobip | Continue or reconstruct engineering work across agent sessions. | Important state disappears when it exists only in conversational history. | The workflow externalizes durable state into Git, task state, logs, and other artifacts. | contextlearning |
| Infobip | Carry a coding task from research through planning, task definition, and implementation. | One growing context across the workflow becomes noisy and allows upstream errors to compound into implementation. | Infobip structures work into four phases and repeatedly writes, selects, compresses, and isolates context between phases. | context |
| Infobip | Delegate implementation to fresh coding-agent sessions with sufficient task context. | Fresh sessions need goals, dependencies, relevant code paths, success criteria, and validation methods without relying on conversational history. | Research and planning produce validated artifacts, and task definitions carry goals, dependencies, code paths, acceptance criteria, and validation methods. | contextverification |
| Destefanis and Aste | Decompose a software change across agents with dependent outputs. | Each contribution can be locally correct while a convention spanning two owners remains unresolved and causes the integrated system to fail. | Identify cross-agent interfaces explicitly and assign one agent responsibility for validating each end-to-end contract. | contextcoordinationverification |
| Destefanis and Aste | Choose between direct messaging and persistent shared artifacts for an agent team. | One-to-one messages repeat shared information, while mandatory files add overhead when the work already communicates through file interfaces. | Measure the task's emergent coordination network and require shared files only where they replace message-heavy many-to-many traffic. | contextcoordinationeconomics |
| Cloudflare | Help agents find resources and help resource owners understand whether agents can discover them. | Human-oriented search and visibility mechanisms do not directly expose what agents can find or how visible a resource is to them. | Cloudflare presents AI Search for agent-oriented retrieval and AEO for measuring visibility to models and agents. | contextobservability |
| Cloudflare | Let agents consume web content without paying to parse human-oriented presentation markup. | HTML intended for humans consumes bandwidth, tokens, and context-window capacity that does not contribute to the agent's task. | Cloudflare presents Markdown for Agents as a server-side representation optimized for machine consumption. | contexteconomics |
| Cursor | Coordinate hundreds of agents building a substantial software system concurrently. | Homogeneous agents make conflicting architectural decisions and spend context on both global planning and local implementation. | Cursor separates planner agents that decompose and delegate goals from worker agents that execute bounded pieces. | contextcoordination |
| Stripe | Move from code generation to runnable software without requiring a human to configure each provider dashboard. | Provisioning infrastructure, connecting services, and managing credentials remain adjacent bottlenecks even when an agent can write the application code. | Expose infrastructure provisioning through a structured agent skill or embedded CLI and connect it to multiple providers through delegated, scoped authority. | contextcoordinationexecution |
| Stripe | Continue multi-day or multi-week agent-built software projects without rediscovering project state on every session. | Session-local agents lose project context and repeatedly reacquire state before they can act. | Carry project context across sessions so the agent can return to the same project with persistent state. | context |
| Spotify | Give coding agents organizational context about software components before and during engineering work. | Agents need component ownership, documentation, and responsible-team information that is not necessarily present in source files. | Spotify exposes Backstage's software catalog to agents through MCPs and command-line tools. | context |
| Spotify | Find concrete source locations that require changes during a fleet-wide migration. | Knowing the downstream consumer does not identify the actual repository usages that need modification. | Spotify uses Codesearch queries to locate migration targets across its GitHub Enterprise landscape. | context |
| Spotify | Identify repositories affected by a downstream dataset migration. | The migration system must know which repositories consume the changed dataset before an agent can modify them. | Spotify uses Backstage endpoint lineage to identify downstream consumers. | context |
| Cloudflare | Make organizational engineering standards directly usable by coding and review agents. | Standards remain weak selection pressure when they live only in human-readable documents disconnected from agent execution and review. | Distill engineering standards into structured Codex rules and an agent skill, then cite those rules during automated review and use them in multi-agent compliance audits. | contextlearningverification |
| Cloudflare | Give coding agents repository-specific operational context across thousands of repositories. | Agents otherwise infer ownership, architecture, build commands, testing conventions and standards from incomplete repository exploration. | Generate AGENTS.md from Backstage metadata, repository structure and Engineering Codex rules, open it as a merge request for owner review, and flag it for refresh when repository changes make it stale. | contextlearningverification |
| Cloudflare | Increase the chance that agent-generated changes already fit repository and organizational constraints before human review. | Agent outputs remain plausible but locally wrong when context acquisition, execution and standards enforcement are disconnected. | Wire Backstage context, repository AGENTS.md, centralized tools and Codex-backed AI review into one engineering path; Cloudflare reports that the resulting first draft is now usually close enough to ship. | contextcoordinationeconomicsexecutionlearningobservabilityverification |
| Cloudflare | Evaluate whether documentation restructuring changes agent cost and task speed. | Standards conformance alone does not demonstrate that the environment actually improves agent efficiency. | Run the same agent architecture against multiple technical documentation sites and compare token use and time to the correct answer. | contexteconomicsobservability |
| Cloudflare | Reduce the exploration required for agents to find precise technical documentation. | Large or HTML-heavy documentation forces iterative search, fragments context, and increases token and latency cost. | Restructure documentation into bounded agent-readable Markdown paths and context-sized product directories. | contexteconomics |
| Cloudflare | Assess whether a web environment exposes the information and capabilities agents need to operate efficiently. | Agent compatibility remains difficult to improve when discoverability, content access, permissions, and callable capabilities are implicit or unmeasured. | Score sites across Discoverability, Content, Bot Access Control, and Capabilities using machine-checkable standards. | contextexecutionobservability |
| Cloudflare | Delegate a bounded task without sharing the parent's entire context and mutable state. | Subagents need independent context and tools while the parent still needs a defined mechanism for delegation and result collection. | Create isolated child agents with their own state, model, memory, and tools, connected to the parent through typed streaming RPC. | contextcoordinationexecution |
| Cloudflare | Continue agent work across long conversations, interruptions, and session branches. | Long-running work exceeds model context windows and loses continuity when conversation state exists only in the active process. | Persist full tree-structured session history, compact context non-destructively, and expose searchable history plus updateable context blocks. | contextobservability |
| Meta | Keep repository context useful as code and dependencies change. | Stale context can misdirect coding agents and may be more harmful than missing context. | Periodically validate paths, detect coverage gaps, rerun quality critics, and automatically repair stale references. | contextlearningobservability |
| Meta | Help coding agents locate the right files, commands, conventions, and dependencies before editing a proprietary codebase. | Without repository-specific tribal knowledge, agents repeatedly explore the codebase and can make plausible but subtly wrong changes. | Generate compact context files containing quick commands, key files, non-obvious patterns, and cross-references instead of exhaustive repository summaries. | contexteconomics |
| Meta | Answer cross-repository impact questions without repeated exploratory searches. | A change in one subsystem can propagate across repositories, making dependency discovery expensive and error-prone. | Pre-compute dependency indexes and data-flow maps so agents can retrieve affected relationships directly. | contexteconomics |
| Meta | Create structured AI-readable repository knowledge across a large multi-repository system. | Manually documenting thousands of files and hidden engineering conventions does not scale. | Orchestrate specialized explorer, analyst, writer, critic, fixer, tester, and gap-filling agent roles to pre-compute repository context. | contextcoordination |
| Stripe | Measure whether coding agents can complete end-to-end API integration work rather than isolated code-generation tasks. | Function-level benchmarks can miss the planning, state management, browser use, database work, and glue code required by real integrations. | Evaluate agents in complete coding environments with repositories, databases, scripts, test credentials, browsers, and real API interactions. | contextexecutionverification |
| Stripe | Help coding agents verify API migrations and integrations against meaningful runtime behavior. | Testing with nonexistent or unrealistic data can produce technically valid errors that an agent misreads as proof that the implementation works. | Provide realistic test credentials and state, and grade behavior against valid end-to-end outcomes rather than accepting error responses as verification. | contextverification |
| Gloaguen et al. | Provide coding agents repository-specific instructions, workflow guidance, and tool information. | Agents need repository-specific knowledge, but extra context can be redundant and increase exploration and inference cost without improving task resolution. | The evaluated CTXbench repositories use developer-committed repository context files that agents read and follow during coding tasks. | contexteconomics |
| Lulla et al. | Provide coding agents repository-specific guidance while they execute pull-request tasks. | Agents otherwise need to infer repository-specific instructions during execution, which can increase completion time and generated output. | The evaluated repositories provide a version-controlled root AGENTS.md file as repository-level context for the coding agent. | contexteconomics |
| Dae-Kyoo Kim | Design agent boundaries for tasks that depend on information accumulated across a development phase. | Overly narrow task boundaries can discard useful continuity between related activities. | Use phase-level specialization where contextual continuity across related tasks improves artifact validity, such as operation identification or test design in the study. | contextcoordination |
| Spotify | Give different background coding agents instructions they can execute effectively. | A single prompt style did not work equally well across agent architectures. | Spotify tailored prompt structure to the agent, using strict step-by-step prompts for its homegrown agent and outcome-oriented prompts for Claude Code. | context |
| Spotify | Make autonomous background coding runs more predictable. | Additional dynamically fetched context and tools increase dimensions of unpredictability. | Spotify limits tools and hooks, constrains Git and Bash operations, and exposes standardized verification. | contextexecutionverification |
| Shafin et al. | Generate complete classes through a structured multi-agent software-development process. | Direct code generation does not separate requirements, design, implementation, and verification responsibilities, making it difficult to study how process structure changes outcomes. | Assign Requirement Engineer, Architect, Developer, and Tester agents distinct responsibilities and pass artifacts sequentially between them with feedback-based refinement. | contextcoordination |
| Spotify | Turn developer requests from Slack or GitHub into background coding work. | The background coding agent needs sufficient task information before it can independently produce a pull request. | An interactive agent gathers task information and hands a resulting prompt to the background coding agent. | contextcoordination |
| Google Cloud | Repair build and test failures encountered while migrating applications across instruction-set architectures. | A migration can reach repository-specific build or test failures that do not lend themselves to a simple mechanical edit. | Invoke an AI migration agent when a build or test fails and let it iteratively edit files, execute the failing check, and continue until verification succeeds or the agent gives up. | contextexecutionverification |
| Google Cloud | Allow an agent to adapt its next repair action to the result of the previous engineering command. | A repair loop needs current build and test evidence rather than relying only on the initial task description. | Execute each selected tool and attach its output back into the agent context before the next reasoning and tool-use step. | contextexecutionverification |
| Google Cloud / DORA | Improve the usefulness of AI-assisted engineering beyond generic model knowledge. | General-purpose AI lacks organization-specific repository, documentation, and workflow context. | Invest in connecting AI tools to internal context so engineering assistance is grounded in company-specific information. | context |
| Google Cloud / DORA | Keep accelerated AI-assisted work aligned with useful product outcomes. | Faster engineering output can move a team in the wrong direction when the problem and user outcome are unclear. | Use explicit user-centric focus to provide direction for AI-assisted engineering work. | context |
| OpenAI | Make delegated agent execution resemble the repository's real engineering environment. | An agent can produce misleading results when dependencies, tooling, or environment setup differ from the environment in which the code is normally built and tested. | Configure the agent sandbox with the repository's dependencies, tooling, and setup so execution matches the real development environment as closely as practical. | contextexecutionverification |
| OpenAI | Delegate repository changes to a background coding agent without sharing mutable execution state across tasks. | Concurrent coding tasks can interfere with one another or with a developer's local workspace when they share execution state. | Run each delegated coding task independently in its own isolated cloud environment preloaded with the target repository. | contextexecution |
| OpenAI | Give a coding agent repository-specific navigation, testing, and convention guidance before it edits code. | A generic agent prompt does not encode how a particular repository is organized, tested, or expected to be changed. | Store repository-local AGENTS.md files that tell the agent how to navigate the codebase, which commands to run, and which project practices to follow. | context |
| Airbnb | Migrate complex tests with project-specific setup, indirection, and conventions. | Retrying alone stopped being effective when the model lacked the repository context needed to resolve complex test behavior. | Expand prompts with selected source files, related tests, passing project-local examples, migration guidance, and validation failures, reaching roughly 40K-100K tokens and up to 50 related files. | contexteconomics |