Evidence-backed practices

Emerging Practices

Evidence-backed engineering practices emerging across public engineering organizations, shown with the use case, problem encountered, reported response, and Selection conditions behind each observation.

47Evidence

47Assessed

0Pending

165Observations

100%Coverage

Assessment coverage measures evidence records explicitly assessed for emerging practices. Assessed evidence may legitimately contain zero practice observations.

25 evidence-backed practice observations · learning

Company	Specific use case being solved	Problem encountered	Reported practice	Selection condition
Will Larson Evidence →	Keep agent execution aligned when project understanding or operating results change over time.	Continuing to drain an old task list can optimize against a stale description after the project or its environment has changed.	Continue to the next task only when the project description is fresh; otherwise restart the loop from goal and measurement assessment.	contextlearningobservability
Cursor Evidence →	Continue a migration across hundreds of pull requests without requiring equally intensive review of every change.	Human review becomes the throughput constraint when every repetitive agent-produced change receives the same level of scrutiny.	Establish a safe approach, review early pull requests closely, and reduce review intensity as repeated fixes hold up.	learningverification
Cursor Evidence →	Make discoveries from one agent execution available to later agents working on the same body of work.	Session-local agents repeatedly rediscover codebase procedures, architectural context, and user preferences.	Synchronize project files containing research, artifacts, codebase knowledge, and preferences across every cloud and local machine used by the Project.	contextlearning
Cursor Evidence →	Turn completed agent work into reusable improvements for future workflows.	Learnings from completed tasks disappear unless they are reviewed and encoded into durable workflow artifacts.	The Reflect skill runs three parallel reviewers, synthesizes durable learnings, checks whether they should be encoded structurally, and requires approval before skill edits.	learningverification
Meta Evidence →	Convert expert corrections into durable improvements to agent knowledge and reasoning.	Manually tracing dependencies, editing interdependent files, and proving that a correction does not break existing behavior can take weeks.	Diagnose feedback, compile minimal edits with parallel impact analysis, perform independent review and structural checks, then submit a validated pull request for expert approval.	coordinationlearningverification
Meta Evidence →	Verify that a proposed knowledge or reasoning change fixes the reported failure without losing existing behavior.	An automatically drafted correction can solve one case while introducing regressions elsewhere in the institutional knowledge system.	Run blind targeted replay and domain regression tests, retry failed compilations, and add every landed failure case to the permanent regression suite.	learningverification
Meta Evidence →	Give an agent durable organizational positions, terminology, and routing rules without encoding them in model weights.	Retrieving raw documents forces the agent to re-derive institutional reasoning on every run, producing slow and inconsistent analysis.	Distill frequently used organizational knowledge into versioned files with explicit boundaries, dependencies, consumers, and deterministic routing.	contextlearning
Teresa Torres Evidence →	Determine whether a change to an AI workflow actually improves its behavior.	Anecdotal testing cannot reliably distinguish an improvement from a regression.	Establish a baseline, change prompt/context/model/decomposition/orchestration, rerun the same inputs, apply evals, and compare scores.	learningverification
Teresa Torres Evidence →	Turn failures observed in real AI usage into regression protection.	Production failures remain isolated incidents unless they become reusable evaluation criteria.	Review traces, log and categorize recurring mistakes, and turn those failure categories into eval candidates.	learningobservabilityverification
Infobip Evidence →	Continue or reconstruct engineering work across agent sessions.	Important state disappears when it exists only in conversational history.	The workflow externalizes durable state into Git, task state, logs, and other artifacts.	contextlearning
Uber Evidence →	Find expensive behavioral patterns in live agent sessions.	Repeated unnecessary tool use and other poor session behaviors can silently consume compute.	Uber uses live cost counters and a session-analysis dashboard that identifies 16 anti-pattern classes with financial impact and targeted remediation.	economicslearningobservability
Cloudflare Evidence →	Make organizational engineering standards directly usable by coding and review agents.	Standards remain weak selection pressure when they live only in human-readable documents disconnected from agent execution and review.	Distill engineering standards into structured Codex rules and an agent skill, then cite those rules during automated review and use them in multi-agent compliance audits.	contextlearningverification
Cloudflare Evidence →	Give coding agents repository-specific operational context across thousands of repositories.	Agents otherwise infer ownership, architecture, build commands, testing conventions and standards from incomplete repository exploration.	Generate AGENTS.md from Backstage metadata, repository structure and Engineering Codex rules, open it as a merge request for owner review, and flag it for refresh when repository changes make it stale.	contextlearningverification
Cloudflare Evidence →	Increase the chance that agent-generated changes already fit repository and organizational constraints before human review.	Agent outputs remain plausible but locally wrong when context acquisition, execution and standards enforcement are disconnected.	Wire Backstage context, repository AGENTS.md, centralized tools and Codex-backed AI review into one engineering path; Cloudflare reports that the resulting first draft is now usually close enough to ship.	contextcoordinationeconomicsexecutionlearningobservabilityverification
Cloudflare Evidence →	Turn an environmental readiness gap into an actionable implementation task.	A readiness assessment has limited value if teams must independently determine how to repair every failing condition.	For each failed readiness check, generate a prompt that can be given to a coding agent to implement the missing support.	executionlearning
Cloudflare Evidence →	Measure how widely agent-friendly environmental traits are spreading across the Web.	Individual site assessments do not show whether agent-compatible traits are becoming common across the wider environment.	Scan 200,000 high-traffic domains and publish standards-adoption data in Cloudflare Radar with weekly updates and API access.	learningobservability
Meta Evidence →	Keep repository context useful as code and dependencies change.	Stale context can misdirect coding agents and may be more harmful than missing context.	Periodically validate paths, detect coverage gaps, rerun quality critics, and automatically repair stale references.	contextlearningobservability
Stripe Evidence →	Improve an agent workflow after observing recurring execution or recovery failures.	Without replayable tasks, changes to prompts, skills, tools, or documentation cannot be compared against the same engineering problem.	Use replayable benchmark environments as a test bed for prompt, skill, browser-tool, and documentation interventions, then measure whether task performance changes.	learningobservabilityverification
Dae-Kyoo Kim Evidence →	Assess whether an agent configuration generalizes beyond one favorable engineering task.	A configuration can appear effective on one task or domain while performing poorly elsewhere.	Evaluate specialization strategies across multiple lifecycle tasks and application domains using structural and semantic artifact-validity criteria.	learningverification
Dae-Kyoo Kim Evidence →	Choose how narrowly to specialize agents across a software-development workflow.	Increasing role specialization does not consistently improve artifact validity across all engineering tasks.	Compare task-specialized, phase-specialized, and process-generalist configurations and choose specialization granularity according to the task rather than assuming narrower roles are universally better.	coordinationlearningverification
Spotify Evidence →	Automatically repair an agent-generated change that fails quality evaluation.	A verifier or judge veto would otherwise leave failed work for a human to recover.	Judge vetoes are fed back into the coding-agent loop, and Spotify reports that the agent course-corrects about half of vetoed cases.	learningverification
Shafin et al. Evidence →	Identify which parts of a multi-agent development process materially change correctness and code quality.	A complete multi-agent workflow can change outcomes without revealing which development activity caused the effect.	Remove Requirement, Design, or Testing one stage at a time and compare each configuration with the full workflow and RawPrompt baseline.	learningobservabilityverification
Yin et al. Evidence →	Choose between lightweight and more elaborate agent orchestration for a software-engineering task.	Agent quantity is an unreliable proxy for execution quality or efficiency.	Compare reasoning depth, coordination strategy, feedback integration, effectiveness, and cost instead of assuming that adding agents improves the workflow.	coordinationeconomicslearning
Airbnb Evidence →	Recover automatically when an LLM-generated migration step fails validation.	A first attempt frequently leaves validation errors that require another targeted correction.	Retry failed steps with dynamic prompts containing the validation errors and latest file version, up to a configurable limit.	executionlearningverification
Airbnb Evidence →	Improve completion across the remaining long tail without tuning one file at a time.	Repeated failures clustered around common issues that required changes to prompts or migration tooling.	Sample representative failures, tune prompts and scripts, verify the change on the sample, then sweep the remaining corpus again.	learningobservabilityverification