Practice Observations
Observations of specific engineering use cases, encountered problems, and reported practices extracted from the evidence corpus.
18Evidence
18Assessed
0Pending
39Observations
100%Coverage
Assessment coverage measures evidence records explicitly assessed for practice observations. Assessed evidence may legitimately contain zero observations.
6 practice observations · observability
| Company | Specific use case being solved | Problem encountered | Reported practice | Selection condition |
|---|---|---|---|---|
| Teresa Torres | Turn failures observed in real AI usage into regression protection. | Production failures remain isolated incidents unless they become reusable evaluation criteria. | Review traces, log and categorize recurring mistakes, and turn those failure categories into eval candidates. | learningobservabilityverification |
| Uber | Find expensive behavioral patterns in live agent sessions. | Repeated unnecessary tool use and other poor session behaviors can silently consume compute. | Uber uses live cost counters and a session-analysis dashboard that identifies 16 anti-pattern classes with financial impact and targeted remediation. | economicslearningobservability |
| Spotify | Coordinate agent-generated migrations across thousands of repositories while directing scarce human attention. | At fleet scale, teams need to know which changes were created or merged and which require human intervention. | Fleetshift schedules and orchestrates Honk changes and tracks migration progress, pull-request state, and changes needing human attention. | coordinationobservability |
| Google Cloud | Run agent jobs that continue for hours or days and survive infrastructure or human interruptions. | A long-running agent cannot depend on one process or session remaining alive for the duration of the work. | Google Agent Executor uses durable event logs and snapshots to support execution, resumption, and distributed deployment. | executionobservability |
| OpenAI | Run more concurrent Codex coding tasks than an engineer can comfortably supervise as separate sessions. | Engineers lost track of work and debugged stalled agents beyond roughly three to five simultaneous sessions. | Symphony moves task assignment and workspace lifecycle into an always-on orchestrator where open tasks receive agents in isolated workspaces and humans review outcomes. | coordinationexecutionobservabilityverification |
| Spotify | Operate background coding agents predictably while agent and model components evolve. | Agent execution needs formatting, linting, diff evaluation, logs, traces, and the ability to change agent or model components without rebuilding the surrounding workflow. | Spotify built a pluggable CLI harness with MCP-based formatting and linting, LLM-as-judge diff evaluation, GCP logs, MLflow traces, and swappable agents and models. | executionobservabilityverification |