Expert corrections become regression-tested institutional memory
Source → Observed → Interpretation → Model implication
An Organizational Second Brain: Building an AI That Learns From Experts
View source →Meta built a domain-specific agent with four interdependent layers: a structured knowledge system, composable reasoning procedures, an evaluation framework, and a self-improvement loop.
The knowledge system contains more than 200 structured files with explicit boundaries, routing rules, dependencies, and consumers; Meta separates frequently used distilled knowledge from sparse sources retrieved when needed.
Meta reports that replacing a flat instruction file and broad semantic retrieval with recipe-driven progressive disclosure reduced tokens consumed per turn by around 80%.
Human checkpoints and escalations let experts confirm, correct, or redirect intermediate reasoning, and each correction becomes input to the improvement loop.
The improvement pipeline diagnoses feedback, compiles minimal file edits, performs independent adversarial review and deterministic structural checks, runs targeted replay and regression tests, and presents a pull request to a domain expert for final review.
When a fix lands, the triggering failure and validated answer are added to the regression suite so future changes must preserve the corrected behavior.
After three development sprints spanning six weeks, Meta reports reducing individual assessment time from days to minutes, producing validated knowledge edits at a rate that previously required full engineering sprints, and observing zero regressions across improvement cycles.
The source refines Learning as an engineering-environment capability rather than an attribute confined to model weights or transient prompts. Expert corrections persist through versioned knowledge and reasoning files, while replay, regression testing, and human review regulate which changes survive. The architecture also separates persistent roles for knowledge, procedures, evaluation, and maintenance, and Meta treats their integration as the operating system: each layer solves a distinct problem and the others degrade if one is removed. This provides evidence of Specialization in an agentic system, although it does not compare the integrated system with useful isolated operation.
REFINES. Refines the model by showing that organizational agent learning can be externalized into auditable, regression-tested environmental state. Selection operates on proposed knowledge and procedure changes through deterministic checks, evaluations, adversarial review, and expert approval. The persistent, differentiated, interdependent layers also support Specialization: the integrated system, rather than any single agent invocation, becomes the meaningful unit that preserves and improves institutional capability.
What this does not establish
- The implemented domain is organizational compliance rather than a comparative study of software-engineering agents, although Meta identifies engineering standards compliance as an applicable domain.
- The source does not compare the integrated architecture with useful isolated agent operation, so it does not establish the relative fitness advantage required to support Cooperation directly.
- The reported outcomes are first-party, do not publish benchmark sizes or raw results, and have not been independently replicated.
- Zero regressions across the reported improvement cycles does not establish that the process prevents regressions across arbitrary domains, longer time horizons, or substantially larger knowledge systems.
- Human experts remain responsible for consequential checkpoints, ambiguity resolution, and approval of proposed knowledge changes.
As structured organizational knowledge and its regression suite grow, does the specialized maintenance system preserve accuracy and expert-review efficiency across repeated software-engineering work better than less integrated agent configurations?