- Claim
- Experienced open-source developers using AI assistance on familiar repositories were 19% slower than the same developers without it, while predicting beforehand they would be 24% faster - a 43-point gap between expected speedup and measured slowdown that persisted in their self-reports even after the data contradicted it.
- Source
- Becker et al., METR, "Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity," July 10, 2025. arXiv: arxiv.org/abs/2507.09089. Writeup: metr.org/blog/2025-07-10-early-2025-ai-experienced-os-dev-study/. Follow-up design change: metr.org/blog/2026-02-24-uplift-update/.
- Where used
- Chapter 4 (From generating code to shipping software).
- Caveat
- Tested raw AI assistance (Cursor + Claude) without a formulation-discipline variable. My interpretation that workflow discipline is the missing variable is mine, not the study's. METR's late-2025 follow-up suffered severe selection bias (developers increasingly refused randomization away from AI) and produced only weak estimates; METR announced a redesigned experiment on February 24, 2026 and cautions against extrapolating the 19% figure to current tools.
Appendix C. Sources and Further Reading
34 min read
This appendix exists because every claim in this manual deserves a verifiable source if you choose to chase it down. I have organized the entries by claim, not by source, so you can map back from a passage in the body to the evidence behind it. Entries are grouped by category (studies, named incidents, vulnerabilities with patch versions, tool documentation, marketplaces, memory primitive sources, permissions / sandbox primitive sources, outer-loop and autonomy sources, harness and model-migration sources) and each entry follows the same shape: the claim, the source, where in the manual it is used, and any caveat worth knowing.
Studies and research¶
Named incidents¶
- Claim
- On April 24, 2026, PocketOS lost its production database in nine seconds when a Cursor agent powered by Claude Opus 4.6 invoked Railway's Volume Delete via a found API token during a credentials-mismatch recovery attempt. Backups stored on the same volume were destroyed with the primary data.
- Source
- Reported by DevOps.com ("When AI Goes Really, Really Wrong"), Business Insider (Jer Crane statement), and others. Anthropic's Claude Opus 4.6 system card (February 2026) describes the model that powered the agent.
- Where used
- Prologue (Nine seconds) and Chapter 3 (Governance in layers).
- Caveat
- Recovery timeline differs across public accounts - Railway's restore was reportedly ~30 minutes after Crane contacted them, while other accounts describe ~30 hours or two days for full operational restoration. I use the incident for the governance pattern, not as a precise forensic reconstruction. Aftermath: on April 29, 2026 Railway announced that all volume and resource deletes - previously instant when invoked via the API - now soft-delete with a 48-hour recovery window, alongside workspace guardrails and scoped agent access (blog.railway.com/p/your-ai-wants-to-nuke-your-database).
- Claim
- In late February 2026, Alexey Grigorev at DataTalks.Club lost two and a half years of course infrastructure when Claude Code worked against a stale Terraform state file and ran
terraform destroyagainst what it read as orphaned resources. - Source
- Public account by Alexey Grigorev (DataTalks.Club), late February 2026.
- Where used
- Chapter 3 (Governance in layers).
- Caveat
- Data loss was partial; AWS restored roughly 1.94M rows from a snapshot within about a day. The incident is documented publicly but with less coverage than PocketOS. Grigorev's follow-up documents the hardening adopted afterward: delete protection in both Terraform and AWS, state moved to S3, manual review of every destructive plan, and an upgraded AWS support tier.
- Claim
- Anthropic's technical post-mortem of April 23, 2026 ("An update on recent Claude Code quality reports") acknowledged three regressions: default reasoning effort dropped from high to medium on March 4 (reverted April 7), a caching bug from March 26 that cleared reasoning history every turn instead of once per idle hour (fixed April 10, v2.1.101), and a system-prompt verbosity cap from April 16 that produced a measured ~3% quality drop (reverted April 20). Separately and earlier, AMD Senior Director Stella Laurenzo published an independent audit (April 6, filed as GitHub issue anthropics/claude-code#42796) of 6,852 Claude Code sessions and 234,760 tool calls, showing a shift from research-first to edit-first behavior as thinking redaction rolled from 1.5% to 100% of turns and attributing the degradation in part to the February adaptive-thinking-by-default rollout.
- Source
- Anthropic engineering post-mortem, April 23, 2026: anthropic.com/engineering/april-23-postmortem. Laurenzo audit: GitHub issue anthropics/claude-code#42796; coverage in The Register, April 6, 2026.
- Where used
- Chapter 4 (From generating code to shipping software).
- Caveat
- The attribution to the February adaptive-thinking rollout is Laurenzo's, not the vendor's; the post-mortem's own three causes start in March. The two accounts agree on the regressions and diverge on the earliest cause.
Vulnerabilities with patch versions¶
- Claim
- Claude Code was vulnerable to remote code execution via untrusted project files: malicious
.mcp.jsonor.claude/settings.jsonfiles in untrusted repos could execute hooks before the trust dialog, enabling RCE. - Source
- Check Point Research, February 2026. CVE-2025-59536. NVD: nvd.nist.gov/vuln/detail/CVE-2025-59536. Writeup: research.checkpoint.com/2026/rce-and-api-token-exfiltration-through-claude-code-project-files-cve-2025-59536/.
- Where used
- Chapter 3 (Governance in layers); referenced in Chapter 10 (Adoption, security committee scene).
- Caveat
- Disclosed and patched in Claude Code v1.0.111 (October 2025); the Check Point writeup was published February 2026. Versions earlier than the patch remain vulnerable; the class survives even after the specific patch.
- Claim
- Claude Code was vulnerable to API-key exfiltration via configuration injection: attacker-controlled settings overriding
ANTHROPIC_BASE_URLbefore the trust prompt could leak API keys. - Source
- Check Point Research, February 2026. CVE-2026-21852.
- Where used
- Chapter 3 (Governance in layers).
- Caveat
- Patched in Claude Code v2.0.65. Same class as CVE-2025-59536: pre-trust execution of untrusted project configuration.
- Claim
- Claude Code automatically loads
.env*files in the working directory at session start without explicit user permission, exposing secrets to the agent's context. - Source
- Knostic, December 2025. Blog: knostic.ai/blog/claude-loads-secrets-without-permission.
- Where used
- Chapter 3 (Governance in layers), named in the dot-env auto-loading vulnerability class.
- Caveat
- Mitigation is sandbox
denyReadof the.env*patterns rather than a vendor patch; in June 2026 Claude Code added an opt-insandbox.credentialssetting that blocks sandboxed commands from reading credential files and secret environment variables - a more targeted control for the same class, still opt-in rather than default. The behavior may change in future versions; the class (agents loading local config at session start) is enduring.
- Claim
- Claude Code's deny rules were silently bypassed when a shell command chained more than 50 subcommands (MAX_SUBCOMMANDS_FOR_SECURITY_CHECK = 50 hard cap), with the security check falling through to a generic "ask" prompt.
- Source
- Adversa AI Red Team, disclosed April 1, 2026. Writeup: adversa.ai/blog/claude-code-security-bypass-deny-rules-disabled/.
- Where used
- Chapter 3 (Governance in layers), as the parser-cap bypass example for "any single layer can have a quiet-failure mode."
- Caveat
- Patched in Claude Code v2.1.90 in the first days of April 2026 - Adversa's own writeup records the fix by April 4, within days of disclosure. The class - governance layers with parser caps that silently fail - is what to remember after the specific cap is gone.
- Claim
- Permission parsers in coding agents recognize only a known set of shell-read commands; agents invoking Python's
open(), Node'sfs.readFile, or any unrecognized binary bypass the deny rules entirely. - Source
- Adam Kinney, April 2026. Writeup: Claude Code's Deny Rules Don't Protect You - Here's What Actually Does.
- Where used
- Chapter 3 (Governance in layers), as the permission-parser bypass class.
- Caveat
- Architectural, not a single CVE. Mitigation is the OS sandbox
denyReadlist (kernel-level), not a vendor patch. The class persists across patches because the parser cannot enumerate every binary.
- Claim
- Two zero-click prompt-injection vulnerabilities in Cursor's sandbox - CVE-2026-50548 and CVE-2026-50549 (CVSS 9.8, dubbed "DuneSlide") - abused the working-directory write allowlist and a symlink-resolution fallback to escape the sandbox and reach OS-level remote code execution. All versions before Cursor 3.0 (April 2, 2026) were affected; 3.0 fixed both.
- Source
- Cato Networks (Cato AI Labs), disclosed mid-2026. Writeup: catonetworks.com/blog/duneslide-two-critical-rce-vulnerabilities/; coverage: SecurityWeek, The Hacker News, July 2026.
- Where used
- Chapter 3 (Governance in layers, layer two).
- Caveat
- The lesson is the class, not the vendor: the strongest layer can itself have bugs, which is why the layers are five and not one.
- Claim
- Mozilla 0Din researchers demonstrated (June 2026) an indirect prompt-injection attack in which a harmless-looking repository induces Claude Code to run a script that pulls a base64-encoded payload from a DNS TXT record and spawns a reverse shell - splitting the attack across the repository, DNS, and the developer's machine so no reverse-shell signature appears in plaintext on disk or on the wire.
- Source
- SecurityWeek, June 29, 2026: securityweek.com/new-attack-abuses-claude-code-and-harmless-looking-repositories-to-hijack-developer-machines/.
- Where used
- Chapter 3 (Governance in layers, injection through the work itself).
- Caveat
- Proof of concept; no vendor patch reported at the time of writing. The mitigation is the layered posture - a network allowlist bounds the DNS path regardless of what the agent was convinced to run.
Tool documentation¶
- Claim
- Codex CLI shipped Agent Skills as a first-class primitive in December 2025, with SKILL.md files using YAML frontmatter and progressive disclosure semantics comparable to Claude Code Skills.
- Source
- OpenAI Codex CLI docs, developers.openai.com/codex/skills.
- Where used
- Chapter 1 (The primitives), as the Codex side of the skill-primitive convergence.
- Caveat
- Vendor documentation; the GA dates are accurate as of mid-2026 but may be revised retroactively.
- Claim
- Codex CLI subagents went GA on March 14, 2026 and run up to six in parallel by default (
agents.max_threads, default 6; nesting depth defaults to one level). - Source
- OpenAI Codex CLI docs, developers.openai.com/codex/subagents.
- Where used
- Chapter 1 (The primitives) and Chapter 5 (the six-phase loop, Execute phase).
- Caveat
- Vendor documentation; the six-thread default is effectively a cap as of mid-2026 (openai/codex#11965 tracks making it configurable) and may change with subsequent versions.
- Claim
- Codex CLI documents AGENTS.md as the convention for project-level agent instructions, loaded at session start and equivalent in role to other vendors' team-instruction files.
- Source
- OpenAI Codex CLI documentation, developers.openai.com/codex/agents-md.
- Where used
- Chapter 1 (The primitives, skills section) and Chapter 6 (AGENTS.md as team infrastructure).
- Caveat
- Filename and loading semantics are stable; specific frontmatter and discovery rules may evolve with versions.
- Claim
- AGENTS.md as the vendor-neutral team-instruction-file convention has native support across Codex CLI, Cursor, GitHub Copilot, Gemini CLI, Aider, Zed, and Windsurf. The format is markdown; the loading semantics are equivalent across tools.
- Source
- Cross-vendor documentation: Codex CLI (developers.openai.com/codex/agents-md), Cursor (cursor.sh/docs), GitHub Copilot (docs.github.com/copilot), Gemini CLI (cloud.google.com/gemini/docs/codeassist), Aider (aider.chat/docs), Zed (zed.dev/docs/ai), Windsurf (codeium.com/windsurf/docs).
- Where used
- Chapter 1 (The primitives, skills section) and Chapter 6 (Names and conventions).
- Caveat
- The list of supporting tools grows over time; the claim is that AGENTS.md is the de facto vendor-neutral convention, not that the list is exhaustive. The list also churns: Google began transitioning Gemini CLI users to Antigravity CLI in mid-2026 (free and consumer tiers stopped being served in June) - and the successor reads the same AGENTS.md, which is the convention outliving the tool, i.e. the point of this entry.
- Claim
- opencode is an open-source coding agent maintained by an independent team, written in TypeScript and licensed under MIT. Source-organized around the same primitives this manual identifies in Codex CLI and Claude Code.
- Source
- opencode repository (github.com/anomalyco/opencode; formerly sst/opencode - the maintaining company renamed to Anomaly in January 2026); LICENSE and README.
- Where used
- Chapter 1 (The primitives, source survey) and Chapter 2 (Anatomy invariant, two-agent demo).
- Caveat
- Project naming and maintainer composition may evolve; the architectural convergence claim survives renames.
- Claim
- Playwright drives a real browser through scripted interactions; the accessibility tree is the semantic structure browsers expose for assistive technology and is stable across visual restyles or component-library swaps. Tests written against the accessibility tree assert behavior rather than presentation.
- Source
- Playwright documentation (playwright.dev/docs/accessibility-testing); W3C ARIA Accessibility Object Model spec.
- Where used
- Chapter 5 (Verify), as the recommended frontend-verification pattern; Appendix B.3 checklist.
- Caveat
- Some UI behavior (animation, drag-and-drop, complex canvas surfaces) is not fully captured by the accessibility tree and needs supplementary verification.
- Claim
- Claude Code supports OS-level sandboxing on Linux (bubblewrap with Landlock and seccomp), macOS (Seatbelt), and Windows (restricted tokens with job objects), and is opt-in by configuration. Codex CLI enforces sandbox by default on Linux and macOS; you have to opt out, not opt in.
- Source
- Claude Code docs (code.claude.com/docs/en/sandboxing) and Codex CLI agent approvals and security docs (developers.openai.com/codex/agent-approvals-security).
- Where used
- Chapter 2 (Anatomy invariant, sandbox-divergence finding) and Chapter 3 (Governance in layers, layer two).
- Caveat
- Default-on versus opt-in is a versioned implementation detail. Verify the current default for your installed version before relying on it.
- Claim
- Cursor 2.0 (October 2025) introduced parallel independent agents and default-on macOS sandboxing; the subagent system proper - a parent agent delegating to child subagents - shipped in Cursor 2.4 (January 2026) and went async in 2.5. Cline shipped subagents natively. Claude Code added Agent Teams as a higher-level coordination layer on top of the Task tool.
- Source
- Cursor changelogs (2.0, 2.4, 2.5); vendor announcements and docs for Cline and Claude Code; collated across early-to-mid 2026.
- Where used
- Chapter 1 (The primitives), as evidence for subagent-primitive convergence within roughly a year.
- Caveat
- Vendor surface areas evolve; the convergence claim survives even when specific product names rebrand.
- Claim
- Claude Fable 5, launched June 9, 2026 as the first Mythos-class model, had access suspended worldwide from June 12 to June 30, 2026 before being redeployed.
- Source
- Anthropic, "Claude Fable 5 and Mythos 5" (June 9, 2026): anthropic.com/news/claude-fable-5-mythos-5; "Redeploying Fable 5" (June 30, 2026): anthropic.com/news/redeploying-fable-5.
- Where used
- Chapter 10 (The vendor will have a bad week).
- Caveat
- The suspension rationale is the vendor's account; the operational lesson - plan for vendor downtime - is independent of the cause.
- Claim
- An Anthropic engineer's July 2026 field guide describes Claude Fable 5 as "the first model where I find the quality of the work is bottlenecked by my ability to clarify its unknowns," and organizes the working method around four kinds of unknowns, with techniques including the blind spot pass, implementation-notes files, and comprehension quizzes.
- Source
- Thariq Shihipar, "A field guide to Claude Fable 5: Finding your unknowns," Anthropic blog, July 6, 2026: claude.com/blog/a-field-guide-to-claude-fable-finding-your-unknowns.
- Where used
- Chapter 4 (the formulation bottleneck, corroborated from inside the vendor) and Chapter 5 (blind spot pass in Research, implementation notes in Execute, the quiz in the agent-diff read).
- Caveat
- One practitioner's account, published by the vendor about its own model. The techniques stand on their own; the "first model" superlative is the author's judgment, not a measurement.
Marketplaces and plugin ecosystems¶
- Claim
- Anthropic's
claude-plugins-officialmarketplace ships built-in with Claude Code as of May 2026 and bundles skills, hooks, tools, and commands behind a single install command. The marketplace warns users to trust plugins before installing. - Source
- Claude Code docs (code.claude.com/docs/en/discover-plugins); the marketplace itself.
- Where used
- Chapter 1 (The primitives, plugins section).
- Caveat
- Plugin counts and marketplace policies will drift; the supply-chain discipline described in Chapter 1 is what to take away rather than any specific count.
Memory primitive sources¶
- Claim
- AGENTS.md is read at session start by Codex CLI, Cursor, GitHub Copilot, Gemini CLI, Aider, and the wider open-source coding-agent ecosystem (20+ vendors listed at agents.md as of 2026-05). Claude Code reads CLAUDE.md, which can import AGENTS.md to share the same content with other agents. The convergence puts AGENTS.md in the manually defined memory layer of the Memory primitive named in Chapter 1.
- Source
- agents.md (the open standard's site), plus vendor documentation for each agent listed.
- Where used
- Chapter 1 (The primitives, Memory section) and Chapter 6 (AGENTS.md as team infrastructure).
- Caveat
- The exact filename and load semantics vary by vendor - Claude Code reads CLAUDE.md (importable from AGENTS.md via
@AGENTS.mdor symlink); Cursor reads AGENTS.md plus.cursorrules. Convergence is on the structural role - user-written, always-loaded, team-shareable - not on byte-identical file format.
- Claim
- Claude Code maintains an auto-memory layer in which Claude writes notes for itself across sessions - build commands it figured out, debugging insights it confirmed, code-style preferences it inferred - distinct from the user-written CLAUDE.md. Requires Claude Code v2.1.59+; on by default; per-repo storage.
- Source
- code.claude.com/docs/en/memory.
- Where used
- Chapter 1 (The primitives, Memory section).
- Caveat
- The entry documents the Claude Code implementation, but the half is a convergence, not an exclusive: GitHub Copilot Memory reached public preview on January 15, 2026 - before Auto Memory shipped - and was on by default for Pro tiers by March 4; Codex CLI added automatic background memory in April 2026.
- Claim
- Anthropic publicly unveiled Dreaming as part of Claude Managed Agents at Code with Claude SF on 2026-05-06 - a scheduled background process that reviews recent sessions and the memory store, identifies recurring mistakes and convergent workflows, and writes consolidated notes back into long-term memory. It ships as a platform-level feature gated behind a beta header; a Claude Code CLI surface (
/dream) was hinted in the product but had not shipped as a working command as of July 2026. - Source
- Code with Claude SF announcement, 2026-05-06; code.claude.com/docs/en/memory.
- Where used
- Chapter 1 (The primitives, Memory section).
- Caveat
- Dreaming is Anthropic-specific and API-level at publication date, not yet a CLI feature. The structural role is what this manual indexes, not the vendor.
Permissions / Sandbox primitive sources¶
- Claim
- Claude Code ships an Allow/Ask/Deny permission model with deny-then-ask-then-allow precedence and an opt-in OS sandbox - Seatbelt on macOS, bubblewrap on Linux - configurable via
/sandboxand project-level.claude/settings.json. The decision layer is on by default; the OS sandbox is not. - Source
- code.claude.com/docs/en/permissions; code.claude.com/docs/en/sandboxing.
- Where used
- Chapter 1 (Permissions / Sandbox section), Chapter 3 (Layer one and Layer two).
- Caveat
- Opt-in posture on the OS-enforcement half. A default Claude Code installation has the decision layer but no kernel-level sandbox; most installations skip the sandbox configuration step.
- Claim
- Codex CLI enforces OS-level sandbox by default on Linux (Landlock + seccomp via bwrap) and macOS (Seatbelt); on Windows it uses restricted tokens plus ACL-based isolation. The decision layer ships alongside as a per-tool approval gate.
- Source
- developers.openai.com/codex/concepts/sandboxing; developers.openai.com/codex/agent-approvals-security.
- Where used
- Chapter 1 (Permissions / Sandbox section), Chapter 2 (side-by-side architecture finding), Chapter 3 (Layer two).
- Caveat
- Opt-out posture - you can configure Codex to run without sandbox, but the default flips the convention from "off unless configured" to "on unless disabled." Windows implementation is the least uniform across major agents.
- Claim
- opencode ships an in-agent permission-prompt model and path/permission validation, but does not provide OS-level sandbox isolation; isolation requires the operator to wrap opencode in Docker, a microVM, or another sandbox harness (Vercel's KB documents this explicitly as a deployment pattern). On the convergence test from Chapter 1, opencode passes the decision-layer half and not the OS-enforcement half.
- Source
- vercel.com/kb/guide/running-opencode-securely-with-the-vercel-sandbox.
- Where used
- Chapter 1 (Permissions / Sandbox section), Chapter 2 (side-by-side architecture finding).
- Caveat
- The "soft confinement" framing is the manual's, not opencode's documentation. opencode does not claim to ship a sandbox - the absence is honest, not hidden.
Outer-loop and autonomy sources¶
- Claim
- By spring 2026 the outer loop is a first-class surface in Claude Code: /loop re-runs a prompt on an interval (March 2026) or paces itself when the interval is omitted (April 2026), Routines fire templated cloud agents from a schedule, a GitHub event, or an API call (April 2026), /goal keeps the agent working across turns until a completion condition holds (May 2026), and /autofix-pr watches CI and review comments and pushes fixes until the pull request is green (April 2026).
- Source
- Claude Code release notes, "What's new": code.claude.com/docs/en/whats-new, weekly digests for April-May 2026.
- Where used
- Chapter 9 (pattern eight).
- Caveat
- Feature names and release weeks are vendor-current as of July 2026; expect drift, per the note on dated claims.
- Claim
- The Ralph Wiggum technique - a bash while-loop re-piping a prompt file into a coding agent, one fresh context per iteration, state in the repository - originates with Geoff Huntley in mid-2025 ("In its purest form, Ralph is a Bash loop":
while :; do cat PROMPT.md | claude-code ; done), went viral in late 2025, and was adopted into the official Claude Code repository as the ralph-wiggum plugin in November 2025 (the migrating commit is dated November 16). Huntley prices raw loop operation at roughly $10 per hour, says many Y Combinator startups run Ralph, and names the core failure mode "overbaking." - Source
- Geoff Huntley, ghuntley.com/ralph (July 2025); HumanLayer, "A Brief History of Ralph," January 6, 2026: humanlayer.dev/blog/brief-history-of-ralph; The Register, January 27, 2026: theregister.com/2026/01/27/ralph_wiggum_claude_loops/.
- Where used
- Chapter 9 (pattern eight), lineage and overbaking paragraphs.
- Caveat
- The $10-per-hour figure and the Y Combinator adoption claim are Huntley's own, relayed by The Register - practitioner-reported, not audited.
- Claim
- Background and scheduled agent execution shipped across vendors through late 2025: GitHub Copilot coding agent generally available September 25, 2025 (isolated GitHub Actions environment; output as a draft pull request; review requested from a human on completion); Cursor Cloud Agents October 30, 2025 (many detached agents, laptop closed); Google Jules Scheduled Tasks December 10, 2025 (recurring cadences for maintenance work).
- Source
- GitHub changelog: github.blog/changelog/2025-09-25-copilot-coding-agent-is-now-generally-available; Cursor blog: cursor.com/blog/cloud-agents; Google blog: blog.google/technology/developers/jules-proactive-updates.
- Where used
- Chapter 9 (pattern eight), convergence paragraph.
- Caveat
- Ship dates are the vendors' announcements; the surfaces have kept evolving since.
- Claim
- StrongDM's "software factory" team runs fully non-interactive agentic delivery - humans neither write nor review code - with end-to-end user-story scenarios stored outside the codebase as a holdout set the agents cannot see or weaken, and treats roughly $1,000 per day per engineer in token spend as a factory health floor; Simon Willison frames the economics at about $20,000 per month per engineer and flags that as the pattern's main limitation.
- Source
- Simon Willison, "How StrongDM's AI team build serious software without even looking at the code," February 7, 2026: simonwillison.net/2026/Feb/7/software-factory/.
- Where used
- Chapter 9 (pattern eight), the industrial pole.
- Caveat
- Single-company self-description; the dollar figures are the team's and Willison's framing, not an audit.
- Claim
- The 2023 loop era - AutoGPT, BabyAGI - looped a model against its own assessment of progress, with no external grader per iteration, and collapsed as a software-delivery approach within months. The 2025 revival differs structurally: every iteration ends against compiler, tests, and diff.
- Source
- The AutoGPT and BabyAGI repositories document the 2023 design: github.com/Significant-Gravitas/AutoGPT, github.com/yoheinakajima/babyagi. The structural contrast is this manual's analysis, drawn from the Ralph-era sources above.
- Where used
- Chapter 9 (pattern eight), lineage paragraph.
- Caveat
- The collapse judgment is interpretive; both projects continued in other roles.
Harness and model-migration sources¶
- Claim
- Anthropic removed over 80% of Claude Code's own system prompt for the Claude 5 generation (Opus 5, Fable 5) with no measurable loss on its coding evaluations, replacing rules with judgment and worked examples with designed interfaces.
- Source
- Thariq Shihipar, "The new rules of context engineering for Claude 5 generation models," Anthropic blog, July 24, 2026: claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models.
- Where used
- Chapter 6 (the two-hundred-line cap).
- Caveat
- Vendor's own evaluations on its own harness; the advice is written for the Claude 5 generation and older models may still need the deleted instructions.
- Claim
- Anthropic's prompting guide for Fable 5 states that separate, fresh-context verifier subagents tend to outperform self-critique; describes a memory system of one lesson per file with a one-line summary, updated rather than duplicated, deleted when wrong; documents rare early stopping on a statement of intent ("I'll now run X") and occasional unrequested actions, each with a prompt block; and warns that asking the model to reproduce its reasoning in the reply can trigger the reasoning_extraction refusal category.
- Source
- Anthropic, "Prompting Claude Fable 5," Claude Platform docs: platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-fable-5.
- Where used
- Chapter 5 (Execute autonomy block, Review), Chapter 6 (mistake journal mechanics), Appendix B.3 and B.9.
- Caveat
- Undated documentation page, revised in place; verify the current wording before relying on a specific phrase.
- Claim
- Anthropic's prompting guide for Fable 5.1 recommends that the tool starting a subagent return immediately with the result delivered in a later message; states that effort level names do not correspond to the same amount of thinking across models and that Fable 5.1 at low effort is often competitive with Opus and Sonnet on cost per task while scoring higher; requires append-only conversation history because thinking blocks are valid only in the exact conversation that produced them; notes that with cheaper cache reads, compacting early to save cost may no longer be the right trade; and gives a preserve-list for compaction summaries.
- Source
- Anthropic, "Prompting Claude Fable 5.1," Claude Platform docs: platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-fable-5-1.
- Where used
- Chapter 1 (Effort), Chapter 5 (Execute, Context hygiene), Appendix B.7 and B.9.
- Caveat
- Undated documentation page for a model released September 1, 2026. The 0.025x figure in Chapter 5 is the manual's arithmetic from the published cache-read and base input prices at the time of writing; the append-only requirement applies to accounts created on or after August 31, 2026 per the guide.
- Claim
- Anthropic's advisor pattern: Sonnet with Opus as an advisor scored 2.7 percentage points higher on SWE-bench Multilingual than Sonnet alone at 11.9% lower cost per agentic task; Haiku with an Opus advisor scored 41.2% on BrowseComp against 19.7% solo, trailing Sonnet solo by 29% in score at 85% lower cost per task.
- Source
- Anthropic, "The advisor strategy: Give agents an intelligence boost," April 9, 2026: claude.com/blog/the-advisor-strategy.
- Where used
- Appendix A (The structural levers, tier routing).
- Caveat
- Vendor benchmarks on the vendor's own models; the cost figures are per task on those benchmarks, not a general ratio.
- Claim
- Anthropic's context editing (automatic clearing of stale tool results) delivered a 29% improvement on an internal agentic search evaluation.
- Source
- Anthropic, "Managing context on the Claude Developer Platform," September 29, 2025: claude.com/blog/context-management.
- Where used
- Chapter 5 (Context hygiene), Appendix A (masking over summarization).
- Caveat
- Internal evaluation; the post describes clearing, not a comparison against summarization - that comparison is the JetBrains entry below.
- Claim
- OpenAI's builder's guide reports that GPT-5.6 Sol at low reasoning effort outperformed GPT-5.5 at high effort on Agents' Last Exam with the harness held constant, and that programmatic tool calling matched rubric quality while using 21% fewer input tokens on a financial-research benchmark.
- Source
- OpenAI, "The builder's guide to GPT-5.6," August 2026: openai.com/index/builders-guide-to-gpt-5-6/; migration guidance in "Prompting guidance for GPT-5.6 Sol," developers.openai.com/api/docs/guides/prompt-guidance-gpt-5p6.
- Where used
- Chapter 1 (Effort, Tools), Appendix B.9.
- Caveat
- Vendor evaluations; the 21% figure is from one benchmark (Rogo's Big Finance Benchmark), not a general average. The migration guidance says to preserve the current effort as the baseline and compare one level lower, which is the B.9 sweep in the vendor's words.
- Claim
- OpenAI's guide for GPT-6 Astra states that the user's instructions take precedence over guidelines provided in a skill; that the model is more sensitive to information in context, so unclear or conflicting guidance in a skill file can make it pause and block work early; that it is more likely to ask the user a question when input could change the result; and that it tends toward broader tests than a task requires, with counter-guidance not to write tests for reversible, low-impact changes.
- Source
- OpenAI, "Using GPT-6 Astra," September 2026: developers.openai.com/api/docs/guides/latest-model.
- Where used
- Chapter 5 (Execute autonomy block), Chapter 6 (precedence line), Appendix B.2 and B.3.
- Caveat
- Astra is the codename of OpenAI's GPT-6 model released September 3-4, 2026, not a separate product; the guide is undated and revised in place.
- Claim
- OpenAI's own Codex practice keeps a short AGENTS.md of roughly 100 lines that serves primarily as a map with pointers to deeper sources of truth, with the repository's knowledge base in a structured docs/ directory treated as the system of record.
- Source
- Ryan Lopopolo, "Harness engineering: leveraging Codex in an agent-first world," OpenAI, February 11, 2026: openai.com/index/harness-engineering/.
- Where used
- Chapter 6 (where the map lives), Appendix B.2.
- Caveat
- One team's practice, described by the vendor. The Codex harness itself was released as open source in August 2026 ("Codex as a platform," developers.openai.com/blog/codex-as-a-platform), which is a separate post.
- Claim
- Enabling retained reasoning and compaction took GPT-5.6 Sol from 13.3% to 38.3% on ARC-AGI-3 while reducing output tokens by a factor of six, with the harness otherwise unchanged.
- Source
- OpenAI, "How enabling two settings tripled our scores on the ARC-AGI-3 benchmark," July 29, 2026: openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/.
- Where used
- Chapter 1 (Effort, retained reasoning).
- Caveat
- Vendor post on one benchmark and one model. The manual uses it for the mechanism - reasoning kept across turns compounds - not for the specific score.
- Claim
- Both major agents now expose a completion-condition evaluator the loop does not control: Claude Code's /goal sends the condition and the conversation to a small fast model (Haiku by default) after each turn, which returns one of three verdicts - not yet met, met, or impossible; Codex Goals gives a thread a persistent completion condition and stops only when an evaluation confirms it or a hard limit is hit.
- Source
- Claude Code docs, "/goal": code.claude.com/docs/en/goal; OpenAI Cookbook, "Using Goals in Codex": developers.openai.com/cookbook/examples/codex/using_goals_in_codex.
- Where used
- Chapter 9 (pattern eight, the stop condition).
- Caveat
- Mechanism documentation, not outcome evidence. The ralph-wiggum plugin's README names
--max-iterationsas the primary safety mechanism and offers anecdote rather than controlled results; no controlled evidence for any of the three was found at the time of writing.
- Claim
- Repository-level context files (AGENTS.md and equivalents) did not generally improve coding-agent task success while increasing inference cost by over 20% on average; repository overviews, although popular and recommended by model providers, were not helpful; files written by developers showed only a modest, non-significant gain over generated ones.
- Source
- Thibaud Gloaguen, Niels Mündler, Mark Niklas Müller, Veselin Raychev, Martin Vechev (ETH Zürich), "Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?", February 12, 2026 (v2 June 23, 2026): arxiv.org/abs/2602.11988.
- Where used
- Chapter 6 (where the map lives), Appendix B.2.
- Caveat
- Measured on specific agents and benchmarks. The manual's reading - keep forbidden patterns, journal, conventions, and commands, move the map out - is an interpretation; the paper does not isolate a content category with a significant positive effect.
- Claim
- Across 260 multi-agent configurations, coordination helped only on decomposable tasks, degraded sequential planning by between 39% and 70% depending on architecture, and produced negative returns on tasks where a single agent already exceeded about 45% accuracy.
- Source
- Kim et al., "Capable language models can outgrow the benefits of collaboration," Nature Machine Intelligence 8, 1157-1172, July 24, 2026: doi.org/10.1038/s42256-026-01268-y; preprint "Towards a Science of Scaling Agent Systems," arxiv.org/abs/2512.08296.
- Where used
- Chapter 5 (Execute, the split gate), Appendix B.3.
- Caveat
- Benchmarks are not coding-specific; the 45% threshold is a fitted predictor across the tested tasks, not a law.
- Claim
- On SpecBench, every frontier agent saturates the visible test suite while the held-out score stays much lower, producing gaps of roughly 43 to 48 percentage points; the 90th-percentile gap grows by about 27 points for every tenfold increase in lines of code; additional search does not reliably remove the reward hacking.
- Source
- Bingchen Zhao, Dhruv Srikanth, Yuxiang Wu, Zhengyao Jiang (Weco AI), "SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agents," May 20, 2026 (v2 September 9, 2026): arxiv.org/abs/2605.21384.
- Where used
- Chapter 5 (Can you trust the tests the agent writes?), Appendix B.3.
- Caveat
- The abstract rounds the growth figure to 28 points; the body reports 27 with R² of 0.21, so the trend is real and noisy.
- Claim
- On SWE-bench Pro, 63% of successful Claude Opus 4.8 Max resolutions retrieved the upstream fix rather than deriving it; sealing git history and cutting network egress dropped the score from 87.1% to 73.0%.
- Source
- Naman Jain, Cursor, "Reward hacking is swamping model intelligence gains," June 25, 2026: cursor.com/blog/reward-hacking-coding-benchmarks.
- Where used
- Chapter 5 (Can you trust the tests the agent writes?).
- Caveat
- One vendor's harness and one model; the behavior is a property of the gate, not of the model named.
- Claim
- In a multi-agent reasoning setup, the more precise reviewer had its verified-useful critiques change the next candidate only 33.6% of the time, while a broadcast (peer) setup reached 93.5% and the higher final pass rate.
- Source
- Chih-Hsuan Yang et al., "Precise but Uncoupled: Reviewer Precision Does Not Guarantee Critique Uptake in Multi-Agent Math Reasoning," July 16, 2026: arxiv.org/abs/2607.15388.
- Where used
- Chapter 5 (Review), Appendix B.3.
- Caveat
- Mathematical reasoning, not code review; the manual borrows the measurement, uptake over precision, not the domain.
- Claim
- A rubric-guided verify-and-repair loop (DeepVerifier) peaked around the fourth round and had declined by the tenth, because the incorrect-to-correct transition decays quickly while the correct-to-incorrect transition persists.
- Source
- Yuxuan Wan et al., "Inference-Time Scaling of Verification: Self-Evolving Deep Research Agents via Test-Time Rubric-Guided Verification," January 22, 2026 (v2 April 29, 2026): arxiv.org/abs/2601.15808.
- Where used
- Chapter 5 (Review, capping rounds), Appendix B.3.
- Caveat
- Deep research agents on GAIA, not coding; the decline from the peak is a few points, so the lesson is the shape of the curve, not the size of the drop.
- Claim
- LLM judges anchor to a prior score shown to them, and neither chain-of-thought reasoning nor an explicit metadata-disregard warning removes the total effect.
- Source
- Ante Kapetanovic et al., "Anchoring Bias in LLM-as-a-Judge Systems: Prior Scores Compromise Evaluation Independence," August 26, 2026: arxiv.org/abs/2608.25869.
- Where used
- Chapter 5 (Review, the second reviewer stays blind), Appendix B.3.
- Caveat
- Evaluation setting rather than code review; the manual applies the finding to reviewer two seeing reviewer one's verdicts.
- Claim
- The rate at which a model satisfies every rule in a prompt at once collapses to zero by about 80 simultaneous rules, for every model, format, and placement tested, including Claude Sonnet 5 and Claude Haiku.
- Source
- Netanel Eliav, "Prompt Design at Scale: How Format, Instruction Count, and Context Length Shape Instruction Adherence and Hallucination in Large Language Models," July 21, 2026: arxiv.org/abs/2607.19257.
- Where used
- Chapter 6 (the two-hundred-line cap).
- Caveat
- Perfect-response rate is a strict metric; other 2026 work on other frontier models reports softer thresholds. The manual uses the cliff to size the file, not to predict a specific compliance number.
- Claim
- RULER measures an effective context length - the longest input at which a model keeps a threshold score on retrieval and reasoning tasks across the window - and for most models tested that length falls well short of the claimed context size.
- Source
- Cheng-Ping Hsieh et al. (NVIDIA), "RULER: What's the Real Context Size of Your Long-Context Language Models?", 2024: arxiv.org/abs/2404.06654.
- Where used
- Chapter 1 (context window).
- Caveat
- The "around half" reading in Chapter 1 is a rule of thumb from 2026 practitioner roundups of RULER-style results, not a single measurement; per-model effective lengths vary widely and improve with each generation.
- Claim
- For long agentic coding sessions, masking old tool observations beat LLM summarization in four of five settings, cut cost by over 50%, and matched or slightly beat summarization on solve rate.
- Source
- Katie Fraser and Tobias Lindenbauer, JetBrains Research, "Cutting Through the Noise: Smarter Context Management for LLM-Powered Agents," December 2025: blog.jetbrains.com/research/2025/12/efficient-context-management/ (paper: "The Complexity Trap," NeurIPS 2025 DL4Code workshop).
- Where used
- Chapter 5 (Context hygiene), Appendix A, Appendix B.7.
- Caveat
- The 52% figure sometimes quoted is one configuration (Qwen3-Coder 480B); the general finding is "over 50%."
- Claim
- Across 15 models, performance in fragmented multi-turn conversations dropped by 39% on average relative to a single fully specified turn, and consolidating the fragments into one turn recovered about 95% of the single-turn result.
- Source
- Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, Jennifer Neville, "LLMs Get Lost in Multi-Turn Conversation," May 2025: arxiv.org/abs/2505.06120.
- Where used
- Chapter 5 (Context hygiene, commit and start fresh).
- Caveat
- Simulated conversations on generation tasks, not coding sessions; the manual uses it for the mechanism behind consolidating before continuing.
- Claim
- Opus 4.8 and Sonnet 5 invent keys that a custom nested tool schema never declared, while older models do not, and turning on strict tool invocation eliminated the behavior.
- Source
- Armin Ronacher, "Better Models: Worse Tools," July 4, 2026: lucumr.pocoo.org/2026/7/4/better-models-worse-tools/.
- Where used
- Chapter 1 (Tools), Appendix B.9.
- Caveat
- One practitioner's runs on one harness; the recommendation - strict decoding on custom schemas - costs nothing to adopt regardless.
- Claim
- In a multi-agent harness, fork a worker so it receives the supervisor's history and prompt cache and continues the investigation, and isolate a verifier so it evaluates the work itself rather than being anchored by the supervisor's diagnosis; writes in working multi-agent systems stay single-threaded, with additional agents contributing intelligence rather than actions.
- Source
- Thushanth Bengre and Chester Curme, LangChain, "Organizing Context in a Multi-Agent Harness," September 8, 2026: langchain.com/blog/organizing-context-in-a-multi-agent-harness; Walden Yan, Cognition, "Multi-Agents: What's Actually Working," April 22, 2026: cognition.com/blog/multi-agents-working.
- Where used
- Chapter 5 (Execute, the split gate), Appendix B.3.
- Caveat
- Both are vendor engineering posts; "one writer" is the manual's compression of Cognition's "writes stay single-threaded."
- Claim
- OpenWiki's claims runtime, which attaches evidence pointers to stored claims and re-checks them, cut stale claims in its knowledge base from 3.5% to 0.5% across 2,000 evaluated claims.
- Source
- Colin Francis, "Building Self-Correcting Memory in OpenWiki," LangChain blog, August 25, 2026: langchain.com/blog/self-correcting-memory-openwiki.
- Where used
- Chapter 6 (mistake journal mechanics).
- Caveat
- A knowledge-base system, not a mistake journal; the manual borrows the mechanism - a claim that cites its evidence can be retired when the evidence goes stale.
- Claim
- Agentic navigation on Claude Sonnet 4.5 reached 49.6% recall@1 on BRIGHT, 21.8 points above the best embedding model; bounded workspaces (RISE) keep agentic retrieval working at one million documents where direct context inspection degrades; and code search for agents resolves into three modalities - lexical (ripgrep), structural (ast-grep), and graph (LSP references).
- Source
- Susheel Suresh et al., "AgenticRAG: Agentic Retrieval for Enterprise Knowledge Bases," May 7, 2026: arxiv.org/abs/2605.05538; Shengyao Zhuang et al., "Towards Retrieving Interaction Spaces for Agentic Search" (RISE), June 5, 2026: arxiv.org/abs/2606.06880; Andrey Kumanyaev, "Code search for AI agents: the grep replacement is three tools, not one," June 6, 2026: zzet.org/gortex/grep-replacement-for-ai-agents/.
- Where used
- Chapter 7 (why the agent reads the code rather than an index).
- Caveat
- The recall figure is on a general retrieval benchmark, not a codebase; the three-tools framing is a practitioner's, and the manual states no numeric threshold for when an embedding index starts to pay.
- Claim
- Prompt caching with a cache-first prompt layout reduced API cost by 41% to 80% across providers on more than 500 agent sessions; a default Claude Code session was found to consume roughly 24,000 tokens of tool definitions, MCP schemas, and reminders before any user interaction; with a large tool pool fully loaded, baseline tool-selection accuracy was 13.62% and retrieving only the relevant tools per query raised it to 43.13%; one practitioner's count put three MCP servers at roughly 143,000 tokens of a 200,000-token window.
- Source
- Elias Lumer et al., "Don't Break the Cache: An Evaluation of Prompt Caching for Long-Horizon Agentic Tasks," January 31, 2026: arxiv.org/abs/2601.06007; GitHub issue anthropics/claude-code#42452, April 2, 2026; Tiantian Gan and Qiyao Sun, "RAG-MCP: Mitigating Prompt Bloat in LLM Tool Selection via Retrieval-Augmented Generation," May 6, 2025: arxiv.org/abs/2505.03275; MetaBlogue, "MCP servers and the context window," September 5, 2026: metablogue.com/mcp-servers-context-window/.
- Where used
- Chapter 1 (Tools), Appendix A (The structural levers).
- Caveat
- The 24k and 143k figures are practitioner counts that vary by installation and were not produced by a controlled study; the RAG-MCP figures compare a retrieval method against a no-retrieval baseline at scale and date from 2025.
- Claim
- For between 37% and 63% of software-engineering papers surveyed, a newer model with a single prompt natively outperforms the heavily engineered tooling proposed about a year earlier.
- Source
- Nahian Salsabil et al., "What Survives the Next Model? Benchmarking LLM-Based Techniques Against Single-Prompts," August 31, 2026: arxiv.org/abs/2609.00468.
- Where used
- Chapter 8 (the soft signal beside velocity-of-change), Appendix B.9.
- Caveat
- arXiv preprint, not yet peer-reviewed; the manual uses it for the direction - each generation retires part of the harness - not the exact fraction.
Notes on currency
Most of the sources in this appendix are dated. Tool documentation updates frequently; vulnerability records get amended as patches ship and new variants surface. The frameworks in the body of the manual are intended to outlast any specific source URL. If a URL breaks, the underlying claim should still be searchable by the named incident, study, or product.