COMPANY
METR
companytopic-note
Overview
METR (Model Evaluation & Threat Research) is a nonprofit model-evaluation research organization focused on auditing frontier AI labs for dangerous capabilities and AI-R&D acceleration. It has previously red-teamed Anthropic’s internal agent monitoring systems and is one of the external evaluators referenced in frontier-lab Responsible Scaling Policy frameworks.
Timeline
- 2026-11-08-AI-Digest — METR’s own
2026-11-05blog post is now the primary source for the~1.9×/~80%figures The Decoder trailed in 2026-11-07-AI-Digest. METR states~1.9×AECI-benchmark growth post-October2025with a95%CI of1.5–2.8×and~80%best-guess attribution to Anthropic researchers “directly using AI systems in R&D” — with METR’s own note that the80%figure carries “substantial uncertainty” and that a1.3×–3×range would not surprise them. AP independently corroborates the acceleration-two-months-after-Dario Amodei‘s-slowdown-call framing. Load-bearing reframe: yesterday’s carried-forward “may have crossed its own AI-R&D acceleration threshold” language should now soften — METR itself does not call this autonomous recursive self-improvement; it is an AI-assisted R&D productivity multiplier on human researchers, not a model-improving-itself loop. The1.9×is real, measurable, and new; the RSI framing is faster than the evidence. Anthropic redacted detailed attribution data in the published report, constraining third-party replication. Separately, a new study on METR-shaped AI-auditor tooling for agent oversight reports high recall on agent misbehaviour but a false-alarm rate risking overwhelming the reviewers the audits were meant to support — qualitative finding is “recall holds, precision does not, and the reviewer bottleneck moves rather than disappears”. Log against MOC - Agent Security, MOC - Major Companies. - 2026-04-05-AI-Digest — METR red-teams Anthropic‘s internal agent monitoring systems over a three-week engagement; discovers novel vulnerabilities (some patched) but confirms major claims in the Claude Opus 4.6 Sabotage Risk Report hold.
- 2026-11-07-AI-Digest — The Decoder reports METR had an 11-week audit-access window (
2026-08-12–2026-10-29) to Anthropic‘s internal benchmarks, training metrics, and researcher interviews; METR’s write-up reportedly frames Anthropic’s test-score progress since October2025as running at roughly1.9×the prior rate (range~1.3×–3×), with the bulk attributed to AI being used inside Anthropic’s own research — effectively meeting the “<2×” AI-R&D acceleration threshold Anthropic’s August2026risk report flagged as a warning level. The Decoder also reports Anthropic canceled METR’s planned follow-up audit on its capabilities-vs-safety compute split. Load-bearing hedge: specific figures (~1.9×,~80%internal-AI attribution,1.97unredacted internal figure, canceled compute-split audit) sit behind The Decoder’s paywalled body and do not independently corroborate against primary METR output or Anthropic’s own filings in a single pass — carry as “reportedly” / “per The Decoder’s write-up of METR”, not as settled fact. Anthropic’s amended S-1/A uses forward-looking “may imminently meet” language on the RSP AI-R&D threshold, so The Decoder’s “crossed the threshold” framing is sharper than Anthropic’s own public one. Log against MOC - Agent Security, MOC - Major Companies.
Key Developments
- Anthropic AI-R&D acceleration audit (Aug–Oct 2026): METR’s reportedly-leaked internal estimate of
~1.9×prior-rate progress on Anthropic test scores (range~1.3×–3×) per The Decoder’s write-up, with~80%attributed to internal AI use; frames Anthropic as having “may have crossed” its own AI-R&D acceleration warning threshold. Specific figures carry as reported-not-corroborated pending primary METR output. METR’s separately-published preliminary estimate reportedly sits closer to~1.5×with meaningful probability of2×. - Red-team of Anthropic internal agent monitoring systems (April 2026): Three-week engagement surfaced novel vulnerabilities (some patched) but confirmed major claims in the Claude Opus 4.6 Sabotage Risk Report held.