🔍 Read the full analysis: The Open Questions In METR’s Study Of AI At Anthropic on ThorstenMeyerAI.com
Get smart everyday buys delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
METR’s November 5 assessment estimates that Anthropic’s AI capabilities improved about 1.9 times as quickly after October 2025 as during the preceding period. The nonprofit says AI use likely accounts for much of the acceleration, but Anthropic redacted most of the evidence METR considered most informative, limiting public scrutiny.
METR, a nonprofit that evaluates frontier AI systems, published an assessment on November 5 estimating that Anthropic’s AI capability progress accelerated about 1.9 times after October 2025 compared with its earlier pace. The report says AI assistance likely accounts for most of the increase, but Anthropic redacted a majority of the evidence METR found most informative, leaving the public with limited ability to independently examine the findings.
METR based its estimate on 11 weeks of access to internal Anthropic data, including benchmark results for unreleased models, training metrics, staff surveys and about 15 interviews. Roughly 2.5 full-time-equivalent staff worked on the assessment from August 12 to October 29 using Anthropic devices. METR said it took no payment; Anthropic supplied laptops and free tokens and retained the right to redact non-public information.
The report’s main measure, AECI, combines benchmark scores into a capability index. METR estimates that Anthropic’s score rose by about 15 points per year before October 2025 and about 29 points per year afterward. It places the estimated break point at October 7, 2025, and gives the acceleration estimate as 1.9x, with a 95% confidence interval of 1.5x to 2.8x. METR says it would not be surprised by estimates between 1.3x and 3x.
METR’s estimate is above the 1.7x figure in Anthropic’s Haiku 5.5 system card. METR says its inclusion of newer, unreleased models is the main reason for the difference. When it compares frontier models across developers rather than Anthropic alone, the estimated acceleration is smaller: about 1.4x, from 17 to 24 index points per year. The report says changes in assumptions about task difficulty move its estimate from 1.1x to 2.9x.
Infographic 01 · How fast is AI speeding up AI?
About twice the old pace, most likely because of AI
An outside evaluator's estimate from internal Anthropic data, plus Anthropic's own measurements. Wide uncertainty, much of the evidence redacted.
Acceleration of Anthropic’s benchmark progress (AECI) since October 2025, 95% confidence interval 1.5x to 2.8x. METR’s overall judgment: around twice the old rate; it “would not be surprised” by 1.3x or 3x.
Capability points per year
AECI trend of Anthropic models, break point 7 Oct 2025
All labs before October 2025: +17 a year, so 1.4x. Different task-difficulty assumptions: 1.1x to 2.9x.
Sources: METR assessment, 5 Nov 2026 · METR risk report review · Anthropic Institute measurements.
Powered by Thorsten Meyer AI
Why the Acceleration Estimate Matters
The report addresses a high-stakes question for AI safety and governance: whether AI systems are helping researchers build more capable successors quickly enough to change how much time labs and regulators have to prepare. A pace increase could affect the timing of safety evaluations, safeguards and policy decisions, though METR’s measurement of benchmark progress does not by itself establish that AI can autonomously develop a successor.
METR attributes perhaps about 80% of the acceleration to AI use, while saying it is confident faster growth in training compute is not the explanation. That is the nonprofit’s estimate, not a directly verified breakdown available to the public. Anthropic’s own internal figures, as described in the source material, say Claude led 26% of its AI research and development work in August 2026, up from under 1% in February, with about 30,000 agents working at a time on its main internal platform. Those company measurements offer a picture of growing use, but do not independently establish how much that use caused the measured capability gains.
The evidence-access issue matters because the report concerns a company’s pace of AI development, yet the public cannot inspect much of the material behind the conclusion. METR conducted an outside assessment, but its findings still rely on access controlled in part by the company being assessed. That makes the report informative while leaving a gap between an independent evaluator’s judgment and a result that outsiders can fully reproduce.
Products Worth Considering
Dell XPS 13 9350 Copilot+ Laptop, 13.4-inch 120Hz Display, Intel Core Ultra 7 256V, 16GB RAM, 512GB SSD, Ubuntu Linux 24.04 LTS
Dell XPS 13 9350 Laptop, 13.4-Inch WUXGA Display, Intel Core Ultra 7 256V, 16GB RAM, 1TB SSD
Apple MacBook Pro MPHF3LL/A 14.2-Inch Laptop | M2 Pro 12-Core Chip 16GB Unified Memory 1TB SSD | 19-Core GPU Liquid Retina XDR Display Space
As an affiliate, we earn on qualifying purchases.
How METR Measured Anthropic’s Progress
METR’s assessment is an examination of internal research and development data, rather than a standard evaluation of a finished, publicly released model. Its AECI measure combines performance across benchmarks. METR describes a 2x acceleration as achieving in one year the amount of progress historically seen in two years. That is a measure of the rate of capability progress on the selected tasks, not a claim that every part of AI development or research work has become twice as fast.
Anthropic’s Responsible Scaling Policy version 3.4, effective July 8, 2026, sets out triggers related to automated research and development. One is a model that could replace the company’s research staff at competitive cost; another is a doubling in the pace of progress plausibly attributable in substantial part to automated research or engineering. Anthropic describes the threshold as a potential early warning. The source material says Anthropic wrote in a November 2 SEC filing that it “may imminently meet” its definition of dramatically accelerated progress. That statement is not a finding that the threshold has been met.
Anthropic describes recursive self-improvement as a system capable of autonomously designing and developing its own successor, and says that this has not yet been achieved and is not inevitable. The current situation described in the report is narrower: AI assists human researchers and engineers, while people continue to set goals. METR’s estimate is relevant to that developing process, not proof of fully autonomous self-improvement.
““roughly two years of historical progress over the last year.””
— METR
What the Redactions Leave Unknown
The report’s central limitation is that Anthropic redacted a majority of the evidence METR found most informative. The source material does not specify enough detail to establish exactly which findings were withheld or how much each redaction affects confidence in the estimate. METR also dropped a planned check of how Anthropic splits compute between safety work and capability development, leaving that allocation unresolved.
The acceleration figure depends on the benchmark set, the index and assumptions about task difficulty. METR notes that its Anthropic-only result could be higher because the company lagged the frontier in 2025, or lower if its earlier models were less tuned to the benchmarks. The report’s range of 1.1x to 2.9x under different task-difficulty assumptions underscores that the 1.9x estimate is not a precise, settled measurement.
It is also not clear from the available material how much AI use caused the measured progress, as opposed to other changes in research methods or processes. METR’s “perhaps ≈80%” attribution is an estimate. Nor does a faster rate on benchmark capability scores establish that all research tasks are faster, that the trend will continue, or that Anthropic has reached the policy’s threshold for automated R&D.
Tests of the Policy Threshold
The next major question is whether Anthropic determines that its Responsible Scaling Policy’s automated R&D threshold has been met and what safeguards would follow. The November 2 filing says the company may imminently meet its definition; the material provided here does not confirm a subsequent determination or action. METR’s assessment makes the threshold a live issue, but does not decide it for Anthropic.
Further assessment could clarify whether the measured acceleration persists, how it compares with other frontier developers, and how AI assistance contributes to research output. Greater disclosure of the redacted evidence, or a repeatable evaluation using comparable data, would help outside researchers test METR’s interpretation. Until then, the 1.9x result should be treated as an independent estimate based on substantial but partly restricted access—not as a fully reproducible public finding.
Key Questions
What did METR conclude about Anthropic’s AI progress?
METR estimated that Anthropic’s capability progress accelerated about 1.9x after October 2025 compared with its earlier rate. Its estimate has a 95% confidence interval of 1.5x to 2.8x, and it says different task-difficulty assumptions produce a wider range of 1.1x to 2.9x.
Does the report show that AI is building AI autonomously?
No. The report concerns AI assisting human research and engineering. Anthropic says fully autonomous design and development of a successor has not been achieved; people still set the goals in the process described here.
How much of the acceleration does METR attribute to AI?
METR estimates that perhaps about 80% is due to AI use. That is a qualified estimate, not a public causal measurement that readers can independently verify from the disclosed evidence.
Why is the report’s evidence contested or difficult to check?
METR had access to internal data and interviews, but Anthropic redacted a majority of the evidence METR found most informative. METR also did not complete a planned check of how compute is divided between safety and capability work.
Has Anthropic confirmed that it crossed its acceleration threshold?
The provided material does not confirm that. Anthropic said in a November 2 SEC filing that it “may imminently meet” its definition, while METR said it was plausible the early-warning threshold had been crossed by July 15. Neither statement establishes a confirmed policy determination.
Source: ThorstenMeyerAI.com
