Is AI Accelerating AI R&D? METR's High-Access Assessment of Anthropic

A note on evidence before the result. This is not a normal capability report. Anthropic supplied METR with unusually broad internal data, but it also retained the unilateral right to redact non-public material. METR says it could publish its highest-level risk conclusions, while Anthropic removed a majority of the evidence that most influenced those conclusions. The redactions were described as protecting confidential information that could accelerate other AI developers. The consequence is unavoidable: readers can inspect the headline measurement and several robustness checks, but they cannot independently audit the strongest evidence behind METR's judgment. This distinction runs through the entire assessment.

From August 12 through October 29, 2026, a METR team equivalent to 2.5 full-time staff worked inside Anthropic for eleven weeks. The team examined whether AI is accelerating AI research and development, how much of any acceleration should be attributed to AI, and whether visible bottlenecks make continued acceleration unlikely. This was not a benchmark run performed from the outside. METR reviewed internal model scores, algorithmic-efficiency data, training-compute measurements, staff surveys, internal reports, code activity, inference spending, forecasts, and roughly fifteen interviews. It did not receive direct query access to unreleased models and did not observe employees in an unmediated way.

The central estimate is that, as of October 2026, Anthropic may be improving model capabilities on well-specified tasks at roughly twice the rate observed between March 2024 and October 2025. The phrase twice the rate is doing precise work. It does not mean that a model is twice as capable, that a task takes half as long, or that two years of progress occurred solely because of AI. It means that the estimated rate of change after October 2025 is about 1.9 times the earlier rate. METR's 95% confidence interval for its main index is 1.5 to 2.8, and its broader subjective uncertainty is wide enough that it would not be surprised by 1.3 or 3.

1. The result in one page

METR reports an estimated acceleration of 1.9x in Anthropic's AECI trend since October 2025, with a 95% confidence interval of 1.5x to 2.8x. AECI is Anthropic's internal version of the Epoch Capabilities Index. It combines results from many benchmarks into a latent capability score. If the estimate is taken at face value, Anthropic made approximately as much benchmark progress during the last year as it made during the two preceding years, roughly the interval from Claude 2.1 to Sonnet 4.5.

The result is not isolated to one regression specification. METR varied model snapshots, benchmark inclusion, regularization, and the statistical link that maps latent capability to observed scores. Most choices produce an estimate near 2x. The widest sensitivity comes from the link function, where estimates span 1.1x to 2.9x. Including non-Anthropic frontier models lowers the estimate to about 1.4x. Anthropic's Haiku 5.5 system card reported 1.7x; METR's higher estimate comes mainly from adding newer unreleased Anthropic models.

The assessment's most important negative finding is about scaling. METR says it obtained convincing evidence that the acceleration was not caused by an acceleration in the rate at which training compute was growing. This does not mean compute is irrelevant. It means that the observed bend in capability progress cannot be reduced to Anthropic suddenly increasing the growth rate of training compute. Further evidence supporting this conclusion was redacted.

The most consequential speculative finding is about compounding. METR did not identify a convincing bottleneck in algorithms, training compute, experimental compute, or data that would bind during the next twelve months. It repeatedly revised its understanding as new context arrived, so confidence is limited. If progress in capabilities improves the tools used to make the next models, and no input constraint stops the loop, the rate of acceleration could rise further. That possibility is why a 1.9x historical ratio matters beyond an accounting exercise.

2. Why an AI R&D acceleration assessment matters for robotics and physical AI

This study measures Anthropic's software and model-development process, not robots. It contains no estimate for manipulation, locomotion, navigation, or sim-to-real transfer. Its relevance to embodied AI is nevertheless structural rather than rhetorical. Frontier models increasingly serve as perception, planning, language, and policy components in robot systems. Improvements in model capability, experimentation throughput, evaluation design, and compute allocation can propagate into the infrastructure on which embodied systems are built.

The more direct issue is the research loop. Robot learning is expensive in exactly the places where better AI tools may help: writing experiment code, checking data pipelines, generating evaluation variants, extracting results, and iterating on training recipes. If AI can shorten these loops, the effect could show up first as more experiments per researcher rather than as a fully autonomous scientist. METR's interviews are consistent with this pattern. Researchers described reducing idea-to-experiment time, trying more variants, and removing implementation overhead. One researcher estimated an 80% reduction in the time between an idea and an experiment. Another said the work was now bottlenecked by ideas and compute rather than code.

That claim should not be transferred mechanically to robotics. Physical experiments still face fabrication, hardware reliability, safety review, scarce robot time, and data-collection costs. A coding-agent speedup may remove only a fraction of the total loop. Yet those physical constraints make measurement more important, not less. If software-side iteration accelerates while hardware-side iteration does not, organizations may generate more candidate policies and evaluations than they can validate on real systems. The bottleneck can move rather than disappear. The METR report is useful here because it separates speedup of an individual activity from acceleration of an overall outcome.

There is also a governance lesson. A lab can publish a capability chart while withholding the internal evidence that best explains how the result was produced. Robotics and physical-AI developers should expect the same tension as model-development automation becomes more relevant to safety cases. A clean external benchmark is not a substitute for access to the actual development loop, and an evaluation report is not independently auditable if the decisive observations cannot be described.

4. The AECI evidence: a bend of about 1.9x

AECI turns many benchmark results into a common latent-capability scale. The method assumes that an observed benchmark score is a noisy function of latent model strength and benchmark difficulty. For the main estimate, that function is logistic. A logistic link is reasonable when task difficulties have symmetric, exponential tails. A long tail of unusually hard tasks would call for a log-log link; too many easy or guessable tasks would call for a complementary log-log link.

The main estimate is 1.9x, but the link-function sensitivity is large. The complementary log-log specification gives 1.1x with an interval from 0.8 to 1.5. The cauchit specification gives 2.8x, and log-log gives 2.9x. A learned two-parameter link gives 2.6x. Out-of-sample $R^2$ is similar across most specifications, so the data do not decisively select one distributional story. METR keeps the standard logistic specification because Epoch and Anthropic use it, not because the alternatives have been ruled out.

Snapshot and benchmark choices are less destabilizing. Using only the highest snapshot per model gives 1.9x; using only the first gives 2.2x; using all snapshots gives 1.9x. Dropping vision benchmarks gives 2.2x. Keeping only benchmarks that Claude labels relevant to AI R&D gives 2.0x. Dropping benchmarks with decreasing scores, poor logistic fit, or scores above 85% leaves the estimate at 1.9x. Varying regularization produces 1.9x to 2.1x. A deliberately omitted model subset, selected for redacted reasons, also produces 1.9x but cannot be independently examined.

METR openly reports two measurement problems. First, Model 4 scores above 50% on all fifteen AECI benchmarks on which it was tested. Nine months earlier, Opus 4.6 still had many benchmarks below 50%. A benchmark suite that becomes too easy cannot measure progress indefinitely. METR tests saturation by removing benchmarks once they reach 85%, and the acceleration estimate remains 1.9x, but confidence in future discrimination is lower.

Second, benchmark elicitation may use too few tokens. A typical internal task used about 2.5 million input and 72,000 output tokens; agentic coding tasks used about 10 million input and 94,000 output tokens. Those figures are larger than some public evaluation budgets, but METR suspects that further inference spending could raise scores by a handful of points. If later models benefit more from extra inference than earlier models, the measured acceleration would be too low. METR does not resolve this because it lacks full returns-to-expenditure curves across models.

The all-frontier comparison is particularly important. When METR combines Anthropic's internal models with other frontier systems, the estimate falls to about 1.4x. One interpretation is that Anthropic was below the broader frontier during part of 2025, so its historical slope was unusually low. That would make the Anthropic-only 1.9x an overstatement. Another interpretation is that Anthropic models were less optimized for these benchmarks historically, making the earlier cross-model trend too steep and the 1.4x too low. METR cannot fully adjudicate between these stories. This is not a cosmetic robustness note: the historical denominator is one of the main reasons the headline ratio is uncertain.

Previous system cards considered whether progress rose in a one-time jump around Mythos Preview and then returned to the old trend. METR says the newer internal models make a permanent slope change more plausible than a temporary level shift. That conclusion is still based partly on model snapshots that only METR and Anthropic can inspect.

5. Attribution: why METR guesses AI explains about 80%

Observing an acceleration is easier than assigning its cause. METR considered compute scaling, hiring, data improvements, algorithmic discoveries, organizational changes, and the removal of other bottlenecks. It says the evidence does not support a simple story in which Anthropic accelerated the growth rate of training compute. Beyond that, the picture is not decisive. Anthropic staff themselves did not converge on one causal account.

METR's current subjective estimate is that a majority of the October 2026 acceleration comes from AI, perhaps around 80%. This is not a regression coefficient. It is a synthesis of interviews, self-reports, internal analyses, staff behavior, and redacted evidence. METR also believes the acceleration in late 2025 had a substantial non-AI component. Hiring, organizational changes, research choices, and other factors may have initiated or amplified the trend before AI's contribution became dominant.

The distinction between instantaneous and cumulative attribution is essential. Suppose AI contributes 80% of today's acceleration. It does not follow that AI caused 80% of all progress since October 2025. Some of the prior gain came from unrelated changes, and AI's share may have grown only gradually. Conversely, a smaller current causal share could still correspond to a large cumulative effect if the rate of acceleration is increasing. METR's interviews sometimes used the two concepts interchangeably, so the report does not treat a researcher's statement about total progress as a clean estimate of today's growth ratio.

The report's accounting framework moves from observable capability outcomes backward through pretraining and reinforcement learning, then to algorithms, data quality, and compute. The aim is to ask where an input growth rate changed rather than to infer causality from one headline curve. METR says the framework was used in the full report, but the relevant applications and detailed estimates were redacted. Readers therefore see the structure of the argument but not the evidence that would let them duplicate its calculation.

One plausible explanation deserves special attention: if AI improves research productivity but the organization continues to face diminishing returns, more output per researcher may sustain a higher growth rate without producing an ever-increasing growth rate. A 2x acceleration in the overall pace requires the relevant inputs, their productivity, or their allocation to change substantially. METR's concern about compounding arises because AI can affect both productivity and the production of better research tools. The same loop can look modest in one annual comparison and much larger when measured as a derivative of the growth rate.

The 80% figure should therefore be read as a compact expression of a qualitative judgment: METR thinks AI is now the largest contributor, but it does not think the other causes vanished, and it cannot identify the exact counterfactual. The report explicitly says the evidence is compatible with a large AI contribution; compatibility is weaker than a controlled causal estimate.

6. Interviews and self-reports: useful, but not a measurement shortcut

METR and Anthropic interviewed researchers about how AI changed their work. The interviews varied in depth and often mixed personal, team, cumulative, and instantaneous interpretations of speedup. METR used Claude to collect and classify relevant quotations. Of the eleven people represented in the published table, two clearly supported a 2x-or-greater acceleration in their area, three leaned yes, four were borderline or unclear, none leaned no, and two did not support it. This is evidence of a possible effect, not a clean distribution estimate.

The strongest quotations are vivid. One researcher said the AI acceleration was larger than all other effects. Another said it was hard to see how they were not three times more productive. A third estimated that the time from idea to experiment had fallen by 80%. Another said the organization was now limited by ideas and compute rather than code. These statements are plausible descriptions of local bottlenecks becoming cheaper. They do not prove that the organization's overall capability growth rate doubled, because a researcher can spend the saved time on low-value experiments, because coordination costs can rise, and because physical or review bottlenecks may dominate the next stage.

Anthropic's Slack polls provide a second, weaker signal. Staff voted on the speedup they associated with the latest model. The average self-report reached roughly 12x for the most capable internal model in October 2026. METR calls this very loose evidence: respondents are self-selected, the vote is informal, and people may confuse time saved on one task with output value at the organizational level. The 12x number should not be substituted for the 1.9x AECI estimate. What matters for acceleration is whether the growth rate of individual uplift changed and whether enough of that uplift reached the overall production function.

The report also looks for fully autonomous contributions. It finds that AI is not yet responsible for a large share of the most important end-to-end training work. Researchers nevertheless described "signs of life" in several parts of the pipeline. Internal reports and analyses of whether tasks were completed by humans or AI support the possibility that autonomy could grow. The exact evidence and pipeline-specific estimates were redacted. The distinction is important: assisted speedup is already visible, while autonomous acceleration is a threshold risk that could change the trend rather than merely raise its level.

7. Bottlenecks, saturation, and the possibility of compounding

METR investigated whether four broad inputs were likely to stop rapid progress over the next twelve months: algorithms, training compute, experiment compute, and data. It did not find convincing evidence for a binding bottleneck in that window. It also did not find steep diminishing returns strong enough to rule out another acceleration. The conclusion is deliberately probabilistic. The team's understanding of several constraints changed multiple times as it received more detailed information, and Anthropic redacted its analyses of specific bottlenecks.

A useful way to read the claim is as an absence of evidence under restricted access. If a company's ability to run experiments is limited by chips, power, data quality, or reliable evaluation, then better coding agents may increase the number of proposed experiments without increasing the rate of validated progress. Conversely, if code implementation was the dominant local constraint, removing it can expose a new bottleneck while still producing a large short-run speedup. METR's interviews support the second pattern in some teams but do not establish that no company-wide limit will bind.

The possibility of compounding is more consequential than the point estimate. Suppose each generation of models improves the tools used to build the next generation. That creates a feedback path from capabilities to the production function for capabilities. A system can show a modest acceleration now and a larger acceleration later, especially if the improvement in research tools is itself accelerating. METR calls this a significant risk, but it does not claim to have measured a runaway loop. It says the evidence would be more alarming if the growth rate were already increasing rather than settling at a new constant slope. Some redacted evidence made continuous or hyperbolic growth more plausible, but the public report cannot show it.

Benchmark saturation is the mirror image of this concern. If models solve nearly all existing tests, AECI may stop discriminating at the exact moment the underlying research process changes. Model 4's scores above 50% on all tested AECI benchmarks are a warning about the measurement instrument, not direct evidence that capabilities have stopped improving. METR's saturation variants still estimate about 1.9x, which reduces the immediate concern, but the historical baseline and the next year's extrapolation become less trustworthy.

The real-world-impact question is even harder. AECI weights tasks that can be graded, many of which are well specified. Real research contains ambiguous goals, failed hardware, tacit knowledge, product constraints, and coordination. If benchmark performance accelerates while those messy tasks do not, the report overstates the impact on actual R&D. If capability affects outcomes through thresholds, such as crossing a level needed to automate a whole workflow, AECI may understate sudden real-world effects. METR considers both directions plausible and does not collapse them into one net correction.

8. Access, redaction, and the limits of independent review

The access arrangement was unusually generous by the standards of external model evaluations. Anthropic supplied internal model benchmarks, sample tasks, rubric answers, agent spending, historical algorithmic-efficiency data, pretraining and RL strength measures, compute-utilization and inference-efficiency metrics, headcount attribution, training-compute metrics, internal surveys, code and pull-request activity, inference spend, autonomous-research documents, and about fifteen interviews. It also provided substantial staff time from two people METR describes as unusually well informed about AI R&D acceleration measurement.

But access was not the same as autonomy. METR staff never had direct access to Anthropic systems. Most data came through a few technical contacts. Requests could be rejected. The assessment did not include query access to non-public models. This matters for adversarial robustness: if the primary contacts supplied a coherent but incomplete picture, METR had limited ability to discover the missing context. METR says it found no systematic discrepancies when comparing those contacts with roughly fifteen other interviewees, many selected by METR, and considers deliberate deception extremely unlikely. It also notes that fuller employee-level access would have made the assessment significantly deeper.

The redaction problem is sharper. METR's full report is about one hundred pages. Its public post can state high-level conclusions but cannot show the strongest evidence, detailed attribution across the training pipeline, forecasts for a future model, or the methodology needed for independent reconstruction. The table below is the report's own distinction between what Anthropic shared and what the public can inspect.

METR says Anthropic described the redactions as removing confidential information that could accelerate other developers. It agrees that the redacted material was confidential and potentially useful to competitors, but observes that much of it was also very high-level. METR does not claim that every redaction was unjustified; it says it is poorly positioned to balance a company's business interest against the public's interest in understanding AI risk. Its complaint is narrower and stronger: the public cannot independently judge the evidence that determines the conclusion.

Two items in the AEF-1 checklist receive a "No" that directly undermines transparency. The evaluator did not share enough methodological detail for independent review, and the provider had authority to redact results in a way that could conceal concerning findings. METR also reports "No" on direct system access and on having sufficient rights to disclose all necessary results to the intended audience. These are not minor procedural blemishes. They explain why the report should be cited for its measured conclusion, not treated as a fully reproducible demonstration.

The original contract also included verification of how compute was allocated between capabilities and safety. Anthropic canceled that component midway, before METR reached substantive conclusions. Anthropic said it wanted the engagement to remain focused on risk assessment and would revisit compute verification later. The public assessment therefore cannot answer a basic governance question: whether the organization's stated safety effort corresponds to its actual allocation of scarce compute.

METR discloses a conflict too. One team member had a role discussion with Anthropic, received an offer, and declined it; the same person held equity in a competitor as a former employee. All of that person's work was checked by an unconflicted team member who took responsibility for its accuracy. No one working on the project held Anthropic equity. METR's later conflict-of-interest policy would require recusal in this situation. These disclosures do not invalidate the result, but they belong in the record because the report asks readers to trust institutional judgment.

There is a subtler adversarial risk in the analysis itself. Claude was involved in much of the data collection and analysis reviewed by METR, wrote a significant share of the documents METR reviewed, and was used for much of METR's own analysis. This creates a possible shared-model bias: both the evidence producer and the evaluator may be systematizing the same interpretation. METR checked for discrepancies against human interviews and did not detect a pattern consistent with systematic distortion. That is a useful check, not a proof that model-mediated analysis is independent.

9. What could make the assessment wrong?

METR lists ways it could underestimate acceleration. First, the growth rate may still be increasing. AECI does not show an obvious hyperbolic curve, but redacted evidence made a continuing acceleration more plausible. If the true process has a rising derivative, a single post-2025 slope understates the instantaneous rate and badly understates the near future. Second, autonomous contributions may be closer than the interviews suggest. Once a human bottleneck is removed, progress can jump sharply rather than rise smoothly. Third, benchmark scores may understate real-world impact when capabilities cross thresholds. A model can appear to improve gradually and then become useful for an entire class of work.

It could also overestimate acceleration. Non-AI factors may explain more of the bend than METR assigns them. AECI may privilege clean, automatically gradable tasks over messy research. Anthropic's historical trend may have been unusually low because it was below the frontier, making the denominator small and the ratio large. And accelerated benchmark scores can reflect benchmark-specific optimization rather than a general improvement in useful capability. The report treats these as live alternatives, not as caveats to be dismissed after the recommendation.

There is a measurement-lag problem in both directions. METR observes a model snapshot only after training finishes, so the measured capability outcome lags the research practices that produced it by roughly the training time. That can hide a recent change in the production process. It can also make a temporary burst look like a new trend if the next training run has not yet revealed its result. The eleven-week assessment window is long enough to interview and analyze, but short compared with the multi-year dynamics being inferred.

Finally, the counterfactual is not identified. METR can separate an observed pre-trend from an observed post-trend. It cannot directly observe what Anthropic would have achieved without AI. Hiring, data, algorithms, compute, organizational focus, and random research outcomes all move together. The report compensates with a broad evidence portfolio rather than a single model. That is appropriate for an exploratory high-access assessment, but it means the 80% attribution should not be treated with the same confidence as the 1.9x descriptive ratio.

10. A practical governance checklist

The most useful lesson is not that every AI lab is necessarily accelerating at 2x. It is that a credible acceleration assessment needs access, a counterfactual, and public evidence. The following checklist turns the report's strengths and failures into requirements for future evaluations.

1. Grant employee-level access, not only curated datasets. Evaluators should be able to follow questions on demand, inspect adjacent documents, observe internal debate, and discover evidence they did not know to request.
2. Define observation, historical, and counterfactual rates in advance. Publish the model, breakpoint search, dating rule, benchmark-selection rule, and the difference between instantaneous and cumulative attribution.
3. Keep the provider from unilaterally deleting decisive evidence. Redaction can protect genuine secrets, but the intended audience must still be able to review the reasoning that supports the headline. A right to suppress potentially concerning results is incompatible with independent verification.
4. Release a reproducible synthetic or privacy-preserving dataset. If raw internal metrics cannot be shared, the evaluator should publish a transformation that preserves the relevant growth ratios and permits independent re-analysis.
5. Interview a provider-independent sample. Primary technical contacts are valuable, but evaluators should select additional staff, document disagreement, and preserve counter-evidence.
6. Measure multiple bottleneck classes at once. Code speed, experiment throughput, training compute, data quality, evaluation capacity, and physical validation should not be collapsed into a single productivity number.
7. Test for autonomous thresholds. Assisted uplift and end-to-end automation are different risks. Track the fraction of important pipeline stages that can be completed without a human owner, not only survey estimates of time saved.
8. Preserve compute-allocation audits. A safety case is incomplete if an evaluator cannot verify how training, inference, and safety work share compute, especially when the original scope promised that verification.
9. Publish a falsification plan. State which future observations would lower the estimate: slowing AECI, a binding data or compute constraint, no growth in autonomous task share, or evidence that non-AI inputs explain the change.
10. Treat the report as a risk signal, not a certification. A high-access evaluation can improve judgment while remaining non-reproducible. The confidence label should travel with the estimate whenever it is used in policy or investment decisions.

11. What to watch next

For AI labs, the next useful observations are whether the post-2025 slope remains near 2x, whether the growth rate itself rises, whether new models continue to saturate AECI, and whether autonomous contributions move from "signs of life" to measurable ownership of major pipeline stages. A single model release cannot answer these questions. A sequence of snapshots with consistent dating and benchmark coverage can.

For robotics and physical AI, the key bridge is not an assumption that software acceleration transfers one-for-one. It is a measurement agenda. Which parts of a robot-learning loop are actually accelerated by AI? Does code generation shorten the cycle when hardware time is fixed? Do better models improve data selection, failure analysis, simulation design, or policy iteration? Does the organization generate more validated physical behavior, or only more unvalidated candidates? These questions would make the METR framework useful without overclaiming its scope.

The central empirical claim is still strong enough to matter: Anthropic's internal benchmark trend appears to have bent upward after October 2025, and the most likely ratio is near two. The central causal claim is softer: AI is probably a major contributor, perhaps the majority contributor today, but non-AI factors and measurement artifacts remain large. The central governance claim is the least ambiguous: the public cannot independently verify the evidence that most influenced METR's risk conclusion because Anthropic retained and exercised broad redaction rights.

That combination is the real finding. A company may be accelerating its own AI R&D while publishing enough aggregate evidence to support concern but not enough evidence to permit independent adjudication. For frontier AI and the embodied systems that will depend on it, the demand should be for both faster measurement and stronger rights to inspect the mechanism behind the number.

Source and citation

Original report: Independent Assessment of AI R&D Acceleration at Anthropic in October 2026, METR, November 5, 2026. Authors: Tom Cunningham, Nate Rush, Parker Whitfill, and Hjalmar Wijk.

This article is a complete editorial adaptation of the public report, with the redaction and access limitations kept alongside the findings. Numerical estimates, confidence intervals, interview summaries, and process details are attributed to METR. Interpretations about robotics and physical AI are editorial context, not claims measured by the original study.