Anthropic system card doubts rapid AI self-improvement
As frontier labs deploy thousands of agents, Anthropic system cards and expert predictions suggest recursive self-improvement will yield gradual efficiency rather than sudden superintelligence.
Anthropic recently shed light on the limits of recursive self-improvement in its Claude Fable 5.1 and Mythos 5.1 System Card. The company noted that while internal model usage helps maintain their current pace of development, they have not observed signs of dramatic acceleration beyond that rate. This admission comes amid intense industry debate over whether AI agents can autonomously upgrade themselves to trigger an intelligence explosion, or if progress will follow a more modest path of lossy self-improvement.
Industry experts remain divided on when these technologies will achieve major milestones. In a recent discussion summarized by GPT-6-Astra, researchers John Schulman, Beren Millidge, and Charlie O’Neill offered varying timelines. For a drop-in remote worker, Schulman estimated about 1 year for an okay version, while O’Neill predicted 1 year with programmatic tools and 2 years via a browser, and Millidge estimated 3 years for full generality. Regarding a tenfold productivity boost for AI researchers, O’Neill estimated 5 to 10 years, while Millidge found Schulman's 2-year estimate plausible. For artificial superintelligence, Schulman predicted 3 to 4 years, Millidge estimated 5 years for focused areas, and O’Neill anticipated 5 to 10 years. Meanwhile, researcher Richard Ngo predicted that superintelligence will not arrive within the next 8 years.
For AI practitioners, these projections highlight that recursive self-improvement is currently more effective at driving down inference costs than expanding peak intelligence. Achieving linear improvements in intelligence still requires exponential compute, meaning that resource bottlenecks and post-training complexities remain significant hurdles. Schulman pointed out that automating post-training is exceptionally difficult because humans must still define how models should behave in specific areas, and errors are easy to make without them showing up on standard benchmarks.
Instead of preparing for an immediate singularity, developers should focus on the economic transformation of scaled, multi-agent systems. The near-term integration of agent swarms will likely optimize software engineering, log monitoring, and experimental workflows. While these tools will make LLMs vastly cheaper to run, human intuition and hypothesis generation will remain the primary bottlenecks in scientific discovery.
This is our own summary of reporting by Interconnects
