OpenAI and Anthropic are already running thousands of concurrent AI agents internally. That operational reality is reshaping how employees at both labs perceive risk timelines, and the cultural anxiety this creates is doing more to drive extinction-level discourse than any verified technical breakthrough. The author, drawing on Richard Ngo's LessWrong post, argues the AI safety community will be directionally correct but factually wrong: no superintelligence within 8 years, but progress fast enough that short-timeline advocates will feel vindicated anyway.
The core technical argument here is called lossy self-improvement, the author's alternative to true recursive self-improvement. Three constraints cap RSI's ceiling: automatable research is too narrow to overcome exponential scaling costs, parallel agents face real diminishing returns, and resource and political bottlenecks are largely AI-resistant. Two Dwarkesh podcasts, one with Noam Brown and one with John Schulman, Beren Millidge, and Charlie O'Neill, sharpen this. Brown's episode clarifies that massive inference-time scaling is real and predictable, but that is not the same thing as RSI. The trio's episode is worth reading for the gap between their sober technical analysis of RL, distillation, and scaling in the first hour, and their surprisingly short timeline predictions at the end.
The author holds genuine uncertainty about one thing: whether the labs have seen specific, non-public breakthroughs that would force an update. Absent that, the current AI safety temperature looks more like a cultural artifact of scaled agents entering production than evidence of imminent singularity. Read the full piece for the breakdown of those trio timeline predictions and the exact conditions the author says would change his mind.
[READ ORIGINAL →]