AI alignment research is solving the wrong problem. The field, championed by organizations like OpenAI and Anthropic and ranked the single highest-impact career path by EA organization 80,000 Hours, is structured around preventing superintelligence from causing human extinction. Meanwhile, real, present-day harms from algorithmic systems get subordinated to speculative doomsday scenarios that conveniently justify building more powerful products.
The core contradiction is in the mission statements. OpenAI publicly names building superintelligence as a primary goal, justifying it with two arguments: it will produce enormous economic growth, and someone else will build it anyway. Nick Bostrom, author of Superintelligence and the intellectual architect of EA, defines alignment as ensuring AI systems do what their builders want. The problem is that the builders are private companies with revenue obligations. Anthropic was founded by OpenAI alumni worried about OpenAI's commercial turn, then built its own product page with pull quotes and feature illustrations.
The piece is worth reading in full because the argument is not simply that AI is overhyped. It traces how the valorization of extinction-risk thinking creates the exact conditions it claims to prevent, by concentrating power and capital in the handful of labs best positioned to frame safety as their competitive advantage. The structural critique of how EA ideology and startup incentives reinforce each other is the part no summary can replace.
[READ ORIGINAL →]