Kelly Bets On Civilization
Read the original on Astral Codex Ten →
Summary
Scott's response to Scott Aaronson's 'don't be maximally hostile to AI' argument (the nuclear-power-fear cautionary tale + IRB/YIMBY analogies). Scott concedes the progress-fear pattern is real, but counters: (1) you're allowed Inside View ('you are Osama bin Laden, and this is a supervirus lab' - AI's failure mode is uniquely deceptive: it pretends to work, bides time, is smarter than you); (2) the Kelly-betting argument - even on a great bet, never bet EVERYTHING; science/tech is a great bet but a world-destroying technology is betting 100%, and 10 AI-like 50-50 bets = a 1023/1024 chance everyone's dead.
Why this score
Quality 74 · Strong. A sharp, memorable AI-risk piece: the Kelly-betting framing of civilizational x-risk (don't bet everything even on a great bet; world-ending tech is betting 100%) is a genuinely good, portable contribution, and the Inside-View/Osama-supervirus rebuttal is sharp. Concedes Aaronson's valid point while rigorously carving out the x-risk exception. Top-Strong, with the AI cluster.
Claude’s paradigm shift 54 · Moderate. Moderate-high - the Kelly-betting framing of x-risk is a fresh, sharp, portable application.
Real-world impact 2 · Minor. 2 - a sharp AI-risk framing but a brief response piece; within-discourse.