The AI Superforecasters Are Here
Read the original on Astral Codex Ten →
Follow-up reading
↳ Does Forecasting Have Room At The Top? — further analysis: how far above humans AI forecasters can go · Aug 2026
Summary
A rich, forward-looking synthesis on AI reaching top-human forecasting ability (2026: prediction markets went billion-dollar; AI superforecasters made real Kalshi/stock-market profits). Hands-on test (FutureSearch/Preseen give ~7-9% on 'will colds halve by 2040,' matching a human superforecaster), the man-vs-machine state (statistical dead heat; the Metaculus Cup 'John Henry' framing; ~0.9 Elo/month trendline -> parity within ~6-12 months). Three underappreciated AI-forecaster advantages (feels-like-what-AI-should-do so escapes 'that's too precise'; standardized brand like a universal Nate Silver; non-adversarial unlike prediction markets). Flags Kapoor & Narayanan's forecasting-as-irreducible-error prediction as the decisive test case for near-term superintelligence. Two big ideas: superforecasting as the certified 'opinion layer' of AI (a middle ground between fact and opinion -- 'AI, should I marry this person? -> 85% divorce in 5 years'); and prediction markets surviving on CANONICITY (not accuracy) to arbitrate between biased AIs. Self-aware coda: the AIs give his own AI-treaty movement 1-2%, and he still rejects them.
Why this score
Quality 78 · Excellent. Excellent (low)/upper-Strong: a lucid, forward-looking synthesis with several genuinely fresh, portable ideas (superforecasting as AI's 'opinion layer'; forecasting as the S&A irreducible-error test case; canonicity-over-accuracy for markets), grounded in hands-on testing; among the strongest recent posts.
Claude’s paradigm shift 54 · Moderate. Notable (upper): 'superforecasting as the certified opinion layer of AI' and 'forecasting as the decisive test of the irreducible-error thesis' are fresh, non-obvious frames on where AI capability is heading.
Real-world impact 2 · Minor. Minor: analysis/advocacy that nudged one product (FutureSearch added causal conditional forecasting after his draft), but the post's own material footprint is small.