Scott Alexander, curated
← Back to curation

Mantic Monday: Scoring Rule Controversy

Quality
65
Strong
Claude Shift
44
Moderate
RWI
1
of 10

Summary

A Mantic Monday built around two genuine forecasting-mechanism problems. First, the Metaculus scoring-rule controversy (Zvi/Ross): the rule is 'proper' (incentivizes honest probabilities) but positive-sum, so on many questions you gain points even when maximally wrong — rewarding spam over careful work; Aguirre's defense is that spammers optimally just echo the existing consensus, so accuracy is unharmed even if the leaderboard is polluted. Second, the 'how do you run a 50-100-year prediction market?' problem — you can profit before resolution if information trickles out (but not if it all arrives at the end), the index-fund-the-stake fix only marginally beats just investing, and Scott's realistic conclusion that markets lose reliability beyond a few decades, with the fallback of using proven short-term forecasters pro bono for long-term questions.

Why this score

Quality 65 · Strong. Strong-floor (65, top Monday band). Two substantive mechanism-design discussions (the proper-but-spammable scoring rule, the long-horizon-market problem) lift it above the roundups.

Claude’s paradigm shift 44 · Moderate. Moderate (44). Sharpens real forecasting-design frames without originating them.

Real-world impact 1 · Negligible. 1 — a forecasting roundup; negligible real-world effect.