Scott Alexander, curated
← Back to curation

Highlights From The Comments On Motivated Reasoning And Reinforcement Learning

Quality
72
Strong
Claude Shift
52
Moderate
RWI
1
of 10

Summary

A rich Highlights companion on the 'why can we always see the lion but not open the IRS letter' motivated-reasoning-as-misapplied-RL thesis. Working neuroscientists (Gabriel, Fox, Steve Byrnes) supply real mechanism — value is learned as a topographic map and acted on by gradient; inferotemporal cortex/superior colliculus are rewarded for 'scary/exciting' not 'life going well', so noticing the lion is reinforced by a different signal than deciding to open an email. Then several commenters argue the discrepancies dissolve into ordinary long-horizon RL cost-benefit, and Scott pushes back hard with two strong original moves: the HODL Bitcoin-arrow thought experiment (no final reinforcement ever arrives, yet you don't hallucinate green arrows) and the small-genome argument (self-deception is cheap to implement as a corollary of existing RL, expensive to hand-code from the genome), landing on motivated-reasoning-as-misapplied-RL as the 'how' behind Trivers' 'why'.

Why this score

Quality 72 · Strong. Strong, upper: among the richer companions of the run — genuine neuroscience content plus two memorable Scott-original arguments (the Bitcoin arrow, the genome-bandwidth penalty) that do real explanatory work. Consistent with the rich-companion band (72). 72.

Claude’s paradigm shift 52 · Moderate. Moderate/Notable edge: the RL-implementation framing of self-deception is a fresh, non-obvious synthesis for informed readers, though it builds on Trivers and standard RL. 52.

Real-world impact 1 · Negligible. Influential only within the rationalist/epistemics discussion; no material reach. RWI 1.