Book Review: Reframing Superintelligence
Read the original on Slate Star Codex →
Summary
Reviews Eric Drexler's Reframing Superintelligence (Comprehensive AI Services). The CAIS thesis: future AI might be an ecosystem of narrow superintelligent SERVICES (a superintelligent Google Translate that just translates, doesn't want to take over the world) rather than a unified Bostromian agent; services are safer, don't self-improve, and could help contain rogue agents ('a good guy with an AI to stop a bad guy with an AI'). Scott finds it common-sense-in-retrospect (worrying he didn't generate it after reading Superintelligence in 2014), raises three sharp confusions (many tasks need general intelligence; is it easier to train services or agents; can agency be separated from cognition), and flags the Gwern/Yudkowsky tool-AI-becomes-agent-AI counterargument. A clear, useful exposition of an influential AI-safety framing.
Why this score
Quality 71 · Strong. Strong (71): a lucid, valuable review that made the CAIS framing accessible and honestly surfaces its open problems; held FIRM below Excellent as an exposition of Drexler's ideas with no firm conclusion.
Claude’s paradigm shift 54 · Moderate. Moderate (54): the services-vs-agents reframe is a fresh, non-obvious lens on AI risk (Drexler's, clarified by Scott).
Real-world impact 4 · Moderate. Moderate (4): CAIS became a reference point in AI-safety discussion, and this review is a common entry point to it; discourse-level influence in the field.