Scott Alexander, curated
← Back to curation

ACX Reader Research Survey: Call For Submissions

Quality
53
Solid
Claude Shift
36
Slight
RWI
3
of 10

Follow-up reading

Reader Survey Final Check-In — the confirmed roster for this call · Jul 2021

Summary

The founding document of the ACX Reader Research Survey, and considerably more than a notice. Scott explains the problem it solves: he had run yearly December surveys, researchers kept asking to piggyback questions onto them, he always declined because saying yes to everyone would need a whole extra survey — and then realised he should just run the extra survey. The immediate spur was meditation researcher Daniel Ingram, who had asked to attach questions to the SSC survey before Scott shut the blog down in 2020, killing the plan; this is explicitly restitution for that. He makes the case for the readership as a research panel by naming the demographics it over-samples (tech, science, meditation/drugs/biohacking, unusual genders and sexualities, psychiatric issues), then sets out the actual mechanism: researchers email a description plus an un-password-protected Google Form of ideally under 25 questions by July 10; a central form assigns every respondent a User ID and collects age, sex, country, race and education; each respondent then fills out some random subset of the submitted surveys, carrying their User ID across all of them; the ID-keyed demographic file goes to every participating researcher at the end. He allows targeted non-random surveys aimed at particular demographics, warns that submission does not guarantee inclusion (he may triage projects that look unprofessional or boring, or if he is overwhelmed), and offers to write up interesting findings as posts and link any resulting papers.

Why this score

Quality 53 · Solid. 53 — upper-mid Solid, level with the two call-for-X anchors it most resembles: SSC-896 'Book Review Contest: Call For Entries' (53) and SSC-661 'Call For Adversarial Collaborations' (53). Like those it founds a recurring program rather than merely announcing an event, and roughly half its length is genuine design reasoning — the User-ID linkage across independently-authored forms is the load-bearing idea, since it is what turns a pile of separate Google Forms into one linkable dataset. Held at 53 and no higher because it is still scaffolding: no findings, no argument, and its value is entirely instrumental to work other people would do.

Claude’s paradigm shift 36 · Slight. 36 — Slight-to-Moderate. Reader surveys were long-established at SSC and outside researchers had already been asking to piggyback, so the raw idea was in circulation. What is mildly novel is the formalisation: treating a blog readership as a shared, ID-linked research panel that third-party researchers apply to use, with the blogger as triage layer. That is a real organisational step beyond 'I will ask your question in my survey'.

Real-world impact 3 · Moderate. 3 — Moderate, niche/professional. Unlike most admin posts this one is the actual recruitment mechanism for a program that produced real research by outside professional and amateur investigators; without this call there is no panel and no dataset. Level with SSC-661, the other call that founded a research-shaped program, and above the within-blog 2s — but nowhere near policy or institutional change.