A recent article by the Open Science Collaboration (a group of 270 coauthors) gained considerable academic and public attention due to its sensational conclusion that the replicability of psychological science is surprisingly low. Science magazine lauded this article as one of the top 10 scientific breakthroughs of the year across all fields of science reports of which appeared on the front pages of newspapers worldwide. We show that OSC's article contains three major statistical errors and, when corrected, provides no evidence of a replication crisis. Indeed, the evidence is also consistent with the opposite conclusion -- that the reproducibility of psychological science is quite high and, in fact, statistically indistinguishable from 100%. The moral of the story is that meta-science must follow the rules of science.
Our Technical Comment: Gilbert, Daniel; Gary King; Stephen Pettigrew; and Timothy Wilson. "Comment on 'Estimating the reproducibility of psychological science,'" Science. (4 March 2016), Vol 351, Issue 6277, Pp. 1037a-1037b. [download article, with supplementary appendix] [download replication data]
Related materials:
- "Study that undercut psych research got it wrong" by Peter Reuell
- Open Science Collaboration reply to our Technical Comment
- Our response to Open Science Collaboration's reply to our Technical Comment
- Our reply to post-publication discussions of our Technical Comment
Authors
Edgar Pierce Professor of Psychology, Harvard University |
Albert J. Weatherhead III University Professor, Harvard University |
PhD candidate, Department of Government, Harvard University |
Sherrell J. Ashton Professor of Psychology, University of Virginia |