aurora network logo

Social Science Replicability

VU International News and Reviews No. 119 September 10 2018

July 2019

A group of scholars with Colin Camerer from CalTech as corresponding author, report on an analysis of the replicability of social science experiments and if peers could predict the predictable.

The article “Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015” reports on the replication of 21 systematically selected experimental studies in social science, with replication samples that were substantially larger than the original. They found a significant effect in the same direction as the original study for 62% of the studies, and the effect size of the replications is on average about  50% of the original effect size.

More interestingly and explained in the Supplementary Information, they investigated the predictive power of peers both through a survey and through a market model in which peers could put their money of the likelihood of replicability. As it turned out, peers have a pretty shrewd idea about which social science experiments are less likely to yield replicable results.

Back to newsletters