Reproducibility Crisis
Understanding Research Methods
To better grasp the replication crisis, it’s worth exploring some of the statistical methods used in psychology experiments. Flexibility in research methodology can help explain why researchers unconsciously (and sometimes consciously) produce unreliable results.
When conducting an experiment, a researcher develops a hypothesis. For example, they may hypothesize that spending time with friends makes people happier. They then seek to reject the null hypothesis —the possibility that there is no association or effect of the sort the researchers propose. In this case, the null hypothesis would be that there is no relationship between happiness and spending time with friends.
A finding is said to be statistically significant if the results of a study based on a particular sample of people are thought to be likely to generalize to the broader population of interest. A traditional benchmark of statistical significance in psychology is a p-value of .05, though more stringent benchmarks have recently been proposed.
The p-value is a measure to determine statistical significance. Roughly speaking, a p-value is the probability of obtaining a study result by random chance if the null hypothesis is true. The smaller the p-value, the less likely it is that an observed result would be found in the absence of a real effect or association between variables. The threshold for significance is traditionally a p-value of less than .05 , although the replication crisis has led researchers to rethink relying on p-values or to propose changing the threshold for what counts as "significant" to a lower p-value (such as .005). The fact that .05 is an arbitrary benchmark is, for some, further evidence that p-values are given too much credence.
Explore More About Replication Crisis
For a comprehensive understanding of replication crisis, read our complete guide: