7 Reasons Everything You Know Is Probably Wrong

4. The Irreproducibility Problem

So, p-hacking is bad, and there are a lot of false positives floating around science, and a lot of bulls**t in our newspaper headlines, but one good way to sort out the truth from the lies is to simply take the studies and try to reproduce the results.

Worryingly, since people have been doing this, it has become woefully apparent that a vast number of studies, some of which are seen as groundbreaking, can't be replicated and this is a massive problem. One of the most famous cases of this is a replication study in psychology, that found that less than half of psychology studies can be replicated. However, that replication study itself has since been accused of containing replication errors - this gives you an idea of just how pant-crappingly complicated this is getting.

It's not just a problem in the admittedly wooly field of psychology either, with a case a few years ago in which an American drug company tried to replicate 53 studies that they considered landmarks in the basic science of cancer. Despite working closely with many of the original researchers to ensure they stuck to their methods, they were only able to reproduce the original results in just six of the studies.

So, the next time a headline screams something like "NEW STUDY SHOWS OWNING A PARROT CAN PREVENT CANCER", it might be worth taking it with a pinch of salt. Speaking of media hype...