Robert J. Niewiadomski
17th September 2012, 12:46
Is medical science built on shaky foundations?
17 September 2012 by Elizabeth Iorns
More than half of biomedical findings cannot be reproduced – we urgently need a way to ensure that discoveries are properly checked
REPRODUCIBILITY is the cornerstone of science. What we hold as definitive scientific fact has been tested over and over again. Even when a fact has been tested in this way, it may still be superseded by new knowledge. Newtonian mechanics became a special case of Einstein's general relativity; molecular biology's mantra "one gene, one protein" became a special case of DNA transcription and translation.
One goal of scientific publication is to share results in enough detail to allow other research teams to reproduce them and build on them. However, many recent reports have raised the alarm that a shocking amount of the published literature in fields ranging from cancer biology to psychology is not reproducible.
Pharmaceuticals company Bayer, for example, recently revealed that it fails to replicate about two-thirds of published studies identifying possible drug targets (Nature Reviews Drug Discovery, vol 10, p 712).
Bayer's rival Amgen reported an even higher rate of failure - over the past decade its oncology and haematology researchers could not replicate 47 of 53 highly promising results they examined (Nature, vol 483, p 531). Because drug companies scour the scientific literature for promising leads, this is a good way to estimate how much biomedical research cannot be replicated. The answer: the majority.
The reasons for this are myriad. The natural world is complex, and experimental methods do not always capture all possible variables. Funding is limited and the need to publish quickly is increasing.
There are human factors, too. The pressure to cut corners, to see what one wants and believes to be true, to extract a positive outcome from months or years of hard work, and the impossibility of being an expert in all the experimental techniques required in a high-impact paper are all contributing factors.
The cost of this failure is high. As I have experienced at first hand as a researcher, attempts to reproduce others' published findings can be expensive and frustrating. Drug companies have spent vast amounts of time and money trying and failing to reproduce potential drug targets reported in the scientific literature - resources that should have contributed towards curing diseases.
Failed replications also quite often go unpublished, thereby leading others to repeat the same failed efforts. In the modern fast-paced world, the normal self-correcting process of science is too slow and too inefficient to continue unaided.
Many have wrung their hands and proposed various penalties for scientific studies that cannot be reproduced. But instead of punishing investigators, what if there was a way of rewarding them for pursuing independent replication of their most significant scientific results - the ones they want to see cited and built on - before or shortly after publication? I believe this could be a substantial boon to science and society, which is why I started the Reproducibility Initiative.
(...)
Full story: http://www.newscientist.com/article/mg21528826.000-is-medical-science-built-on-shaky-foundations.html
17 September 2012 by Elizabeth Iorns
More than half of biomedical findings cannot be reproduced – we urgently need a way to ensure that discoveries are properly checked
REPRODUCIBILITY is the cornerstone of science. What we hold as definitive scientific fact has been tested over and over again. Even when a fact has been tested in this way, it may still be superseded by new knowledge. Newtonian mechanics became a special case of Einstein's general relativity; molecular biology's mantra "one gene, one protein" became a special case of DNA transcription and translation.
One goal of scientific publication is to share results in enough detail to allow other research teams to reproduce them and build on them. However, many recent reports have raised the alarm that a shocking amount of the published literature in fields ranging from cancer biology to psychology is not reproducible.
Pharmaceuticals company Bayer, for example, recently revealed that it fails to replicate about two-thirds of published studies identifying possible drug targets (Nature Reviews Drug Discovery, vol 10, p 712).
Bayer's rival Amgen reported an even higher rate of failure - over the past decade its oncology and haematology researchers could not replicate 47 of 53 highly promising results they examined (Nature, vol 483, p 531). Because drug companies scour the scientific literature for promising leads, this is a good way to estimate how much biomedical research cannot be replicated. The answer: the majority.
The reasons for this are myriad. The natural world is complex, and experimental methods do not always capture all possible variables. Funding is limited and the need to publish quickly is increasing.
There are human factors, too. The pressure to cut corners, to see what one wants and believes to be true, to extract a positive outcome from months or years of hard work, and the impossibility of being an expert in all the experimental techniques required in a high-impact paper are all contributing factors.
The cost of this failure is high. As I have experienced at first hand as a researcher, attempts to reproduce others' published findings can be expensive and frustrating. Drug companies have spent vast amounts of time and money trying and failing to reproduce potential drug targets reported in the scientific literature - resources that should have contributed towards curing diseases.
Failed replications also quite often go unpublished, thereby leading others to repeat the same failed efforts. In the modern fast-paced world, the normal self-correcting process of science is too slow and too inefficient to continue unaided.
Many have wrung their hands and proposed various penalties for scientific studies that cannot be reproduced. But instead of punishing investigators, what if there was a way of rewarding them for pursuing independent replication of their most significant scientific results - the ones they want to see cited and built on - before or shortly after publication? I believe this could be a substantial boon to science and society, which is why I started the Reproducibility Initiative.
(...)
Full story: http://www.newscientist.com/article/mg21528826.000-is-medical-science-built-on-shaky-foundations.html