‘Robot scientist’ Eve finds that less than one third of scientific results are reproducible

Science Daily  April 6, 2022
The reproducibility and robustness of only a small fraction of published biomedical results has been tested. An international team of researchers (UK, USA – University of Chicago) used a combination of automated text analysis and the ‘robot scientist’ Eve to semi-automate the process of finding papers reporting reproducible results. Out of the more than 12,000 research papers on breast cancer cell biology, after narrowing down to 74 papers of high scientific interest, only 22 papers were found to be reproducible. Two different human teams used Eve and two breast cancer cell lines and attempted to reproduce the 74 results. Statistically significant evidence for repeatability was found for 43 papers, meaning that the results were replicable under identical conditions; and significant evidence for reproducibility or robustness was found in 22 papers, meaning the results were replicable by different scientists under similar conditions. In two cases, the automation made serendipitous discoveries…read more. Open Access TECHNICAL ARTICLE