Online games and remote experiments could reduce scientific fraud, cherry-picking

(Press-News.org) PITTSBURGH—One way to combat the rising level of errors and fraud in life sciences research is through massive online laboratories, which use videogames to engage large numbers of non-professional investigators and prevent scientists from manually testing their own hypotheses, researchers from Carnegie Mellon University and Stanford University say.

Though unconventional, CMU's Adrien Treuille and Stanford's Rhiju Das argue that this online, game-like approach actually is more scientifically rigorous than the standard practice of scientists proposing an explanation for some phenomenon and then testing that hypothesis through experimentation. In a commentary published online today by the journal Trends in Biochemical Sciences, they maintain that massive online labs could be a model for the entirety of science.

Earlier this year, Treuille and Das reported in the Proceedings of the National Academy of Sciences on the success of their own online lab, an RNA-design project called EteRNA, which has produced unprecedented design insights that have advanced knowledge of RNA (ribonucleic acid).

"If you strip away the game part, projects such as EteRNA present a fundamentally new model of remote science that can prevent many common forms of scientific fraud," said Treuille, an assistant professor of computer science and robotics at Carnegie Mellon. "We registered more than 150,000 participants who contributed in excess of 2 million human-hours to EteRNA. That means there were a lot of eyes, a lot of people looking over each other's shoulders as hypotheses were developed and experimental results evaluated. Everything is out in the open."

Online participants use computer design tools to propose RNA designs that meet certain criteria. The designs are then synthesized in the Stanford lab of Das, an assistant professor of biochemistry. The results are made available to the entire EteRNA community for analysis and use in future design challenges.

Treuille said that transparency makes it difficult for any individual to retrospectively adjust scientific hypotheses to match experimental results, or to cherry-pick data to reflect a scientist's biases. "It's not typically acknowledged, but having the same team both develop a hypothesis and test it in the lab creates a conflict of interest — something that may be contributing to a plague of irreproducible results in many research studies," Treuille said. "EteRNA separates these two important processes without inhibiting the science."

The sheer number of people involved in the research also addresses a problem associated with a hallmark of modern biology — the use of high-throughput experimental techniques.

"A single DNA or RNA sequencing run can generate billions of data points," Das noted. "Following up those experiments with cycles of hypothesis generation and testing is critical to establishing scientific truth. But it can be expensive and time-consuming. It's tempting to skip the extra work and cherry-pick the data to produce a publishable manuscript.

"If you have thousands of people helping you, however, the data deluge isn't so bad," Das added.

Though people attracted to EteRNA by its game-like elements were not necessarily trained scientists, many developed deep understandings of RNA design as they used computerized tools to design molecules and then saw which designs were successful when synthesized in Das' lab.

Treuille and Das said setting up a massive online lab isn't cheap, but should be within the budget of a conventional life sciences grant.

"The major current barrier may instead be the career risks that these projects pose for their creators," they write in the new position paper. "In particular, videogames, which appear critical for recruiting scientifically engaged citizens, are generally viewed as incompatible with 'serious,' or rigorous research."

INFORMATION:

The EteRNA project was supported in part by the Burroughs-Wellcome Foundation and the W.M. Keck Medical Research Foundation. In addition to the main EteRNA website, http://eterna.cmu.edu/web/, the project also can be accessed as RNA VirtuaLab at NOVA Labs, a digital platform for citizen science created by the PBS science series NOVA.

About Carnegie Mellon University: Carnegie Mellon is a private, internationally ranked research university with programs in areas ranging from science, technology and business, to public policy, the humanities and the arts. More than 12,000 students in the university's seven schools and colleges benefit from a small student-to-faculty ratio and an education characterized by its focus on creating and implementing solutions for real problems, interdisciplinary collaboration and innovation. A global university, Carnegie Mellon has campuses in Pittsburgh, Pa., California's Silicon Valley and Qatar, and programs in Africa, Asia, Australia, Europe and Mexico.