(Press-News.org) A group of Cambridge computer scientists have set a new gold standard for openness and reproducibility in research by sharing the more than 200GB of data and 20,000 lines of code behind their latest results - an unprecedented degree of openness in a peer-reviewed publication. The researchers hope that this new gold standard will be adopted by other fields, increasing the reliability of research results, especially for work which is publicly funded.
The researchers are presenting their results at a talk today (4 May) at the 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI) in Oakland, California.
In recent years there's been a great deal of discussion about so-called 'open access' publications - the idea that research publications, particularly those funded by public money, should be made publicly available.
Computer science has embraced open access more than many disciplines, with some publishers sub-licensing publications and allowing authors to publish them in open archives. However, as more and more corporations publish their research in academic journals, and as academics find themselves in a 'publish or perish' culture, the reliability of research results has come into question.
"Open access isn't as open as you think, especially when there are corporate interests involved," said Matthew Grosvenor, a PhD student from the University's Computer Laboratory, and the paper's lead author. "Due to commercial sensitivities, corporations are reluctant to make their code and data sets available when they publish in peer-reviewed journals. But without the code or data sets, the results are irrelevant - we can't know whether an experiment is the same if we try to recreate it."
Beyond computer science, a number of high-profile incidents of errors, fraud or misconduct have called quality standards in research into question. This has thrown the issue of reproducibility - that a result can be reliably repeated given the same conditions - into the spotlight.
"If a result cannot be reliably repeated, then how can we trust it?" said Grosvenor. "If you try to reproduce other people's work from the paper alone, you often end up with different numbers. Unless you have access to everything, it's useless to call a piece of research open source. It's either open source or it's not - you can't open source just a little bit."
With their most recent publication, Grosvenor and his colleagues have gone several steps beyond typical open access standards - setting a new gold standard for open and reproducible research. All of the experimental figures and tables in the award-winning final version of their paper, which describes a new method of making data centres more efficient, are clickable.
By clicking on any of the figures or tables in the paper, readers are taken to a website where the researchers have produced technically detailed descriptions of the methods for every one of their experiments. These descriptions include the original data sets and tools that were used to produce the figures as well as free and open source access to all of the source code that they wrote and modified.
In the past this might not have been possible, but thanks to cheap cloud storage, the researchers have put nearly 200GB of data and 20,000 lines of code on to the internet and made it freely available to all under a permissive open-source license.
"It now should be possible for anyone with a collection of computers to follow our instructions and produce our exact graphs," said Grosvenor. "We think that this is the way forward for all scientific publications and so we've put our money where our mouth is and done it."
INFORMATION:
PHILADELPHIA - Gene therapy for Leber congenital amaurosis (LCA), an inherited disorder that causes loss of night- and day-vision starting in childhood, improved patients' eyesight within weeks of treatment in a clinical trial of 15 children and adults at the Scheie Eye Institute at the Perelman School of Medicine at the University of Pennsylvania. New results involving a subset of patients from the ongoing trial show that these benefits peaked one to three years after treatment and then diminished. The findings are published today in The New England Journal of Medicine.
In ...
MEDFORD/SOMERVILLE, Mass. (May 3, 2015) -- A Tufts University study sheds new light on the process by which DNA repair occurs within the cell. In research published in the May 15 edition of the journal Genes & Development and available May 4 online in advance of print, Tufts University biologist Catherine Freudenreich and her co-authors show that expanded repeats of the CAG/CTG trinucleotide (CAG) in yeast shift to the periphery of the cell nucleus for repair. This shift is important for preventing repeat instability and genetic disease.
CAG expansions are significant ...
Young people are aware of the risks of cyberbullying but perceive others as being more at risk than themselves. Young women are more vulnerable to this perception than young men.
This is the finding of a study by Dr Lucy Betts and Sondos Metwally from Nottingham Trent University (NTU) that will be presented as part of the poster presentation session at the British Psychological Society's Annual Conference next week (Thursday 7 May 2015) hosted in Liverpool.
A survey, designed to measure how vulnerable young people felt to cyberbullying and how vulnerable they felt ...
PHOENIX and SAN FRANCISCO -- May 1, 2015 -- Led by the Translational Genomics Research Institute (TGen) and UC San Francisco (UCSF), a comprehensive genetic review of treatment strategies for glioblastoma brain tumors was published today in the Oxford University Press journal Neuro-Oncology.
The study, Towards Precision Medicine in Glioblastoma: The Promise and The Challenges, covers how these highly invasive and almost-always-deadly brain cancers may be treated, reviews the continuing challenges faced by researchers and clinicians, and presents the hope for better treatments ...
Young children with autism spectrum disorder, who also have serious behavioral problems, showed improved behavior when their parents were trained with specific, structured strategies to manage tantrums, aggression, self-injury, and non-compliance.
The findings from this parent training study by Yale and Emory University researchers were published recently in the Journal of the American Medical Association (JAMA).
Autism spectrum disorder (ASD) is a chronic condition beginning in early childhood and defined by impaired social communication and repetitive behavior. ...
Expanding what we know about college students mixing alcohol with energy drinks, investigators from Dartmouth's Norris Cotton Cancer Center found teens aged 15-17 years old who had ever mixed alcohol with energy drinks were four times more likely to meet the criteria for alcohol use disorder than a teen who has tried alcohol but never mixed it with an energy drink. The Dartmouth team, led by James D. Sargent, MD with first author Jennifer A. Emond, MSc, PhD published "Energy drink consumption and the risk of alcohol use disorder among a national sample of adolescents and ...
Procrastination is the thief of time that derails New Year's resolutions and delays saving for college or retirement, but researchers have found a way to collar it.
The trick? Think of the future as now.
"The simplified message that we learned in these studies is if the future doesn't feel imminent, then, even if it's important, people won't start working on their goals," said Daphna Oyserman, lead researcher and co-director of the USC Dornsife Mind and Society Center.
Through a series of scenarios, Oyserman and co-author Neil Lewis Jr. of the University of Michigan ...
Shopping online or in catalogs is great for many reasons: to while away time on a snowy day; to avoid the holiday crush at the local mall; to do ultra-efficient comparison shopping; to enjoy a world of choice at your fingertips. But if you need a pair of shoes for the party tonight? Not so much.
Online and catalog retailers pondering whether to add physical stores to their customers' buying options can look to recent research by marketing professors Koen Pauwels and Scott A. Neslin for valuable insights on the interplay among the various channels.
In "Building with ...
With major funding from several groups, including NSERC, an SFU doctoral student has made a key discovery regarding Fraser River sockeye's vulnerability to sea lice.
Recently published research indicates that juvenile Fraser River sockeye salmon that are highly infected with sea lice are 20 per cent less successful at consuming food than their lightly infected counterparts. Sean Godwin, a Simon Fraser University doctoral biology student is the lead author of a study, co-authored by SFU biologists John Reynolds and Larry Dill (emeritus), and University of Toronto researcher ...
Researchers at UC San Diego School of Medicine conducted the first population-based study that characterizes the association and temporal relationship between gastrointestinal stromal tumors (GIST) and other cancers. The results, published by Cancer on April 30, indicate that one in 5.8 patients with GIST will develop additional malignancies before and after their diagnosis.
Specifically, patients with GIST are more likely to develop other sarcomas, non-Hodgkin's lymphoma, carcinoid tumors, melanoma, colorectal, esophageal, pancreatic, hepatobiliary, non-small cell lung, ...
