Study explores effects of review setting on scientific peer review

New research finds that using teleconference technology may not alter the outcomes of research grant review panels

2013-08-08

(Press-News.org) RESTON, VIRGINIA – Research findings published today in PLOS ONE report that the setting in which a scientific peer review panel evaluates grant applications does not necessarily impact the outcome of the review process. However, the research found that the average amount of discussion panelists engage in during the review is reduced. The investigation examined more than 1,600 grant application reviews coordinated by the American Institute of Biological Sciences Scientific Peer Advisory and Review Services (AIBS SPARS) on behalf of a federal agency over a four-year period.

The researchers compared two years when grant applications were reviewed using an in-person peer review panel to two years when panels were conducted via teleconference or videoconference.

Funding organizations routinely bring experts together to review research grant applications. A process known as scientific peer review, the goal of these panels is to identify the best research applications.

"There are no studies exploring whether the review setting significantly alters the quality of the peer review process," stated Dr. Stephen Gallo, the lead author of this study and Technical Operations Manager for AIBS SPARS.

"Our goal is always a reliable and high-quality peer-review process. It is important that we understand the strengths and weaknesses of different peer review methods," said Scott Glisson, Director of AIBS SPARS and an author of this study.

The findings appear in "Teleconference Versus Face-To-Face Scientific Peer Review of Grant Applications: Effects on Review Outcomes" published in PLOS ONE.

"Little difference was found in most of the review metrics between face-to-face and teleconference settings," said Gallo. Application scoring was only modestly affected and reviewers used the full scoring range regardless of review setting. The reviewer ratings were highly reliable in both settings.

Often, the greatest anticipated difference between in-person and teleconference panels is the amount of time allocated to discussing applications. This study found teleconference or videoconference panels allocated less time to application discussions than in-person panels.

"This is a first of its kind study that provides valuable data to help research program managers select appropriate models for conducting peer review," said Glisson.

More research is needed. "We should know whether the reduced amount of discussion and peripheral interactions that occur in a teleconference setting influence the final panel outcomes, and, ultimately the productivity of the research that is funded," said Gallo.

### For more information about AIBS visit http://www.aibs.org, or follow AIBS SPARS on Twitter: @AIBS_SPARS.

ELSE PRESS RELEASES FROM THIS DATE:

Heat intensifies Siberian wildfires

2013-08-08

The summer of 2012 was the most severe wildfire season Russia had faced in a decade. 2013 might be headed in the same direction after an unusual heat wave brought a surge of fire activity in northern Siberia in July. A persistent high-pressure weather pattern in the Russian Arctic—a blocking high—contributed to the heat wave, which saw temperatures reach 32° Celsius (90° Fahrenheit) in the northern city of Norilsk. For comparison, daily July highs in Norilsk average 16° Celsius (61° Fahrenheit). Blocking highs are so named because they block the jet stream from moving ...

Carbon under pressure exhibits interesting traits

2013-08-08

High pressures and temperatures cause materials to exhibit unusual properties, some of which can be special. Understanding such new properties is important for developing new materials for desired industrial uses and also for understanding the interior of Earth, where everything is hot and squeezed. A paper in Nature Geoscience highlights a new technique in which small amounts of a sample can be studied while being hot and squeezed within an electron microscope. Use of such a microscopy method permits determination of details down to the scale of a few atoms, including ...

Is Europa habitable?

2013-08-08

New Rochelle, NY, August 7, 2013—Europa, the ice-covered moon of the planet Jupiter, may be able to support life. NASA has commissioned a team of expert scientists to consider the science goals for a landed spacecraft mission to the surface of Europa, and to investigate the composition and geology of its icy shell and the potential for life within its interior ocean. The NASA-appointed Science Definition Team outlines the main priorities of a future lander mission to Europa to study its potential habitability in an article in Astrobiology, a peer-reviewed journal from Mary ...

Protein involved in nerve-cell migration implicated in spread of brain cancer

2013-08-08

The invasion of brain-tumor cells into surrounding tissue requires the same protein molecule that neurons need to migrate into position as they differentiate and mature, according to new research from the University of Illinois at Chicago College of Medicine and published August 7 in the online journal PLOS ONE. The researchers investigated similarities between the transition of neural stem cells into neurons and the process whereby cancer cells invade surrounding tissues. "Both processes involve the mobilization of cells," says Anjen Chenn, director of clinical pathology ...

Study casts doubt on theory that retired NFL players suffer unique cognitive disorder

2013-08-08

MAYWOOD, Il. – The media have widely reported that retired NFL players are at risk for a neurodegenerative disorder called chronic traumatic encephalopathy (CTE), which causes symptoms such as aggression, depression, suicidality and progressive dementia. But a study of retired NFL players, led by Christopher Randolph, PhD, of Loyola University Medical Center, has found no evidence to support this theory. Randolph and colleagues report their findings in the Journal of the International Neuropsychological Society. While there are no clearly defined diagnostic criteria ...

Eavesdropping plants prepare to be attacked

2013-08-08

MADISON — In a world full of hungry predators, prey animals must be constantly vigilant to avoid getting eaten. But plants face a particular challenge when it comes to defending themselves. "One of the things that makes plants so ecologically interesting is that they can't run away," says John Orrock, a zoology professor at the University of Wisconsin–Madison. "You can't run, you can't necessarily hide, so what can you do? Some plants make themselves less tasty." Some do this either by boosting their production of toxic or unpleasant-tasting chemicals (think cyanide, ...

Of stars and stripes: NASA satellites used to predict zebra migrations

2013-08-08

VIDEO: Botswana's Okavango Delta and the Makgadikgadi Salt Pans are two ends of a 360-mile round trip zebra migration, the second longest on Earth. In this animation, shades of red show... Click here for more information. One of the world's longest migrations of zebras occurs in the African nation of Botswana, but predicting when and where zebras will move has not been possible until now. Using NASA rain and vegetation data, researchers can track when and where arid lands ...

Family members of children with cancer may also be at risk

2013-08-08

SALT LAKE CITY—When a child is diagnosed with cancer, one of the first questions the parents ask is "Will my other children get cancer?" A new study from Huntsman Cancer Institute (HCI) at the University of Utah suggests the answer to that question depends on whether a family history of cancer exists. The research results were published online in the International Journal of Cancer and will appear in the November 15 print issue. The study, led by Joshua Schiffman, M.D., medical director of HCI's High Risk Pediatric Cancer Clinic and a pediatric hematologist/oncologist ...

Cell maturity pathway is deleted or weak in glioblastoma multiforme

2013-08-08

HOUSTON -- A program that pushes immature cells to grow up and fulfill their destiny as useful, dedicated cells is short-circuited in the most common and deadly form of brain tumor, scientists at The University of Texas MD Anderson Cancer Center report this week in the Early Edition of the Proceedings of the National Academy of Sciences (PNAS). Stuck in what amounts to cellular adolescence, these precursor cells accumulate, contributing to the variability among glioblastoma multiforme (GBM) cells that make it so difficult to treat, said first author Jian Hu, Ph.D., instructor ...

An extra hour of TV beyond recommendations diminishes toddlers' kindergarten chances

2013-08-08

Every hourly increase in daily television watching at 29 months of age is associated with diminished vocabulary and math skills, classroom engagement (which is largely determined by attention skills), victimization by classmates, and physical prowess at kindergarten, according to Professor Linda Pagani of the University of Montreal and the CHU Sainte-Justine children's hospital. "This is the first time ever that a stringently controlled associational birth cohort study has looked at and found a relationship between too much toddler screen time and kindergarten risks for ...