Potential neurological treatments often advance to clinical trials on shaky evidence, study says

(Press-News.org) STANFORD, Calif. - Clinical trials of drug treatments for neurological diseases such as Alzheimer's and Parkinson's often fail because the animal studies that preceded them were poorly designed or biased in their interpretation, according to a new study from an international team of researchers. More stringent requirements are needed to assess the significance of animal studies before testing the treatments in human patients, the researchers say.

The team - led by John Ioannidis, MD, DSc, a professor of medicine at the Stanford University School of Medicine and an expert in clinical trial design - assessed the results of more than 4,000 animal studies in 160 meta-analyses of potential treatments for neurological disorders from Alzheimer's disease, Parkinson's disease, stroke, spinal-cord injury and a form of multiple sclerosis. (A meta-analysis is a study that compiles and assesses information and conclusions from many independent experiments of a treatment, or intervention, for a particular condition.).

They determined that only eight of the 160 studies of potential treatments yielded the statistically significant, unbiased data necessary to support advancing the treatment to clinical trials. In contrast, 108 of the treatments were deemed at least somewhat effective at the time they were published.

Ioannidis and his collaborators at the University of Edinburgh in Scotland and the University of Ioannina School of Medicine in Greece say that animal studies of potential interventions can be made more efficient and reliable by increasing average sample size, being aware of statistical bias, publishing negative results and making all the results of all experiments on the effectiveness of a particular treatment - regardless of their outcome - freely accessible to scientists.

"Some researchers have postulated that animals may not be good models for human diseases," said Ioannidis. "I don't agree. I think animal studies can be useful and perfectly fine. The problem is more likely to be related to the selective availability of information about the studies conducted on animals." Although the researchers focused here on neurological disorders, they believe it is likely that similar bias exists in animal studies of other types of disorders.

Ioannidis, who directs the Stanford Prevention Research Center, is the senior author of the research, which will be published online in PLoS Biology on July 16. Lecturer Konstantinos Tsilidis, PhD, and postgraduate fellow Orestis Panagiotou, MD, of the University of Ioannina share lead authorship of the study. Panagiotou is currently a researcher at the National Cancer Institute's Division of Cancer Epidemiology and Genetics.

Ioannidis is known for his efforts to strengthen the way that research is planned, carried out and reported. He was called "one of the world's foremost experts on the credibility of medical research" in a profile published in The Atlantic magazine in 2010. He outlined some of the problems he observed in a 2005 essay in PLoS-Medicine titled, "Why most published research findings are false." The essay is one of the most-downloaded articles in the history of the Public Library of Science, according to the journal's media relations office.

For the new study, Ioannidis and his colleagues evaluated results in a database of the thousands of animal studies compiled over the years through the CAMARADES initiative (Collaborative Approach to Meta-Analysis and Review of Animal Data in Experimental Studies), led by professor Malcolm MacLeod, PhD, from the University of Edinburgh, who is also a co-author of the study.

The team compared the number of experiments in the meta-analyses that would have been expected to yield positive results (based on their predicted statistical power) with the actual number of experiments with published positive results. The difference was striking: 919 expected versus the 1,719 that were published, implying that either negative results were not published, or that the results of the experiments were interpreted too optimistically.

"We saw that it was very common for these interventions to have published evidence that they would work," said Ioannidis. "It was extremely common to have results that suggest they would be effective in humans."

Furthermore, nearly half (46 percent) of the 160 meta-analyses showed evidence of small-study effects - a term used to describe the fact that a small study using fewer numbers of animals is more likely to find the intervention more effective than a larger study with many animals.

Ioannidis speculated that a reluctance to publish negative findings (that is, those that conclude that a particular intervention did not work any better than the control treatment) and a perhaps unconscious desire on the part of researchers to find a promising treatment has colored the field of neurological research. Obscuring access to studies that conclude a particular treatment is ineffective, while also publishing positive results that are likely to be statistically flawed, tilts the perception toward the potential effectiveness of an intervention and encourages unwarranted human clinical trials.

"There are no standard rules that guide a decision to move from animal studies into human clinical trials," said Ioannidis, who also holds C.F. Rehnborg Professorship at Stanford. "Sometimes interventions are tested in humans with very little evidence that they may be effective. Of the 160 analyses we studied, only eight had what we would call strong evidence of potential effectiveness with no hint of bias in the preliminary animal studies. And of these eight, only two have given positive results in humans."

Ioannidis believes the development of consortiums of groups of researchers studying a particular intervention, coupled with the free sharing of all data about its effectiveness, or lack thereof, is a good first step in reducing bias in animal studies.

"Under the current conditions, only a tiny proportion of interventions that have published some promising results in animals have shown to be at all effective in humans. For example, while dozens of treatments on ischemic or hemorrhagic stroke seem to work in the animal literature, almost none of them have worked in humans," said Ioannidis. "It is hard to believe we could not improve upon that translation record. If we raise the bar for moving into human trials, centralize researchers' efforts and make all results available, it will be much easier for researchers to know whether they have a potential winner, and it would increase the efficiency of human clinical trials enormously."

### The study was supported by the MRC Trials Methodology Hub at the University of Edinburgh.

The Stanford University School of Medicine consistently ranks among the nation's top medical schools, integrating research, medical education, patient care and community service. For more news about the school, please visit http://mednews.stanford.edu. The medical school is part of Stanford Medicine, which includes Stanford Hospital & Clinics and Lucile Packard Children's Hospital. For information about all three, please visit http://stanfordmedicine.org/about/news.html.

Print media contacts: Krista Conger at (650) 725-5371 (kristac@stanford.edu), John Sanford at (650) 723-8309 (jsanford@stanford.edu) Broadcast media contact: M.A. Malone at (650) 723-6912 (mamalone@stanford.edu)