(Press-News.org) CHICAGO – In an examination of the characteristics of studies that yield large treatment effects from medical interventions, these studies were more likely to be smaller in size, often with limited evidence, and when additional trials were performed, the effect sizes became typically much smaller, according to a study in the October 24/31 issue of JAMA.
"Most effective interventions in health care confer modest, incremental benefits," according to background information in the article. "Large effects are important to document reliably because in a relative scale they represent potentially the cases in which interventions can have the most impressive effect on health outcomes and because they are more likely to be adopted rapidly and with less evidence. Consequently, it is important to know whether, when observed, very large effects are reliable and in what sort of experimental outcomes they are commonly observed. … Some large treatment effects may represent entirely spurious observations. It is unknown how often studies with seemingly very large effects are repeated."
Tiago V. Pereira, Ph.D., of the Health Technology Assessment Unit, German Hospital Oswaldo Cruz, Sao Paulo, Brazil, and colleagues conducted a study to evaluate the frequency and features of very large treatment effects of medical interventions that are first recorded in a clinical trial. For the study, the researchers used data from the Cochrane Database of Systematic Reviews (CDSR) and assessed the types of treatments and outcomes in trials with very large effects, examined how often large-effect trials were followed up by other trials on the same topic, and how these effects compared against the effects of the respective meta-analyses.
Among 3,545 available reviews, 3,082 contributed usable information on 85,002 forest plots (a graphical display designed to illustrate the relative strength of treatment effects in multiple studies). Overall, 8,239 forest plots (9.7 percent) had a nominally statistically significant very large effect in the first published trial, group A; 5,158 (6.1 percent) had a nominally statistically significant very large effect found only after the first published trial, group B; and 71,605 (84.2 percent) had no trials with significant very large effects, group C. The researchers found that nominally significant very large effects arose mostly from small trials with few events. For the index trials, the median [midpoint] number of events was only 18 in group A and 15 in the group B. The median number of events in the group C index trials was 14.
The authors also observed that 90 percent and 98 percent of the very large effects observed in first and subsequently published trials, respectively, became smaller in meta-analyses that included other trials; the median odds ratio decreased from approximately 12 to 4 for first trials, and from 10 to 2.5 for subsequent trials.
Topics with very large effects were less likely than other topics to address mortality. Across the whole CDSR, there was only 1 intervention with large beneficial effects on mortality and no major concerns about the quality of the evidence (for a trial on extracorporeal oxygenation for severe respiratory failure in newborns).
"… this empirical evaluation suggests that very large effect estimates are encountered commonly in single trials. Conversely, genuine very large effects with extensive support from substantial evidence appear to be rare in medicine and large benefits for mortality are almost entirely nonexistent. As additional evidence accumulates, caution may still be needed, especially if there is repetitive testing of accumulating trials. Patients, clinicians, investigators, regulators, and the industry should consider this in evaluating very large treatment effects when the evidence is still early and weak," the researchers write.
(JAMA. 2012;308[16]:1676-1684. Available pre-embargo to the media at http://media.jamanetwork.com)
Editor's Note: Dr. Pereira was supported in part by grants from Fundacao de Amparo a Pesquisa do Estado de Sao Paulo (Sao Paulo Research Foundation). All authors have completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.
Editorial: Improving the Health of Patients and Populations Requires Humility, Uncertainty, and Collaboration
Andrew D. Oxman, M.D., of the Norwegian Knowledge Centre for the Health Services, Oslo, Norway, comments on the findings of this study in an accompanying editorial.
"Clinicians should be humble about the ability to prevent or treat most health problems, although a large range of effective interventions are available, some with large effects, most have modest (albeit important) effects and the effects of many are uncertain. Acknowledging uncertainty is the first, essential step to reducing important uncertainties through well-designed evaluations. Collaboration is essential to reduce those uncertainties by identifying and agreeing to priorities for evaluation and making sure not to continue to waste scarce resources for research on unimportant questions or poorly designed evaluations. Clinicians need to collaborate on needed evaluations and synthesizing and making the results of evaluations readily available to inform decisions about how best to improve the health of individuals and populations."
(JAMA. 2012;308[16]:1691-1692. Available pre-embargo to the media at http://media.jamanetwork.com)
Editor's Note: The author has completed and submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest and none were reported.
###
To contact corresponding author John P. A. Ioannidis, M.D., D.Sc., call Krista Conger at 650-725-5371 or email kristac@stanford.edu. To contact editorial author Andrew D. Oxman, M.D., email oxman@online.no.
END
CHICAGO –Among patients without acute respiratory distress syndrome, protective mechanical ventilation with use of lower tidal volumes (the volume of air inhaled and exhaled during each breath) was associated with better outcomes including less lung injury, lower mortality, fewer pulmonary infections and a shorter hospital length of stay, according to a meta-analysis of previous studies, reported in the October 24/31 issue of JAMA.
"Mechanical ventilation is a life-saving strategy in patients with acute respiratory failure. However, unequivocal evidence suggests that ...
Lives could be saved by removing age restrictions on rotavirus vaccination
A study published in this week's PLOS Medicine, which suggests that the additional children's lives saved by removing the age restrictions for rotavirus vaccination in low- and middle-income countries would be much greater than any extra deaths from vaccine-associated complications (namely, intussusception-a form of bowel obstruction), has informed policy regarding the age restrictions for this vaccine.
Hundreds of thousands of infants world-wide have been vaccinated against rotavirus (which ...
A new study reported in this week's PLOS Medicine reports findings from a study carried out in four African countries by Carla Makhlouf Obermeyer and colleagues on approaches towards expanding testing and counselling for HIV. Provider-initiated HIV testing has the potential to expand access to treatment and prevention services, but there have been concerns as to whether consent practices, client confidentiality, and the referral to care will be acceptable under provider-initiated testing modes.
The study, conducted in Burkina Faso, Kenya, Malawi and Uganda, found that ...
When regulatory systems for medical products in low-and middle-income countries work, people live but when such systems fail, people die, according to experts from the US Food and Drug Administration (FDA) writing in this week's PLOS Medicine.
Charles Preston, Mary Lou Valdez, and Katherine Bond from the Office of International Programs at the FDA, argue that few global initiatives focus on strengthening the medical product regulatory systems in low-and middle-income countries but that globalization and the scaling up of medicines and vaccines to these countries are highlighting ...
STANFORD, Calif. — Use of colonoscopy for colorectal cancer screening could explain a significant decrease in the cancer's incidence over the past decade, according to a new study from researchers at the Stanford University School of Medicine. Although colonoscopy is now the most common colorectal cancer screening method, there has been conflicting evidence as to its effectiveness compared with sigmoidoscopy, a method that examines only a portion of the colon.
The team scrutinized data collected from more than 2 million patients over the past 20 years, and found that ...
SALT LAKE CITY, Oct. 24, 2012 – Computer simulations provide new mathematical support for the "grandmother hypothesis" – a famous theory that humans evolved longer adult lifespans than apes because grandmothers helped feed their grandchildren.
"Grandmothering was the initial step toward making us who we are," says Kristen Hawkes, a distinguished professor of anthropology at the University of Utah and senior author of the new study published Oct. 24 by the British journal Proceedings of the Royal Society B.
The simulations indicate that with only a little bit of grandmothering ...
BOSTON – Although placebos have played a critical role in medicine and clinical research for more than 70 years, it has been a mystery why these inactive treatments help to alleviate symptoms in some patients – and not others. Now researchers have for the first time identified genetic differences between placebo responders and non-responders, providing an important new clue to what has come to be known as "the placebo effect."
Led by investigators at Beth Israel Deaconess Medical Center (BIDMC) and Harvard Medical School (HMS), the new findings demonstrate that genetic ...
The laws that govern how intricate surface patterns, such as those found in the cauliflower, develop over time have been described, for the first time, by a group of European researchers.
In a study published today, 24 October, in the Institute of Physics and German Physical Society's New Journal of Physics, researchers have provided a mathematical formula to describe the processes that dictate how cauliflower-like patterns – a type of fractal pattern – form and develop.
The term fractal defines a pattern that, when you take a small part of it, looks similar, although ...
It has long been known that breast density, or mammographic density, is a strong risk factor for breast cancer, and that estrogen and progestin hormone therapy increases dense breast tissue. Now, a study published in BioMed Central's open access journal Breast Cancer Research has identified several gene variants in hormone metabolism and growth factor pathways that may be associated with breast density and, hence, breast cancer risk.
Mammographic density relates to the fact that x-rays permeate different types of breast tissue in different ways, leading to white areas ...
Patients who have received radiotherapy for head and neck cancer often suffer from the unpleasant and distressing side-effect of a dry mouth, caused by damage to their salivary glands from the radiation.
Now, a new study has shown that acupuncture can relieve the symptoms of dry mouth (known as xerostomia). The findings from the largest trial yet to investigate this are published in the cancer journal Annals of Oncology [1] today (Wednesday).
Around half a million people worldwide develop head and neck cancer each year and, at present, there are few effective treatments ...
