(Press-News.org) BOSTON – Physician-investigators at Beth Israel Deaconess Medical Center (BIDMC) compared a chatbot’s probabilistic reasoning to that of human clinicians. The findings, published in JAMA Network Open, suggest that artificial intelligence could serve as useful clinical decision support tools for physicians.
“Humans struggle with probabilistic reasoning, the practice of making decisions based on calculating odds,” said the study’s corresponding author Adam Rodman, MD, an internal medicine physician and investigator in the department of Medicine at BIDMC. “Probabilistic reasoning is one of several components of making a diagnosis, which is an incredibly complex process that uses a variety of different cognitive strategies. We chose to evaluate probabilistic reasoning in isolation because it is a well-known area where humans could use support.”
Basing their study on a previously published national survey of more than 550 practitioners performing probabilistic reasoning on five medical cases, Rodman and colleagues fed the publicly available Large Language Model (LLM), Chat GPT-4, the same series of cases and ran an identical prompt 100 times to generate a range of responses.
The chatbot—just like the practitioners before them—was tasked with estimating the likelihood of a given diagnosis based on patients’ presentation. Then, given test results such as chest radiography for pneumonia, mammography for breast cancer, stress test for coronary artery disease and a urine culture for urinary tract infection, the chatbot program updated its estimates.
When test results were positive, it was something of a draw; the chatbot was more accurate in making diagnoses than the humans in two cases, similarly accurate in two cases and less accurate in one case. But when tests came back negative, the chatbot shone, demonstrating more accuracy in making diagnoses than humans in all five cases.
“Humans sometimes feel the risk is higher than it is after a negative test result, which can lead to overtreatment, more tests and too many medications,” said Rodman.
But Rodman is less interested in how chatbots and humans perform toe-to-toe than in how highly skilled physicians’ performance might change in response to having these new supportive technologies available to them in the clinic, added Rodman. He and colleagues are looking into it.
“LLMs can’t access the outside world – they aren’t calculating probabilities the way that epidemiologists, or even poker players, do. What they're doing has a lot more in common with how humans make spot probabilistic decisions,” he said. “But that’s what is exciting. Even if imperfect, their ease of use and ability to be integrated into clinical workflows could theoretically make humans make better decisions,” he said. “Future research into collective human and artificial intelligence is sorely needed.”
Co-authors included Thomas A. Buckley, University of Massachusetts Amherst; Arun K. Manrai, PhD, Harvard Medical School; Daniel J. Morgan, MD, MS, University of Maryland School of Medicine.
Rodman reported receiving grants from the Gordon and Betty Moore Foundation. Morgan reported receiving grants from the Department of Veterans Affairs, the Agency for Healthcare Research and Quality, the Centers for Disease Control and Prevention, and the National Institutes of Health, and receiving travel reimbursement from the Infectious Diseases Society of America, the Society for Healthcare Epidemiology of America. The American College of Physicians and the World Heart Health Organization outside the submitted work. No other disclosures were reported.
About Beth Israel Deaconess Medical Center
Beth Israel Deaconess Medical Center is a leading academic medical center, where extraordinary care is supported by high-quality education and research. BIDMC is a teaching affiliate of Harvard Medical School, and consistently ranks as a national leader among independent hospitals in National Institutes of Health funding. BIDMC is the official hospital of the Boston Red Sox.
Beth Israel Deaconess Medical Center is a part of Beth Israel Lahey Health, a health care system that brings together academic medical centers and teaching hospitals, community and specialty hospitals, more than 4,800 physicians and 38,000 employees in a shared mission to expand access to great care and advance the science and practice of medicine through groundbreaking research and education.
# # #
END
People who have been subject to abuse are more likely to experience physical and mental health effects than previously thought, according to a new study.
In a global review and meta-analysis of evidence published in Nature Medicine today, researchers have found that there are elevated risks between intimate partner violence or childhood sexual abuse, and some health conditions including major depressive disorder, maternal miscarriage for partners, and alcohol misuse and self-harm among children.
Globally, one in three ever-partnered women have experienced ...
McGill University researchers have made a breakthrough in diagnostic technology, inventing a ‘lab on a chip’ that can be 3D-printed in just 30 minutes. The chip has the potential to make on-the-spot testing widely accessible.
As part of a recent study, the results of which were published in the journal Advanced Materials, the McGill team developed capillaric chips that act as miniature laboratories. Unlike other computer microprocessors, these chips are single-use and require no external power source—a simple paper strip suffices. They function through capillary action – ...
BEER-SHEVA, Israel, December 11, 2023 – Fairy circles, a nearly hexagonal pattern of bare-soil circular gaps in grasslands, initially observed in Namibia and later in other parts of the world, have fascinated and baffled scientists for years. Theories for their appearance range from spatial self-organization induced by scale-dependent water-vegetation feedback to pre-existing patterns of termite nests.
Prof. Ehud Meron of Ben-Gurion University of the Negev has been studying the Namibian fairy circles as a case study for understanding how ecosystems respond to water stress. He believes that all theories ...
WASHINGTON (Dec. 11, 2023)--A new study published in Proceedings of the National Academy of Sciences Nexus provides a better understanding of how the brain responds to injuries. Researchers at the George Washington University discovered that a protein called Snail plays a key role in coordinating the response of brain cells after an injury.
The study shows that after an injury to the central nervous system (CNS) a group of localized cells start to produce Snail, a transcription factor or protein that has been implicated in the repair process.The GW researchers show that changing how much Snail is produced can significantly affect whether the injury starts ...
About The Study: In participants with obesity or overweight, withdrawing tirzepatide led to substantial regain of lost weight, whereas continued treatment maintained and augmented initial weight reduction in this randomized clinical trial that included 670 adults.
Authors: Louis J. Aronne, M.D., of Weill Cornell Medicine in New York, is the corresponding author.
To access the embargoed study: Visit our For The Media website at this link https://media.jamanetwork.com/
(doi:10.1001/jama.2023.24945)
Editor’s Note: Please see the article for additional information, including other authors, author contributions and affiliations, conflict ...
Lymphatic fluid from surgical drains, which is usually tossed in the trash, is a treasure in the hands of University of Pittsburgh and Washington University School of Medicine in St. Louis researchers who found that this liquid could inform more precise treatments for patients with head and neck cancer caused by human papillomavirus (HPV).
The new study, published in Clinical Cancer Research, a journal of the American Association for Cancer Research, shows for the first time that HPV DNA in lymphatic fluid collected ...
INDIANAPOLIS -- LOINC® and Health Data Standards at Regenstrief Institute has received a $4.4 million grant from the Regenstrief Foundation to support expansion of work on a global initiative to categorize and standardize social determinants of health (SDOH) information into electronic health records (EHRs), an ambitious project to integrate social needs data into clinical care to more effectively address health inequities.
The unequal distribution of income, food, safety and access to healthcare are recognized as root causes of health disparities. But, currently, data on these factors are rarely well documented within a patient’s ...
Frontline nurses who learned the Transcendental Meditation® (TM®) technique during the COVID-19 pandemic showed rapid and significant improvements in flourishing, PTSD, anxiety, and burnout over 3 months compared to controls, according to a study published today in the Journal of Nursing Administration.
The objective of this study was to evaluate the impact of Transcendental Meditation on nurses’ multidimensional well-being, conceptualized as the presence of flourishing and the absence of PTSD, anxiety, and burnout.
A total of 104 nurses in three Florida hospitals participated. ...
Intimate partner violence is widespread and can have severe physical and psychological health repercussions, but there is a shortage of research on reliable predictors of abuse before it occurs. New research, published in Social Psychological and Personality Science, identifies several warning signs that preceded and predicted intimate partner violence.
“Although future research is required to fully understand the associations between warning signs and abuse, these red flags could eventually be used ...
It may feel unforgivably rude to reject an invitation – even one to an event you would much prefer not to attend – but people often overestimate the social consequences of saying no, according to research published by the American Psychological Association.
“I was once invited to an event that I absolutely did not want to attend, but I attended anyways because I was nervous that the person who invited me would be upset if I did not – and that appears to be a common experience,” said lead author Julian Givi, PhD, an assistant professor at West Virginia University. “Our research shows, however, ...
