PRESS-NEWS.org - Press Release Distribution
PRESS RELEASES DISTRIBUTION

ChatGPT’s potential and limits in summarizing medical research for clinicians

Large language models (LLMs) are neural network–based computer programs that use a detailed statistical understanding of written language to perform many tasks, including text generation, summarization, software development, and prediction.

2024-03-25
(Press-News.org) Large language models (LLMs) are neural network–based computer programs that use a detailed statistical understanding of written language to perform many tasks, including text generation, summarization, software development, and prediction. However, LLMs can produce text that, while may seem correct, is not fact-based. This study investigates whether a popular LLM, ChatGPT-3.5, could produce high-quality, accurate, and bias-free summaries of medical research abstracts and determine the relevance of various journals and their articles to different medical specialties. Ten articles published in 2022 (not yet “seen” by ChatGPT, as ChatGPT was trained on data before 2022) were randomly sampled from each of 14 selected journals. ChatGPT was then prompted to summarize the abstract, “self-reflect” on the quality, accuracy, and bias of its own summaries, and evaluate its performance in classifying articles’ and journals' relevance to various areas of medicine (cardiology, pulmonary medicine, family medicine, internal medicine, public health, primary care, neurology, psychiatry, obstetrics and gynecology, and general surgery).

The quality of summaries, relevant classification of journal and article to medical specialty were also assessed by human physicians. The results include a total of 140 abstract summaries across 14 journals. ChatGPT produced summaries that were 70% shorter than the abstracts. The summaries were rated as high quality, high accuracy, and low bias by both ChatGPT and physician reviewers. Serious inaccuracies occurred in only four of the 140 summaries. Minor inaccuracies were noted in 20 of 140 articles and mostly related to the introduction of ambiguity in meaning or summarization of details that would have provided additional content but not completely changed the meaning. ChatGPT was able to classify journals to relevant medical specialties but was much less able to classify specific articles to relevant medical specialties. The summaries were found to have rare—but important—inaccuracies that preclude them from being considered a definitive source of truth.

What We Know: The availability of medical knowledge is increasing. However, due to the demands of their jobs, clinicians have little time to review academic literature, even within their own specialty. Large language models (eg, ChatGPT) could be helpful and save time, but they are not always accurate as they can include bias from their training models and the human feedback which reinforces their learning, and sometimes include information that is not fact-based. 

What This Study Adds: Clinicians are strongly cautioned against solely relying on ChatGPT-based summaries to understand study methods and study results, especially in high-risk situations. Critical medical decisions should—for obvious reasons—remain based on a full evaluation of the full text of articles in context with available evidence from meta-analyses and professional guidelines. However, this study suggests ChatGPT can be useful as a screening tool to help busy clinicians and scientists more rapidly evaluate whether further review of an article is likely to be worthwhile. 

Quality, Accuracy, and Bias in ChatGPT-Based Summarization of Medical Abstracts 

Daniel J. Parente, MD, PhD, et al

Department of Family Medicine and Community Health, University of Kansas Medical Center, Kansas City, Kansas

PRE-EMBARGO LINK (Link expires at 5 p.m. EDT March 25, 2024)

PERMANENT LINK

END


ELSE PRESS RELEASES FROM THIS DATE:

Pediatric health care disrupted by COVID-19 pandemic, compounded by existing barriers such as systemic racism

2024-03-25
This study explores the extent to which pediatric health care was interrupted during and as a result of the COVID-19 pandemic. This was measured based on three primary outcomes of interest: foregone care, foregone well-child or vaccination-related visits, and complete absence of well-child or vaccination-related visits. Researchers extracted data from a nationwide longitudinal survey known as CovEx (COVID Experiences Survey) that had been administered in two “waves'' to a cohort of parents of children between five and 12 years of age. Wave 1 took place October 8-November 13, 2020, and Wave 2 took place March 24-May 7, 2021, with an 82% retention rate of participants. Data was ...

Fathers’ perceptions of their teen sons’ readiness for sex linked to their likelihood to provide guidance on condom use 

2024-03-25
The purpose of this study was to delve deeper into the link between fathers’ perceptions of their adolescent sons’ preparedness to engage in sexual intercourse, and how likely they were to provide their sons with guidance on using condoms correctly and consistently. Researchers recruited 191 father-son pairs from among Black and Latino residents of the South Bronx in New York City; adolescent participants ranged in age from 15 to 19 years old. Using a sequential mixed-methods explanatory design, all participants completed a confidential survey, after which a random ...

Survey on genital tucking among transgender and gender diverse individuals: Majority of respondents desire open conversation with health care providers

2024-03-25
Genital tucking is the practice of hiding or minimizing the appearance of one’s genitals and gonads. It is practiced by transgender women and gender diverse individuals who were assigned male at birth. In this study, researchers designed a 27-question survey related to such factors as the length of time individuals spent tucked; how commonly and frequently; and understandings of, concerns for, and positive/negative history of potential adverse effects. Only 23% of the 98 respondents reported discussing ...

Annual vaccination campaign with second dose protects high-risk groups from SARS-CoV-2 and may save health care costs

2024-03-25
Embargoed for release until 5:00 p.m. ET on Monday 25 March 2024    Annals of Internal Medicine Tip Sheet     @Annalsofim    Below please find summaries of new articles that will be published in the next issue of Annals of Internal Medicine. The summaries are not intended to substitute for the full articles as a source of information. This information is under strict embargo and by taking it into possession, media representatives are committing to the terms of the embargo not only on their own behalf, but also on behalf of the organization they represent.    ----------------------------    1. Annual vaccination campaign with ...

DEI and antiracism curriculum must connect the classroom to the clinic to educate medical students about racism in medicine

2024-03-25
There's a growing awareness of the role medicine as an institution has played in creating and perpetuating health inequities facing historically marginalized groups, yet systemic racism and implicit biases continue to shape aspects of clinical practice such as care management decisions and patient communication.  Addressing medical racism has become an essential part of the medical education curriculum, though most published curricula treat health equity as a lecture topic separate from clinical practice. In this report, authors from the Emory University School of Medicine present a new approach to threading (cohesively incorporating) diversity, equity, and inclusion (DEI) and ...

A national quality improvement initiative provides insight into how to successfully implement change in primary care practices

2024-03-25
In 2015, the Agency for Healthcare Research and Quality (AHRQ) launched EvidenceNOW: Advancing Heart Health. This three-year initiative, AHRQ’s largest-ever investment in primary care research, was designed to provide external quality improvement support to help small and medium-size primary care practices implement the latest evidence and improve their delivery of cardiovascular care.  AHRQ awarded grants to primary care practices in seven regions across the U.S. to implement this initiative and to evaluate their own experiences with it, thereby accounting for regional differences in needs and existing resources. Additionally, ...

Annals of Family Medicine March/April 2024 Tip Sheet

2024-03-25
Collaboration Between Medical Department Staff and Their IT Colleagues on an EHR Optimization Project Leads to Improved Productivity Between June and October 2021, members of Marshall University’s Department of Family and Community Health collaborated with their school’s Information Technology (IT) team to conduct a four-month, department-wide optimization project to improve their usage of the Electronic Health Record (EHR) software. The software was implemented across their institution ...

Annals of Family Medicine: Study shows Latino and Black fathers' perception of sons' sexual readiness is key to effective condom guidance

2024-03-25
Providence, R.I. -- A new study from Annals of Family Medicine reveals that a father's understanding of his son's readiness for sexual relationships plays a crucial role in the timing and effectiveness of discussions regarding condom use guidance. The findings offer valuable insights for family-focused health care providers, guiding them to encourage fathers to initiate conversations about sexual readiness and the importance of correct and consistent condom use with their adolescent sons. In 2021, 47% of adolescent males in the 12th grade reported engaging in sexual activity, yet condom use has declined. These trends contribute to negative sexual health outcomes among ...

The future of independent primary care practices serving vulnerable populations depends on new policies that support health equity

2024-03-25
In recent years, the U.S. government has invested substantially in Federally Qualified Health Centers (FQHCs), which have become synonymous in policy discussions with primary care for the socially vulnerable. Conversely, no such investment has been made in independent practices serving socially vulnerable patients. As independent practices become less financially viable, this disparity could severely limit primary care options for socially vulnerable patients. This mixed-methods study considers the extent to which independent family physicians ...

Comparative study of type 2 diabetes medications show differences in medication acceptance, quality-of-life, insulin secretion and mortality

2024-03-25
Diabetes affects more than 1 in 10 — or more than 38 million — Americans. People with diabetes who keep their blood sugar levels in the near-normal range generally have a much lower risk of developing diabetes complications such as heart, kidney, and eye diseases. The challenge is that most people with diabetes require more than one medication to control blood sugar levels over time.  The Glycemia Reduction Approaches in Diabetes: A Comparative Effectiveness, or GRADE, Study was designed to compare ...

LAST 30 PRESS RELEASES:

Underserved youth less likely to visit emergency department for concussion in Ontario, study finds

‘Molecular shield’ placed in the nose may soon treat common hay fever trigger

Beetles under climate stress lay larger male eggs: Wolbachia infection drives adaptive reproduction strategy in response to rising temperature and CO₂

Groundbreaking quantum study puts wave-particle duality to work

Weekly injection could be life changing for Parkinson’s patients

Toxic metals linked to impaired growth in infants in Guatemala

Being consistently physically active in adulthood linked to 30–40% lower risk of death

Nerve pain drug gabapentin linked to increased dementia, cognitive impairment risks

Children’s social care involvement common to nearly third of UK mums who died during perinatal period

‘Support, not judgement’: Study explores links between children’s social care involvement and maternal deaths

Ethnic minority and poorer children more likely to die in intensive care

Major progress in fertility preservation after treatment for cancer of the lymphatic system

Fewer complications after additional ultrasound in pregnant women who feel less fetal movement

Environmental impact of common pesticides seriously underestimated

The Milky Way could be teeming with more satellite galaxies than previously thought

New study reveals surprising reproductive secrets of a cricket-hunting parasitoid fly

Media Tip Sheet: Symposia at ESA2025

NSF CAREER Award will power UVA engineer’s research to improve drug purification

Tiny parasitoid flies show how early-life competition shapes adult success

New coating for glass promises energy-saving windows

Green spaces boost children’s cognitive skills and strengthen family well-being

Ancient trees dying faster than expected in Eastern Oregon

Study findings help hone precision of proven CVD risk tool

Most patients with advanced melanoma who received pre-surgical immunotherapy remain alive and disease free four years later

Introducing BioEmu: A generative AI Model that enables high-speed and accurate prediction of protein structural ensembles

Replacing mutated microglia with healthy microglia halts progression of genetic neurological disease in mice and humans

New research shows how tropical plants manage rival insect tenants by giving them separate ‘flats’

Condo-style living helps keep the peace inside these ant plants

Climate change action could dramatically limit rising UK heatwave deaths

Annual heat-related deaths projected to increase significantly due to climate and population change

[Press-News.org] ChatGPT’s potential and limits in summarizing medical research for clinicians
Large language models (LLMs) are neural network–based computer programs that use a detailed statistical understanding of written language to perform many tasks, including text generation, summarization, software development, and prediction.