(Press-News.org) UCSF-led study finds artificial intelligence is as good as a physician at prioritizing which patients need to be seen first.
Emergency departments nationwide are overcrowded and overtaxed, but a new study suggests artificial intelligence (AI) could one day help prioritize which patients need treatment most urgently.
Using anonymized records of 251,000 adult emergency department (ED) visits, researchers at UC San Francisco evaluated how well an AI model was able to extract symptoms from patients’ clinical notes to determine their need to be treated immediately. They then compared the AI analysis with the patients’ scores on the Emergency Severity Index, a 1-5 scale that ED nurses use when patients arrive to allocate care and resources by highest need, a process known as triage.
The patients’ data were separated from their actual identities (de-identified) for the study, which publishes May 7, 2024, in JAMA Network Open. The researchers evaluated the data using the ChatGPT-4 large language model (LLM), accessing it via UCSF’s secure generative AI platform, which has broad privacy protections.
The researchers tested the LLM’s performance with a sample of 10,000 matched pairs – 20,000 patients in total – that included one patient with a serious condition, such as stroke, and another with a less urgent condition, such as a broken wrist. Given only the patients’ symptoms, the AI was able to identify which ED patient in the pair had a more serious condition 89% of the time.
In a sub-sample of 500 pairs that were evaluated by a physician as well as the LLM, the AI was correct 88% of the time, compared to 86% for the physician.
Having AI assist in the triage process could free up critical physician time to treat patients with the most serious conditions, while offering backup decision-making tools for clinicians who are juggling multiple urgent requests.
“Imagine two patients who need to be transported to the hospital but there is only one ambulance. Or a physician is on call and there are three people paging her at the same time, and she has to determine who to respond to first,” said lead author Christopher Williams, MB, BChir, a UCSF postdoctoral scholar at the Bakar Computational Health Sciences Institute.
Not quite ready for prime time
The study is one of only a few to evaluate an LLM using real-world clinical data, rather than simulated scenarios, and is the first to use more than 1,000 clinical cases for this purpose. It’s also the first study to use data from visits to the emergency department, where there is a wide array of possible medical conditions.
Despite its success within this study, Williams cautioned that AI is not ready to use responsibly in the ED without further validation and clinical trials.
“It’s great to show that AI can do cool stuff, but it’s most important to consider who is being helped and who is being hindered by this technology,” said Williams. “Is just being able to do something the bar for using AI, or is it being able to do something well, for all types of patients?”
One important issue to untangle is how to eliminate bias from the model. Previous research has shown these models may perpetuate racial and gender biases in health care, due to biases within the data used to train them. Williams said that before these models can be used, they will need to be modified to strip out that bias.
“First we need to know if it works and understand how it works, and then be careful and deliberate in how it is applied,” Williams said. “Upcoming work will address how best to deploy this technology in a clinical setting.”
Funding and additional authors: Co-authors include Travis Zack, MD; Brenda Y. Miao, BA; Madhumita Sushil, PhD; Michelle Wang, PharmD, PhD; Aaron E. Kornblith, MD; and Atul J. Butte, MD, PhD, all from UCSF. The authors' department affiliations, funding and consulting disclosures can be found in the paper.
About UCSF: The University of California, San Francisco (UCSF) is exclusively focused on the health sciences and is dedicated to promoting health worldwide through advanced biomedical research, graduate-level education in the life sciences and health professions, and excellence in patient care. It includes UCSF Health, which comprises three top-ranked hospitals, as well as affiliations throughout the Bay Area. Learn more at https://ucsf.edu, or see our Fact Sheet.
###
Follow UCSF
ucsf.edu | Facebook.com/ucsf | YouTube.com/ucsf
END
Emergency department packed to the gills? Someday, AI may help
2024-05-07
ELSE PRESS RELEASES FROM THIS DATE:
Asthma education is key to reducing deaths worldwide, say respiratory health associations
2024-05-07
NEW YORK, NY - May 7, 2024 – On World Asthma Day 2024 the message is clear: "Asthma Education Empowers." The Forum of International Respiratory Societies (FIRS), of which the American Thoracic Society is a founding member, stresses the crucial role of education in empowering people with asthma to manage their condition effectively and to know when to seek medical assistance.
FIRS also urges health care professionals to enhance their awareness of the preventable morbidity and mortality from asthma and of the published evidence on effective asthma management, so they are equipped to provide reliable information and optimal treatment for their patients.
Asthma ...
60% of women with disabilities view cannabis as a ‘harmless’ drug
2024-05-07
A growing number of states and territories in the United States have legalized medical and recreational cannabis use. As such, recreational cannabis has been associated with a lower perception of risk of harm in the general U.S. population.
However, in women of childbearing age, evidence has shown that cannabis use may increase the risk of adverse reproductive and perinatal health outcomes. Furthermore, research on the perception of risk from using cannabis among vulnerable populations such as those with disabilities is lacking.
Using data from the 2021 National Survey on Drug Use and Health, researchers from Florida ...
Years after his death, late scientist's work could yield new cancer treatments
2024-05-07
Some of the final work of a late University of Virginia School of Medicine scientist has opened the door for life-saving new treatments for solid cancer tumors, including breast cancer, lung cancer and melanoma.
Prior to his sudden death in 2016, John Herr, PhD, had been collaborating with UVA Cancer Center’s Craig L. Slingluff Jr., MD, to investigate the possibility that a discovery from Herr’s lab could help treat cancer.
Eight years of research has borne that idea out: Herr’s research into the SAS1B protein could lead to “broad and profound” new treatments ...
SwRI evaluates reliability of pressure relief valves for liquid natural gas tanks in train derailment scenarios
2024-05-07
SAN ANTONIO — May 7, 2024 —Southwest Research Institute (SwRI) has helped determine the viability of pressure relief valves for liquid natural gas tanks in the event of a train derailment for the Federal Rail Administration (FRA). The report from the FRA shows that a study conducted by SwRI demonstrates that the pressure relief valves work as designed to prevent overpressurization and explosion if a derailment occurs.
“The pressure relief valves on tanks that transport liquid ...
Study: You’re breathing potential carcinogens inside your car
2024-05-07
The air inside all personal vehicles is polluted with harmful flame retardants—including those known or suspected to cause cancer—according to a new peer-reviewed study published in Environmental Science & Technology. Car manufacturers add these chemicals to seat foam and other materials to meet an outdated federal flammability standard with no proven fire-safety benefit.
“Our research found that interior materials release harmful chemicals into the cabin air of our cars,” said lead author Rebecca ...
Using AI to predict GPA from college application essays
2024-05-07
Jonah Berger and Olivier Toubia used natural language processing to understand what drives academic success. The authors analyzed over 20,000 college application essays from a large public university that attracts students from a range of racial, cultural, and economic backgrounds and found that the semantic volume of the writing, or how much ground an application essay covered predicted college performance, as measured by grade point average. Essays that covered more semantic ground predicted higher grades. Similarly, essays with smaller conceptual jumps between successive parts of ...
Few tenure-track jobs for engineering PhDs
2024-05-07
A study finds that most engineering PhD graduates will never secure a tenure-track faculty position. Over the past 50 years, the number of full-time faculty positions in US universities has steadily declined while production of science and engineering PhD graduates has nearly doubled. Siddhartha Roy and colleagues analyzed data on PhD graduates and tenure-track and tenured faculty members across all engineering disciplines from 2006–2021. The average annual likelihood of securing a tenure-track faculty position in engineering during this 16-year period was 12.4%. The likelihood of securing a tenure-track faculty position was 18.5% in ...
Hidden citations in physics
2024-05-07
In the scientific literature, a citation acts as a mechanism to signal prior knowledge, enhance credibility, and protect against plagiarism. But it also gives credit to the individual or team who established or discovered the knowledge in question, and citations have thus emerged as a metric to measure the impact of a work or researcher. However, when a discovery or technique becomes common knowledge, scientists often stop bothering to cite it. Thus, the most impactful work is often undercited. Albert-László Barabási ...
Wildfire risk management in the era of climate change
2024-05-07
A Perspective explores lessons learned from recent deadly wildfires and proposes a strategy for managing wildfire risk. Wildfire risk and wildfire deaths are on the rise due to climate change, policies of fire suppression, and development in the wildland-urban interface. The August 8, 2023, fire that destroyed the historic town of Lahaina, Hawaii, claimed 98 lives, in part due to a failure to alert residents to the danger. In 2018, 104 lives were lost in a fire in Mati, Greece, for which there were also no alerts. For both ...
A smart neckband for tracking dietary intake
2024-05-07
A smart neckband allows wearers to monitor their dietary intake. Automatically monitoring food and fluid intake can be useful when managing conditions including diabetes and obesity, or when maximizing fitness. But wearable technologies must be able to distinguish eating and drinking from similar movements, such as speaking and walking. Chi Hwan Lee and colleagues propose a machine-learning enabled neckband that can differentiate body movements, speech, and fluid and food intake. The neckband’s sensor module includes ...