(Press-News.org) Where we live and work, our age, and the conditions we grew up in can influence our health and lead to disparities, but these factors can be difficult for clinicians and researchers to capture and address. A new study by investigators from Mass General Brigham demonstrates that large language models (LLMs), a type of generative artificial intelligence (AI), can be trained to automatically extract information on social determinants of health (SDoH) from clinicians’ notes which could augment efforts to identify patients who may benefit from resource support. Findings, published in npj Digital Medicine, show that the finely tuned models could identify 93.8 percent of patients with adverse SDoH, whereas official diagnostic codes included this information in only 2 percent of cases. These specialized models were less prone to bias than generalist models such as GPT-4.
“Our goal is to identify patients who could benefit from resource and social work support, and draw attention to the under-documented impact of social factors in health outcomes,” said corresponding author Danielle Bitterman, MD, a faculty member in the Artificial Intelligence in Medicine (AIM) Program at Mass General Brigham and a physician in the Department of Radiation Oncology at Brigham and Women’s Hospital. “Algorithms that can pass major medical exams have received a lot of attention, but this is not what doctors need in the clinic to help take better care of patients each day. Algorithms that can notice things that doctors may miss in the ever-increasing volume of medical records will be more clinically relevant and therefore more powerful for improving health.”
Health disparities are widely recognized to be linked to SDoH, which include employment, housing and other non-medical circumstances that impact medical care. For example, the distance a cancer patient lives from a major medical center or the support they have from a partner can substantially influence outcomes. While clinicians may summarize relevant SDoH in their visit notes, this vital information is rarely systematically organized in the electronic health record (EHR).
The emergence of AI tools in health has the potential to positively reshape the continuum of care. Mass General Brigham, as one of the nation’s top integrated academic health systems and largest innovation enterprises, is leading the way in conducting rigorous research on new and emerging technologies to inform the responsible incorporation of AI into care delivery, workforce support and administrative processes.
To create LMs capable of extracting information on SDoH, the researchers manually reviewed 800 clinician notes from 770 patients with cancer who received radiotherapy at the Department of Radiation Oncology at Brigham and Women’s Hospital. They tagged sentences that referred to one or more of six, pre-determined SDoH: employment status, housing, transportation, parental status (if the patient has a child under 18 years old), relationships and presence or absence of social support.
Using this “annotated” dataset, the researchers trained existing LMs to identify references to SDoH in clinician notes. They tested their models using an additional 400 clinic notes from patients treated with immunotherapy at Dana-Farber Cancer Institute and from patients admitted to the critical care units at Beth Israel Deaconess Medical Center.
The researchers found that fine-tuned LMs, especially Flan-T5 LMs, could consistently identify rare references to SDoH in clinician notes. The “learning capacity” of these models was limited by the rarity of SDoH documentation in the training set, where the researchers found that only 3 percent of sentences in clinician notes contained any mention of SDoH. To address this issue, the researchers used ChatGPT, another LM, to produce an additional 900 synthetic examples of SDoH sentences that could be used as an extra training dataset.
A major criticism of generative AI models in health care is that they can potentially perputate bias and widen health disparities. The researchers found that their fine-tuned LM was less likely than OpenAI’s GPT-4, a generalist LM, to change its determination about an SDoH based on individuals’ race/ethnicity and gender. The researchers state that it is difficult to understand exactly how biases are formed and deconstructed — both in humans and in computer models. Understanding the origins of algorithmic bias is an ongoing endeavor for the researchers.
“If we don’t monitor algorithmic bias when we develop and implement large language models, we could make existing health disparities much worse than they currently are,” Bitterman said. “This study demonstrated that fine-tuning LMs may be a strategy to reduce algorithmic bias, but more research is needed in this area.”
Authorship: Mass General Brigham co-authors include co-first authors Marco Guevara, MS, and Shan Chen, MS, of the AIM Program at Mass General Brigham and the Department of Radiation Oncology at the Brigham, Spencer Thomas, Tafadzwa L. Chaunzwa, Benjamin H. Kann, Jack M. Qian, Hugo JWL Aerts and Raymond H. Mak (Brigham and Women’s Hospital and AIM Program) and Idalid Franco and Shalini Moningi (AIM Program, Brigham and Women’s Hospital). Additional co-authors include Madeleine Golstein, Susan Harper, Paul J. Catalano and Guergana K. Savova..
Disclosures: Bitterman is an Associate Editor of Radiation Oncology, HemOnc.org and receives funding from the American Association for Cancer Research. Chen and Guevara report no disclosures. A complete list of disclosures is included in the paper.
Funding: Bitterman received funding for this work from the Woods Foundation, Jay Harris Junior Faculty Award, Joint Center for Radiation Therapy Foundation and National Institutes of Health (U54CA274516-01A1). A complete list of funding sources is included in the paper.
Paper cited: Guevara, M et al. “Large Language Models to Identify Social Determinants of Health in Electronic Health Records” npj Digital Medicine DOI: 10.1038/s41746-023-00970-0
For More Information:
Artificial Intelligence in Medicine Program
Data Science Office
2024 Predictions about Artificial Intelligence
ChatGPT Shows Limited Ability to Recommend Guidelines-Based Cancer Treatments
Study Finds Limitations to CPR Directions Given by Artificial Intelligence Voice Assistants, Recommends Use of Emergency Services
ChatGPT Shows ‘Impressive’ Accuracy in Clinical Decision Making
###
About Mass General Brigham
Mass General Brigham is an integrated academic health care system, uniting great minds to solve the hardest problems in medicine for our communities and the world. Mass General Brigham connects a full continuum of care across a system of academic medical centers, community and specialty hospitals, a health insurance plan, physician networks, community health centers, home care, and long-term care services. Mass General Brigham is a nonprofit organization committed to patient care, research, teaching, and service to the community. In addition, Mass General Brigham is one of the nation’s leading biomedical research organizations with several Harvard Medical School teaching hospitals. For more information, please visit massgeneralbrigham.org.
END
Researchers at the University of California San Diego have developed a neural implant that provides information about activity deep inside the brain while sitting on its surface. The implant is made up of a thin, transparent and flexible polymer strip that is packed with a dense array of graphene electrodes. The technology, tested in transgenic mice, brings the researchers a step closer to building a minimally invasive brain-computer interface (BCI) that provides high-resolution data about deep neural activity by using recordings from the brain surface.
The work was published on Jan. 11 in Nature Nanotechnology.
“We are expanding the spatial ...
Cities play a key role in climate change and biodiversity and are one of the most recognizable features of the Anthropocene. They also accelerate innovation and shape social networks, while perpetuating and intensifying inequalities. Today over half of all humanity lives in cities, a threshold which will rise to nearly 70% by the mid-21st century. Yet despite their importance for the Anthropocene, cities are not a recent phenomenon.
In a new study, an interdisciplinary team of authors from the Max Planck Institute of Geoanthropology ...
Francis Crick Institute press release
Under strict embargo: 10:00hrs GMT 11 January 2024
Peer reviewed
Observational study
People
First prehistoric person with Turner syndrome identified from the Iron Age
Earliest known person with Jacob’s syndrome identified from Early Medieval Period
Individuals with Klinefelter syndrome identified across a range of time periods
New technique developed to measure number of chromosomes in ancient genomes more precisely
Researchers at the Francis Crick Institute, working with University of Oxford, University of York ...
Researchers have linked a decade-long decline in the blood lead levels of American Indian adults to long-term cardiovascular health benefits, including reduced blood pressure levels and a reduction in a marker associated with hypertrophic cardiomyopathy and heart failure. The research team, supported by the National Institutes of Health, found that adults who had the greatest reductions in blood lead levels saw their systolic blood pressure fall by about 7 mm Hg, an amount comparable to the effects of blood pressure-lowering medication. Lead exposure is known to harm the health of children by damaging the brain and nervous system and slowing growth and development. It has also been associated ...
Through the Strong Heart Family Study, National Institutes of Health-supported researchers found that small declines in blood lead levelswere associated with long-term cardiovascular health improvements in American Indian adults. Participants who had the greatest reductions in blood lead levels saw their systolic blood pressure fall by about 7 mm Hg, an amount comparable to the effects of blood pressure-lowering medication.
The findings as reported from researchers at Columbia ...
A multi-national team of scientists (China, USA, New Zealand, Italy, and France) analyze the temperature of the Earth annually. These scientists have found a fever that increases every year. For the past decade, each year has been hotter than the prior year in the ocean. And there are other changes in the ocean that also matter.
The ocean is an important part of the Earth's climate system – it covers 70% of the planet and absorbs about 90% of the heat from global warming. The ocean helps control the atmosphere – a warmer ocean leads to a warmer and moister atmosphere with wilder weather. The ocean also controls ...
In a new study, environmentally benign inverse-perovskites with high energy conversion efficiency have been reported by Tokyo Tech scientists with potential for practical application as thermoelectric materials (TEMs). Addressing the limitations typically faced with TEMs, such as insufficient energy conversion efficiency and environmental toxicity due to heavy elements, the new TEMs provide a suitable alternative to TEMs based on toxic elements with better thermoelectric properties than conventional eco-friendly TEMs.
Thermoelectric materials (TEMs) capable of converting thermal energy to electrical energy and vice versa have become an essential part of our world, which needs ...
Humans learn which behaviors pay off and which don’t from watching others. Based on this, we may draw conclusions about how to act – or eat. In the case of the latter, people may use each other as guides to determine what and how much to eat. This is called social modelling and is one of the most powerful social influences on eating behavior.
In a new study, researchers in the UK investigated whether observing others’ facial expressions while eating raw broccoli influenced young women’s liking and desire to ...
Osaka, Japan – Approximately 5% of global carbon emissions are attributable to producing the chemicals that are essential to modern life. Creating a sustainable solution to one chemical reaction in particular – the autoxidation of aldehydes – has challenged researchers for decades.
Now, in a study recently published in Green Chemistry, researchers from Osaka University, Shizuoka Institute of Science & Technology, and collaborating partners have solved this problem. Through reaction kinetics and mathematical ...
Controlling the direction of magnetization using low electric field is necessary for developing efficient spintronic devices. In spintronics, properties of an electron’s spin or magnetic moment are used to store information. The electron spins can be manipulated by straining orbital magnetic moments to create a high-performance magnetoelectric effect.
Japanese researchers, including Jun Okabayashi from the University of Tokyo, revealed a strain-induced orbital control mechanism in interfacial multiferroics. In multiferroic material, the magnetic property ...
