(Press-News.org) OAK BROOK, Ill. – An AI chest X-ray foundation model for disease detection demonstrated racial and sex-related bias leading to uneven performance across patient subgroups and may be unsafe for clinical applications, according to a study published today in Radiology: Artificial Intelligence, a journal of the Radiological Society of North America (RSNA). The study aims to highlight the potential risks for using foundation models in the development of medical imaging artificial intelligence.
“There’s been a lot of work developing AI models to help doctors detect disease in medical scans,” said lead researcher Ben Glocker, Ph.D., professor of machine learning for imaging at Imperial College London in the U.K. “However, it can be quite difficult to get enough training data for a specific disease that is representative of all patient groups.”
Due to the difficulty of collecting large volumes of high-quality training data, the AI field has moved toward using deep-learning foundation models that have been trained for other purposes. Foundation models are AI neural networks that have been trained on large, often unlabeled datasets which handle jobs from translating text to analyzing medical images.
“Despite their increasing popularity, we know little about potential biases in foundation models that could affect downstream uses,” Dr. Glocker said.
Dr. Glocker’s research team compared the performance of a recently published chest X-ray foundation model and a reference model built by the team in evaluating 127,118 chest X-rays with associated diagnostic labels. The pre-trained foundation model was built with more than 800,000 chest X-rays from India and the U.S.
The researchers completed a comprehensive performance analysis to determine how well the models performed for individual subgroups. The 42,884 patients (mean age, 63; 23,623 male) in the study group included Asian, Black and white patients.
Bias analysis showed significant differences between features related to disease detection across biological sex and race.
“Our bias analysis showed that the foundation model consistently underperformed compared to the reference model,” Dr. Glocker said. “We observed a decline in disease classification performance and specific disparities in protected subgroups.”
Significant differences were found between male and female and Asian and Black patients in the features related to disease detection. Compared with the average model performance across all subgroups, classification performance on the ‘no finding’ label dropped between 6.8% and 7.8% for female patients, and performance in detecting ‘pleural effusion’—a buildup of fluid around the lungs—dropped between 10.7% and 11.6% for Black patients.
“Dataset size alone does not guarantee a better or fairer model,” Dr. Glocker said. “We need to be very careful about data collection to ensure diversity and representativeness.”
He noted that it’s important that foundation models are published and shared.
“To minimize the risk of bias associated with the use of foundation models for clinical decision-making, these models need to be fully accessible and transparent,” he said.
Dr. Glocker is an advocate for comprehensive bias analysis as an integral part of the development and auditing of foundation models.
“AI is often seen as a black box, but that’s not entirely true,” he said. “We can open the box and inspect the features. Model inspection is one way of continuously monitoring and flagging issues that need a second look.”
The work doesn’t start with the AI model, it starts with the data used to build it, Dr. Glocker noted.
“As we collect the next dataset, we need to, from day one, make sure AI is being used in a way that will benefit everyone,” he said.
“Risk of Bias in Chest Radiography Deep Learning Foundation Models.” Collaborating with Dr. Glocker were Charles Jones, M.Eng., Mélanie Roschewitz, M.Sc., and Stefan Winzeck, Ph.D.
Radiology: Artificial Intelligence is edited by Charles E. Kahn Jr., M.D., M.S., Perelman School of Medicine at the University of Pennsylvania, and owned and published by the Radiological Society of North America, Inc. (https://pubs.rsna.org/journal/ai)
RSNA is an association of radiologists, radiation oncologists, medical physicists and related scientists promoting excellence in patient care and health care delivery through education, research and technologic innovation. The Society is based in Oak Brook, Illinois. (RSNA.org)
For patient-friendly information on chest X-rays, visit RadiologyInfo.org.
A joint research team in China wrote a review on in-sensor visual computing, a three-in-one hardware solution that is more efficient, economical and secure than conventional machine vision systems, which collect, store, and interpret visual signals on separate hardware units. This review was published Sept. 26 in Intelligent Computing, a Science Partner Journal.
In-sensor visual computing systems are inspired by how humans and other mammals collect, extract and process visual signals, an intricate biological mechanism showing low latency and low energy cost. By integrating sensing, storage and computation onto the ...
A cellular signal essential to the development of the skeleton increases during aging to weaken bones, finds a new study in mice.
The study, led by researchers from NYU Grossman School of Medicine, found that blocking the signaling pathway, called Notch, in aging skeletal stem cells caused a “massive increase” in bone mass and restored lost bone-healing ability during aging.
The study results revolve around immature stem cells, which have the capacity to mature into more than one cell type. Bone is among the tissues that keep pools of stem cells on hand into adulthood, ready to mature into replacement cells that maintain healthy tissue and repair ...
DETROIT (Sept. 27, 2023) – Henry Ford Health is the first health system in Michigan to offer ExactVu, a cutting-edge technology for diagnosing and evaluating prostate cancer. Enhancing the precision and speed of diagnosis, this new technology ensures patients receive timely and appropriate care, while also providing a more efficient and convenient experience than other diagnostic methods.
"Throughout the years, advancements in prostate cancer diagnostic and treatment modalities have made a tremendous difference for patients,” said Craig Rogers, M.D., Chair of the Department of Urology at Henry Ford Cancer. “This is ...
Because figuring out what is new and what is familiar in what we see is such a critically important ability for prioritizing our attention, neuroscientists have spent decades trying to figure out how our brains are typically so good at it. Along the way they’ve made key observations that seem outright contradictory, but a new study shows that the mystifying measures are really two sides of the same coin, paving the way for a long-sought understanding of “visual recognition memory” (VRM).
VRM is the ability to quickly recognize the familiar things in scenes, which can then be de-prioritized so that we can focus on the new things that might be more important ...
The Buck Institute for Research on Aging and Phenome Health are joining forces in the quest to understand the biology of aging. Phenome Health, a Seattle-based nonprofit research organization led by CEO Lee Hood, MD, PhD, uses a data-driven approach to health and disease that integrates diverse types of biological big data. The new Center for Phenomic Health at the Buck will be co-led by Dr. Hood, who joins the Buck as Chief Innovation Officer and Distinguished Professor, and Eric Verdin, MD, Buck President ...
A groundbreaking development in biomedical engineering has led to the creation of a human stomach micro-physiological system (hsMPS), representing a significant leap forward in understanding and treating various gastrointestinal diseases, including stomach cancer. The research team, led by Professor Tae-Eun Park from the Department of Biomedical Engineering at UNIST and Professor Seong-Ho Kong from Seoul National University Hospital, has successfully developed a biomimetic chip that combines organoid and organ-on-a-chip technologies to simulate the complex defense mechanisms of the human gastric mucosa.
Organoids, which mimic human organs using stem cells, have ...
The Department of Energy’s Oak Ridge National Laboratory announced the establishment of its Center for AI Security Research, or CAISER, to address threats already present as governments and industries around the world adopt artificial intelligence and take advantage of the benefits it promises in data processing, operational efficiencies and decision-making.
In partnership with federal agencies such as the Air Force Research Laboratory’s Information Directorate and the Department of Homeland Security (DHS) Science and Technology Directorate ...
Cambridge, Mass. — A newly discovered nearby supernova whose star ejected up to a full solar mass of material in the year prior to its explosion is challenging the standard theory of stellar evolution. The new observations are giving astronomers insight into what happens in the final year prior to a star’s death and explosion.
SN 2023ixf is a new Type II supernova discovered in May 2023 by amateur astronomer Kōichi Itagaki of Yamagata, Japan shortly after its progenitor, or origin star, ...
DALLAS, September 27, 2023 — The American Heart Association and American Medical Association (AMA) nationally recognized 1,709 health care organizations (HCOs) — 400 more than in 2022 — for their efforts to prioritize control of their patients’ blood pressure (BP), a leading preventable risk factor for heart disease, stroke and premature death.
According to the 2022 American Heart Association Statistical Update, nearly half of U.S. adults — 121.5 million ...
HOUSTON – (Sept. 27, 2023) Small and precise: These are the ideal characteristics for CRISPR systems, the Nobel-prize winning technology used to edit nucleic acids like RNA and DNA.
Rice University scientists have described in detail the three-dimensional structure of one of the smallest known CRISPR-Cas13 systems used to shred or modify RNA and employed their findings to further engineer the tool to improve its precision. According to a study published in Nature Communications, the molecule works differently than other proteins in the same family.
“There are different types of CRISPR systems, and the one ...