(Press-News.org) OAK BROOK, Ill. – Incorrect advice by an AI-based decision support system could seriously impair the performance of radiologists at every level of expertise when reading mammograms, according to a new study published in Radiology, a journal of the Radiological Society of North America (RSNA).
Often touted as a “second set of eyes” for radiologists, AI-based mammographic support systems are one of the most promising applications for AI in radiology. As the technology expands, there are concerns that it may make radiologists susceptible to automation bias—the tendency of humans to favor suggestions from automated decision-making systems. Several studies have shown that the introduction of computer-aided detection into the mammography workflow could impair radiologist performance. However, no studies have looked at the influence of AI-based systems on the performance of accurate mammogram readings by radiologists.
Researchers from institutions in Germany and the Netherlands set out to determine how automation bias can affect radiologists at varying levels of experience when reading mammograms aided by an AI system.
In the prospective experiment, 27 radiologists read 50 mammograms. They then provided their Breast Imaging Reporting and Data System (BI-RADS) assessment assisted by an AI system. BI-RADS is a standard system used by radiologists to describe and categorize breast imaging findings. While BI-RADS categorization is not a diagnosis, it is crucial in helping doctors determine the next steps in care.
Researchers presented the mammograms in two randomized sets. The first was a training set of 10 in which the AI suggested the correct BI-RADS category. The second set contained incorrect BI-RADS categories, purportedly suggested by AI, in 12 of the 40 mammograms.
The results showed that the radiologists were significantly worse at assigning the correct BI-RADS scores for the cases in which the purported AI suggested an incorrect BI-RADS category. For example, inexperienced radiologists assigned the correct BI-RADS score in almost 80% of cases in which the AI suggested the correct BI-RADS category. When the purported AI suggested the wrong category, their accuracy fell to less than 20%. Experienced radiologists—those with more than 15 years of experience on average—saw their accuracy fall from 82% to 45.5% when the purported AI suggested the incorrect category.
“We anticipated that inaccurate AI predictions would influence the decisions made by radiologists in our study, particularly those with less experience,” said study lead author Thomas Dratsch, M.D., Ph.D., from the Institute of Diagnostic and Interventional Radiology, at University Hospital Cologne in Cologne, Germany. “Nonetheless, it was surprising to find that even highly experienced radiologists were adversely impacted by the AI system’s judgments, albeit to a lesser extent than their less seasoned counterparts.”
The researchers said the results show why the effects of human-machine interaction must be carefully considered to ensure safe deployment and accurate diagnostic performance when combining human readers and AI.
“Given the repetitive and highly standardized nature of mammography screening, automation bias may become a concern when an AI system is integrated into the workflow,” Dr. Dratsch said. “Our findings emphasize the need for implementing appropriate safeguards when incorporating AI into the radiological process to mitigate the negative consequences of automation bias.”
Possible safeguards include presenting users with the confidence levels of the decision support system. In the case of an AI-based system, this could be done by showing the probability of each output. Another strategy involves teaching users about the reasoning process of the system. Ensuring that the users of a decision support system feel accountable for their own decisions can also help decrease automation bias, Dr. Dratsch said.
The researchers plan to use tools like eye-tracking technology to better understand the decision-making process of radiologists using AI.
“Moreover, we would like to explore the most effective methods of presenting AI output to radiologists in a way that encourages critical engagement while avoiding the pitfalls of automation bias,” Dr. Dratsch said.
###
“Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance.” Collaborating with Dr. Dratsch were Xue Chen, M.D., Mohammad Rezazade Mehrizi, Ph.D., Roman Kloeckner, M.D., Aline Mähringer-Kunz, M.D., Michael Püsken, M.D., Bettina Baeßler, M.D., Stephanie Sauer, M.D., David Maintz, M.D., and Daniel Pinto dos Santos, M.D.
In 2023, Radiology is celebrating its 100th anniversary with 12 centennial issues, highlighting Radiology’s legacy of publishing exceptional and practical science to improve patient care.
Radiology is edited by Linda Moy, M.D., New York University, New York, N.Y., and owned and published by the Radiological Society of North America, Inc. (https://pubs.rsna.org/journal/radiology)
RSNA is an association of radiologists, radiation oncologists, medical physicists and related scientists promoting excellence in patient care and health care delivery through education, research, and technologic innovation. The Society is based in Oak Brook, Illinois. (RSNA.org)
For patient-friendly information on mammography, visit RadiologyInfo.org.
END
An AI model has been developed to automatically translate Akkadian text written in cuneiform into English. Hundreds of thousands of clay tablets from ancient Mesopotamia, written in cuneiform and dating back as far as 3,400 BCE, have been found by archeologists, far more than could easily be translated by the limited number of experts who can read them. Shai Gordin and colleagues present a new machine learning model that can automatically translate Akkadian cuneiform into English. Two versions of the model were trained. One version translates the Akkadian from representations of the cuneiform signs in Latin ...
A popular explanation for climate denialism is that humans will adopt beliefs that accord with their political orientation, using their cognitive abilities to explain away identity-inconsistent information in a process called “motivated reasoning.” To test this hypothesis, Bence Bago and colleagues challenged volunteers’ ability to think rationally using time pressure and cognitive loads of varying intensity. The team recruited American participants from Lucid, a website that connects academics with online survey participant pools. The authors found that people who had the ability to deliberate free of cognitive load or time ...
SAN ANTONIO – May 2, 2023 - Southwest Research Institute (SwRI) and The University of Texas at San Antonio (UTSA) have developed a method to synthesize the highly potent antimalarial drug artemisinin, which could lead to a more cost-effective treatment for malaria. The work, recently featured on the cover of the scientific journal Organic Letters, was supported by the Bill and Melinda Gates Foundation as well as a grant from the Connecting through Research Partnerships (Connect) program, a joint effort by SwRI and UTSA to enhance scientific collaboration between the two institutions.
In 2021, 247 million cases of malaria led to 619,000 deaths worldwide. The most effective ...
The American Society for Biochemistry and Molecular Biology sent recommendations to the National Institutes of Health on April 14 to improve working conditions for postdocs.
The society recommended that the NIH:
Ensure a livable wage, benefits, etc. for postdocs
Require that principle investigators help postdocs craft a training plan
Require that principal investigators complete mentor training programs
Expand programs to create more academic staff scientist positions
Collect data on postdoc career outcomes
“Many of our members who run labs are struggling to recruit and retain qualified postdocs,” Sarina Neote, public affairs director ...
A multi-lab replication of a 2006 study supports the role of peer sanction in promoting human cooperation. Cooperation is imperative for society to successfully solve complex problems, including climate change. One approach many groups have adopted is a system of peer sanctions for noncooperators. Such a system incurs costs to participants, who must impose the sanctions, but can allow cooperators to feel more secure that their investments in the shared project will be matched by others. A 2006 study suggested that groups with peer sanctions outgrew and outperformed groups without a peer-sanctioning institution. In light ...
In a world that is slowly distancing itself from carbon-based energy, there has been a meteoric rise in the use of lithium-ion batteries as a next-generation energy storage solution. However, this has resulted in another problem - an increase in the amount of lithium battery waste. Lithium-ion batteries degrade slowly over their lifetime, losing anywhere from 12% to 24% of their total capacity over 500 charging and discharging cycles. The electrolyte and other materials inside the battery can also degrade, causing a decrease in capacity over time. The disposal of lithium batteries ...
WASHINGTON, D.C. – The Patient-Centered Outcomes Research Institute (PCORI) today began inviting proposals for new studies and implementation projects through nine funding opportunities. These PCORI Funding Announcements (PFAs) include four offering up to $452 million for comparative clinical effectiveness research (CER) studies comparing health care approaches to help patients and their caregivers make better-informed health choices given their particular circumstances. Additional PFAs offer funding for a range of projects that support and advance patient-centered CER and use of PCORI-funded research results.
“With ...
New York, NY (May 2, 2023)—Applying machine learning models, a type of artificial intelligence (AI), to data collected passively from wearable devices can identify a patient’s degree of resilience and well-being, according to investigators at the Icahn School of Medicine at Mount Sinai in New York.
The findings, reported in the May 2nd issue of JAMIA Open, support wearable devices, such as the Apple Watch®, as a way to monitor and assess psychological states remotely without requiring the completion of mental health questionnaires.
The ...
Three high school students – Andrea Olsen from Oslo, Norway; Zachary Harpaz from Boca Raton, Florida; and Chris Ren from Shanghai, China – co-authored a paper using a generative artificial intelligence (AI) engine for target discovery from Insilico Medicine (“Insilico”) called PandaOmics to identify new therapeutic targets for glioblastoma multiforme (GBM). GBM is the most aggressive and common malignant brain tumor, accounting for 16% of all primary brain tumors. The findings were published on April 26 in the journal Aging.
Olsen, a student at Sevenoaks School in Kent, UK, began interning at Insilico Medicine in 2021, after discovering ...
People often assume that the products they use every day are safe. Now a new study by Silent Spring Institute and University of California, Berkeley exposes how much people come into contact with toxic ingredients in products, used at home and at work, that could harm their health. Findings from the analysis could help state and federal agencies strengthen chemical regulations and guide manufacturers in making safer products.
Many common products like shampoos, body lotions, cleaners, mothballs, and paint removers contain toxic volatile organic compounds or VOCs—chemicals that escape as gases, accumulate in indoor air, and cause a variety of health ...
