(Press-News.org) MIAMI, FLORIDA (EMBARGOED UNTIL SEPT 3, 2025, AT 6:00 A.M. EDT) – Patients are increasingly turning to AI for medical information and even advice, but how should they approach using AI-powered services? A new study published Sept. 3 in the peer-reviewed journal, Future Science OA, provides insight into this question for the fast-moving field of blood cancer, evaluating the quality of responses by ChatGPT to a set of 10 medical questions.
The study investigated ChatGPT 3.5, a version of the popular chatbot from OpenAI that was freely available when the study was conducted, in July 2024. Four anonymous hematology-oncology physicians evaluated the answers.
ChatGPT 3.5 performed best at general questions but struggled with providing information about newer therapies and approaches, the study showed.
“I would warn patients to have some skepticism, especially about answers dealing with specific types of cancer and treatments, and check with their doctor,” said senior author Justin Taylor, M.D., a physician-scientist at Sylvester Comprehensive Cancer Center, part of the University of Miami Miller School of Medicine.
When Taylor was first training to be a physician, patients were increasingly using Google to search for medical information. Both patients and physicians adapted over time. Physicians learned how to direct patients to credible sources, and patients became more capable of finding accurate information.
He sees a similar process playing out with chatbots powered by large language models (LLMs). LLMs are trained on vast amounts of information. Ask a question, and the model will provide an answer. It’s often not accurate or complete, but the technology is evolving rapidly – and so is its uptake.
Prior to the new study, information on how LLMs performed on hematology-oncology tasks was lacking. Other researchers had evaluated LLMs for their ability to address general medical information or other areas of cancer.
For instance, ChatGPT 3.5 provided correct answers about cervical cancer prevention and survivorship, but was far less accurate about diagnosis and treatment.
Focusing on hematology-oncology provides an opportunity to test performance in a field with rapidly shifting treatment options, often tailored to unique patient profiles.
The team chose to evaluate ChatGPT 3.5 because “we wanted to pick something that was popular, freely available, and that we thought most people would use,” said Taylor.
The researchers posed 10 questions to the bot, similar to patient questions as they progress through treatment. Five were general questions often asked when patients are first diagnosed, such as, “What are the common side effects of chemotherapy and how can they be managed?” The other questions were more specific, such as “What is a BCL-2 inhibitor?” BCL-2 inhibitors are a class of drugs under active investigation.
The physician evaluators graded answers on a scale of 1 to 5, from “strongly disagree” to “strongly agree.” A score of 3 was neutral, meaning it was “neither accurate nor inaccurate; it is ambiguous or incomplete.” ChatGPT earned an average score of 3.38 on general cancer questions and 3.06 on questions about newer therapies. None of the evaluators gave the bot a score of 5 on any of the answers.
“Physician oversight remains essential for vetting AI-generated medical information before patient use,” concluded the researchers.
One limitation was that the study did not test other LLMs or newer versions of ChatGPT. After all, ChatGPT 3.5 was trained on datasets with a 2021 cutoff date, limiting its analysis of new medical developments. But Taylor said the message of caution still holds.
“When new drugs or research findings emerge, oncologists check in with their colleagues, discuss the implications and think about how to adapt them to their patients.” Chatbots can’t provide that kind of nuance and personalized understanding, he said.
However, there’s still a place for the technology, he added. ChatGPT and similar tools may help patients prepare for medical visits and devise questions for their physicians. Then, tools can also help direct patients to primary sources with more accurate, detailed information.
The study dovetails with other AI initiatives at Sylvester and the Miller School of Medicine. AI is already easing the paperwork burden for physicians, who can access tools that help them summarize patient encounters or fill out forms. The Miller School has launched the Office of AI in Medical Education, an elective course on AI for medical students, and a self-paced course on AI ethics for Spanish-language medical professionals in Latin American countries.
One Sylvester team built an AI-powered system for brain tumor diagnosis during optical imaging. Another team leveraged machine learning to develop a risk-prediction model for guiding therapeutic decisions in multiple myeloma patients.
Meanwhile, Taylor has his eye on the future as LLMs and associated technology become more powerful. He and his colleagues may take a fresh look at the accuracy of newer versions of ChatGPT within the next year or two.
Study collaborators include first author Tiffany Nong at Florida State University College of Medicine, and researchers at the University of Vermont and Florida Cancer Specialists.
about Sylvester research on the InventUM blog and follow @SylvesterCancer on X for the latest news on its research and care.
# # #
Authors: A complete list of authors is available in the paper.
Article Title: “ChatGPT’s Role in the Rapidly Evolving Hematologic Cancer Landscape”
DOI: 10.1080/20565623.2025.2546259
Funding and Disclosures: Disclosures and funding information are included in the article.
# # #
END
New research from Michigan State University reveals that photorespiration – long considered a wasteful process – is essential for producing a crucial nutrient for preventing birth defects.
For the first time, scientists have measured how much carbon flows through photorespiration to make folates, a class of compounds that includes vitamin B9 – known for its importance as a prenatal vitamin. According to the study, led by MSU researcher Berkley Walker, about 6 percent of the carbon absorbed by plants is used to make folates. That number plummets by fivefold when photorespiration is suppressed.
These findings, published in Nature Plants, could help scientists engineer ...
A team of scientists at Kyoto University’s Institute for Integrated Cell-Material Sciences (iCeMS) has created a protein-based therapeutic tool that could change the way we treat diseases caused by harmful or unnecessary cells. The new tool, published in Nature Biomedical Engineering, involves a synthetic protein called Crunch, short for Connector for Removal of Unwanted Cell Habitat. Crunch uses the body’s natural waste removal system to clear out specific target cells, offering hope for improved treatments for cancer, autoimmune diseases, and other diseases where harmful cells cause damage.
Every day, billions of our body’s ...
Osaka, Japan – The nostalgic “glow-in-the-dark” stars that twinkle on the ceilings of childhood bedrooms operate on a phenomenon called phosphorescence. Here, a material absorbs energy and later releases it in the form of light. However, recent demand for softer, phosphorescent materials has presented researchers with a unique challenge, as producing organic liquids with efficient phosphorescence at room temperature is considered difficult.
Now, researchers at The University of Osaka have attempted to tackle this problem by producing an organic liquid that phosphoresces in the ambient environment. ...
Climate action is falling behind on the goals as stated in the Paris Agreement. To meet those goals, countries must act according to their ‘fair share’ targets. However, researchers from Utrecht University found a bias in how ambition and fairness assessments were calculated until now: “previous studies assessing countries climate ambition share a feature that rewards high emitters at the expense of the most vulnerable ones.” This finding influences climate change mitigations globally. The research, led by Yann Robiou du Pont, was published on 3 September in Nature Communications.
The ...
The world’s methane emissions continue to rise steadily with no signs of slowing, as global trade contributes some 30% to the total amount of the greenhouse gas swirling around the planet, a new study reveals.
As major trade patterns shift, South-South transactions now dominate global supply chains with developing countries increasingly participating in global supply chains. Asia and the developing Pacific region emerge as the largest contributors to global methane emissions, driven by rapid industrialisation and population growth.
Publishing their research today (3 Sep) in Nature Communications, an international team led by researchers at the Universities of Birmingham ...
A study in Health Economics reveals that comprehensive state bans on flavored e-cigarettes may reduce vaping among young adults, but they can also lead to increased use of traditional cigarettes.
Using information from national datasets and advanced statistical methods, researchers found evidence that young adults 18–24 years of age decrease their use of e-cigarettes by about 2–3 percentage points after state bans, while increasing traditional cigarette use by a similar amount. Because cigarettes are more dangerous to health than e-cigarettes, there appears to be a net negative effect on health for this age group.
For youth ...
Research in Economic Inquiry reveals that Major League Baseball teams appear to manage player injuries differently depending on contract status, with players on temporary contracts missing significantly fewer games due to injury. This suggests that teams may invest less in the long-term health of non-permanent players.
For the study, investigators analyzed 2009–2017 data, with 4,125 observations for 1,184 unique players, so that the average player had available data for approximately 3.5 years.
The findings raise ...
Society is experiencing increased global demand for lithium, a critical resource for rechargeable batteries in electric vehicles, consumer electronics, and energy storage systems. Conventional lithium extraction methods from ore reserves or liquid lithium resources are saddled with low efficiency, high costs, and environmental concerns, but researchers have now developed a promising new method.
As described in Advanced Functional Materials, the method involves a solar-powered evaporator that extracts lithium from saline, or salt, lakes. The method relies ...
Traditional polymers such as plastics are widely utilized for their chemical inertness and durability. However, these very properties make them non-degradable in nature and cause long-term environmental damage due to their persistence. In this light, biodegradable polymers that can be broken down by microbes have gained a lot of attention and scientists have turned towards cyclic ketene acetals (CKAs), a group of organic compounds containing carbon atoms and oxygen arranged in a ring-like structure, to develop biodegradable polymers.
CKAs ...
SDE BOKER, Israel, September 3, 2025 — A groundbreaking study from researchers at Ben-Gurion University of the Negev reveals that desert soils can emit powerful greenhouse gases within minutes of being wetted—even in the absence of microbial life.
Published by Dr. Isaac Yagle and Prof. Ilya Gelfand at BGU's Blaustein Institutes for Desert Research in Scientific Reports, the study challenges long-standing assumptions that soil microbes are solely responsible for post-rain “pulse emissions” of gases like carbon dioxide (CO₂), nitrous oxide (N₂O), and nitric oxide (NO). ...
