Head-to-head against AI, pharmacy students won

A study showed that when compared with students, ChatGPT 3.5 was less likely to correctly answer questions on therapeutics exams focused on clinical applications and cases.

2025-08-11

(Press-News.org) TUCSON, Ariz. — Students pursuing a Doctor of Pharmacy degree routinely take – and pass – rigorous exams to prove competency in several areas. Can ChatGPT accurately answer the same questions? A new study by University of Arizona R. Ken Coit College of Pharmacy researchers said no, it can’t.

Researchers found that ChatGPT 3.5, a form of artificial intelligence, fared worse than PharmD students in answering questions on therapeutics examinations that ensure students have the knowledge, skills, and critical thinking abilities to provide safe, effective and patient-centered care.

ChatGPT was less likely to correctly answer application-based questions (44%) compared with questions focused on recall of facts (80%). It also was less likely to answer case-based questions correctly (45%) compared with questions that weren’t focused on patient cases (74%). Overall, ChatGPT answered only 51% of the questions correctly.

The results provide additional insights into the uses and limitations of the technology and may also prove valuable in the development of pharmacy exam questions. The study findings appear in Currents in Pharmacy Teaching and Learning.

“AI has many potential uses in health care and education, and it’s not going away,” said Christopher Edwards, PharmD, an associate clinical professor of pharmacy practice and science. “One of the things we were hoping to answer with the study was if students wanted to use AI on an exam, how would they perform? I wanted to have data to show the students and tell them they can do well in the exams by studying hard and they don’t necessarily need these tools.”

A secondary goal was to find out what kinds of questions AI would struggle with. Coit College of Pharmacy Interim Dean Brian Erstad, PharmD, wasn’t surprised that ChatGPT did better with straightforward multiple choice and true-false questions and was less successful with application-based questions.

“The kinds of places where evidence is limited and judgment is required, which is often in a clinical setting, was where we found the technology somewhat lacking,” he said. “Ironically those are the kinds of questions clinicians are always facing.”

Edwards, Erstad, and Bernadette Cornelison, PharmD, an associate professor of pharmacy practice and science, evaluated answers to 210 questions from six exams in two pharmacotherapeutics courses that are part of the university’s Coit College of Pharmacy PharmD program.

The questions came from a first-year PharmD course focused on disorders related to nonprescription medications for heartburn, diarrhea, atopic dermatitis, cold and allergies. The other class was a second-year course that covered cardiology, neurology and critical care topics.

To compare the exam performances of pharmacy students and ChatGPT, they calculated mean composite scores as a measure of the ability to correctly answer questions. For ChatGPT, they added individual scores for each exam and divided by the number of exams. To figure out the mean composite score for the students, they divided the sum of the mean class performance on each exam by the number of exams. The mean composite score for six exams for ChatGPT was 53 compared to 82 for pharmacy students.

Educators, clinicians and others continue to debate the value of AI large language models, such as ChatGPT, in academic medicine. While such models will continue to play a range of roles in health care, pharmacy practice and other areas, many are concerned that relying too much on the technology could hamper the development of needed reasoning and critical thinking skills in students.

Both Erstad and Edwards acknowledged that in time, newer and more advanced technology may change these results.

END

ELSE PRESS RELEASES FROM THIS DATE:

Only some emotions help posts go viral

2025-08-11

The positive emotions of love and joy might seem to have much in common. Similarly, anxiety and anger appear to be close emotional siblings. But on social media, seemingly similar emotions can lead to very different responses. According to new research by Yifan Yu, assistant professor of information, risk, and operations management at Texas McCombs, certain emotions pack a bigger punch than others when it comes to how posts spread on social media. “Some emotions, like anxiety and love, help content go viral, while others, like anger or sadness, tend to slow ...

Predicting risk in children with heart defects

2025-08-11

A multi-center study has identified critical risk factors that increase the likelihood of death in children with a heart defect who are awaiting or have recently undergone heart transplantation, according to findings published in Circulation. Fontan circulatory failure (FCF) is a long-term complication in children born with single-ventricle heart defects who have undergone a series of surgeries that culminates with the Fontan procedure. While this surgery helps reroute blood flow and extend life expectancy, it can lead to chronic health problems, including ...

Test performance improves when children can exercise briefly beforehand, UNCG researchers find

2025-08-11

A new study from UNC Greensboro (UNCG) researchers suggests giving children just nine minutes to engage in high-intensity interval exercise can boost their academic performance. “In the classroom, you have teachers that say, ‘Let’s take a movement break to get you focused again,’” said lead author and UNCG Assistant Professor Eric Drollette, Ph.D.. “We know that’s the case anecdotally in the classroom, but we hadn’t put the science to it.” Investigating the science behind this classroom wisdom, the researchers created a short sequence of exercises that can be completed in ...

Meet IDEA: An AI assistant to help geoscientists explore Earth and beyond

2025-08-11

A new artificial intelligence tool developed by researchers at the University of Hawai‘i (UH) at Mānoa is making it easier for scientists to explore complex geoscience data—from tracking sea levels on Earth to analyzing atmospheric conditions on Mars. Called the Intelligent Data Exploring Assistant (IDEA), the software framework combines the power of large language models, like those used in ChatGPT, with scientific data, tailored instructions, and computing resources. By simply providing questions in everyday ...

Ready for market: New process boosts clean, cost-efficient chemical production

2025-08-11

Building on their success developing a cleaner way to make valuable organic acids, researchers from the Center for Advanced Bioenergy and Bioproducts Innovation (CABBI) have pushed one product closer to commercialization with a major upgrade in yield. A CABBI team from the University of Illinois Urbana-Champaign and Princeton University re-engineered the metabolism of the yeast Issatchenkia orientalis to supercharge its fermentation of plant glucose into succinic acid – an important industrial chemical used in food additives and a diverse array of agricultural and pharmaceutical products. Since ...

Losing weight before IVF may increase chance of pregnancy

2025-08-11

Embargoed for release until 5:00 p.m. ET on Monday 11 August 2025 Follow @Annalsofim on X, Facebook, Instagram, Bluesky, and Linkedin Below please find summaries of new articles that will be published in the next issue of Annals of Internal Medicine. The summaries are not intended to substitute for the full articles as a source of information. This information is under strict embargo and by taking it into possession, media representatives are committing to the terms of the embargo ...

New study uncovers how genetics and lifestyle drive the heart disease dilated cardiomyopathy

2025-08-11

An international team, led by scientists from the Victor Chang Cardiac Research Institute has studied around 3000 people affected by the heart disease dilated cardiomyopathy (DCM) – a driver of heart failure and sudden cardiac arrest. They discovered those who had a mutation in a particular gene called TTN were 21 times more likely to develop the disease than family members who did not carry a mutation. For the first time, the team found that a person's general health and lifestyle factors; such as being overweight or having high alcohol consumption, contributed to an earlier DCM diagnosis. The study published in the European Heart Journal involving 1000 families affected ...

City of Hope study shows childhood cancer survivors face new health problems later in life

2025-08-11

LOS ANGELES — Researchers at City of Hope®, one of the largest and most advanced cancer research and treatment organizations in the United States with its National Medical Center ranked among the nation’s top cancer centers by U.S. News & World Report, today published a new study which found that some survivors of childhood cancer are more at risk for serious health issues as they grow older, including new cancers and chronic conditions like heart disease. While a cause for concern, the findings published in the Journal of Clinical Oncology also point to a silver ...

An innovative system that dehydrates fruit without heat

2025-08-11

Dried fruit is a tasty snack or sweet addition to recipes, but the water removal process often requires heat and energy. In a step toward more sustainable food preservation, researchers reporting in ACS Food Science & Technology have developed a method to dry food at room temperature by adjusting air pressure conditions and using food-safe calcium chloride. In a proof-of-concept, the system successfully dried mango and apple slices to commercial levels. Dehydrating food turns perishable items such as fruit into long-lasting pantry ...

The Optica Foundation names Cara Green Executive Director of Development

2025-08-11

WASHINGTON, D.C. – The Optica Foundation today announces its new Executive Director of Development, Cara Green. Building on more than a decade of fundraising experience, Green will lead fundraising efforts for the Optica Foundation as it launches students and early-career professionals into successful careers. There has never been a greater need – or a greater opportunity – to assist the next generation entering the field of optics and photonics. The Optica Foundation is dedicated to doing so, and to recognizing and fostering excellence in students and early-career professionals who ...

LAST 30 PRESS RELEASES:

An ECOG-ACRIN imaging study solves a long-standing gap in metastatic breast cancer research and care: accurately measuring treatment response in patients with bone metastases

Cleveland Clinic presents final results of phase 1 clinical trial of preventive breast cancer vaccine study

Nationally renowned anesthesiology physician-scientist and clinical operations leader David Mintz, MD, PhD, named Chair of the Department of Anesthesiology at the UM School of Medicine