(Press-News.org) A study of ChatGPT found the artificial intelligence tool answered less than half of the test questions correctly from a study resource commonly used by physicians when preparing for board certification in ophthalmology.
The study, published in JAMA Ophthalmology and led by St. Michael’s Hospital, a site of Unity Health Toronto, found ChatGPT correctly answered 46 per cent of questions when initially conducted in Jan. 2023. When researchers conducted the same test one month later, ChatGPT scored more than 10 per cent higher.
The potential of AI in medicine and exam preparation has garnered excitement since ChatGPT became publicly available in Nov. 2022. It’s also raising concern for the potential of incorrect information and cheating in academia. ChatGPT is free, available to anyone with an internet connection, and works in a conversational manner.
“ChatGPT may have an increasing role in medical education and clinical practice over time, however it is important to stress the responsible use of such AI systems,” said Dr. Rajeev H. Muni, principal investigator of the study and a researcher at the Li Ka Shing Knowledge Institute at St. Michael’s. “ChatGPT as used in this investigation did not answer sufficient multiple choice questions correctly for it to provide substantial assistance in preparing for board certification at this time.”
Researchers used a dataset of practice multiple choice questions from the free trial of OphthoQuestions, a common resource for board certification exam preparation. To ensure ChatGPT’s responses were not influenced by concurrent conversations, entries or conversations with ChatGPT were cleared prior to inputting each question and a new ChatGPT account was used. Questions that used images and videos were not included because ChatGPT only accepts text input.
Of 125 text-based multiple-choice questions, ChatGPT answered 58 (46 per cent) questions correctly when the study was first conducted in Jan. 2023. Researchers repeated the analysis on ChatGPT in Feb. 2023, and the performance improved to 58 per cent.
“ChatGPT is an artificial intelligence system that has tremendous promise in medical education. Though it provided incorrect answers to board certification questions in ophthalmology about half the time, we anticipate that ChatGPT’s body of knowledge will rapidly evolve,” said Dr. Marko Popovic, a co-author of the study and a resident physician in the Department of Ophthalmology and Vision Sciences at the University of Toronto.
ChatGPT closely matched how trainees answer questions, and selected the same multiple-choice response as the most common answer provided by ophthalmology trainees 44 per cent of the time. ChatGPT selected the multiple-choice response that was least popular among ophthalmology trainees 11 per cent of the time, second least popular 18 per cent of the time, and second most popular 22 per cent of the time.
“ChatGPT performed most accurately on general medicine questions, answering 79 per cent of them correctly. On the other hand, its accuracy was considerably lower on questions for ophthalmology subspecialties. For instance, the chatbot answered 20 per cent of questions correctly on oculoplastics and zero per cent correctly from the subspecialty of retina. The accuracy of ChatGPT will likely improve most in niche subspecialties in the future,” said Andrew Mihalache, lead author of the study and undergraduate student at Western University.
END
ChatGPT scores nearly 50 per cent on board certification practice test for ophthalmology, study shows
AI tool scored more than 10 per cent higher one month later
2023-04-27
ELSE PRESS RELEASES FROM THIS DATE:
Ecology: Over 64% of suitable elephant habitat lost across Asia since 1700
2023-04-27
Habitats suitable for Asian elephants (Elephas maximus) across Asia have decreased by over 64% – equating to 3.3 million square kilometres of land – since the year 1700, estimates a study published in Scientific Reports. The authors suggest that habitat loss from 1700, after centuries of relative stability, coincides with the colonial-era use of land and subsequent agricultural intensification in South Asia.
Asian elephants live in a range of habitats including grasslands and rainforests, but with increasing human use of land and habitat loss, elephants can come ...
Citing growing evidence of harm to child health and learning ability, advocates call for faster replacement of diesel school buses
2023-04-27
Health and environmental advocates today called on communities, school boards and governments at every level to accelerate the electrification of school buses, replacing tens of thousands of diesel-powered school buses spewing toxic fumes that can seriously harm child health and interfere with learning.
Led by the Canadian Partnership for Children’s Health and Environment (CPCHE), a coalition of 34 organisations made the goal of all-electric school bus fleets in Canada the central focus of this year’s national Healthy Environments for Learning Day (April 27).
The joint call for urgent action by ...
Association of SARS-CoV-2 vaccination or infection with Bell palsy
2023-04-27
About The Study: This systematic review and meta-analysis suggests a higher incidence of Bell palsy (BP) among SARS-CoV-2–vaccinated versus placebo groups. The occurrence of BP did not differ significantly between recipients of the Pfizer/BioNTech versus Oxford/AstraZeneca vaccines. SARS-CoV-2 infection posed a significantly greater risk for BP than SARS-CoV-2 vaccination.
Authors: Amir Kheradmand, M.D., of Johns Hopkins University in Baltimore, and Mehran Rahimlou, Ph.D., of the Zanjan University of Medical Sciences in Zanjan, Iran, are the corresponding authors.
To access the embargoed study: Visit our For The Media website at this link https://media.jamanetwork.com/
(doi:10.1001/jamaoto.2023.0160)
Editor’s ...
Performance of an artificial intelligence chatbot in ophthalmic knowledge assessment
2023-04-27
About The Study: In this study that included 125 text-based multiple-choice questions provided by the OphthoQuestions free trial for ophthalmic board certification examination preparation, ChatGPT answered approximately half of the questions correctly. Medical professionals and trainees should appreciate the advances of AI in medicine while acknowledging that ChatGPT as used in this investigation did not answer sufficient multiple-choice questions correctly for it to provide substantial assistance in preparing for board certification at this time.
Authors: Rajeev H. Muni, M.D., M.Sc., of St. Michael’s Hospital/Unity Health Toronto in Toronto, is the corresponding ...
Comparison between ChatGPT and Google search as sources of postoperative patient instructions
2023-04-27
About The Study: The findings of this study suggest that ChatGPT provides postoperative instructions that are helpful for patients with a fifth-grade reading level or different health literacy levels. However, ChatGPT generated instructions scored lower in understandability, actionability, and procedure-specific content than Google Search– and institution-specific instructions.
Authors: Noel Ayoub, M.D., M.B.A., of the Stanford University School of Medicine in Stanford, California, is the corresponding author.
To access the embargoed study: Visit our For The Media website at this link https://media.jamanetwork.com/
(doi:10.1001/jamaoto.2023.0704)
Editor’s ...
Arthropods in high-diversity forests contribute to improved productivity
2023-04-27
An international team of researchers led by Prof. MA Keping from the Institute of Botany of the Chinese Academy of Sciences (IBCAS) has shown that forests with higher tree species richness tend to have greater arthropod diversity.
The researchers showed that higher tree diversity promotes productivity through the suppression of herbivores by enemy arthropods.
These findings, published recently in Nature Ecology & Evolution, underscore the importance of arthropod diversity as a mediator of the effects of tree diversity ...
Molecular autopsy sheds light on cause of sudden death of a child with COVID-19
2023-04-27
Researchers from Tokyo Medical and Dental University (TMDU) utilize an advanced DNA sequencing technique to reveal the factors that contributed to the death of a 5-year-old child with COVID-19
Tokyo, Japan – Sudden, unexplained child mortality is a tragedy; determining the cause of death is important for improving healthcare and providing loved ones with closure. Now, researchers from Japan have reported the use of an advanced DNA sequencing technique, whole-exome sequencing (WES), to determine why a young child died after a relatively mild infection.
In a study published ...
Bioinformatics specialists in Saarbrücken explore the molecular mechanisms of ageing
2023-04-27
A team led by bioinformatics experts Andreas Keller and Fabian Kern from Saarland University together with researchers at Stanford University have gained new insights into manifestations of ageing at the molecular level. They found that the process of reading genetic information does not run as smoothly in older individuals as it does in younger ones. These changes in the transcription process are due to particular RNA molecules that influence the activity of individual genes and thus determine which proteins the body produces – physiological ...
University of Cincinnati research examines the role of genetics in opioid use disorder
2023-04-27
New research out of the University of Cincinnati examines the association between genetics and the presence of opioid use disorder (OUD). The study identified six single nucleotide polymorphisms (SNPs) or genetic variants that are linked to OUD.
The study was published in the journal Clinical Pharmacology & Therapeutics.
“We are trying to identify some of the genetic variants that might play into OUD,” says Caroline Freiermuth, MD, associate professor in the Department of Emergency ...
Light-based computing scheme reduces power needed to mine cryptocurrencies
2023-04-27
WASHINGTON — Researchers have developed a new light-based computing scheme that uses a photonic integrated circuit to reduce the energy necessary for cryptocurrency and blockchain applications. Mining cryptocurrencies like Bitcoin—a process of verifying transactions and adding new cryptocurrency to the blockchain—consumes up to 1% of the world’s energy. This energy expenditure is expected to grow as cryptocurrency and blockchain applications become increasingly mainstream.
Cryptocurrencies are digital currencies created using encryption algorithms. These alternative currencies require ...
LAST 30 PRESS RELEASES:
Public take the lead in discovery of new exploding star
What are they vaping? Study reveals alarming surge in adolescent vaping of THC, CBD, and synthetic cannabinoids
ECMWF - delivering forecasts over 10 times faster and cutting energy usage by 1000
Brazilian neuroscientist reveals how viral infections transform the brain through microscopic detective work
Turning social fragmentation into action through discovering relatedness
Cheese may really be giving you nightmares, scientists find
Study reveals most common medical emergencies in schools
Breathable yet protective: Next-gen medical textiles with micro/nano networks
Frequency-engineered MXene supercapacitors enable efficient pulse charging in TENG–SC hybrid systems
Developed an AI-based classification system for facial pigmented lesions
Achieving 20% efficiency in halogen-free organic solar cells via isomeric additive-mediated sequential processing
New book Terraglossia reclaims language, Country and culture
The most effective diabetes drugs don't reach enough patients yet
Breast cancer risk in younger women may be influenced by hormone therapy
Strategies for staying smoke-free after rehab
Commentary questions the potential benefit of levothyroxine treatment of mild hypothyroidism during pregnancy
Study projects over 14 million preventable deaths by 2030 if USAID defunding continues
New study reveals 33% gap in transplant access for UK’s poorest children
Dysregulated epigenetic memory in early embryos offers new clues to the inheritance of polycystic ovary syndrome (PCOS)
IVF and IUI pregnancy rates remain stable across Europe, despite an increasing uptake of single embryo transfer
It takes a village: Chimpanzee babies do better when their moms have social connections
From lab to market: how renewable polymers could transform medicine
Striking increase in obesity observed among youth between 2011 and 2023
No evidence that medications trigger microscopic colitis in older adults
NYUAD researchers find link between brain growth and mental health disorders
Aging-related inflammation is not universal across human populations, new study finds
University of Oregon to create national children’s mental health center with $11 million federal grant
Rare achievement: UTA undergrad publishes research
Fact or fiction? The ADHD info dilemma
Genetic ancestry linked to risk of severe dengue
[Press-News.org] ChatGPT scores nearly 50 per cent on board certification practice test for ophthalmology, study showsAI tool scored more than 10 per cent higher one month later