(Press-News.org) PULLMAN, Wash. – While large language models like ChatGPT can do well when choosing multiple-choice answers on financial licensing exams, they falter when dealing with more nuanced tasks.
A Washington State University-led study analyzed more than 10,000 responses to financial exam questions by the artificial intelligence language models BARD, Llama and ChatGPT.
The researchers asked the models to not only choose answers but also explain the reasoning behind them, then compared those text answers to those by human professionals. While two versions of ChatGPT performed the best at these tasks, they still showed a high level of inaccuracy with more advanced topics.
“It’s far too early to be worried about ChatGPT taking finance jobs completely,” said study author DJ Fairhurst of WSU’s Carson College of Business. “For broad concepts where there have been good explanations on the internet for a long time, ChatGPT can do a very good job at synthesizing those concepts. If it’s a specific, idiosyncratic issue, it’s really going to struggle.”
For this study, published in the Financial Analysts Journal, Fairhurst and co-author Daniel Greene of Clemson University used questions from licensing exams including the Securities Industry Essentials exam as well as the Series 6, 7, 65 and 66.
To move beyond the AI models’ ability to simply pick the right answer, the researchers asked the models to provide written explanations. They also chose questions based on specific job tasks financial professionals might actually perform.
“Passing certification exams is not enough. We really need to dig deeper to get to what these models can really do,” said Fairhurst.
Of all the models, the paid version of ChatGPT, version 4.0, performed the best, providing answers that were the most similar to human experts. Its accuracy was also 18 to 28 percentage points higher than the other models. However, this changed when the researchers fine-tuned the earlier, free version of ChatGPT 3.5, by feeding it examples of correct responses and explanations. After this tuning, it came close to ChatGPT 4.0 in accuracy and even surpassed it in providing answers’ that were similar to those of human professionals.
Both models still fell short, though, when it came to certain types of questions. While they did well reviewing securities transactions and monitoring financial market trends, the models gave more inaccurate answers for specialized situations such as determining clients insurance coverage and tax status.
Fairhurst and Greene, along with WSU doctoral student Adam Bozman, are now working on other ways to determine what ChatGPT can and cannot do with a project that asks it to evaluate potential merger deals. For this, they are taking advantage of the fact that ChatGPT is trained on data up until September 2021, and using deals made after that date where the result is known. Preliminary findings are showing that so far, the AI model isn’t very good at this task.
Overall, the researchers said that ChatGPT is still probably better used as a tool to assist rather than as a replacement for an established financial professional. On the other hand, AI may change the way some investment banks employ entry-level analysts.
“The practice of bringing a bunch of people on as junior analysts, letting them compete and keeping the winners – that becomes a lot more costly,” said Fairhurst. “So it may mean a downturn in those types of jobs, but it’s not because ChatGPT is better than the analysts, it's because we’ve been asking junior analysts to do tasks that are more menial.”
END
ChatGPT errors show it cannot replace finance professionals, yet
2024-12-18
ELSE PRESS RELEASES FROM THIS DATE:
Bias in AI amplifies our own biases
2024-12-18
Artificial intelligence (AI) systems tend to take on human biases and amplify them, causing people who use that AI to become more biased themselves, finds a new study by UCL researchers.
Human and AI biases can consequently create a feedback loop, with small initial biases increasing the risk of human error, according to the findings published in Nature Human Behaviour.
The researchers demonstrated that AI bias can have real-world consequences, as they found that people interacting with biased AIs became more likely to underestimate women’s ...
Paris Declaration calls for data-driven forensics to spearhead the fight against fake science
2024-12-18
Supporters of research integrity have signed a new declaration calling for data-driven forensics – known as Forensic Scientometrics (FoSci) – to lead the charge in detecting, exposing and even preventing fake science.
The Forensic Scientometrics (FoSci) Paris Declaration was drafted during an event in Paris last week organized and run by Digital Science’s VP of Research Integrity, Dr Leslie McIntosh. The event was hosted at Institut Universitaire de France (IUF) by Dr Guillaume Cabanac, research integrity “sleuth” and professor at the University of Toulouse, as part of his research chair titled ...
Coral reef nightlife becomes more predatory with artificial light
2024-12-18
Artificial light can wake sleeping fish and attract predators, changing nighttime coral reef communities, according to new research using novel underwater infrared cameras.
The study, published today in Global Change Biology, was conducted by an international team of scientists from the UK, France, French Polynesia and Chile and the first large-scale experiment to investigate the impact of light pollution on the nightlife of coral reefs.
Lead author, Dr Emma Weschke, from the University of Bristol’s School of Biological Sciences, ...
Young exoplanet’s atmosphere unexpectedly differs from its birthplace
2024-12-18
Just as some children physically resemble their parents, many scientists have long thought that developing planets should resemble the swirling disk of gas and dust that births them.
But, in a new study, a Northwestern University-led team of astrophysicists discovered the resemblance might be looser than previously thought. By studying a still-forming exoplanet and its surrounding natal disk, the researchers uncovered a mismatched composition of gases in the planet’s atmosphere compared to gases within the disk.
The surprising finding potentially confirms long-held skepticism that scientists’ current model of planet formation is too simplified.
The ...
Carnivorous squirrels documented in California
2024-12-18
A ground squirrel with cheeks stuffed with nuts, seeds or grains, is a common sight. But a new study provides the first evidence that California ground squirrels also hunt, kill and eat voles. The study, led by the University of Wisconsin-Eau Claire and University of California, Davis, is the first to chronicle widespread carnivorous behavior among squirrels.
Published in the Journal of Ethology, the study fundamentally changes our understanding of ground squirrels. It suggests that what was considered a granivorous species actually is an opportunistic omnivore and more flexible in its diet than was assumed.
The observations occurred in 2024 — the 12th year of the Long-term ...
Researchers develop new test for early osteoarthritis diagnosis
2024-12-18
Diagnosing osteoarthritis often occurs in the late stages when cartilage degradation is severe, making it difficult to distinguish it from other types of arthritis and to determine the best treatment plan. In work published in the Journal of Orthopaedic Research, investigators have developed and tested a new diagnostic test that uses two markers found in the synovial fluid of patients’ joints.
After assessing levels of cartilage oligomeric matrix protein and interleukin‐8, the team came up with an algorithm based on the ratio of these two markers and validated its efficacy in differentiating osteoarthritis from inflammatory arthritis in 171 human knee synovial ...
Research uncovers high extinction risk for many Amazonian tree species
2024-12-18
Among tree species in the Ecuadorian Amazon, investigators at the Universidad de las Américas, in Ecuador, found that 14% are critically endangered and 47% are endangered. The Plants, People, Planet study indicates that trees with smaller fruits face the greatest threats due to declines of specific animal species that disperse them.
The findings reveal that the extinction risk for endemic trees is associated not only with extrinsic factors such as deforestation but also with complex relationships with other living organisms in their environment.
“Thus, our results highlight the importance ...
How did the COVID-19 pandemic and lockdown affect the identity of trans and gender diverse youth?
2024-12-18
Research published in the British Journal of Developmental Psychology indicates that the COVID-19 pandemic and lockdown had largely positive impacts on gender identity development in trans and gender diverse youth.
For the study, 295 transgender and gender diverse U.S. youth, ages 13–22 years, were asked the open-ended question “How has the COVID pandemic changed or affected your own understanding of your gender identity?”
Responses revealed several themes. The most prevalent was “time for identity development,” ...
Does having more chronic conditions increase older adults’ risk of declining kidney function?
2024-12-18
Older adults with multiple chronic conditions may face a high risk of kidney function decline, according to new research published in the Journal of the American Geriatrics Society.
In the study, 3,094 older individuals from the Swedish National Study on Aging and Care in Kungsholmen were followed for 15 years.
There was an independent dose-response relationship between the number of chronic conditions and kidney function decline, so that as the number of chronic conditions increased, kidney function decline became more likely and steeper.
When considering the makeup of conditions rather than just ...
Experts issue new ethical standards for body donation programs
2024-12-18
A new report in the journal Anatomical Sciences Education outlines best practices and standards for human body donation programs across the United States, which accept whole body donations after death for research and education.
The report, issued by a task force of American Association for Anatomy members, seeks to align body donation programs with evolving societal values and legal frameworks. It aims to maintain the highest ethical standards for donors by upholding the principles of informed consent, oversight, and dignity.
The report emphasizes the importance of ensuring potential donors and their families ...