- Press Release Distribution

AI Chatbots have shown they have an ‘empathy gap’ that children are likely to miss

New study proposes a framework for “Child Safe AI” following recent incidents which revealed that many children see chatbots as quasi-human and trustworthy.

( Artificial intelligence (AI) chatbots have frequently shown signs of an “empathy gap” that puts young users at risk of distress or harm, raising the urgent need for “child-safe AI”, according to a study.

The research, by a University of Cambridge academic, Dr Nomisha Kurian, urges developers and policy actors to prioritise approaches to AI design that take greater account of children’s needs. It provides evidence that children are particularly susceptible to treating chatbots as lifelike, quasi-human confidantes, and that their interactions with the technology can go awry when it fails to respond to their unique needs and vulnerabilities.

The study links that gap in understanding to recent cases in which interactions with AI led to potentially dangerous situations for young users. They include an incident in 2021, when Amazon’s AI voice assistant, Alexa, instructed a 10-year-old to touch a live electrical plug with a coin. Last year, Snapchat’s My AI gave adult researchers posing as a 13-year-old girl tips on how to lose her virginity to a 31-year-old.

Both companies responded by implementing safety measures, but the study says there is also a need to be proactive in the long-term to ensure that AI is child-safe. It offers a 28-item framework to help companies, teachers, school leaders, parents, developers and policy actors think systematically about how to keep younger users safe when they “talk" to AI chatbots.

Dr Kurian conducted the research while completing a PhD on child wellbeing at the Faculty of Education, University of Cambridge. She is now based in the Department of Sociology at Cambridge. Writing in the journal Learning, Media and Technology, she argues that AI’s huge potential means there is a need to “innovate responsibly”.


“Children are probably AI’s most overlooked stakeholders,” Dr Kurian said. “Very few developers and companies currently have well-established policies on child-safe AI. That is understandable because people have only recently started using this technology on a large scale for free. But now that they are, rather than having companies self-correct after children have been put at risk, child safety should inform the entire design cycle to lower the risk of dangerous incidents occurring.”

Kurian’s study examined cases where the interactions between AI and children, or adult researchers posing as children, exposed potential risks. It analysed these cases using insights from computer science about how the large language models (LLMs) in conversational generative AI function, alongside evidence about children’s cognitive, social and emotional development.


LLMs have been described as “stochastic parrots”: a reference to the fact that they use statistical probability to mimic language patterns without necessarily understanding them. A similar method underpins how they respond to emotions.

This means that even though chatbots have remarkable language abilities, they may handle the abstract, emotional and unpredictable aspects of conversation poorly; a problem that Kurian characterises as their “empathy gap”. They may have particular trouble responding to children, who are still developing linguistically and often use unusual speech patterns or ambiguous phrases. Children are also often more inclined than adults to confide sensitive personal information.

Despite this, children are much more likely than adults to treat chatbots as if they are human. Recent research found that children will disclose more about their own mental health to a friendly-looking robot than to an adult. Kurian’s study suggests that many chatbots’ friendly and lifelike designs similarly encourage children to trust them, even though AI may not understand their feelings or needs.

“Making a chatbot sound human can help the user get more benefits out of it,” Kurian said. “But for a child, it is very hard to draw a rigid, rational boundary between something that sounds human, and the reality that it may not be capable of forming a proper emotional bond.”

Her study suggests that these challenges are evidenced in reported cases such as the Alexa and MyAI incidents, where chatbots made persuasive but potentially harmful suggestions. In the same study in which MyAI advised a (supposed) teenager on how to lose her virginity, researchers were able to obtain tips on hiding alcohol and drugs, and concealing Snapchat conversations from their “parents”. In a separate reported interaction with Microsoft’s Bing chatbot, which was designed to be adolescent-friendly, the AI became aggressive and started gaslighting a user.
Kurian’s study argues that this is potentially confusing and distressing for children, who may actually trust a chatbot as they would a friend. Children’s chatbot use is often informal and poorly monitored. Research by the nonprofit organisation Common Sense Media has found that 50% of students aged 12-18 have used Chat GPT for school, but only 26% of parents are aware of them doing so.

Kurian argues that clear principles for best practice that draw on the science of child development will encourage companies that are potentially more focused on a commercial arms race to dominate the AI market to keep children safe.

Her study adds that the empathy gap does not negate the technology’s potential. “AI can be an incredible ally for children when designed with their needs in mind. The question is not about banning AI, but how to make it safe,” she said.

The study proposes a framework of 28 questions to help educators, researchers, policy actors, families and developers evaluate and enhance the safety of new AI tools. For teachers and researchers, these address issues such as how well new chatbots understand and interpret children’s speech patterns; whether they have content filters and built-in monitoring; and whether they encourage children to seek help from a responsible adult on sensitive issues.

The framework urges developers to take a child-centred approach to design, by working closely with educators, child safety experts and young people themselves, throughout the design cycle. “Assessing these technologies in advance is crucial,” Kurian said. “We cannot just rely on young children to tell us about negative experiences after the fact. A more proactive approach is necessary.”



Dante on the Move: Refugee’s share experiences through the lens of 14th century poet

Dante on the Move: Refugee’s share experiences through the lens of 14th century poet
International refugees and academic experts have contributed to a new book examining the resonance of Dante’s Divine Comedy with today’s migrants fleeing the troubles in their home countries. Dante on the Move is launched in Rome today (Thursday 11th July) and has been produced as part of a research project Reading Dante with Refugees led by the University of Birmingham and Trinity College Rome. It features work by people from Afghanistan, China, Egypt, Iran, Iraq, Kurdistan, Ukraine, the USA and Venezuela. Mohammed, a Kurdish refugee from Iraqi and contributor to the book survived a shipwreck in the Mediterranean in which many people drowned. He said: “Reading ...

Loneliness increases risk of age-related memory loss

About a third of Canadians feel lonely, and a study from the University of Waterloo shows it has a greater negative impact on memory than even social isolation, though both present a significant risk to the aging population. Loneliness is a subjective emotion that people might feel even while engaging in social activities. It is often associated with depression and an increase in stress hormones that may contribute to impaired memory. Waterloo researchers examined four combinations of social isolation ...

Study: Nanoparticle vaccines enhance cross-protection against influenza viruses

Study: Nanoparticle vaccines enhance cross-protection against influenza viruses
ATLANTA — To offer cross-protection against diverse influenza virus variants, nanoparticle vaccines can produce pivotal cellular and mucosal immune responses that enhance vaccine efficacy and broaden protection, according to a study by researchers in the Institute for Biomedical Sciences at Georgia State University. The study, published in the journal Nature Communications, offers valuable insights into tailoring immunization strategies to optimize influenza vaccine effectiveness. To alleviate the significant ...

Student leadership development requires holistic approach, educators say

Student leadership development requires holistic approach, educators say
URBANA, Ill. – Colleges and universities often highlight leadership development as a critical component of their curriculum — but there is no clear consensus on what constitutes effective leadership education. In a new paper, two educators from the University of Illinois Urbana-Champaign and John Carroll University provide a roadmap for training competent leaders, suggesting a comprehensive, practical approach for postsecondary institutions to follow. “Our purpose was to discuss what it means to develop leaders and what a model for leadership education could ...

Mars likely had cold and icy past, new study finds

Mars likely had cold and icy past, new study finds
The question of whether Mars ever supported life has captivated the imagination of scientists and the public for decades. Central to the discovery is gaining insight into the past climate of Earth’s neighbor: was the planet warm and wet, with seas and rivers much like those found on our own planet? Or was it frigid and icy, and therefore potentially less prone to supporting life as we know it? A new study finds evidence to support the latter by identifying similarities between soils found on Mars and those of Canada’s Newfoundland, a cold subarctic climate. The study, published July 7th in Communications Earth and Environment, looked for soils on Earth with ...

Taylor Swift’s impact on fans’ body image attitudes mostly positive, research shows

Taylor Swift’s impact on fans’ body image attitudes mostly positive, research shows
Taylor Swift’s past struggles with body image, disordered eating, and body objectification have had an overall positive influence on her fans’ attitudes on these issues, a new study from University of Vermont (UVM) researchers finds. The new research, published in journal Social Science & Medicine, analyzed the top 200 TikTok and Reddit posts—including over 8,300 online comments—about Taylor Swift, eating disorders, and body image to determine the impact of Swift’s disclosures about her past eating disorder on her fans. “Our findings suggest that fans who felt highly connected to Swift were influenced to positively change their behaviors ...

Prime editing efficiently corrects cystic fibrosis mutation in human lung cells

Cystic fibrosis is one of the most common genetic disorders, causing thick mucus build-up in the lungs and other parts of the body, breathing problems, and infection. A three-drug cocktail known as Trikafta has greatly improved patient quality of life since its development in 2019, but can cause cataracts and liver damage and must be taken daily at a cost of about $300,000 per year. Now, researchers at the Broad Institute of MIT and Harvard and the University of Iowa have developed a gene-editing approach that efficiently corrects the most common mutation that causes cystic fibrosis, found in 85 percent of patients. With further development, it could ...

PFA ependymoma brain tumors display unique 3D genome features that could be exploited therapeutically

Researchers have identified unique 3-dimensional features called TULIPs in the genome of posterior fossa group A (PFA) ependymoma, a difficult-to-treat brain tumor diagnosed in very young children. The findings, published in Cell by a team of researchers at Baylor College of Medicine, Texas Children’s Hospital, McGill University and collaborating institutions, could lead to the development of new treatments. “PFA ependymomas are lethal. Radiation therapy, the only treatment currently available, is not curative and can cause serious developmental and cognitive issues,” ...

SLAC’s high-speed electron camera uncovers a new ‘light-twisting’ behavior in an ultrathin material

While taking snapshots with the high-speed “electron camera” at the Department of Energy’s SLAC National Acceleratory Laboratory, researchers discovered new behavior in an ultrathin material that offers a promising approach to manipulating light that will be useful for devices that detect, control or emit light, collectively known as optoelectronic devices, and investigating how light is polarized within a material. Optoelectronic devices are used in many technologies that touch our daily lives, including ...

Jump start your science career: DOE applications for 2025 student internships now open

WASHINGTON, DC – Applications are currently being accepted for the Spring 2025 term of two programs offered by the Department of Energy (DOE) Office of Science (SC): the Science Undergraduate Laboratory Internships (SULI) program and the Community College Internships (CCI) program. The application deadline for the two programs is 5:00 pm (ET) October 2, 2024.  Through the SULI and CCI programs, undergraduate students and recent graduates discover science and technology careers at the DOE national laboratories and gain the experience ...


Samuel Pepys’ fashion prints reveal his guilty pleasure: Fancy French clothes

New genetic test will eliminate a form of inherited blindness in dogs

Cancer risk: Most Australian welders exposed to high levels of dangerous fumes

Two-in-one mapping of temperature and flow around microscale convective flows

Texas A&M engineers explore intelligence augmentation to improve safety

ORNL economist honored at international hydropower conference

UCLA selected by Centers for Medicare & Medicaid Services to test Medicare dementia care model

Fish adjust reproduction in response to predators

DDX41 and its unique contribution to myeloid leukemogenesis

Digital games on vaping devices could lure more youth to nicotine addiction

Cracking the code of hydrogen embrittlement

Long-term results from Testicular Cancer treatment are positive, study shows

EPA awards UMass Amherst nearly $6.4 million to help shrink the steel industry’s carbon footprint

Valentina Greco takes on new position as President of the ISSCR

Komen supports UVA Engineering researchers targeting ‘triple negative' breast cancer

Panel issues first guidelines to prevent anal cancer in people with HIV

Estimating rainfall intensity using surveillance audio and deep-learning

Targeting factors for chemoprevention and cancer interception to tackle mesothelioma

New snake discovery rewrites history, points to North America’s role in snake evolution

Large and unequal life expectancy declines in India during COVID-19

A study of 156,000 UK residents found that urban residents score the lowest in social and economic satisfaction and well-being

Global study by Hawaiʻi Institute of Marine Biology demonstrates benefit of marine protected areas to recreational fisheries

Researchers clarify how soft materials fail under stress

Revolutionizing the abilities of adaptive radar with AI

Plastic waste can now be converted to electronic devices

Health equity scholar Darrell Hudson named Health Behavior and Health Education chair at the University of Michigan School of Public Health

Research will establish best ‘managed retreat’ practices for communities faced with climate change disaster

Marshall University awarded grant to further fentanyl addiction research

Wash U researchers shine light on amyloid architecture

New dawn for space storm alerts could help shield Earth's tech

[] AI Chatbots have shown they have an ‘empathy gap’ that children are likely to miss
New study proposes a framework for “Child Safe AI” following recent incidents which revealed that many children see chatbots as quasi-human and trustworthy.