PRESS-NEWS.org - Press Release Distribution
PRESS RELEASES DISTRIBUTION

PolyU-led research reveals that sensory and motor inputs help large language models represent complex concepts

2025-07-10
(Press-News.org) Can one truly understand what “flower” means without smelling a rose, touching a daisy or walking through a field of wildflowers? This question is at the core of a rich debate in philosophy and cognitive science. While embodied cognition theorists argue that physical, sensory experience is essential to concept formation, studies of the rapidly evolving large language models (LLMs)suggest that language alone can build deep, meaningful representations of the world.

By exploring the similarities between LLMs and human representations, researchers at The Hong Kong Polytechnic University (PolyU) and their collaborators have shed new light on the extent to which language alone can shape the formation and learning of complex conceptual knowledge. Their findings also revealed how the use of sensory input for grounding or embodiment – connecting abstract with concrete concepts during learning – affects the ability of LLMs to understand complex concepts and form human-like representations. The study, in collaboration with scholars from Ohio State University, Princeton University and City University of New York, was recently published in Nature Human Behaviour.

Led by Prof. LI Ping, Sin Wai Kin Foundation Professor in Humanities and Technology, Dean of the PolyU Faculty of Humanities and Associate Director of the PolyU-Hangzhou Technology and Innovation Research Institute, the research team selected conceptual word ratings produced by state-of-the-art LLMs, namely ChatGPT (GPT-3.5, GPT-4) and Google LLMs (PaLM and Gemini). They compared them with human-generated word ratings of around 4,500 words across non-sensorimotor (e.g., valence, concreteness, imageability), sensory (e.g., visual, olfactory, auditory) and motor domains (e.g., foot/leg, mouth/throat) from the highly reliable and validated Glasgow Norms and Lancaster Norms datasets.

The research team first compared pairs of data from individual humans and individual LLM runs to discover the similarity between word ratings across each dimension in the three domains, using results from human-human pairs as the benchmark. This approach could, for instance, highlight to what extent humans and LLMs agree that certain concepts are more concrete than others. However, such analyses might overlook how multiple dimensions jointly contribute to the overall representation of a word. For example, the word pair “pasta” and “roses” might receive equally high olfactory ratings, but “pasta” is in fact more similar to “noodles” than to “roses” when considering appearance and taste. The team therefore conducted representational similarity analysis of each word as a vector along multiple attributes of non-sensorimotor, sensory and motor dimensions for a more complete comparison between humans and LLMs.

The representational similarity analyses revealed that word representations produced by the LLMs were most similar to human representations in the non-sensorimotor domain, less similar for words in sensory domain and most dissimilar for words in motor domain. This highlights LLM limitations in fully capturing humans’ conceptual understanding. Non-sensorimotor concepts are understood well but LLMs fall short when representing concepts involving sensory information like visual appearance and taste, and body movement. Motor concepts, which are less described in language and rely heavily on embodied experiences, are even more challenging to LLMs than sensory concepts like colour, which can be learned from textual data.

In light of the findings, the researchers examined whether grounding would improve the LLMs’ performance. They compared the performance of more grounded LLMs trained on both language and visual input (GPT-4, Gemini) with that of LLMs trained on language alone (GPT-3.5, PaLM). They discovered that the more grounded models incorporating visual input exhibited a much higher similarity with human representations.

Prof. Li Ping said, “The availability of both LLMs trained on language alone and those trained on language and visual input, such as images and videos, provides a unique setting for research on how sensory input affects human conceptualisation. Our study exemplifies the potential benefits of multimodal learning, a human ability to simultaneously integrate information from multiple dimensions in the learning and formation of concepts and knowledge in general. Incorporating multimodal information processing in LLMs can potentially lead to a more human-like representation and more efficient human-like performance in LLMs in the future.”

Interestingly, this finding is also consistent with those of previous human studies indicating the representational transfer. Humans acquire object-shape knowledge through both visual and tactile experiences, with seeing and touching objects activating the same regions in human brains. The researchers pointed out that – as in humans – multimodal LLMs may use multiple types of input to merge or transfer representations embedded in a continuous, high-dimensional space. Prof. Li added, “The smooth, continuous structure of embedding space in LLMs may underlie our observation that knowledge derived from one modality could transfer to other related modalities. This could explain why congenitally blind and normally sighted people can have similar representations in some areas. Current limits in LLMs are clear in this respect”.

Ultimately, the researchers envision a future in which LLMs are equipped with grounded sensory input, for example, through humanoid robotics, allowing them to actively interpret the physical world and act accordingly. Prof. Li said, “These advances may enable LLMs to fully capture embodied representations that mirror the complexity and richness of human cognition, and a rose in LLM’s representation will then be indistinguishable from that of humans.”

END


ELSE PRESS RELEASES FROM THIS DATE:

Premature babies should have early skin-to-skin contact with their mother

2025-07-10
More premature babies who had early skin-to-skin contact with their mother were being breastfed at the time of discharge from hospital and for up to one year afterwards. However, this is far from the only benefit. A team from St. Olavs Hospital and the Norwegian University of Science and Technology (NTNU) have looked at this issue in a number of articles. They now hope that more hospitals will change their practice so that premature babies are not separated from their mother during the first few hours after birth. “The first few ...

New research in JNCCN offers reassurance about localized prostate cancer prognosis

2025-07-10
PLYMOUTH MEETING, PA [July 10, 2025] — New research in the July 2025 issue of JNCCN—Journal of the National Comprehensive Cancer Network finds that for people diagnosed with nonmetastatic low-risk prostate cancer later in life, and treated according to NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines®), 90% were likely to survive their cancer for their remaining life-expectancy. Of those with nonmetastatic higher-risk cancer and a longer life expectancy, that likelihood was still greater than 65%. The researchers studied 62,839 people diagnosed with non-metastatic prostate ...

Fluorinated polyimide: High toughness and low dielectric properties pave new path for high-frequency communication materials

2025-07-10
Conventional polyimides (PIs) exhibit excellent thermal stability and mechanical performance, yet their dielectric properties (dielectric constant (Dk) > 3.2, dissipation factor (Df) > 0.005 @ 10 GHz). In previous reports, the introduction of trifluoromethyl reduced the dielectric constant and dissipation factor, but it increased chain rigidity, weakened hydrogen bonds interaction, and reduced free volume, which definitely reduced mechanical performance (such as poor toughness leading to crack risks in advanced packaging). Therefore, it is necessary to design PI materials with high toughness and low dielectric properties to meet ...

Radar-based control of a helical microswimmer in 3-Dimensional space with dynamic obstacles

2025-07-10
Recent advances have yielded significant progress in actuation, navigation, and control of magnetic microrobots. Nevertheless, dynamic obstacle avoidance in 3D environments remains a critical challenge, often relying on computationally intensive path-planning methods that limit real-time performance. "Using a hierarchical radar system to enable high-frequency direction updates minimizes computational load while ensuring collision-free navigation," explained corresponding author Jiangfan Yu, a professor at The Chinese University of Hong Kong, Shenzhen. The radar framework comprises (a) a motion sphere for directional ...

Short-term physical activity reduces metabolic-associated steatohepatitis by promoting the degradation of branched-chain amino acids in skeletal muscle

2025-07-10
Background and Aims Metabolic-associated steatohepatitis (MASH) is an advanced and progressive liver disease that potentially causes cirrhosis and hepatocellular carcinoma. Exercise is a crucial and effective intervention for ameliorating metabolic dysfunction-associated steatotic liver disease. This study aimed to provide a comprehensive understanding of the underlying mechanisms of MASH, which benefit a broad spectrum of MASH patients, including those who have difficulty engaging in physical activity. Methods We established a mouse model of MASH and selectively knocked down ...

A multimodal amphibious robot driven by soft electrohydraulic flippers

2025-07-10
The key component of this robot is the soft electrohydraulic actuator. “Unlike traditional rigid robots, soft robots have better environmental adaptability and safety, and electrohydraulic actuation technology is one of the breakthroughs in the field of soft robots in recent years. It has higher energy efficiency and less noise,” said Fuyi Fang, a researcher at School of Mechanical Engineering, Shanghai Jiao Tong University. The soft electrohydraulic actuator is a sealed pouch filled with silicone ...

US vacation renters waste $2 billion worth of food annually

2025-07-10
COLUMBUS, Ohio – If you find yourself routinely throwing away groceries and leftovers the night before you check out of an Airbnb, you’re not alone: A new study values the food wasted by U.S. vacation renters at about $2 billion each year. Based on survey results, researchers estimated that groceries, takeout and restaurant leftovers averaging $12 in value per night of short-term lodging remains uneaten. Though the analysis didn’t pinpoint how much remaining food landed in the trash, that was the most common outcome reported, along with taking food home or leaving it behind. On average, ...

Automatized vocabulary knowledge in predicting speech fluency

2025-07-10
When learning a second language (L2), many are likely familiar with the challenge of memorizing vocabulary, only to struggle with recalling and using it fluently in speech. Studies have found that speaking fluently in L2 depends not only on knowing what words mean but also on how quickly and automatically you can access and use them appropriately in contexts. This ability to retrieve contextually appropriate word meanings without conscious effort is known as automatized vocabulary knowledge (AVK). In a new study, a team of researchers led by Mr. Kotaro Takizawa from Waseda University, Japan, along with Prof. Kazuya Saito and Dr. ...

Uncovering the relationship between oral function and lifestyle-related diseases

2025-07-10
The term ‘oral health’ refers to the health of the mouth, teeth, gums, and other related structures, and it is closely linked with our well-being and quality of life. Recently, there has been a lot of interest in understanding how oral health is linked to and influences our overall health. For example, it was found that gum disease may be associated with various diseases such as diabetes and kidney disease. Poor oral health can also affect a person’s diet, which further increases a person’s ...

Drone herbicide applications prove effective for common reed control

2025-07-10
WESTMINSTER, Colorado – 10 July 2025 – New research from the University of Waterloo shows that a single, targeted herbicide application from a Remotely Piloted Aircraft System (RPAS) can suppress common reed invasions with more than 99% effectiveness. This result is among many research findings recently published online in Invasive Plant Science and Management (IPSM), volume 18, 2025, by Cambridge University Press, a Weed Science Society of America (WSSA) research journal. “The 99% reduction in live common reed stems observed with RPAS-based herbicide application demonstrates its capacity to suppress invasive common reed effectively,” says Rebecca ...

LAST 30 PRESS RELEASES:

Introducing BioEmu: A generative AI Model that enables high-speed and accurate prediction of protein structural ensembles

Replacing mutated microglia with healthy microglia halts progression of genetic neurological disease in mice and humans

New research shows how tropical plants manage rival insect tenants by giving them separate ‘flats’

Condo-style living helps keep the peace inside these ant plants

Climate change action could dramatically limit rising UK heatwave deaths

Annual heat-related deaths projected to increase significantly due to climate and population change

Researchers discover new way cells protect themselves from damage

Rivers choose their path based on erosion — a discovery that could transform flood planning and restoration

New discovery reveals dopamine operates with surgical precision, not as a broad signal

New AI tool gives a helping hand to x ray diagnosis

New Leicester study reveals hidden heart risks in women with Type 2 Diabetes

Over 400 different types of nerve cell have been grown – far more than ever before

Newly discovered molecule may explain reduced muscle mass in type 2 diabetes

Rheumatoid arthritis and muscle wasting: New review points to overlooked complications

Overcoming intrinsic dispersion locking by misaligned bilayer metagratings

Vaccines work: Cohort data from Denmark show real-world evidence of stable protection against HPV-related cervical cancer

Underwater shaped charge explosions: a comprehensive experimental study on coupling dynamics

Wristband sensor provides all-in-one monitoring for diabetes and cardiovascular care

Unveiling the spatiotemporal landscape of Ganoderma lingzhi: insights into ganoderic acid distribution and biosynthesis

Quality and antibiotic resistance risks in livestock probiotics in China

Genomic study reveals deep roots of human survival and adaptation in Himalayas

Differential obesity trends in Asian and Native Hawaiian or other Pacific Islander US adults

Cumulative anticholinergic exposure and change in gait speed and grip strength in older adults

Study suggests lemurs age differently than humans

Hypothermia alters glucose metabolism and may reveal mechanisms of metabolic disease

Content or form? The two possible paths of our memories

Research team produces low-loss spin waveguide network

PolyU-led research reveals that sensory and motor inputs help large language models represent complex concepts

Premature babies should have early skin-to-skin contact with their mother

New research in JNCCN offers reassurance about localized prostate cancer prognosis

[Press-News.org] PolyU-led research reveals that sensory and motor inputs help large language models represent complex concepts