Search robot thinks for itself

Tracking objects with the help of language models

2026-03-17

(Press-News.org) A robot that can locate lost items on command – this is the latest development at the Technical University of Munich (TUM). It combines knowledge from the internet with a spatial map of its surroundings to efficiently find the objects being sought.

The new robot from Prof. Angela Schoellig’s TUM Learning Systems and Robotics Lab looks like a broomstick on wheels with a camera mounted at the top. It is one of the first robots that not only integrates image understanding but also applies it to a clearly defined task.

To find a pair of glasses misplaced in the kitchen, for example, the robot has to look around and build a three-dimensional image of the room. The camera initially provides two-dimensional images, but these pixels also contain depth information. This creates a spatial map of the environment that is accurate to the centimeter and is constantly updated. A laptop also provides the robot with information about which objects are visible in the image and what significance they have for humans.

“We have taught the robot to understand its surroundings,” says Prof. Angela Schoellig. The head of the Robotics Lab at the TUM Chair of Safety, Performance and Reliability for Learning Systems aims to develop robots that can navigate any environment independently. Humanoid robots working in factories or robots in care settings in private homes require this newly developed basic understanding, which, as Schoellig explains, “is important for all robots that move in spaces that are constantly changing”.

Internet knowledge translated into the robot’s language

The robot therefore understands that a table or window sill can be used to briefly set down a pair of glasses, whereas a stovetop or a sink are not suitable for this purpose. “The language model captures the relationships between the objects and we convert this information into the robot’s language,” explains Prof. Schoellig. Two-digit numbers appear on the three-dimensional map of the environment, constantly recalculating the likelihood that the object being searched for is located there. According to the research results, the robot then searches the probable locations almost 30 per cent more efficiently than if it searched randomly throughout the room. Artificial intelligence is used in two ways: on the one hand in image recognition and on the other hand through the use of a language model.

Another special capability of the robot: it remembers previous images and is able to compare them with new images of its surroundings. If a new object suddenly appears in the kitchen, it recognizes the change with a high degree of certainty (95 per cent) and marks these areas as “highly probable” search locations.

Next step: searching behind cupboard doors

In the next step, the TUM scientist and board member at the Munich Institute of Robotics and Machine Intelligence (TUM MIRMI) also wants to search for objects that are in a drawer or behind a door. To do this, however, the robot will not merely have to draw on knowledge from the internet but will also have to interact with its surroundings. Robotic arms and hands must open a cupboard and determine whether it opens upwards or sideways and how best to grasp the handle. This will enable the robot to search even in closed spaces such as cupboards or drawers.

Publications:

Where did I leave my glasses? Open-Vocabulary Semantic Exploration in Real-World Semi-Static Environments; Benjamin Bogenberger, Oliver Harrison, Orrin Dahanaggamaarachchi, Lukas Brunke, Jingxing Qian, Siqi Zhou, Angela P. Schoellig; IEEE Robotics and Automation Letters, 3 March 2026; https://ieeexplore.iee...t/11359697

Further information:

- Prof. Angela Schoellig is a member of the board of the Munich Institute of Robotics and Machine Intelligence (TUM MIRMI). The institute is an integrative research institute at the Technical University of Munich (TUM) that focuses on robotics and AI. The institute brings together expertise in key areas of robotics, including perception and data science. Nearly 80 TUM chairs are networked within MIRMI to develop innovative robotic and AI-supported solutions for the environment, health, mobility, work, as well as security and defense. TUM MIRMI is headed by Prof. Lorenzo Masia. Further information can be found at https://www.mirmi.tum.de/.

Additional material for media outlets:

Photos for download: https://mediatum.ub.tum.de/1846659 Scientific video: https://utiasdsl.githu...ploration/ Subject experts:

Prof. Angela Schoellig

Chair of Safety, Performance and Reliability for Learning Systems

Technical University of Munich (TUM)

angela.schoellig@tum.de

END

ELSE PRESS RELEASES FROM THIS DATE:

Researchers find more effective approach to revealing Majorana zero modes in superconductors

2026-03-17

An international team of researchers, including physicists from HSE MIEM, has demonstrated that nonmagnetic impurities can help more accurately reveal Majorana zero modes—quantum states considered promising building blocks for quantum computing. The researchers found that these impurities shift the energy levels that typically obscure the Majorana signal, while leaving the mode itself largely unaffected, thereby making its spectral peak more distinct. The study has been published in Research. Majorana ...

HSE biologists identify factors that accelerate breast cancer recurrence

2026-03-17

Scientists at HSE University have identified a molecular mechanism underlying aggressive breast cancer. They found that the signals supporting tumour growth originate not from the tumour itself but from its microenvironment. The researchers also demonstrated that reduced levels of the IGFBP6 protein in the tumour microenvironment lead to the accumulation of macrophages—immune cells associated with a higher risk of cancer recurrence. These findings already make it possible to assess patient risk more accurately and may, in the future, enable the development of drugs that target cells of the ...

Using AI to improve standard-of-care cardiac imaging

2026-03-17

Heart disease is the leading cause of adult death worldwide, making cardiovascular disease diagnosis and management a global health priority. An echocardiogram, or cardiac ultrasound, is one of the most commonly used imaging tools employed by physicians to diagnose a variety of heart diseases and conditions.  Most standard echocardiograms provide two-dimensional visual images (2D) of the three-dimensional (3D) cardiac anatomy. These echocardiograms often capture hundreds of 2D slices or views of a beating heart that can enable physicians to make clinical assessments ...

Stanford researchers develop novel "scaffold-free" approach for treating damaged muscles

2026-03-17

Traumatic muscle injury can be associated with volumetric muscle loss (VML), often leading to permanent functional loss. Until recently, experimental therapies to support muscle regeneration have faced several key limitations, including the challenge of delivering sufficient healing cells to the traumatized area and the inability of conventional tissue transplants to conform to the specific shape of a muscle defect. A recent study, led by senior author Ngan F. Huang, PhD, Associate Professor of Cardiothoracic Surgery (Research) in the Stanford Department of Cardiothoracic Surgery, highlights a unique approach her research team has developed to address this problem and potentially ...

Qubits created using unexpected materials

2026-03-17

For the first time, researchers have demonstrated that the properties of the perovskite family of materials can be used to create so-called quantum bits. The findings, published in the journal Nature Communications, pave the way for more affordable materials in future quantum computers. According to the researchers from Linköping University, Sweden, behind the study, few within the field believed it would be possible. The reason is that the atoms in perovskite materials should, in theory, interact ...

Superconductor advance could unlock ultra-energy-efficient electronics

2026-03-17

Superconducting materials could play a crucial role in the energy-efficient applications of the future. However, several technical challenges still stand in the way of their practical use. Now, researchers at Chalmers University of Technology in Sweden have developed a new material design that addresses a major obstacle in the field: enabling superconductivity to operate at higher temperatures while also withstanding strong magnetic fields. This breakthrough could pave the way for far more energy-efficient electronics and quantum technologies. Digital devices, ...

Closing your eyes might not help you hear better after all

2026-03-17

WASHINGTON, March 17, 2026 — Most people will close their eyes when trying to concentrate on a faint sound. Many of us have been told that keeping our eyes closed helps us hear better — that it frees up our brains’ processing abilities and increases our auditory sensitivity. However, that strategy may sometimes backfire, particularly in environments with a lot of loud background noise. In JASA, published on behalf of the Acoustical Society of America by AIP Publishing, researchers from Shanghai Jiao Tong University tested whether a person closing their eyes can really hear better in noisy environments. To test this, volunteers listened to a collection of sounds through ...

New computational biology tool automates and standardizes genome sequencing analysis

2026-03-17

In a single experiment, scientists can decipher the entire genomes of many patient samples, animal models or cultured cells. To fully realize the potential to study biology at this unprecedented scale, researchers must be equipped to analyze the titanic troves of data generated by these new methods. Scientists at Sanford Burnham Prebys Medical Discovery Institute and the University of California Los Angeles published findings March 17, 2026, in Cell Reports Methods discussing building and testing a new computational tool for tackling massive and complex sequencing datasets. The new resource, ...

Climate change is fueling disease outbreaks

2026-03-17

Diseases historically absent from the United States have been showing up in Florida, Texas, California and other U.S. states in recent years. To understand why, look to Peru. That’s where researchers from Stanford and other institutions analyzed the connection between a cyclone and a massive outbreak of dengue fever, a mosquito-borne viral disease that can cause fever, rash, and life-threatening symptoms like hemorrhage and shock. Their findings, published March 17 in One Earth, reveal that warmer, wetter weather linked to climate change is making disease epidemics more likely. "Health impacts of climate change aren't something we're waiting for,” ...

Three anesthesia drugs all have the same effect in the brain, MIT researchers find

2026-03-17

CAMBRIDGE, MA -- When patients undergo general anesthesia, doctors can choose among several drugs. Although each of these drugs acts on neurons in different ways, they all lead to the same result: a disruption of the brain’s balance between stability and excitability, according to a new MIT study. This disruption causes neural activity to become increasingly unstable, until the brain loses consciousness, the researchers found. The discovery of this common mechanism could make it easier to develop new technologies for monitoring patients while ...

Muscle-healing 'Ally' turns 'Enemy': A novel immune cell subset that controls muscle regeneration and ossification in FOP

Waterpipe smoking can cause carbon monoxide poisoning even after brief use, during outdoor smoking, or through indoor secondhand exposure

Impact of Japan's indoor smoke-free laws on the prevalence of smoke-free establishments

New study fills research gap in food safety to better protect pregnant people from Listeria

[Press-News.org] Search robot thinks for itself
Tracking objects with the help of language models

Search robot thinks for itself

ELSE PRESS RELEASES FROM THIS DATE:

LAST 30 PRESS RELEASES: