(Press-News.org) New York, NY--January 11, 2021--Like a longtime couple who can predict each other's every move, a Columbia Engineering robot has learned to predict its partner robot's future actions and goals based on just a few initial video frames.
When two primates are cooped up together for a long time, we quickly learn to predict the near-term actions of our roommates, co-workers or family members. Our ability to anticipate the actions of others makes it easier for us to successfully live and work together. In contrast, even the most intelligent and advanced robots have remained notoriously inept at this sort of social communication. This may be about to change.
The study, conducted at Columbia Engineering's Creative Machines Lab led by Mechanical Engineering Professor Hod Lipson, is part of a broader effort to endow robots with the ability to understand and anticipate the goals of other robots, purely from visual observations.
The researchers first built a robot and placed it in a playpen roughly 3x2 feet in size. They programmed the robot to seek and move towards any green circle it could see. But there was a catch: Sometimes the robot could see a green circle in its camera and move directly towards it. But other times, the green circle would be occluded by a tall red carboard box, in which case the robot would move towards a different green circle, or not at all.
After observing its partner puttering around for two hours, the observing robot began to anticipate its partner's goal and path. The observing robot was eventually able to predict its partner's goal and path 98 out of 100 times, across varying situations--without being told explicitly about the partner's visibility handicap.
"Our initial results are very exciting," says Boyuan Chen, lead author of the study, which was conducted in collaboration with Carl Vondrick, assistant professor of computer science, and published today by Nature Scientific Reports. "Our findings begin to demonstrate how robots can see the world from another robot's perspective. The ability of the observer to put itself in its partner's shoes, so to speak, and understand, without being guided, whether its partner could or could not see the green circle from its vantage point, is perhaps a primitive form of empathy."
When they designed the experiment, the researchers expected that the Observer Robot would learn to make predictions about the Subject Robot's near-term actions. What the researchers didn't expect, however, was how accurately the Observer Robot could foresee its colleague's future "moves" with only a few seconds of video as a cue.
The researchers acknowledge that the behaviors exhibited by the robot in this study are far simpler than the behaviors and goals of humans. They believe, however, that this may be the beginning of endowing robots with what cognitive scientists call "Theory of Mind" (ToM). At about age three, children begin to understand that others may have different goals, needs and perspectives than they do. This can lead to playful activities such as hide and seek, as well as more sophisticated manipulations like lying. More broadly, ToM is recognized as a key distinguishing hallmark of human and primate cognition, and a factor that is essential for complex and adaptive social interactions such as cooperation, competition, empathy, and deception.
In addition, humans are still better than robots at describing their predictions using verbal language. The researchers had the observing robot make its predictions in the form of images, rather than words, in order to avoid becoming entangled in the thorny challenges of human language. Yet, Lipson speculates, the ability of a robot to predict the future actions visually is not unique: "We humans also think visually sometimes. We frequently imagine the future in our mind's eyes, not in words."
Lipson acknowledges that there are many ethical questions. The technology will make robots more resilient and useful, but when robots can anticipate how humans think, they may also learn to manipulate those thoughts.
"We recognize that robots aren't going to remain passive instruction-following machines for long," Lipson says. "Like other forms of advanced AI, we hope that policymakers can help keep this kind of technology in check, so that we can all benefit."
INFORMATION:
About the Study
The study is titled "Visual Behavior Modelling for Robotic Theory of Mind"
Authors are: Boyuan Chen, Carl Vondrick and Hod Lipson, Mechanical Engineering and Computer Science, Columbia Engineering.
The study was supported by NSF NRI 1925157 and DARPA MTO grant L2M Program HR0011-18-2-0020.
The authors declare no financial or other conflicts of interest.
LINKS:
Paper: http://www.nature.com/articles/s41598-020-77918-x
DOI: 10.1038/s41598-020-77918-x
VIDEO: https://youtu.be/f2U7_jZVxcU
PROJECT WEBSITE: https://www.creativemachineslab.com/robot-visual-behavior-modeling.html
https://engineering.columbia.edu/faculty/hod-lipson
http://www.cs.columbia.edu/~bchen/
https://me.columbia.edu/
https://www.cs.columbia.edu/
http://engineering.columbia.edu/
Columbia Engineering
Columbia Engineering, based in New York City, is one of the top engineering schools in the U.S. and one of the oldest in the nation. Also known as The Fu Foundation School of Engineering and Applied Science, the School expands knowledge and advances technology through the pioneering research of its more than 220 faculty, while educating undergraduate and graduate students in a collaborative environment to become leaders informed by a firm foundation in engineering. The School's faculty are at the center of the University's cross-disciplinary research, contributing to the Data Science Institute, Earth Institute, Zuckerman Mind Brain Behavior Institute, Precision Medicine Initiative, and the Columbia Nano Initiative. Guided by its strategic vision, "Columbia Engineering for Humanity," the School aims to translate ideas into innovations that foster a sustainable, healthy, secure, connected, and creative humanity.
The Third Pole centered on the Tibetan Plateau is home to headwaters of over 10 major Asian rivers. These glacier-based water systems, also known as the Asian Water Towers, will have to struggle to quench the thirst of downstream communities despite more river runoff brought on by a warmer climate, according to a recent study published in Nature Climate Change.
By constraining earth system models for precipitation projections, together with estimated glacier melt contributions, the study quantified the wet-season runoff of seven rivers at the Third Pole, and found it would increase 1.0-7.2% by the end of the 21st century for warming in the range of 1.5-4°C. However, the study also showed that rising water demands from the growing population will outweigh ...
To perform calculations, quantum computers need qubits to act as elementary building blocks that process and store information. Now, physicists have produced a new type of qubit that can be switched from a stable idle mode to a fast calculation mode. The concept would also allow a large number of qubits to be combined into a powerful quantum computer, as researchers from the University of Basel and TU Eindhoven have reported in the journal Nature Nanotechnology.
Compared with conventional bits, quantum bits (qubits) are much more fragile and can lose their information content very quickly. The challenge for quantum computing is therefore to keep the sensitive ...
Diets rich in healthy and plant-based foods encourages the presence of gut microbes that are linked to a lower risk of common illnesses including heart disease, research has found.
A large-scale international study using metagenomics and blood chemical profiling has uncovered a panel of 15 gut microbes associated with lower risks of common conditions such as obesity and type 2 diabetes. The study has been published today in Nature Medicine from researchers at King's College London, Massachusetts General Hospital (MGH), Harvard T.H. Chan School of Public Health, the University ...
With the Proof of Concept funding line, the ERC grants recipients of ERC frontier research funds (Starting, Consolidator, Advanced or Synergy grants) with 150.000 Euro to develop promising ideas with commercial or societal potential to the proof of concept stage. With this funding, Olaf Groß and his team in the Metabolism and Inflammation Group at the Institute of Neuropathology of the Medical Center - University of Freiburg will test whether a new class of immune activating drugs they discovered can boost the effectiveness of cancer immunotherapies ...
Heidelberg/Germany, 11 January 2021 - Development of an in vitro human-derived tissue model for studying virus infection and disease progression in the alveolar cells of the lungs responsible for oxygen and carbon dioxide exchange with the blood might enable the study of possible therapies for acute respiratory distress syndrome (ARDS) triggered by SARS-CoV-2. Researchers in the Netherlands have demonstrated that the SARS-CoV-2 replicates efficiently in their model resembling the human bronchioalveolar system that is thought to play a critical role in progression of infection towards pneumonia and ARDS.
It is already ...
Two billion years after the Big Bang, the Universe was still very young. However, thousands of huge galaxies, rich in stars and dust, were already formed. An international study, led by SISSA - Scuola Internazionale Superiore di Studi Avanzati, now explains how this was possible. Scientists combined observational and theoretical methods to identify the physical processes behind their evolution and, for the first time, found evidence for a rapid growth of dust due to a high concentration of metals in the distant Universe. The study, published in Astronomy&Astrophysics, offers a new approach to investigate the evolutionary phase of massive objects.
Since their initial discovery 20 years ago, very distant and massive galaxies that form prodigious amount of ...
Patients with severe COVID-19 disease have significantly shorter telomeres, according to a study conducted by researchers at the Spanish National Cancer Research Centre (CNIO) in collaboration with the COVID-IFEMA Field Hospital, published in the journal Aging. The study, led by Maria A. Blasco and whose first authors are Raúl Sánchez and Ana Guío-Carrión, postulates that telomere shortening as a consequence of the viral infection impedes tissue regeneration and that this is why a significant number of patients suffer prolonged sequelae.
Blasco was already developing a therapy to regenerate lung tissue in pulmonary fibrosis patients; she now believes that this treatment -which should still take at least a year and a half to become available- ...
NEW YORK, NY (Jan. 11, 2021)--A new study has found that up to 20% of glioblastomas--an aggressive brain cancer--are fueled by overactive mitochondria and may be treatable with drugs currently in clinical trials.
Mitochondria are responsible for creating the energy that fuels all cells. Though they are usually less efficient at producing energy in cancer, tumor cells in this newly identified type of glioblastoma rely on the extra energy provided by overactive mitochondria to survive.
The study, by cancer scientists at Columbia University's Vagelos College ...
For some years, an active substance from the leaves of an ornamental plant has been regarded as a possible forerunner of a new group of potent drugs. So far, however, it has been very laborious to manufacture it in large quantities. That could now change: Researchers at the University of Bonn (Germany) have identified a bacterium that produces the substance and can also be easily cultivated in the laboratory. The results are published in the journal Nature Communications.
The coralberry currently once again adorns many living rooms: In winter it bears bright red fruits, which make it a popular ornamental plant at this time of year. For pharmacists, however, it is interesting for a different reason: It contains ...
CHAMPAIGN, Ill. -- For years, researchers have worked to repurpose excess atmospheric carbon dioxide into new chemicals, fuels and other products traditionally made from hydrocarbons harvested from fossil fuels. The recent push to mitigate the climactic effects of greenhouse gases in the atmosphere has chemists on their toes to find the most efficient means possible. A new study introduces an electrochemical reaction, enhanced by polymers, to improve CO2-to-ethylene conversion efficiency over previous attempts.
The results of the study led by University of Illinois Urbana-Champaign chemistry ...
