(Press-News.org) Imagine a coffee cup sitting on a table. Now, imagine a book partially obscuring the cup. As humans, we still know what the coffee cup is even though we can't see all of it. But a robot might be confused.
Robots in warehouses and even around our houses struggle to identify and pick up objects if they are too close together, or if a space is cluttered. This is because robots lack what psychologists call "object unity," or our ability to identify things even when we can't see all of them.
Researchers at the University of Washington have developed a way to teach robots this skill. The method, called THOR for short, allowed a low-cost robot to identify objects — including a mustard bottle, a Pringles can and a tennis ball — on a cluttered shelf. In a recent paper published in IEEE Transactions on Robotics, the team demonstrated that THOR outperformed current state-of-the-art models.
UW News reached out to senior author Ashis Banerjee, UW associate professor in both the industrial & systems engineering and mechanical engineering departments, for details about how robots identify objects and how THOR works.
How do robots sense their surroundings?
Ashis Banerjee: We sense the world around us using vision, sound, smell, taste and touch. Robots sense their surroundings using one or more types of sensors. Robots "see" things using either standard color cameras or more complex stereo or depth cameras. While standard cameras simply record colored and textured images of the surroundings, stereo and depth cameras also provide information on how far away the objects are, just like our eyes do.
On their own, however, the sensors cannot enable the robots to make "sense" of their surroundings. Robots need a visual perception system, similar to the visual cortex of the human brain, to process images and detect where all the objects are, estimate their orientations, identify what the objects might be and parse any text written on them.
Why is it hard for robots to identify objects in cluttered spaces?
AB: There are two main challenges here. First, there are likely a large number of objects of varying shapes and sizes. This makes it difficult for the robot’s perception system to distinguish between the different object types. Second, when several objects are located close to each other, they obstruct the views of other objects. Robots have trouble recognizing objects when they don't have a full view of the object.
Are there any types of objects that are especially hard to identify in cluttered spaces?
AB: A lot of that depends on what objects are present. For example, it is challenging to recognize smaller objects if there are a variety of sizes present. It is also more challenging to differentiate between objects with similar or identical shapes, such as different kinds of balls, or boxes. Additional challenges occur with soft or squishy objects that can change shape as the robot collects images from different vantage points in the room.
So how does THOR work and why is it better than previous attempts to solve this problem?
AB: THOR is really the brainchild of lead author Ekta Samani, who completed this research as a UW doctoral student. The core of THOR is that it allows the robot to mimic how we as humans know that partially visible objects aren't broken or entirely new objects.
THOR does this by using the shape of objects in a scene to create a 3D representation of each object. From there it uses topology, an area of mathematics that studies the connectivity between different parts of objects, to assign each object to a "most likely" object class. It does this by comparing its 3D representation to a library of stored representations.
THOR does not rely on training machine learning models with images of cluttered rooms. It just needs images of each of the different objects by themselves. THOR does not require the robot to have specialized and expensive sensors or processors, and it also works well with commodity cameras.
This means that THOR is very easy to build, and is, more importantly, readily useful for completely new spaces with diverse backgrounds, lighting conditions, object arrangements and degree of clutter. It also works better than the existing 3D shape-based recognition methods because its 3D representation of the objects is more detailed, which helps identify the objects in real time.
How could THOR be used?
AB: THOR could be used with any indoor service robot, regardless of whether the robot operates in someone’s home, an office, a store, a warehouse facility or a manufacturing plant. In fact, our experimental evaluation shows that THOR is equally effective for warehouse, lounge and family room-type spaces.
While THOR performs significantly better than the other existing methods for all kinds of objects in these cluttered spaces, it does the best at identifying kitchen-style objects, such as a mug or a pitcher, that typically have distinctive but regular shapes and moderate size variations.
There are several additional problems that need to be addressed, and we are working on some of them. For example, right now, THOR considers only the shape of the objects, but future versions could also pay attention to other aspects of appearance, such as color, texture or text labels. It is also worth looking into how THOR could be used to deal with squishy or damaged objects, which have shapes that are different from their expected configurations.
Also, some spaces may be so cluttered that certain objects might not be visible at all. In these scenarios, a robot needs to be able to decide to move around to "see" the objects better, or, if allowed, move around some of the objects to get better views of the obstructed objects.
Last but not least, the robot needs to be able to deal with objects it hasn't seen before. In these scenarios, the robot should be able to place these objects into a "miscellaneous" or "unknown" object category, and then seek help from a human to correctly identify these objects.
This research was funded in part by an Amazon Research Award.
For more information, contact Banerjee at firstname.lastname@example.org.
Q&A: Helping robots identify objects in cluttered spaces
ELSE PRESS RELEASES FROM THIS DATE:
Evaluation of ruxolitinib, a Janus Kinase inhibitor, in multiple myeloma
“[...] the results of the studies presented in this review will hopefully provide the impetus for conducting additional preclinical and clinical studies to evaluate RUX in the setting of MM as well as other types of cancer.” BUFFALO, NY- February 7, 2024 – A new research perspective was published in Oncotarget's Volume 15 on February 5, 2024, entitled, “Preclinical and clinical evaluation of the Janus Kinase inhibitor ruxolitinib in multiple myeloma.” In this new paper, researchers Ashley Del Dosso, Elizabeth Tadevosyan, and James ...
Blood thinners added to clot-busting medication did not improve stroke outcomes
Research Highlights: Although giving blood thinners along with clot-busting medication may help in treating heart attacks, it did not improve 90-day outcomes in people with clot-caused strokes. Enrollment in a large clinical trial, which had been planned to include more than 1,200 patients with ischemic stroke (clot-caused stroke), was halted after an independent data and safety monitoring board found no indication of benefit among the first 500 patients. The data analysis for the 500 enrolled participants found giving blood thinners along with clot-busting medication did not increase the risk of bleeding into the brain. Embargoed ...
Reinforcing the diverse ways people access seafood can ensure healthy communities in the face of change
(Santa Barbara, Calif.) — As climate change affects the oceans, coastal communities, particularly those at the front lines of ocean warming and sea level rise, are facing pressures that could threaten their access to aquatic foods. “Climate change and other economic shocks are impacting how people access seafood, and typically households that are most reliant on seafood, such as those in Pacific Island countries, are most at risk,” said Jacob Eurich, a research associate at UC Santa Barbara’s Marine Science Institute, and a fisheries scientist at the Environmental Defense Fund. Which is why, he added, it is necessary to increase food system resilience in the area, ...
Non-white victims of lethal violence and suicide in the US die significantly younger than their white counterparts
In the US, people of color who are killed by violence or die by suicide lose more potential years of life than white victims, according to a new study, which also explored factors that may contribute to these disparities. Gregory Zimmerman of Northeastern University in Boston, US, and colleagues present these findings in the open-access journal PLOS ONE on February 7, 2024. Mounting evidence suggests that, among victims of violence in the US, the average number of potential years of life lost—how much longer the victim would have lived if they survived—is greater for people of color than for white people. To deepen understanding, Zimmerman ...
Almost 1 in 5 Indian adults aged 60+ show signs of mild neurocognitive disorder, according to nationally representative data - more than previously recognized
Almost 1 in 5 Indian adults aged 60+ show signs of mild neurocognitive disorder, according to nationally representative data - more than previously recognized ### Article URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0297220 Article Title: Prevalence of DSM-5 mild and major neurocognitive disorder in India: Results from the LASI-DAD Author Countries: USA, UK, India Funding: This work was supported by the National Institute on Aging (R01 AG051125 (JL), R01 AG030153 (JL), U01 AG064948 (JL), R01 AG070953 (ALG)). The funders had no role in study design, ...
More biodiverse nature landscapes may better buffer against stress - but only if you notice the difference, per experiment using videos of urban woodland
More biodiverse nature landscapes may better buffer against stress - but only if you notice the difference, per experiment using videos of urban woodland ### Article URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0297179 Article Title: Does increasing biodiversity in an urban woodland setting promote positive emotional responses in humans? A stress recovery experiment using 360-degree videos of an urban woodland Author Countries: UK Funding: SF received a PhD Studentship funding from the UKRI Economic and Social Research Council, ...
UK austerity politics correlated with increased frailty in the oldest adults
The period of austerity politics from 2012 to 2018 was associated with steeper increases in frailty with age compared to pre-austerity between 2002 and 2010, according to a new study published February 7th in the open-access journal PLOS ONE by Carys Pugh of the University of Edinburgh, UK, and colleagues. Previous research has linked a stalling in life expectancy growth to austerity politics implemented in response to the 2008-2009 financial crisis. However, the mechanism through which public spending cuts are associated with decreased life expectancy ...
Psychology study unearths ways to bolster global climate awareness and climate action
An international team of scientists has created a tool that can aid in increasing climate awareness and climate action globally by highlighting messaging themes shown to be effective through experimental research. The web-based tool, and the methods undergirding its creation, appears in the journal Science Advances. The tool stems from a study involving nearly 250 researchers that drew more than 59,000 participants from 63 countries, including Algeria, China, Denmark, Germany, Israel, Japan, New Zealand, Peru, and the United States. “We ...
Scientists reveal why blueberries are blue
Tiny external structures in the wax coating of blueberries give them their blue colour, researchers at the University of Bristol can reveal. This applies to lots of fruits that are the same colour including damsons, sloes and juniper berries. In the study, published today in Science Advances, researchers show why blueberries are blue despite the dark red colour of the pigments in the fruit skin. Their blue colour is instead provided by a layer of wax that surrounds the fruit which is made up of miniature structures that scatter blue and UV light. This gives blueberries their blue appearance to humans and blue-UV to birds. The chromatic blue-UV reflectance ...
McMaster and ALK researchers discover new cell that remembers allergies
Researchers with McMaster University and Denmark-based pharmaceutical company ALK-Abello A/S have made a groundbreaking discovery: a new cell that remembers allergies. The discovery gives scientists and researchers a new target in treating allergies and could lead to new therapeutics. The research, published in Science Translational Medicine on Feb. 7, 2024, coins the brand-new cell as a type-2 memory B cell (MBC2). “We’ve discovered a type of memory B cell that had unique characteristics and a unique gene ...