NYU Tandon School of Engineering Researchers unveil tool to help developers create augmented reality task assistants
2023-09-28
(Press-News.org)
Augmented reality (AR) technology has long fascinated both the scientific community and the general public, remaining a staple of modern science fiction for decades.
In the pursuit of advanced AR assistants – ones that can guide people through intricate surgeries or everyday food preparation, for example – a research team from NYU Tandon School of Engineering has introduced Augmented Reality Guidance and User-Modeling System, or ARGUS.
An interactive visual analytics tool, ARGUS is engineered to support the development of intelligent AR assistants that can run on devices like Microsoft HoloLens 2 or MagicLeap. It enables developers to collect and analyze data, model how people perform tasks, and find and fix problems in the AR assistants they are building.
Claudio Silva, NYU Tandon Institute Professor of Computer Science and Engineering and Professor of Data Science at the NYU Center for Data Science, leads the research team that will present its paper on ARGUS at IEEE VIS 2023 on October 26, 2023, in Melbourne Australia. The paper received Honorable Mention in that event’s Best Paper Awards.
“Imagine you’re developing an AR AI assistant to help home cooks prepare meals,” said Silva. “Using ARGUS, a developer can monitor a cook working with the ingredients, so they can assess how well the AI is performing in understanding the environment and user actions. Also, how the system is providing relevant instructions and feedback to the user. It is meant to be used by developers of such AR systems.”
ARGUS works in two modes: online and offline.
The online mode is for real-time monitoring and debugging while an AR system is in use. It lets developers see what the AR system sees and how it's interpreting the environment and user actions. They can also adjust settings and record data for later analysis.
The offline mode is for analyzing historical data generated by the AR system. It provides tools to explore and visualize this data, helping developers understand how the system behaved in the past.
ARGUS’ offline mode comprises three key components: the Data Manager, which helps users organize and filter AR session data; the Spatial View, providing a 3D visualization of spatial interactions in the AR environment; and the Temporal View, which focuses on the temporal progression of actions and objects during AR sessions. These components collectively facilitate comprehensive data analysis and debugging.
“ARGUS is unique in its ability to provide comprehensive real-time monitoring and retrospective analysis of complex multimodal data in the development of systems,” said Silva. “Its integration of spatial and temporal visualization tools sets it apart as a solution for improving intelligent assistive AR systems, offering capabilities not found together in other tools.”
ARGUS is open source and available on GitHub under VIDA-NYU. The work is supported by the DARPA Perceptually-enabled Task Guidance (PTG) program.
About New York University Tandon School of Engineering
The NYU Tandon School of Engineering is home to a community of renowned faculty, undergraduate and graduate students united in a mission to understand and create technology that powers cities, enables worldwide communication, fights climate change, and builds healthier, safer, and more equitable real and digital worlds. The school’s culture centers on encouraging rigorous, interdisciplinary collaboration and research; fostering inclusivity, entrepreneurial thinking, and diverse perspectives; and creating innovative and accessible pathways for lifelong learning in STEM. NYU Tandon dates back to 1854, the founding year of both the New York University School of Civil Engineering and Architecture and the Brooklyn Collegiate and Polytechnic Institute. Located in the heart of Brooklyn, NYU Tandon is a vital part of New York University and its unparalleled global network. For more information, visit engineering.nyu.edu.
END
ELSE PRESS RELEASES FROM THIS DATE:
2023-09-28
Normal aging is usually associated with a decline in memory, although it is unclear what factors play a role. In a new study, researchers studied specific interneurons, which serve as communication centers that connect other neurons, in the regions of the brain that are important for learning and memory.
Increasing age places people at risk, whether it is because of a normal decrease in cognitive ability or due to postoperative cognitive disorders. In the latter, the deficits can persist for many months after surgery especially when the patients are older than 60. Unfortunately, the ...
2023-09-28
A new study in JAMA Network Open may provide key answers to how to help people experiencing chronic back pain.
The study, published today, examined the critical connection between the brain and pain for treating chronic pain. Specifically, they looked at the importance of pain attributions, which are people’s beliefs about the underlying causes of their pain, to reduce chronic back pain severity.
“Millions of people are experiencing chronic pain and many haven’t found ways to help with the pain, making it clear that something is missing in the way we’re diagnosing and treating people,” said the ...
2023-09-28
RICHLAND, Wash.—For decades, researchers have assumed that the inevitable filmy buildup on electrodes inside rechargeable batteries is the driver of performance loss. Now, we know that view is backward.
The buildup of mossy or tree-like structured lithium metal deposits on battery electrodes is not the root cause of performance loss, but rather a side effect. The first direct measurement of the electrical properties at the boundary between the solid electrode and the liquid electrolyte inside a rechargeable battery is reported today in Nature Energy.
The study, led by a research team at the Department of Energy’s Pacific Northwest National Laboratory, shows that the so-called ...
2023-09-28
UCLA-led research suggests that diabetes control can significantly improve for Latinos when a pharmacist implements an intervention that addresses these patients’ barriers to medication adherence.
In this pharmacist-led intervention, pharmacists reviewed the patients’ lab results, vital signs, and barriers to adherence. They then used this information to design a personally-tailored intervention to boost medication adherence and optimize the treatment regimen.
The study is the first to evaluate a collaboration between pharmacist and physicians aimed at improving diabetes care in a large healthcare system among Latino patients, said study lead Dr. Kimberly Narain, ...
2023-09-28
CAMBRIDGE, MA -- Alzheimer’s disease affects more than 6 million people in the United States, and there are very few FDA-approved treatments that can slow the progression of the disease.
In hopes of discovering new targets for potential Alzheimer’s treatments, MIT researchers have performed the broadest analysis yet of the genomic, epigenomic, and transcriptomic changes that occur in every cell type in the brains of Alzheimer’s patients.
Using more than 2 million cells from more than 400 postmortem brain samples, the researchers analyzed how gene expression is disrupted as Alzheimer’s progresses. They also tracked changes in ...
2023-09-28
Key takeaways:
Scientists have developed “multielement ink” – the first “high-entropy” semiconductor that can be processed at low-temperature or room temperature.
Multielement ink could enable cost-effective and energy-efficient semiconductor manufacturing.
The new semiconducting material could accelerate the sustainable production of next-gen microelectronics, photovoltaics, solid state lighting, and display devices.
Semiconductors are the heart of almost every ...
2023-09-28
Neuroscientists at the Flatiron Institute in New York City and their colleagues have made a big breakthrough using one of the world’s smallest brains.
Using an ion beam, an electron microscope and a lot of patience, the neuroscientists mapped out the entire early visual system — from the eyes that take in light to the neurons processing the information — for a parasitic wasp smaller than a grain of table salt. The feat marks the first time scientists have fully reconstructed such a system at the synaptic level from a single specimen of any animal species, the neuroscientists report ...
2023-09-28
Key Takeaways
In a survey-based study of US adults, those living in communities in which most people seldom left home at certain times during the COVID-19 pandemic were more likely to report symptoms of depression
The link remained strong even after considering COVID-19 activity, weather, and county-level economics
Accounting for state-level pandemic restrictions only modestly attenuated the association
BOSTON – Higher levels of depressive symptoms have been reported during the COVID-19 pandemic compared with other times in history, and as much as three times higher than prior to the ...
2023-09-28
An international team of researchers from the Max Planck Institute for the Structure and Dynamics of Matter (MPSD) in Hamburg and ETH Zurich has now demonstrated that it is possible to probe electron dynamics in liquids using intense laser fields and to retrieve the electron mean free path – the average distance an electron can travel before colliding with another particle. They found that the mechanism by which liquids emit a particular light spectrum known as the high-harmonic spectrum is markedly different from the one in other phases of matter like gases and solids. The team’s findings open the door to a deeper understanding of ultrafast dynamics in liquids.
Using ...
2023-09-28
In some patients, infection with the pandemic virus SARS-CoV-2 can trigger a dangerous immune response in hardened fatty deposits (plaques) lining the heart’s largest blood vessels, a new study shows.
The findings are based on the body’s immune system, which evolved to destroy invading microbes but also drives disease when triggered in the wrong context. Doing so brings on a set of responses termed inflammation, including swelling, which results as immune cells and signaling proteins home in on infection sites. ...
LAST 30 PRESS RELEASES:
[Press-News.org] NYU Tandon School of Engineering Researchers unveil tool to help developers create augmented reality task assistants
