(Press-News.org)
Northwestern University engineers have developed a new system for full-body motion capture — and it doesn’t require specialized rooms, expensive equipment, bulky cameras or an array of sensors.
Instead, it requires a simple mobile device.
Called MobilePoser, the new system leverages sensors already embedded within consumer mobile devices, including smartphones, smart watches and wireless earbuds. Using a combination of sensor data, machine learning and physics, MobilePoser accurately tracks a person’s full-body pose and global translation in space in real time.
“Running in real time on mobile devices, MobilePoser achieves state-of-the-art accuracy through advanced machine learning and physics-based optimization, unlocking new possibilities in gaming, fitness and indoor navigation without needing specialized equipment,” said Northwestern’s Karan Ahuja, who led the study. “This technology marks a significant leap toward mobile motion capture, making immersive experiences more accessible and opening doors for innovative applications across various industries.”
Ahuja’s team will unveil MobilePoser on Oct. 15, at the 2024 ACM Symposium on User Interface Software and Technology in Pittsburgh. “MobilePoser: Real-time full-body pose estimation and 3D human translation from IMUs in mobile consumer devices” will take place as a part of a session on “Poses as Input.”
An expert in human-computer interaction, Ahuja is the Lisa Wissner-Slivka and Benjamin Slivka Assistant Professor of Computer Science at Northwestern’s McCormick School of Engineering, where he directs the Sensing, Perception, Interactive Computing and Experience (SPICE) Lab.
Limitations of current systems
Most movie buffs are familiar with motion-capture techniques, which are often revealed in behind-the-scenes footage. To create CGI characters — like Gollum in “Lord of the Rings” or the Na’vi in “Avatar” — actors wear form-fitting suits covered in sensors, as they prowl around specialized rooms. A computer captures the sensor data and then displays the actor’s movements and subtle expressions.
“This is the gold standard of motion capture, but it costs upward of $100,000 to run that setup,” Ahuja said. “We wanted to develop an accessible, democratized version that basically anyone can use with equipment they already have.”
Other motion-sensing systems, like Microsoft Kinect, for example, rely on stationary cameras that view body movements. If a person is within the camera’s field of view, these systems work well. But they are impractical for mobile or on-the-go applications.
Predicting poses
To overcome these limitations, Ahuja’s team turned to inertial measurement units (IMUs), a system that uses a combination of sensors — accelerometers, gyroscopes and magnetometers — to measure a body’s movement and orientation. These sensors already reside within smartphones and other devices, but the fidelity is too low for accurate motion-capture applications. To enhance their performance, Ahuja’s team added a custom-built, multi-stage artificial intelligence (AI) algorithm, which they trained using a publicly available, large dataset of synthesized IMU measurements generated from high-quality motion capture data.
With the sensor data, MobilePoser gains information about acceleration and body orientation. Then, it feeds this data through AI algorithm, which estimates joint positions and joint rotations, walking speed and direction, and contact between the user’s feet and the ground.
Finally, MobilePoser uses a physics-based optimizer to refine the predicted movements to ensure they match real-life body movements. In real life, for example, joints cannot bend backward, and a head cannot rotate 360 degrees. The physics optimizer ensures that captured motions also cannot move in physically impossible ways.
The resulting system has a tracking error of just 8 to 10 centimeters. For comparison, the Microsoft Kinect has a tracking error of 4 to 5 centimeters, assuming the user stays within the camera’s field of view. With MobilePoser, the user has freedom to roam.
“The accuracy is better when a person is wearing more than one device, such as a smartwatch on their wrist plus a smartphone in their pocket,” Ahuja said. “But a key part of the system is that it’s adaptive. Even if you don’t have your watch one day and only have your phone, it can adapt to figure out your full-body pose.”
Potential use cases
While MobilePoser could give gamers more immersive experiences, the new app also presents new possibilities for health and fitness. It goes beyond simply counting steps to enable the user to view their full-body posture, so they can ensure their form is correct when exercising. The new app also could help physicians analyze patients’ mobility, activity level and gait. Ahuja also imagines the technology could be used for indoor navigation — a current weakness for GPS, which only works outdoors.
“Right now, physicians track patient mobility with a step counter,” Ahuja said. “That’s kind of sad, right? Our phones can calculate the temperature in Rome. They know more about the outside world than about our own bodies. We would like phones to become more than just intelligent step counters. A phone should be able to detect different activities, determine your poses and be a more proactive assistant.”
To encourage other researchers to build upon this work, Ahuja’s team has released its pre-trained models, data pre-processing scripts and model training code as open-source software. Ahuja also says the app will soon be available for iPhone, AirPods and Apple Watch.
END
Distinct immune “signatures” in patients who develop adverse events while taking immunotherapy for cancer may help oncologists identify patients at risk and treat them early to prevent serious side effects, suggests a study by researchers from the Johns Hopkins Kimmel Cancer Center and its Bloomberg~Kimmel Institute for Cancer Immunotherapy.
The study, published Oct. 15 in the Journal of Clinical Investigation, is the largest to date to look at immune signatures in patients with a range of cancers. It included a diverse sample of 111 patients treated with immune checkpoint inhibitors ...
The North Texas Genome Center (NTGC) at UT Arlington is getting an upgrade in the form of a next-generation genetic sequencer that will allow researchers to study genetic links between health and disease at a large scale. The new $1 million instrument, the only one of its kind in North Texas, will allow faculty and students to more deeply analyze rare genetic variants, an important step in discovering new insights for the future of health care.
“I’m excited to be able to expand and upgrade the technological capabilities of the North Texas Genome Center, a collaborative ...
A research group led by Joshua Neunuebel at the University of Delaware, USA, tracked the behavior of mice using machine learning to understand how they handle aggressive behavior from other mice. The researchers’ findings, published on October 15th in the open-access journal PLOS Biology, show that male mice deescalate aggressive encounters by running over to a female mouse to distract the aggressive male mouse.
The researchers recorded groups of two male and two female mice interacting over five hours. Like many other animals, mice have social hierarchies, and in almost each group recorded, one male was always significantly more aggressive towards the other.
Social interactions ...
In an era where mental health awareness is at the forefront of public discourse, a new historical review is shedding light on the often-overlooked French contributions to our understanding of depression. Published in Genomic Psychiatry, the study by Dr. Kenneth S. Kendler and Virginia Justis of Virginia Commonwealth University examines a seminal 1897 French monograph that helped shape modern concepts of melancholia and depression.
The review focuses on "La Mélancolie" by Jacques Roubinovitch ...
New Study Key Takeaways:
New model focuses on more equity and accessibility in opioid overdose treatment locations and resource allocation.
Utilizing this model, the results project that within 2 years, there will be a decrease in the number of people with opioid use disorder, an increase in the number of people getting treatment and a decrease in opioid-related deaths.
Policymakers should target adding treatment facilities to counties that have significantly fewer facilities than their population share and are more socially vulnerable.
BALTIMORE, MD, October 15, 2024 – The opioid epidemic is a crisis that has plagued the United States for decades. ...
Key takeaways
Voting eligibility for people with felony convictions has expanded since 1997, with more than two million individuals now able to vote.
Several factors, like misinformation, distrust of government or lack of clarity around procedures, impede many from exercising their right to vote.
Direct outreach, including through trusted, on-the-ground community organizations, as well as informational text messaging, can help eligible voters navigate the process and make sure their voices are included in elections.
As get-out-the-vote efforts hit high gear nationwide, a team of sociologists, political scientists ...
Alzheimer’s disease may damage the brain in two distinct phases, based on new research funded by the National Institutes of Health (NIH) using sophisticated brain mapping tools. According to researchers who discovered this new view, the first, early phase happens slowly and silently — before people experience memory problems — harming just a few vulnerable cell types. In contrast, the second, late phase causes damage that is more widely destructive and coincides with the appearance of symptoms and the rapid ...
Advancements in quantum information technology are paving the way for faster and more efficient data transfer. A key challenge has been ensuring that qubits, the fundamental units of quantum information, can be transferred between different wavelengths without losing their essential properties, such as coherence and entanglement. As reported in Advanced Photonics, researchers from Shanghai Jiao Tong University (SJTU) recently made significant strides in this area by developing a novel method for broadband frequency conversion, a crucial step for future quantum networks.
The SJTU team focused on a technique using X-cut thin film lithium niobate ...
Aston University is leading a new £5.5 million EU research project
Will focus on converting fibre-optic cables into sensors to detect natural hazards
Could identify earthquakes and tsunamis and assess civil infrastructure.
Aston University is leading a new £5.5 million EU research project to explore converting existing telecommunication fibre-optic cables into sensors which can detect natural hazards, such as earthquakes and tsunamis, and assess the condition of civil infrastructure.
The project is called ECSTATIC (Engineering Combined Sensing and Telecommunications Architectures for Tectonic and Infrastructure Characterisation) ...
About The Study: Existing evaluations of large language models mostly focus on accuracy of question answering for medical examinations, without consideration of real patient care data. Dimensions such as fairness, bias, and toxicity and deployment considerations received limited attention. Future evaluations should adopt standardized applications and metrics, use clinical data, and broaden focus to include a wider range of tasks and specialties.
Corresponding Author: To contact the corresponding author, Nigam H. Shah, MBBS, PhD, email nigam@stanford.edu.
To ...
