- Press Release Distribution

Smooth avatar-user synchronization for the metaverse

Smooth avatar-user synchronization for the metaverse
( Mobile phones, smartwatches and earbuds are some gadgets that we carry around physically without much thought. The increasingly digitalised world sees a shrinking gap between human and technology, and many researchers and companies are interested in how technology can be further integrated into our lives.


What if, instead of incorporating technology into our physical world, we assimilate ourselves into a virtual environment? This is what Assistant Professor Xiong Zehui from the Singapore University of Technology and Design (SUTD) hopes to achieve in his research. Working with researchers from the Nanyang Technological University and the Guangdong University of Technology, this fruitful collaboration yielded a preprint, ‘Vision-based semantic communications for metaverse services: A contest theoretic approach’. The research will be presented at the IEEE Global Communications Conference in December 2023.


The joint effort focused on the notion of the metaverse—a virtual reality (VR) universe where users can control avatars to interact with the virtual environment. In this world, people can meet others (through their avatars), visit virtual locations and even make online purchases. In a sense, the metaverse hopes to extend past the limits of our physical reality.


One challenge for mainstream adoption of metaverse services is the demand for real-time synchronisation between human actions and avatar responses. “In the metaverse, avatars must be updated and rendered to reflect users’ behaviour. But achieving real-time synchronisation is complex, as it places high demands on the rendering resource allocation scheme of the metaverse service provider (MSP),” explained Asst Prof Xiong.


MSPs take on an enormous burden, relaying gargantuan amounts of data between users and the server. The more immersive the experience, the larger the data payload. Individuals that perform fast actions, such as running or jumping, will be more likely to face a lapse in smoothness of their avatars, as the MSP struggles to keep up.


A common solution is to restrict the number of users in a single virtual environment, ensuring the MSP has sufficient resources, or bandwidth, to simulate all users regardless of activity. This is a highly inefficient approach as users who are standing still will be afforded excess resources that they do not need. Only users with large movements require constant updates to their avatar, and hence the surplus bandwidth. The problem then leaves the question hanging—how can resources be allocated without wastage?


Asst Prof Xiong and team proposed a novel framework to optimise resource allocation in MSPs, with the overall aim of ensuring a smooth and immersive experience for all users. The scheme uses a semantic communication technique dubbed human pose estimation (HPE) to first reduce the information payload for users. Picking the most efficient distribution of resources among users was performed using contest theory, with user devices competing for just enough resources to simulate their avatars.


The first step for a seamless avatar-user interface requires efficient encoding of information to the MSPs. Consider a camera capturing the movements of a human to be translated into motions of their avatar. Each image captured by the camera is packed with redundant background information that is not useful for modelling the virtual characters.


In HPE, the computer is tasked to identify humans as the object, and highlight only the skeletal joints. Based on the joints, the algorithm can reconstruct a simple stickman-like model that can be sent to the MSPs. This caricature then guides the MSPs to model the actions taken by the avatar. In the research, Asst Prof Xiong and team managed to reduce the data overhead by a million-fold, from megabytes to bytes.


With this massive savings in bandwidth, the team then turned to modelling interactions between the MSPs and the network of users using contest theory. In this approach, users (or rather, their devices) are competitors fighting for the resources of the MSP. The algorithm seeks to minimise the latency across all users over a fixed amount of available resources. At the same time, the individual devices decide on their own update rates, depending on the actions taken by the user.


To test for lag, the algorithm measures the differences in the avatar position with different update rates. Users that face lag will have large discrepancies between their HPE stickmen and their avatars. At the same time, the MSP’s resources are treated as an award given out to competitors that performed well without lag.


However, each user still needs to be able to accurately deduce the right amount of resources to request from the MSP. Faced with the complexity of the task, the team turned to using machine learning. A neural network, dubbed the deep Q-network (DQN), optimises the resources distributed. Under this framework, the team effort yielded a 66% improvement in lag across all users, compared to traditional methods.


Asst Prof Xiong is optimistic for the future of the metaverse, citing healthcare, education, and marketing as potential areas that could benefit from metaverse services. He said, “Some developments or advancements that I’m most looking forward to include integrating cutting-edge technologies such as generative AI and VR, as well as the growth of global, digital, and virtual economies. It will be exciting to see how these advancements shape the future of the metaverse.”


[Attachments] See images for this press release:
Smooth avatar-user synchronization for the metaverse Smooth avatar-user synchronization for the metaverse 2 Smooth avatar-user synchronization for the metaverse 3


Defense against the enemy within

The research teams of Professor René Ketting at the Institute of Molecular Biology (IMB) in Mainz, Germany, and Dr. Sebastian Falk at the Max Perutz Labs in Vienna, Austria, have identified a new enzyme called PUCH, which plays a key role in preventing the spread of parasitic DNA in our genomes. These findings may reveal new insights into how our bodies detect and fight bacteria and viruses to prevent infections. Our cells are under constant attack from millions of foreign intruders, such as viruses and bacteria. To keep us from getting sick, ...

New tool reveals how drugs affect men, women differently -- and will make for safer medications

New tool reveals how drugs affect men, women differently -- and will make for safer medications
UVA Health researchers have developed a powerful new tool to understand how medications affect men and women differently, and that will help lead to safer, more effective drugs in the future. Women are known to suffer a disproportionate number of liver problems from medications. At the same time, they are typically underrepresented in drug testing. To address this, the UVA scientists have developed sophisticated computer simulations of male and female livers and used them to reveal sex-specific differences in how the tissues are affected by drugs. The new model has already provided ...

Researchers develop mixture of compounds to help preserve organs before transplantation

Using zebrafish as a model, investigators have determined a suitable combination of chemical compounds in which to store hearts, and potentially other organs, when frozen for extended periods of time before transplantation. The work, which is published in The FASEB Journal, involved a variety of methods, including assays at multiple developmental stages, techniques for loading and unloading agents, and the use of viability scores to quantify organ function. These methods allowed scientists to perform the largest and most comprehensive screen of cryoprotectant agents to determine their toxicity and efficiency at preserving ...

A surprising way to disrupt sleep

A surprising way to disrupt sleep
Osaka, Japan - Circadian rhythms, the internal biological clocks that regulate our daily activities, are essential for maintaining health and well-being. While the role of transcription in these rhythms is well-established, a new study sheds light on the critical importance of post-transcriptional processes. The research, titled “Circadian ribosome profiling reveals a role for the Period2 upstream open reading frame in sleep,” to be published in PNAS, redefines our understanding of how translation and post-transcriptional processes influence the body’s internal clock and its impact on sleep patterns. Timing Is Everything: ...

To Eat or Not to Eat: Targeting autophagy to enhance memory immune responses

To Eat or Not to Eat: Targeting autophagy to enhance memory immune responses
Osaka, Japan – Memory B cells depend on autophagy for their survival, but the protein Rubicon is thought to hinder this process. Researchers from Osaka University have discovered a shorter isoform of Rubicon called RUBCN100, which enhances autophagy in B cells. Mice that lacked the longer isoform, RUBCN130, produced more memory B cells in a way that relied on autophagy. These findings provide further insight into the role of Rubicon in autophagy. Autophagy is a mechanism that degrades and ...

Researchers find a cause of Parkinson’s disease

Until recently, our understanding of Parkinson's disease has been quite limited, which has been apparent in the limited treatment options and management of this debilitating condition. Our recent understanding has primarily revolved around the genetic factors responsible for familial cases, while the causative factors in the vast majority of patients remained unknown. However, in a new study, researchers from the University of Copenhagen have unveiled new insights into the workings of the brain in Parkinson's patients. Leading the groundbreaking discovery is Professor Shohreh Issazadeh-Navikas. “For the first time, we can show that mitochondria, ...

Pheromones influence death feigning behavior in beetles

Pheromones influence death feigning behavior in beetles
Predation is a driving force in the evolution of anti-predator strategies, and death feigning, characterized by immobility in response to threats, is a common defensive mechanism across various animal species. While this behavior can enhance an individual's survival prospects by reducing a predator's interest, it also carries costs, such as limited opportunities for feeding and reproduction. Recently, researchers from Okayama University, Japan, investigated how pheromones, important chemical signals that affect foraging and reproduction, might influence death-feigning behavior in the red flour beetle, Tribolium castaneum.  “Male beetles release an aggregation pheromone called ...

Genetics of attraction: mate choice in fruit flies

Genetics of attraction: mate choice in fruit flies
Genetic quality or genetic compatibility? What do female fruit flies prioritize when mating? Researchers at the University of Zurich show that both factors are important at different stages of the reproductive process and that females use targeted strategies to optimize the fitness of their offspring. Breeding female fruit flies face a difficult decision: do they mate with the male that has the best genes, or with the one whose genes best match their own? Evolutionary biologists from the University of Zurich and Concordia University have now investigated ...

FAU Engineering study employs deep learning to explain extreme events

FAU Engineering study employs deep learning to explain extreme events
Identifying the underlying cause of extreme events such as floods, heavy downpours or tornados is immensely difficult and can take a concerted effort by scientists over several decades to arrive at feasible physical explanations. Extreme events cause significant deviation from expected behavior and can dictate the overall outcome for a number of scientific problems and practical situations. For example, practical scenarios where a fundamental understanding of extreme events can be of vital importance include rogue waves in the ocean that could endanger ships and offshore structures or increasingly ...

New study uncovers potential treatment for non-alcoholic fatty liver disease

New study uncovers potential treatment for non-alcoholic fatty liver disease
A breakthrough study, jointly led by Professor Jang Hyun Choi and Professor Sung Ho Park from the Department of Biological Sciences at UNIST has identified an important factor involved in the development of non-alcoholic fatty liver disease (NAFLD) caused by obesity. The research team discovered that Thrap3, a protein associated with thyroid hormone receptors, plays a significant role in exacerbating NAFLD by inhibiting the activity of adenosine monophosphate-activated protein kinase (AMPK), a key regulator of fat metabolism in the liver. NAFLD encompasses various metabolic diseases such as fatty hepatitis and cirrhosis resulting from excessive fat accumulation. ...


Tracing how the infant brain responds to touch with near-infrared spectroscopy

These are the world's most effective charities

When is an aurora not an aurora?

Advisory panel issues field-defining recommendations for US government investments in particle physics research

Doctors discover many patients at UNC’s Inflammatory Bowel Disease Clinic screen positive for malnutrition

BNL: Advisory panel issues field-defining recommendations for U.S. government investments in particle physics research

International collaboration uses faculty member’s research on ancient Roman migration, seeks to understand Balkan genomic history

USF Health Heart Institute doctors are upbeat about cardiac regeneration

AI-driven breakthroughs in cells study: SFU-UBC collaboration introduces "MCS-detect" for advancements in super-resolution microscopy

Advisory panel issues field-defining recommendations for investments in particle physics research

$3.8 million NIH grant to fund Southwest Center on Resilience for Climate Change and Health

What happens when the brain loses a hub? 

Study reveals Zika’s shape-shifting machinery—and a possible vulnerability

RIT leading STEM co-mentoring network

Genetic mutations that promote reproduction tend to shorten human lifespan, study shows

CAMH develops potential new drug treatment for multiple sclerosis

Polyethylene waste could be a thing of the past

A dynamic picture of how we respond to high or low oxygen levels

University of Toronto researchers discover new lipid nanoparticle that shows muscle-specific mRNA delivery, reduces off-target effects.

Evolving insights in blood-based liquid biopsies for prostate cancer interrogation

Finding the most heat-resistant substances ever made

Time-tested magnesium oxide: Unveiling CO2 absorption dynamics

Engaging heterosexual men more effectively could slash HIV infections in Uganda

A fork in the rhod: Janelia researchers unveil comprehensive collection of rhodamine-based fluorescent dyes

The Gerontological Society of America congratulates new 2023 awardees

Texas A&M Institute part of national effort to harness nuclear laser fusion for limitless energy

How health system hesitancies contributed to COVID risks

Stand Up to Cancer names Julian Adams, Ph.D., President and CEO

Immersive VR goggles for mice unlock new potential for brain science

Racial and ethnic differences in hospice use among Medicaid-only and dual-eligible decedents

[] Smooth avatar-user synchronization for the metaverse