(Press-News.org) ADELPHI, Md. -- Army researchers developed a pioneering framework that provides a baseline for the development of collaborative multi-agent systems.
The framework is detailed in the survey paper Survey of recent multi-agent reinforcement learning algorithms utilizing centralized training, which is featured in the SPIE Digital Library. Researchers said the work will support research in reinforcement learning approaches for developing collaborative multi-agent systems such as teams of robots that could work side-by-side with future Soldiers.
"We propose that the underlying information sharing mechanism plays a critical role in centralized learning for multi-agent systems, but there is limited study of this phenomena within the research community," said Army researcher and computer scientist Dr. Piyush K. Sharma of the U.S. Army Combat Capabilities Development Command, known as DEVCOM, Army Research Laboratory. "We conducted this survey of the state-of-the-art in reinforcement learning algorithms and their information sharing paradigms as a basis for asking fundamental questions on centralized learning for multi-agent systems that would improve their ability to work together."
Sharma's collaborators on this project include DEVCOM ARL researchers Drs. Erin Zaroukian, Rolando Fernandez, Michael Dorothy, Derrik Asher and Anjon Basak, a postdoctoral fellow under the Oak Ridge Associated Universities fellowship program.
This survey of state-of-the-art in reinforcement learning establishes a baseline for researchers seeking to develop autonomous multi-agent systems through enhanced information sharing mechanisms, for example rewarding functions or observation and state space sharing.
Training multiple agents simultaneously is more difficult due to the dynamic nature of complex environments that can suffer from the curse of dimensionality, the more agents there are, the more complicated the coordination becomes, Sharma said. This paper develops a framework to characterize the key information sharing parameters that are often confusing and not easy to understand.
The researchers predict that centralization in training may be the solution towards more rapidly developing autonomous systems that can flexibly work alongside Soldiers in the future.
"Consistent, centralized training can result in multi-agent systems that work more reliably together, increasing the level of trust from the Soldier of the artificial intelligence," Sharma said. "Specifically, we focused on identifying and characterizing the underlying mathematical framework of the most recent centralized learning algorithms."
Such a mathematical model can provide an avenue to explore alternate centralized learning techniques to gauge their effect on the learning rate and emergent collaborative behaviors, he said.
The survey exceeds prior research literature in two ways:
Creates a consolidated view of the latest state-of-the-art in reinforcement learning algorithms
Outlines a novel approach to describing information shared during centralized learning
The researchers focused on the algorithms published within five to six years. As these algorithms are very recent, they have not yet been explored extensively by the research community. At the time of publication, they did not find comprehensive prior work.
The researchers attempted to define and categorize the mechanisms for sharing, orienting on what is actually being shared as opposed to how it is shared. They are optimistic that they have identified gaps in the recent reinforcement learning techniques worthy of further study that could potentially enhance the agent training process.
The researchers said they are optimistic the survey will generate discussion and further exploration of the problem space of machine learning to train autonomous multi-agent systems.
"As the demand for multi-agent systems working cooperatively has become more prevalent in the commercial industry, for example, Amazon Warehouse Robots, Intel's drone show at winter Olympics 2018. There is also an emerging need for these multi-agent system technologies to assist the Army in collaborative tactical operations," Sharma said. "The research resulting from this survey document can make the goal of reliable, collaborative AI achievable."
Moving forward, the team feels better equipped to investigate particular aspects of multi-agent reinforcement learning based approaches that train agents in centralized fashion.
Centralized techniques come with certain limitations, so they will also conduct empirical analysis of existing decentralized learning techniques, Sharma said. They plan to move towards modeling and simulation of multi-agent reinforcement learning training to validate and extend theories of agent learning, behavior and coordination.
INFORMATION:
WASHINGTON -- For more than 25 years, Burmese pythons have been living and breeding in the Florida Everglades where they prey on native wildlife and disrupt the region's delicate ecosystems. A new study shows that infrared cameras could make it easier to spot these invasive snakes in the Florida foliage, providing a new tool in the effort to remove them.
In the Optical Society (OSA) journal Applied Optics, researchers led by Dr. Kyle Renshaw from the University of Central Florida College of Optics and Photonics report that a near infrared camera helped people detect Burmese pythons at distances up to 1.3 times farther away than was possible using a traditional visible-wavelength ...
AMES, Iowa - Scientists have invested great time and effort into making connections between a plant's genotype, or its genetic makeup, and its phenotype, or the plant's observable traits. Understanding a plant's genome helps plant biologists predict how that plant will perform in the real world, which can be useful for breeding crop varieties that will produce high yields or resist stress.
But environmental conditions play a role as well. Plants with the same genotype will perform differently when grown in different environments. A new study led by an Iowa State University scientist uses advanced data analytics to help scientists understand ...
A study conducted by a group of Brazilian researchers contributes to a deeper understanding of the molecular basis for schizophrenia, and potentially to the development of more specific and effective treatments for the disease. The medications currently available on the market act generically on the brain and can have severe adverse side effects.
Treatment of post-mortem samples from the hippocampus of schizophrenic patients with an NMDA receptor antagonist pointed to biological processes associated with the disease that are specific to neurons and oligodendrocytes. NMDA receptors are neurotransmitter receptors located in the postsynaptic ...
In those with fatty liver disease, a person's fat goes to their liver instead of their fat tissue, either because of an absence of fat depots, which is seen in the rare genetic disease lipodystrophy, or because the depots are too full, which is seen in people with obesity.
One third of these people will go on to develop nonalcoholic steatohepatitis, or NASH - an advanced form of fatty liver disease brought on by progressive inflammation and scarring in the organ.
In 2002, Michigan Medicine endocrinologist Elif Oral, M.D., who had just moved from the National Institutes of Health at the time, published her discovery that patients with severe lipodystrophy lack leptin, a hormone that helps curb appetite and control weight gain. When given ...
BOSTON - Automated emails and letters that provide personalized feedback related to cafeteria purchases at work may help employees make healthier food choices. That's the conclusion of a new study that was led by investigators at Massachusetts General Hospital (MGH) and is published in END ...
RUDN University chemists and their colleagues from Novosibirsk State University, Novosibirsk Institute of Organic Chemistry and The State Research Center of Virology and Biotechnology VECTOR have obtained a new class of compounds that inhibit the replication of the deadly Hantaan virus that affects blood vessels and internal organs of humans. The resulting substances were 5 times more effective than existing antiviral drugs. The results have been published Bioorganic & Medicinal Chemistry Letters.
The Hantaan virus causes acute haemorrhagic fever with renal syndrome (HFRS). The disease ...
Microgravity in space perturbs human physiology and is detrimental for astronaut health, a fact first realized during early Apollo missions when astronauts experienced inner ear disturbances, heart arrhythmia, low blood pressure, dehydration, and loss of calcium from their bones after their missions.
One of the most striking observations from Apollo missions was that just over half of astronauts became sick with colds or other infections within a week of returning to Earth. Some astronauts have even experienced re-activation of dormant viruses, such as the chickenpox virus. These findings stimulated studies on the effects of weak gravity, or ...
A new study from researchers at Johns Hopkins Bloomberg School of Public Health and colleagues suggests a slowdown in the use of convalescent plasma to treat hospitalized COVID-19 patients led to a higher COVID-19 mortality during a critical period during this past winter's surge.
U.S. hospitals began treating COVID-19 patients with convalescent plasma therapy--which uses antibody-rich blood from recovered COVID-19 patients--in the summer of 2020 when doctors were looking to identify treatments for the emerging disease. By the spring of 2021, doctors in the United States had treated over 500,000 COVID-19 patients with convalescent plasma. The use ...
Male mice more susceptible to stress can pass down their behaviors to offspring via changes in their sperm's genetic code, according to new research published in JNeurosci.
Stressful experiences alter gene expression, which parents can pass down to their offspring. But it was unclear if sperm itself transmits this information, or if behavioral cues between the parents play a larger role.
Cunningham et al. tracked the stress response of male mice after ten days of chronic stress and sorted them into resilient and susceptible groups, based on the severity of their response. The offspring of resilient and control mice showed decreased stress behaviors ...
Researchers at the University of Sydney's Charles Perkins Centre conducted the largest ever study of nutrient interactions by examining the health of mice on 33 different diets containing various combinations of protein to carbs, and different sources of carbohydrate.
They found that a low-protein (10% of dietary energy), high-carbohydrate (70%) diet produced either the healthiest or unhealthiest metabolic outcomes of all 33 diets, depending on the kind of carbs.
When carbs were made up mainly of resistant starch, a form of starch that is resistant to digestion and is fermented by bacteria in the gut, the low protein diet was the healthiest of all diets. When the ...
