(Press-News.org) ADELPHI, Md. -- Multi-domain operations, the Army's future operating concept, requires autonomous agents with learning components to operate alongside the warfighter. New Army research reduces the unpredictability of current training reinforcement learning policies so that they are more practically applicable to physical systems, especially ground robots.
These learning components will permit autonomous agents to reason and adapt to changing battlefield conditions, said Army researcher Dr. Alec Koppel from the U.S. Army Combat Capabilities Development Command, now known as DEVCOM, Army Research Laboratory.
The underlying adaptation and re-planning mechanism consists of reinforcement learning-based policies. Making these policies efficiently obtainable is critical to making the MDO operating concept a reality, he said.
According to Koppel, policy gradient methods in reinforcement learning are the foundation for scalable algorithms for continuous spaces, but existing techniques cannot incorporate broader decision-making goals such as risk sensitivity, safety constraints, exploration and divergence to a prior.
Designing autonomous behaviors when the relationship between dynamics and goals are complex may be addressed with reinforcement learning, which has gained attention recently for solving previously intractable tasks such as strategy games like go, chess and videogames such as Atari and Starcraft II, Koppel said.
Prevailing practice, unfortunately, demands astronomical sample complexity, such as thousands of years of simulated gameplay, he said. This sample complexity renders many common training mechanisms inapplicable to data-starved settings required by MDO context for the Next-Generation Combat Vehicle, or NGCV.
"To facilitate reinforcement learning for MDO and NGCV, training mechanisms must improve sample efficiency and reliability in continuous spaces," Koppel said. "Through the generalization of existing policy search schemes to general utilities, we take a step towards breaking existing sample efficiency barriers of prevailing practice in reinforcement learning."
Koppel and his research team developed new policy search schemes for general utilities, whose sample complexity is also established. They observed that the resulting policy search schemes reduce the volatility of reward accumulation, yield efficient exploration of an unknown domains and a mechanism for incorporating prior experience.
"This research contributes an augmentation of the classical Policy Gradient Theorem in reinforcement learning," Koppel said. "It presents new policy search schemes for general utilities, whose sample complexity is also established. These innovations are impactful to the U.S. Army through their enabling of reinforcement learning objectives beyond the standard cumulative return, such as risk sensitivity, safety constraints, exploration and divergence to a prior."
Notably, in the context of ground robots, he said, data is costly to acquire.
"Reducing the volatility of reward accumulation, ensuring one explores an unknown domain in an efficient manner, or incorporating prior experience, all contribute towards breaking existing sample efficiency barriers of prevailing practice in reinforcement learning by alleviating the amount of random sampling one requires in order to complete policy optimization," Koppel said.
The future of this research is very bright, and Koppel has dedicated his efforts towards making his findings applicable for innovative technology for Soldiers on the battlefield.
"I am optimistic that reinforcement-learning equipped autonomous robots will be able to assist the warfighter in exploration, reconnaissance and risk assessment on the future battlefield," Koppel said. "That this vision is made a reality is essential to what motivates which research problems I dedicate my efforts."
The next step for this research is to incorporate the broader decision-making goals enabled by general utilities in reinforcement learning into multi-agent settings and investigate how interactive settings between reinforcement learning agents give rise to synergistic and antagonistic reasoning among teams.
According to Koppel, the technology that results from this research will be capable of reasoning under uncertainty in team scenarios.
INFORMATION:
This research, conducted in collaboration with Princeton University, University of Alberta and Google Deepmind, was a spotlight talk at NeurIPS 2020, one of the premiere conferences that fosters the exchange of neural information processing systems research in biological, technological, mathematical and theoretical aspects.
Gallium is a highly useful element that has accompanied the advancement of human civilization throughout the 20th century. Gallium is designated as a technologically critical element, as it is essential for the fabrication of semiconductors and transistors. Notably, gallium nitride and related compounds allowed for the discovery of the blue LED, which was the final key in the development of an energy-efficient and long-lasting white LED lighting system. This discovery has led to the awarding of the 2014 Nobel Prize in Physics. It is estimated that up to 98% of the demand for gallium originates from the semiconductor ...
About one in 100 children has a common brain disorder called Chiari 1 malformation, but most of the time such children grow up normally and no one suspects a problem. But in about one in 10 of those children, the condition causes headaches, neck pain, hearing, vision and balance disturbances, or other neurological symptoms.
In some cases, the disorder may run in families, but scientists have understood little about the genetic alterations that contribute to the condition. In new research, scientists at Washington University School of Medicine in St. Louis have shown that Chiari 1 malformation can be caused by variations ...
Use of the delivery system in mouse models results in unprecedented siRNA penetration across the intact blood brain barrier
Technology could offer potential for a variety of human neurological disorders
In the past few decades, researchers have identified biological pathways leading to neurodegenerative diseases and developed promising molecular agents to target them. However, the translation of these findings into clinically approved treatments has progressed at a much slower rate, in part because ...
Fluorescence microscopy is widely used in biochemistry and life sciences because it allows scientists to directly observe cells and certain compounds in and around them. Fluorescent molecules absorb light within a specific wavelength range and then re-emit it at the longer wavelength range. However, the major limitation of conventional fluorescence microscopy techniques is that the results are very difficult to evaluate quantitatively; fluorescence intensity is significantly affected by both experimental conditions and the concentration of the fluorescent substance. Now, a new study by scientists from Japan is set to revolutionize the field of fluorescence lifetime ...
LOS ANGELES (Dec. 30, 2020) -- People who move due to unaffordable housing are at increased risk of failing to receive the medical care they need, according to a new study from Cedars-Sinai and the University of California, Los Angeles. The study, published online in the Journal of General Internal Medicine, concludes that the result could be long-term health problems.
The findings were based on 146,417 adults who responded from 2011 to 2017 to the California Health Interview Survey, the largest such state survey in the U.S. ...
One of the physiopathological characteristics of COVID-19 that has most baffled the scientific and medical community is what is known as "silent hypoxemia" or "happy hypoxia". Patients suffering this phenomenon, the causes of which are still unknown, have severe pneumonia with markedly decreased arterial blood oxygen levels (known as hypoxemia). However, they do not report dyspnea (subjective feeling of shortness of breath) or increased breathing rates, which are usually characteristic symptoms of people with hypoxemia from pneumonia or any other cause.
Patients with "silent hypoxemia" often suffer ...
BOSTON -- Pregnant women may be especially vulnerable to developing more severe cases of COVID-19 following SARS-CoV-2 infection, but little is known about their anti-SARS-CoV-2 immune response or how it may affect their offspring. In a study published in JAMA Network Open, a group led by investigators at Massachusetts General Hospital (MGH) provides new insights that could help improve care for these women and their newborns and emphasizes the need for pregnant women to be considered in vaccine rollout plans.
The study included 127 pregnant women in their third trimester who received care at three Boston hospitals between April 2 and June 13, ...
Pregnant women who are infected with SARS-CoV-2, the virus that causes COVID-19, during the third trimester are unlikely to pass the infection to their newborns, suggests a study funded by the National Institutes of Health. The study followed 127 pregnant women who were admitted to Boston hospitals during the spring of 2020. Among the 64 pregnant women who tested positive for SARS-CoV-2, no newborns tested positive for the virus. NIH support was provided by the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), the National Heart, Lung, and Blood Institute (NHLBI), and the National Institute of Allergy and Infectious ...
Surgery, in addition to treatments like chemotherapy and radiation therapy, may increase the length of survival for metastatic breast cancer patients, according to Penn State College of Medicine and Penn State Cancer Institute researchers. They studied nearly 13,000 stage four breast cancer patients and found that those who had surgery in addition to their other treatments had a survival advantage over those who had other treatments alone.
Stage four breast cancer accounts for 6% of newly diagnosed breast cancer cases. Systemic therapy, which may include treatments like chemotherapy, hormone therapies and immunotherapies, ...
Toronto, CANADA - Attention Deficit Hyperactive Disorder (ADHD) can have negative consequences on mental health into adulthood. A nationally representative Canadian study reported that the lifetime prevalence of suicide attempts was much higher for women who had ADHD (24%) compared to women who had not (3%). Men with ADHD were also more likely to have attempted suicide compared to men without ADHD (9% vs. 2%).
"ADHD casts a very long shadow. Even when we took into account history of mental illness, and the higher levels of poverty and early adversities that adults with ADHD often experience, those with ...
