(Press-News.org) Computer scientists often encounter problems relevant to real-life scenarios. For instance, "multiagent problems," a category characterized by multi-stage decision-making by multiple decision makers or "agents," has relevant applications in search-and-rescue missions, firefighting, and emergency response.
Multiagent problems are often solved using a machine learning technique known as "reinforcement learning" (RL), which concerns itself with how intelligent agents make decisions in an environment unfamiliar to them. An approach usually adopted in such an endeavor is policy iteration (PI), which starts off with a "base policy" and then improves on it to generate a "rollout policy" (with the process of generation called a "rollout"). Rollout is simple, reliable, and well-suited for an on-line, model-free implementation.
There is, however, a serious issue. "In a standard rollout algorithm, the amount of total computation grows exponentially with the number of agents. This can make the computations prohibitively expensive even for a modest number of agents," explains Prof. Dimitri Bertsekas from Massachusetts Institute of Technology and Arizona State University, USA, who studies large-scale computation and optimization of communication and control.
In essence, PI is simply a repeated application of rollout, in which the rollout policy at each iteration becomes the base policy for the next iteration. Usually, in a standard multiagent rollout policy, all agents are allowed to influence the rollout algorithm at once ("all-agents-at-once" policy). Now, in a new study published in the IEEE/CAA Journal of Automatica Sinica, Prof. Bertsekas has come up with an approach that might be a game changer.
In his paper, Prof. Bertsekas focused on applying PI to problems with a multiple-component control, each component selected by a different agent. He assumed that all agents had perfect state information and shared it among themselves. He then reformulated the problem by trading off control space complexity with state space complexity. Additionally, instead of an all-agents-at-once policy, he adopted an "agent-by-agent" policy wherein only one agent was allowed to execute a rollout algorithm at a time, with coordinating information provided by the other agents.
The result was impressive. Instead of an exponentially growing complexity, Prof. Bertsekas found only a linear growth in computation with the number of agents, leading to a dramatic reduction in the computation cost. Moreover, the computational simplification did not sacrifice the quality of the improved policy, performing at par with the standard rollout algorithm.
Prof. Bertsekas then explored exact and approximate PI algorithms using the new version of agent-by-agent policy improvement and repeated application of rollout. For highly complex problems, he explored the use of neural networks to encode the successive rollout policies, and to precompute signaling policies that coordinate the parallel computations of different agents.
Overall, Prof. Bertsekas is optimistic about his findings and future prospects of his approach. "The idea of agent-by-agent rollout can be applied to challenging multidimensional control problems, as well as deterministic discrete/combinatorial optimization problems, involving constraints that couple the controls of different stages," he observes. He has published two books on RL, one of which, titled "Rollout, Policy Iteration, and Distributed Reinforcement Learning" soon to be published by Tsinghua Press, China, deals with the subject of his study in detail.
The new approach to multiagent systems might very well revolutionize how complex sequential decision problems are solved.
INFORMATION:
Reference
D. Bertsekas, "Multiagent reinforcement learning: Rollout and policy iteration," IEEE/CAA J. Autom. Sinica, vol. 8, no. 2, pp. 249-272, Feb. 2021.
http://www.ieee-jas.net/en/article/doi/10.1109/JAS.2021.1003814
https://ieeexplore.ieee.org/document/9317713
IEEE/CAA Journal of Automatica Sinica aims to publish high-quality, high-interest, far-reaching research achievements globally, and provide an international forum for the presentation of original ideas and recent results related to all aspects of automation.
The first Impact Factor of IEEE/CAA Journal of Automatica Sinica is 5.129, ranking among Top 17% (11/63, SCI Q1) in the category of Automation & Control Systems, according to the latest Journal Citation Reports released by Clarivate Analytics in 2020. In addition, its latest CiteScore is 8.3, and has entered Q1 in all three categories it belongs to (Information System, Control and Systems Engineering, Artificial Intelligence) since 2018.
Why publish with us: Fast and high quality peer review; Simple and effective online submission system; Widest possible global dissemination of your research; Indexed in SCIE, EI, IEEE, Scopus, Inspec. JAS papers can be found at http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=6570654 or http://www.ieee-jas.net
CHAMPAIGN, Ill. -- For more than a decade, ecologists have been warning of a downward trend in bumble bee populations across North America, with habitat destruction a primary culprit in those losses. While efforts to preserve wild bees in the Midwest often focus on restoring native flowers to prairies, a new Illinois-based study finds evidence of a steady decline in the availability of springtime flowers in wooded landscapes.
The scarcity of early season flowers in forests - a primary food source for bumble bees at this time of year - likely endangers ...
WASHINGTON - The Society for Cardiovascular Angiography and Interventions (SCAI) has released an expert consensus statement providing cardiologists, cath lab directors, and hospital leadership guidance for contemporary cath lab standards. The document, "SCAI Expert Consensus Update on Best Practices in the Cardiac Catheterization Laboratory" will be presented today at the SCAI 2021 Virtual Scientific Sessions with simultaneous publication in Catheterization & Cardiovascular Interventions. The statement has been endorsed by the American College of Cardiology Clinical Policy Approval Committee, American Heart Association, ...
Cells breathe, to an extent, exchanging gases, taking in energy sources from the environment and processing it. Now, researchers from Tohoku University in Japan have shone a light on the process in a new way.
Their demonstrated visualization method in model systems was made available online on March 12th in Biosensors and Bioelectronics, ahead of the June print edition.
The researchers used spheroids - cultured cells within a close-to-natural environment - to mimic a biological tissue using mesenchymal stem cells (MSCs). Due to MCSs' ability to self-renew and differentiate into various ...
Our bodies and behaviors often seem to have rhythms of their own. Why do we go to the bathroom at the same time every day? Why do we feel off if we can't go to sleep at the right time? Circadian rhythms are a behind-the-scenes force that shape many of our behaviors and our health. Michihiro Mieda and his team at Kanazawa University in Japan are researching how the brain's circadian rhythm control center regulates behavior.
Termed the superchiasmatic nucleus, or SCN, the control center contains many types of neurons that transmit signals using the molecule GABA, but little is known about how each type contributes to our bodily rhythms. In their newest study, the researchers focused on GABA neurons that produce arginine vasopressin, a hormone that regulates ...
WASHINGTON, D.C, (April 28, 2021) - A new study reveals the use of a potassium ferrate hemostatic patch (PFHP) reduces the time to hemostasis for patients receiving cardiac catherization. The findings indicate a faster approach to removing the compression band used during the procedure, without compromising safety. Positive results of the STAT2 trial follow an initial pilot study and are being presented as late-breaking clinical science at the Society for Cardiovascular Angiography and Interventions (SCAI) 2021 Virtual Scientific Sessions.
Cardiac ...
WASHINGTON, D.C, (April 28, 2021) - A retrospective analysis of risk factors for coronary artery disease (CAD) in young African American patients is being presented today at the Society for Cardiovascular Angiography and Interventions (SCAI) 2021 Virtual Scientific Sessions. The findings reveal this specific patient segment, African-Americans under age 45, experiences greater CAD risk factors related to smoking, drug and alcohol abuse, HIV as well as mental health conditions including anxiety and depression.
CAD is the most common type of heart disease, with high blood pressure, obstructive sleep apnea and diabetes among traditional risk factors. African Americans are disproportionally ...
WASHINGTON, D.C., (April 28, 2021) - Results from a retrospective observational study, presented today at Society for Cardiovascular Angiography and Interventions (SCAI) 2021 Virtual Scientific Sessions, reveal a 70% decline in the number of patients presenting with acute myocardial infarction (AMI) during April 2020 compared to April 2019. While the number of patients with AMI seeking care at hospitals dropped during the pandemic, those that did receive care experienced more severe symptoms because of delays in patients seeking emergency services.
AMI, commonly recognized as a heart attack, is responsible for more than one million deaths in the U.S. every year. For the best patient outcomes, seeking care within the ...
During the summer of 2020, especially June and July, periods of extreme heavy rainfall occurred in China's Yangtze River Valley (YRV). These rain events caused the severest floods for the region since the summer of 1998. Despite this, the 2020 western North Pacific (WNP) typhoon season started slowly, but eventually produced 23 named tropical cyclones, still slightly below 27, the WNP seasonal average. As summer transitioned to winter, three severe cold surges swept most parts of China during late 2020 and early 2021, prompting the National Meteorological Center to issue its highest cold surge warning alert ...
The research by RMIT University looked at the ramifications on the stock market following Google's withdrawal from mainland China in 2010.
It found access to unbiased information about companies' performance - aided by unrestricted internet search results - led to investors making more informed decisions.
On the flip side, search results manipulated to show overly positive information led to stocks for those companies being overvalued temporarily, increasing the stock market crash risk by 19%.
The study has been published in the Journal of Financial Economics.
Lead researcher Dr Gaoping Zheng, a lecturer in finance at RMIT, said the study showed search results influenced decisions, a challenge to previous thinking that they merely justified people's existing ideas.
"Until ...
In almost ten per cent of myocardial infarctions, no obvious cause in the coronary artery can be found. Some of the patients are diagnosed with broken-heart syndrome, while others are left without a diagnosis. A new study from Karolinska Institutet in Sweden suggests that early magnetic resonance (MR) imaging of the heart can greatly increase the rate of diagnosis. The study has been published in the journal JACC Cardiovascular Imaging.
Myocardial infarction is one of the most common diseases in the West, and is usually caused by a blood clot that blocks the coronary artery on the heart's surface. However, in up to ten per cent of all myocardial infarctions, no obvious cause ...
