(Press-News.org) Recently, model-based reinforcement learning has been considered a crucial approach to applying reinforcement learning in the physical world, primarily due to its efficient utilization of samples. However, the supervised learned model, which generates rollouts for policy optimization, leads to compounding errors and hinders policy performance. To address this problem, the research team led by Yang YU published their new research on 15 August 2024 in Frontiers of Computer Science co-published by Higher Education Press and Springer Nature.
The team proposed a novel model-based learning approach that unifies the objectives of model learning and policy learning. By directly maximizing the policy’s performance in the real world, this research proposes the Model Gradient algorithm (MG). Compared with existing model-based methods, this approach achieves both higher sample efficiency and better performance.
This research identifies the limitation of current supervised-learned model-based reinforcement learning methods, where the model inaccuracy leads to compounding error. The authors suggest addressing the problem by modifying model learning objective. A supervised model learning approach may not be designed to assist policy learning in achieving better performance because the objective does not align with the ultimate goal of reinforcement learning, i.e., maximizing the real-world policy performance. Therefore, this research aims to unify the objective of model learning and policy learning starting with policy gradient. By maximizing the real-world performance of the policy learned in the model, this research derives the gradient of model, which represents the direction of policy improvement with the form of enhancing the similarity between the policy gradient in the real environment and that in the model. By adopting this model update approach, the authors develops a novel model-based reinforcement learning algorithm called the Model Gradient algorithm (MG).
Experimental results demonstrate that MG outperforms other model-based reinforcement learning baselines with supervised model fitting in multiple continuous control tasks. MG especially exhibits stable performance in sparse reward tasks, even when compared to state-of-the-art Dyna-style model-based reinforcement learning methods with short-horizon rollouts.
For the future work, this research considers extending this form to more policy optimization such as off-policy methods.
DOI: 10.1007/s11704-023-3150-5
END
A unified objective for dynamics model and policy learning in model-based reinforcement learning
2024-09-04
[Attachments] See images for this press release:
ELSE PRESS RELEASES FROM THIS DATE:
How to solve the challenges faced by the carbon sequestration function of Chinese plantations in the future?
2024-09-04
Since the first industrial revolution, the rapid development of the human economy and society has directly exacerbated the process of CO2 emission from human activities such as fossil fuel combustion, industrial processes, agriculture, and land use activities. With the continuous increase of global greenhouse gas concentration dominated by CO2, the greenhouse effect is becoming more and more obvious, and the trend of global warming is becoming more and more serious. To cope with the continuous warming of the global climate and mitigate ...
Sleep-deprived, cyberbullied teenagers addicted to smartphones now a common phenomenon
2024-09-04
Combine cyberbullying, smartphone use, lack of sleep and poor mental health, and you have the perfect storm for a teenage meltdown.
Australian researchers have polled more than 50,000 primary and secondary school students aged 7-19 years about the link between their sleep and nighttime phone habits, experience of cyberbullying and stress levels.
Researchers from the Behaviour-Brain-Body Research Centre at the University of South Australia found that across all genders and age groups, phone use overnight not only robbed children of sleep, but it also had a negative impact on their mental health, ...
Auburn researchers show novel drug rescues memory loss in Alzheimer’s mouse model
2024-09-04
AUBURN, AL — In a recent development in Alzheimer's disease research, Auburn University scientists have studied a new drug, troriluzole, that can prevent brain changes leading to memory loss and cognitive decline in a mouse model of the disease. This study, recently published in the Journal of Neurochemistry, is the first to show how troriluzole can target early-stage alterations associated with Alzheimer’s, providing new hope for potential treatments.
Dr. Miranda Reed, a Professor in the department ...
Study at Pennington Biomedical Research Center to evaluate THC, CBD benefits for dementia-related agitation
2024-09-04
Pennington Biomedical Research Center’s Dr. Jeff Keller is evaluating the potential for delta-9-tetrahydrocannabinol, or THC, and cannabidiol, or CBD, to reduce the behaviors indicating agitation, distress or anxiety in patients with Alzheimer’s disease or other forms of dementia. The study is designed for hospice-eligible patients who are either receiving hospice care or who are eligible for hospice, and who are exhibiting agitation concurrently with a diagnosis of dementia. There are currently no FDA-approved medications to treat agitation at the end-of-life stages in dementia patients.
The “Life’s End Benefits of Cannabidiol and ...
Illinois scientists to test modernized genetic model for optimized crop breeding
2024-09-04
URBANA, Ill. — The National Science Foundation (NSF) has funded University of Illinois Urbana-Champaign research that aims to connect the dots between quantitative and molecular genetics and improve crop breeding.
The four-year, $795,000 grant investigates new theories on how genetics influence complex crop traits, such as yield or grain quality. These traits are controlled by lots of different genes — sometimes hundreds or thousands — which makes untangling their contributions difficult. Crop breeders use a host of advanced genetic tools to predict and ...
Adolescent glioma subtype responds to CDK4/6 inhibitor
2024-09-04
Boston – CDK4/6 inhibitors, which are already FDA approved for the treatment of other forms of cancer, show early signs of promise in the treatment of a subtype of pediatric high-grade glioma, according to new research from Dana-Farber Cancer Institute and the Institute of Cancer Research in London. Treatment of a patient with a second relapse of this glioma subtype and no other treatment options resulted in 18 months of progression-free survival.
“We are finally starting to see more targeted therapies come out for different forms of brain cancer,” says senior author Mariella Filbin, MD, PhD, co-director ...
Study highlights importance of social media influencers in information dissemination during mpox outbreak
2024-09-04
A recent study shows social media influencers are more important than previously thought when it comes to getting out vital information in a crisis.
The study suggested partnerships that could improve public communication between governments, non-profits and social media influencers during crises. The study, conducted by UF/IFAS assistant professor Kimberly Kay Wiley, a researcher in the family, youth and community sciences department, and Bridgewater State University associate professor Seth Meyer, shows how these groups can collaborate to effectively disseminate information and manage public health emergencies on social media.
“In ...
Ability to cope well with adversity in older age linked to lower death risk
2024-09-04
The ability to cope well with, and adapt to, challenging life circumstances and events in older age is linked to a lower risk of death, suggests a large nationally representative study, published in the open access journal BMJ Mental Health.
The findings underscore the importance of efforts to bolster mental resilience, conclude the researchers.
The available evidence suggests that mental resilience is a dynamic and active process influenced by various factors, including sex, hormones, and the genes regulating ...
Number of general practices shrinking but patient lists ballooning in England
2024-09-04
Over the past decade the number of NHS general practices in England has shrunk by 20%, but patient list sizes have expanded by 40% to just under 10,000, on average, finds an analysis of three national primary care datasets, published in the open access journal BMJ Open.
And while the total NHS general practice workforce grew 20% between 2015 and 2022, as a result of increases in admin staff and other practitioners, the number of GPs per 1000 patients fell by 15% over the same period, when accounting for working hours, the analysis shows.
Major structural and organisational changes have taken place in general practice in England over the past decade,but it’s difficult ...
Women, Black people, and disadvantaged less likely to get heart surgery in England
2024-09-04
Women, people of Black ethnicity, and those from low income households in England are less likely to be offered heart surgery than men, White people, and those who are affluent, finds research published online in the journal Heart.
And when they do have these procedures, they are more likely to die within a year, prompting the researchers to call for prompt action to tackle these health inequalities.
Cardiac surgery is one of the costliest ways of treating cardiovascular disease, with around 28,000 adults a year in the UK undergoing the procedure, note the researchers. While previously published research shows that gender, ethnicity, and social/economic deprivation can affect ...