(Press-News.org) In this Policy Forum, Michael Cohen and colleagues highlight the unique risks presented by a particular class of artificial intelligence (AI) systems: reinforcement learning (RL) agents that plan more effectively than humans over long horizons. “Giving [such] an advanced AI system the objective to maximize its reward and, at some point, withholding reward from it, strongly incentivizes the AI system to take humans out of the loop,” write Cohen and colleagues. This incentive also arises for long-term planning agents (LTPAs) more generally, say the authors, and in ways empirical testing is unlikely to cover. It is thus critical to address extinction risk from these systems, say Cohen et al., and this will require new forms of government intervention. Although governments have expressed some concern about existential risks from AI and taken promising first steps in the U.S. and U.K, in particular, regulatory proposals to date do not adequately address this particular class of risk – losing control of advanced LTPAs. Even empirical safety testing – the prevailing regulatory approach for AI – is likely to be either dangerous or uninformative, for a sufficiently capable LTPA, say the authors. Accordingly, Cohen and colleagues propose that developers not be permitted to build sufficiently capable LTPAs, and that the resources required to build them be subject to stringent controls. When it comes to determining how capable is “sufficiently capable,” for an LTPA, the authors offer insight to guide regulators and policymakers. They note they do not believe that existing AI systems exhibit existentially dangerous capabilities, nor do they exhibit several of the capabilities mentioned in President Biden’s recent executive order on AI, “and it is very difficult to predict when they could.” The authors note that although their proposal for governing LTPAs fills an important gap, “further institutional mechanisms will likely be needed to mitigate the risks posed by advanced artificial agents.”
END
Governance frameworks should address the prospect of AI systems that cannot be safely tested
2024-04-04
ELSE PRESS RELEASES FROM THIS DATE:
Revealed: Mechanical damage during an asthma attack
2024-04-04
In asthma, the tightening of muscles around the bronchi causes damage to the airway by squeezing and destroying epithelial cells, which promotes the airway inflammation and mucus production often associated with an asthma attack, researchers report. The findings suggest that preventing the mechanical damage caused by an asthma attack, rather than treating only its downstream symptoms, could pave the way for therapies that stop the whole asthma inflammatory cycle. Asthma is a common airway disorder affecting more than 300 million people worldwide. Although it is primarily considered an inflammatory disease, a diagnostic feature of asthma is mechanical bronchoconstriction – the ...
Agricultural diversification yields joint environmental and social benefits
2024-04-04
Promoting livestock biodiversity and soil conservation strategies provides both social and environmental benefits, according to a new study. The findings suggest that well-designed polices aimed at incentivizing the adoption of multiple diversification strategies could mitigate simplified agriculture’s negative environmental, health, and social impacts. “Our interdisciplinary analysis spanning a wide array of regions provides convincing evidence that agricultural diversification is a promising win-win strategy for providing social and environmental benefits,” write the authors. Agricultural lands tend to be simplified ecosystems designed ...
Nerve cells not entirely “young at heart”
2024-04-04
After two decades in the United States, Martin Hetzer returned home to Austria in 2023 to become the 2nd President of the Institute of Science and Technology Austria (ISTA). A year into his new role, the molecular biologist remains engaged in the realm of aging research.
Hetzer is fascinated by the biological puzzles surrounding the aging processes in organs like the brain, heart, and pancreas. Most cells comprising these organs are not renewed throughout a human’s entire life span. Nerve cells (neurons) in the human brain, for instance, can be as old as the organism, even ...
Discovery of how limiting damage from an asthma attack could stop disease
2024-04-04
Scientists at King’s College London have discovered a new cause for asthma that sparks hope for treatment that could prevent the life-threatening disease.
Most current asthma treatments stem from the idea that it is an inflammatory disease. Yet, the life-threatening feature of asthma is the attack or the constriction of airways, making breathing difficult. The new study, published today in Science, shows for the first time that many features of an asthma attack—inflammation, mucus secretion, and damage to the airway barrier that ...
Less extensive breast cancer surgery results in fewer swollen arms
2024-04-04
It is possible to leave most of the lymph nodes in the armpit, even if one or two of them have metastases larger than two millimetres? This is shown in a trial enrolling women from five countries, led by researchers at Karolinska Institutet and published in the New England Journal of Medicine. The results open up for gentler surgery for patients with breast cancer.
Breast cancer can spread to the lymph nodes in the armpit. However, tumours found only in the breast and armpit lymph nodes are considered a localized disease, with the goal of curing the patient.
A challenging question for breast cancer surgeons revolves around what should ...
Body mapping links our responses to music with their degree of uncertainty and surprise
2024-04-04
Music holds an important place in human culture, and we’ve all felt the swell of emotion that music can inspire unlike almost anything else. But what is it exactly about music that can bring on such intense sensations in our minds and bodies? A new study reported in the journal iScience on April 4 has insight from studies that systematically examine the way perception of unique musical chords elicits specific bodily sensations and emotions.
“This study reveals the intricate interplay between musical uncertainty, prediction ...
Shaking tiny clusters of brain cells, scientists reveal an overlooked protein’s role in traumatic brain injury
2024-04-04
Clinicians often find limited success in treating patients with traumatic brain injury, a condition long linked to contact sports and military services. A new study, published April 4 in the journal Cell Stem Cell, may offer new clues to better solutions. Scientists found a protein, TDP-43, that appears to drive nerve damage right after injury. Moreover, blocking a certain cell surface protein can correct faulty TDP-43 and curb nerve death in mouse and human cells.
“There’s really nothing out there that can prevent the injury ...
Mistreatment in childbirth is common in the US especially among the disadvantaged
2024-04-04
Lack of respectful maternity care in the U.S. culminating in mistreatment in childbirth is a regular occurrence, according to a new study at Columbia University Mailman School of Public Health. Yet until now experiences of this mistreatment had not been widely documented in the United States. The findings are published in JAMA Network Open.
To estimate the prevalence of mistreatment by care providers in childbirth, the researchers collected survey data from a representative sample of people who had a live birth in 2020 ...
New findings shed light on the expanding universe
2024-04-04
An astrophysicist from The University of Texas at Dallas and his colleagues from the Dark Energy Spectroscopic Instrument (DESI) collaboration are at the forefront of an ambitious experiment to study the expansion of the universe and its acceleration.
Dr. Mustapha Ishak-Boushaki, professor of physics in the School of Natural Sciences and Mathematics (NSM) at UT Dallas, is a member of the DESI collaboration, an international group of more than 900 researchers from over 70 institutions around the world engaged in a multiyear experiment to increase understanding of the ...
Feeling the music
2024-04-04
Music which surprises us can be felt in the heart, while music which matches our expectations can bring feelings of calmness and satisfaction, according to a new study. Researchers played eight short tunes made up of just four chords each to over 500 participants. Each tune had a varied mix of surprising and unsurprising, and certain and uncertain chord progressions. When asked to report how the tunes made them feel and where they were affected, participants’ answers showed that fluctuations in predictions about chord sequences were felt in specific parts of the body, notably the heart and abdomen. Researchers also ...