(Press-News.org) Risk assessment instruments (RAIs) are widely used to inform high-stakes decision making in the criminal justice system and other areas, such as health care and child welfare. These tools typically assume a relation between predictors and outcomes that does not vary with time. But because societies change, this assumption may not hold in all settings, generating what a new study calls cohort bias—a bias resulting from cohort-wide influences not experienced by past or future cohorts.
The study, by researchers at Carnegie Mellon University (CMU), Harvard, and University of Pennsylvania, appears in PNAS. The researchers analyzed criminal histories of individuals in Chicago over a 25 year period, 1995-2020. They found that regardless of predictors, a “machine learning” tool predicting the likelihood of arrest between ages 17 and 24 for older cohorts born in the 1980s substantially overpredicted the likelihood of arrest for younger cohorts born in the mid-1990s. This suggests that RAIs are likely to fail over time for more recent cohorts if social change dynamics are ignored.
“Our aim is to improve the science and use of RAIs in high-stakes contexts,” says Daniel Nagin, professor of public policy and statistics at CMU’s Heinz College who goes on to point out that “Cohort bias can generate inequality in criminal justice that is distinct from racial bias.”
In predicting criminal behavior and justice system involvement, RAIs typically predict risk using a combination of features that measure individual characteristics, family background, and prior criminal history.
Past research has identified early-life, psychosocial, and neighborhood predictors of criminal involvement in adolescence and beyond. These include family instability and poverty, low self-control and growing up in poor neighborhoods. While the association between many of these features and later criminal involvement is well established, whether the predictive strength of these features is constant across historical periods is uncertain.
In this study, researchers examined whether an RAI trained on individual-level features of an older age cohort accurately predicted the likelihood of criminal justice involvement of a younger cohort. Specifically, they studied the performance of models trained to predict arrests in early adulthood across multiple birth cohorts born between 1979 and 1995.
Their data came from the longitudinal Project on Human Development in Chicago Neighborhoods and included more than 1,000 individuals from four age cohorts. Researchers also examined criminal records from the state of Illinois between 1995 and 2020. Among the study’s findings:
Cohort bias was significant: RAIs trained on an older cohort overpredicted the probability of arrest of the younger cohort by up to 89%.
Cohort bias was substantial within racial-ethnic groups (Whites and others, Latinos, and Blacks): This establishes cohort bias as an underappreciated mechanism generating inequality in criminal justice that is distinct from racial bias.
Cohort bias persisted: It lasted even when measures of arrest from immediately before the ages for which the study predicted arrest were included as predictors and even when analysis was limited to high-risk participants.
As these findings suggest, individuals’ future behaviors are not only a function of their stable traits, earlier life circumstances, prior behaviors, and age, but also of ongoing social changes that affect all members of a birth cohort. While the field understands that an algorithm’s performance can degrade over time, the implications of that ongoing change are typically not recognized in RAIs or in the general conceptualization of future risk.
In the context of high-stakes decisions, RAIs have the potential to do harm if not well calibrated. While human judgment is also prone to biases, an RAI can make millions more judgments, amplifying its impact.
“Social science research and policy increasingly rely on predictive RAIs, including those that use machine learning algorithms,” explains Erika Montana, a Ph.D. student in machine learning and public policy at CMU’s Heinz College, who coauthored the study and took the lead role in conducting the analyses. “Our findings show that the relations between risk factors and future arrest are not stable over time. As a result, prediction models that rely on these risk factors are prone to systematic and substantial error.”
The study was funded by the Office of Juvenile Justice and Delinquency Prevention and managed by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice.
Tools to assess crime risk for young cohorts are likely to fail over time if they ignore social change
ELSE PRESS RELEASES FROM THIS DATE:
Direct air capture technology licensed to Knoxville-based Holocene
An innovative and sustainable chemistry developed at the Department of Energy’s Oak Ridge National Laboratory for capturing carbon dioxide from air has been licensed to Holocene, a Knoxville-based startup focused on designing and building plants that remove carbon dioxide from atmospheric air. “ORNL is tackling climate change by developing numerous technologies that reduce or eliminate emissions,” said Susan Hubbard, ORNL deputy for science and technology. “But with billions ...
Fetal exposure to PCBs affects hearing health later in life
Music, mice, and microscopic imaging combine to provide new insight into the effects of environmental chemicals on hearing loss. Researchers at the Beckman Institute for Advanced Science and Technology found that early exposure to an environmental chemical called polychlorinated biphenyls, or PCBs, made it more difficult for mice to recover from sound-related trauma sustained later in life. Their paper appears in the Journal of Neuroscience. PCBs are carcinogenic compounds formerly used in industrial and consumer products. Although they were banned in the United States in 1979 and haven’t seen industrial use in decades, their highly ...
Quantum computers are better at guessing, new study demonstrates
Daniel Lidar, the Viterbi Professor of Engineering at USC and Director of the USC Center for Quantum Information Science & Technology, and first author Dr. Bibek Pokharel, a Research Scientist at IBM Quantum, achieved this quantum speedup advantage in the context of a “bitstring guessing game.” They managed strings up to 26 bits long, significantly larger than previously possible, by effectively suppressing errors typically seen at this scale. (A bit is a binary number that is either zero or one). Quantum computers promise to solve certain problems with an advantage that increases as the ...
New discoveries about where atherosclerotic plaques rupture can lead to preventive treatments
A common cause of myocardial infarction and stroke is the rupture of atherosclerotic plaques. The exact location of plaque ruptures has previously been unknown, but now researchers at Lund University have mapped this. The research team has also identified an enzyme, a marker, that they hope will help predict who is at risk of having a myocardial infarction or a stroke due to a ruptured atherosclerotic plaque. In atherosclerosis , fat is accumulated in the artery walls creating atherosclerotic ...
Webb Space Telescope detects universe’s most distant complex organic molecules
CHAMPAIGN, Ill. — Researchers have detected complex organic molecules in a galaxy more than 12 billion light-years away from Earth – the most distant galaxy in which these molecules are now known to exist. Thanks to the capabilities of the recently launched James Webb Space Telescope and careful analyses from the research team, a new study lends critical insight into the complex chemical interactions that occur in the first galaxies in the early universe. University of Illinois Urbana-Champaign astronomy and physics professor Joaquin Vieira and graduate student Kedar Phadke collaborated with researchers at Texas A&M ...
Zoonoses: Welcome to Professor Fernando Rosado Spilki, the new Executive Editor-in-Chief
As the Co-Editors-in-Chief of Zoonoses, Dr. Lynn Soong (University of Texas Medical Branch, TX, USA) and Dr. Xiaoping Dong (Chinese Center for Disease Control & Prevention, Beijing, China) extend a warm welcome to Dr. Fernando Rosado Spilki, new Executive Editor-in-Chief (Vector Biology/Epidemiology) of Zoonoses. Dr. Spilki is currently a Professor in the Institute of Health Sciences at Feevale University, Novo Hamburgo, Brazil. He received B.S. in Veterinary Medicine (2001), M.S. in Veterinary Sciences/Animal Virology (2004) both from the Universidade Federal do Rio Grande do Sul, and his Ph.D. in Genetics & Molecular ...
Fungi stores a third of carbon from fossil fuel emissions and could be essential to reaching net zero, new study reveals
Fungi stores a third of carbon from fossil fuel emissions and could be essential to reaching net zero, new study reveals Mycorrhizal fungi are responsible for holding up to 36 per cent of yearly global fossil fuel emissions below ground - more than China emits each year The fungi make up a vast underground network all over the planet underneath grasslands and forests, as well as roads, gardens, and houses on every continent on Earth It is not only crucial to storing carbon and keeping the planet cooler, but are also essential to global biodiversity Researchers ...
Climate justice: Global North owes $170 trillion for excessive CO2 emissions, says study
Industrialised nations responsible for excessive levels of carbon dioxide emissions could be liable to pay a total of $170 trillion in compensation or reparations by 2050 to ensure climate change targets are met, say researchers. This money, which amounts to nearly $6 trillion per year or about 7% of annual global Gross Domestic Product (GDP), would be distributed as compensation to low-emitting countries that must decarbonise their economies far more rapidly than would otherwise be required. Financial redress for the losses and damages that climate-vulnerable countries face due to the excessive CO2 emissions of others is seen as an increasingly important ...
BRIDGEcereal: Self-teaching web app improves speed, accuracy of classifying DNA variations among cereal varieties
Kim Kaplan 301-588-5314 Kim.Kaplan@usda.gov BRIDGEcereal: Self-Teaching Web App Improves Speed, Accuracy of Classifying DNA Variations Among Cereal Varieties PULLMAN, WA, June 5, 2023—Agricultural Research Service and Washington State University scientists have developed an innovative web app called BRIDGEcereal [https://bridgecereal.scinet.usda.gov/] that can quickly and accurately analyze the vast amount of genomic data now available for cereal crops and organize the material into intuitive charts that identify patterns locating genes of interest. With the rapid advancements in ...
Availability of LGBTQ mental health services for youth
About The Study: This survey study found that 28% of youth-serving U.S. mental health facilities offered LGBTQ-specific mental health services in 2020. Although some states had relatively high levels of LGBTQ service availability as a percentage of facilities, many of these states had few facilities available to children per capita. Public mental health facilities were less likely to offer LGBTQ-specific mental health services, a concern given that the cost of care is a barrier to services. The findings suggest a need to expand availability of LGBTQ services ...