former NIJ director proposes new framework to enhance rigor, impact of criminal justice intervention evaluations

(Press-News.org) Experimental research is fundamental to criminology, but reaching consensus on rigorous evidence and using that evidence to determine what works remains an ongoing challenge to the field. In a new article, the former director of the National Institute of Justice (NIJ) within the U.S. Department of Justice’s Office of Justice programs, proposes a framework to situate both the role of rigorous evaluation and its results in a more robust understanding of the effectiveness of social programs. According to her new framework, the more deliberate the implementation of a social program, the more likely it will yield its intended impact.

“Deliberate implementation can enhance our understanding of what constitutes an effective intervention and what factors explain each outcome,” says Nancy La Vigne, a criminologist who previously directed the NIJ and authored the article, which is published in the Journal of Experimental Criminology. La Vigne is an expert whose work is promoted by the NCJA Crime and Justice Research Alliance, which is funded by the National Criminal Justice Association.

“This information is essential in guiding policymaking and program investments by government agencies and philanthropy,” adds La Vigne. “It yields more credible findings for decision makers, boosts the likelihood of successful replication, and helps bridge the evidence-to-action divide.”

The impetus for the current debate in criminology over the role of rigorous evaluation is a 2023 article by researcher Megan Stevenson in the Boston University Law Review. Stevenson argued that rigorous evaluations, which she defines solely as randomized controlled trials (RCTs), frequently yield null results and that interventions previously found to be effective often fail to replicate. Stevenson suggested that the null results are because the criminal justice system requires systemic changes, which are unable to be captured by RCTs. Reaction by the criminology community fueled a robust debate over the role, definition, and value of rigorous research in public forums.

In her article, La Vigne notes that absent from the debate was any recognition of the importance of researchers and programmatic partners having a shared understanding of the local context and quality of criminal justice interventions. She introduces the Titanium Law of Evaluation, which emphasizes the importance of deliberate implementation, as defined by adherence to model fidelity and application to local context, elements that are often inadequately addressed in evaluations of criminal justice interventions.

La Vigne advocates for using an ongoing feedback loop, based on psychologist Kurt Lewin’s Action Research Model, in which researchers routinely share findings about opportunities to improve implementation fidelity that are often necessitated by local contexts and other factors. Sharing this information during the evaluation enables program implementers to make midcourse corrections to improve the robustness of the intervention and its likelihood of yielding desired impacts.

She uses two case studies—one of the Honest Opportunity Probation with Enforcement program in Hawaii, designed to implement swift and certain sanctions for people on probation who have substance use disorders, and the other of Center for Employment Opportunities, one of the largest employment programs for people re-entering society after incarceration in the United States—to illustrate the importance of measuring implementation fidelity, making mid-course corrections, and adapting programs to local contexts to foster desired outcomes.

La Vigne concludes that the Titanium Law emphasizes the need for a nuanced understanding of implementation processes, promoting collaborative evaluations that consider both fidelity and local context—both key components of implementation science. She recommends integrating these principles into criminology curricula and fostering partnerships between researchers and practitioners to improve evaluation outcomes.

“The Titanium Law of Evaluation elevates a vital but often overlooked component of current evaluation practice: focusing on adapting program components to local contexts and ensuring implementation fidelity, rather than focusing solely on the theory that underlies an intervention,” explains La Vigne. “By doing so, adherence to the Titanium Law can promote both successful implementation as well as successful replication in different settings, all in the interests of public safety.”

END