UC Riverside pioneers way to remove private data from AI models

(Press-News.org) A team of computer scientists at UC Riverside has developed a method to erase private and copyrighted data from artificial intelligence models—without needing access to the original training data.

This advance, detailed in a paper presented in July at the International Conference on Machine Learning in Vancouver, Canada, addresses a rising global concern about personal and copyrighted materials remaining in AI models indefinitely—and thus accessible to model users—despite efforts by the original creators to delete or guard their information with paywalls and passwords.

The UCR innovation compels AI models to “forget” selected information while maintaining the models’ functionality with the remaining data. It’s a significant advancement that can amend models without having to re-make them with the voluminous original training data, which is costly and energy-intensive. The approach also enables the removal of private information from AI models even when the original training data is no longer available.

“In real-world situations, you can’t always go back and get the original data,” said Ümit Yiğit Başaran, a UCR electrical and computer engineering doctoral student and lead author of the study. “We’ve created a certified framework that works even when that data is no longer available.”

The need is pressing. Tech companies face new privacy laws, such as the European Union’s General Data Protection Regulation and California’s Consumer Privacy Act, which govern the security of personal data embedded in large-scale machine learning systems.

Moreover, The New York Times is suing OpenAI and Microsoft over the use of its many copyrighted articles to train Generative Pre-trained Transformer, or GPT, models.

AI models “learn” the patterns of words from vast amount texts scraped from the Internet. When queried, the models predict the most likely word combinations, generating natural-language responses to user prompts. Sometimes they generate near-verbatim reproductions of the training texts, allowing users to bypass the paywalls of the content creators.

The UC Riverside research team—comprising Başaran, professor Amit Roy-Chowdhury, and assistant professor Başak Güler—developed what they call a “source-free certified unlearning” method. The technique allows AI developers to remove targeted data by using a substitute, or “surrogate,” dataset that statistically resembles the original data.

The system adjusts model parameters and adds carefully calibrated random noise to ensure the targeted information is erased and cannot be reconstructed.

Their framework builds on a concept in AI optimization that efficiently approximates how a model would change if it had been retrained from scratch. The UCR team enhanced this approach with a new noise-calibration mechanism that compensates for discrepancies between the original and surrogate datasets. 

The researchers validated their method using both synthetic and real-world datasets and found it provided privacy guarantees close to those achieved with full retraining—yet required far less computing power.

The current work applies to simpler models—still widely used—but could eventually scale to complex systems like ChatGPT, said Roy-Chowdhury, the co-director of UCR’s Riverside Artificial Intelligence Research and Education (RAISE) Institute and a professor in the Marlan and Rosemary Bourns College of Engineering. 

Beyond regulatory compliance, the technique holds promise for media organizations, medical institutions, and others handling sensitive data embedded in AI models, the researchers said. It also could empower people to demand the removal of personal or copyrighted content from AI systems.

“People deserve to know their data can be erased from machine learning models—not just in theory, but in provable, practical ways,” Güler said.

The team’s next steps involve refining the method to work with more complex model types and datasets and building tools to make the technology accessible to AI developers worldwide.

The title of the paper is “A Certified Unlearning Approach without Access to Source Data.” It was done in collaboration with Sk Miraj Ahmed, a computational science research associate at the Brookhaven National Laboratory in Upton, NY who received his doctoral degree at UCR. Both Roy-Chowdary and Güler are faculty members in the Department of Electrical and Computer Engineering with secondary appointments in the Department of Computer Science and Engineering.

END