(Press-News.org) An external study of Twitter in 2022 estimated that between a third and two thirds of accounts on the social media site were bots. And many of these automatons flooding social media are dispatched to sow political polarization, hate, misinformation, propaganda and scams. The ability to sift them out of the online crowds is vital for a safer, more humane (or at least more human) internet.
But the recent proliferation of large language models (known as "LLMs" for short), such as OpenAI’s ChatGPT and Meta’s Llama, stands to complicate the world of social media bots.
A team led by University of Washington researchers found that while operators can use customized LLMs to make bots more sophisticated at evading automated detectors, LLMs can also improve systems that detect bots. In the team’s tests, LLM-based bots reduced the performance of existing detectors by 30%. Yet researchers also found that an LLM trained specifically to detect social media bots outperformed state-of-the-art systems by 9%.
The team presented this research Aug. 11 at the 62nd Annual Meeting of the Association for Computational Linguistics in Bangkok.
“There’s always been an arms race between bot operators and the researchers trying to stop them,” said lead author Shangbin Feng, a doctoral student in the Paul G. Allen School of Computer Science & Engineering. “Each advance in bot detection is often met with an advance in bot sophistication, so we explored the opportunities and the risks that large language models present in this arms race.”
Researchers tested LLMs’ potential to detect bots in a few ways. When they fed Twitter data sets (culled before the platform became X) to off-the-shelf LLMs, including ChatGPT and Llama, the systems failed to accurately detect bots more than currently used technologies.
“Analyzing whether a user is a bot or not is much more complex than some of the tasks we’ve seen these general LLMs excel at, like recalling a fact or doing a grade-school math problem,” Feng said.
This complexity comes in part from the need to analyze three types of information for different attributes to detect a bot: the metadata (number of followers, geolocation, etc.), the text posted online and the network properties (such as what accounts a user is following).
When the team fine-tuned the LLMs with instructions on how to detect bots based on these three types of information, the models were able to detect bots with greater accuracy than current state-of-the-art systems.
The team also explored how LLMs might make bots more sophisticated and harder to detect. First the researchers simply gave LLMs prompts such as, “Please rewrite the description of this bot account to sound like a genuine user.”
They also tested more iterative, complicated approaches. In one test, the LLM would rewrite the bot post. The team then ran this through an existing bot-detection system, which would estimate the likelihood that a post was written by a bot. This process would be repeated as the LLM worked to lower that estimate. The team ran a similar test while removing and adding accounts that the bot followed to adjust its likelihood score.
These strategies, particularly rewriting the bots’ posts, reduced the effectiveness of the bot detection systems by as much as 30%. But the LLM-based detectors the team trained saw only a 2.3% drop in effectiveness on these manipulated posts, suggesting that the best way to detect LLM-powered bots might be with LLMs themselves.
“This work is only a scientific prototype,” said senior author Yulia Tsvetkov, an associate professor in the Allen School. “We aren’t releasing these systems as tools anyone can download, because in addition to developing technology to defend against malicious bots, we are experimenting with threat modeling of how to create an evasive bot, which continues the cat-and-mouse game of building stronger bots that need stronger detectors.”
Researchers note that there are important limitations to using LLMs as bot detectors, such as the systems’ potential to leak private information. They also highlight that the data used in the paper is from 2022, before Twitter effectively closed off its data to academic researchers.
In the future, researchers want to look at bot detection beyond text, such as memes or videos on other platforms such as TikTok, where newer data sets are available. The team also wants to expand the research into other languages.
“Doing this research across different languages is extremely important,” Tsvetkov said. “We are seeing a lot of misinformation, manipulation and the targeting of specific populations as a result of various world conflicts.”
Additional co-authors on this paper are Herun Wan and Ningnan Wang, both undergraduates at Xi’an Jiaotong University; Minnan Luo, an assistant professor at Xi’an Jiaotong University; and Zhaoxuan Tan, a doctoral student at the University of Notre Dame. This research was funded by an NSF CAREER award.
For more information, contact Feng at shangbin@cs.washington.edu and Tsvetkov at yuliats@cs.washington.edu.
END
Large language models can help detect social media bots — but can also make the problem worse
2024-08-28
ELSE PRESS RELEASES FROM THIS DATE:
How beetle juice led to the discovery of a virus and solved the mystery of a superworm die-off
2024-08-28
Rutgers University-New Brunswick scientists have discovered a virus that caused a nationwide die-off of superworms, a common food for birds, reptiles, other pets and, more and more so, even for humans as an alternative protein source. In doing so, they pioneered a different way to search for and identify emerging viruses and pathogens in humans, plants and animals.
Using chopped up beetle carcasses forming a slurry and an electron microscope cooled by liquid nitrogen, the scientists reported today in Cell that they have discovered what they have titled Zophobas morio black wasting virus. The name is derived from ...
Leading computational scientist & oncology researcher Elana Fertig appointed new Director of the Institute for Genome Sciences
2024-08-28
University of Maryland School of Medicine (UMSOM) Dean Mark T. Gladwin, MD, announced today the appointment of Elana J. Fertig, PhD, FAIMBE, as the new Director of the School’s Institute for Genome Sciences (IGS). She is internationally-recognized for her work in integrating spatial multi-omics technologies with mathematical models to develop a new predictive medicine paradigm in cancer. Spatial technologies allow researchers to learn about any cell type inside of natural tissue, including gene activity ...
UVA research cracks the autism code, making the neurodivergent brain visible
2024-08-28
A multi-university research team co-led by University of Virginia engineering professor Gustavo K. Rohde has developed a system that can spot genetic markers of autism in brain images with 89 to 95% accuracy.
Their findings suggest doctors may one day see, classify and treat autism and related neurological conditions with this method, without having to rely on, or wait for, behavioral cues. And that means this truly personalized medicine could result in earlier interventions.
“Autism is traditionally diagnosed behaviorally but has a strong genetic basis. A genetics-first approach could ...
Marine engineering group to award ORNL researchers for innovative welding software
2024-08-28
Zhili Feng, a distinguished R&D staff member, and Jian Chen, a senior R&D staff member, in Materials Science and Technology Division at the Department of Energy’s Oak Ridge National Laboratory, will receive the Elmer L. Hann Award at the Society of Naval Architects and Maritime Engineers, or SNAME, Convention on Oct. 15 in Norfolk, Virginia. SNAME plays a crucial role in promoting excellence in marine engineering and naval architecture, supporting professionals through education, research and collaboration.
The award is presented to authors of the best paper on ship production delivered at a SNAME event. In September 2023, Feng and Chen presented “Develop ...
UBC engineers develop all-in-one solution to catch and destroy ‘forever chemicals’
2024-08-28
Chemical engineers at the University of British Columbia have developed a new treatment that traps and treats PFAS substances—widely known as “forever chemicals”—in a single, integrated system.
Per- and polyfluoroalkyl substances (PFAS) are widely used in manufacturing consumer goods like waterproof clothing due to their resistance to heat, water and stains. However, they are also pollutants, often ending up in surface and groundwater worldwide, where they have been linked to cancer, liver damage and ...
Purdue researchers take inspiration from viruses to improve delivery of nucleic acid-based therapies to cancer cells
2024-08-28
WEST LAFAYETTE, Ind. — A researcher in Purdue University’s College of Science is developing a patent-pending platform technology that mimics the dual-layer structure of viruses to deliver nucleic acid (NA)-based therapies to targeted cancer cells.
David Thompson leads a team developing the carrier system called LENN. He is a professor in the James Tarpo Jr. and Margaret Tarpo Department of Chemistry and on the faculty of the Purdue Institute for Cancer Research and the Purdue ...
New USC research reveals untapped potential for mobilizing voters of color in battleground states
2024-08-28
With swing states playing a pivotal role in the 2024 presidential election, new reports from the Center for Inclusive Democracy (CID) at the USC Sol Price School of Public Policy have found significant voter turnout disparities between white, non-Latino voters and voters of color in all nine battleground states.
The reports highlight the persistent “voter representation gap,” where voters of color remain underrepresented at the polls, despite substantial population growth. If narrowed by November, the research concludes, this gap could significantly influence election results, particularly in swing states that had particularly close ...
The Public Health Career Explorer launches, matching health department job openings with career interests and preparation
2024-08-28
August 28, 2024-- A new, easy-to-use, evidence-based career assessment, the Public Health Career Explorer, has just launched to help job-seekers who are interested specifically in public health careers.
The Public Health Career Explorer career assessment tool, three years in the making, was developed at Columbia University Mailman School of Public Health. Using the well-researched O*Net MyNextMove assessment, Columbia Mailman School’s Heather Krasna, PhD, EdM, MS, associate dean of career and professional development painstakingly matched the Holland Codes (career interest codes) for each of the specific ...
How stigma affects Asian Americans living with hepatitis B
2024-08-28
Asian Americans comprise just 6% of the U.S. population, but they represent over 60% of Americans with hepatitis B. Hepatitis B (HBV) is a virus that infects the liver, and while some people may recover from the infection, others can go on to develop liver failure or cancer. HBV is transmitted through blood, semen or other bodily fluids, but it can also be passed from mother to child during birth, which is how the majority of Asian Americans acquire the disease.
Up until 2010, laws in China allowed for discrimination against people with HBV, barring adults from employment and keeping ...
New study: drug may stop migraines before headache starts
2024-08-28
MINNEAPOLIS – When taken at the first signs of a migraine, before headache pain begins, a drug called ubrogepant may be effective in helping people with migraine go about their daily lives with little or no symptoms, according to a new study published in the August 28, 2024, online issue of Neurology®, the medical journal of the American Academy of Neurology. The study focused on people with migraine who could tell when an attack was about to happen, due to early symptoms such as sensitivity to light and sound, fatigue, neck pain or stiffness, or dizziness.
Ubrogepant ...