PRESS-NEWS.org - Press Release Distribution
PRESS RELEASES DISTRIBUTION

Large language models can help detect social media bots — but can also make the problem worse

2024-08-28
(Press-News.org) An external study of Twitter in 2022 estimated that between a third and two thirds of accounts on the social media site were bots. And many of these automatons flooding social media are dispatched to sow political polarization, hate, misinformation, propaganda and scams. The ability to sift them out of the online crowds is vital for a safer, more humane (or at least more human) internet.

But the recent proliferation of large language models (known as "LLMs" for short), such as OpenAI’s ChatGPT and Meta’s Llama, stands to complicate the world of social media bots.

A team led by University of Washington researchers found that while operators can use customized LLMs to make bots more sophisticated at evading automated detectors, LLMs can also improve systems that detect bots. In the team’s tests, LLM-based bots reduced the performance of existing detectors by 30%. Yet researchers also found that an LLM trained specifically to detect social media bots outperformed state-of-the-art systems by 9%.

The team presented this research Aug. 11 at the 62nd Annual Meeting of the Association for Computational Linguistics in Bangkok.

“There’s always been an arms race between bot operators and the researchers trying to stop them,” said lead author Shangbin Feng, a doctoral student in the Paul G. Allen School of Computer Science & Engineering. “Each advance in bot detection is often met with an advance in bot sophistication, so we explored the opportunities and the risks that large language models present in this arms race.”

Researchers tested LLMs’ potential to detect bots in a few ways. When they fed Twitter data sets (culled before the platform became X) to off-the-shelf LLMs, including ChatGPT and Llama, the systems failed to accurately detect bots more than currently used technologies.

“Analyzing whether a user is a bot or not is much more complex than some of the tasks we’ve seen these general LLMs excel at, like recalling a fact or doing a grade-school math problem,” Feng said.

This complexity comes in part from the need to analyze three types of information for different attributes to detect a bot: the metadata (number of followers, geolocation, etc.), the text posted online and the network properties (such as what accounts a user is following).

When the team fine-tuned the LLMs with instructions on how to detect bots based on these three types of information, the models were able to detect bots with greater accuracy than current state-of-the-art systems.

The team also explored how LLMs might make bots more sophisticated and harder to detect. First the researchers simply gave LLMs prompts such as, “Please rewrite the description of this bot account to sound like a genuine user.”

They also tested more iterative, complicated approaches. In one test, the LLM would rewrite the bot post. The team then ran this through an existing bot-detection system, which would estimate the likelihood that a post was written by a bot. This process would be repeated as the LLM worked to lower that estimate. The team ran a similar test while removing and adding accounts that the bot followed to adjust its likelihood score.

These strategies, particularly rewriting the bots’ posts, reduced the effectiveness of the bot detection systems by as much as 30%. But the LLM-based detectors the team trained saw only a 2.3% drop in effectiveness on these manipulated posts, suggesting that the best way to detect LLM-powered bots might be with LLMs themselves.

“This work is only a scientific prototype,” said senior author Yulia Tsvetkov, an associate professor in the Allen School. “We aren’t releasing these systems as tools anyone can download, because in addition to developing technology to defend against malicious bots, we are experimenting with threat modeling of how to create an evasive bot, which continues the cat-and-mouse game of building stronger bots that need stronger detectors.”

Researchers note that there are important limitations to using LLMs as bot detectors, such as the systems’ potential to leak private information. They also highlight that the data used in the paper is from 2022, before Twitter effectively closed off its data to academic researchers.

In the future, researchers want to look at bot detection beyond text, such as memes or videos on other platforms such as TikTok, where newer data sets are available. The team also wants to expand the research into other languages.

“Doing this research across different languages is extremely important,” Tsvetkov said. “We are seeing a lot of misinformation, manipulation and the targeting of specific populations as a result of various world conflicts.”

Additional co-authors on this paper are Herun Wan and Ningnan Wang, both undergraduates at Xi’an Jiaotong University; Minnan Luo, an assistant professor at Xi’an Jiaotong University; and Zhaoxuan Tan, a doctoral student at the University of Notre Dame. This research was funded by an NSF CAREER award.

For more information, contact Feng at shangbin@cs.washington.edu and Tsvetkov at yuliats@cs.washington.edu.

END


ELSE PRESS RELEASES FROM THIS DATE:

How beetle juice led to the discovery of a virus and solved the mystery of a superworm die-off

How beetle juice led to the discovery of a virus and solved the mystery of a superworm die-off
2024-08-28
Rutgers University-New Brunswick scientists have discovered a virus that caused a nationwide die-off of superworms, a common food for birds, reptiles, other pets and, more and more so, even for humans as an alternative protein source. In doing so, they pioneered a different way to search for and identify emerging viruses and pathogens in humans, plants and animals. Using chopped up beetle carcasses forming a slurry and an electron microscope cooled by liquid nitrogen, the scientists reported today in Cell that they have discovered what they have titled Zophobas morio black wasting virus. The name is derived from ...

Leading computational scientist & oncology researcher Elana Fertig appointed new Director of the Institute for Genome Sciences

Leading computational scientist & oncology researcher Elana Fertig appointed new Director of the Institute for Genome Sciences
2024-08-28
University of Maryland School of Medicine (UMSOM) Dean Mark T. Gladwin, MD, announced today the appointment of Elana J. Fertig, PhD, FAIMBE, as the new Director of the School’s Institute for Genome Sciences (IGS). She is  internationally-recognized for her work in integrating spatial multi-omics technologies with mathematical models to develop a new predictive medicine paradigm in cancer. Spatial technologies allow researchers to learn about any cell type inside of natural tissue, including gene activity ...

UVA research cracks the autism code, making the neurodivergent brain visible

UVA research cracks the autism code, making the neurodivergent brain visible
2024-08-28
A multi-university research team co-led by University of Virginia engineering professor Gustavo K. Rohde has developed a system that can spot genetic markers of autism in brain images with 89 to 95% accuracy. Their findings suggest doctors may one day see, classify and treat autism and related neurological conditions with this method, without having to rely on, or wait for, behavioral cues. And that means this truly personalized medicine could result in earlier interventions. “Autism is traditionally diagnosed behaviorally but has a strong genetic basis. A genetics-first approach could ...

Marine engineering group to award ORNL researchers for innovative welding software

Marine engineering group to award ORNL researchers for innovative welding software
2024-08-28
Zhili Feng, a distinguished R&D staff member, and Jian Chen, a senior R&D staff member, in Materials Science and Technology Division at the Department of Energy’s Oak Ridge National Laboratory, will receive the Elmer L. Hann Award at the Society of Naval Architects and Maritime Engineers, or SNAME, Convention on Oct. 15 in Norfolk, Virginia. SNAME plays a crucial role in promoting excellence in marine engineering and naval architecture, supporting professionals through education, research and collaboration. The award is presented to authors of the best paper on ship production delivered at a SNAME event. In September 2023, Feng and Chen presented “Develop ...

UBC engineers develop all-in-one solution to catch and destroy ‘forever chemicals’

UBC engineers develop all-in-one solution to catch and destroy ‘forever chemicals’
2024-08-28
Chemical engineers at the University of British Columbia have developed a new treatment that traps and treats PFAS substances—widely known as “forever chemicals”—in a single, integrated system. Per- and polyfluoroalkyl substances (PFAS) are widely used in manufacturing consumer goods like waterproof clothing due to their resistance to heat, water and stains. However, they are also pollutants, often ending up in surface and groundwater worldwide, where they have been linked to cancer, liver damage and ...

Purdue researchers take inspiration from viruses to improve delivery of nucleic acid-based therapies to cancer cells

Purdue researchers take inspiration from viruses to improve delivery of nucleic acid-based therapies to cancer cells
2024-08-28
WEST LAFAYETTE, Ind. — A researcher in Purdue University’s College of Science is developing a patent-pending platform technology that mimics the dual-layer structure of viruses to deliver nucleic acid (NA)-based therapies to targeted cancer cells. David Thompson leads a team developing the carrier system called LENN. He is a professor in the James Tarpo Jr. and Margaret Tarpo Department of Chemistry and on the faculty of the Purdue Institute for Cancer Research and the Purdue ...

New USC research reveals untapped potential for mobilizing voters of color in battleground states

New USC research reveals untapped potential for mobilizing voters of color in battleground states
2024-08-28
With swing states playing a pivotal role in the 2024 presidential election, new reports from the Center for Inclusive Democracy (CID) at the USC Sol Price School of Public Policy have found significant voter turnout disparities between white, non-Latino voters and voters of color in all nine battleground states. The reports highlight the persistent “voter representation gap,” where voters of color remain underrepresented at the polls, despite substantial population growth. If narrowed by November, the research concludes, this gap could significantly influence election results, particularly in swing states that had particularly close ...

The Public Health Career Explorer launches, matching health department job openings with career interests and preparation

2024-08-28
August 28, 2024-- A new, easy-to-use, evidence-based career assessment, the Public Health Career Explorer, has just launched to help job-seekers who are interested specifically in public health careers. The Public Health Career Explorer career assessment tool, three years in the making, was developed at Columbia University Mailman School of Public Health. Using the well-researched O*Net MyNextMove assessment, Columbia Mailman School’s Heather Krasna, PhD, EdM, MS, associate dean of career and professional development  painstakingly matched the Holland Codes (career interest codes) for each of the specific ...

How stigma affects Asian Americans living with hepatitis B

2024-08-28
Asian Americans comprise just 6% of the U.S. population, but they represent over 60% of Americans with hepatitis B. Hepatitis B (HBV) is a virus that infects the liver, and while some people may recover from the infection, others can go on to develop liver failure or cancer. HBV is transmitted through blood, semen or other bodily fluids, but it can also be passed from mother to child during birth, which is how the majority of Asian Americans acquire the disease. Up until 2010, laws in China allowed for discrimination against people with HBV, barring adults from employment and keeping ...

New study: drug may stop migraines before headache starts

2024-08-28
MINNEAPOLIS – When taken at the first signs of a migraine, before headache pain begins, a drug called ubrogepant may be effective in helping people with migraine go about their daily lives with little or no symptoms, according to a new study published in the August 28, 2024, online issue of Neurology®, the medical journal of the American Academy of Neurology. The study focused on people with migraine who could tell when an attack was about to happen, due to early symptoms such as sensitivity to light and sound, fatigue, neck pain or stiffness, or dizziness.    Ubrogepant ...

LAST 30 PRESS RELEASES:

Reality check: making indoor smartphone-based augmented reality work

Overthinking what you said? It’s your ‘lizard brain’ talking to newer, advanced parts of your brain

Black men — including transit workers — are targets for aggression on public transportation, study shows

Troubling spike in severe pregnancy-related complications for all ages in Illinois

Alcohol use identified by UTHealth Houston researchers as most common predictor of escalated cannabis vaping among youths in Texas

Need a landing pad for helicopter parenting? Frame tasks as learning

New MUSC Hollings Cancer Center research shows how Golgi stress affects T-cells' tumor-fighting ability

#16to365: New resources for year-round activism to end gender-based violence and strengthen bodily autonomy for all

Earliest fish-trapping facility in Central America discovered in Maya lowlands

São Paulo to host School on Disordered Systems

New insights into sleep uncover key mechanisms related to cognitive function

USC announces strategic collaboration with Autobahn Labs to accelerate drug discovery

Detroit health professionals urge the community to act and address the dangers of antimicrobial resistance

3D-printing advance mitigates three defects simultaneously for failure-free metal parts 

Ancient hot water on Mars points to habitable past: Curtin study

In Patagonia, more snow could protect glaciers from melt — but only if we curb greenhouse gas emissions soon

Simplicity is key to understanding and achieving goals

Caste differentiation in ants

Nutrition that aligns with guidelines during pregnancy may be associated with better infant growth outcomes, NIH study finds

New technology points to unexpected uses for snoRNA

Racial and ethnic variation in survival in early-onset colorectal cancer

Disparities by race and urbanicity in online health care facility reviews

Exploring factors affecting workers' acquisition of exercise habits using machine learning approaches

Nano-patterned copper oxide sensor for ultra-low hydrogen detection

Maintaining bridge safer; Digital sensing-based monitoring system

A novel approach for the composition design of high-entropy fluorite oxides with low thermal conductivity

A groundbreaking new approach to treating chronic abdominal pain

ECOG-ACRIN appoints seven researchers to scientific committee leadership positions

New model of neuronal circuit provides insight on eye movement

Cooking up a breakthrough: Penn engineers refine lipid nanoparticles for better mRNA therapies

[Press-News.org] Large language models can help detect social media bots — but can also make the problem worse