(Press-News.org) COLUMBUS, Ohio – In an effort to make the internet more accessible for people with disabilities, researchers at The Ohio State University have begun developing an artificial intelligence agent that could complete complex tasks on any website using simple language commands.
In the three decades since it was first released into the public domain, the world wide web has become an incredibly intricate, dynamic system. Yet because internet function is now so integral to society’s well-being, its complexity also makes it considerably harder to navigate.
Today there are billions of websites available to help access information or communicate with others, and many tasks on the internet can take more than a dozen steps to complete. That’s why Yu Su, co-author of the study and an assistant professor of computer science and engineering at Ohio State, said their work, which uses information taken from live sites to create web agents — online AI helpers — is a step toward making the digital world a less confusing place.
“For some people, especially those with disabilities, it’s not easy for them to browse the internet,” said Su. “We rely more and more on the computing world in our daily life and work, but there are increasingly a lot of barriers to that access, which, to some degree, widens the disparity.”
The study was presented in December at the Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS), a flagship conference for AI and machine learning research.
By taking advantage of the power of large language models, the agent works similarly to how humans behave when browsing the web, said Su. The Ohio State team showed that their model was able to understand the layout and functionality of different websites using only its ability to process and predict language.
Researchers started the process by creating Mind2Web, the first dataset for generalist web agents. Though previous efforts to build web agents focused on toy simulated websites, Mind2Web fully embraces the complex and dynamic nature of real-world websites and emphasizes an agent’s ability of generalizing to entirely new websites it has never seen before. Su said that much of their success is due to their agent’s ability to handle the internet’s ever-evolving learning curve. The team lifted over 2,000 open-ended tasks from 137 different real-world websites, which they then used to train the agent.
Some of the tasks included booking one-way and round-trip international flights, following celebrity accounts on Twitter, browsing comedy films from 1992 to 2017 streaming on Netflix, and even scheduling car knowledge tests at the DMV. Many of the tasks were very complex – for example, booking one of the international flights used in the model would take 14 actions. Such effortless versatility allows for diverse coverage on a number of websites, and opens up a new landscape for future models to explore and learn in an autonomous fashion, said Su.
“It’s only become possible to do something like this because of the recent development of large language models like ChatGPT,” said Su. Since the chatbot became public in November 2022, millions of users have used it to automatically generate content, from poetry and jokes to cooking advice and medical diagnoses.
Still, because one website could contain thousands of raw HTML elements, it would be too costly to feed so much information to a single large language model. To address this gap, the study also introduces a framework called MindAct, a two-pronged agent that uses both small and large language models to carry out these tasks. The team found that by using this strategy, MindAct significantly outperforms other common modeling strategies and is able to understand various concepts at a decent level.
With more fine-tuning, the study points out, the model could likely be used in tandem with both open-and closed-source large language models such as Flan-T5 or GPT-4. However, their work does highlight an increasingly relevant ethical problem in creating flexible artificial intelligence, said Su. While it could certainly serve as a helpful agent to humans surfing the web, the model could also be used to enhance systems like ChatGPT and turn the entire internet into an unprecedentedly powerful tool, said Su.
“On the one hand, we have great potential to improve our efficiency and to allow us to focus on the most creative part of our work,” he said. “But on the other hand, there’s tremendous potential for harm.” For instance, autonomous agents able to translate online steps into the real world could influence society by taking potentially dangerous actions, such as misusing financial information or spreading misinformation.
“We should be extremely cautious about these factors and make a concerted effort to try to mitigate them,” said Su. But as AI research continues to evolve, he notes that it’s likely society will experience major growth in the commercial use and performance of generalist web agents in the years to come, especially as the technology has already gained so much popularity in the public eye.
“Throughout my career, my goal has always been trying to bridge the gap between human users and the computing world,” said Su. “That said, the real value of this tool is that it will really save people time and make the impossible possible.”
The research was supported by the National Science Foundation, the U.S. Army Research Lab and the Ohio Supercomputer Center. Other co-authors were Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang and Huan Sun, all of Ohio State.
#
Contact: Yu Su, Su.809@osu.edu
Written by: Tatyana Woodall, Woodall.52@osu.edu
END
CORVALLIS, Ore. – An international team of scientists led by Oregon State University researchers has used a novel 500-year dataset to frame a “restorative” pathway through which humanity can avoid the worst ecological and social outcomes of climate change.
In addition to charting a possible new course for society, the researchers say their “paradigm shifting” plan can support climate modeling and discussion by providing a set of actions that strongly emphasize social and economic justice as well ...
Chemical recycling of widely used vinyl polymers (VPs) is one of the key technologies required for realizing a sustainable society. In this regard, a team of researchers from Shinshu University have recently reported a new chemical process that facilitates the depolymerization of cyclic styrene-based VPs, resulting in the recovery of a monomer precursor. This highly efficient chemical recycling system can help with effective resource circulation and the development of new plastic recycling technologies.
Vinyl polymers (VPs) are ...
The quantum Cheshire cat effect draws its name from the fictional Cheshire Cat in the Alice in Wonderland story. That cat was able to disappear, leaving only its grin behind. Similarly, in a 2013 paper, researchers claimed quantum particles are able to separate from their properties, with the properties travelling along paths the particle cannot. They named this the quantum Cheshire cat effect. Researchers since have claimed to extend this further, swapping disembodied properties between particles, disembodying multiple properties simultaneously, ...
Complimentary press passes are now available for Discover BMB, the annual meeting of the American Society for Biochemistry and Molecular Biology (ASBMB). Join us March 23–26 in San Antonio to experience an engaging agenda showcasing the newest developments and current trends in the field.
As the flagship meeting for one of the largest molecular life science organizations in the world, #DiscoverBMB brings together researchers in academia and industry from across the globe.
Explore captivating science stories and connect with leading experts during the scientific symposia, which will encompass 12 themes. Topics include:
Exciting ...
Children and adults with rare, deadly genetic diseases have fresh hope for curative therapies, thanks to a new collaboration between the Innovative Genomics Institute (IGI) and Danaher Corporation, a global life sciences and diagnostics innovator.
The new Danaher-IGI Beacon for CRISPR Cures center will use genome editing to address potentially hundreds of diseases, including rare genetic disorders that have no cure. The goal is to ensure treatments can be developed and brought to patients ...
DALLAS, Jan. 9, 2024 — A physician-scientist from Massachusetts researching whether chemicals naturally occurring in foods could help treat heart disease, a genetics expert from Pennsylvania exploring the molecular mechanisms of lipid metabolism and cardiovascular diseases and a California-based professor of cardiovascular medicine studying how vaping impacts the development of abdominal aortic aneurysms are the most recent American Heart Association Merit Award recipients. Over the next five years, each researcher will receive a total of $1 million in funding from the Association, the world’s leading voluntary organization focused on heart and brain health and research, ...
Phages, the viruses that infect bacteria, will pay a high growth-rate cost to access environmental information that can help them choose which lifecycle to pursue, according to a study. Yigal Meir and colleagues developed a model of a bacteria-phage system to investigate how much the viruses should be willing to invest to acquire information about their local environment. A temperate phage, once inside a bacterium, can choose one of two life cycles. In the lytic cycle, the phage turns the bacterium ...
BINGHAMTON, N.Y. -- Researchers from Binghamton University, State University of New York are unraveling the workings of Group B Strep (GBS) infections in pregnant women, which could someday lead to a vaccine.
One in five pregnant women carry Streptococcus agalactiae (Group B Strep or GBS) in the vaginal tract, which is typically harmless — except when it isn’t.
The bacterial infection poses serious and even fatal consequences for newborns, including pneumonia, sepsis and meningitis, which can have long-term effects on the child’s cognitive function.
Researchers ...
Steatotic liver disease (SLD), previously known as non-alcoholic fatty liver disease, which includes a range of conditions caused by fat build-up in the liver due to abnormal lipid metabolism, affects about 25% of the population worldwide, making it the most common liver disorder. Often referred to as “silent liver disease,” SLD progresses without noticeable symptoms and can lead to more severe conditions like cirrhosis (liver scarring) and liver cancer.
A liver biopsy—an invasive procedure involving liver tissue sample extraction from the body—is ...
Robotics and autonomous vehicles are among the most rapidly growing domains in the technological landscape, with the potential to make work and transportation safer and more efficient. Since both robots and self-driving cars need to accurately perceive their surroundings, 3D object detection methods are an active area of study. Most 3D object detection methods employ LiDAR sensors to create 3D point clouds of their environment. Simply put, LiDAR sensors use laser beams to rapidly scan and measure the ...