PRESS-NEWS.org - Press Release Distribution
PRESS RELEASES DISTRIBUTION

Automated system teaches users when to collaborate with an AI assistant

MIT researchers develop a customized onboarding process that helps a human learn when a model’s advice is trustworthy

2023-12-07
(Press-News.org)

Artificial intelligence models that pick out patterns in images can often do so better than human eyes — but not always. If a radiologist is using an AI model to help her determine whether a patient’s X-rays show signs of pneumonia, when should she trust the model’s advice and when should she ignore it? 

A customized onboarding process could help this radiologist answer that question, according to researchers at MIT and the MIT-IBM Watson AI Lab. They designed a system that teaches a user when to collaborate with an AI assistant.

In this case, the training method might find situations where the radiologist trusts the model’s advice — except she shouldn’t because the model is wrong. The system automatically learns rules for how she should collaborate with the AI, and describes them with natural language. 

During onboarding, the radiologist practices collaborating with the AI using training exercises based on these rules, receiving feedback about her performance and the AI’s performance.

The researchers found that this onboarding procedure led to about a 5 percent improvement in accuracy when humans and AI collaborated on an image prediction task. Their results also show that just telling the user when to trust the AI, without training, led to worse performance. 

Importantly, the researchers’ system is fully automated, so it learns to create the onboarding process based on data from the human and AI performing a specific task. It can also adapt to different tasks, so it can be scaled up and used in many situations where humans and AI models work together, such as in social media content moderation, writing, and programming.

“So often, people are given these AI tools to use without any training to help them figure out when it is going to be helpful. That’s not what we do with nearly every other tool that people use — there is almost always some kind of tutorial that comes with it. But for AI, this seems to be missing. We are trying to tackle this problem from a methodological and behavioral perspective,” says Hussein Mozannar, a graduate student in the Social and Engineering Systems doctoral program within the Institute for Data, Systems, and Society (IDSS) and lead author of a paper about this training process.

The researchers envision that such onboarding will be a crucial part of training for medical professionals. 

“One could imagine, for example, that doctors making treatment decisions with the help of AI will first have to do training similar to what we propose. We may need to rethink everything from continuing medical education to the way clinical trials are designed,” says senior author David Sontag, a professor of EECS, a member of the MIT-IBM Watson AI Lab and the MIT Jameel Clinic, and the leader of the Clinical Machine Learning Group of the Computer Science and Artificial Intelligence Laboratory (CSAIL).

Mozannar, who is also a researcher with the Clinical Machine Learning Group, is joined on the paper by Jimin J. Lee, an undergraduate in electrical engineering and computer science; Dennis Wei, a senior research scientist at IBM Research; and Prasanna Sattigeri and Subhro Das, research staff members at the MIT-IBM Watson AI Lab. The paper will be presented at the Conference on Neural Information Processing Systems.

Training that evolves

Existing onboarding methods for human-AI collaboration are often composed of training materials produced by human experts for specific use cases, making them difficult to scale up. Some related techniques rely on explanations, where the AI tells the user its confidence in each decision, but research has shown that explanations are rarely helpful, Mozannar says.

“The AI model’s capabilities are constantly evolving, so the use cases where the human could potentially benefit from it are growing over time. At the same time, the user’s perception of the model continues changing. So, we need a training procedure that also evolves over time,” he adds.

To accomplish this, their onboarding method is automatically learned from data. It is built from a dataset that contains many instances of a task, such as detecting the presence of a traffic light from a blurry image.

The system’s first step is to collect data on the human and AI performing this task. In this case, the human would try to predict, with the help of AI, whether blurry images contain traffic lights. 

The system embeds these data points onto a latent space, which is a representation of data in which similar data points are closer together. It uses an algorithm to discover regions of this space where the human collaborates incorrectly with the AI. These regions capture instances where the human trusted the AI’s prediction but the prediction was wrong, and vice versa. 

Perhaps the human mistakenly trusts the AI when images show a highway at night.

After discovering the regions, a second algorithm utilizes a large language model to describe each region as a rule, using natural language. The algorithm iteratively fine-tunes that rule by finding contrasting examples. It might describe this region as “ignore AI when it is a highway during the night.”

These rules are used to build training exercises. The onboarding system shows an example to the human, in this case a blurry highway scene at night, as well as the AI’s prediction, and asks the user if the image shows traffic lights. The user can answer yes, no, or use the AI’s prediction. 

If the human is wrong, they are shown the correct answer and performance statistics for the human and AI on these instances of the task. The system does this for each region, and at the end of the training process, repeats the exercises the human got wrong. 

“After that, the human has learned something about these regions that we hope they will take away in the future to make more accurate predictions,” Mozannar says.

Onboarding boosts accuracy

The researchers tested this system with users on two tasks — detecting traffic lights in blurry images and answering multiple choice questions from many domains (such as biology, philosophy, computer science, etc.).

They first showed users a card with information about the AI model, how it was trained, and a breakdown of its performance on broad categories. Users were split into five groups: Some were only shown the card, some went through the researchers’ onboarding procedure, some went through a baseline onboarding procedure, some went through the researchers’ onboarding procedure and were given recommendations of when they should or should not trust the AI, and others were only given the recommendations.

Only the researchers’ onboarding procedure without recommendations improved users’ accuracy significantly, boosting their performance on the traffic light prediction task by about 5 percent without slowing them down. However, onboarding was not as effective for the question-answering task. The researchers believe this is because the AI model, ChatGPT, provided explanations with each answer that convey whether it should be trusted.

But providing recommendations without onboarding had the opposite effect — users not only performed worse, they took more time to make predictions.

“When you only give someone recommendations, it seems like they get confused and don’t know what to do. It derails their process. People also don’t like being told what to do, so that is a factor as well,” Mozannar says. 

Providing recommendations alone could harm the user if those recommendations are wrong, he adds. With onboarding, on the other hand, the biggest limitation is the amount of available data. If there aren’t enough data, the onboarding stage won’t be as effective, he says.

In the future, he and his collaborators want to conduct larger studies to evaluate the short- and long-term effects of onboarding. They also want to leverage unlabeled data for the onboarding process, and find methods to effectively reduce the number of regions without omitting important examples. 

This work is funded, in part, by the MIT-IBM Watson AI Lab. 

###

Written by Adam Zewe, MIT News

Paper: “Effective Human-AI Teams via Learned Natural Language Rules and Onboarding”

https://arxiv.org/pdf/2311.01007.pdf

END



ELSE PRESS RELEASES FROM THIS DATE:

CCNY team develops pioneering indoor navigation system

2023-12-07
In a major stride toward revolutionizing indoor navigation, a City College of New York-led team has developed groundbreaking technology that could chart real-time paths, delivering users—both sighted and low vision—a seamless and accurate indoor navigation experience complete with turn-by-turn guidance. The invention has earned a U.S. patent  titled "System and Method for Real-time Indoor Navigation." The innovation is the brainchild of the City College-based CUNY Computational Vision and Convergence Laboratory (CCVCL) headed by Zhigang ...

SLAC brings rapid-fire laser and target expertise to national fusion energy research hubs

SLAC brings rapid-fire laser and target expertise to national fusion energy research hubs
2023-12-07
The U.S. Department of Energy’s SLAC National Accelerator Laboratory and Stanford University will partner with Colorado State University (CSU), Lawrence Livermore National Laboratory (LLNL) and other institutions in the creation of two inertial fusion science and technology hubs that were announced today by the DOE.  Following on last year’s fusion ignition breakthrough at LLNL’s National Ignition Facility, the hubs aim to accelerate foundational inertial fusion research and technology development toward a potential clean and abundant energy source. In total, $42 million was awarded to collaborations between universities, ...

Cooling down the hot takes on Twitch 

2023-12-07
Twitch. Some see it as a fun online community of gamers and good-natured e-sports fandom. For others, it’s a perilous stream of potentially toxic content and hate speech.   In the ever-evolving landscape of digital communication, the real-time nature of messages on live-stream platforms like Twitch and YouTube Live brings with it unique challenges for content moderation. At present, effective tools for moderating content in live streams are lacking because existing models have been trained on non-real-time social media platforms like Facebook or Twitter. Research Assistant Dong-Ho Lee and Principal Scientist Jay Pujara, ...

Study: How farmers decide to store or sell their grain

Study: How farmers decide to store or sell their grain
2023-12-07
URBANA, Ill. – When farmers harvest their grain, they can choose to sell it right away or store it to obtain better prices later in the season. A new study from the University of Illinois Urbana-Champaign explores how Illinois corn and soybean producers make those decisions and why the cost-benefit evaluation of storage may differ across farms. Agricultural commodity prices fluctuate in response to changes in supply and demand, which depends on the stockpile of grain inventories around the country — but economists don’t really don't know how farmers decide to sell versus store their grain. “Economic theory provides guidance that Extension ...

Black patients less likely to get referral for home health care after hospital stay

2023-12-07
    When discharging Black patients from the hospital, nurses are less likely to refer them to home health care than white patients, a new University of Michigan study found.   About 22% of Black patients are referred by discharge nurses to home health care compared to 27% of white patients.    The study found that despite a higher likelihood that Black patients were unmarried, lived alone and had more chronic conditions—all risk factors for hospital readmission—they were routinely rated equally ready for hospital discharge as white patients, and ...

Damon Runyon launches post-baccalaureate research internship

2023-12-07
The Damon Runyon Cancer Research Foundation is thrilled to announce the launch of the Damon Runyon Scholars Program for Advancing Research and Knowledge (SPARK), a one-year intensive cancer research internship program for post-baccalaureate students who come from backgrounds underrepresented in the sciences. The goal of the program is to provide students who have the potential to become leaders in cancer research with rigorous scientific training and a network of mentors and peers to support their next steps into graduate school and beyond.  Damon ...

The Morton Arboretum tree root scientist recognized as top-cited researcher

The Morton Arboretum tree root scientist recognized as top-cited researcher
2023-12-07
LISLE, Ill. (Dec. 7, 2023)— The Morton Arboretum Tree Root Biologist Luke McCormack, Ph.D., has been recognized as one of the most cited and influential researchers worldwide by global information services provider Clarivate’s esteemed 2023 list of “Highly Cited Researchers.”  The list includes influential researchers at universities, research institutes and commercial organizations around the world who have demonstrated significant and broad influence in their field(s) of research. McCormack is the first Arboretum ...

Discrimination during pregnancy may alter circuits in infants’ brains

Discrimination during pregnancy may alter circuits in infants’ brains
2023-12-07
Racial discrimination and bias are painful realities and increasingly recognized as detrimental to the health of adults and children. These stressful experiences also appear to be transmitted from mother to child during pregnancy, altering the strength of infants’ brain circuits, according to a new study from researchers at Columbia, Yale, and Children’s Hospital of Los Angeles. The study found similar brain changes in infants whose mothers experienced stress from adapting to a new culture during pregnancy. “A leading hypothesis would be that the connectivity changes that we see could ...

Wayne State University announces creation of two research centers and institutes that aim to impact the health of Detroiters and beyond

2023-12-07
DETROIT – Wayne State University Interim Vice President for Research Timothy Stemmler, Ph.D., announced today the university’s Board of Governors approved the creation of two research initiatives that aim to improve the health and lives of the Detroit community and beyond. Center for Emerging and Infectious Diseases The Center for Emerging and Infectious Diseases (CEID) will contribute to the ongoing advancement of diagnostic testing, enabling rapid and accurate identification of infectious ...

New method is better able to map immune response and paves way for new treatments

2023-12-07
A new method, developed at Karolinska Institutet, KTH Royal Institute of Technology and SciLifeLab in Sweden, can identify unique immune cell receptors and their location in tissue, a study published in the journal Science reports. The researchers predict that the method will improve the ability to identify which immune cells contribute to disease processes and open up opportunities to develop novel therapies for numerous diseases. Immune cells such as T and B cells are central to the body’s defence against both infections and tumours. Both types of immune cells express unique receptors that specifically recognise different parts of unwanted and foreign elements, such as bacteria, viruses ...

LAST 30 PRESS RELEASES:

Nanoscale topcoat can turbocharge supported gold nanoparticle catalysts

Beyond the ink: Painting with physics

Only 9 percent of older Americans were vaccinated against RSV before the disease hit this fall and winter

Evolution-capable AI promotes green hydrogen production using more abundant chemical elements

In wake of powerful cyclone, remarkable recovery of Pacific island’s forests

PSU study sheds light on 2020 extreme weather event that brought fires and snow to western US

Rice physicist earns NSF CAREER Award to revolutionize quantum technology

Mining the treasures locked away in produced water

Minoritized groups face high anxiety when taking part in research experiments

Orcas demonstrating they no longer need to hunt in packs to take down the great white shark

Scientists discover a novel vehicle for antibiotic resistance

Large-scale study explores link between smoking and DNA changes across six racial and ethnic groups

EU funding for outstanding early-career researcher Pieter Gunnink

Associate Professor Ron Korstanje, Ph.D., of The Jackson Laboratory named Evnin Family Chair

Researchers create coating solution for safer food storage

An overgrowth of nerve cells appears to cause lingering symptoms after recurrent UTIs

New findings on the immune system

Most smokers in England wrongly believe vaping is at least as harmful as smoking

New antibodies target “dark side” of influenza virus protein

Fred Hutch announces 2024 Harold M. Weintraub Graduate Student Award recipients

New academic journal on artificial intelligence launched

UMaine researchers use GPS-tracked icebergs in novel study to improve climate models

A mental process that leads to putting off an unpleasant task

The role of history in how efficient color names evolve

AI outperforms humans in standardized tests of creative potential

Study results show 25% of pregnant people are not getting enough omega-3 fatty acids from their diet or dietary supplements

Cleveland Clinic researchers uncover how virus causes cancer, point to potential treatment

SLU professor studies link between adversity, psychiatric and cognitive decline

Warwick to benefit from £2.5 million funding into “phenomenal” metamaterials

More schooling is linked to slowed aging and increased longevity

[Press-News.org] Automated system teaches users when to collaborate with an AI assistant
MIT researchers develop a customized onboarding process that helps a human learn when a model’s advice is trustworthy