(Press-News.org) WEST LAFAYETTE, Ind. – In the background of image recognition software that can ID our friends on social media and wildflowers in our yard are neural networks, a type of artificial intelligence inspired by how own our brains process data. While neural networks sprint through data, their architecture makes it difficult to trace the origin of errors that are obvious to humans — like confusing a Converse high-top with an ankle boot — limiting their use in more vital work like health care image analysis or research. A new tool developed at Purdue University makes finding those errors as simple as spotting mountaintops from an airplane.
“In a sense, if a neural network were able to speak, we’re showing you what it would be trying to say,” said David Gleich, a Purdue professor of computer science in the College of Science who developed the tool, which is featured in a paper published in Nature Machine Intelligence. "The tool we’ve developed helps you find places where the network is saying, ‘Hey, I need more information to do what you’ve asked.’ I would advise people to use this tool on any high-stakes neural network decision scenarios or image prediction task.”
Code for the tool is available on GitHub, as are use case demonstrations. Gleich collaborated on the research with Tamal K. Dey, also a Purdue professor of computer science, and Meng Liu, a former Purdue graduate student who earned a doctorate in computer science.
In testing their approach, Gleich’s team caught neural networks mistaking the identity of images in databases of everything from chest X-rays and gene sequences to apparel. In one example, a neural network repeatedly mislabeled images of cars from the Imagenette database as cassette players. The reason? The pictures were drawn from online sales listings and included tags for the cars’ stereo equipment.
Neural network image recognition systems are essentially algorithms that process data in a way that mimics the weighted firing pattern of neurons as an image is analyzed and identified. A system is trained to its task — such as identifying an animal, a garment or a tumor — with a “training set” of images that includes data on each pixel, tagging and other information, and the identity of the image as classified within a particular category. Using the training set, the network learns, or “extracts,” the information it needs in order to match the input values with the category. This information, a string of numbers called an embedded vector, is used to calculate the probability that the image belongs to each of the possible categories. Generally speaking, the correct identity of the image is within the category with the highest probability.
But the embedded vectors and probabilities don’t correlate to a decision-making process that humans would recognize. Feed in 100,000 numbers representing the known data, and the network produces an embedded vector of 128 numbers that don’t correspond to physical features, although they do make it possible for the network to classify the image. In other words, you can’t open the hood on the algorithms of a trained system and follow along. Between the input values and the predicted identity of the image is a proverbial “black box” of unrecognizable numbers across multiple layers.
“The problem with neural networks is that we can’t see inside the machine to understand how it’s making decisions, so how can we know if a neural network is making a characteristic mistake?” Gleich said.
Rather than trying to trace the decision-making path of any single image through the network, Gleich’s approach makes it possible to visualize the relationship that the computer sees among all the images in an entire database. Think of it like a bird’s-eye view of all the images as the neural network has organized them.
The relationship among the images (like network’s prediction of the identity classification of each of the images in the database) is based on the embedded vectors and probabilities the network generates. To boost the resolution of the view and find places where the network can’t distinguish between two different classifications, Gleich’s team first developed a method of splitting and overlapping the classifications to identify where images have a high probability of belonging to more than one classification.
The team then maps the relationships onto a Reeb graph, a tool taken from the field of topological data analysis. On the graph, each group of images the network thinks are related is represented by a single dot. Dots are color coded by classification. The closer the dots, the more similar the network considers groups to be, and most areas of the graph show clusters of dots in a single color. But groups of images with a high probability of belonging to more than one classification will be represented by two differently colored overlapping dots. With a single glance, areas where the network cannot distinguish between two classifications appear as a cluster of dots in one color, accompanied by a smattering of overlapping dots in a second color. Zooming in on the overlapping dots will show an area of confusion, like the picture of the car that’s been labeled both car and cassette player.
“What we’re doing is taking these complicated sets of information coming out of the network and giving people an ‘in’ into how the network sees the data at a macroscopic level,” Gleich said. “The Reeb map represents the important things, the big groups and how they relate to each other, and that makes it possible to see the errors.”
“Topological Structure of Complex Predictions” was produced with the support of the National Science Foundation and the U.S. Department of Energy.
The mind’s eye of a neural network system
ELSE PRESS RELEASES FROM THIS DATE:
Study finds motorist disorientation syndrome is not only caused by vestibular dysfunction
Amsterdam, November 16, 2023 – A large case series aimed at understanding the factors underlying Motorist Disorientation Syndrome (MDS) has found that patients experience severe, consistent symptoms comparable to vestibular migraine. Previously there has been speculation that underlying peripheral vestibular hypofunction, when the inner ear part of the balance system is not working properly, contributes to this presentation. However, vestibular deficits were not a consistent feature in the patients studied. The findings have been published in the Journal of Vestibular Research. In ...
Rabies virus variants from marmosets are found in bats
Rabies virus variants closely related to variants present in White-tufted marmosets (Callithrix jacchus) have been detected in bats in Ceará state, Northeast Brazil. Rabies is a deadly disease for humans. Its emergence in distinct wildlife species is a potential source of human infection and hence a public health concern. Marmosets are common in forests and conservation units throughout Brazil. In or near urban areas, they are often captured as pets and later abandoned. They have been linked ...
How a mutation in microglia elevates Alzheimer’s risk
A rare but potent genetic mutation that alters a protein in the brain’s immune cells, known as microglia, can give people as much as a three-fold greater risk of developing Alzheimer’s disease. A new study by researchers in The Picower Institute for Learning and Memory at MIT details how the mutation undermines microglia function, explaining how it seems to generate that higher risk. “This TREM2 R47H/+ mutation is a pretty important risk factor for Alzheimer’s disease,” said study lead author Jay Penney, a former postdoc in the MIT lab of Picower Professor Li-Huei ...
International team uses Insilico Medicine’s AI platform to find dual targets for aging and cancer
An international research team is the first to use artificial intelligence (AI) analysis to identify dual-purpose target candidates for the treatment of cancer and aging, the most promising of which was experimentally validated. The findings were published in the journal Aging Cell. Researchers from the University of Oslo, University of Chicago Pritzker School of Medicine, and clinical stage AI-driven drug discovery company Insilico Medicine used Insilico’s AI target discovery engine, PandaOmics, to analyze transcriptomic data derived from ...
New therapeutic strategy to reduce neuronal death in amyotrophic lateral sclerosis
Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disease that affects neurons in the brain and spinal cord causing loss of muscle control. A study by the University of Barcelona has designed a potential therapeutic strategy to tackle this pathology that has no treatment to date. It is a molecular trap that prevents one of the most common genetic ALS-causing peptide compounds, the Poly-GR dipeptide, from causing its toxic effects in the body. The results show that this strategy reduces the death of neurons in patients and in an animal model (vinegar flies) of the disease. The first authors of this international research study published in the journal Science Advances are ...
Breakthrough in bladder cancer research
After 40 years of treating metastatic bladder cancer with chemotherapy as a primary treatment, scientists now present a new approach using immunotherapy combinations. The results of not just one, but two studies have been presented at the European Society for Medical Oncology (ESMO) conference in Madrid. The outcomes of these studies are going to revolutionize the landscape of bladder cancer treatment. Traditionally, cisplatin-based chemotherapy has been the standard treatment for bladder cancer patients who are able to tolerate this drug. However, responses have been limited, and durable outcomes rare. Over the past years, two phase-3 clinical trials studied the effects ...
Study: Temperature variability reduces nesting success
Ithaca, N.Y.—Many songbirds are nesting earlier in spring because of warmer temperatures brought about by climate change. But the shift brings another danger that is especially deadly for nestlings: greater exposure to temperature variability in the form of cold snaps and heat waves. Such extremes result in more nest failures. These findings come from a Cornell Lab of Ornithology study just published in the journal Nature Communications. "When we talk about temperature changes, the focus is mostly on averages," said co-lead author Conor Taff, a researcher in Cornell University's ...
Understanding survival factors in critically ill patients on extracorporeal membrane oxygenation
Extracorporeal membrane oxygenation (ECMO) is a widely used advanced life support procedure that provides cardiac and respiratory support to critically ill patients. ECMO use has been increasing exponentially over the last decade as it has shown success in resuscitating patients in critical situations like the COVID-19 pandemic and is now a lifesaving treatment modality in intensive care units (ICUs). However, prolonged ECMO use may be associated with an increased risk of mortality. Identifying risk factors for in-hospital mortality and developing standardized nursing practice guidelines for ECMO management may improve the survival rates in patients. In this vein, ...
Dr. Ralph and Marian Falk Medical Research Trust awards $1.35M to two Case Western Reserve University researchers
CLEVELAND—A combined $1.35 million from the Dr. Ralph and Marian Falk Medical Research Trust was awarded to two researchers from the Case Western Reserve University School of Medicine to advance their work on finding more effective treatments—and better options—for two debilitating diseases. The Falk Trust awarded Reshmi Parameswaran, an assistant professor in the Department of Medicine, Division of Hematology/Oncology, pathology and pediatrics at the School of Medicine, $1 million over three years for her work in cancer cell therapy. Carlos Subauste, a professor of medicine and pathology at the School of Medicine, received a two-year, ...
High levels of maternal stress during pregnancy linked to children’s behavior problems
WASHINGTON – Children whose mothers are highly stressed, anxious or depressed during pregnancy may be at higher risk for mental health and behavior issues during their childhood and teen years, according to research published by the American Psychological Association. “Our research suggests that psychological distress during the pregnancy period has a small but persistent effect on children’s risk for aggressive, disinhibited and impulsive behaviors,” said study author Irene Tung, PhD, of California State University Dominguez Hills. “These findings add to the evidence that providing widely accessible mental health care and support ...