(Press-News.org) Training neural networks to perform tasks, such as recognizing images or navigating self-driving cars, could one day require less computing power and hardware thanks to a new artificial neuron device developed by researchers at the University of California San Diego. The device can run neural network computations using 100 to 1000 times less energy and area than existing CMOS-based hardware.
Researchers report their work in a paper published Mar. 18 in Nature Nanotechnology.
Neural networks are a series of connected layers of artificial neurons, where the output of one layer provides the input to the next. Generating that input is done by applying a mathematical calculation called a non-linear activation function. This is a critical part of running a neural network. But applying this function requires a lot of computing power and circuitry because it involves transferring data back and forth between two separate units--the memory and an external processor.
Now, UC San Diego researchers have developed a nanometer-sized device that can efficiently carry out the activation function.
"Neural network computations in hardware get increasingly inefficient as the neural network models get larger and more complex," said Duygu Kuzum, a professor of electrical and computer engineering at the UC San Diego Jacobs School of Engineering. "We developed a single nanoscale artificial neuron device that implements these computations in hardware in a very area- and energy-efficient way."
The new study, led by Kuzum and her Ph.D. student Sangheon Oh, was performed in collaboration with a DOE Energy Frontier Research Center led by UC San Diego physics professor Ivan Schuller, which focuses on developing hardware implementations of energy-efficient artificial neural networks.
The device implements one of the most commonly used activation functions in neural network training called a rectified linear unit. What's particular about this function is that it needs hardware that can undergo a gradual change in resistance in order to work. And that's exactly what the UC San Diego researchers engineered their device to do--it can gradually switch from an insulating to a conducting state, and it does so with the help of a little bit of heat.
This switch is what's called a Mott transition. It takes place in a nanometers-thin layer of vanadium dioxide. Above this layer is a nanowire heater made of titanium and gold. When current flows through the nanowire, the vanadium dioxide layer slowly heats up, causing a slow, controlled switch from insulating to conducting.
"This device architecture is very interesting and innovative," said Oh, who is the study's first author. Typically, materials in a Mott transition experience an abrupt switch from insulating to conducting because the current flows directly through the material, he explained. "In this case, we flow current through a nanowire on top of the material to heat it and induce a very gradual resistance change."
To implement the device, the researchers first fabricated an array of these so-called activation (or neuron) devices, along with a synaptic device array. Then they integrated the two arrays on a custom printed circuit board and connected them together to create a hardware version of a neural network.
The researchers used the network to process an image--in this case, a picture of Geisel Library at UC San Diego. The network performed a type of image processing called edge detection, which identifies the outlines or edges of objects in an image. This experiment demonstrated that the integrated hardware system can perform convolution operations that are essential for many types of deep neural networks.
The researchers say the technology could be further scaled up to do more complex tasks such as facial and object recognition in self-driving cars. With interest and collaboration from industry, this could happen, noted Kuzum.
"Right now, this is a proof of concept," Kuzum said. "It's a tiny system in which we only stacked one synapse layer with one activation layer. By stacking more of these together, you could make a more complex system for different applications."
INFORMATION:
Paper: "Energy Efficient Mott Activation Neuron for Full Hardware Implementation of Neural Networks."
This work was supported by the Office of Naval Research, Samsung Electronics, the National Science Foundation, the National Institutes of Health, a Qualcomm Fellowship and the U.S. Department of Energy, Office of Science through an Energy Frontier Research Center.
From swallowing pills to injecting insulin, patients frequently administer their own medication. But they don't always get it right. Improper adherence to doctors' orders is commonplace, accounting for thousands of deaths and billions of dollars in medical costs annually. MIT researchers have developed a system to reduce those numbers for some types of medications.
The new technology pairs wireless sensing with artificial intelligence to determine when a patient is using an insulin pen or inhaler, and flags potential errors in the patient's administration method. "Some past work reports that up to 70% of patients do not ...
A study by Monash University has uncovered that liver metabolism is disrupted in people with obesity-related type 2 diabetes, which contributes to high blood sugar and muscle loss - also known as skeletal muscle atrophy.
Using human trials as well as mouse models, collaborative research led by Dr Adam Rose at Monash Biomedicine Discovery Institute has found the liver metabolism of the amino acid alanine is altered in people with obesity-related type 2 diabetes. By selectively silencing enzymes that break down alanine in liver cells, high blood sugar and muscle loss can be reversed by the restoration of skeletal muscle protein synthesis, a critical determinant of muscle size and strength.
The research, published today in Nature Metabolism, has shown the altered liver metabolism ...
Scientists have witnessed bonobo apes adopting infants who were born outside of their social group for the first time in the wild.
Researchers, including psychologists at Durham University, UK, twice saw the unusual occurrence among bonobos in the Democratic Republic of Congo, in central Africa.
They say their findings give us greater insight into the parental instincts of one of humans' closest relatives and could help to explain the emotional reason behind why people readily adopt children who they have had no previous connection with.
The research, led by Kyoto University, in Japan, is published in the journal Scientific Reports.
Researchers observed a number of bonobo groups over several years in the Wamba area of ...
Researchers at Seattle's Institute for Systems Biology and their collaborators looked at the electronic health records of nearly 630,000 patients who were tested for SARS-CoV-2, and found stark disparities in COVID-19 outcomes -- odds of infection, hospitalization, and in-hospital mortality -- between White and non-White minority racial and ethnic groups. The work was published in the journal Clinical Infectious Diseases.
The team looked at sociodemographic and clinical characteristics of patients who were part of the Providence healthcare system in Washington, Oregon and California. These ...
Metformin, a drug used to treat type-2 diabetes, could help reduce chronic inflammation in people living with HIV (PLWH) who are being treated with antiretroviral therapy (ART), according to researchers at the University of Montreal Hospital Research Centre (CRCHUM).
Although ART has helped improved the health of PLWH, they are nevertheless at greater risk of developing complications related to chronic inflammation, such as cardiovascular disease. These health problems are mainly due to the persistence of HIV reservoirs in the patients' long-lived memory T cells and to the constant activation of their immune system.
In a pilot study published recently in ...
MADISON, Wis. -- Scientists at the University of Wisconsin-Madison have developed a way to use a cell's own recycling machinery to destroy disease-causing proteins, a technology that could produce entirely new kinds of drugs.
Some cancers, for instance, are associated with abnormal proteins or an excess of normally harmless proteins. By eliminating them, researchers believe they can treat the underlying cause of disease and restore a healthy balance in cells.
The new technique builds on an earlier strategy by researchers and pharmaceutical companies to remove proteins residing inside of a cell, and expands on this system to include proteins ...
For the first time, scientists have documented biofluorescence in an Arctic fish species. The study, led by researchers at the American Museum of Natural History who spent hours in the icy waters off of Greenland where the red-and-green-glowing snailfish was found, is published today in the American Museum Novitates.
"Overall, we found marine fluorescence to be quite rare in the Arctic, in both invertebrate and vertebrate lineages," said John Sparks, a curator in the American Museum of Natural History's Department of Ichthyology and one of the authors of the ...
The ongoing COVID-19 pandemic changed the higher education experience for students across the United States, with more than 90 percent of institutions reporting a shift in education delivery with the arrival of COVID-19.
The rapid transition to remote study came with its own learning curve for students and faculty alike. But for many students with disabilities, the shift offered new educational modalities as well as challenges - and the hope that some changes will continue after the threat of the virus subsides.
"This was a really unique, historical moment," says Nicholas Gelbar '06 (ED), '07 MEd, '13 Ph.D., an associate research professor with the Neag School of Education. "Remote learning, ...
Millions of children weighing less than 15kg are currently denied access to Ivermectin treatment due to insufficient safety data being available to support a change to the current label indication. The WorldWide Antimalarial Resistance Network (WWARN)'s new meta-analysis published today provides evidence that supports removing this barrier and improving treatment equity. ...
DURHAM, N.H.-- Researchers at the University of New Hampshire have found that more than 50% of children in high-risk populations in the United States are not receiving behavioral health services that could improve their developmental outcomes when it comes to mental and physical health problems.
In their END ...
