(Press-News.org) Training the artificial intelligence models that underpin web search engines, power smart assistants and enable driverless cars, consumes megawatts of energy and generates worrying carbon dioxide emissions. But new ways of training these models are proven to be greener.
Artificial intelligence models are used increasingly widely in today's world. Many carry out natural language processing tasks - such as language translation, predictive text and email spam filters. They are also used to empower smart assistants such as Siri and Alexa to 'talk' to us, and to operate driverless cars.
But to function well these models have to be trained on large sets of data, a process that includes carrying out many mathematical operations for every piece of data they are fed. And the data sets they are being trained on are getting ever larger: one recent natural language processing model was trained on a data set of 40 billion words.
As a result, the energy consumed by the training process is soaring. Most AI models are trained on specialised hardware in large data centres. According to a recent paper in the journal Science, the total amount of energy consumed by data centres made up about 1% of global energy use over the past decade - equalling roughly 18 million US homes. And in 2019, a group of researchers at the University of Massachusetts estimated that training one large AI model used in natural language processing could generate around the same amount of CO2 emissions as five cars would generate over their total lifetime.
Concerned by this, researchers at the University of Cambridge set out to investigate more energy-efficient approaches to training AI models. Working with collaborators at the University of Oxford, University College London, and Avignon Université, they explored the environmental impact of a different form of training - called federated learning - and discovered that it had a significantly greener impact.
Instead of training the models in data centres, federated learning involves training models across a large number of individual machines. The researchers found that this can lead to lower carbon emissions than traditional learning.
Senior Lecturer Dr Nic Lane explains how it works when the training is performed not inside large data centres but over thousands of mobile devices - such as smartphones - where the data is usually collected by the phone users themselves.
"An example of an application currently using federated learning is the next-word prediction in mobile phones," he says. "Each smartphone trains a local model to predict which word the user will type next, based on their previous text messages. Once trained, these local models are then sent to a server. There, they are aggregated into a final model that will then be sent back to all users."
And this method has important privacy benefits as well as environmental benefits, points out Dr Pedro Porto Buarque De Gusmao, a postdoctoral researcher working with Dr Lane.
"Users might not want to share the content of their texts with a third party," he explains. "In federated learning, we can keep data local and use the collective power of millions of mobile devices together to train AI models without users' raw data ever leaving the phone."
"And besides these privacy-related gains," says Dr Lane, "in our recent research, we have shown that federated learning can also have a positive impact in reducing carbon emissions.
"Although smartphones have much less processing power than the hardware accelerators used in data centres, they don't require as much cooling power as the accelerators do. That's the benefit of distributing the training of models across a wide pool of devices."
The researchers recently co-authored a paper on this called 'Can Federated Learning save the planet?' and will be discussing their findings at an international research conference, the Flower Summit 2021, on 11 May.
In their paper, they offer the first-ever systematic study of the carbon footprint of federated learning. They measured the carbon footprint of a federated learning setup by training two models -- one in image classification, the other in speech recognition - using a server and two chipsets popular in the simple devices targeted by federated methods. They recorded the energy consumption during training, and how it might vary depending on where in the world the chipsets and server were located.
They found that while there was a difference between CO2 emission factors among countries, federated learning under many common application settings was reliably 'cleaner' than centralised training.
Training a model to classify images in a large image dataset, they found any federated learning setup in France emitted less CO2 than any centralised setup in both China and the US. And in training the speech recognition model, federated learning was more efficient than centralised training in any country.
Such results are further supported by an expanded set of experiments in a follow-up study ('A first look into the carbon footprint of federated learning') by the same lab that explores an even wider variety of data sets and AI models. And this research also provides the beginnings of necessary formalism and algorithmic foundation of even lower carbon emissions for federated learning in the future.
Based on their research, the researchers have made available a first-of-its-kind 'Federated Learning Carbon Calculator' so that the public and other researchers can estimate how much CO2 is produced by any given pool of devices. It allows users to detail the number and type of devices they are using, which country they are in, which datasets and upload/download speeds they are using and the number of times each device will train on its own data before sending its model for aggregation.
They also offer a similar calculator for estimating the carbon emissions of centralised machine learning.
"The development and usage of AI is playing an increasing role in the tragedy that is climate change," says Dr Lane, "and this problem will only worsen as this technology continues to proliferate through society. We urgently need to address this which is why we are keen to share our findings showing that federated learning methods can produce less CO2 than data centres under important application scenarios.
"But even more importantly, our research also shines a light as to how federated learning should evolve towards being even more broadly environmentally friendly. Decentralized methods like this will be key in the invention of future sustainable forms of AI in the years ahead."
To observe the swift neuronal signals in a fish brain, scientists have started to use a technique called light-field microscopy, which makes it possible to image such fast biological processes in 3D. But the images are often lacking in quality, and it takes hours or days for massive amounts of data to be converted into 3D volumes and movies.
Now, EMBL scientists have combined artificial intelligence (AI) algorithms with two cutting-edge microscopy techniques - an advance that shortens the time for image processing from days to mere seconds, while ensuring that the resulting images are crisp and accurate. The findings are published in Nature Methods.
"Ultimately, we were able to take 'the best of both worlds' in this approach," says Nils Wagner, one of the paper's two lead ...
PHILADELPHIA-- A gene called GAS2 plays a key role in normal hearing, and its absence causes severe hearing loss, according to a study led by researchers in the Perelman School of Medicine at the University of Pennsylvania.
The researchers, whose findings are published online today in Developmental Cell, discovered that the protein encoded by GAS2 is crucial for maintaining the structural stiffness of support cells in the inner ear that normally help amplify incoming sound waves. They showed that inner ear support cells lacking functional GAS2 lose their amplifier abilities, causing severe hearing ...
Depressive symptoms are more common in teenage girls than in their male peers. However, boys' mental health appears to be affected more if they suffer from obesity. Irrespective of gender, bullying is a considerably greater risk factor than overweight for developing depressive symptoms. These conclusions are drawn by researchers at Uppsala University who monitored adolescents for six years in a questionnaire study, now published in the Journal of Public Health.
"The purpose of our study was to investigate the connection ...
Nearly four-fifths of 16- and 17-year-olds have encountered pornographic content on the Internet
Pornography is a multibillion-dollar business. Pornographic content is virtually ubiquitous on the Internet, and surveys suggest that 25% of all searches lead to explicit content. Given the size of the market, it's not surprising that young people are drawn to such sites, which are only a couple of clicks away.
Professor Neil Thurman of the Department of Media and Communication (IfKW) at Ludwig-Maximilians-Universitaet (LMU) in Munich, in collaboration with statistician Fabian Obster (Universität der Bundeswehr München), has carried out a study on the use of pornographic sites by young people. Based on a survey involving a sample ...
Great reed warblers normally migrate by night during its month-long migration from northern Europe to Sub-Saharan Africa. However, researchers have now discovered that during the few occasions when it continues to fly during daytime, it flies at extremely high altitudes (up to 6300 meters). One possible explanation for this unexpected and consistent behaviour could be that the birds want to avoid overheating. The study is published in Science.
Most of the many millions of songbirds that migrate every year between Europe and Africa fly by night and spend the daytime hours resting and eating. Some species, which normally only fly by night, occasionally fly for over 24 consecutive ...
If you are reading this, chances are you have several other tabs open in your browser that you mean to get to eventually.
Internet browser tabs are a major source of friction on the internet. People love them. People hate them. For some users, tabs bring order and efficiency to their web browsing. For others, they spiral out of control, shrinking at the top of the screen as their numbers expand.
A research team at Carnegie Mellon University recently completed the first in-depth study of browser tabs in more than a decade. They found that many people struggle with tab overload, an underlying reason being that while tabs serve a variety of functions, ...
ITHACA, N.Y. - An 18.5 million-year-old fossil found in Panama provides evidence of a new species and is the oldest reliable example of a climbing woody vine known as a liana from the soapberry family. The discovery sheds light on the evolution of climbing plants.
The new species, named Ampelorhiza heteroxylon, belongs to a diverse group of tropical lianas called Paullinieae, within the soapberry family (Sapindaceae). More than 475 species of Paullinieae live in the tropics today.
Researchers identified the species from fossilized roots that revealed features known to be unique to the wood of modern climbing vines, adaptations that allow them to twist, grow and climb.
The study, "Climbing ...
The functions of water-dominated ecosystems can be considerably influenced and changed by hydrological fluctuation. The varying states of redox-active substances are of crucial importance here. Researchers at the University of Bayreuth have discovered this, in cooperation with partners from the Universities of Tübingen and Bristol and the Helmholtz Centre for Environmental Research, Halle-Leipzig. They present their discovery in the journal Nature Geoscience. The new study enables a more precise understanding of the biogeochemical processes that contribute to the degradation of pollutants and the reduction of greenhouse gas ...
More than ever, patients are using telehealth to ask doctors and nurses about worrying blood-pressure readings, nauseating migraines and stubborn foot ulcers. But for patients with chronic conditions, how frequent should telehealth appointments be? Can that frequency change? Under what conditions?
West Virginia University researcher Jennifer Mallow is trying to answer these questions. In a new project, she and her colleagues completed a systematic review of studies that dealt with telehealth and chronic conditions. They found that--in general--telehealth services benefitted patients more if they ...
A comprehensive review into what we know about COVID-19 and the way it functions suggests the virus has a unique infectious profile, which explains why it can be so hard to treat and why some people experience so-called "long-COVID", struggling with significant health issues months after infection.
There is growing evidence that the virus infects both the upper and lower respiratory tracts - unlike "low pathogenic" human coronavirus sub-species, which typically settle in the upper respiratory tract and cause cold-like symptoms, or "high pathogenic" viruses such as those that cause SARS and ARDS, which typically settle in the lower respiratory tract.
Additionally, more frequent multi-organ impacts, and blood clots, and ...