PRESS-NEWS.org - Press Release Distribution
PRESS RELEASES DISTRIBUTION

'Audeo' teaches artificial intelligence to play the piano

2021-02-04
(Press-News.org) Anyone who's been to a concert knows that something magical happens between the performers and their instruments. It transforms music from being just "notes on a page" to a satisfying experience.

A University of Washington team wondered if artificial intelligence could recreate that delight using only visual cues -- a silent, top-down video of someone playing the piano. The researchers used machine learning to create a system, called Audeo, that creates audio from silent piano performances. When the group tested the music Audeo created with music-recognition apps, such as SoundHound, the apps correctly identified the piece Audeo played about 86% of the time. For comparison, these apps identified the piece in the audio tracks from the source videos 93% of the time.

The researchers presented Audeo Dec. 8 at the NeurIPS 2020 conference.

"To create music that sounds like it could be played in a musical performance was previously believed to be impossible," said senior author Eli Shlizerman, an assistant professor in both the applied mathematics and the electrical and computer engineering departments. "An algorithm needs to figure out the cues, or 'features,' in the video frames that are related to generating music, and it needs to 'imagine' the sound that's happening in between the video frames. It requires a system that is both precise and imaginative. The fact that we achieved music that sounded pretty good was a surprise."

Audeo uses a series of steps to decode what's happening in the video and then translate it into music. First, it has to detect which keys are pressed in each video frame to create a diagram over time. Then it needs to translate that diagram into something that a music synthesizer would actually recognize as a sound a piano would make. This second step cleans up the data and adds in more information, such as how strongly each key is pressed and for how long.

"If we attempt to synthesize music from the first step alone, we would find the quality of the music to be unsatisfactory," Shlizerman said. "The second step is like how a teacher goes over a student composer's music and helps enhance it."

The researchers trained and tested the system using YouTube videos of the pianist Paul Barton. The training consisted of about 172,000 video frames of Barton playing music from well-known classical composers, such as Bach and Mozart. Then they tested Audeo with almost 19,000 frames of Barton playing different music from these composers and others, such as Scott Joplin.

Once Audeo has generated a transcript of the music, it's time to give it to a synthesizer that can translate it into sound. Every synthesizer will make the music sound a little different -- this is similar to changing the "instrument" setting on an electric keyboard. For this study, the researchers used two different synthesizers.

"Fluidsynth makes synthesizer piano sounds that we are familiar with. These are somewhat mechanical-sounding but pretty accurate," Shlizerman said. "We also used PerfNet, a new AI synthesizer that generates richer and more expressive music. But it also generates more noise."

Audeo was trained and tested only on Paul Barton's piano videos. Future research is needed to see how well it could transcribe music for any musician or piano, Shlizerman said.

"The goal of this study was to see if artificial intelligence could generate music that was played by a pianist in a video recording -- though we were not aiming to replicate Paul Barton because he is such a virtuoso," Shlizerman said. "We hope that our study enables novel ways to interact with music. For example, one future application is that Audeo can be extended to a virtual piano with a camera recording just a person's hands. Also, by placing a camera on top of a real piano, Audeo could potentially assist in new ways of teaching students how to play."

INFORMATION:

Kun Su and Xiulong Liu, both doctoral students in electrical and computer engineering, are co-authors on this paper. This research was funded by the Washington Research Foundation Innovation Fund as well as the applied mathematics and electrical and computer engineering departments.

For more information, contact Shlizerman at shlizee@uw.edu.



ELSE PRESS RELEASES FROM THIS DATE:

In symbiosis: Plants control the genetics of microbes

In symbiosis: Plants control the genetics of microbes
2021-02-04
Researchers from the University of Ottawa have discovered that plants may be able to control the genetics of their intimate root symbionts - the organism with which they live in symbiosis - thereby providing a better understanding of their growth. In addition to having a significant impact on all terrestrial ecosystems, their discovery may lead to improved eco-friendly agricultural applications. We talked to research lead Nicolas Corradi, Associate Professor in the Department of Biology and Research Chair in Microbial Genomics at the University of Ottawa, ...

Experimental vaccine blunts the deadliest of synthetic opioids

2021-02-04
LA JOLLA, CA--As the opioid epidemic raged on with an even greater force during COVID-19, the Scripps Research laboratory of chemist Kim Janda, PhD, has been working on new therapeutic interventions that may be able to prevent the bulk of deaths from opioid overdose. Janda and his team have developed experimental vaccines that have shown in rodents to blunt the deadly effects of fentanyl--which has been driving the boom in opioid deaths--as well as its even more fatal cousin, carfentanil, a growing source of overdoses and a chemical terrorist threat. "Synthetic opioids are not only extremely deadly, but also addictive and easy to manufacture, making them a formidable public health threat, especially when the coronavirus crisis is negatively impacting mental health," says Janda, the Ely ...

Switching nanolight on and off

Switching nanolight on and off
2021-02-04
A team of researchers led by Columbia University has developed a unique platform to program a layered crystal, producing imaging capabilities beyond common limits on demand. The discovery is an important step toward control of nanolight, which is light that can access the smallest length scales imaginable. The work also provides insights for the field of optical quantum information processing, which aims to solve difficult problems in computing and communications.   "We were able to use ultrafast nano-scale microscopy to discover a new way to control our crystals with light, turning elusive photonic properties on and off at will," said Aaron Sternbach, postdoctoral researcher ...

Scientists establish multiple primate models of SARS-CoV-2 airborne infection

2021-02-04
Army scientists evaluated three nonhuman primate species as potential models of SARS-CoV-2 airborne infection, according to results published online this week in PLOS ONE. Their work demonstrates that any of these species may be useful for testing vaccines and therapies in response to the COVID-19 pandemic, which has resulted in over 104 million cases and more than 2 million deaths worldwide in the past year. Given the global impact of COVID-19, experts are working rapidly to develop medical countermeasures, and testing in animal models is critically important to evaluate the efficacy of these products. Recent studies suggest that aerosol ...

New test provides fast and accurate diagnosis of liposarcomas

New test provides fast and accurate diagnosis of liposarcomas
2021-02-04
Philadelphia, February 4, 2021 - Researchers have leveraged the latest advances in RNA technology and machine learning methods to develop a gene panel test that allows for highly accurate diagnosis of the most common types of liposarcoma. It quickly and reliably distinguishes benign lipomas from liposarcomas and can be performed in laboratories at a lower cost than current "gold standard" tests. The new assay is described in The Journal of Molecular Diagnosis, published by Elsevier. "Liposarcomas are a type of malignant cancer that is difficult to diagnose because, even under a microscope, it is hard to differentiate liposarcomas from benign tumors or other types ...

New study examines addiction medicine treatment in Vietnam

2021-02-04
An assessment published this week in the journal The Lancet HIV provides new insight about an initiative to integrate treatment of opioid use disorder along with HIV in Vietnam. The study marks one of the first scientifically robust assessments of a new model of treating HIV in lower or middle income countries where injection drug use is a major cause of HIV infection. It also suggests the importance of building support for peer and community connections to tackle the opioid epidemic that continues to ravage the United States in the midst of the COVID-19 pandemic. The study was led by scientists and physicians at Hanoi Medical University and Oregon Health & Science University. "Our study suggests that countries that ...

Imaging the first moments of a body plan emerging in the embryo

2021-02-04
WOODS HOLE, Mass. -- Egg cells start out as round blobs. After fertilization, they begin transforming into people, dogs, fish, or other animals by orienting head to tail, back to belly, and left to right. Exactly what sets these body orientation directions has been guessed at but not seen. Now researchers at the Marine Biological Laboratory (MBL) have imaged the very beginning of this cellular rearrangement, and their findings help answer a fundamental question. "The most interesting and mysterious part of developmental biology is the origin of the body axis in animals," said researcher Tomomi Tani. An MBL scientist in the Eugene Bell Center at the time of the research, Tani is now with Japan's National Institute of Advanced Industrial Science and Technology. The work by Tani and ...

Mysterious organic scum boosts chemical reaction efficiency, may reduce chemical waste

Mysterious organic scum boosts chemical reaction efficiency, may reduce chemical waste
2021-02-04
CHAMPAIGN, Ill. -- Chemical manufacturers frequently use toxic solvents such as alcohols and benzene to make products like pharmaceuticals and plastics. Researchers are examining a previously overlooked and misunderstood phenomenon in the chemical reactions used to make these products. This discovery brings a new fundamental understanding of catalytic chemistry and a steppingstone to practical applications that could someday make chemical manufacturing less wasteful and more environmentally sound. The study led by University of Illinois Urbana-Champaign researcher David Flaherty, University of Minnesota, Twin Cities researcher Matthew Neurock and Virginia Tech researcher Ayman Karim is published in the journal Science. Combining ...

An optical coating like no other

An optical coating like no other
2021-02-04
For more than a century, optical coatings have been used to better reflect certain wavelengths of light from lenses and other devices or, conversely, to better transmit certain wavelengths through them. For example, the coatings on tinted eyeglasses reflect, or "block out," harmful blue light and ultraviolet rays. But until now, no optical coating had ever been developed that could simultaneously reflect and transmit the same wavelength, or color. In a paper in Nature Nanotechnology, researchers at the University of Rochester and Case Western Reserve University describe a new class of optical coatings, so-called Fano Resonance Optical Coatings (FROCs), that can be used on filters to reflect and transmit colors of remarkable purity. In addition, ...

Comb-like etching regulated growth for large-area graphene nanoribbon arrays

Comb-like etching regulated growth for large-area graphene nanoribbon arrays
2021-02-04
The rapid development of silicon-based transistors leads to its integration getting closer to the limit of Moore's law. Graphene is expected to become the next generation of mainstream chip materials due to its ultra-high carrier mobility. However, it is difficult to obtain a high on/off current ratio for intrinsic graphene-based transistor owing to the absence of bandgap. Graphene nanoribbons, which possess a tunable bandgap and unique optoelectrical properties, have attracted extensive attention and exploration. Nowadays, the preparation of graphene nanoribbons is underdeveloped, and common strategies include the clip of carbon materials (graphene films, carbon nanotubes, or graphite) ...

LAST 30 PRESS RELEASES:

Pink skies

Monkeys are world’s best yodellers - new research

Key differences between visual- and memory-led Alzheimer’s discovered

% weight loss targets in obesity management – is this the wrong objective?

An app can change how you see yourself at work

NYC speed cameras take six months to change driver behavior, effects vary by neighborhood, new study reveals

New research shows that propaganda is on the rise in China

Even the richest Americans face shorter lifespans than their European counterparts, study finds

Novel genes linked to rare childhood diarrhea

New computer model reveals how Bronze Age Scandinavians could have crossed the sea

Novel point-of-care technology delivers accurate HIV results in minutes

Researchers reveal key brain differences to explain why Ritalin helps improve focus in some more than others

Study finds nearly five-fold increase in hospitalizations for common cause of stroke

Study reveals how alcohol abuse damages cognition

Medicinal cannabis is linked to long-term benefits in health-related quality of life

Microplastics detected in cat placentas and fetuses during early pregnancy

Ancient amphibians as big as alligators died in mass mortality event in Triassic Wyoming

Scientists uncover the first clear evidence of air sacs in the fossilized bones of alvarezsaurian dinosaurs: the "hollow bones" which help modern day birds to fly

Alcohol makes male flies sexy

TB patients globally often incur "catastrophic costs" of up to $11,329 USD, despite many countries offering free treatment, with predominant drivers of cost being hospitalization and loss of income

Study links teen girls’ screen time to sleep disruptions and depression

Scientists unveil starfish-inspired wearable tech for heart monitoring

Footprints reveal prehistoric Scottish lagoons were stomping grounds for giant Jurassic dinosaurs

AI effectively predicts dementia risk in American Indian/Alaska Native elders

First guideline on newborn screening for cystic fibrosis calls for changes in practice to improve outcomes

Existing international law can help secure peace and security in outer space, study shows

Pinning down the process of West Nile virus transmission

UTA-backed research tackles health challenges across ages

In pancreatic cancer, a race against time

Targeting FGFR2 may prevent or delay some KRAS-mutated pancreatic cancers

[Press-News.org] 'Audeo' teaches artificial intelligence to play the piano