(Press-News.org) BELLINGHAM, Washington, USA -- Those formerly silent walls can "talk" now: Researchers have demonstrated a simple optical technique by which audio information can be extracted from high-speed video recordings. The method uses an image-matching process based on vibration from sound waves, and is reported in an article appearing in the November issue of the journal Optical Engineering, published by SPIE, the international society for optics and photonics.
"One of the intriguing aspects of the paper is the ability to recover spoken words from a video of objects in the room," said journal Associate Editor Reiner Eschbach, a Research Fellow at Xerox Corp. "The paper shows that the sound creates minute vibrations in objects and that these vibrations ― given the right equipment ― can be picked up from a video signal. This is an interesting foray into a new application space and will, in my view, trigger interesting research in the field,"
The article, "Audio extraction from silent high-speed video using an optical technique," was authored by Zhaoyang Wang, Hieu Nguyen, and Jason Quisberth of the Department of Engineering of the Catholic University of America, and is available from the SPIE Digital Library.
The technique is based on the fact that sound waves are mechanical waves that cause air to vibrate when traveling, the paper notes. That vibration through air can cause vibration of objects located in its traveling path, especially if the objects are lightweight, thin, and flexible, such as a piece of paper. The vibrations, although usually with small amplitudes, can be detected and analyzed algorithmically, and audio reconstructed based on those calculations.
The authors used a subset-based image-correlation approach to detect the motions of points on the surface of an object, capturing target images with a high-speed camera and applying the Gauss-Newton algorithm and a few other measures to achieve very fast and highly accurate image matching. Because the detected vibrations are directly related to sound waves, a simple model was used to reconstruct the original audio information of the sound waves.
While other recent work in the area reports on more sophisticated techniques to compute motion signals, the authors chose a simpler image-matching approach to measure vibration. Because light can travel through air considerably farther than sound and can pass through glass, they anticipate that the technique may find applications such as the passive detection of conversations inside of a building from a far distance, Wang said. "We are currently improving the technique to increase its accuracy and sensitivity, make the measurements in real-time, and remove interference from other sources."
INFORMATION:
Optical Engineering is published in print and digitally in the SPIE Digital Library, which contains more than 420,000 articles from SPIE journals and proceedings, as well as more than 200 eBooks. Abstracts are freely searchable, and an increasing number of full articles in the society's 10 peer-reviewed journals are published with open access. Approximately 18,000 new research papers, eBooks, and other publications are added each year.
Michael Eismann, Senior Scientist for Electro-Optical and Infrared Sensors at the Sensors Directorate of the U.S. Air Force Research Lab, is the journal's Editor-in-Chief.
SPIE is the international society for optics and photonics, a not-for-profit organization founded in 1955 to advance light-based technologies. The Society serves nearly 256,000 constituents from approximately 155 countries, offering conferences, continuing education, books, journals, and a digital library in support of interdisciplinary information exchange, professional networking, and patent precedent. SPIE provided more than $3.2 million in support of education and outreach programs in 2013.
In what is believed to be the first interview-style qualitative study of its kind among health care providers in the trenches, a team led by a Johns Hopkins geriatrician has further documented barriers to better care of older adults as they are transferred from hospital to rehabilitation center to home, and too often back again.
Using comments and concerns drawn from in-depth interviews of 18 physicians and two home health care agency administrators -- all experienced in trying to coordinate care of older adults -- the researchers created a framework for evaluating what ...
VIDEO:
A bat lands on the rewarded object in complete darkness (movie taken in IR). The movie shows that the bat has a general knowledge of the location of the object,...
Click here for more information.
In a discovery that overturns conventional wisdom about bats, researchers reporting in the Cell Press journal Current Biology on December 4 have found that Old World fruit bats--long classified as "non-echolocating"--actually do use a rudimentary form of echolocation. Perhaps ...
A new study shows that increasing sugar in the diet of male fruit flies for just 1 or 2 days before mating can cause obesity in their offspring through alterations that affect gene expression in the embryo. There is also evidence that a similar system regulates obesity susceptibility in mice and humans. The research, which is published online December 4 in the Cell Press journal Cell, provides insights into how certain metabolic traits are inherited and may help investigators determine whether they can be altered.
Research has shown that various factors that are passed ...
New Caledonian crows--well known for their impressive stick-wielding abilities--show preferences when it comes to holding their tools on the left or the right sides of their beaks, in much the same way that people are left- or right-handed. Now researchers reporting in the Cell Press journal Current Biology on December 4 suggest that those bill preferences allow each bird to keep the tip of its tool in view of the eye on the opposite side of its head. Crows aren't so much left- or right-beaked as they are left- or right-eyed.
"If you were holding a brush in your mouth ...
Blood cancers are more common in men than in women, but it has not been clear why this is the case. A study published by Cell Press December 4th in Cell Stem Cell provides an explanation, revealing that female sex hormones called estrogens regulate the survival, proliferation, and self-renewal of stem cells that give rise to blood cancers. Moreover, findings in mice with blood neoplasms--the excessive production of certain blood cells--suggest that a drug called tamoxifen, which targets estrogen receptors and is approved for the treatment of breast cancer, may also be a ...
In a breakthrough study to be published on the December 4th issue of the prestigious scientific journal Cell, a research team led by Miguel Soares at the Instituto Gulbenkian de Ciência (IGC; Portugal) discovered that specific bacterial components in the human gut microbiota can trigger a natural defense mechanism that is highly protective against malaria transmission.
Over the past few years, the scientific community became aware that humans live under a continuous symbiotic relationship with a vast community of bacteria and other microbes that reside in the gut. ...
This discovery has a potential application in the treatment of certain blood disorders for which there is currently no cure. The study was led by Dr. Simón Méndez-Ferrer of the CNIC, working in partnership with the laboratories of Doctors Jürg Schwaller and Radek Skoda of the University Hospital in Basel (Switzerland). The study's authors have demonstrated in mice that tamoxifen, a drug already approved and widely used for the treatment of breast cancer, blocks the symptoms and the progression of a specific group of blood disorders known as myeloproliferative ...
PROVIDENCE, R.I. [Brown University] -- In a study in the journal Neuron, scientists describe a new high data-rate, low-power wireless brain sensor. The technology is designed to enable neuroscience research that cannot be accomplished with current sensors that tether subjects with cabled connections.
Experiments in the paper confirm that new capability. The results show that the technology transmitted rich, neuroscientifically meaningful signals from animal models as they slept and woke or exercised.
"We view this as a platform device for tapping into the richness of ...
The bacterium Salmonella Typhi causes typhoid fever in humans, but leaves other mammals unaffected. Researchers at University of California, San Diego and Yale University Schools of Medicine now offer one explanation -- CMAH, an enzyme that humans lack. Without this enzyme, a toxin deployed by the bacteria is much better able to bind and enter human cells, making us sick. The study is published in the Dec. 4 issue of Cell.
In most mammals (including our closest evolutionary cousins, the great apes), the CMAH enzyme reconfigures the sugar molecules found on these animals' ...
The link between obesity and cardiovascular diseases is well acknowledged. Being obese or overweight is a major risk factor for the development of elevated blood pressure, and cardiovascular diseases. But it has net been known how obesity increases the risk of high blood pressure, making it difficult to develop evidence based therapies for obesity, hypertension and heart disease.
In a ground-breaking study, published today in the prestigious journal, Cell (embargo midday EST), researchers from Monash University in Australia, Warwick, Cambridge in the UK and several American ...
