(Press-News.org) Computers will someday soon automatically provide short video digests of a day in your life, your family vacation or an eight-hour police patrol, say computer scientists at The University of Texas at Austin.
The researchers are working to develop tools to help make sense of the vast quantities of video that are going to be produced by wearable camera technology such as Google Glass and Looxcie.
"The amount of what we call 'egocentric' video, which is video that is shot from the perspective of a person who is moving around, is about to explode," said Kristen Grauman, associate professor of computer science in the College of Natural Sciences. "We're going to need better methods for summarizing and sifting through this data."
Grauman and her colleagues developed a superior technique that uses machine learning to automatically analyze recorded videos and assemble a better short "story" of the footage than what is available from existing methods.
Better video summarization should prove important in helping military commanders managing data coming in from soldiers' cameras, investigators trying to sift through cellphone video data in the wake of disasters like the Boston Marathon bombing, and senior citizens using video summaries of their days to compensate for memory loss, said Grauman.
"There's research showing that if people suffering from memory loss wear a camera that takes a snapshot once a minute, and then they review those images at the end of the day, it can help their recall," said Grauman. "That's pretty inspiring. What if instead of images that were selected just because they were a minute apart, they had a video or photographic summary that was selected because it told a good story? Maybe that would help even more. That's the kind of thing we're hoping to achieve."
Grauman, her postdoc Lu Zheng and doctoral student Yong Jae Lee presented their method, which they call "story-driven" video summarization, at the IEEE Conference on Computer Vision and Pattern Recognition this summer.
VIDEO:
This 12-frame summary is distilled from an uninterrupted three hour video taken by a person going about her day while wearing a $200 Looxcie camera.
Click here for more information.
Their findings are based on video amassed by volunteers wearing commercially available Looxcie cameras, which cost about $200, record five hours of video at a stretch, connect to smartphones and fit in an ear as a large Bluetooth device does.
"The task is to take a very long video and automatically condense it into very short video clips, or a series of stills, that convey the essence of the story," said Grauman. "To do that, though, we first have to ask: What makes a good visual story? Our answer is that beyond displaying important persons, objects and scenes, it must also convey how one thing leads to the next."
To tackle the challenge, Grauman and her colleagues took a two-step approach. The first step involved using machine learning techniques to teach their system to "score" the significance of objects in view based on egocentric factors such as how often the objects appeared in the center of the frame, which is a good proxy for where the camera wearer's gaze is, or whether they are touched by the wearer's hands.
"If you give us a region in the video, then we will give back an importance level, based on all those properties that we have extracted and learned how to combine," said Grauman. "So at that point you can select frames that will maximize the importance."
The next step was to use those important frames, through the video, and look for early ones that influence later ones. To do that they adapted a method developed by researchers at Carnegie Mellon University that could predict how one news article leads to another, assembling a series of articles to transition from a starting point to a known end point.
For the text work, researchers used word frequencies and correlations across articles to quantify influence. For the video work, Grauman and Lu used their significant objects and frames to do the same. Then they were able to identify a chain of video clips that efficiently filled in the story from beginning to end.
"We ran human 'taste tests' comparing our method to previous methods," said Grauman, "and between 75 and 90 percent of people evaluating the summaries, depending on the datasets and method being compared, found that our system is superior."
Grauman said that as video summarization techniques continue to improve, they will become invaluable aids not just to people with very specialized needs, like police investigators and those suffering from memory loss, but to everyday Web surfers as well.
"My hope is that we'll be able to get video browsing much closer to what we experience with image browsing," she said. "Consider browsing 50 images on a webpage. It's manageable, since you can scroll down and see them all in one pass. Now imagine trying to browse 50 videos online. It's simply not efficient. We need summarization algorithms in order to improve video search considerably."
INFORMATION:
Researchers use machine learning to boil down the stories that wearable cameras are telling
New methods automatically distill hours of footage into short summaries
2013-09-13
ELSE PRESS RELEASES FROM THIS DATE:
EORTC at 2013 ECCO-ESMO-ESTRO meeting in Amsterdam
2013-09-13
The EORTC will have an active presence at the 2013 ECCO-ESMO-ESTRO Meeting in Amsterdam, The Netherlands, from 27 September to 01 October 2013 and would like to call your attention to the following presentations.
Society Session
F. Meunier and R. Stupp are co-Chairs of the session "European Organisation for Research and Treatment of Cancer (EORTC) - Multidisciplinary Cancer Clinical Research: What are we up to in 2013?" on Saturday, 28 September 2013 from 16:00 - 18:00 in Room G102.
Special sessions
J. Bogaerts will chair a Special session, "Dilemma of Crossover ...
Software may be able to take over from hardware in managing caches
2013-09-13
CAMBRIDGE, Mass-- In today's computers, moving data to and from main memory consumes so much time and energy that microprocessors have their own small, high-speed memory banks, known as "caches," which store frequently used data. Traditionally, managing the caches has required fairly simple algorithms that can be hard-wired into the chips.
In the 21st century, however, in order to meet consumers' expectations for steadily increasing computational power, chipmakers have had to begin equipping their chips with more and more cores, or processing units. And as cores proliferate, ...
NASA satellite sees 2 vortices circling newborn Tropical Storm Man-yi's center
2013-09-13
NASA's Terra satellite passed over newborn Tropical Storm Man-yi and captured and image that clearly showed two vortices rotating around a large center of circulation. Man-yi formed on Sept. 12 in the northwestern Pacific Ocean as the sixteenth tropical depression and by Sept. 13 it strengthened into a tropical storm.
When NASA's Terra satellite passed over newborn Tropical Storm Man-yi in the northwestern Pacific Ocean on Sept. 13 at 01:15 UTC, the Moderate Resolution Imaging Spectroradiometer or MODIS instrument captured a visible image of the storm. The MODIS image ...
Catalysts team up with textiles
2013-09-13
This news release is available in German.
In future, it will be much easier to produce some active pharmaceutical substances and chemical compounds than was the case to date. An international team working with chemists from the Max-Planck-Institut für Kohlenforschung in Mülheim an der Ruhr have immobilised various catalysts on nylon in a very simple way. Catalysts mediate between the reagents in a chemical reaction and control the process leading to the desired end product. When textile material is used as a support for the chemical auxiliaries, the reaction can ...
NIH clinical study establishes human model of influenza pathogenesis
2013-09-13
WHAT:
A National Institutes of Health (NIH) clinical study of healthy adult volunteers who consented to be infected with the 2009 H1N1 influenza virus under carefully controlled conditions has provided researchers with concrete information about the minimum dose of virus needed to produce mild-to-moderate illness. The study also gives a clearer picture of how much time elapses between a known time of infection, the start of viral shedding (a signal of contagiousness), the development of an immune response, and the onset and duration of influenza symptoms. The data obtained ...
The '50-50' chip: Memory device of the future?
2013-09-13
WASHINGTON, D.C. Sept. 13, 2013 -- A new, environmentally-friendly electronic alloy consisting of 50 aluminum atoms bound to 50 atoms of antimony may be promising for building next-generation "phase-change" memory devices, which may be the data-storage technology of the future, according to a new paper published in the journal Applied Physics Letters, which is produced by AIP Publishing.
Phase-change memory is being actively pursued as an alternative to the ubiquitous flash memory for data storage applications, because flash memory is limited in its storage density and ...
NASA sees southwesterly wind shear weakened hurricane Humberto
2013-09-13
Southwesterly wind shear has taken its toll on hurricane Humberto, and NASA's TRMM satellite noticed that in rainfall data.
When NASA's Tropical Rainfall Measuring Mission or TRMM satellite passed over Hurricane Humberto on September 12, 2013 at 1625 UTC/12:25 p.m. EDT the eye was no longer visible. An analysis derived from
TRMM Microwave Imager (TMI) and Precipitation Radar (PR) data showed that most of the precipitation with Humberto was located in the northwestern quadrant, pushed there by the strong southwesterly wind shear.
TRMM found that the most intense rain ...
To touch the microcosmos
2013-09-13
WASHINGTON, D.C. Sept. 13, 2013 -- What if you could reach through a microscope to touch and feel the microscopic structures under the lens? In a breakthrough that may usher in a new era in the exploration of the worlds that are a million times smaller than human beings, researchers at Université Pierre et Marie Curie in France have unveiled a new technique that allows microscope users to manipulate samples using a technology known as "haptic optical tweezers."
Featured in the journal Review of Scientific Instruments, which is produced by AIP Publishing, the new technique ...
Diets low in polyunsaturated fatty acids may be a problem for youngsters
2013-09-13
In the first study to closely examine the polyunsaturated fatty acid (PUFA) intake among U.S. children under the age of 5, Sarah Keim, PhD, principal investigator in the Center for Biobehavioral Health at The Research Institute at Nationwide Children's Hospital, has found what might be a troubling deficit in the diet of many youngsters. The study, published online today by Maternal and Child Nutrition, used data on nearly 2500 children age 12 to 60 months from the U.S. National Health and Nutrition Examination Survey.
PUFAs are essential to human health. A proper ratio ...
Immune to aging
2013-09-13
This news release is available in German.
While ageing remains an inevitable fact of life, Max Planck researchers have discovered a microbe that stays forever young by rejuvenating every time it reproduces. The findings, published in Current Biology, provide fundamental insights into the mechanisms of aging.
In general, even symmetrically dividing microbes do not split into two exactly identical halves. Detailed investigations revealed that there are mechanisms in place that ensure that one half receives older, often defective, cell material, whereas the other ...
LAST 30 PRESS RELEASES:
New take on immunotherapy reinvigorates T cells by blocking uptake of energy-sapping cancer byproducts
How much climate change is in the weather?
Flagship AI-ready dataset released in type 2 diabetes study
Shaking it up: An innovative method for culturing microbes in static liquid medium
Greener and cleaner: Yeast-green algae mix improves water treatment
Acquired immune thrombotic thrombocytopenic purpura (TTP) associated with inactivated COVID-19 vaccine CoronaVac
CIDEC as a novel player in abdominal aortic aneurysm formation
Artificial intelligence: a double-edged sword for the environment?
Current test accommodations for students with blindness do not fully address their needs
Wide-incident-angle wideband radio-wave absorbers boost 5G and beyond 5G applications
A graph transformer with boundary-aware attention for semantic segmentation
C-Path announces key leadership appointments in neurodegenerative disease research
First-of-its-kind analysis of U.S. national data reveals significant disparities in individual well-being as measured by lifespan, education, and income
Exercise programs help cut new mums’ ‘baby blues’ severity and major depression risk
Gut microbiome changes linked to onset of clinically evident rheumatoid arthritis
Signals from the gut could transform rheumatoid arthritis treatment
Pioneering research reveals some of the world’s least polluting populations are at much greater risk of flooding fuelled by climate change
UK’s health data should be recognized as critical national infrastructure, says independent review
A 36-gene predictive score of anti-cancer drug resistance anticipates cancer therapy outcomes
Someone flirts with your spouse. Does that make your partner appear more attractive?
Hourglass-shaped stent could ease severe chest pain from microvascular disease
United Nations ratifies framework to protect people on cash app
Oklahoma State basketball team joins the Nation of Lifesavers
Power of aesthetic species on social media boosts wildlife conservation efforts, say experts
Researchers develop robotic sensory cilia that monitor internal biomarkers to detect and assess airway diseases
Could crowdsourcing hold the key to early wildfire detection?
Reconstruction of historical seasonal influenza patterns and individual lifetime infection histories in humans based on antibody profiles
New study traces impact of COVID-19 pandemic on global movement and evolution of seasonal flu
Presenting a Janus channel of membranes for complete oil-and-water separation
COVID-19 restrictions altered global dispersal of influenza viruses
[Press-News.org] Researchers use machine learning to boil down the stories that wearable cameras are tellingNew methods automatically distill hours of footage into short summaries