(Press-News.org) Computer vision technology is increasingly used in areas such as automatic surveillance systems, self-driving cars, facial recognition, healthcare and social distancing tools. Users require accurate and reliable visual information to fully harness the benefits of video analytics applications but the quality of the video data is often affected by environmental factors such as rain, night-time conditions or crowds (where there are multiple images of people overlapping with each other in a scene). Using computer vision and deep learning, a team of researchers led by Yale-NUS College Associate Professor of Science (Computer Science) Robby Tan, who is also from the National University of Singapore's (NUS) Faculty of Engineering, has developed novel approaches that resolve the problem of low-level vision in videos caused by rain and night-time conditions, as well as improve the accuracy of 3D human pose estimation in videos.
The research was presented at the 2021 Conference on Computer Vision and Pattern Recognition (CVPR), a top ranked computer science conference.
Combating visibility issues during rain and night-time conditions
Night-time images are affected by low light and man-made light effects such as glare, glow, and floodlights, while rain images are affected by rain streaks or rain accumulation (or rain veiling effect).
"Many computer vision systems like automatic surveillance and self-driving cars, rely on clear visibility of the input videos to work well. For instance, self-driving cars cannot work robustly in heavy rain and CCTV automatic surveillance systems often fail at night, particularly if the scenes are dark or there is significant glare or floodlights," explained Assoc Prof Tan.
In two separate studies, Assoc Prof Tan and his team introduced deep learning algorithms to enhance the quality of night-time videos and rain videos, respectively. In the first study, they boosted the brightness yet simultaneously suppressed noise and light effects (glare, glow and floodlights) to yield clear night-time images. [See Figure 1a below] This technique is new and addresses the challenge of clarity in night-time images and videos when the presence of glare cannot be ignored. In comparison, the existing state-of-the-art methods fail to handle glare. [See Figure 1b below]
Night-time images
In tropical countries like Singapore where heavy rain is common, the rain veiling effect can significantly degrade the visibility of videos. In the second study, the researchers introduced a method that employs a frame alignment, which allows them to obtain better visual information without being affected by rain streaks that appear randomly in different frames and affect the quality of the images. Subsequently, they used a moving camera to employ depth estimation in order to remove the rain veiling effect caused by accumulated rain droplets. [See Figure 2 below]. Unlike existing methods, which focus on removing rain streaks, the new methods can remove both rain streaks and the rain veiling effect at the same time.
Rain images
Figure 2: Top image shows the input (existing method), middle image shows the intermediate output of removing rain streaks, and bottom image shows the final output of removing both rain streaks and the rain veiling effect using the Yale-NUS research team's new method.
3D Human Pose Estimation: Tackling inaccuracy caused by overlapping, multiple humans in videos
At the CVPR conference, Assoc Prof Tan also presented his team's research on 3D human pose estimation, which can be used in areas such as video surveillance, video gaming, and sports broadcasting.
In recent years, 3D multi-person pose estimation from a monocular video (video taken from a single camera) is increasingly becoming an area of focus for researchers and developers. Instead of using multiple cameras to take videos from different locations, monocular videos offer more flexibility as these can be taken using a single, ordinary camera - even a mobile phone camera.
However, accuracy in human detection is affected by high activity, i.e. multiple individuals within the same scene, especially when individuals are interacting closely or when they appear to be overlapping with each other in the monocular video.
In this third study, the researchers estimate 3D human poses from a video by combining two existing methods, namely, a top-down approach or a bottom-up approach. By combining the two approaches, the new method can produce more reliable pose estimation in multi-person settings and handle distance between individuals (or scale variations) more robustly.
The researchers involved in the three studies include members of Assoc Prof Tan's team at the NUS Department of Electrical and Computer Engineering where he holds a joint appointment, and his collaborators from City University of Hong Kong, ETH Zurich and Tencent Game AI Research Center. His laboratory focuses on research in computer vision and deep learning, particularly in the domains of low level vision, human pose and motion analysis, and applications of deep learning in healthcare.
"As a next step in our 3D human pose estimation research, which is supported by the National Research Foundation, we will be looking at how to protect the privacy information of the videos. For the visibility enhancement methods, we strive to contribute to advancements in the field of computer vision, as they are critical to many applications that can affect our daily lives, such as enabling self-driving cars to work better in adverse weather conditions," said Assoc Prof Tan.
INFORMATION:
Researchers at the RIKEN Center for Integrative Medical Sciences (IMS) have discovered that acetate, a major metabolite produced by some intestinal bacteria, is involved in regulating other intestinal bacteria. Specifically, experiments showed that acetate could trigger an immune response against potentially harmful bacteria. The findings, published in the scientific journal Nature, will lead to the development of new ways to regulate the balance of intestinal bacteria.
You may be surprised to know that 40 trillion important bacteria live in our intestines. They help keep us healthy by producing essential nutrients and eliminating foreign pathogens. On the other ...
Some COVID-19 patients who experience acute respiratory failure respond by significantly increasing their respiratory effort - breathing faster and more deeply
There is concern among some doctors that this level of respiratory effort can lead to further damage to these patients' lungs.
Working with an international team of leading intensive care clinicians, engineering researchers at the University of Warwick have used computational modelling to provide new evidence that high respiratory efforts in COVID-19 patients can produce pressures and strains inside the lung that can result in injury.
The impact of high breathing efforts on the lungs of patients suffering with acute respiratory failure due to COVID-19 ...
The carbon-hydrogen bonds in alkanes--particularly those at the ends of the molecules, where each carbon has three hydrogen atoms bound to it--are very hard to "crack" if you want to replace the hydrogen atoms with other atoms. Methane (CH(4)) and ethane (CH(3)CH(3)) are made up, exclusively, of such tightly bound hydrogen atoms. In the journal Angewandte Chemie, a team of researchers has now described how they break these bonds while forming new carbon-nitrogen bonds (amidation).
If it were possible to easily break the C-H bonds in hydrocarbons, it would be possible to synthesize complex organic ...
The struggle to get your child to go to sleep and stay asleep is something most parents can relate to. Once the bedtime battle is over and the kids have finally nodded off, many parents tune out as well.
But University of South Australia researcher Professor Kurt Lushington is calling for parents to check on their small snoozers before switching off.
He says knowing the quality of a child's sleep is important, as it could be an indicator of sleep-disordered breathing - an under-reported medical condition that can affect a child's health and wellbeing.
"During sleep, the muscles keeping the upper airway stiff relax, and as a consequence, the airway narrows, which ...
Computer vision has progressed much over the past decade and made its way into all sorts of relevant applications, both in academia and in our daily lives. There are, however, some tasks in this field that are still extremely difficult for computers to perform with acceptable accuracy and speed. One example is object tracking, which involves recognizing persistent objects in video footage and tracking their movements. While computers can simultaneously track more objects than humans, they usually fail to discriminate the appearance of different objects. This, in turn, can lead to the algorithm to mix up objects in a scene and ultimately produce incorrect tracking results.
At the Gwangju Institute of Science ...
A new Tel Aviv University study found that, like humans, bats living in Tel Aviv enjoy the wide variety and abundance of food that the city has to offer, in contrast to rural bats living in Beit Guvrin, who are content eating only one type of food. The study was led by research student Katya Egert-Berg, under the guidance of Prof. Yossi Yovel, head of Tel Aviv University's Sagol School of Neuroscience and a faculty member of the School of Zoology in the George S. Wise Faculty of Life Sciences and the Steinhardt Museum of Natural History, as well as a recipient of the 2021 Kadar Family Award for Outstanding ...
The July issues of two of the American Psychiatric Association journals, The American Journal of Psychiatry and Psychiatric Services are available online.
The American Journal of Psychiatry is the most widely read psychiatric journal in the world. Its July issue offers a collection of articles discussing the impacts of structural racism, socioeconomic deprivation and stigmatization on mental health. This includes the article From Womb to Neighborhood: A Racial Analysis of Social Determinants of Psychosis in the United States, which was featured at the APA Annual Meeting in May. Among other highlights:
Dismantling Structural Racism in Psychiatry: A Path to Mental Health Equity
Modification of Heritability for Educational Attainment and Fluid Intelligence by Socioeconomic ...
When it comes to making eggs, female flies and female humans are surprisingly similar. And that could be a boon for women seeking better birth control methods, a UConn researcher reports in the July 5 issue of PNAS.
There are about 61 million women of reproductive age in the US, and about 43 million of them are sexually active but don't want a pregnancy right now, according to the Guttmacher Institute. And while there are a dozen or so different methods of birth control available, most have undesirable side effects for some of the women who try them. Despite the need, ...
Using advanced remote sensing techniques can help the early detection of oak tree decline and control many other forest diseases worldwide, says new research from Swansea University.
The research published in Remote Sensing of Environment examined holm oak decline, which in its early stages causes changes to the tree's physiological condition that is not readily visible. It is only later, when the tree is more advanced in its decline, that outward changes to its leaf pigment and canopy structure become apparent.
The researchers used an integrated approach of hyperspectral and thermal imaging, a 3-D radiative transfer ...
Philadelphia, July 19, 2021--Researchers at Children's Hospital of Philadelphia (CHOP) have developed a novel method for producing new antibiotics to combat resistant bacteria. Through an approach that would target bacteria with an antibiotic that is masked by a prodrug, which the bacteria would themselves remove, the researchers identified a method that would allow for development of new, effective antibiotics that could overcome issues of resistance. The findings were published today in eLife.
"We've created a sort of 'Trojan Horse' that would allow antibiotics to reach desired tissues undisturbed, ...
