(Press-News.org) A research team has developed a computer vision technique that can perform dichotomous image segmentation, high-resolution salient object detection, and concealed object detection in the same framework. Their novel bilateral reference framework (BiRefNet) is able to capture tiny-pixel features and holds potential for a wide range of practical computer vision applications.
The work is published in the journal CAAI Artificial Intelligence Research on August 22.
In computer vision research, image segmentation technology involves separating digital images into meaningful parts. Through this process, images are easier to analyze. As high-resolution image acquisition has advanced, scientists are now able to achieve highly precise object segmentation. This new technology is called high-resolution dichotomous image segmentation (DIS), and companies such as Samsung, Adobe, and Disney are now using it. However, current strategies used in DIS are not sufficient to capture the very finest features. To meet these existing challenges in high-resolution DIS, the research team has developed a bilateral reference module.
The team achieved high-resolution DIS with high accuracy through their BiRefNet. “With the proposed bilateral reference module, BiRefNet shows much higher precision on high-resolution images, especially those with fine details. Our BiRefNet is, so far, the best open-source and commercially available model for foreground object extraction,” said Deng-Ping Fan, a professor at Nankai University.
The team’s novel progressive bilateral reference network BiRefNet handles the high-resolution DIS task with separate localization and reconstruction modules. For the localization module, they extracted hierarchical features from the vision transformer backbone, which are then combined and squeezed. For the reconstruction module, they further designed the inward and outward references as bilateral references, in which the source image and the gradient map are fed into the decoder at different stages. Instead of resizing the original images to lower-resolution versions to ensure consistency with decoding features at each stage, they kept the original resolution for intact detail features in inward reference and adaptively cropped them into patches for compatibility with decoding features.
Their BiRefNet provides a simple yet strong baseline that performs high-quality DIS. Its inward reference with source image guidance fills in the mission information in the fine parts and its outward reference with gradient supervision allows it to focus more on regions with richer details.
Because of its extremely accurate segmentation results, BiRefNet has many useful applications. It can be employed in scenarios that common segmentation models cannot handle. For instance, it can accurately find cracks in walls, help maintain them, and determine when to repair them. It can also achieve highly accurate extraction of objects with fine grids and dense holes.
BiRefNet has already been widely used in the computer vision community. It has been integrated into the web app ComfyUI system as the so far best image matting node for better stable-diffusion-based image synthesis. BiRefNet is also widely used for human or portrait segmentation in both images and videos.
Looking ahead, the team plans to extend BiRefNet to more related tasks, including DIS, high-resolution salient object detection, camouflaged object detection, portrait segmentation, and prompt-guided object extraction. The team has already provided well-trained models for most of the aforementioned tasks.
They are also working to adapt BiRefNet to a more lightweight architecture for faster inference on high-resolution images and easier deployment on edge devices. “We have already provided BiRefNet in different parameter magnitudes, some of which have achieved 30 frames per second on images in 1024 x 1024 resolution,” said Fan.
“The ultimate goal is to keep our BiRefNet as the best open-source model for a series of related tasks, such as foreground object extraction, image matting, and portrait segmentation, making it strong, free, and open-source forever for everyone,” said Fan.
About CAAI Artificial Intelligence Research
CAAI Artificial Intelligence Research (CAAI AIR) is an Open Access, peer-reviewed scholarly journal, published by Tsinghua University Press, released exclusively on SciOpen. CAAI AIR aims to publish the state-of-the-art achievements in the field of artificial intelligence and its applications, including knowledge intelligence, perceptual intelligence, machine learning, behavioral intelligence, brain and cognition, AI chips and applications, etc. Original research and review articles on but not limited to the above topics are welcome. The journal is completely Open Access with no article processing fees for authors.
About SciOpen
SciOpen is an open access resource of scientific and technical content published by Tsinghua University Press and its publishing partners. SciOpen provides end-to-end services across manuscript submission, peer review, content hosting, analytics, identity management, and expert advice to ensure each journal’s development. By digitalizing the publishing process, SciOpen widens the reach, deepens the impact, and accelerates the exchange of ideas.
END
In the ever-evolving field of robotics, a groundbreaking approach has emerged, revolutionizing how robots perceive, navigate, and interact with their environments. This new frontier, known as brain-inspired navigation technology, integrates insights from neuroscience into robotics, offering enhanced capabilities and efficiency.
Brain-inspired navigation technologies are not just a mere improvement over traditional methods; they represent a paradigm shift. By mimicking the neural mechanisms of animals, these technologies provide robots with the ability to navigate through complex and unknown terrains with unprecedented accuracy ...
On behalf of the Community Health Impact Coalition (CHIC), Dr. Madeleine Ballard, global health leader and CEO of CHIC, is the Institute for Health Metrics and Evaluation’s 2024 Roux Prize winner. The award recognizes Dr. Ballard’s work alongside thousands of community health workers (CHWs) to secure quality care for all, through evidence-based health systems benefiting millions of people across the world.
Half of the world's population lacks access to essential health services. Around the globe, CHWs have stepped up to address this critical gap and deliver care in a way that improves access, increases equity, and saves lives. Despite their ...
The death of a massive, rapidly spinning star can shake the universe. And the resulting ripples — known as gravitational waves — could be felt by instruments on Earth, according to new research published August 22 in The Astrophysical Journal Letters. These new sources of gravitational waves just await discovery, the scientists behind the research predict.
The gravitational waves emerge following the violent deaths of rapidly rotating stars 15 to 20 times the mass of the sun. Upon running ...
INDIANAPOLIS – Chronic cough, with symptoms lasting more than eight weeks, affects approximately one in 10 adults. Cough is among the most common reasons for seeking medical care in the United States, yet chronic cough is difficult to treat. One of the largest studies of chronic cough and one of the first to explore the use of opioids, which are known to suppress cough, to treat these patients, has found that 20 percent of patients with chronic cough received a prescription for a cough suppressant containing an opioid.
With the goals of estimating opioid prescription in the chronic cough population and of informing alternative treatment ...
SAN ANTONIO — August 22, 2024 —Scientists at Southwest Research Institute have developed a new screening method to identify drug formulations that can penetrate the blood-brain barrier (BBB), to facilitate treatment of brain diseases and conditions.
“The BBB protects the brain and central nervous system from potentially harmful substances in the bloodstream, regulating the transport of essential nutrients and ions while maintaining the stability of the central nervous system,” said Research Engineer Nicholas McMahon, from SwRI’s Bioengineering group. “However, the very characteristics that make the BBB such an ...
Bioactive compounds like polyphenols and their health benefits have long captured public attention and interest. Commonly present in plant-based food like fruits, vegetables, seeds, coffee, and tea, the polyphenols have a strong bitter taste and, in the normal course, is excreted by our body due to poor absorption.
The polyphenols interact with human bitter taste receptors also known as Type 2 taste receptors (T2R) expressed within and outside the oral cavity. Notably, the activation of T2R expressed along the ...
A new treatment for locally advanced rectal cancer shows favourable results in that surgery can sometimes be avoided completely. It also reduces the risk of recurrence. The method has been confirmed as effective in a comprehensive study conducted at Uppsala University and published in eClinicalMedicine.
“The tumour disappears completely more often, thereby increasing the chance of avoiding surgery and retaining normal rectum and rectal function. Moreover, there are fewer metastases,” says Bengt Glimelius, Professor of Oncology ...
Chronic cough is among the most common reasons for seeking medical care, with middle-aged women the group most affected. New studies at Uppsala University also show that this condition appears to be a hereditary phenomenon. The studies have been published in ERJ Open Research and PLOS ONE.
“More than 10% of the population has a chronic cough, which has been shown to entail several negative consequences: reduced quality of life, reduced ability to work and voice problems. At present, we have insufficient knowledge about ...
Highlights:
Flu vaccine efficacy varies year to year.
A universal flu vaccine would protect people against all influenza strains that infect humans and last more than a season.
A new vaccine candidate incorporates proteins from 8 strains of influenza.
Recent tests of the candidate show efficacy in animal models, and the researchers hope to move to clinical trials soon.
Washington, D.C.—Annual flu vaccines protect against severe infection, but they vary in efficacy and may not match the most virulent strains ...
DALLAS, August 22, 2024 — A new $20 million research initiative will engage the people most impacted by health disparities in developing solutions that may help improve their overall health and well-being. The American Heart Association, celebrating 100 years of lifesaving service as the world’s leading nonprofit organization focused on heart and brain health for all, and the Robert Wood Johnson Foundation (RWJF), a leading national philanthropy dedicated to taking bold leaps to transform health, are ...
