(Press-News.org) Despite significant progress in developing AI systems that can understand the physical world like humans do, researchers have struggled with modelling a certain aspect of our visual system: the perception of light.
“Determining the influence of light in a given photograph is a bit like trying to separate the ingredients out of an already baked cake.” explains Chris Careaga, a PhD student in the Computational Photography Lab at SFU. The task requires undoing the complicated interactions between light and surfaces in a scene. This problem is referred to as intrinsic decomposition, and has been studied for nearly half a century.
In a new paper published in the journal ACM Transactions on Graphics, researchers in the Computational Photography Lab develop an AI approach to intrinsic decomposition that works on a wide range of images. Their method automatically separates an image into two layers: one with only lighting effects and one with the true colours of objects in the scene. “The main innovation behind our work is to create a system of neural networks that are individually tasked with easier problems. They work together to understand the illumination in a photograph”, Careaga adds.
Although intrinsic decomposition has been studied for decades, SFU’s new invention is the first in the field to accomplish this task for any HD image that a person might take with their camera. “By editing the lighting and colours separately, a whole range of applications that are reserved for CGI and VFX become possible for regular image editing”, says Dr. Yağız Aksoy, who leads the Computational Photography Lab at SFU. “This physical understanding of light makes it an invaluable and accessible tool for content creators, photo editors, and post-production artists, as well as for new technologies such as augmented reality and spatial computing.”
The group has since extended their intrinsic decomposition approach, applying it to the problem of image compositing: “When you insert an object or person from one image into another, it’s usually obvious that it’s edited since the lighting and colours don’t match” explains Careaga. “Using our intrinsic decomposition technique, we can alter the lighting of the inserted object to make it appear more realistic in the new scene”. In addition to publishing a paper on this, presented at SIGGRAPH Asia last December, the group has also developed a computer interface that allows users to interactively edit the lighting of these “composited” images. S. Mahdi H. Miangoleh, a PhD student in Aksoy’s lab, also contributed to this work.
Aksoy and his team plan to extend their methods to video for use in film post-production, and further develop AI capabilities in terms of interactive illumination editing. They emphasize a creativity-driven approach to AI in film production, aiming to empower independent and low-budget productions. To better understand the challenges in these production settings, the group has developed a computational photography studio at the Simon Fraser University campus where they conduct research in an active production environment. They also produce videos explaining their work which you can check out here:
Intrinsic Image Decomposition via Ordinal Shading - ACM TOG
Intrinsic Harmonization for Illumination-Aware Compositing - SIGGRAPH Asia 2023
The above publications represent some of the group’s initial steps towards providing AI-driven editing capabilities to the rich filmmaking industry in British Columbia. Their focus on intrinsic decomposition enables even low-budget productions to adjust lighting easily, without requiring costly reshoots. These innovations support local filmmakers, maintaining BC's position as a global filmmaking hub, and will serve as the foundation of many more AI-enabled applications to come from the Computational Photography Lab at SFU.
END
SFU researchers develop AI that can understand light in photographs
2024-02-21
ELSE PRESS RELEASES FROM THIS DATE:
Minority ethnic NHS staff more likely to face workplace discrimination during pandemic than White colleagues
2024-02-21
Minority ethnic NHS staff were more likely to face workplace harassment, discrimination, and unavailability of personal protective equipment (PPE) than their White British colleagues during the pandemic, reveals research published online in the journal Occupational & Environmental Medicine.
Urgent action is needed to redress ongoing race inequalities in the health service, insist the researchers, who call for the inclusion of diversity and inclusion training in professional development, and the expansion of the NHS Workforce Race Equality Standard.
Staff from minority ethnic groups make up nearly ...
J-shaped curve apparent between dietary thiamine and worsening mental acuity
2024-02-21
There seems to be a J-shaped curve between dietary thiamine (vitamin B1) and worsening mental acuity among cognitively healthy older people, suggests research published in the open access journal General Psychiatry.
The sweet spot seems to be a daily intake of 0.68 mg, below which there is relatively little impact. But higher daily intake was strongly associated with cognitive decline, with the optimal maintenance dose 0.6 to less than 1 mg/day, the findings show.
Thiamine is an essential water-soluble B vitamin involved in energy metabolism and brain neurotransmitter activity. Good dietary sources include whole grains, fortified ...
Adopting healthy lifestyle strongly linked to lower irritable bowel syndrome risk
2024-02-21
Adopting a healthy lifestyle is strongly linked to a lower risk of irritable bowel syndrome or IBS for short, finds research published online in the journal Gut.
Of the big 5 healthy behaviours, not smoking, a high level of vigorous physical activity, and getting enough sleep were independently associated with keeping the condition at bay.
Characterised by abdominal pain, bloating, and abnormal bowel habit, IBS is thought to affect up to 1 in 10 people worldwide. Exactly what causes IBS isn’t fully understood, but disordered functioning of the gut–brain ...
Avid appetite in childhood linked to later eating disorder symptoms
2024-02-21
An enthusiastic response to food in early childhood may be linked to a higher likelihood of experiencing eating disorder symptoms in adolescence, according to a new study led by researchers at UCL and Erasmus University Rotterdam.
The study, published in The Lancet Child & Adolescent Health, looked at survey data from 3,670 young people in the UK and the Netherlands to investigate how appetite traits in early childhood might relate to the likelihood of developing eating disorder symptoms up to 10 years later.
The researchers found that a particularly high food responsiveness, defined as the urge to eat when you see, smell ...
Red light can reduce blood glucose levels, says study
2024-02-21
The researchers found that 670 nanometres (nm) of red light stimulated energy production within mitochondria, the tiny powerhouses within cells, leading to increased consumption of glucose. In particular, it led to a 27.7% reduction in blood glucose levels following glucose intake, and it reduced maximum glucose spiking by 7.5%.
While the study was conducted in healthy individuals, the non-invasive, non-pharmacological technique has the potential to have an impact on diabetes control after meals, as it can reduce damaging fluctuations of blood glucose in the body that contribute to ageing.
The study also highlights the significant long-term consequences for human health, including ...
UT-Battelle donates $186,000 to support SEEED’s green construction program
2024-02-21
Oak Ridge National Laboratory’s managing contractor, UT-Battelle, presented a donation of $186,000 to Socially Equal Energy Efficient Development, or SEEED, to support the nonprofit’s third green solar home as part of their Green Construction Program.
“We are committed to serving the communities that we live in,” UT-Battelle CEO and ORNL Director Stephen Streiffer said. “Our partnership with SEEED has provided a tremendous opportunity to share our scientific expertise, and we are excited that our support will help make this green solar home become a reality.”
Streiffer joined SEEED for a groundbreaking ceremony in Knoxville, Tenn., at the home project ...
Spinning, magnetic micro-robots help researchers probe immune cell recognition
2024-02-21
Researchers at the Pritzker School of Molecular Engineering and the Department of Chemistry at the University of Chicago have engineered tiny, spinning micro-robots that bind to immune cells to probe their function. The robot, or “hexapod,” gives scientists a new, highly adaptable way to study immune cells and to aid in the design of immunotherapies against cancer, infection, or autoimmune diseases.
Each hexapod robot has six arms containing molecules that might be recognized as foreign by the immune system — such as protein fragments from a tumor, virus, or ...
Helping patients with low income overcome eating disorders
2024-02-21
Individuals with eating disorders who have low income are frequently misdiagnosed and lack adequate access to appropriate therapy, according to researchers from Weill Cornell Medicine and Columbia University Vagelos College of Physicians and Surgeons.
Their paper, published in The Cognitive Behaviour Therapist on Feb. 19, identified the barriers to care that come with having low income and offered guidelines on how therapists can make accommodations for these patients to improve diagnoses and access to treatment.
Approximately 30 million people in the United States experience ...
Preventing relapse by restoring an opioid-weakened brain pathway governing behavior
2024-02-21
Medical University of South Carolina scientists report in Neuron that they have uncovered a way to restore an opioid-weakened brain pathway in a preclinical model.
With funding from the National Institute on Drug Abuse, part of the National Institutes of Health, the MUSC research team, led by neuroscientist James Otis, Ph.D., used advanced neuroscience tools to return a pathway between the thalamus and basal ganglia to healthy functioning in mice. As a result, this restoration prevented mice that were opioid-dependent from seeking or self-administering heroin. Results also suggested that sustained opioid use was the cause of this weakened pathway, rather than being caused by ...
Water quality monitor, locust-inspired electronic nose under development
2024-02-21
By Beth Miller
Two teams of engineers led by faculty in the McKelvey School of Engineering at Washington University in St. Louis will work toward developing products to monitor drinking water quality and to detect explosives with an electronic nose with one-year, $650,000 Convergence Accelerator Phase 1 grants from the National Science Foundation (NSF).
Barani Raman, professor of biomedical engineering, and Daniel Giammar, the Walter E. Browne Professor of Environmental Engineering, will lead teams of researchers from Washington University and other institutions and entities funded under the NSF’s Convergence Accelerator program, designed to address national-scale ...