The process of image relighting, shown here, is the automatic adjusting of the illumination settings of an image to match a new illumination source and geometry. IMAGE: PROVIDED BY VISHAL MONGA
Penn State-led team recognized for image relighting research
5/25/2021
By Sarah Small
UNIVERSITY PARK, Pa. — A collaborative Penn State and Amazon team of researchers, led by Vishal Monga, professor of electrical engineering, recently received second place in an international image-processing challenge.
The challenge was part of the New Trends in Image Restoration and Enhancement (NTIRE) workshop at the Institute of Electrical and Electronic Engineers Conference on Computer Vision and Pattern Recognition (CVPR). CVPR is “the premier annual computer vision event” for “students, academics and industry researchers,” according to its website.
In addition to Monga, the team comprised Amir Yazdani, a Penn State electrical engineering doctoral student advised by Monga, and Tiantong Guo, a researcher from Amazon Research. Their work, “Physically Inspired Dense Fusion Networks for Relighting,” received the NTIRE runner-up award for the Depth Guided Image Relighting Challenge out of 152 participants globally and 17 finalists.
Image relighting is the automatic adjusting of the illumination settings of an image to match a new illumination source and geometry. According to the researchers, this work has implications for applications in augmented reality, gaming and synthetic scene generation and enhancement in motion pictures.
The team reports in their paper that a novel mathematical model for images would allow for successful relighting even with the presence of dense shadows or limited training — two problem areas for existing relighting methods.
“Our success is built on the foundation that we build artificial intelligence frameworks that are not black box but physically inspired,” Monga said, referring to black box methods where a system or device’s inputs and outputs are observed in an effort to learn about it without knowledge of its internal workings. “In particular, our model predicts multiple image parameters, including material reflectance and illumination and geometry, leading to high accuracy relighting.”
Yazdani and Guo will attend the CVPR from June 19-25 virtually to accept the award and present their work on behalf of their team. The research was supported by a grant from the Office of Naval Research.