The objective of this project is to develop image processing algorithms for cinema that allow people watching a movie on a screen to see the same details and colors as people at the shooting location can.
It is due to camera and display limitations that the shooting location and the images on the screen are perceived very differently. We want to be able to use common cameras and displays (as opposed to highly expensive hardware systems) and work solely on processing the video so that our perception of the scene and of the images on the screen match, without having to add artificial lights when shooting (other than for artistic purposes) or to manually correct the colors to adapt to a particular display device.
Given that in terms of sensing capabilities cameras are in most regards better than human photoreceptors, the superiority of human vision over camera systems lies in the better processing which is carried out in the retina and visual cortex. Therefore, rather than working on the hardware, improving lenses and sensors, we will instead use, whenever possible, existing knowledge on visual neuroscience and models on visual perception to develop software methods mimicking neural processes in the human visual system, and apply these methods to images captured with a regular camera. We will also use variational methods coupled with perceptual metrics to optimize the final outputs.
From a technological standpoint, reaching our goal will be a remarkable achievement which will impact how movies are made (in less time, with less equipment, with smaller crews, with more artistic freedom) but also which movies are made (since good-visual-quality productions will become more affordable.) We also anticipate a considerable technological impact in the realm of consumer video. From a scientific standpoint, this will imply finding solutions for several challenging open problems in image processing and computer vision, but it also has a strong potential to bring methodological advances to other domains like experimental psychology and visual neuroscience.
Our research is funded by the following public institutions: European Research Council, Ministerio de Economía y Competitividad, Institució Catalana de Recerca i Estudis Avançats.