Intrinsic image decomposition
from multiple photographs

Figure 1: Intrinsic image decomposition separates material colors from effects of lighting. This allows casual users to perform advanced image editing such as modifying material colors, inserting virtual objects, and relighting photographs.

Figure 2: A coherent intrinsic decomposition, applied on a photo collection, enables the transfer of illumination between photographs: here, the top photograph has been transformed into a nighttime picture.

People

This doctorate thesis was prepared by Pierre-Yves Laffont in the REVES team at Inria Sophia-Antipolis (France), and defended on October 12, 2012.

The thesis committee was composed of:

George Drettakis (advisor),
Adrien Bousseau (co-advisor),
Brian Curless (reviewer),
Hendrik Lensch (reviewer),
Diego Gutierrez (examiner),
Frédéric Precioso (president).

Thesis abstract

Editing materials and lighting is a common image manipulation task that requires significant expertise to achieve plausible results. Each pixel aggregates the effect of both material and lighting, therefore standard color manipulations are likely to affect both components.

Intrinsic image decomposition separates a photograph into independent layers: reflectance, which represents the color of the materials, and illumination, which encodes the effect of lighting at each pixel.

In this thesis, we tackle this ill-posed problem by leveraging additional information provided by multiple photographs of the scene. We combine image-guided algorithms with sparse 3D information reconstructed from multi-view stereo, in order to constrain the decomposition.

We first present an approach to decompose images of outdoor scenes, using photographs captured at a single time of day. This method not only separates reflectance from illumination, but also decomposes the illumination into sun, sky, and indirect layers. We then develop a methodology to extract lighting information about a scene solely from a few images, thus simplifying the capture and calibration steps of our intrinsic decomposition. In a third part, we focus on image collections gathered from photo-sharing websites or captured with a moving light source. We exploit the variations of lighting to process complex scenes without user assistance, nor precise and complete geometry.

The methods described in this thesis enable advanced image manipulations such as lighting-aware editing, insertion of virtual objects, and image-based illumination transfer between photographs of a collection.

Keywords: computational photography, relighting, image editing, image-based rendering, image-guided propagation, multi-view stereo, light estimation

Thesis

Thesis (PDF, 73MB)

Accompanying video for Chapter 3 (MOV, 45MB),
also on youtube

Accompanying video for Chapter 5 (MP4 format, 84MB),
also on youtube

Additional files described in the Thesis appendix can be downloaded here (.RAR format, 238MB).

BibTeX citation

@PHDTHESIS{Laffont2012,
    author = {Pierre-Yves Laffont},
    title = {Intrinsic image decomposition from multiple photographs},
    school = {Inria / University of Nice Sophia-Antipolis},
    year = {2012},
    month = {October}
}

Funding

This work was funded by a CORDI-S Doctoral Fellowship at Inria Sophia-Antipolis.