Spatially-Varying Diffuse Reflectance Capture Using Irradiance Map Rendering for Image-Based Modeling Applications

Image-based 3D modelling using Structure-form-Motion (SfM) has
matured significantly over the last decade. Standard SfM methods
create the object’s texture from the appearance of the physical object
at the time of acquisition. We propose a method for acquiring
the diffuse per-point reflectance of the modelled object, as part of
the image acquisition work flow, only adding one extra captured
image and an irradiance rendering step, making it easy for anyone
to digitize physical objects to create 3D content for AR/VR using
only consumer grade hardware. Current state of the art of spatially
varying reflectance capture requires either large, expensive, and
purpose built setups or are optimization based approaches, whereas
the proposed approach is model based.
This paper proposes adding a render of irradiance with modelled
camera and light source, using off the shelf hardware for image capture. The key element is taking two images at each imaging location:
one with just the ambient illumination conditions, and one where the
light from an on-camera flash is included. It is demonstrated how to
get the ambient illumination to cancel out, and by assuming Lambertian materials, render the irradiance corresponding to the flash-only
image, enabling computation of spatially varying diffuse reflectance
rather than appearance. Qualitative results demonstrate the added
realism of the modelled objects when used as assets in renders under
varying illumination conditions, including limited outdoor scenarios.
Quantitative tests demonstrate that the reflectance can be estimated
correctly to within a few percent even in cases with severe un-even
ambient illumination.