CoSPE: Computer vision based Scene Parameter Estimation

CoSPE started in 2005 and is a 7 year research project funded by the Danish Technical Research Council. The project aims at estimating light source and material properties, i.e., illumination conditions and reflectances, given an image or a sequence of images of a scene. The approach is to exploit what we call a “hypothesize-synthesize-correlate-update” verification strategy. This is a special version of the classical “predict-match-update” loop in model-based control systems where the state of a system is predicted from a model, the predicted state is matched to the measured state and any discrepancies are used to update the model. The “hypothesize-synthesize-correlate-update” strategy will combine information from two of the three image forming components (illumination, reflectance, geometry) with actual image information and use this combination to generate information concerning the third component. Applications are, e.g., in synthesizing artificial images of real world environments that look photorealistic. This requires an accurate 3D description of the scene to be imaged including light source and material properties. Small inaccuracies, such as a too sharp shadow or a too glossy reflection on an object, can make the images look artificial to human observers. The project results will also be very valuable to Augmented Reality systems, where virtual objects are inserted into real scenes. This can only be done realistically if the real scene illumination conditions are known. Another important application is in robust computer vision systems, like autonomous outdoor vehicle navigation. Funded by the Danish Technical Research Council, 2005-2012. (Claus B. Madsen, Hans J. Andersen, Erik Granum)