A scalable GPU-based approach to shading and shadowing for photorealistic real-time Augmented Reality

Visually realistic Augmented Reality (AR) entails addressing several difficult problems. The most difficult problem is that of rendering the virtual objects with illumination which is consistent with the illumination of the real scene. The paper describes a complete AR rendering system centered around the use of High Dynamic Range environment maps for representing the real scene illumination. The main contribution lies in a novel, physically-based approach to rendering shadows cast by virtual objects without changing the shadows already present in the images of the real scene. The proposed approach effectively involves real-time estimation of the diffuse albedos of the real scene, and essentially relighting these areas to take virtual shadows into account. Another contribution lies in the fact that the proposed approach is designed to run on graphics hardware and is scalable in the sense that it offers a simple way to balance performance with visual quality.