Modeling from reality: creating virtual reality models through observation

In this paper, we present an overview of our project to construct a digital archive of cultural heritages. Among the efforts in our project, we briefly overview our research on geometric and photometric preservation of cultural assets and restoration of their original appearance. Digital geometric modeling is achieved through a pipeline consisting of scanning, registering and merging multiple range images. For these purposes, we have developed a robust simultaneous registration method and an efficient and robust voxel based integration method. On top of the geometrical model, we align texture images acquired at the scanning. Because the geometrical relation between the range sensor and the image sensor are calibrated, we automatically align texture images onto the geometrical models. For photometric modeling, we have developed a surface light field based method, which captures the appearance variation of real world objects under different viewpoints and illumination conditions from a series of images. As an attempt to restore the original appearance of historical heritages, we have reconstructed several buildings and statues that have been lost in the past. In this paper, we overview these techniques and show several results of applying the proposed methods to existing ancestral assets.