Summary of full paper, presented at
San Francisco, October 20-21, 1999
The full article is published in the printed "Proceedings of the 2nd IWAR'99"
and copyright protected by IEEE Computer Society.
Photometric Image-Based Rendering for Virtual Lighting Image Synthesis
Sadahiko MIHASHI and Takeshi SHAKUNAGA
A concept named Photometric Image-Based Rendering (PIBR) is
introduced for a seamless augmented reality. The PIBR is
defined as Image-Based Rendering which covers appearance
changes caused by the lighting condition changes, while
Geometric Image-Based Rendering (GIBR) is defined as
Image-Based Rendering which covers appearance changes caused
by the view point changes. The PIBR can be applied to image
synthesis to keep photometric consistency between virtual
objects and real scenes in an arbitrary lighting condition.
We analyze the conventional IBR algorithms, and formalize the
PIBR in the whole IBR framework. A specific algorithm is also
presented for realizing the PIBR. The photometric
linearization makes a controllable framework for the PIBR,
which consists of four processes; (1) separation of
environmental illumination effects, (2) estimation of lighting
directions, (3) separation of specular reflections and
cast-shadows, and (4) linearization of self-shadows. After the
photometric linearization of input images, we can synthesize
any realistic images which include not only diffuse
reflections but also self-shadows, cast-shadows and specular
reflections. Experimental results show that realistic images
can be successfully synthesized with keeping photometric
Copyright (c) 1999 IEEE. Reprinted, with permission, from IWAR'99 proceedings.
This web page contains only experimental results. If you want to know
the details of our PIBR method, please read the proceedings of IWAR'99.
Photometric Image-Based Rendering
First, we show image syntheses of a glossy ceramic pot. Keeping a
halogen light in the long distance from the pot, we took 27 images
with changing the lighting source position. The input images are
linearized, and the principal component analysis was accomplished to
make three optimal base images. Several lighting directions
corresponding to the coefficients were specified, and virtual images
were synthesized. We show one of the synthesized images as follows.
The left is a synthesized image without linear factors. Both diffuse
reflections and self-shadows are correctly synthesized. The right is
a synthesized image with linear factors, which were selected by the
nearest neighbor method. We can see that the realistic images with
the appropriate surface properties can be synthesized by the proposed
Synthesized image by the PIBR.
LEFT: Synthesized image without nonlinear factors.
RIGHT: Synthesized image with nonlinear factors.
You can see a movie sequence by clicking on each image.
Application for Augmented Reality
Next, we show some results of mixing virtual objects and real scenes
with keeping the photometric consistency. The left images are real
scenes in which the virtual object is to be mixed. The virtual object
is synthesized by the PIBR to have the same lighting direction as the
real scene. The right images are the mixed images, a mouse is a
virtual object while a cup and a book are real ones. Since the
photometric property is consistent between the real scene and the
virtual object, the mixed image looks realistic.
This material is posted here with permission of the IEEE. Internal or
personal use of this material is permitted. However, permission to
reprint/republish this material for advertising or promotional purposes or
for creating new collective works for resale or redistribution must be
obtained from the IEEE by sending a blank email message to
By choosing to view this document, you agree to all provisions of the
copyright laws protecting it.