Summary of full paper, presented at
San Francisco, October 20-21, 1999
The full article is published in the printed "Proceedings of the 2nd IWAR'99"
and copyright protected by IEEE Computer Society.
Vision-based Pose Computation: Robust and Accurate Augmented Reality Tracking
Computer Graphics and Immersive Technology Laboratory
Computer Science Department
University of Southern California
Vision-based tracking systems have advantages for augmented
reality (AR) applications. Their registration can be very
accurate, and there is no delay between the motions of real and
virtual scene elements. However, vision-based tracking often
suffers from limited range, intermittent errors, and dropouts.
These shortcomings are due to the need to see multiple
calibrated features or fiducials in each frame. To address these
shortcomings, features in the scene can be dynamically
calibrated and pose calculations can be made robust to noise and
numerical instability. In this paper, we survey classic
vision-based pose computations and present two methods that
offer increased robustness and accuracy in the context of
real-time AR tracking.
Copyright (c) 1999 IEEE. Reprinted, with permission, from IWAR'99 proceedings.
- Problems in Current Pose Computation Method
- Our current pose computation method (3-point based analytical solutions) has numerical instability and is not robust in the presence of noise and outliers (figure on the left).
- In a simulated experiment of tracking with dynamic calibration, the system started tracking with 6 calibrated features. The camera was then panned and rotated while the system estimated the positions of 94 initially uncalibrated features placed in a 100x30x20 volume. Figure on the right hand side shows the errors in the camera position computed from the estimated features. After about 500 frames (~16 seconds) the five inch accumulated error exceeds 5% of the largest operating volume dimension. This performance may be adequate to compensate for several frames of fiducial occlusion, but it does not allow significant tracking area extension.
- Summary of Pose Computation Methods
- We present two new pose computation methods that are accurate,
robust, and perform in real-time. RA3 (Robust Averages of
3-point solutions) is embedded with real-time approximation
version of robust M-estimator to a set of 3-point based
solutions. iEKF (Iterative Extended Kalman Filter) iterates EKF
in measurement space to updates the camera pose point by point.
It is a variation of SCAAT (Single Constrain At A Time) filter
that is designed specifically for video frame rates and
over-constrained measurements per frame that are typical of
passive vision systems.
- RA3 is fast (about 4 times faster than iEKF); robust under
sudden camera motion and in the presence of outliers; accurate with
about 1.0 pixel re-projection error (s =0.5 pixel measurement
noise); and capable of using a wide range of points (3-6 in our
- iEKF is also fast enough for real time applications, though it
is slower than RA3; robust under sudden motion and in the
presence of outliers; accurate within about 0.52 pixel of
re-projection error (s =0.5 pixel measurement noise); and fully
n-point-based. iEKF also has no minimum-points requirement for a given frame. It can estimate pose if only one point is available in a frame
- Reduced propagation error
The propagated errors were significantly reduced for both methods compared to the simple 3-point method. These results indicate that it may be feasible to use autocalibration over a long term and large area with modest error propagation. More tests and a real system are needed to verify or demonstrate the viability of extendible tracking for real applications.
- Movie files of real-time experiments (to compare the new pose
computation methods with 3-point based method)
- mpeg1 movie file of 3-point method: 2.2M
- mpeg1 movie file of iEK Fmethod: 2.2M
- mpeg1 movie file of RA3 method: 2.2M
This material is posted here with permission of the IEEE. Internal or
personal use of this material is permitted. However, permission to
reprint/republish this material for advertising or promotional purposes or
for creating new collective works for resale or redistribution must be
obtained from the IEEE by sending a blank email message to
By choosing to view this document, you agree to all provisions of the
copyright laws protecting it.