IWAR'99 Breakout Session:
Tracker Shootout at ISAR 2000
Summary: Suya You (firstname.lastname@example.org)
In this breakout session we discussed the issue central to a Tracker
Shootout for IWAR 2000. The topics posed to the discussion were:
1. Necessity for live comparison of various
2. Predefined Scenario.
3. Result evaluation.
1. Necessity for live comparison of various tracking
A wealth of tracking methods, employing a variety of sensing technologies,
have been proposed.
Need some kinds of "standard", quantitative measurements.
Predefine a scenario to live compare proposed techniques.
Most participants felt
this was necessary and would be helpful.
2. Predefined Scenario:
known "ground truth".
Suit for different methods (fiducials, vision, inertial, magnetic...) and
Simple scenario (for first year) to test tracking accuracy (maybe in the
form of annotation).
3. Result evaluation:
Suggested scenario includes a simple cube box with known 3D information.
User may track and annotate a virtual cube on it to evaluate the tracking
approach. The scenario may be the only constraint, but user can recofigurate
it (for example, add fiducials on the surface or background) as required
of the tracking approach.
Some people suggested the community could make the scenario and release
Measure directly in 3D space.
Measure in 2D image domain, e.g. project the 3D back to image plane.
Time allowed for only a brief discussion above. There was much controversy,
but most participants agreed that this was necessary and would be helpful.
It also seemed that many were in agreement that starting a simple scenario
might be the first step towards the goal.