Multi-Object Pose Estimation Challenge

ICCV 2015 Workshop

   
   
   

Description

The Multi-Object Pose Estimation Challenge is part of the ICCV 2015 workshop on Recovering 6D Object Pose and aims in evaluating various methods on the problem of 6D pose estimation of multiple objects in a scene.

   

Dataset

The dataset can be downloaded from here. It contains 6 objects and about 1000 test images per object. Test images contain multiple instances of the object with partial occlusions. For training, the 3D models of the objects are provided. To refere to this dataset please cite the following paper: A. Tejani, D. Tang, R. Kouskouridas and T-K. Kim, “Latent-Class Hough Forests for 3D Object Detection and Pose Estimation”, European Conference on Computer Vision (ECCV), Zurich, Switzerland, 2014 [1]

   

Evaluation

To evaluate the results, the percentage of correctly estimated poses per object is calculated, along with false positives and false negatives. In order to assess whether a 6D pose solution is correct we use the following three metrics:
1. The AD criterion (Hinterstoisser et al. [2]): We calculate the Average Distance (AD) between all vertices in the 3D model in the estimated pose and the ground truth pose. A pose is considered correct, when this average distance is below 10% of the object diameter.
2. 5cm, 5deg (Shotton et al. [3]): A pose is considered correct when the translational error is below 5cm and the rotational error is below 5deg.
3. The IOU criterion: We calculate the 2D axis aligned bounding boxes in the estimated pose and ground truth pose. We calculate the IOU (Intersection Over Union) of the bounding boxes and a pose is considered correct, when this value is above a threshold (to be defined).

   

Participation

In order to participate, you should send us the results at icvl.challenge.2015@gmail.com of the 3D pose estimation of the objects in the test images.

You should create separate folders for each object, and for each test image of each object you should include a file called poseXXX.txt where XXX is the number of the test image. The file poseXXX.txt should include N 4x4 rotation matrices, one under the other, one line per row. N is the number of objects that your algorithm detected.

To evaluate the results, please send us a zip file containing one folder per object with the pose files, and we will measure the accuracy using all our metrics. Results will be published here after the challenge.

   

References

1. A. Tejani, D. Tang, R. Kouskouridas and T-K. Kim, “Latent-Class Hough Forests for 3D Object Detection and Pose Estimation”, European Conference on Computer Vision (ECCV), Zurich, Switzerland, 2014
2. Stefan Hinterstoisser, Vincent Lepetit, Slobodan Ilic, Stefan Holzer, Gary R. Bradski, Kurt Konolige, Nassir Navab: Model Based Training, Detection and Pose Estimation of Texture-Less 3D Objects in Heavily Cluttered Scenes. ACCV 2012
3. [Shotton2013]: Jamie Shotton, Ben Glocker, Christopher Zach, Shahram Izadi, Antonio Criminisi, Andrew Fitzgibbon: Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images. CVPR 2013