Detection-based Object Labeling in 3D Scenes
Kevin Lai, Liefeng Bo, Xiaofeng Ren, and Dieter Fox. ICRA 2012, May 2012.

Software Authors:
Kevin Lai
Liefeng Bo

This software implements the 3D scene labeling technique in the above paper. The software has been run with MATLAB R2012b (8.0.0.783) 64-bit (glnxa64) on a linux machine. The package also includes an unmodified copy of liblinear for convenience.

Before you run, you may need to compile the .cc files in rgbd/ and setup gco and qrot3d in the util folder for your machine architecture.

The main MATLAB scripts to run are:

rgbdtrain(category) - Train a category sliding window detector
rgbdscenelabeling(seq) - Run sliding window detectors on a video sequence to generate object probability maps
rgbdvideotest(seq) - Stitch together the video sequence into a 3D scene and label it

Navigate to object_labeling/rgbd and run the above scripts from there. They will dump results into the following directories:
object_labeling/rgbd/model -  The category detector models
object_labeling/rgbd/results_det - probabilistic score maps for each video sequence
object_labeling/rgbd/results_mrf - 3D scene labeling results. The ply files are labeled 3D point clouds and can be opened with MeshLab ( http://meshlab.sourceforge.net/ )


To run the code and reproduce the results in the paper, you will also need to download the data at:
http://www.cs.washington.edu/rgbd-dataset/dataset/rgbd-scene-labeling

Place the train, background, test, and groundtruth tarballs under object_labeling/rgbd/rgbddata and the aligned_rgbd_scenes under object/labeling/rgbd

See rgbdrunall.m for an example of how to run everything. Note that actually running it serially like that will take a long time. You can train all 5 category detectors in parallel, then run scene labeling on all 8 video sequences in parallel).


If you use this software, please cite:

Detection-based Object Labeling in 3D Scenes
Kevin Lai, Liefeng Bo, Xiaofeng Ren, and Dieter Fox. ICRA 2012, May 2012.