RGB-D Object Dataset

Software

In this page we include some code snippets and software for processing the data. If you have any suggestions or would like to contribute software to this page, please contact Kevin Lai.

Hierarchical Matching Pursuit Features (Matlab)

Hierarchical Matching Pursuit (HMP) is an unsupervised feature learning technique for RGB, depth, and 3D point cloud data. Code for HMP features now available here. It achieves state-of-the-art results on the RGB-D Object Dataset.

RGB-D Kernel Descriptors (Matlab, C++)

Kernel Descriptors are a family of features for extracting rich cues from RGB-D data. They have been shown to outperform existing features such as SIFT and spin images for RGB-D object recognition. For more information and to download the software, visit the project page.

Reading Depth Images (C++, OpenCV)

All depth images in the RGB-D Object Dataset are stored as PNG where each pixel stores the depth in millimeters as a 16-bit unsigned integer. The code snippet below uses OpenCV to read a depth image and convert the depth into floats - thanks to Daniel Ricao Canelhas for suggesting this.

cv::Mat depthImage;
depthImage = cv::imread(filename, CV_LOAD_IMAGE_ANYDEPTH | CV_LOAD_IMAGE_ANYCOLOR ); // Read the file
depthImage.convertTo(depthImage, CV_32F); // convert the image data to float type

Reading Point Clouds (C++, PCL)

read_rgbd_pcd.cpp - Point clouds in the RGB-D Object Dataset are stored in the PCD file format. This sample code reads a point cloud in the dataset using the Point Cloud Library (PCL).

Note: If you encounter point clouds that are incorrectly colored black, see this for a fix. Thanks to Walter Lucetti for pointing this out.

Reading Point Clouds (MATLAB)

readPcd.m - Point clouds in the RGB-D Object Dataset are stored in the PCD file format. This MATLAB function reads a point cloud in the dataset. Warning: While this function will read PCD files from the RGB-D Object Dataset, it may not be compatible with arbitrary PCD files created with the latest version of PCL.

unpackRGBFloat.m - The color information of each point is stored as a float. Use this function to unpack the data as follows:

data = readPcd('apple_1_1_1.pcd');
rgb = unpackRGBFloat(single(data(:,4)));

Depth Image to Point Cloud (MATLAB)

depthToCloud.m - This MATLAB function will convert the depth images in the RGB-D Object Dataset into 3D point clouds. In order to convert the depth images into 3D point clouds, you need to use one of the following set of instructions, depending on which dataset you downloaded:

1. RGB-D Object Dataset (cropped images):

loc = load('apple_1_1_1_loc.txt');
depth = imread('apple_1_1_1_depthcrop.png');
[pcloud distance] = depthToCloud(depth,loc);

2. RGB-D Object Dataset (full 640x480 images) or RGB-D Scenes:

depth = imread('apple_1_1_1_depth.png');
[pcloud distance] = depthToCloud(depth);

The "loc" parameter is necessary for the cropped images because the depthToCloud function needs to know the location in the CCD sensor of the top-left pixel in the image. With the full, uncropped images, this is just 1,1 and so the parameter is not needed.

The depthToCloud function returns the point cloud in a N*M*3 matrix, where pcloud(x,y,:) is the 3D point at image location (x,y), with NaN denoting missing depth pixels.

Spin Images (MATLAB)

compSpinImages.m - This MATLAB function computes spin images for every point in a point cloud. It uses a third-party implementation of k-d trees here. In our experiments we used radius=0.04, imgW=16, minNeighbors=10.

Detection-based Object Labeling in 3D Scenes

Detection-based Object Labeling in 3D Scenes
Kevin Lai, Liefeng Bo, Xiaofeng Ren, and Dieter Fox. ICRA 2012, May 2012.

Software and Data for the technique from the above paper is now available.