This video demonstrates combining HMP sliding window and HMP3D voxel features in an MRF framework for labeling objects in 3D scenes reconstructed from RGB-D (Kinect) videos. The top left shows the original RGB and depth video frames. The 3D scene labeling is shown as it is reconstructed from the Kinect video, with objects colored by category (sofa=maroon, coffee table=purple, bowl=red, cap=green, mug=yellow, soda=cyan).
For technical details and more results, see the paper Unsupervised Feature Learning for 3D Scene Labeling.
In this video we demonstrate a view-based approach for labeling objects in 3D scenes reconstructed from RGB-D (Kinect) videos. The top row shows the original RGB and depth video frames, with high scoring bounding box object detections plotted on the RGB image. The 3D scene labeling is shown at the bottom, with objects color coded by category (bowl=red, cap=green, cereal=blue, mug=yellow, soda=cyan).
For technical details and more results, see the paper Detection-based Object Labeling in 3D Scenes.
OASIS is a software architecture that enables the prototyping of applications that use RGB-D cameras and underlying computer vision algorithms to recognize and track objects and gestures, combined with interactive projection. Object recognition is an important component of OASIS. The system recognizes objects that are placed within the interactive projection area so that the appropriate animations and augmented reality scenarios can be created. Our approach uses both depth and color information from the RGB-D camera to recognize different objects. Novel objects can be trained on the fly and recognized in the future.
One example application of OASIS is the following interactive LEGO playing scenario that was shown at the Consumer Electronics Show (CES) 2011.