RGBD Data set

1. RGB-D Object Dataset
http://www.cs.washington.edu/rgbd-dataset/software.html

2. B3DO: Berkeley 3-D Object Dataset
http://kinectdata.com/

3. Kinect with ground truth from Vicon (ETH)
http://www.asl.ethz.ch/research/datasets

4.

The Mobile Robot Programming Toolkit

http://www.mrpt.org/Kinect_and_MRPT

OpenNI

Introducing OpenNI

The OpenNI organization is an industry-led, not-for-profit organization formed to certify and promote the compatibility and interoperability of Natural Interaction (NI) devices, applications and middleware.

As a first step towards this goal, the organization has made available an open source framework – the OpenNI framework – which provides an application programming interface (API) for writing applications utilizing natural interaction. This API covers communication with both low level devices (e.g. vision and audio sensors), as well as high-level middleware solutions (e.g. for visual tracking using computer vision).

On this website you will be able to download the open source OpenNI framework, and get information on third-party hardware and middleware solutions which are OpenNI compliant

More?
http://www.openni.org/

PCL - Point Cloud Library

What is PCL?

The Point Cloud Library (or PCL) is a large scale, open project [1] for 3D point cloud processing. The PCL framework contains numerous state-of-the art algorithms including filtering, feature estimation, surface reconstruction, registration, model fitting and segmentation. These algorithms can be used, for example, to filter outliers from noisy data, stitch 3D point clouds together, segment relevant parts of a scene, extract keypoints and compute descriptors to recognize objects in the world based on their geometric appearance, and create surfaces from point clouds and visualize them -- to name a few.

PCL is released under the terms of the BSD license and is open source software. It is free for commercial and research use.
PCL is cross-platform, and has been successfully compiled and deployed on Linux, MacOS, Windows, and Android. To simplify development, PCL is split into a series of smaller code libraries, that can be compiled separately. This modularity is important for distributing PCL on platforms with reduced computational or size constraints (for more information about each module see the documentation page). Another way to think about PCL is as a graph of code libraries, similar to the Boost set of C++ libraries.

Want to learn more? Flow this link.
http://pointclouds.org/about.html

The PCD (Point Cloud Data) file format

This document describes the PCD (Point Cloud Data) file format, and the way it is used inside Point Cloud Library (PCL).

Why a new file format?

The PCD file format is not meant to reinvent the wheel, but rather to complement existing file formats that for one reason or another did not/do not support some of the extensions that PCL brings to n-D point cloud processing.
PCD is not the first file type to support 3D point cloud data. The computer graphics and computational geometry communities in particular, have created numerous formats to describe arbitrary polygons and point clouds acquired using laser scanners. Some of these formats include:

• PLY - a polygon file format, developed at Stanford University by Turk et al
• STL - a file format native to the stereolithography CAD software created by 3D Systems
• OBJ - a geometry definition file format first developed by Wavefront Technologies
• X3D - the ISO standard XML-based file format for representing 3D computer graphics data
• and many others

All the above file formats suffer from several shortcomings, as explained in the next sections – which is natural, as they were created for a different purpose and at different times, before today’s sensing technologies and algorithms have been invented.

PCD versions

PCD file formats might have different revision numbers, prior to the release of Point Cloud Library (PCL) version 1.0. These are numbered with PCD_Vx (e.g., PCD_V5, PCD_V6, PCD_V7, etc) and represent version numbers 0.x for the PCD file.
The official entry point for the PCD file format in PCL however should be version 0.7 (PCD_V7).

Want to learn more, click here:

SIFT Features

The website talking about Affine Covariant Features:
http://www.robots.ox.ac.uk/~vgg/research/affine/index.html#publications

SiftGPU: A GPU Implementation of Scale Invariant Feature Transform (SIFT):
http://cs.unc.edu/~ccwu/siftgpu/

Indoor Scene Segmentation using a Structured Light Sensor

Indoor Scene Segmentation using a Structured Light Sensor
http://cs.nyu.edu/~silberman/papers/indoor_seg_struct_light.pdf

Pre-processing:
1)remove holes in the depth image <- filter image by the cross-bilateral filter of pairs

2)rotate RGB image and depth map and labels to eliminate any pith and roll< - 3-axis accelerometer provided by Kinect

There exists some offset between RGB image and Depth image provided by Kinect. So, calibration is carried out to obtain precise spatial alignment between the depth and RGB images.
One method: using a set of checkerboard images in conjunction with the calibration tool of Burrus. This also provided the homography between the two cameras.

The energy function E(y) measures the cost of a latent label yi over each pixel in the image N. The energy is composed of three potential terms: 1) unary cost function, which depends on the pixel location I, local descriptor xi and learned parameters theta; 2) a class transition cost between pairs of adjacent pixels I and j; and 3) a spatial smoothness term (inhibit or encourage a class transition).

In this paper, the class transition penalty is very simple, which is as follows:

  

In some applications, the penalty can be learned from training set, and different penalty should be assigned among different class transitions.