From TouchDesigner Documentation
Jump to: navigation, search

TouchDesigner acquires multiple data streams from the Intel RealSense and outputs data from the RealSense CHOP and RealSense TOP.

RealSense.1.jpg RealSense.2.jpg

The RealSense CHOP connects to RealSense devices and outputs data from Intel's RealSense camera.

  1. Marker Tracking, where you provide it a TOP image and it will do object recognition to find that image in the camera view and output its position and rotation relative to the camera.
  2. Finger Tracking, which detects 10 gesture types.
  3. Face Tracking, which outputs a mesh patching key points of your the face it detects.
  4. Person Tracking for R200 camera. Outputs tracked points on the persons skeleton.

The RealSense TOP node outputs color, depth and IR data in various forms. It has a number of setups:

  1. Color, which is the video from the RealSense camera color sensor.
  2. Depth, a calculation of the depth of each pixel. 0 means the pixel is 0 meters from the camera and 1 means the pixel is the maximum distance or more from the camera.
  3. Raw Depth, values taken directly from the RealSense camera SDK. Once again 0 means 1 meter from the camera and 1 is the maximum range or more away from the camera.
  4. Visualized Depth, a gray scale image from the RealSense SDK which can help you visualize the depth but it cannot be used to actually determine a pixel’s exact distance from the camera.
  5. Depth To Color UV Map, the UV values from a 32 bit floating RG texture (note, no blue) that are needed to remap the depth image to line up with the color image. You can use the Remap TOP node to align the images to match.
  6. Color To Depth UV Map, the UV values from a 32 bit floating RG texture (note, no blue) that are needed to remap the color image to line up with the depth image. You can use the Remap TOP node to align the two.
  7. Infrared, the raw video from the infrared sensor of the RealSense camera.
  8. Point cloud, literally a cloud of points in 3d space (X, Y, Z coordinates) or data points created by the scanner of the RealSense Camera.
  9. Point Cloud Color UVs which can be used to get each point’s color from the Color image stream.
  10. Segmented Color (with Alpha) which outputs masked color image of the detected person in front of the camera.

See the point cloud sample at:

Installing RealSense Software for Windows SDK[edit]

if you are using the Windows SDK API in the RealSense TOP, or using the RealSense CHOP, you need to use the older RealSense SDK and drivers. You will need to install two pieces of software from Intel, The first is the camera driver software (DCM) for the particule model of RealSense camera you have (F200, R200, SR300). The second is the the SDK Runtime Distributable. If you already have the full RealSense SDK installed you don't need to install the Runtime separately.

These downloads are available here.

For builds 2019.10000 or later, you should install the 2016 R3 SDK/Runtime (version 11). For builds 2018.20000 and earlier, you should install the 2016 R2 SDK/Runtime (version 10).

See RealSense CHOP, RealSense TOP

An Operator Family that creates, composites and modifies images, and reads/writes images and movies to/from files and the network. TOPs run on the graphics card's GPU.

Each SOP has a list of Points. Each point has an XYZ 3D position value plus other optional attributes. Each polygon Primitive is defined by a vertex list, which is list of point numbers.