Palette:camSchnappr

From Derivative
Jump to: navigation, search


Summary
[edit]

camSchnappr is a interactive mapping application entirely inspired by MAPAMOK, created by Kyle McDonald at the YCAM Interlab. With Kyle publishing the source code for MAPAMOK, we had a chance to look at it and convert it into TouchDesigner. For the original tool and source have a look here:

Kyle describes the significance of the tool as the use of openCVs cameraCalibrate to calibrate a projector via a model of the to be mapped structure instead of using a checkerboard. With TouchDesigner’s integration of Python, we were able to implement the same functionality into a standalone tool which lets you map complex structures in a small amount of time.

PythonIcon.pngPalette/camSchnappr Ext


What it does

If you have a physical 3D structure plus a virtual 3D model of that structure, you will be able to project a rendered virtual model onto that physical structure perfectly-aligned. You will be selecting points on your 3D Model and align them with the real world positions as you see them on the projector output. In order to support multiple projectors, the output of blend-masks from multiple camSchnapprs is now supported.

Using your 6 or more alignment points, CamSchnappr will compute the position, rotation, scale and viewing angle of a specified TouchDesigner Camera component used for the rendering. Internally, an openCV function called calibrateCamera will be run and the “intrinsics and extrinsics” of the camera will be calculated and stored in the Camera component.


What you need

  • TouchDesigner
  • a projector
  • a physical structure to project on
  • a 3D Model of the to-be-mapped structure


Interface

Start TouchDesigner and drag camSchnappr from the Palette>Tools onto your network pane. camSchnappr will present itself as a Camera COMP and can be configured via its Custom Parameters.


Parameters - camSchnappr Page

Render TOP Rendertop - Assign a Render TOP. camSchnappr will use the referenced TOP's resolution.  

Geo SOP Geosop - Assign the SOP which holds the geometry you are calibrating. Do not specify a SOP that is a Material SOP or is from a branch with a Material SOP in it as this would break the blending. Instead reference the Texture via the Color Map Parameter.

Open camSchnappr Opencs - Open the camSchnappr editing window.  

Close camSchnappr Closecs - Close the camSchnappr editing window. Pulsing this parameter after completion of the calibration will ensure that unnecessary nodes are cook disabled.  

Output Monitor Monitor - Assign which Monitor the output window will open on. 

Open Output Open - Open the output window.  

Close Output Close - Close the output window.  

Hide Guides Guides - Hide the guides on the output monitor.  

Reverse Geo Vertex Reversevertex - Reverse both U and V for the Hull.  

Color Geo Randomly Random - Color each primitive of the geometry by random. This can be useful when schnapping complex objects.  

Color Map Colormap - Assign a TOP which is used as texture in the editing viewport and the output.  

Texture Alpha Texturealpha - Control the texture alpha.  

Show Wireframe Wireframe - Show or hide the geometries wireframe.  

Wireframe Line Width Wirewidth - Control the wireframe strength of the geometry in the output window.  

Wireframe Color Wirecolor - Set the wireframe color of the geometry in the output window.

  • Wireframe Color Wirecolorr -
  • Wireframe Color Wirecolorg -
  • Wireframe Color Wirecolorb -

Backlight Color Blightcolor - Set the background color of the output window.

  • Backlight Color Blightcolorr -
  • Backlight Color Blightcolorg -
  • Backlight Color Blightcolorb -

Backlight Blightamount - Control the backlight amount of the output window.  

Clear Clear - Clear the calibration data.  

Parameters - Auto Blend Page

camSchnappr Cameras Camschnapprcams - Specify all other camSchnapprs in order to calculate the blend-mask for the projector.

Parameters - Advanced Page

Near Near - This control allows you to designate the near and far clipping planes. Geometry closer / further away from the lens than these distances will not be visible.  

Far Far -  

FOV Fov - Specify the initial FOV (field of view) estimation for the camera matrix given to the cv2.calibrateCamera function. 

Intrinsic Guess Intrinsic - the initial cameraMatrix passed to calibrateCamera contains valid initial values of focal length and principal point that are optimized further. Otherwise, the principal point is initially set to the image center ( imageSize is used), and focal distances are computed in a least-squares fashion.  

Fix Aspect Ratio Aspect - This considers only the vertical focal length as a free parameter. This should always be enabled unless you have an unusual projector with non-square pixels.  

Zero Tangent Distance Tangent - Tangential distortion is set to zero. It's enabled by default because most projectors have very little tangential distortion.  

Fix Principal Point Principal - The principal point is not changed during the optimization. You should enable this if you have a high quality lens with zero lens shift.  

Fix K1 Fixk1 - The corresponding radial distortion coefficient is not changed during the optimization. For extremely wide fisheye lenses or lenses with radial distortion try enabling these.  

Fix K2 Fixk2 - The corresponding radial distortion coefficient is not changed during the optimization. For extremely wide fisheye lenses or lenses with radial distortion try enabling these.  

Fix K3 Fixk3 - The corresponding radial distortion coefficient is not changed during the optimization. For extremely wide fisheye lenses or lenses with radial distortion try enabling these.  

Max Iterations Maxiterations - Specify the number of iterations the algorithm should terminate after. 

Precision Precision - Specify the value of accuracy at which, when reaching it, the algorithm should terminate.  

The Viewport

Open the viewport by clicking the Open camSchnappr parameter.

CamSchnappr Viewport.jpg

You can interact with the viewport as follows:

  • ctrl + left-mouse click and drag to tumble
  • ctrl + middle-mouse click and drag to zoom
  • ctrl + right-mouse click and drag to pan
  • h to home the geometry
  • create control points by left-mouse clicking on the geometry
  • make control points active by left-mouse clicking on blue spheres
  • delete control points by right-mouse clicking onto spheres
  • tab to cycle through selected points

When you select points a little flag with the point number will be displayed.


The Mapping Viewport

Once you selected the correct output Monitor via the Output Monitor parameter, click the Open Output parameter.

CamSchnappr Output01.jpg

The interface is quite simple:

  • a pink crosshair indicates your current cursor position on the screen
  • a blue crosshair with number indicates an inactive point previously selected to be mapped in the main Viewport
  • a yellow crosshair with number indicates an active point previously selected to be mapped in the main viewport

The numbers beside each point correspond to the numbers of the selected points in the main viewport

You can drag points to their real-world position on the screen or use the arrow keys on the keyboard. For faster movement, use shift+arrowkey to move points.

Tab will cycle through the available points.


Workflow

  • Create a simple render setup with the model of the object you want to map onto.
  • select the camSchnappr Camera COMP.
  • Drag your Render TOP to onto the Render TOP parameter.
  • Drag the SOP containing the geometry of the to be mapped object onto the Geo SOP parameter.
  • Select the number of the output your Projector is connected to via the Output Monitor parameter.
  • Open the mapping Viewport by clicking the Open Output parameter, if somehow the output now overlays your main viewport, just hit escape and select a different Monitor Output number via the Output Monitor parameter.
  • In the main viewport align the geometry to the camera so it’s similar to what the projector sees.
  • Select a point in the main viewport and move it in the mapper viewport to the corresponding position on the real world object.
  • Repeat the last step for at least 6 points total.
  • After you have aligned 6 points, the camera should be calibrated and you should see the projection mapped onto the object.
  • If everything is correct you can close the mapping output and the camSchnappr viewport by clicking the Close Output and Close camSchnappr parameter. The Camera Calibration values are saved inside your camSchnappr Camera COMP as 2 Table DATs.


Multiple Projectors

  • CamSchnappr can be run on multiple render setups making it possible to calibrate multiple projectors or multiple objects.
  • specify the path to all other calibrated camSchnapprs in the 'camSchnappr Cameras' parameter on the 'Auto Blend' parameter page.
  • the TOP output of each camSchnappr will be the blend-mask for each respective projector.
  • multiply the blend-mask with the final projector outputs to achieve a soft edge blended result.

CamSchnappr Output02.jpg


An Operator Family that contains its own Network inside. There are twelve 3D Object Component and eight 2D Panel Component types. See also Network Path.

An Operator Family that creates, composites and modifies images, and reads/writes images and movies to/from files and the network. TOPs run on the graphics card's GPU.

An Operator Family that reads, creates and modifies 3D polygons, curves, NURBS surfaces, spheres, meatballs and other 3D surface data.

An Operator Family that associates a shader with a SOP or Geometry Object for rendering textured and lit objects.

Parameters in TouchDesigner are an operator's settings (numbers, menus, flags, strings, paths) which the user can alter. Parameters for any operator can be opened by right-clicking on the operator and selecting "Parameters..." A currently selected operator's parameters can be viewed in a Network Editor by pressing the keyboard shortcut 'p'.

Each SOP has a list of Points. Each point has an XYZ 3D position value plus other optional attributes. Each polygon Primitive is defined by a vertex list, which is list of point numbers.