From TouchDesigner Documentation
Jump to: navigation, search

OpenCV (Open Source Computer Vision) is a library of programming functions mainly aimed at real-time computer vision, originally developed by an Intel research center and now open source.

TouchDesigner comes pre-installed with OpenCV 3.2, and numpy which interface with TouchDesigner's Python 3.7.2, making it possible for TouchDesigner to access the OpenCV functions directly.

OpenCV is used in the Blob Track TOP and camSchnappr.

Testing OpenCV[edit]

  • open TouchDesigner
  • open the Textport with Alt+t
  • run following script:
TouchDesigner Build 2020.20610 compile on Fri Mar 4 16:03:29 2020
Python 3.7.2 (heads/3.7-Derivative:052feb9e72, Mar  7 2019, 16:29:45) [MSC v.1900 64 bit (AMD64)]Python 3.5.1+ (default, Apr 21 2016, 11:06:11) [MSC v.1900 64 bit (AMD64)]

python >>> import cv2

python >>> image = cv2.imread(app.samplesFolder+'/Map/Trillium.jpg')

python >>> print(image)
  • if the output is an array of numbers and no errors, OpenCV is working properly.

Example: Finding Features in a Texture[edit]

Every TOP can directly be converted into a NumPy Array by calling the myTop.numpyArray() Method (Also see: TOP Class#Methods)

NumPy arrays are the default data structure openCV saves it's data in. In general NumPy can be understood as a library for Python to support large, multi-dimensional arrays and matricies, along with a large collection of high level functions to operate on these arrays. (Compare: NumPy)

The following example is the content of a Script CHOP. First, parameters for the Script CHOP are specified. For each cook of the Script CHOP, the operator specified in the Top custom parameter is read into a numPy array and then passed on to an openCV function called goodFeaturesToTrack.

Note: The referenced TOP should be a monochrom image. Converting color textures to grayscale can be done using the Monochrome TOP. Doing so, and using the Luminance function as the convertion, makes it unnecessary to do this step in the script (many openCV functions expect a grayscale image as input).

The resulting output can be used in an Instancing setup.

 1 # me - this DAT
 2 # scriptOp - the OP which is cooking
 4 # press 'Setup Parameters' in the OP to call this function to re-create the parameters.
 5 def onSetupParameters(scriptOp):
 6 	# create a custom page
 7 	page = scriptOp.appendCustomPage('Good Features')
 9 	# create a custom TOP reference parameter
10 	topPar = page.appendTOP('Top', label='TOP (monochrome)')
12 	# create a custom parameter to specify number of features to detect
13 	p = page.appendInt('Features', label='Number of Features')
14 	p[0].default = 25
15 	p[0].normMin = 1
16 	p[0].normMax = 250
18 	# create a custom parameter to specify minimum quality level
19 	# under which detected features would be rejected
20 	p = page.appendFloat('Quality', label='Minimum Quality Level')
21 	p[0].default = 0.01
22 	p[0].normMin = 0.001
23 	p[0].normMax = 1
25 	# create a custom parameter to specify the minimum distance
26 	# between detected features
27 	p = page.appendInt('Distance', label='Minimum Distance')
28 	p[0].default = 10
29 	p[0].normMin = 1
30 	p[0].normMax = 1200
31 	return
33 # called whenever custom pulse parameter is pushed
34 def onPulse(par):
35 	return
37 import numpy as np
38 import cv2
40 def onCook(scriptOp):
41 	scriptOp.clear()
43 	# read in parameters to see how many features to detect
44 	topRef = scriptOp.par.Top.eval()
45 	features = scriptOp.par.Features
46 	quality = scriptOp.par.Quality
47 	distance = scriptOp.par.Distance
49 	# default values
50 	xVals = []
51 	yVals = []
52 	corners = []
54 	if topRef:
55 		# read top as numpyArray
56 		img = topRef.numpyArray()
58 		# since we are reading from a gray scale TOP, throw out everything but red channel
59 		# we also can skip the cv2.cvtColor function you would see here otherwise for converting a color image to gray scale
60 		img = img[:,:,:1]
62 		# run goodFeaturesToTrack openCV function
63 		#
64 		corners = cv2.goodFeaturesToTrack(img,features,quality,distance)
66 		# slice array to have x and y positions split into 2 variables
67 		xVals = corners[:,:,0:1]
68 		yVals = corners[:,:,1:2]
70 	# setup the scriptOp with 2 channels
71 	# also set length to number of features that were detected
72 	scriptOp.rate = me.time.rate
73 	scriptOp.numSamples = len(corners)
74 	tx = scriptOp.appendChan('tx')
75 	ty = scriptOp.appendChan('ty')
77 	# assign values to channels
78 	tx.vals = xVals
79 	ty.vals = yVals
80 	return

Next Steps[edit]

There is a selection of introductory tutorials on the OpenCV website and OpenCV Tutorials

OpenCV C++ Documentation is here.

OpenCV License Agreement[edit]

TouchDesigner uses parts of OpenCV (the Blob Track TOP) under the following license.

Intel License Agreement.

Copyright (C) 2000, Intel Corporation, rights reserved. Third party copyrights are property of their respective owners.

Redistribution of OpenCV and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

  • Redistribution's of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
  • Redistribution's in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
  • The name of Intel Corporation may not be used to endorse or promote products derived from this software without specific prior written permission.

The OpenCV software is provided by the copyright holders and contributors "as is" and any express or implied warranties, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose are disclaimed. In no event shall the Intel Corporation or contributors be liable for any direct, indirect, incidental, special, exemplary, or consequential damages (including, but not limited to, procurement of substitute goods or services; loss of use, data, or profits; or business interruption) however caused and on any theory of liability, whether in contract, strict liability, or tort (including negligence or otherwise) arising in any way out of the use of this software, even if advised of the possibility of such damage.

A dialog box in which commands and scripts can entered manually. Script errors and python print() messages are also output to the textport.

An Operator Family that creates, composites and modifies images, and reads/writes images and movies to/from files and the network. TOPs run on the graphics card's GPU.

A set of commands located in a Text DAT that are triggered to run under certain conditions. There are two scripting languages in TouchDesigner: Python and the original Tscript. Scripts and single-line commands can also be run in the Textport.

An Operator Family which operate on Channels (a series of numbers) which are used for animation, audio, mathematics, simulation, logic, UI construction, and many other applications.

An Operator Family that manipulates text strings: multi-line text or tables. Multi-line text is often a command Script, but can be any multi-line text. Tables are rows and columns of cells, each containing a text string.