Experimental:Palette:tdPyEnvManager
Summary[edit]
The TDPyEnvManager is a custom TouchDesigner component that helps manage Python environments as well as Conda environments and helps TouchDesigner users and developers integrate easily and efficiently third party Python libraries that are installed in an external environment.
Users can rely on this component to create, activate, and manage Python vEnvs and Conda environments. Users can easily get to the activated environment's CLI and use pip or conda commands to install packages. Users can also export the environment to a requirements.txt or environment.yml file, and create a new environment from it.
Selected environments are added to the TouchDesigner Python path and packages installed in the environment are available to TouchDesigner.
Additionally, the Conda Env mode allows for users to search for environments available globally on the system, granted a valid Conda installation is found and the environments are matching the Python version of the TouchDesigner installation.
Users that wish to automatize the creation of environments can use the TDPyEnvManagerHelper as a CLI. More details at TDPyEnvManagerHelper.
💡 Support for UV can be added after creating a Python vEnv and activating the environment through the Open CLI pulse parameter. In the CLI, typepip install uv.
Considerations
When using the TDPyEnvManager in your projects, and when custom components are relying on third party libraries loaded from side loaded environments, it is preferable that you let the initialization of custom components being handheld lazily.
i.e. Do not force initialization of components using an Execute DAT or the Init On Start toggle.
The TDPyEnvManager should be the first component initializing on start so that the environment is setup before any additional component attempt to import modules from it.
To fail initialization gracefully, you can add a try / except condition around your import statements at the top of your extension.
try:
import tensorrt as trt
import torch
import torchvision.transforms as transforms
except Exception as e:
logger.error(f'TDDepthAnythingHF - An error occured trying to import some of the required libraries. Make sure that the environment is setup properly.')
logger.error(f'TDDepthAnythingHF - {e}')
When setting up your TDPyEnvManager and side environment, you can safely ignore issues from custom components. When your environment is setup properly and you installed the required dependencies, restart TouchDesigner and any import issue should be fixed.
Tips
When developing using VSCode, you can setup your code-workspace to load both TouchDesigner TDI library as well as the libraries installed in the sideloaded environment:
{
"folders": [
{
"path": "."
}
],
"settings": {
"python.defaultInterpreterPath": "C:\\Program Files\\Derivative\\TouchDesigner.2023.32465\\bin\\python.exe",
"python.analysis.autoSearchPaths": true,
"python.autoComplete.extraPaths": [
"C:\\Program Files\\Derivative\\TouchDesigner.2023.32465\\bin\\python\\Lib\\site-packages",
".\\scripts",
".\\TDDepthAnythingHF_vEnv\\Lib\\site-packages",
],
"python.analysis.extraPaths": [
"C:\\Program Files\\Derivative\\TouchDesigner.2023.32465\\bin\\python\\Lib\\site-packages",
".\\scripts",
".\\TDDepthAnythingHF_vEnv\\Lib\\site-packages",
],
}
}
In the example above, the interpreter is pointing to TouchDesigner’s own python binary and we added TouchDesigner `site-packages` as well as a Python vEnv plus a relative `scripts` folder.
Note
- Sideloading Python environments and third party packages is at your own risk and can cause instabilities. For more details, head to https://docs.derivative.ca/Python
- When using the Conda mode, users silently install Miniconda and agree to the Anaconda Inc. ToS. While the TDPyEnvManager locks on
conda-forgechannels when first installing Miniconda and creating environments, it is possible that an external configuration on the user's machine forceconda installcalls to install third party packages on the conda default channels. The default channel is subject to Anaconda Inc. ToS. More details regarding the license of the default channel at: https://www.anaconda.com/blog/is-conda-free
Parameters - TDPyEnvManager Page
General -
Active - Enable or disable the startup and initialization process of the component. Toggling active off avoid detection of a matching vEnv or conda env during startup or initialization of the component. This can be useful when debugging and prototyping but should be kept turned on for project delivery.
Openparametersdialog - Open a floating window of internal logger parameters dialog.
Status - The current status of the component.
Mode - ⊞ - The mode in which the component should run. Python vEnv are plug n play and are create relative to the .toe file with a name matching the parent folder. Conda envs require downloading and installing Miniconda if a valid Conda Install Folder is not specified upon creation of the conda env.
- Python vEnv
Python vEnv-
- Conda Env
Conda Env-
Reset - Reset the component internal states and trigger the post init sequence.
Restart - Restart TouchDesigner. While the component is trying its best at keeping the PATH and other variables clean when changing mode, creating environments... etc, restarting will guarantee a clean state using the latest settings. This will restart TouchDesigner and prompt the user for saving if unsaved changes are found. External components, if any, should be saved manually.
Pythonvenv -
Createvenv - If no existing Python vEnv is found relative to the .toe file, create a vEnv using TouchDesigner. The vEnv folder will be next to the .toe file and be named after the parent folder name followed by '_vEnv'. i.e. If the parent folder is named 'Banana', the vEnv folder will be 'Banana_vEnv'.
Exportrequirementstxt - Export a requirements.txt file with all unique packages currently installed in the vEnv, that are not part of the TouchDesigner build site-packages. Note: This is not an optimal requirements.txt file. You need to go through it manually and curate it to avoid setting too many version restrictions and possibly add a --extra-index-url https://some.url line within the requirements.txt file.
Createfromrequirementstxt - If a requirements.txt file is available next to the .toe file in which this TDPyEnvManager instance is used, create a vEnv and install all the packages in the requirements.txt file. If the vEnv already exists, install the packages in the existing vEnv.
Condaenv -
Condainstallfolder - The root folder of an exiting Anaconda or Miniconda installation, or the path where a Miniconda installation should be created. A Miniconda installation relative to the .toe file is a fast and efficient setup to prototype, develop and ship a project relying on a Miniconda environment.
Environment - ⊞ - The name of the environment to be created, or the name of an already existing environment in that installation. An environment name followed by a star (*) is a global environment. See parameter "Include global" for details. The environment selected, when / if exisiting, will be added to the PATH and its packages will be available to TouchDesigner.
- TDEnv
TDEnv-
Createcondaenv - Create a conda environment of the name matching the parameter Environment, in the conda install folder, if a valid conda installation is found. Otherwise, install Miniconda and create the environment of the name matching the parameter Environment in the freshly installed Miniconda installation. The created conda envs are using the conda-forge channels by default.
Keepcondainstaller - Keep the Miniconda installer once downloaded.
Includeglobal - When toggled on, include all the publicly available conda environments of the machine. Environments that are created in the conda installation specified in the Conda Install Folder parameter are always listed in the Environment parameter. The environments that are found using the conda API and are not in the specified conda install folder will be followed by a star (*).
Exportenvironmentyml - Export an environment.yml file with all unique packages currently installed in the selected conda environment. This include default packages, pip packages, conda specific packages and the python version the environment was created with as well as the conda channels used for that environment.
Createfromenvironmentyml - If an environment.yml file is available next to the .toe file in which this TDPyEnvManager instance is used, create a conda env and install all the packages listed in the environment.yml file. If the environment already exists, install the packages in the existing environment.
Refresh - Refresh the list of environments listed in Environment parameter drop down. Can be useful if an environment was created through the CLI or outside of the TouchDesigner context.
Extra -
Opencli - Open a CLI and activate the environment, whether it's a Python vEnv or a conda environment. You can use pip (in both Python vEnv and conda) and conda install as well as other environment specific commands.
Parameters - About Page
Help -
Version -
Toxsavebuild -
Operator Outputs
- Output 0 -
TouchDesigner Build:
| Palette |
|---|
| • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • • |