USD Asset Resolver

This repository holds reference implementations for Usd asset resolvers. The resolvers are compatible with the AR 2.0 standard proposed in the Asset Resolver 2.0 Specification. As the Usd documentation offers quite a good overview over the overall asset resolution system, we will not be covering it in this repository's documentation. You can find a video tutorial here (or on Youtube/Vimeo).

Installation

To build the various resolvers, follow the instructions in the install guide.

Feature Overview

Asset resolvers that can be compiled via this repository:

• Production Resolvers
  • File Resolver - A file system based resolver similar to the default resolver with support for custom mapping pairs as well as at runtime modification and refreshing.
  • Cached Resolver - A resolver that first consults an internal resolver context dependent cache to resolve asset paths. If the asset path is not found in the cache, it will redirect the request to Python and cache the result. This is ideal for smaller studios, as this preserves the speed of C++ with the flexibility of Python.
• RnD Resolvers
  • Python Resolver - Python based implementation of the file resolver. The goal of this resolver is to enable easier RnD by running all resolver and resolver context related methods in Python. It can be used to quickly inspect resolve calls and to setup prototypes of resolvers that can then later be re-written in C++ as it is easier to code database interactions in Python for initial research.
• Proof Of Concept Resolvers
  • Http Resolver - A proof of concept http resolver. This is kindly provided and maintained by @charlesfleche in the arHttp: Offloads USD asset resolution to an HTTP server repository. For documentation, feature suggestions and bug reports, please file a ticket there. This repo handles the auto-compilation against DCCs and exposing to the automatic installation update manager UI.

For more information check out the building guide as well as the individual resolvers to see their full functionality.

Video Tutorials

You can also find our OpenUSD playlists on Youtube and Vimeo.

Overview, installation and customization

In this video we cover how to (build and) install the resolvers in this repository as well as how you can customize them to fit your needs.

Installation

• Automatic Installation
• System Requirements
• Building/Deployment

Automatic/Quick Installation

Currently we only support doing this in Houdini and Maya.

Update Manager

Installing is done via the "USD Asset Resolver - Update Manager". Depending on the application, running the installer is a bit different, for more information check the specific app instructions below.

The update dialog will prompt you for an installation directory and offers you to choose between different releases and resolvers. Once you have made your choices, you can press install and the chosen resolver will be installed.

As mentioned in our Resolvers Plugin Configuration section, we need to setup a few environment variables before launching our application so that USD detects our resolver.

In your install directory you will find a "launch.sh/.bat" file, which does this for you based on what host app you ran the installer in. All you then have to do is run the "launch.sh/.bat" file by double clicking it and then your app should open as usual with the resolver running. In the launch file we have enabled the "TF_DEBUG=AR_RESOLVER_INIT" environment variable, so there will be debug logs where you can see if everything worked correctly.

Houdini

In Houdini we simply need to open the "Python Source Editor" from the "Windows" menu and run the following code to get access to the update manager. You should preferably do this in a clean Houdini session as a safety measure.

import ssl; from urllib import request
update_manager_url = 'https://raw.githubusercontent.com/LucaScheller/VFX-UsdAssetResolver/main/tools/update_manager.py?token=$(date+%s)'
exec(request.urlopen(update_manager_url,context=ssl._create_unverified_context()).read(), globals(), locals())
run_dcc()

Maya

In Maya we simply need to open the "Script Editor" and run the following code to get access to the update manager. You should preferably do this in a clean Maya session as a safety measure.

import ssl; from urllib import request
update_manager_url = 'https://raw.githubusercontent.com/LucaScheller/VFX-UsdAssetResolver/main/tools/update_manager.py?token=$(date+%s)'
exec(request.urlopen(update_manager_url,context=ssl._create_unverified_context()).read(), globals(), locals())
run_dcc()

Requirements

System dependencies

Currently building on Linux and Windows is documented. We use CMake as our build system and link against Houdini/Maya to avoid having to compile our own version of Usd.

It is also possible to compile against a self-compiled Usd build, this is not covered by this guide though.

VFX DCC vendors try to keep in sync with the versions specified in the VFX Reference Platform, so if something doesn't work, first make sure that your software versions are supported.

Linux

Windows

When compiling against Houdini/Maya on Windows, make sure you use the Visual Studio version that Houdini/Maya was compiled with as noted in the HDK/SDK. You'll also need to install the Visual Studio build tools that match the Visual Studio release if you want to run everything from the terminal.

Building

Currently we support building against Houdini and Maya on Linux and Windows. If you don't want to self-compile, you can also download pre-compiled builds on our release page. To load the resolver, you must specify a few environment variables, see our Resolvers > Environment Variables section for more details.

Setting up our build environment

After installing the requirements, we first need to set a couple of environment variables that our cmake file depends on.

Using our convenience setup script

On Linux we provide a bash script that you can source that sets up our development environment. This sets a few environment variables needed to build the resolver as well as for Houdini/Maya to load it. This can be done by running the following from the source directory:

source setup.sh

In the setup.sh file you can define what resolver to compile by setting the RESOLVER_NAME variable to one of the resolvers listed in resolvers in camelCase syntax (for example fileResolver or pythonResolver). Here you'll also have to define what Houdini/Maya version to compile against.

It will then automatically set the PATH, PYTHONPATH, PXR_PLUGINPATH_NAME and LD_LIBRARY_PATH environment variables to the correct paths so that after your run the compile, the resolver will be loaded correctly (e.g. if you launch Houdini via houdinifx, it will load everything correctly). The build process also logs this information again.

By default it also sets the TF_DEBUG env var to AR_RESOLVER_INIT so that you'll get logs of what resolver is loaded by USD's plugin system, which you can use to verify that everything is working correctly.

Manually setting up the environment

If you don't want to use our convenience script, you can also setup the environment manually.

# Linux
## Houdini
export AR_DCC_NAME=houdini
export HFS=<PathToHoudiniRoot> # For example "/opt/hfs<HoudiniVersion>"
## Maya
export AR_DCC_NAME=maya
export MAYA_USD_SDK_ROOT="/path/to/maya/usd/sdk/root/.../mayausd/USD"
export MAYA_USD_SDK_DEVKIT_ROOT="/path/to/maya/usd/sdk/root/.../content/of/devkit.zip"
export PYTHON_ROOT="/path/to/python/root"
## Resolver
export AR_RESOLVER_NAME=fileResolver

# Windows
## Houdini
set AR_DCC_NAME=houdini
set HFS=<PathToHoudiniRoot> # For example "C:\Program Files\Side Effects Software\<HoudiniVersion>"
## Maya
set AR_DCC_NAME=maya
set MAYA_USD_SDK_ROOT="/path/to/maya/usd/sdk/root/.../mayausd/USD"
set MAYA_USD_SDK_DEVKIT_ROOT="/path/to/maya/usd/sdk/root/.../content/of/devkit.zip"
set PYTHON_ROOT="/path/to/python/root"
## Resolver
set AR_RESOLVER_NAME=fileResolver

Running the build

To run the build, run:

    ...
    add_compile_definitions(_GLIBCXX_USE_CXX11_ABI=0)
    ...

# Linux
./build.sh
# Windows
build.bat

The build.sh/.bat files also contain (commented out) the environment definition part above, so alternatively just comment out the lines and you are good to go.

Testing the build

Unit tests are automatically run post-build on Linux using the Houdini/Maya version you are using. You can find each resolvers tests in its respective src//testenv folder.

Alternatively you can run Houdini/Maya and check if the resolver executes correctly. If you didn't use our convenience script as noted above, you'll have to specify a few environment variables, so that our plugin is correctly detected by USD.

Head over to our Resolvers > Environment Variables section on how to do this.

After that everything should run smoothly, you can try loading the examples in the "files" directory or work through our example setup section for a simple production example.

Customize build

If you want to further configure the build, you can head into the CMakeLists.txt in the root of this repo. In the first section of the file, you can configure various things, like the environment variables that the resolvers use, Python module namespaces and what resolvers to compile. This is a standard CMakeLists.txt file that you can also configure via CMake-GUI. If you don't want to use the build.sh bash script, you can also configure and compile this project like any other C++ project via this file.

Documentation

If you want to locally build this documentation, you'll have to download mdBook and mdBook-admonish and add their parent directories to the PATHenv variable so that the executables are found.

You can do this via bash (after running source setup.sh):

export MDBOOK_VERSION="0.4.28"
export MDBOOK_ADMONISH_VERSION="1.9.0"
curl -L https://github.com/rust-lang/mdBook/releases/download/v$MDBOOK_VERSION/mdbook-v$MDBOOK_VERSION-x86_64-unknown-linux-gnu.tar.gz | tar xz -C ${REPO_ROOT}/tools
curl -L https://github.com/tommilligan/mdbook-admonish/releases/download/v$MDBOOK_ADMONISH_VERSION/mdbook-admonish-v$MDBOOK_ADMONISH_VERSION-x86_64-unknown-linux-gnu.tar.gz | tar xz -C ${REPO_ROOT}/tools
export PATH=${REPO_ROOT}/tools:$PATH

You then can just run the following to build the documentation in html format:

./docs.sh

The documentation will then be built in docs/book.

Resolvers

Asset resolvers that can be compiled via this repository:

• Production Resolvers
  • File Resolver - A file system based resolver similar to the default resolver with support for custom mapping pairs as well as at runtime modification and refreshing.
  • Cached Resolver - A resolver that first consults an internal resolver context dependent cache to resolve asset paths. If the asset path is not found in the cache, it will redirect the request to Python and cache the result. This is ideal for smaller studios, as this preserves the speed of C++ with the flexibility of Python.
• RnD Resolvers
  • Python Resolver - Python based implementation of the file resolver. The goal of this resolver is to enable easier RnD by running all resolver and resolver context related methods in Python. It can be used to quickly inspect resolve calls and to setup prototypes of resolvers that can then later be re-written in C++ as it is easier to code database interactions in Python for initial research.
• Proof Of Concept Resolvers
  • Http Resolver - A proof of concept http resolver. This is kindly provided and maintained by @charlesfleche in the arHttp: Offloads USD asset resolution to an HTTP server repository. For documentation, feature suggestions and bug reports, please file a ticket there. This repo handles the auto-compilation against DCCs and exposing to the automatic installation update manager UI.

USD Plugin Configuration

In order for our plugin to be found by USD, we have to specify a few environment variables. Run this in your terminal before running your USD capable app. If your are using a pre-compiled release build, redirect the paths accordingly.

# Linux
export PYTHONPATH=${REPO_ROOT}/dist/${RESOLVER_NAME}/lib/python:${PYTHONPATH}
export PXR_PLUGINPATH_NAME=${REPO_ROOT}/dist/${RESOLVER_NAME}/resources:${PXR_PLUGINPATH_NAME}
export LD_LIBRARY_PATH=${REPO_ROOT}/dist/${RESOLVER_NAME}/lib
export TF_DEBUG=AR_RESOLVER_INIT # Debug Logs
# Windows
set PYTHONPATH=%REPO_ROOT%\dist\%RESOLVER_NAME%\lib\python;%PYTHONPATH%
set PXR_PLUGINPATH_NAME=%REPO_ROOT%\dist\%RESOLVER_NAME%\resources;%PXR_PLUGINPATH_NAME%
set PATH=%REPO_ROOT%\dist\%RESOLVER_NAME%\lib;%PATH%
set TF_DEBUG=AR_RESOLVER_INIT # Debug Logs

If it loads correctly, you'll see something like this in the terminal output:

ArGetResolver(): Found primary asset resolver types: [FileResolver, ArDefaultResolver]

Debugging

By using the TF_DEBUG environment variable

To check what resolver has been loaded, you can set the TF_DEBUG env variable to AR_RESOLVER_INIT:

export TF_DEBUG=AR_RESOLVER_INIT

ArGetResolver(): Found primary asset resolver types: [PythonResolver, ArDefaultResolver]
ArGetResolver(): Using asset resolver PythonResolver from plugin ${REPO_ROOT}/dist/pythonResolver/lib/pythonResolver.so for primary resolver
ArGetResolver(): Found URI resolver ArDefaultResolver
ArGetResolver(): Found URI resolver FS_ArResolver
ArGetResolver(): Using FS_ArResolver for URI scheme(s) ["op", "opdef", "oplib", "opdatablock"]
ArGetResolver(): Found URI resolver PythonResolver
ArGetResolver(): Found package resolver USD_NcPackageResolver
ArGetResolver(): Using package resolver USD_NcPackageResolver for usdlc from plugin usdNc
ArGetResolver(): Using package resolver USD_NcPackageResolver for usdnc from plugin usdNc
ArGetResolver(): Found package resolver Usd_UsdzResolver
ArGetResolver(): Using package resolver Usd_UsdzResolver for usdz from plugin usd

By loading the Python Module

When importing the Python module, be sure to first import the Ar module, otherwise you might run into errors, as the resolver is not properly initialized:

# Start python via the aliased `usdpython`
# Our sourced setup.sh aliases Houdini's standalone python to usdpython
# as well as sources extra libs like Usd
usdpython

# First import Ar, so that the resolver is initialized
from pxr import Ar
from usdAssetResolver import FileResolver

Production Resolvers

Here you can find production ready asset resolvers, checkout our Resolvers Overview section for an outline of their features:

• File Resolver
• Cached Resolver

File Resolver

Overview

This resolver is a file system based resolver similar to the default resolver with support for custom mapping pairs.

• A simple mapping pair look up in a provided mapping pair Usd file. The mapping data has to be stored in the Usd layer metadata in an key called mappingPairs as an array with the syntax ["sourcePathA.usd", "targetPathA.usd", "sourcePathB.usd", "targetPathB.usd"]. (This is quite similar to Rodeo's asset resolver that can be found here using the AR 1.0 specification.)
• The search path environment variable by default is AR_SEARCH_PATHS. It can be customized in the CMakeLists.txt file.
• You can use the AR_ENV_SEARCH_REGEX_EXPRESSION/AR_ENV_SEARCH_REGEX_FORMAT environment variables to preformat any asset paths before they looked up in the mappingPairs. The regex match found by the AR_ENV_SEARCH_REGEX_EXPRESSION environment variable will be replaced by the content of the AR_ENV_SEARCH_REGEX_FORMAT environment variable. The environment variable names can be customized in the CMakeLists.txt file.
• The resolver contexts are cached globally, so that DCCs, that try to spawn a new context based on the same mapping file using the Resolver.CreateDefaultContextForAsset, will re-use the same cached resolver context. The resolver context cache key is currently the mapping file path. This may be subject to change, as a hash might be a good alternative, as it could also cover non file based edits via the exposed Python resolver API.
• Resolver.CreateContextFromString/Resolver.CreateContextFromStrings is not implemented due to many DCCs not making use of it yet. As we expose the ability to edit the context at runtime, this is also often not necessary. If needed please create a request by submitting an issue here: Create New Issue
• You can adjust the resolver context content during runtime via exposed Python methods (More info here). Refreshing the stage is also supported, although it might be required to trigger additional reloads in certain DCCs.
• We optionally also support exposing alle path identifiers to our ResolverContext.ResolveAndCache Python method. This can be enabled by setting the AR_CACHEDRESOLVER_ENV_EXPOSE_ABSOLUTE_PATH_IDENTIFIERS environment variable to 1 or by calling pxr.Ar.GetUnderlyingResolver().SetExposeAbsolutePathIdentifierState(True). This then forwards any path to be run through our mapped pairs mapping, regardless of how the identifier is formatted.

Resolver Environment Configuration

• AR_SEARCH_PATHS: The search path for non absolute asset paths.
• AR_SEARCH_REGEX_EXPRESSION: The regex to preformat asset paths before mapping them via the mapping pairs.
• AR_SEARCH_REGEX_FORMAT: The string to replace with what was found by the regex expression.

The resolver uses these env vars to resolve non absolute asset paths relative to the directories specified by AR_SEARCH_PATHS. For example the following substitutes any occurrence of v<3digits> with v000 and then looks up that asset path in the mapping pairs.

```bash
export AR_SEARCH_PATHS="/workspace/shots:/workspace/assets"
export AR_SEARCH_REGEX_EXPRESSION="(v\d\d\d)"
export AR_SEARCH_REGEX_FORMAT="v000"
```

Debug Codes

Adding following tokens to the TF_DEBUG env variable will log resolver information about resolution/the context respectively.

• FILERESOLVER_RESOLVER
• FILERESOLVER_RESOLVER_CONTEXT

For example to enable it on Linux run the following before executing your program:

export TF_DEBUG=FILERESOLVER_RESOLVER_CONTEXT

Python API

Overview

You can import the Python module as follows:

from pxr import Ar
from usdAssetResolver import FileResolver

Tokens

Tokens can be found in FileResolver.Tokens:

FileResolver.Tokens.mappingPairs

Resolver

We also have the opt-in feature to expose any identifier, regardless of absolute/relative/search path based formatting to be run through our mapped pairs mapping by setting the AR_CACHEDRESOLVER_ENV_EXPOSE_ABSOLUTE_PATH_IDENTIFIERS environment variable to 1 or by calling pxr.Ar.GetUnderlyingResolver().SetExposeAbsolutePathIdentifierState(True). This then forwards any path to be run through our mapped pairs mapping, regardless of how the identifier is formatted.

from pxr import Ar, Usd
from usdAssetResolver import FileResolver

file_resolver = Ar.GetUnderlyingResolver()

# Enable absolute identifier resolving
file_resolver.SetExposeAbsolutePathIdentifierState(True)
print("Resolver is currently exposing absolute path identifiers to the mapping pairs | {}".format(file_resolver.GetExposeAbsolutePathIdentifierState()))
# Or set the "AR_FILERESOLVER_ENV_EXPOSE_ABSOLUTE_PATH_IDENTIFIERS" environment variable to 1.
# This can't be done via Python, as it has to happen before the resolver is loaded.
file_resolver.GetExposeAbsolutePathIdentifierState() # Get the state of exposing absolute path identifiers
file_resolver.SetExposeAbsolutePathIdentifierState() # Set the state of exposing absolute path identifiers

Resolver Context

You can manipulate the resolver context (the object that holds the configuration the resolver uses to resolve paths) via Python in the following ways:

from pxr import Ar, Usd
from usdAssetResolver import FileResolver

# Get via stage
stage = Usd.Stage.Open("/some/stage.usd")
context_collection = stage.GetPathResolverContext()
fileResolver_context = context_collection.Get()[0]
# Or context creation
fileResolver_context = FileResolver.ResolverContext()

# To print a full list of exposed methods:
for attr in dir(FileResolver.ResolverContext):
    print(attr)

Refreshing the Resolver Context

from pxr import Ar
resolver = Ar.GetResolver()
context_collection = stage.GetPathResolverContext()
fileResolver_context = context_collection.Get()[0]
# Make edits as described below to the context.
fileResolver_context.AddMappingPair("source.usd", "destination.usd")
# Trigger Refresh (Some DCCs, like Houdini, additionally require node re-cooks.)
resolver.RefreshContext(context_collection)

Search Paths

The search paths get read from the AR_SEARCH_PATHS env var by default. You can inspect and tweak the search paths at runtime by using the following methods:

ctx.GetSearchPaths()                        # Return all search paths (env and custom)
ctx.RefreshSearchPaths()                    # Reload env search paths and re-populates the search paths that the resolver uses. This must be called after changing the env var value or the custom search paths. 
ctx.GetEnvSearchPaths()                     # Return all env search paths 
ctx.GetCustomSearchPaths()                  # Return all custom search paths
ctx.SetCustomSearchPaths(searchPaths: list) # Set custom search paths

Mapping Pairs

To inspect/tweak the active mapping pairs, you can use the following:

ctx.GetMappingFilePath()                      # Get the mapping file path (Defaults file that the context created Resolver.CreateDefaultContextForAsset() opened)
ctx.SetMappingFilePath(p: str)                # Set the mapping file path
ctx.RefreshFromMappingFilePath()              # Reload mapping pairs from the mapping file path
ctx.GetMappingPairs()                         # Returns all mapping pairs as a dict
ctx.AddMappingPair(src: string, dst: str)     # Add a mapping pair
ctx.ClearMappingPairs()                       # Clear all mapping pairs
ctx.RemoveMappingByKey(src: str)              # Remove a mapping pair by key
ctx.RemoveMappingByValue(dst: str)            # Remove a mapping pair by value

To generate a mapping .usd file, you can do the following:

from pxr import Ar, Usd, Vt
from usdAssetResolver import FileResolver
stage = Usd.Stage.CreateNew('/some/path/mappingPairs.usda')
mapping_pairs = {'assets/assetA/assetA.usd':'assets/assetA/assetA_v005.usd', 'shots/shotA/shotA_v000.usd':'shots/shotA/shotA_v003.usd'}
mapping_array = []
for source_path, target_path in mapping_pairs.items():
    mapping_array.extend([source_path, target_path])
stage.SetMetadata('customLayerData', {FileResolver.Tokens.mappingPairs: Vt.StringArray(mapping_array)})
stage.Save()

To change the asset path formatting before it is looked up in the mapping pairs, you can do the following:

ctx.GetMappingRegexExpression()               # Get the regex expression
ctx.SetMappingRegexExpression(regex_str: str) # Set the regex expression
ctx.GetMappingRegexFormat()                   # Get the regex expression substitution formatting
ctx.SetMappingRegexFormat(f: str)             # Set the regex expression substitution formatting

Cached Resolver

Overview

By default (similar to the FileResolver and USD's default resolver), any absolute and relative file path is resolved as an on-disk file path. That means "normal" USD files, that don't use custom identifiers, will resolve as expected (and as fast as usual as this is called in C++).

All non file path identifiers (anything that doesn't start with "/", "./", "../") will forward their request to the PythonExpose.py -> ResolverContext.ResolveAndCache method. If you want to customize this resolver, just edit the methods in PythonExpose.py to fit your needs. You can either edit the file directly or move it anywhere where your "PYTHONPATH"/"sys.path" paths look for Python modules.

We also recommend checking out our unit tests of the resolver to see how to interact with it. You can find them in the "/src/CachedResolver/testenv" folder or on GitHub.

Here is a full list of features:

• We support adding caching pairs in two ways, cache-lookup-wise they do the same thing, except the "MappingPairs" have a higher priority than "CachedPairs":
  • MappingPairs: All resolver context methods that have Mapping in their name, modify the internal mappingPairs dictionary. As with the FileResolver and PythonResolver resolvers, mapping pairs get populated when creating a new context with a specified mapping file or when editing it via the exposed Python resolver context methods. When loading from a file, the mapping data has to be stored in the Usd layer metadata in an key called mappingPairs as an array with the syntax ["sourceIdentifierA.usd", "/absolute/targetPathA.usd", "sourceIdentifierB.usd", "/absolute/targetPathB.usd"]. (This is quite similar to Rodeo's asset resolver that can be found here using the AR 1.0 specification.). See our Python API page for more information.
  • CachingPairs: All resolver context methods that have Caching in their name, modify the internal cachingPairs dictionary. With this dictionary it is up to you when to populate it. In our PythonExpose.py file, we offer two ways where you can hook into the resolve process. In both of them you can add as many cached lookups as you want via ctx.AddCachingPair(asset_path, resolved_asset_path):
    • On context creation via the PythonExpose.py -> ResolverContext.Initialize method. This gets called whenever a context gets created (including the fallback default context). For example Houdini creates the default context if you didn't specify a "Resolver Context Asset Path" in your stage on the active node/in the stage network. If you do specify one, then a new context gets spawned that does the above mentioned mapping pair lookup and then runs the PythonExpose.py -> ResolverContext.Initialize method.
    • On resolve for non file path identifiers (anything that doesn't start with "/"/"./"/"../") via the PythonExpose.py -> ResolverContext.ResolveAndCache method. Here you are free to only add the active asset path via ctx.AddCachingPair(asset_path, resolved_asset_path) or any number of relevant asset paths.
• We optionally also support hooking into relative path identifier creation via Python. This can be enabled by setting the AR_CACHEDRESOLVER_ENV_EXPOSE_RELATIVE_PATH_IDENTIFIERS environment variable to 1 or by calling pxr.Ar.GetUnderlyingResolver().SetExposeRelativePathIdentifierState(True). We then have access in our PythonExpose.py -> Resolver.CreateRelativePathIdentifier method. Here we can then return a non file path (anything that doesn't start with "/"/"./"/"../") identifier for our relative path, which then also gets passed to our PythonExpose.py -> ResolverContext.ResolveAndCache method. This allows us to also redirect relative paths to our liking for example when implementing special pinning/mapping behaviours. For more info check out our production example section. As with our mapping and caching pairs, the result is cached in C++ to enable faster lookups on consecutive calls. As identifiers are context independent, the cache is stored on the resolver itself. See our Python API section on how to clear the cache.
• We optionally also support exposing alle path identifiers to our ResolverContext.ResolveAndCache Python method. This can be enabled by setting the AR_CACHEDRESOLVER_ENV_EXPOSE_ABSOLUTE_PATH_IDENTIFIERS environment variable to 1 or by calling pxr.Ar.GetUnderlyingResolver().SetExposeAbsolutePathIdentifierState(True). This then forwards any path to be run through our Python exposed method, regardless of how the identifier is formatted. Use this with care, we recommend only using this for debugging or when having a large dataset of pre-cached mapping pairs easily available.
• In comparison to our FileResolver and PythonResolver, the mapping/caching pair values need to point to the absolute disk path (instead of using a search path). We chose to make this behavior different, because in the "PythonExpose.py" you can directly customize the "final" on-disk path to your liking.
• The resolver contexts are cached globally, so that DCCs, that try to spawn a new context based on the same mapping file using the Resolver.CreateDefaultContextForAsset, will re-use the same cached resolver context. The resolver context cache key is currently the mapping file path. This may be subject to change, as a hash might be a good alternative, as it could also cover non file based edits via the exposed Python resolver API.
• Resolver.CreateContextFromString/Resolver.CreateContextFromStrings is not implemented due to many DCCs not making use of it yet. As we expose the ability to edit the context at runtime, this is also often not necessary. If needed please create a request by submitting an issue here: Create New Issue
• Refreshing the stage is also supported, although it might be required to trigger additional reloads in certain DCCs.

Debug Codes

Adding following tokens to the TF_DEBUG env variable will log resolver information about resolution/the context respectively.

• CACHEDRESOLVER_RESOLVER
• CACHEDRESOLVER_RESOLVER_CONTEXT

For example to enable it on Linux run the following before executing your program:

export TF_DEBUG=CACHEDRESOLVER_RESOLVER_CONTEXT

Overview

You can import the Python module as follows:

from pxr import Ar
from usdAssetResolver import CachedResolver

Tokens

Tokens can be found in CachedResolver.Tokens:

CachedResolver.Tokens.mappingPairs

Resolver

We optionally can also hook into relative path identifier creation via Python.

This can be enabled by setting the AR_CACHEDRESOLVER_ENV_EXPOSE_RELATIVE_PATH_IDENTIFIERS environment variable to 1 or by calling pxr.Ar.GetUnderlyingResolver().SetExposeRelativePathIdentifierState(True).

We then have access in our PythonExpose.py -> Resolver.CreateRelativePathIdentifier method. Here we can then return a non file path (anything that doesn't start with "/"/"./"/"../") identifier for our relative path, which then also gets passed to our PythonExpose.py -> ResolverContext.ResolveAndCache method.

This allows us to also redirect relative paths to our liking for example when implementing special pinning/mapping behaviours.

For more info check out our production example section.

As with our mapping and caching pairs, the result is cached in C++ to enable faster lookups on consecutive calls.

As identifiers are context independent, the cache is stored on the resolver itself. See below on how to modify and inspect the cache.

We also have the option to expose any identifier, regardless of absolute/relative/search path based formatting to our PythonExpose.py -> ResolverContext.ResolveAndCache method by setting the AR_CACHEDRESOLVER_ENV_EXPOSE_ABSOLUTE_PATH_IDENTIFIERS environment variable to 1 or by calling pxr.Ar.GetUnderlyingResolver().SetExposeAbsolutePathIdentifierState(True). As this then runs all paths through the Python exposed section, make sure that paths are batch added/pre-cached as much as possible to keep the resolve efficient.

from pxr import Ar, Usd
from usdAssetResolver import CachedResolver

cached_resolver = Ar.GetUnderlyingResolver()
# Enable relative identifier modification
cached_resolver.SetExposeRelativePathIdentifierState(True)
print("Resolver is currently exposing relative path identifiers to Python | {}".format(cached_resolver.GetExposeRelativePathIdentifierState()))
# Or set the "AR_CACHEDRESOLVER_ENV_EXPOSE_RELATIVE_PATH_IDENTIFIERS" environment variable to 1.
# This can't be done via Python, as it has to happen before the resolver is loaded.
cached_resolver.GetExposeRelativePathIdentifierState() # Get the state of exposing relative path identifiers
cached_resolver.SetExposeRelativePathIdentifierState() # Set the state of exposing relative path identifiers

# Enable absolute identifier resolving
cached_resolver.SetExposeAbsolutePathIdentifierState(True)
print("Resolver is currently exposing absolute path identifiers to Python | {}".format(cached_resolver.GetExposeAbsolutePathIdentifierState()))
# Or set the "AR_CACHEDRESOLVER_ENV_EXPOSE_ABSOLUTE_PATH_IDENTIFIERS" environment variable to 1.
# This can't be done via Python, as it has to happen before the resolver is loaded.
cached_resolver.GetExposeAbsolutePathIdentifierState() # Get the state of exposing absolute path identifiers
cached_resolver.SetExposeAbsolutePathIdentifierState() # Set the state of exposing absolute path identifiers

# We can also inspect and edit our relative identifiers via the following methods on the resolver.
# You'll usually only want to use these outside of the Resolver.CreateRelativePathIdentifier method only for debugging.
# The identifier cache is not context dependent (as identifiers are not).
cached_resolver.GetCachedRelativePathIdentifierPairs()       # Returns all cached relative path identifier pairs as a dict
cached_resolver.AddCachedRelativePathIdentifierPair()        # Add a cached relative path identifier pair
cached_resolver.RemoveCachedRelativePathIdentifierByKey()    # Remove a cached relative path identifier pair by key
cached_resolver.RemoveCachedRelativePathIdentifierByValue()  # Remove a cached relative path identifier pair by value
cached_resolver.ClearCachedRelativePathIdentifierPairs()     # Clear all cached relative path identifier pairs

Resolver Context

You can manipulate the resolver context (the object that holds the configuration the resolver uses to resolve paths) via Python in the following ways:

from pxr import Ar, Usd
from usdAssetResolver import CachedResolver

# Get via stage
stage = Usd.Stage.Open("/some/stage.usd")
context_collection = stage.GetPathResolverContext()
cachedResolver_context = context_collection.Get()[0]
# Or context creation
cachedResolver_context = CachedResolver.ResolverContext()

# To print a full list of exposed methods:
for attr in dir(CachedResolver.ResolverContext):
    print(attr)

Refreshing the Resolver Context

from pxr import Ar
from usdAssetResolver import CachedResolver
resolver = Ar.GetResolver()
stage = pxr.Usd.Stage.Open("/some/stage.usd")
context_collection = stage.GetPathResolverContext()
cachedResolver_context = context_collection.Get()[0]
# Make edits as described below to the context.
cachedResolver_context.AddMappingPair("identifier.usd", "/absolute/file/path/destination.usd")
# Trigger Refresh (Some DCCs, like Houdini, additionally require node re-cooks.)
resolver.RefreshContext(context_collection)

Editing the Resolver Context

We can edit the mapping and cache via the resolver context. We also use these methods in the PythonExpose.py module.

import json
stage = pxr.Usd.Stage.Open("/some/stage.usd")
context_collection = stage.GetPathResolverContext()
cachedResolver_context = context_collection.Get()[0]

# Mapping Pairs (Same as Caching Pairs, but have a higher loading priority)
cachedResolver_context.AddMappingPair("identifier.usd", "/absolute/file/path/destination.usd")
# Caching Pairs
cachedResolver_context.AddCachingPair("identifier.usd", "/absolute/file/path/destination.usd")
# Clear mapping and cached pairs (but not the mapping file path)
cachedResolver_context.ClearAndReinitialize()
# Load mapping from mapping file
cachedResolver_context.SetMappingFilePath("/some/mapping/file.usd")
cachedResolver_context.ClearAndReinitialize()

# Trigger Refresh (Some DCCs, like Houdini, additionally require node re-cooks.)
resolver.RefreshContext(context_collection)

When the context is initialized for the first time, it runs the ResolverContext.Initialize method as described below. Here you can add any mapping and/or cached pairs as you see fit.

Mapping/Caching Pairs

To inspect/tweak the active mapping/caching pairs, you can use the following:

ctx.ClearAndReinitialize()                    # Clear mapping and cache pairs and re-initialize context (with mapping file path)
ctx.GetMappingFilePath()                      # Get the mapping file path (Defaults to file that the context created via Resolver.CreateDefaultContextForAsset() opened")
ctx.SetMappingFilePath(p: str)                # Set the mapping file path
ctx.RefreshFromMappingFilePath()              # Reload mapping pairs from the mapping file path
ctx.GetMappingPairs()                         # Returns all mapping pairs as a dict
ctx.AddMappingPair(src: string, dst: str)     # Add a mapping pair
ctx.RemoveMappingByKey(src: str)              # Remove a mapping pair by key
ctx.RemoveMappingByValue(dst: str)            # Remove a mapping pair by value
ctx.ClearMappingPairs()                       # Clear all mapping pairs
ctx.GetCachingPairs()                         # Returns all caching pairs as a dict
ctx.AddCachingPair(src: string, dst: str)     # Add a caching pair
ctx.RemoveCachingByKey(src: str)              # Remove a caching pair by key
ctx.RemoveCachingByValue(dst: str)            # Remove a caching pair by value
ctx.ClearCachingPairs()                       # Clear all caching pairs

To generate a mapping .usd file, you can do the following:

from pxr import Ar, Usd, Vt
from usdAssetResolver import CachedResolver
stage = Usd.Stage.CreateNew('/some/path/mappingPairs.usda')
mapping_pairs = {'assets/assetA/assetA.usd':'/absolute/project/assets/assetA/assetA_v005.usd', '/absolute/project/shots/shotA/shotA_v000.usd':'shots/shotA/shotA_v003.usd'}
mapping_array = []
for source_path, target_path in mapping_pairs.items():
    mapping_array.extend([source_path, target_path])
stage.SetMetadata('customLayerData', {CachedResolver.Tokens.mappingPairs: Vt.StringArray(mapping_array)})
stage.Save()

PythonExpose.py Overview

As described in our overview section, the cache population is handled completely in Python, making it ideal for smaller studios, who don't have the C++ developer resources.

You can find the basic implementation version that gets shipped with the compiled code here: PythonExpose.py.

Below we show the Python exposed methods, note that we use static methods, as we just call into the module and don't create the actual object. (This module could just as easily been made up of pure functions, we just create the classes here to make it match the C++ API.)

To enable a similar logging as the TF_DEBUG env var does, you can uncomment the following in the log_function_args function.

...code...
def log_function_args(func):
    ...code...
    # To enable logging on all methods, re-enable this.
    # LOG.info(f"{func.__module__}.{func.__qualname__} ({func_args_str})")
...code...

Resolver

class Resolver:

    @staticmethod
    @log_function_args
    def CreateRelativePathIdentifier(resolver, anchoredAssetPath, assetPath, anchorAssetPath):
        """Returns an identifier for the asset specified by assetPath.
        It is very important that the anchoredAssetPath is used as the cache key, as this
        is what is used in C++ to do the cache lookup.

        We have two options how to return relative identifiers:
        - Make it absolute: Simply return the anchoredAssetPath. This means the relative identifier
                            will not be passed through to ResolverContext.ResolveAndCache.
        - Make it non file based: Make sure the remapped identifier does not start with "/", "./" or"../"
                                  by putting some sort of prefix in front of it. The path will then be
                                  passed through to ResolverContext.ResolveAndCache, where you need to re-construct
                                  it to an absolute path of your liking. Make sure you don't use a "<somePrefix>:" syntax,
                                  to avoid mixups with URI based resolvers.

        Args:
            resolver (CachedResolver): The resolver
            anchoredAssetPath (str): The anchored asset path, this has to be used as the cached key.
            assetPath (str): An unresolved asset path.
            anchorAssetPath (Ar.ResolvedPath): A resolved anchor path.

        Returns:
            str: The identifier.
        """
        remappedRelativePathIdentifier = f"relativePath|{assetPath}?{anchorAssetPath}"
        resolver.AddCachedRelativePathIdentifierPair(anchoredAssetPath, remappedRelativePathIdentifier)
        return remappedRelativePathIdentifier

Resolver Context

class ResolverContext:
    @staticmethod
    def Initialize(context):
        """Initialize the context. This get's called on default and post mapping file path
        context creation.

        Here you can inject data by batch calling context.AddCachingPair(assetPath, resolvePath),
        this will then populate the internal C++ resolve cache and all resolves calls
        to those assetPaths will not invoke Python and instead use the cache.

        Args:
            context (CachedResolverContext): The active context.
        """
        # Very important: In order to add a path to the cache, you have to call:
        #    context.AddCachingPair(assetPath, resolvedAssetPath)
        # You can add as many identifier->/abs/path/file.usd pairs as you want.
        context.AddCachingPair("identifier", "/some/path/to/a/file.usd")

    @staticmethod
    def ResolveAndCache(context, assetPath):
        """Return the resolved path for the given assetPath or an empty
        ArResolvedPath if no asset exists at that path.
        Args:
            context (CachedResolverContext): The active context.
            assetPath (str): An unresolved asset path.
        Returns:
            str: The resolved path string. If it points to a non-existent file,
                 it will be resolved to an empty ArResolvedPath internally, but will
                 still count as a cache hit and be stored inside the cachedPairs dict.
        """
        # Very important: In order to add a path to the cache, you have to call:
        #    context.AddCachingPair(assetPath, resolvedAssetPath)
        # You can add as many identifier->/abs/path/file.usd pairs as you want.
        resolved_asset_path = "/some/path/to/a/file.usd"
        context.AddCachingPair(assetPath, resolved_asset_path)
        return resolved_asset_path

Production Setup

In this example we examine how a possible production setup would look like.

We enable the advanced feature of exposing relative identifiers to Python by setting the AR_CACHEDRESOLVER_ENV_EXPOSE_RELATIVE_PATH_IDENTIFIERS environment variable to 1.

Prerequisites

To run the example, we must first initialize our environment.

If you are using the pre-compiled builds, make sure that you adjust the paths accordingly.

# Linux
export PYTHONPATH=${REPO_ROOT}/files/implementations/CachedResolver/code:${REPO_ROOT}/dist/${RESOLVER_NAME}/lib/python:${PYTHONPATH}
export PXR_PLUGINPATH_NAME=${REPO_ROOT}/dist/${RESOLVER_NAME}/resources:${PXR_PLUGINPATH_NAME}
export LD_LIBRARY_PATH=${REPO_ROOT}/dist/${RESOLVER_NAME}/lib
export AR_CACHEDRESOLVER_ENV_EXPOSE_RELATIVE_PATH_IDENTIFIERS=1
# Windows
set PYTHONPATH=%REPO_ROOT%\files\implementations\CachedResolver\code;%REPO_ROOT%\dist\%RESOLVER_NAME%\lib\python;%PYTHONPATH%
set PXR_PLUGINPATH_NAME=%REPO_ROOT%\dist\%RESOLVER_NAME%\resources;%PXR_PLUGINPATH_NAME%
set PATH=%REPO_ROOT%\dist\%RESOLVER_NAME%\lib;%PATH%
set AR_CACHEDRESOLVER_ENV_EXPOSE_RELATIVE_PATH_IDENTIFIERS=1

You'll also need to adjust the shotA_mapping.usd to point to absolute file paths:

#usda 1.0
(
    customLayerData = {
        string[] mappingPairs = [
            "assets/assetA", "<Change/This/Path/To/Be/Absolute>/files/implementations/CachedResolver/workspace/assets/assetA/assetA_v002.usd",
            "relativeIdentifier|assets/assetA?surface", "<Change/This/Path/To/Be/Absolute>/files/implementations/CachedResolver/workspace/assets/assetA/elements/surface_v001.usd"
        ]
    }
)

Demo

Let's have a look how we can demo this setup in Houdini.

Loading our Shot

If everything was initialized correctly, we can sublayer in the shot A USD file by referring to it via shots/shotsA.

Let's inspect what is happening in our `PythonExpose.py`` file:

...
class ResolverContext:

    @staticmethod
    @log_function_args
    def ResolveAndCache(context, assetPath):
        """Return the resolved path for the given assetPath or an empty
        ...
        """
        ...
        resolved_asset_path = ""
        if assetPath.startswith(RELATIVE_PATH_IDENTIFIER_PREFIX):
            ...
        else:
            ####### DOCS
            """ Since our asset identifier doesn't start with our relative path prefix (more on that below),
            the resolve code here is executed. Here we simply map our entity (asset/shot)
            to a specific version, usually the latest one. You'd replace this with a
            database call or something similar. You can also batch call .AddCachingPair
            for multiple paths here (or preferably in the ResolverContext.Initialize method).
            """
            #######
            entity_type, entity_identifier = assetPath.split("/")
            # Here you would add your custom "latest version" query.
            resolved_asset_path = os.path.join(ENTITY_TYPE_TO_DIR_PATH[entity_type],
                                               entity_identifier,
                                               f"{entity_identifier}_v002.usd")
        # Cache result
        context.AddCachingPair(assetPath, resolved_asset_path)
        return resolved_asset_path
...

So far so good? Alright, then let's look at some pinning examples.

Different pipelines use different mechanisms of loading specific versions. These two ways are the most common ones:

• Opt-In: A user has to manually opt-in to loading specific/the latest versions
• Opt-Out: The user always gets the latest version, but can optionally opt-out and pin/map specific versions that should not change.

With both methods, we need a pinning mechanism (as well as when we typically submit to a render farm as nothing should change there).

We provide this by either live modifying the context (see the Python API section) or by providing a pinning file.

Loading Pinning

We can load our pinning via a configure stage node or via the lop network settings, more info on that can be found in the USD Survival Guide - Asset Resolver section.

Let's have a look at the content of our pinning file.

#usda 1.0
(
    customLayerData = {
        string[] mappingPairs = [
            "assets/assetA", "<Change/This/Path/To/Be/Absolute>/files/implementations/CachedResolver/workspace/assets/assetA/assetA_v002.usd",
            "relativeIdentifier|assets/assetA?surface", "<Change/This/Path/To/Be/Absolute>/files/implementations/CachedResolver/workspace/assets/assetA/elements/surface_v001.usd"
        ]
    }
)

Here we decided to pin/map two things:

• An asset
• An element of an asset

In a production scenario you would pin everything when submitting to the render farm/when saving a scene to keep the state of the scene re-constructable. We decided to not do this here, to showcase the more complicated case of mapping only specific relative paths.

In this case the result is that the v002 of our assetA is being loaded, but with v001 of our surface version and v002 of our model version. Where does the version v002 for our model come from?

When opt-in-ing to expose relative identifiers to Python, the method below gets called.

...
class Resolver:

    @staticmethod
    @log_function_args
    def CreateRelativePathIdentifier(resolver, anchoredAssetPath, assetPath, anchorAssetPath):
        """Returns an identifier for the asset specified by assetPath and anchor asset path.
        ...
        """
        LOG.debug("::: Resolver.CreateRelativePathIdentifier | {} | {} | {}".format(anchoredAssetPath, assetPath, anchorAssetPath))
        # For this example, we assume all identifier are anchored to the shot and asset directories.
        # We remove the version from the identifier, so that our mapping files can target a version-less identifier.
        anchor_path = anchorAssetPath.GetPathString()
        anchor_path = anchor_path[:-1] if anchor_path[-1] == "/" else anchor_path[:anchor_path.rfind("/")]
        entity_type = os.path.basename(os.path.dirname(anchor_path))
        entity_identifier = os.path.basename(anchor_path)
        entity_element = os.path.basename(assetPath).split("_")[0]

        remapped_relative_path_identifier = f"{RELATIVE_PATH_IDENTIFIER_PREFIX}{entity_type}/{entity_identifier}?{entity_element}"
        resolver.AddCachedRelativePathIdentifierPair(anchoredAssetPath, remapped_relative_path_identifier)

        # If you don't want this identifier to be passed through to ResolverContext.ResolveAndCache
        # or the mapping/caching mechanism, return this:
        # resolver.AddCachedRelativePathIdentifierPair(anchoredAssetPath, anchoredAssetPath)
        # return anchoredAssetPath

        return remapped_relative_path_identifier
...

In our resolver we then map all relative file path identifiers to their originally intended paths, either by using our mapping or by using our "fallback"/original encoded version. (As a side note in case you forgot: Only identifiers that do not have a match in our mapping pair dict get piped through to ResolveAndCache method. So in this case our asset element model and surface files.)

...
class ResolverContext:

    @staticmethod
    @log_function_args
    def ResolveAndCache(context, assetPath):
        """Return the resolved path for the given assetPath or an empty
        ...
        """
        ...
        resolved_asset_path = ""
        if assetPath.startswith(RELATIVE_PATH_IDENTIFIER_PREFIX):
            ####### DOCS
            """The v002 version of our model .usd file and v001 of our surface model .usd file come from our resolve cache method.
            For our model file, we extract the version from our identifier, for our surface file we use our mapping pairs.
            The later means that we first have a mapping pair cache hit miss (in the C++ resolver code) and therefore ResolveAndCache gets called, which then
            re-applies the correct mapping. If the identifier is encountered again it will use the C++ cache, which means everything is kept fast.
            """
            #######
            base_identifier = assetPath.removeprefix(RELATIVE_PATH_IDENTIFIER_PREFIX)
            anchor_path, entity_element = base_identifier.split("?")
            entity_type, entity_identifier = anchor_path.split("/")
            entity_element, entity_version = entity_element.split("-")
            # Here you would add your custom relative path resolve logic.
            # We can test our mapping pairs to see if the version is pinned, otherwise we fallback to the original intent.
            versionless_identifier = f"{RELATIVE_PATH_IDENTIFIER_PREFIX}{entity_type}/{entity_identifier}?{entity_element}"
            mapping_pairs = context.GetMappingPairs()
            mapping_hit = mapping_pairs.get(versionless_identifier)
            if mapping_hit:
                resolved_asset_path = mapping_hit
            else:
                resolved_asset_path = os.path.normpath(os.path.join(ENTITY_TYPE_TO_DIR_PATH[entity_type],
                                                                    entity_identifier,
                                                                    "elements", f"{entity_element}_{entity_version}.usd"))
        else:
            ...
        # Cache result
        context.AddCachingPair(assetPath, resolved_asset_path)
        return resolved_asset_path
...

Summary

And that's all folks! We encourage you to also play around with the code or adjusting the mapping files to see how everything works.

If you make live adjustments via the API, don't forget to refresh the context as described in our Python API section.

Content Structure

To make the example setup a bit simpler, our shot setup does not contain any shot layers. In a real production setup it would be setup similar to our assets.

Shots

Content of a USD file located at /workspace/shots/shotA/shot_v002.usd

#usda 1.0
def Xform "testAssetA_1" (
    prepend references = @assets/assetA@</asset>
)
{
}

def Xform "testAssetA_2" (
    prepend references = @assets/assetA@</asset>
)
{
    matrix4d xformOp:transform:transform1 = ( (1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0), (0, 10, 0, 1) )
    uniform token[] xformOpOrder = ["xformOp:transform:transform1"]
}

Content of a USD file located at /workspace/shots/shotA/shot_v001.usd

#usda 1.0
def Xform "testAssetA_1" (
    prepend references = @assets/assetA@</asset>
)
{
}

def Xform "testAssetA_2" (
    prepend references = @assets/assetA@</asset>
)
{
    matrix4d xformOp:transform:transform1 = ( (1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0), (0, 3, 0, 1) )
    uniform token[] xformOpOrder = ["xformOp:transform:transform1"]
}

Assets

Content of a USD file located at /workspace/assets/assetA/assetA_v002.usd

#usda 1.0
def Xform "asset" (
    prepend references = [@./elements/model_v002.usd@</asset>, @./elements/surface_v002.usd@</asset>]
)
{
}

Content of a USD file located at /workspace/assets/assetA/assetA_v001.usd

#usda 1.0
def Xform "asset" (
    prepend references = [@./elements/model_v001.usd@</asset>, @./elements/surface_v001.usd@</asset>]
)
{
}

Asset Elements

Model

Content of a USD file located at /workspace/assets/assetA/elements/model_v001.usd

#usda 1.0
def "asset" ()
{
    def Cube "shape" ()
    {
        double size = 2
    }
}

Content of a USD file located at /workspace/assets/assetA/elements/model_v002.usd

#usda 1.0
def "asset" ()
{
    def Cylinder "shape" ()
    {
    }
}

Surface

Content of a USD file located at /workspace/assets/assetA/elements/surface_v001.usd

#usda 1.0
def "asset" ()
{
    color3f[] primvars:displayColor = [(1, 0, 0)] (
        interpolation = "constant"
    )
}

Content of a USD file located at /workspace/assets/assetA/elements/surface_v002.usd

#usda 1.0
def "asset" ()
{
    color3f[] primvars:displayColor = [(0, 1, 0)] (
        interpolation = "constant"
    )
}

RnD Resolvers

Here you can find RnD asset resolvers, checkout our Resolvers Overview section for an outline of their features:

• Python Resolver

These resolvers can be used as boiler plate code to build more advanced resolvers or to prototype features more easily.

Python Resolver

Overview

Python based implemention of the file resolver:

• A simple mapping pair look up in a provided mapping pair Usd file. The mapping data has to be stored in the Usd layer metadata in an key called mappingPairs as an array with the syntax ["sourcePathA.usd", "targetPathA.usd", "sourcePathB.usd", "targetPathB.usd"]. (This is quite similar to Rodeo's asset resolver that can be found here using the AR 1.0 specification.)
• The search path environment variable by default is AR_SEARCH_PATHS. It can be customized in the CMakeLists.txt file.
• You can use the AR_ENV_SEARCH_REGEX_EXPRESSION/AR_ENV_SEARCH_REGEX_FORMAT environment variables to preformat any asset paths before they looked up in the mappingPairs. The regex match found by the AR_ENV_SEARCH_REGEX_EXPRESSION environment variable will be replaced by the content of the AR_ENV_SEARCH_REGEX_FORMAT environment variable. The environment variable names can be customized in the CMakeLists.txt file.
• The resolver contexts are cached globally, so that DCCs, that try to spawn a new context based on the same mapping file using the Resolver.CreateDefaultContextForAsset, will re-use the same cached resolver context. The resolver context cache key is currently the mapping file path. This may be subject to change, as a hash might be a good alternative, as it could also cover non file based edits via the exposed Python resolver API.
• Resolver.CreateContextFromString/Resolver.CreateContextFromStrings is not implemented due to many DCCs not making use of it yet. As we expose the ability to edit the context at runtime, this is also often not necessary. If needed please create a request by submitting an issue here: Create New Issue
• This resolver has feature parity to the file resolver, but the implementation is slightly different. The goal of this resolver is to enable easier RnD by running all resolver and resolver context related methods in Python. It can be used to quickly inspect resolve calls and to setup prototypes of resolvers that can then later be re-written in C++ as it is easier to code database related pipelines in Python.
• Running in Python does not allow proper multithreading due to Python's Global Interpreter Lock, so this resolver should not be used in (large scale) productions.

Resolver Environment Configuration

• AR_SEARCH_PATHS: The search path for non absolute asset paths.
• AR_SEARCH_REGEX_EXPRESSION: The regex to preformat asset paths before mapping them via the mapping pairs.
• AR_SEARCH_REGEX_FORMAT: The string to replace with what was found by the regex expression.

The resolver uses these env vars to resolve non absolute asset paths relative to the directories specified by AR_SEARCH_PATHS. For example the following substitutes any occurrence of v<3digits> with v000 and then looks up that asset path in the mapping pairs.

```bash
export AR_SEARCH_PATHS="/workspace/shots:/workspace/assets"
export AR_SEARCH_REGEX_EXPRESSION="(v\d\d\d)"
export AR_SEARCH_REGEX_FORMAT="v000"
```

Debug Codes

Adding following tokens to the TF_DEBUG env variable will log resolver information about resolution/the context respectively.

• PYTHONRESOLVER_RESOLVER
• PYTHONRESOLVERR_RESOLVER_CONTEXT

For example to enable it on Linux run the following before executing your program:

export TF_DEBUG=PYTHONRESOLVERR_RESOLVER_CONTEXT

Overview

You can import the Python module as follows:

from pxr import Ar
from usdAssetResolver import PythonResolver

Tokens

Tokens can be found in PythonResolver.Tokens:

PythonResolver.Tokens.mappingPairs
PythonResolver.Tokens.mappingRegexExpression
PythonResolver.Tokens.mappingRegexFormat
PythonResolver.Tokens.searchPaths

Resolver Context

You can manipulate the resolver context (the object that holds the configuration the resolver uses to resolve paths) via Python in the following ways:

from pxr import Ar, Usd
from usdAssetResolver import PythonResolver

# Get via stage
stage = Usd.Stage.Open("/some/stage.usd")
context_collection = stage.GetPathResolverContext()
pythonResolver_context = context_collection.Get()[0]
# Or context creation
pythonResolver_context = PythonResolver.ResolverContext()

# To print a full list of exposed methods:
for attr in dir(PythonResolver.ResolverContext):
    print(attr)

Refreshing the Resolver Context

from pxr import Ar
from usdAssetResolver import PythonResolver
resolver = Ar.GetResolver()
stage = pxr.Usd.Stage.Open("/some/stage.usd")
context_collection = stage.GetPathResolverContext()
pythonResolver_context = context_collection.Get()[0]
# Make edits as described below to the context.
pythonResolver_context.SetData("{'mappingPairs': {'source.usd': 'destination.usd'}, 'searchPaths': ['/path/A']}")
# Trigger Refresh (Some DCCs, like Houdini, additionally require node re-cooks.)
resolver.RefreshContext(context_collection)

Editing the Resolver Context

Since the resolver is implemented in Python, the actual context data is a serialized json dict. It can be retrieved/edited/set as follows:

import json
stage = pxr.Usd.Stage.Open("/some/stage.usd")
context_collection = stage.GetPathResolverContext()
pythonResolver_context = context_collection.Get()[0]
# Load context
data = json.loads(pythonResolver_context.GetData())
# Manipulate data
data[PythonResolver.Tokens.mappingPairs]["sourceAdd.usd"] = "targetAdd.usd"
# Set context
pythonResolver_context.SetData(json.dumps(data))
# Trigger Refresh (Some DCCs, like Houdini, additionally require node re-cooks.)
resolver.RefreshContext(context_collection)

When the context is initialized for the first time, it runs the ResolverContext.LoadOrRefreshData method as described below. After that is is just a serialized .json dict with at minimum the PythonResolver.Tokens.mappingPairsand PythonResolver.Tokens.searchPaths tokens being set.

Additionally the PythonResolver.Tokens.mappingRegexExpression/PythonResolver.Tokens.mappingRegexFormat keys can be set to support regex substitution before doing the mapping pair lookup.

PythonExpose.py Overview

The rest of the Python API is actually the fully exposed resolver.

You can find the fully implemented version that gets shipped with the compiled code here: PythonExpose.py. This file has feature parity to the File Resolver. It is also quite close to the native default file resolver that ships with Usd.

Below we show the Python exposed methods, note that we use static methods, as we just call into the module and don't create the actual object. (This module could just as easily been made up of pure functions, we just create the classes here to make it match the C++ API.)

The method signatures match the C++ signatures, except how the context is injected, as this is necessary due to how the Python exposing works.

To enable a similar logging as the TF_DEBUG env var does, you can uncomment the following in the log_function_args function.

...code...
def log_function_args(func):
    ...code...
    # To enable logging on all methods, re-enable this.
    # LOG.info(f"{func.__module__}.{func.__qualname__} ({func_args_str})")
...code...

Resolver

class Resolver:
    @staticmethod
    def _CreateIdentifier(assetPath, anchorAssetPath, serializedContext, serializedFallbackContext):
        """Returns an identifier for the asset specified by assetPath.
        If anchorAssetPath is not empty, it is the resolved asset path
        that assetPath should be anchored to if it is a relative path.
        Args:
            assetPath (str): An unresolved asset path.
            anchorAssetPath (Ar.ResolvedPath): An resolved anchor path.
            serializedContext (str): The serialized context.
            serializedFallbackContext (str): The serialized fallback context.
        Returns:
            str: The identifier.
        """
        ... code ...
    @staticmethod
    def _CreateIdentifierForNewAsset(assetPath, anchorAssetPath):
        """Return an identifier for a new asset at the given assetPath.
        This is similar to _CreateIdentifier but is used to create identifiers
        for assets that may not exist yet and are being created.
        Args:
            assetPath (str): An unresolved asset path.
            anchorAssetPath (Ar.ResolvedPath): An resolved anchor path.
        Returns:
            str: The identifier.
        """
        ... code ...
    @staticmethod
    def _Resolve(assetPath, serializedContext, serializedFallbackContext):
        """Return the resolved path for the given assetPath or an empty
        ArResolvedPath if no asset exists at that path.
        Args:
            assetPath (str): An unresolved asset path.
        Returns:
            Ar.ResolvedPath: The resolved path.
        """
        ... code ...
    @staticmethod
    def _ResolveForNewAsset(assetPath):
        """Return the resolved path for the given assetPath that may be
        used to create a new asset or an empty ArResolvedPath if such a
        path cannot be computed.
        Args:
            assetPath (str): An unresolved asset path.
        Returns:
            Ar.ResolvedPath: The resolved path.
        """
        ... code ...
    @staticmethod
    def _IsContextDependentPath(assetPath):
        """Returns true if assetPath is a context-dependent path, false otherwise.
        Args:
            assetPath (str): An unresolved asset path.
        Returns:
            bool: The context-dependent state.
        """
        ... code ...
    @staticmethod
    def _GetModificationTimestamp(assetPath, resolvedPath):
        """Return an ArTimestamp representing the last time the asset at assetPath was modified.
        Args:
            assetPath (str): An unresolved asset path.
            resolvePath (Ar.ResolvedPath): A resolved path.
        Returns:
            Ar.Timestamp: The timestamp.
        """
        ... code ...

Resolver Context

class ResolverContext:
    @staticmethod
    def LoadOrRefreshData(mappingFilePath, searchPathsEnv, mappingRegexExpressionEnv, mappingRegexFormatEnv):
        """Load or refresh the mapping pairs from file and the search paths from the
        configured environment variables.
        Args:
            mappingFilePath(str): The mapping .usd file path
            searchPathsEnv(str): The search paths environment variable
            mappingRegexExpressionEnv(str): The mapping regex expression environment variable
            mappingRegexFormatEnv(str): The mapping regex format environment variable
        Returns:
            str: A serialized json dict that can be used as a context.
        """
        ... code ...

Proof Of Concept Resolvers

Here you can find proof of concept asset resolvers, checkout our Resolvers Overview section for an outline of their features:

• Http Resolver

These resolvers demonstrate different concepts of how resolvers can be built and are often minimal in their implementation.

Http Resolver (arHttp @charlesfleche)

This is a proof of concept http resolver. This is kindly provided and maintained by @charlesfleche in the arHttp: Offloads USD asset resolution to an HTTP server repository.

For documentation, feature suggestions and bug reports, please file a ticket there.

This repo handles the auto-compilation against DCCs and exposing to the automatic installation update manager UI.

Running the demo server

As the resolver reacts to http requests, we need to setup a local server to answer to our requests.

Example Usd files and mapping pair files

Simple Example

A very simple setup can be found in In the <REPO_ROOT>/files folder.

Before launching a Usd related software, you'll have to set these env vars:

export AR_SEARCH_PATHS=${REPO_ROOT}/files/generic
export AR_SEARCH_REGEX_EXPRESSION="(bo)"
export AR_SEARCH_REGEX_FORMAT="Bo"

The source setup.sh bash script in the root of this repo also does this for your automatically. After that load the box.usda in the application of your choice. Make sure to load box.usdaand not /path/to/repo/box.usda as the resolver only resolves paths that use the search path mechanism. You should now see a cube. If you set the resolver context to the mapping.usda file, it will be replaced to a cylinder.

Production Example

A larger example scene setup might looks as follows:

• The following files on disk:
  • /workspace/shots/shotA/shotA.usd
  • /workspace/shots/shotA/shotA_mapping.usd
  • /workspace/assets/assetA/assetA.usd
  • /workspace/assets/assetA/assetA_v001.usd
  • /workspace/assets/assetA/assetA_v002.usd
• The AR_SEARCH_PATHS environment variable being set to /workspace/shots:/workspace/assets

In the <REPO_ROOT>/files folder you can also find this setup. To run it, you must set the AR_SEARCH_PATHS env var as follows.

export AR_SEARCH_PATHS=${REPO_ROOT}/files/generic/workspace/shots:${REPO_ROOT}/files/generic/workspace/assets

And then open up the shots/shotA/shotA.usd file and set the resolver context mapping file path to shots/shotA/shotA_mapping.usd.

In Houdini this is done by loading the shot file via a sublayer node and setting the Resolver Context Asset Path parm to the mapping file path in the Scene Graph Tree>Opens the parameter dialog for the current LOP Network button.

You'll see the box being replaced to cylinder.

Content structure

Content of a USD file located at /workspace/shots/shotA/shotA.usd

#usda 1.0
def "testAssetA" (
    prepend references = @assetA/assetA.usd@</asset>
)
{
}

Content of the USD file located at /workspace/shots/shotA/shotA_mapping.usd

#usda 1.0
(
    customLayerData = {
        string[] mappingPairs = ["assetA/assetA.usd", "assetA/assetA_v002.usd"]
    }
)

Content of the USD files located at /workspace/assets/assetA/assetA.usd and /workspace/assets/assetA/assetA_v001.usd

#usda 1.0
def Cube "asset" ()
{
    double size = 2
}

Content of the USD file located at /workspace/assets/assetA/assetA.usd and /workspace/assets/assetA/assetA_v002.usd

#usda 1.0
def Cylinder "asset" ()
{
}