Siggraph Presentation

This guide will be officially introduced at Siggraph 2023 - Houdini Hive on Wednesday, 9. of August 2023 at 11:00 AM PST.

Collections

Collections are USD's mechanism of storing a set of prim paths. We can nest/forward collections to other collections and relationships, which allows for powerful workflows. For example we can forward multiple collections to a light linking relationship or forwarding material binding relationships to a single collection on the asset root prim, which then in return forwards to the material prim.

Table of Contents

  1. Collections In-A-Nutshell
  2. What should I use it for?
  3. Resources
  4. Overview
    1. Creating & querying collections
    2. Inverting a collection

TL;DR - Metadata In-A-Nutshell

  • Collections are encoded via a set of properties. A prim can store any number of collections.
    • collection:<collectionName>:includes relationship: A list of target Sdf.Paths to include, we can als target other collections.
    • collection:<collectionName>:excludes relationship: A list of target Sdf.Paths to exclude. These must be below the include paths.
    • collection:<collectionName>:expansionRuleattribute: Controls how collections are computed, either by running includes - excludes (mode explicitOnly) or by expanding all child prims and then doing includes - excludes (mode expandPrims).
  • Collections can link to other collections, which gives us a powerful mechanism of forwarding hierarchy structure information.
  • Collections can easily be accessed and queried via the Usd.CollectionAPI. The query can be limited via USD filter predicates, e.g. to defined prims only.
  • To help speed up collection creation, USD also ships with util functions:
    • Collection creation: UsdUtils.AuthorCollection(<collectionName>, prim, [<includePathList>], [<excludePathList>])
    • Re-writing a collection to be as sparse as possible: include_paths, exclude_paths = UsdUtils.ComputeCollectionIncludesAndExcludes(target_paths, stage)

What should I use it for?

Tip

We use collections for multiple things:

  • Creating a group of target paths, that are of interest to other departments, e.g. mark prims that are useful for FX/layout/lighting selections (for example character vs. background). Another common thing to use them for is storing render layer selections, that then get applied in our final render USD file.
  • Faster navigation of hierarchies by isolating to collections that interest us.
  • As collections can contain other collections, they are a powerful mechanism of forwarding and aggregating selections.

Resources

Overview

Collections are made up of relationships and attributes:

  • collection:<collectionName>:includes relationship: A list of target Sdf.Paths to include, we can als target other collections.
  • collection:<collectionName>:excludes relationship: A list of target Sdf.Paths to exclude. These must be below the include paths. Excluding another collection does not work.
  • collection:<collectionName>:expansionRuleattribute: This controls how collections are expanded:
    • explicitOnly: Do not expand to any child prims, instead just do an explicit diff between include and exclude paths. This is like a Python set().difference().
    • expandPrims: Expand the include paths to all children and subtract the exclude paths.
    • expandPrimsAndProperties: Same as expandPrims, but expand properties too. (Not used by anything at the moment).
  • (Optional) collection:<collectionName>:includeRoot attribute: When using expandPrims/expandPrimsAndProperties this bool attribute enables the includes to target the / pseudo root prim.

Collection Size

Make sure that you write your collections as sparsely as possible, as otherwise they can take a long time to combine when stitching multiple files, when writing per frame USD files.

Creating & querying collections

We interact with them via the Usd.CollectionAPI class API Docs. The collection api is a multi-apply API schema, so we can add multiple collections to any prim.

Here are the UsdUtils.ComputeCollectionIncludesAndExcludes API docs.

# Usd.CollectionAPI.Apply(prim, collection_name)
# collection_api = Usd.CollectionAPI(prim, collection_nam)
# collection_query = collection_api.ComputeMembershipQuery()
### High Level ###
from pxr import Sdf, Usd, UsdUtils
stage = Usd.Stage.CreateInMemory()
bicycle_prim = stage.DefinePrim(Sdf.Path("/set/yard/biycle"), "Cube")
car_prim = stage.DefinePrim(Sdf.Path("/set/garage/car"), "Sphere")
tractor_prim = stage.DefinePrim(Sdf.Path("/set/garage/tractor"), "Cylinder")
helicopter_prim = stage.DefinePrim(Sdf.Path("/set/garage/helicopter"), "Cube")
boat_prim = stage.DefinePrim(Sdf.Path("/set/garage/boat"), "Cube")
set_prim = bicycle_prim.GetParent().GetParent()
set_prim.SetTypeName("Xform")
bicycle_prim.GetParent().SetTypeName("Xform")
car_prim.GetParent().SetTypeName("Xform")
# Create collection
collection_name = "vehicles"
collection_api = Usd.CollectionAPI.Apply(set_prim, collection_name)
collection_api.GetIncludesRel().AddTarget(set_prim.GetPath())
collection_api.GetExcludesRel().AddTarget(bicycle_prim.GetPath())
collection_api.GetExpansionRuleAttr().Set(Usd.Tokens.expandPrims)
print(Usd.CollectionAPI.GetAllCollections(set_prim)) # Returns: [Usd.CollectionAPI(Usd.Prim(</set>), 'vehicles')]
print(Usd.CollectionAPI.GetCollection(set_prim, "vehicles")) # Returns: Usd.CollectionAPI(Usd.Prim(</set>), 'vehicles')
collection_query = collection_api.ComputeMembershipQuery()
print(collection_api.ComputeIncludedPaths(collection_query, stage))
# Returns: [Sdf.Path('/set'), Sdf.Path('/set/garage'), Sdf.Path('/set/garage/car'), Sdf.Path('/set/yard')]
# Set it to explicit only
collection_api.GetExpansionRuleAttr().Set(Usd.Tokens.explicitOnly)
collection_query = collection_api.ComputeMembershipQuery()
print(collection_api.ComputeIncludedPaths(collection_query, stage))
# Returns: [Sdf.Path('/set')]

# To help speed up collection creation, USD also ships with util functions:
# UsdUtils.AuthorCollection(<collectionName>, prim, [<includePathList>], [<excludePathList>])
collection_api = UsdUtils.AuthorCollection("two_wheels", set_prim, [set_prim.GetPath()], [car_prim.GetPath()])
collection_query = collection_api.ComputeMembershipQuery()
print(collection_api.ComputeIncludedPaths(collection_query, stage))
# Returns:
# [Sdf.Path('/set'), Sdf.Path('/set/garage'), Sdf.Path('/set/yard'), Sdf.Path('/set/yard/biycle')]
# UsdUtils.ComputeCollectionIncludesAndExcludes() gives us the possibility to author 
# collections more sparse, that the include to exclude ratio is kept at an optimal size.
# The Python signature differs from the C++ signature:
"""
include_paths, exclude_paths = UsdUtils.ComputeCollectionIncludesAndExcludes(
    target_paths,
    stage,
    minInclusionRatio = 0.75,
    maxNumExcludesBelowInclude = 5,
	minIncludeExcludeCollectionSize = 3,
    pathsToIgnore = [] # This ignores paths from computation (this is not the exclude list)
)		
"""
target_paths = [tractor_prim.GetPath(), car_prim.GetPath(), helicopter_prim.GetPrimPath()]
include_paths, exclude_paths = UsdUtils.ComputeCollectionIncludesAndExcludes(target_paths,stage, minInclusionRatio=.9)
print(include_paths, exclude_paths)
# Returns:
# [Sdf.Path('/set/garage/car'), Sdf.Path('/set/garage/tractor'), Sdf.Path('/set/garage/helicopter')] []
include_paths, exclude_paths = UsdUtils.ComputeCollectionIncludesAndExcludes(target_paths,stage, minInclusionRatio=.1)
print(include_paths, exclude_paths)
# Returns: [Sdf.Path('/set/garage')] [Sdf.Path('/set/garage/boat')]
# Create a collection from the result
collection_api = UsdUtils.AuthorCollection("optimized", set_prim, include_paths, exclude_paths)

Inverting a collection

When we want to isolate a certain part of the hierarchy (for example to pick what to render), a typical thing to do, is to give users a "render" collection which then gets applied by setting all prims not included to be inactive. Here is an example of how to iterate a stage by pruning (skipping the child traversal) and deactivating anything that is not in the specific collection.

This is very fast and "sparse" as we don't edit leaf prims, instead we find the highest parent and deactivate it, if no children are part of the target collection.

from pxr import Sdf, Usd, UsdUtils
stage = Usd.Stage.CreateInMemory()
# Create hierarchy
prim_paths = [
    "/set/yard/biycle",
    "/set/yard/shed/shovel",
    "/set/yard/shed/flower_pot",
    "/set/yard/shed/lawnmower",
    "/set/yard/shed/soil",
    "/set/yard/shed/wood",
    "/set/garage/car",
    "/set/garage/tractor",
    "/set/garage/helicopter",
    "/set/garage/boat",
    "/set/garage/key_box",
    "/set/garage/key_box/red",
    "/set/garage/key_box/blue",
    "/set/garage/key_box/green",
    "/set/people/mike",
    "/set/people/charolotte"
]
for prim_path in prim_paths:
    prim = stage.DefinePrim(prim_path, "Cube")
print("<< hierarchy >>")
for prim in stage.Traverse():
    print(prim.GetPath())
    parent_prim = prim.GetParent()
    while True:
        if parent_prim.IsPseudoRoot():
            break
        parent_prim.SetTypeName("Xform")
        parent_prim = parent_prim.GetParent()
# Returns:
"""
<< hierarchy >>
/HoudiniLayerInfo
/set
/set/yard
/set/yard/biycle
/set/yard/shed
/set/yard/shed/shovel
/set/yard/shed/flower_pot
/set/yard/shed/lawnmower
/set/yard/shed/soil
/set/yard/shed/wood
/set/garage
/set/garage/car
/set/garage/tractor
/set/garage/helicopter
/set/garage/boat
/set/garage/key_box
/set/garage/key_box/red
/set/garage/key_box/blue
/set/garage/key_box/green
/set/people
/set/people/mike
/set/people/charolotte
"""
# Collections
collection_prim = stage.DefinePrim("/collections")
storage_include_prim_paths = ["/set/garage/key_box", "/set/yard/shed"]
storage_exclude_prim_paths = ["/set/yard/shed/flower_pot"]
collection_api = UsdUtils.AuthorCollection("storage", collection_prim, storage_include_prim_paths, storage_exclude_prim_paths)
collection_query = collection_api.ComputeMembershipQuery()
included_paths = collection_api.ComputeIncludedPaths(collection_query, stage)
# print(included_paths)
# Prune inverse:
print("<< hierarchy pruned >>")
iterator = iter(Usd.PrimRange(stage.GetPseudoRoot()))
for prim in iterator:
    if prim.IsPseudoRoot():
        continue
    if prim.GetPath() not in included_paths and not len(prim.GetAllChildrenNames()):
        iterator.PruneChildren()
        prim.SetActive(False)
    else:    
        print(prim.GetPath())
# Returns:
"""
<< hierarchy pruned >>
/set
/set/yard
/set/yard/shed
/set/yard/shed/shovel
/set/yard/shed/lawnmower
/set/yard/shed/soil
/set/yard/shed/wood
/set/garage
/set/garage/key_box
/set/garage/key_box/red
/set/garage/key_box/blue
/set/garage/key_box/green
/set/people
"""