Siggraph Presentation

This guide will be officially introduced at Siggraph 2023 - Houdini Hive on Wednesday, 9. of August 2023 at 11:00 AM PST.

Advanced Concepts

Table of Contents

  1. Edit Targets
  2. Utility functions in the Usd.Utils module
  3. Utility functions in the Sdf module
    1. Moving/Renaming/Removing prim/property/variant specs with Sdf.BatchNamespaceEdit()
    2. Copying data with Sdf.CopySpec
    3. Delaying change notifications with the Sdf.ChangeBlock
  4. Relationships
    1. Special Relationships
    2. Relationship Forwarding (Binding post)
    3. Collections

Edit Targets

Pro Tip | Edit Targets

A edit target defines, what layer all calls in the high level API should write to.

An edit target's job is to map from one namespace to another, we mainly use them for writing to layers in the active layer stack (though we could target any layer) and to write variants, as these are written "inline" and therefore need an extra name space injection.

We cover edit targets in detail in our composition fundamentals section.

Utility functions in the Usd.Utils module

Usd provides a bunch of utility functions in the UsdUtils module (USD Docs):

For retrieving/upating dependencies:

  • UsdUtils.ExtractExternalReferences: This is similar to layer.GetCompositionAssetDependencies(), except that it returns three lists: [<sublayers>], [<references>], [<payloads>]. It also consults the assetInfo metadata, so result might be more "inclusive" than layer.GetCompositionAssetDependencies().
  • UsdUtils.ComputeAllDependencies: This recursively calls layer.GetCompositionAssetDependencies() and gives us the aggregated result.
  • UsdUtils.ModifyAssetPaths: This is similar to Houdini's output processors. We provide a function that gets the input path and returns a (modified) output path.

For animation and value clips stitching:

  • Various tools for stitching/creating value clips. We cover these in our animation section. These are also what the commandline tools that ship with USD use.

For collection authoring/compression:

Utility functions in the Sdf module

Moving/Renaming/Removing prim/property/variant specs with Sdf.BatchNamespaceEdit()

We've actually used this quite a bit in the guide so far, so in this section we'll summarize its most important uses again:

Using Sdf.BatchNamespaceEdit() moving/renaming/removing prim/property (specs)

We main usage is to move/rename/delete prims. We can only run the name space edit on a layer, it does not work with stages. Thats means if we have nested composition, we can't rename prims any more. In production this means we'll only be using this with the "active" layer, that we are currently creating/editing. All the edits added are run in the order they are added, we have to be careful what order we add removes/renames if they interfere with each other.

Sdf.BatchNamespaceEdit | Moving/renaming/removing prim/property specs | Click to expand!

 
### High Level / Low Level ###
# The Sdf.BatchNamespaceEdit() always runs only on an individual layer.
from pxr import Gf, Sdf, Usd
stage = Usd.Stage.CreateInMemory()
layer = stage.GetEditTarget().GetLayer()
bicycle_prim_path = Sdf.Path("/bicycle")
bicycle_prim = stage.DefinePrim(bicycle_prim_path, "Xform")
bicycle_color_attr = bicycle_prim.CreateAttribute("color", Sdf.ValueTypeNames.Color3h)
bicycle_color_attr.Set(Gf.Vec3h([0,1,2]))
car_prim_path = Sdf.Path("/car")
car_prim = stage.DefinePrim(car_prim_path, "Xform")
soccer_ball_prim_path = Sdf.Path("/soccer_ball")
soccer_ball_prim = stage.DefinePrim(soccer_ball_prim_path, "Xform")
soccer_ball_player_rel = soccer_ball_prim.CreateRelationship("player")
soccer_ball_player_rel.SetTargets([Sdf.Path("/players/mike")])

print(layer.ExportToString())
"""Returns:
#usda 1.0

def Xform "bicycle"
{
    custom color3h color = (0, 1, 2)
}

def Xform "car"
{
}

def Xform "soccer_ball"
{
    custom rel player = </players/mike>
}
"""
with Sdf.ChangeBlock():
    edit = Sdf.BatchNamespaceEdit()
    ## Important: Edits are run in the order they are added.
    # If we try to move and then remove, it will error.
    ## Prim Specs
    # Remove
    edit.Add(car_prim_path, Sdf.Path.emptyPath)
    # Move
    edit.Add(bicycle_prim_path, car_prim_path)
    # Rename
    basket_ball_prim_path = soccer_ball_prim_path.ReplaceName("basket_ball")
    edit.Add(soccer_ball_prim_path, basket_ball_prim_path)
    ## Property Specs
    edit.Add(car_prim_path.AppendProperty("color"), car_prim_path.AppendProperty("style"))
    soccer_ball_player_rel_path = basket_ball_prim_path.AppendProperty("player")
    edit.Add(soccer_ball_player_rel_path, soccer_ball_player_rel_path.ReplaceName("people"))
    
    # We can als 
    if not layer.Apply(edit):
        raise Exception("Failed to apply layer edit!")
        
print(layer.ExportToString())
"""Returns:
#usda 1.0
def Xform "car"
{
    custom color3h style = (0, 1, 2)
}

def Xform "basket_ball"
{
    custom rel people = </players/mike>
}
"""

Using Sdf.BatchNamespaceEdit() for variant creation

We can create variant via the namespace edit, because variants are in-line USD namespaced paths.

Sdf.BatchNamespaceEdit | Moving prim specs into variants | Click to expand!

 
### High Level / Low Level ###
# The Sdf.BatchNamespaceEdit() always runs only on an individual layer.
from pxr import Sdf, Usd
stage = Usd.Stage.CreateInMemory()
prim_path = Sdf.Path("/bicycle")
prim = stage.DefinePrim(prim_path, "Xform")
stage.DefinePrim(Sdf.Path("/bicycle/cube"), "Cube")

layer = stage.GetEditTarget().GetLayer()
with Sdf.ChangeBlock():
    edit = Sdf.BatchNamespaceEdit()
    prim_spec = layer.GetPrimAtPath(prim_path)
    # Move content into variant
    variant_set_spec = Sdf.VariantSetSpec(prim_spec, "model")
    variant_spec = Sdf.VariantSpec(variant_set_spec, "myCoolVariant")
    variant_prim_path = prim_path.AppendVariantSelection("model", "myCoolVariant")
    edit.Add(prim_path.AppendChild("cube"), variant_prim_path.AppendChild("cube"))
    # Variant selection
    prim_spec.SetInfo("variantSetNames", Sdf.StringListOp.Create(prependedItems=["model"]))
    prim_spec.variantSelections["model"] = "myCoolVariant"
        
    if not layer.Apply(edit):
        raise Exception("Failed to apply layer edit!")

We also cover variants in detail in respect to Houdini in our Houdini - Tips & Tricks section.

Copying data with Sdf.CopySpec

We use the Sdf.CopySpec method to copy/duplicate content from layer to layer (or within the same layer).

Copying specs (prim and properties) from layer to layer with Sdf.CopySpec()

The Sdf.CopySpec can copy anything that is representable via the Sdf.Path. This means we can copy prim/property/variant specs. When copying, the default is to completely replace the target spec.

We can filter this by passing in filter functions. Another option is to copy the content to a new anonymous layer and then merge it via UsdUtils.StitchLayers(<StrongLayer>, <WeakerLayer>). This is often more "user friendly" than implementing a custom merge logic, as we get the "high layer wins" logic for free and this is what we are used to when working with USD.

Still under construction!

We'll add some examples for custom filtering at a later time.

Sdf.CopySpec | Copying prim/property specs | Click to expand!

 
### High Level / Low Level ###
# The Sdf.BatchNamespaceEdit() always runs only on an individual layer.
from pxr import Gf, Sdf, Usd
stage = Usd.Stage.CreateInMemory()
layer = stage.GetEditTarget().GetLayer()
bicycle_prim_path = Sdf.Path("/bicycle")
bicycle_prim = stage.DefinePrim(bicycle_prim_path, "Xform")
bicycle_color_attr = bicycle_prim.CreateAttribute("color", Sdf.ValueTypeNames.Color3h)
bicycle_color_attr.Set(Gf.Vec3h([0,1,2]))
car_prim_path = Sdf.Path("/car")
car_prim = stage.DefinePrim(car_prim_path, "Xform")
soccer_ball_prim_path = Sdf.Path("/soccer_ball")
soccer_ball_prim = stage.DefinePrim(soccer_ball_prim_path, "Xform")
soccer_ball_player_rel = soccer_ball_prim.CreateRelationship("player")
soccer_ball_player_rel.SetTargets([Sdf.Path("/players/mike")])

print(layer.ExportToString())
"""Returns:
#usda 1.0
def Xform "bicycle"
{
    custom color3h color = (0, 1, 2)
}

def Xform "car"
{
}

def Xform "soccer_ball"
{
    custom rel player = </players/mike>
}
"""
# When copying data, the target prim spec will be replaced by the source prim spec.
# The data will not be averaged
with Sdf.ChangeBlock():
    # Copy Prim Spec
    Sdf.CopySpec(layer, soccer_ball_prim_path, layer, car_prim_path.AppendChild("soccer_ball"))
    # Copy Property
    Sdf.CopySpec(layer, bicycle_color_attr.GetPath(), layer, car_prim_path.AppendChild("soccer_ball").AppendProperty("color"))
print(layer.ExportToString())
"""Returns:
#usda 1.0
def Xform "bicycle"
{
    custom color3h color = (0, 1, 2)
}

def Xform "car"
{
    def Xform "soccer_ball"
    {
        custom color3h color = (0, 1, 2)
        custom rel player = </players/mike>
    }
}

def Xform "soccer_ball"
{
    custom rel player = </players/mike>
}
"""

Using Sdf.CopySpec() for variant creation

We can also use Sdf.CopySpec for copying content into a variant.

Sdf.CopySpec | Copying prim specs into variants | Click to expand!

 
### High Level / Low Level ###
# The Sdf.CopySpec() always runs on individual layers.
from pxr import Sdf, Usd
stage = Usd.Stage.CreateInMemory()
bicycle_prim_path = Sdf.Path("/bicycle")
bicycle_prim = stage.DefinePrim(bicycle_prim_path, "Xform")
cube_prim_path = Sdf.Path("/cube")
cube_prim = stage.DefinePrim(cube_prim_path, "Cube")

layer = stage.GetEditTarget().GetLayer()
with Sdf.ChangeBlock():
    edit = Sdf.BatchNamespaceEdit()
    prim_spec = layer.GetPrimAtPath(bicycle_prim_path)
    # Copy content into variant
    variant_set_spec = Sdf.VariantSetSpec(prim_spec, "model")
    variant_spec = Sdf.VariantSpec(variant_set_spec, "myCoolVariant")
    variant_prim_path = bicycle_prim_path.AppendVariantSelection("model", "myCoolVariant")
    Sdf.CopySpec(layer, cube_prim_path, layer, variant_prim_path.AppendChild("cube"))
    # Variant selection
    prim_spec.SetInfo("variantSetNames", Sdf.StringListOp.Create(prependedItems=["model"]))
    prim_spec.variantSelections["model"] = "myCoolVariant"
print(layer.ExportToString())
"""Returns:
#usda 1.0

def Xform "bicycle" (
    variants = {
        string model = "myCoolVariant"
    }
    prepend variantSets = "model"
)
{
    variantSet "model" = {
        "myCoolVariant" {
            def Cube "cube"
            {
            }

        }
    }
}

def Cube "cube"
{
}
"""

We also cover variants in detail in respect to Houdini in our Houdini - Tips & Tricks section.

Delaying change notifications with the Sdf.ChangeBlock

Whenever we edit something in our layers, change notifications get sent to all consumers (stages/hydra delegates) that use the layer. This causes them to recompute and trigger updates.

When performing a large edit, for example creating large hierarchies, we can batch the edit, so that the change notification gets the combined result.

Pro Tip | When/How to use Sdf.ChangeBlocks

In theory it is only safe to use the change block with the lower level Sdf API. We can also use it with the high level API, we just have to make sure that we don't accidentally query an attribute, that we just overwrote or perform ops on deleted properties.

We therefore recommend work with a read/write code pattern:

  • We first query all the data via the Usd high level API
  • We then write our data via the Sdf low level API

When writing data, we can also write it to a temporary anonymous layer, that is not linked to a stage and then merge the result back in via UsdUtils.StitchLayers(anon_layer, active_layer). This is a great solution when it is to heavy to query all data upfront.

For more info see the Sdf.ChangeBlock API docs.

from pxr import Sdf, Tf, Usd
def callback(notice, sender):
    print("Changed Paths", notice.GetResyncedPaths())
stage = Usd.Stage.CreateInMemory()
# Add
listener = Tf.Notice.Register(Usd.Notice.ObjectsChanged, callback, stage)
# Edit
layer = stage.GetEditTarget().GetLayer()
for idx in range(5):
    Sdf.CreatePrimInLayer(layer, Sdf.Path(f"/test_{idx}"))
# Remove
listener.Revoke()
# Returns:
"""
Changed Paths [Sdf.Path('/test_0')]
Changed Paths [Sdf.Path('/test_1')]
Changed Paths [Sdf.Path('/test_2')]
Changed Paths [Sdf.Path('/test_3')]
Changed Paths [Sdf.Path('/test_4')]
"""
stage = Usd.Stage.CreateInMemory()
# Add
listener = Tf.Notice.Register(Usd.Notice.ObjectsChanged, callback, stage)
with Sdf.ChangeBlock():
    # Edit
    layer = stage.GetEditTarget().GetLayer()
    for idx in range(5):
        Sdf.CreatePrimInLayer(layer, Sdf.Path(f"/test_{idx}"))
# Remove
listener.Revoke()
# Returns:
# Changed Paths [Sdf.Path('/test_0'), Sdf.Path('/test_1'), Sdf.Path('/test_2'), Sdf.Path('/test_3'), Sdf.Path('/test_4')]

Relationships

Special Relationships

Still under construction!

This sub-section is still under development, it is subject to change and needs extra validation.

USD has a few "special" relationships that infer information on our hierarchy. These are: - `proxyPrim`: This is a relationship from a prim with a render purpose to a prim with a proxy purpose. It can be used by clients to get low-resolution proxy representations (for example for simulations/collision detection etc.). Setting it is optional.

These special relationships have primvar like inheritance from parent to child prims:

  • material:binding: This controls the material binding.
  • coordSys:<customName>: Next to collections, this is currently the only other multi-apply API schema that ships natively with USD. It allows us to target an xform prim, whose transform we can then read into our shaders at render time.
  • skel:skeleton/skel:animationSource:
    • skel:skeleton: This defines what skeleton prims with the skelton binding api schema should bind to.
    • skel:animationSource: This relationship can be defined on mesh prims to target the correct animation, but also on the skeletons themselves to select the skeleton animation prim.

Relationship Forwarding (Binding post)

Still under construction!

This sub-section is still under development, it is subject to change and needs extra validation.

Collections

We cover collections in detail in our collection section with advanced topics like inverting or compressing collections.