Point Instancers ('Copy To Points')

We have four options for mapping Houdini's packed prims to USD:

As transforms
As point instancers
As deformed geo (baking down the xform to actual geo data)
As skeletons, more info in our RBD section

Pro Tip | When to use PointInstancer prims?

We'll always want to use USD's PointInstancer prims, when representing a "replicate a few unique meshes to many points" scenario. In SOPs we usually do this via the "Copy To Points" node.

You can find all the .hip files of our shown examples in our USD Survival Guide - GitHub Repo.

For all options for SOP to LOP importing, check out the official Houdini docs.

In the below examples, we use the path/name attributes to define the prim path. You can actually configure yourself what attributes Houdini should use for defining our prim paths on the "SOP import node" LOP/"USD Configure" SOP node.

Houdini Native Import (and making it convenient for artists)

To import our geometry to "PointInstancer" USD prims, we have to have it as packed prims in SOPs. If you have nested packed levels, they will be imported as nested point instancers. We do not recommend doing this, as it can cause some confusing outputs. The best practice is to always have a "flat" packed hierarchy, so only one level of packed prims.

Houdini gives us the following options for importing:

The usdinstancerpath attribute defines the location of our PointInstancer prim.
The path/name attribute defines the location of the prototype prims. Prototypes are the unique prims that should get instances, they are similar to the left input on your "copy to points" node.

An important thing to note is, that if your path/name attribute does not have any / slashes or starts with ./, the prototypes will be imported with the following path: <usdinstancerpath>/Prototypes/<pathOrName>. Having the /Prototypes prim is just a USD naming convention thing.

To make it easy to use for artists, we recommend mapping the traditional path attribute value to usdinstancerpath and making sure that the name attribute is relative.

Another important thing to know about packed prims is, that the path/name attributes are also used to define the hierarchy within the packed prim content. So before you pack your geometry, it has to have a valid path value.

Good, now that we know the basics, let's have a look at a not so expectable behavior: If you remember, in our Basic Building Blocks of Usd section, we explained that relationships can't be animated. Here's the fun part:

PointInstancer | Varying Prototypes | Problem

The mapping of what point maps to what prototype prim is stored via the protoIndices attribute. This maps an index to the prim paths targetd by the prototypes relationship. Since relationships can't be animated, the protoIndices/prototypes properties has to be aware of all prototypes, that ever get instanced across the whole cache.

This is the reason, why in our LOPs instancer node, we have to predefine all prototypes. The problem is in SOPs, it kind of goes against the usual artist workflow. For example when we have debris instances, we don't want to have the artist managing to always have at least one copy of all unique prims we are trying to instance.

PointInstancer | Varying Prototypes | Solution

The artist should only have to ensure a unique name attribute value per unique instance and a valid usdinstancerpath value. Making sure the protoIndices don't jump, as prototypes come in and out of existence, is something we can fix on the pipeline side.

Luckily, we can fix this behavior, by tracking the prototypes ourselves per frame and then re-ordering them as a post process of writing our caches.

Let's take a look at the full implementation:

As you can see, all looks good, when we only look at the active frame, because the active frame does not know about the other frames. As soon as we cache it to disk though, it "breaks", because the protoIndices map to the wrong prototype.

All we have to do is create an attribute, that per frame stores the relation ship targets as a string list. After the cache is done, we have to map the wrong prototype index to the write one.

Here is the tracker script:

PointInstancer | Re-Order Prototypes | Track Prototypes | Click to expand

import pxr
node = hou.pwd()
layer = node.editableLayer()

ref_node = node.parm("spare_input0").evalAsNode()
ref_stage = ref_node.stage()

with pxr.Sdf.ChangeBlock():
    for prim in ref_stage.TraverseAll():
        prim_path = prim.GetPath()
        if prim.IsA(pxr.UsdGeom.PointInstancer):
            prim_spec = layer.GetPrimAtPath(prim_path)
            # Attrs
            prototypes = prim.GetRelationship(pxr.UsdGeom.Tokens.prototypes)
            protoIndices_attr = prim.GetAttribute(pxr.UsdGeom.Tokens.protoIndices)
            if not protoIndices_attr:
                continue
            if not protoIndices_attr.HasValue():
                continue
            protoTracker_attr_spec = pxr.Sdf.AttributeSpec(prim_spec, "protoTracker", pxr.Sdf.ValueTypeNames.StringArray)
            layer.SetTimeSample(protoTracker_attr_spec.path, hou.frame(), [p.pathString for p in prototypes.GetForwardedTargets()])

And here the post processing script. You'll usually want to trigger this after the whole cache is done writing. It also works with value clips, you pass in all the individual clip files into the layers list. This is also another really cool demo, of how numpy can be used to get C++ like performance.

PointInstancer | Re-Order Prototypes | Track Prototypes | Click to expand

import pxr
import numpy as np
node = hou.pwd()
layer = node.editableLayer()


layers = [layer]

def pointinstancer_prototypes_reorder(layers):
    """Rearrange the prototypes to be the actual value that they were written with
    based on the 'protoTracker' attribute.
    We need to do this because relationship attributes can't be animated,
    which causes instancers with varying prototypes per frame to output wrong
    prototypes once the scene is cached over the whole frame range.
    
    This assumes that the 'protoTracker' attribute has been written with the same
    time sample count as the 'protoIndices' attribute. They will be matched by time
    sample index.

    Args:
        layers (list): A list of pxr.Sdf.Layer objects that should be validated.
                       It is up to the caller to call the layer.Save() command to
                       commit the actual results of this function.
                       
    """
    
    # Constants
    protoTracker_attr_name = "protoTracker"
    
    # Collect all point instancer prototypes
    instancer_prototype_mapping = {}

    def collect_data_layer_traverse(path):
        if not path.IsPrimPropertyPath():
            return
        if path.name != protoTracker_attr_name:
            return
        instancer_prim_path = path.GetPrimPath()
        instancer_prototype_mapping.setdefault(instancer_prim_path, set())
        time_samples = layer.ListTimeSamplesForPath(path)
        if time_samples:
            for timeSample in layer.ListTimeSamplesForPath(path):
                prototype_prim_paths = layer.QueryTimeSample(path, timeSample)
                instancer_prototype_mapping[instancer_prim_path].update(prototype_prim_paths)
    
    for layer in layers:
        layer.Traverse(layer.pseudoRoot.path, collect_data_layer_traverse)
    # Exit if not valid instancers were found
    if not instancer_prototype_mapping:
        return
    # Combine prototype mapping data
    for k, v in instancer_prototype_mapping.items():
        instancer_prototype_mapping[k] = sorted(v)
    # Apply combined targets
    for layer in layers:
        for instancer_prim_path, prototypes_prim_path_strs in instancer_prototype_mapping.items():
            instancer_prim_spec = layer.GetPrimAtPath(instancer_prim_path)
            if not instancer_prim_spec:
                continue
            protoTracker_attr_spec = layer.GetPropertyAtPath(
                instancer_prim_path.AppendProperty(protoTracker_attr_name)
            )
            if not protoTracker_attr_spec:
                continue
            protoIndices_attr_spec = layer.GetPropertyAtPath(
                instancer_prim_path.AppendProperty(pxr.UsdGeom.Tokens.protoIndices)
            )
            if not protoIndices_attr_spec:
                continue
            prototypes_rel_spec = layer.GetRelationshipAtPath(
                instancer_prim_path.AppendProperty(pxr.UsdGeom.Tokens.prototypes)
            )
            if not prototypes_rel_spec:
                continue
            # Update prototypes
            prototypes_prim_paths = [pxr.Sdf.Path(p) for p in prototypes_prim_path_strs]
            prototypes_rel_spec.targetPathList.ClearEdits()
            prototypes_rel_spec.targetPathList.explicitItems = prototypes_prim_paths
            
            # Here we just match the time sample by index, not by actual values
            # as some times there are floating precision errors when time sample keys are written.
            protoIndices_attr_spec_time_samples = layer.ListTimeSamplesForPath(
                protoIndices_attr_spec.path
            )
            # Update protoIndices
            for protoTracker_time_sample_idx, protoTracker_time_sample in enumerate(
                layer.ListTimeSamplesForPath(protoTracker_attr_spec.path)
            ):
                # Reorder protoIndices
                protoTracker_prim_paths = list(
                    layer.QueryTimeSample(
                        protoTracker_attr_spec.path,
                        protoTracker_time_sample,
                    )
                )
                # Skip if order already matches
                if prototypes_prim_paths == protoTracker_prim_paths:
                    continue

                prototype_order_mapping = {}
                for protoTracker_idx, protoTracker_prim_path in enumerate(protoTracker_prim_paths):
                    prototype_order_mapping[protoTracker_idx] = prototypes_prim_paths.index(
                        protoTracker_prim_path
                    )
                # Re-order protoIndices via numpy (Remapping via native Python is slow).
                source_value = np.array(
                    layer.QueryTimeSample(
                        protoIndices_attr_spec.path,
                        protoIndices_attr_spec_time_samples[protoTracker_time_sample_idx],
                    ),
                    dtype=int,
                )
                destination_value = np.copy(source_value)
                for k in sorted(prototype_order_mapping.keys(), reverse=True):
                    destination_value[source_value == k] = prototype_order_mapping[k]
                layer.SetTimeSample(
                    protoIndices_attr_spec.path,
                    protoIndices_attr_spec_time_samples[protoTracker_time_sample_idx],
                    destination_value,
                )
                # Force deallocate
                del source_value
                del destination_value
                # Update protoTracker attribute to reflect changes, allowing
                # this function to be run multiple times.
                layer.SetTimeSample(
                    protoTracker_attr_spec.path,
                    protoTracker_time_sample,
                    pxr.Vt.StringArray(prototypes_prim_path_strs),
                )

pointinstancer_prototypes_reorder(layers)

Phew, now everything looks alright again!

Performance Optimizations

You may have noticed, that we always have to create packed prims on SOP level, to import them as PointInstancer prims. If we really want to go all out on the most high performance import, we can actually replicate a "Copy To Points" import. That way we only have to pass in the prototypes and the points, but don't have the overhead of spawning the packed prims in SOPs.

Is this something you need to be doing? No, Houdini's LOPs import as well as the packed prim generation are highly performant, the following solution is really only necessary, if you are really picky about making your export a few hundred milliseconds faster with very large instance counts.

As you can see we are at a factor 20 (1 seconds : 50 milliseconds). Wow! Now what we don't show is, that we actually have to conform the point instances attributes to what the PointInstancer prim schema expects. So the ratio we just mentioned is the best case scenario, but it can be a bit slower, when we have to map for example N/up to orientations. This is also only this performant because we are importing a single PointInstancer prim, which means we don't have to segment any of the protoIndices.

We also loose the benefit of being able to work with our packed prims in SOP level, for example for collision detection etc.

Let's look at the details:

On SOP level:

We create a "protoIndices" attribute based on all unique values of the "name" attribute
We create a "protoHash" attribute in the case we have multiple PointInstancer prim paths, so that we can match the prototypes per instancer
We conform all instancing related attributes to have the correct precision. This is very important, as USD does not allow other precisions types than what is defined in the PointInstancer schema.
We conform the different instancing attributes Houdini has to the attributes the PointInstancer schema expects. (Actually the setup in the video doesn't do this, but you have to/should, in case you want to use this in production)

On LOP level:

We import the points as a "Points" prim, so now we have to convert it to a "PointInstancer" prim. For the prim itself, this just means changing the prim type to "PointInstancer" and renaming "points" to "positions".
We create the "prototypes" relationship property.

PointInstancer | Custom Import | Click to expand

import pxr
node = hou.pwd()
layer = node.editableLayer()

ref_node = node.parm("spare_input0").evalAsNode()
ref_stage = ref_node.stage()

time_code = pxr.Usd.TimeCode.Default() if not ref_node.isTimeDependent() else pxr.Usd.TimeCode(hou.frame())

with pxr.Sdf.ChangeBlock():
    edit = pxr.Sdf.BatchNamespaceEdit()
    for prim in ref_stage.TraverseAll():
        prim_path = prim.GetPath()
        if prim.IsA(pxr.UsdGeom.Points):
            prim_spec = layer.GetPrimAtPath(prim_path)
            # Prim
            prim_spec.typeName = "PointInstancer"
            purpose_attr_spec = pxr.Sdf.AttributeSpec(prim_spec, pxr.UsdGeom.Tokens.purpose, pxr.Sdf.ValueTypeNames.Token)
            # Rels
            protoTracker_attr = prim.GetAttribute("protoTracker")
            protoHash_attr = prim.GetAttribute("protoHash")
            if protoTracker_attr and protoTracker_attr.HasValue():
                protoTracker_prim_paths = [pxr.Sdf.Path(p) for p in protoTracker_attr.Get(time_code)]
                prototypes_rel_spec = pxr.Sdf.RelationshipSpec(prim_spec, pxr.UsdGeom.Tokens.prototypes)
                prototypes_rel_spec.targetPathList.explicitItems = protoTracker_prim_paths
                # Cleanup
                edit.Add(prim_path.AppendProperty("protoTracker:indices"), pxr.Sdf.Path.emptyPath)
                edit.Add(prim_path.AppendProperty("protoTracker:lengths"), pxr.Sdf.Path.emptyPath)
            elif protoHash_attr and protoHash_attr.HasValue():
                protoHash_pairs = [i.split("|") for i in protoHash_attr.Get(time_code)]
                protoTracker_prim_paths = [pxr.Sdf.Path(v) for k, v in protoHash_pairs if k == prim_path]
                prototypes_rel_spec = pxr.Sdf.RelationshipSpec(prim_spec, pxr.UsdGeom.Tokens.prototypes)
                prototypes_rel_spec.targetPathList.explicitItems = protoTracker_prim_paths
                protoTracker_attr_spec = pxr.Sdf.AttributeSpec(prim_spec, "protoTracker", pxr.Sdf.ValueTypeNames.StringArray)
                layer.SetTimeSample(protoTracker_attr_spec.path, hou.frame(), [p.pathString for p in protoTracker_prim_paths])
                # Cleanup
                edit.Add(prim_path.AppendProperty("protoHash"), pxr.Sdf.Path.emptyPath)
                edit.Add(prim_path.AppendProperty("protoHash:indices"), pxr.Sdf.Path.emptyPath)
                edit.Add(prim_path.AppendProperty("protoHash:lengths"), pxr.Sdf.Path.emptyPath)
            # Children
            Prototypes_prim_spec = pxr.Sdf.CreatePrimInLayer(layer, prim_path.AppendChild("Prototypes"))
            Prototypes_prim_spec.typeName = "Scope"
            Prototypes_prim_spec.specifier = pxr.Sdf.SpecifierDef
            
            # Rename
            edit.Add(prim_path.AppendProperty(pxr.UsdGeom.Tokens.points),
                     prim_path.AppendProperty(pxr.UsdGeom.Tokens.positions))

    if not layer.Apply(edit):
        raise Exception("Failed to modify layer!")

Usd Survival Guide

Point Instancers ('Copy To Points')

Houdini Native Import (and making it convenient for artists)

Performance Optimizations