actor_assets – Rigid Dynamic-Object (Actor) Assets¶

Rigid dynamic-object (“actor”) 3DGS assets — object-local Gaussian bank.

splatsim.sequence_tracks/v2 already says where every dynamic object is at every moment (track_id / class_name / box size / per-frame pose). What it cannot say is what the object looks like: a scene bundle’s Gaussians are one static world-frame cloud, so a moving car is either smeared into the background or cut out of it and gone.

This module supplies the missing half — an asset bank: Gaussian clouds authored in a canonical object-local frame, plus the bindings that say which track is rendered with which asset. A consumer renders frame t by taking the track’s pose (R, p) at t and instancing the asset’s Gaussians through it: means rotated and translated, orientation quaternions pre-multiplied by R, scales / opacity / colour untouched.

Scope of v1: rigid objects only — vehicles, trailers, traffic cones, anything whose shape does not change over time. One cloud, one pose per frame. Articulated (pedestrians, cyclists) and deformable objects are deliberately out of scope; the per-asset motion tag exists so they can arrive later without a schema break, and a v1 reader rejects anything but "rigid" loudly rather than rendering a walking pedestrian as a frozen statue.

Gaussian parameters are identical to the static background: the payload is an NGSP v4 SPZ stream with the same fields (positions, rotations, scales, colours, alphas, SH) and the same optional per-Gaussian LiDAR extension record (SPZ_EXT_TYPE_TIER4_GAUSSIAN_LIDAR), so an actor’s Gaussians travel through exactly the loader, LOD and rasteriser paths a background chunk does. Nothing about an actor is a special kind of Gaussian.

Object-local frame¶

The most common way to get dynamic objects wrong is to leave the object frame implicit and discover downstream that every asset faces the wrong way (NVIDIA’s NuRec asset-insertion workflow, for instance, needs a hand-applied 90° rotation to reconcile Asset-Harvester output with its renderer). The convention is therefore written into every document as object_frame and compared exactly on read:

right-handed, +X forward (the direction the object drives), +Y left, +Z up — the same FLU triad the bundle already uses for sensor rigs;
origin at the centre of the object’s oriented bounding box, in all three axes — the exact point Track.frames[].translation refers to, so instancing is a pure rigid transform with no hidden offset;
metres, right-handed rotations, xyzw quaternions — as everywhere else in a scene bundle.

An asset authored around a ground-level origin (common for CARLA / Autoware vehicle models) must be shifted by +size_z / 2 along +Z before it is packed. An asset in the glTF/SPZ RUB triad must be rotated by RUB_TO_ENU.

Spherical harmonics live in the object frame too: a renderer evaluates them with the view direction rotated into the object frame, so the coefficients never need re-rotating per frame.

Metric scale, not unit scale¶

Assets are stored at true metric scale. NuRec’s external assets are unit-normalised and stretched to each track’s cuboid at render time; that suits a generative asset library but it distorts geometry non-uniformly, which is exactly what a LiDAR simulator must not do — a range return off a car stretched into a truck’s box is wrong by construction. The default binding therefore instances the asset rigidly and ignores the track’s box dimensions. Consumers that want the NuRec behaviour opt in per instance:

fit_mode

"rigid" (default) — use the asset as authored; the track’s size is metadata only. Metrically faithful.
"uniform" — one isotropic scale factor fitting the asset’s declared size into the track’s size. Shape preserved, scale approximate.
"stretch" — per-axis scale (NuRec-equivalent). Interoperable, but not metrically faithful; never use it for LiDAR ground truth.

Assets and tracks are decoupled¶

asset_id is not track_id. One asset can back many tracks (fifty instances of the same sedan), and a track is re-skinned by editing one binding — the same decoupling NuRec’s DynamicObjectTrack.asset_id uses. Bindings live in this document rather than inside sequence_tracks.json so the track schema stays untouched and a bundle’s tracks remain readable by consumers that know nothing about actor assets.

On-disk layout¶

Inside a scene USDZ — and, identically, in a standalone asset-bank directory:

actor_assets.json                        # this schema
actor_assets/<asset_id>/asset.spz        # NGSP v4 SPZ, object-local frame

actor_assets.json is recorded in scene.json.extras.actor_assets.

Document schema — splatsim.actor_assets/v1:

{
  "schema": "splatsim.actor_assets/v1",
  "frame": "object",
  "object_frame": {...},                 # the convention above, verbatim
  "assets": [
    {
      "asset_id":   "sedan_0007",
      "uri":        "actor_assets/sedan_0007/asset.spz",
      "class_name": "automobile",
      "motion":     "rigid",
      "size":       [4.62, 1.90, 1.48],  # box (dx, dy, dz), metres
      "bbox_min":   [-2.41, -0.98, -0.79],
      "bbox_max":   [ 2.40,  0.97,  0.81],
      "n_points":   48213,
      "sh_degree":  3,
      "ext_attributes": {                # present iff the SPZ carries one
        "extension": "EXT_gaussian_lidar",
        "container": "spz_extension",
        "spz_extension_type": "0x54340001",
        "attributes": ["lidar_intensity_raw", "lidar_raydrop_logit"]
      },
      "provenance": {},
      "metadata":   {}
    }
  ],
  "instances": [
    {"track_id": "100", "asset_id": "sedan_0007", "fit_mode": "rigid"}
  ]
}

size is the declared box the track pose refers to; bbox_min / bbox_max is the measured AABB of the Gaussian means, which legitimately overhangs the box (splats have radius, reconstructions have outliers). Both are kept: placement and fitting use size, culling uses the AABB. The AABB must contain the origin — an asset whose means sit at (113, -58, 1.9) is still in world coordinates, and that check catches it at write time rather than at render time.

3dgs_io.actor_assets.ACTOR_ASSETS_ARCHIVE_PATH = 'actor_assets.json'¶: Archive-root path of the bank document inside a scene USDZ.

3dgs_io.actor_assets.ACTOR_ASSETS_PREFIX = 'actor_assets/'¶: Archive prefix owned by the asset payloads.

3dgs_io.actor_assets.FIT_MODES: tuple[str, ...] = ('rigid', 'uniform', 'stretch')¶: How an instance reconciles the asset’s own size with its track’s box size.

3dgs_io.actor_assets.MOTION_MODELS: tuple[str, ...] = ('rigid',)¶: Motion models a document may declare. v1 implements rigid only.

3dgs_io.actor_assets.OBJECT_FRAME_CONVENTION: dict[str, Any] = {'forward': '+x', 'handedness': 'right', 'left': '+y', 'length_units': 'meters', 'origin': 'bbox_center', 'quaternion_order': 'xyzw', 'scale': 'metric', 'up': '+z'}¶: The canonical object-local frame. Written into every document and compared exactly on read, exactly as FRAME_CONVENTION is for world-frame data.

3dgs_io.actor_assets.SH_POLICIES: tuple[str, ...] = ('rotate', 'drop', 'keep')¶: How extract_actor_asset() treats view-dependent SH bands.

class 3dgs_io.actor_assets.ActorAsset(asset_id, class_name, size, bbox_min, bbox_max, n_points, sh_degree=0, motion='rigid', uri=None, ext_attributes=None, provenance=<factory>, metadata=<factory>)¶

Bases: object

One rigid object’s Gaussian cloud, described in the object-local frame.

The payload itself lives at uri inside the archive; this record is the index entry a consumer reads before deciding whether to load it.

uri: str | None = None¶: Archive path of the SPZ payload. Always actor_assets/<asset_id>/asset.spz.

class 3dgs_io.actor_assets.ActorAssetBank(assets=<factory>, instances=<factory>)¶

Bases: object

The asset index plus the track bindings — one actor_assets.json.

asset_for_track(track_id)¶

The asset bound to track_id, or None when it is unbound.

Return type:: ActorAsset | None

class 3dgs_io.actor_assets.ActorAssetSource(asset_id, cloud, class_name, size, ext_attrs=<factory>, motion='rigid', provenance=<factory>, metadata=<factory>)¶

Bases: object

An in-memory asset on its way into a bundle.

cloud holds object-local values numerically — it is written with no implicit axis conversion, exactly like the world-frame background chunks. ext_attrs carries the same optional per-Gaussian LiDAR arrays the background uses, threaded so attr[i] ↔ gaussian[i] keeps holding.

class 3dgs_io.actor_assets.ActorInstance(track_id, asset_id, fit_mode='rigid', metadata=<factory>)¶

Bases: object

Binds one track to the asset it is rendered with.

3dgs_io.actor_assets.build_actor_asset_bank(sources, instances=None)¶

Encode sources into a bank document plus {asset_id: spz_bytes}.

Return type:: tuple[ActorAssetBank, dict[str, bytes]]

3dgs_io.actor_assets.decode_actor_asset(asset, payload)¶

Decode one asset payload into its cloud and per-Gaussian ext attributes.

Return type:: tuple[GaussianCloud, dict[str, ndarray]]

3dgs_io.actor_assets.encode_actor_asset(source)¶

Encode one source into its index record and its SPZ payload bytes.

The cloud is written with no implicit axis conversion (save_spz_world semantics), so the numbers on disk are the object-local numbers in memory.

Return type:: tuple[ActorAsset, bytes]

3dgs_io.actor_assets.extract_actor_asset(cloud, *, asset_id, class_name, size, translation, rotation, ext_attrs=None, margin=0.0, sh_policy='rotate', provenance=None)¶

Cut one object out of a world-frame cloud into an object-local asset.

This is the canonical way to turn a track’s box — its size plus one of its TrackFrame poses, in the bundle’s ENU world frame — into a rigid asset: Gaussians inside the box are selected, then transformed by the inverse pose so the box centre lands on the object-local origin and +X points the way the object drives.

margin grows the selection box on every side (metres). Tracked boxes are rarely tight, and a splat whose centre sits just inside the box can still have its tail outside it.

sh_policy decides what happens to view-dependent bands, whose coefficients are expressed in the source (world) frame: :rtype: ActorAssetSource

"rotate" (default) — re-express them in the object frame. Exact for the yaw-only poses road vehicles have; a pose with more than ~2° of pitch/roll raises, because that case needs full SH rotation.
"drop" — emit a view-independent (sh_degree = 0) asset. Always safe, loses specular highlights.
"keep" — copy the coefficients verbatim, for callers whose SH is already object-aligned.

Colours, opacities and scales are copied untouched; only the means, the orientation quaternions and (under "rotate") the SH bands move, exactly as a rigid transform should.

3dgs_io.actor_assets.load_actor_asset_dir(directory)¶

Read a standalone asset-bank directory.

The directory mirrors the in-archive layout exactly — an actor_assets.json at its root with actor_assets/<asset_id>/asset.spz payloads beside it — so a bank can be authored once and packed into many bundles.

Return type:: tuple[ActorAssetBank, dict[str, bytes]]

3dgs_io.actor_assets.parse_actor_assets(doc)¶

Inverse of serialize_actor_assets().

Return type:: ActorAssetBank

3dgs_io.actor_assets.rotate_sh_about_z(sh, angle_rad)¶

Rotate real-SH coefficients about the +Z axis by angle_rad.

sh is (n, coefs, 3) in the 3DGS / SPZ layout: the DC band is stored separately in colors, so coefs covers bands 1..degree with m ascending from -l to +l inside each band.

Rotation about the SH polar axis mixes only the (m, -m) pair, so this is exact and closed-form (no Wigner-D machinery). angle_rad is the yaw of the source frame relative to the target frame: the returned coefficients evaluate at d to what the input evaluated at R_z(angle_rad) @ d.

Return type:: ndarray

3dgs_io.actor_assets.save_actor_asset_dir(directory, bank, payloads)¶

Write a bank out as a standalone directory (inverse of the loader).

Return type:: Path

3dgs_io.actor_assets.encode_actor_assets_doc(bank)¶

Serialise a bank to the exact bytes written to actor_assets.json.

Every writer goes through here so a bank packed into a bundle, retrofitted onto one, or dumped to a directory is byte-identical — otherwise archives that hold the same bank stop comparing equal.

Return type:: bytes

3dgs_io.actor_assets.resolve_actor_asset_bank(actor_assets, actor_instances=None)¶

Normalise the two ways a caller supplies a bank into (bank, payloads).

actor_assets is either a list of in-memory ActorAssetSource clouds to encode, or a standalone bank directory whose payloads are carried through byte-for-byte. Explicit actor_instances always win: for a directory they replace the bindings recorded there. Returns (None, {}) when neither argument is given, so writers can pass their kwargs straight in.

Both the build path (save_scene_usdz()) and the retrofit path (add_actor_assets_to_usdz()) resolve through here so the override rule has exactly one definition.

Return type:: tuple[ActorAssetBank | None, dict[str, bytes]]

3dgs_io.actor_assets.serialize_actor_assets(bank)¶

Build a frame-explicit object-local actor-asset document.

Return type:: dict[str, Any]

3dgs_io.actor_assets.validate_instances_against_tracks(bank, tracks)¶

Cross-check bindings against the bundle’s tracks.

Every bound track_id must exist. A rigidly-instanced asset whose own box disagrees badly with its track’s box is warned about — it renders, it is just very likely the wrong asset for that track.

Return type:: None

3dgs_io.actor_assets.verify_actor_payload(asset, payload)¶

Check an SPZ payload against the index record that names it.

Reads only the NGSP plaintext header and extension zone, so this stays O(1) in the Gaussian count — the packing paths carry payloads through byte-for-byte and must not pay a full decompress to sanity-check them.

Return type:: None