T4 Scene¶
A run's trajectory log can be written as a T4Scene:
the Protobuf exchange contract of tier4/e2e-devkit, TIER IV's repository (not
public; t4_scene.proto, contract 0.1.0), which the devkit decodes into its
T4Scene -- the scene its data lists, scorers and visualisations work on. It
is not the T4 dataset, the nuScenes-style directory of
sensor data t4.enabled writes during a run: a T4Scene is written from the
run's log alone, holds no sensor data, and needs neither CARLA nor any extra.
As each run ends¶
With t4scene.enabled=true the runner writes each run's scene itself, right
after it closes the run's trajectory log, beside it in the run's output
directory -- a batch's cases and a SLURM task's alike:
| Parameter | Type | Default | Description |
|---|---|---|---|
t4scene.enabled |
bool | false |
Write each run as {ScenarioName}.t4scene.pb and {ScenarioName}.t4scene.map.pb |
t4scene.step_s |
float | 0.1 |
The scenes' time step, a positive multiple of the 0.05 s tick (--step); another value, a bool (true, which would otherwise read as 1 s) or a string is refused before anything runs; null is the default |
t4scene.max_distance_m |
float or null | 100.0 |
The road users within this distance of the ego are a scene's objects: a positive, finite distance, another value is refused before anything runs; null takes all of them (--max-distance, --all-objects; --all-objects ignores --max-distance, which is then not checked) |
A batch takes the first case's settings, as it does t4's -- a SLURM batch
too, whichever task runs a case (SLURM). The conversion
runs on the runner's thread once CARLA is done with the case (the map is read
once however many cases ran on it). Like the log itself it is best effort: a
log that cannot be converted is logged as a warning and the run's verdict
stands.
To a driver policy, every step¶
With driver.t4_scene=true (ego.entity=carla_driver), every drive also
carries the run so far as a T4Scene, in CarlaRendererData.t4_scene
(carla-driver-interface 2.4 and later), for a policy that plans from scenes --
carla_driver_interface's T4SceneDriver, which hands it to a devkit driver
(t4_e2e_devkit.driver.AbstractT4Driver):
uv run scenario scenario=cut_in/left ego.entity=carla_driver \
driver.map_dir=/tmp/maps driver.t4_scene=true
| Parameter | Type | Default | Description |
|---|---|---|---|
driver.t4_scene |
bool | false |
Send each drive's scene; needs driver.map_dir and driver.send_renderer_data |
driver.t4_scene_history_s |
float | 3.0 |
How far back a scene reaches, the current row aside |
driver.t4_scene_wheel_base_m |
float or null | null |
The ego's wheel base in the scenes' ego shape; null leaves it out |
driver.t4_scene_goal |
scenario or road_ahead |
scenario |
The scenes' goal: where the scenario routes the ego (route_to; none without one), or the far end of the road ahead the policy is sent -- where a policy reading the egodriver observations has its goal |
driver.t4_scene_map |
run or map_set |
run |
The scenes' map: the run's Lanelet2 map, or the map set's (map_dir: roadgen's from the world's OpenDRIVE, or lanelet2_path), the one a policy reading the egodriver observations is given. The route and the lights are placed on it |
Each scene is what the log converts to (below), built as the run goes from the
same records (t4_scene.live): history only, the current row last -- the
drive's own tick -- and the rows before it 0.1 s apart, counted back from it,
its states unrounded (the log keeps millimetres). The rows are anchored on the
drive, not on the run's first tick, so a driver.policy_timestep_s that is not
a multiple of 0.1 s (0.05, 0.15 s, ...) still shows the policy its own tick,
with the current sweep on it at its true time; with one that is (the default),
the rows are the ones the log converts to. Every row's time is still
nanoseconds since the run's first tick, so two scenes' rows need not fall on
the same times. A run younger than t4_scene_history_s sends the rows it has.
On top of what the log holds, a scene carries what the policy is told and
a log does not: the ego's velocity and acceleration as the egodriver
observations carry them (of base_link, the acceleration an estimate), the
angular velocity (on a row the policy was not told, the log's) -- each turned
from the rig's axes into the scene's by the row's poses_3d rotation, into
velocities_3d, accelerations_3d and angular_velocities_3d -- front tyres'
angle and odometry times; the first LiDAR's
current sweep on the current row ([N, 5] in the rig frame, intensity on the
0-255 scale, the fifth column zero); the route on every row -- the road ahead
the policy is sent (driver.route_horizon_m) walked along the map's lanelets
as OnePlanner's egodriver policy walked it (drivable lanelets' centrelines,
within 2.5 m and heading the same way; successors and lane changes from a
vehicle routing graph, junction bound ends within 0.3 m welded; the points and
the ego's pose in single precision, as the egodriver observations carry them),
after the few the ego has just passed -- and the goal (t4_scene_goal). A policy
reading scenes is thus shown what one reading the egodriver observations is. Its map is written once into the map set as the t4
format (<map_dir>/<map_id>/t4_map.pb, a MapResource), which the scene
references. Every other observation is sent as before: a policy that does not
read scenes is unchanged. The scenes are built by a trajectory recorder of
their own: if it fails (logged), the history stops there, and the policy is
sent no scene from then on -- as before the first tick -- rather than the
frozen history as now; that is logged once, and the run goes on.
After a run¶
# Every log under a run's output directory, written beside each log
uv run scenario-t4scene outputs/2026-10-29/10-00-00
# One log, somewhere else, at 20 Hz, with every road user however far
uv run scenario-t4scene outputs/.../CutInScenario.trajectory.jsonl -o scenes/ --step 0.05 --all-objects
A path that does not exist, or a directory with no log under it, is a usage
error (exit 2) before anything is written. Ctrl-C prints
scenario-t4scene: interrupted and exits 130.
A log whose last line is cut off -- a run killed while it wrote its last tick -- is converted without that line, with a warning; a line that is not JSON before the last, or a record that cannot be used, fails the log with its file and line number (what a reader has to handle).
from autoware_carla_scenario.t4_scene import trajectory_log_to_t4_scene, write_t4_scene
export = trajectory_log_to_t4_scene("CutInScenario.trajectory.jsonl")
export.scene # t4_scene_pb2.T4Scene
export.map_resource # t4_scene_pb2.MapResource, or None without a map
write_t4_scene("CutInScenario.trajectory.jsonl") # -> the two paths
Files¶
| File | What it holds |
|---|---|
{stem}.t4scene.pb |
One serialised T4Scene message: the whole run |
{stem}.t4scene.map.pb |
One serialised MapResource: the Lanelet2 map the run was on, which the scene's map names by SHA-256 |
The contract carries no map bytes in a scene, so the map is a file of its
own; a scene without one (--no-map, or a log whose map is not on this
machine any more) has no map and no traffic_lights, and writing it
removes the {stem}.t4scene.map.pb an earlier export left, which it does not
name.
Each file is written in one rename (a scratch file beside it, renamed over
it), so a run killed while it writes leaves no truncated message -- one that
would often still parse, as a shorter scene. The scene goes last, after its
map, and an earlier export's scene is removed first: a .t4scene.pb on disk
always has the map it names beside it.
What goes where¶
| T4Scene | From the log |
|---|---|
scene_metadata |
scene_id: the log's stem; recording_id: the world's episode; tags (each key:value, as the devkit requires): source:autoware_carla_scenario, map:{name}, and result:passed (the result's passed is JSON true) or result:failed (anything else) when the run reached a verdict |
timestamps_ns, frame_indices |
One row every --step seconds (default 0.1 s, a multiple of the 0.05 s tick) from the first tick to the last: nanoseconds since the first tick, and the CARLA frame of each row |
current_index |
--current-index (default 0: the whole run is the recorded future) |
ego |
The actor whose role is ego or hero, its motion in the contract's 3D fields, all in the scene's axes: poses_3d at its base_link, velocities_3d of its base_link (below), angular_velocities_3d (rad/s, from the ticks' angular_velocity); the devkit derives the planar pose, velocity and longitudinal speed from them, and refuses the legacy poses, velocities, accelerations, longitudinal_speeds and angular_velocities, which are left unset. Then available, turn_indicators (the log's turn_indicators records: disabled 1, left 2, right 3, hazard 4; unknown before the first), and its shape |
annotations |
Every other vehicle and walker within --max-distance (default 100 m) of the ego, as a box (the boxes group): its box's centre, size (width, length, height), heading and quaternion, velocity, labels (vehicle 1, pedestrian 7) and track_ids (the CARLA actor id) |
traffic_lights |
One group per traffic light regulatory element of the map, its one element the round lamp of the CARLA light it shows: colour (red 1, yellow 2, green 3, else unknown 0), shape circle, status on (off for CARLA's Off, else unknown), confidence 1. A consumer derives each lanelet's state from the map, as for any source |
map |
The MapResource's SHA-256, scope full, with elevation, map_id lanelet2-sha256:{the .osm file's SHA-256} |
routes |
With the map, the lanelets the ego drove along, in order, one route (no preferred_lane_ids, no timestamp_ns) that every row's indices entry points at: its base_link on the rows it is on, a point a metre, walked along the map's lane graph as a live scene's road ahead is (RouteGraph.match: straight on, through a junction's branch, across a lane change). None when the ego hardly moved or drove on no lanelet, or with --no-route |
What the log does not hold is left out, never made up:
- A row the log has no tick for -- a tick without road users is not logged -- has its ego, objects and lights unavailable; rows are never interpolated.
- The ego has no accelerations or steering, and its
shapenowheel_base; nor angular velocity in a log without the ticks'angular_velocity(one written before it was recorded); nor turn indicators in a log withoutturn_indicatorsrecords (one written before they were recorded). - Other road users' turn indicators are in the log but not in the scene: the contract has them for the ego only.
- A vehicle's class is
vehicle: the log has its blueprint, not thebase_typethat says car, truck or bus. - A light's bulbs beyond its one round lamp: CARLA's light has one state.
- The route the ego was given -- the scenario's waypoints, or the one Autoware planned -- is not in the log: the scene's route is the one it drove, without the lanelets beside it that a planned route also holds.
- No goal.
- An entity spawned hidden is on the scene from its
spawnrecord to itsdespawnrecord, not while it is parked out of sight.
Frames¶
- The scene's axes are Autoware's
mapframe, the Lanelet2 map's, converted from CARLA's as the trajectory log says, with themgrs_offsetandz_offsetof itsmaprecord. Both the scene and the map are registered at the identity (global_origin,global_origin_3d), so the devkit draws and scores one against the other. - The ego's
base_linkis its rear axle, as in T4 data: where the run took it to be, which the log'srear_axlerecord says (where the ego put it: the driver policy's rig origin, the ROS 2 ego's odometry or the Autoware ego'sbase_link; else from the wheels, else the box), or--rear-axle-offsetmetres along its x axis; in a log without that record, half its box's half-length (-0.5 * extent.x, a quarter of its length) behind its origin, where the run takes it on CARLA UE5 (no wheel positions).shape.base_link_to_frontis from there to its box's front, andbase_link_to_rear_axleis 0. The offset may be any finite number (negative behind the origin, positive ahead of it); a--rear-axle-offset(orrear_axle_offset_m) ofnanorinfis refused before anything is written, and so is a log whoserear_axlerecord holds one; so is arear_axle_offset_mor record that is not a number (a boolean, a string such as"1.5",nullin a record), the error naming the setting. - The ego's velocity is its
base_link's too. The log records its actor origin's, so the rigid-body lever arm is added:v = v_origin + w x r,wthe tick'sangular_velocityandrthe rotated(rear_axle_offset, 0, 0), in the map frame -- as the driver observations do (ego_observation). The forward speed is the same either way; a turning ego's sideways velocity is not (yaw_rate * rear_axle_offsetof slip at the origin). A log written before ticks carriedangular_velocitykeeps the origin's velocity. - A light shows a traffic-light regulatory element when it switches an
OpenDRIVE signal (its
opendrive_id) of the controller named after the element (Controller_TL_{id}, the Lanelet2-to-OpenDRIVE convention, asTrafficSignalActionmaps them); else, for a map without such controllers (roadgen's, carla-ue5-maps'), when one of its stop-line waypoints is on a lanelet the element governs or within 50 m of lanelets before one -- the first such light, in the log's order. That is howcarla_driver_interface.hdmapfinds a light on a Lanelet2 map: CARLA puts its stop waypoints where its own junction starts, often metres before the map's stop line.
scenario-t4scene reads a map once however many of its logs ran on it.
The map¶
The map is the run's own .osm (its path is in the log's map record;
--lanelet2 and --xodr point at it when it has moved), read with the
projection the run read it with, and written in the contract's one profile,
LANELET2_OSM in LOCAL_METERS: OSM XML whose nodes carry their exact map
coordinates as local_x/local_y/ele, with auxiliary lat/lon that are
not a geolocation. Its SHA-256 is over the encoding, the projection and those
bytes. This is the devkit's own encoding (t4_e2e_devkit.maps.io.osm_bytes),
reimplemented on Lanelet2 so that writing a scene needs neither the devkit nor
its torch; the devkit's MapResourceStore.register verifies it.
Reading it with the devkit¶
from t4_e2e_devkit.serialization.protobuf import MapResourceStore, from_proto, t4_scene_pb2
from t4_e2e_devkit.viz import figure_to_rgb, plot_bev_frame
store = MapResourceStore()
store.register(t4_scene_pb2.MapResource.FromString(open("Run.t4scene.map.pb", "rb").read()))
scene = from_proto(
t4_scene_pb2.T4Scene.FromString(open("Run.t4scene.pb", "rb").read()),
map_resources=store,
)
figure, _ = plot_bev_frame(scene, {"ground_truth": None}) # the BEV at current_index
dataclasses.replace(scene, current_index=i) moves the anchor, so a video of
the run is one plot_bev_frame per row (the devkit's FFmpegVideoWriter
encodes them).
The contract¶
proto/t4_e2e_devkit/t4_scene.proto is the devkit's proto/t4_scene.proto,
copied byte for byte; its Python modules are generated into
t4_scene/_proto/ by scripts/compile_protos.py (proto/README.md).
Its descriptor keeps the devkit's name, t4_scene.proto, and package,
t4.devkit.experimental.v1, so the devkit's bindings and these load into one
process. The contract is experimental on the devkit's side: when it moves,
copy the file again and regenerate.