Skip to content

External Driver Interface

The ego vehicle in a scenario is normally driven by CARLA's TrafficManager. This page describes the alternative: handing the ego to an external driving policy that plans over gRPC, so a scenario becomes a test of that policy rather than of TrafficManager.

The wire contract is alpasim's egodriver.EgodriverService. Its policy side is carla-driver-interface (2.x), a light package (grpcio, protobuf, numpy, Pillow; Python 3.10+) a policy depends on without pulling in the scenario framework. It is versioned on its own: the framework requires carla-driver-interface>=2.4.2,<3, so a policy and a scenario package can share an environment when the policy accepts 2.4.2 or a later 2.x.

A policy run as a separate process may be on another major. Each side declares the contract revision it speaks (carla-driver-interface's major) in gRPC metadata, and the framework declares its cameras in the policy's revision:

Policy on rig_to_camera the framework declares
carla-driver-interface 2.x (revision 2) the camera's optical frame in the rig (x right, y down, z along the optical axis), as alpasim declares it
carla-driver-interface 1.x (declares no revision) the camera body in the rig (x along the optical axis, y left, z up), as 1.x reads it

Either way the pose is in the rig, whose origin is the rear axle (see Coordinate frames below), while a camera's position_* mount is relative to the CARLA actor origin, the vehicle's centre, where CARLA attaches it: the declared pose is the mount moved by the rear-axle offset, as the LiDAR's sweeps and a T4 dataset's camera calibration are. The default camera at position_x: 1.5 on a car whose rear axle is 1.4 m behind its centre is declared at x = 2.9 in the rig.

A 2.x policy is handed the optical frame either way, also by autoware_carla_scenario 4.x, whose cameras its server translates. A revision the framework does not know fails start_session with ContractError, before the scenario starts.

Architecture

The scenario framework plays the runtime role: it owns the world, ticks the simulation, renders observations, and asks the policy what to do. The policy is a separate process serving egodriver.EgodriverService.

flowchart LR
    subgraph scenario["Scenario process (Python 3.10-3.14)"]
        SR["ScenarioRunner<br/>owns the world and the tick loop"]
        CDE["CarlaDriverEntity"]
        CAM["CarlaCameraSensor(s)"]
        TF["TrajectoryFollower<br/>pure pursuit + yaw-rate trim<br/>Autoware's PID longitudinal controller<br/>through the powertrain model"]
        SR -->|on_tick| CDE
        CAM -->|frames| CDE
        CDE --> TF
        TF -->|VehicleControl| SR
    end

    subgraph policy["Policy process"]
        P["egodriver.EgodriverService<br/>e.g. carla-driver-interface serve"]
    end

    CDE -->|"observations + drive()"| P
    P -->|"planned trajectory"| CDE

Each simulation tick, CarlaDriverEntity tracks the most recent plan and applies a control. Every driver.policy_timestep_s it also submits fresh observations — camera frames, the ego's pose and dynamic state at every tick since the previous step (the egomotion history the upstream runtime sends), and a route window rolled forward from the ego's current position (its rear axle) — and asks the policy to re-plan. Between policy steps the cached plan keeps being tracked, which is how a policy slower than the simulation stays usable.

Running a scenario against a policy

The scenario can serve the policy itself, in its own process, so one command runs everything. Name the policy in driver.policy (a reference policy, or package.module:Class for your own BaseDriver):

uv run scenario ego.entity=carla_driver driver.policy=route_follower

The policy is built once per run and served on a free local port; driver.address is not used. Every scenario of a batch opens its own session on it, so a model is loaded once. Over the wire nothing changes: by default the scenario still dials it over gRPC, exactly as it dials a policy in another process, and with driver.transport=direct it exchanges the same messages without a socket (below).

To run the policy as a process of its own (another environment, a GPU host, an alpasim driver), leave driver.policy unset and start the policy first. It is a server, and the scenario is the client:

# In the policy's own environment
uv run carla-driver-interface serve --policy route_follower --port 50051

Then run any scenario with the carla_driver ego entity:

uv run scenario ego.entity=carla_driver driver.address=localhost:50051

How the run reaches the policy: driver.transport

driver.transport driver.policy set (the run serves it) driver.policy unset (another process)
tcp (default) served on a free local port and dialled there dials driver.address (host:port)
unix served on a Unix socket in a directory of the run's own (0700, under $TMPDIR, or /tmp) dials driver.address, which must name a socket (unix:///run/policy.sock)
direct no socket: the run calls the policy's servicer through carla_driver_interface.inprocess refused: a policy in another process has to be dialled
uv run scenario ego.entity=carla_driver driver.policy=route_follower driver.transport=direct
# a policy in its own process, on a socket
uv run carla-driver-interface serve --policy route_follower --host unix:///run/policy.sock
uv run scenario ego.entity=carla_driver driver.transport=unix driver.address=unix:///run/policy.sock

Whichever it is, the policy sees the same alpasim egodriver messages: direct still serializes every request and response to the bytes the wire would carry and parses them on the other side, and its servicer is the one a server runs, with a context that behaves as gRPC's (an abort is a grpc.RpcError with the same code and details, the contract revision is negotiated through the same metadata). A policy that is an alpasim driver therefore drives the same whether it is dialled over TCP or called in-process; only the transport's cost goes (a sweep's megabytes are not copied through the loopback twice per step). The one difference is a deadline (driver.timeout_s): the policy runs on the run's own thread, so an overrun is reported when its call returns rather than cut short.

Under the SLURM launcher every task serves its own policy, so neither unix nor direct uses a port of its slot (slurm.md).

ScenarioRunner logs Autopilot skipped for ego (id=…) — external control expected when the policy has taken over. Pass and fail conditions, recording, the trajectory log, and sweeps all behave exactly as they do for a TrafficManager-driven ego.

Configuration

The driver config group (conf/driver/default.yaml) holds the settings:

driver:
  address: localhost:50051
  policy: null               # e.g. route_follower: served by the run itself
  transport: tcp             # tcp | unix | direct; see "How the run reaches the policy"
  timeout_s: 60.0
  policy_timestep_s: 0.1     # a positive multiple of the 0.05 s tick; others are refused
  image_quality: 90          # JPEG quality, a whole number in 1-100
  warmup_s: 0.0              # run-up onto the spawn pose; see "Warming up a policy"
  route_horizon_m: 80.0
  route_resolution_m: 2.0
  rear_axle_offset_m: null   # null: from the wheels, else the box
  send_ground_truth: false
  send_renderer_data: true       # CARLA ground truth: lights, traffic, speed limit
  send_actor_ground_truth: true
  traffic_light_sight_distance_m: 60.0
  actor_horizon_m: 150.0
  random_seed: null           # null: the run's seed (`seed`); else a whole number in 0..2**31 - 1
  cameras:
    - logical_id: camera_front_wide_120fov
      image_width: 960
      image_height: 604
      fov: 120.0
      position_x: 1.5
      position_z: 1.6
  control:
    lookahead_gain_s: 0.6
    wheelbase_m: 2.8
    max_steer_angle_deg: null   # null: 56 deg (CARLA 0.10: angle = 56 deg * steer**2)
    max_steer_angle_rad: null   # ... or the same in radians; set one of the two
    steer_exponent: 2.0
    yaw_rate_ki: 3.0            # integral trim on the measured yaw rate
    longitudinal_controller: pid  # Autoware's autoware_pid_longitudinal_controller
    delay_compensation_time: 0.17 # ... and any of its parameters, by name
    throttle_deadband: 0.3      # vehicle interface: least throttle to pull away

Every number in driver must be a number: an int or a float, never a boolean or a string. Hydra reads driver.policy_timestep_s=true as a boolean, which would otherwise be taken as 1 (a 1 s policy step), and a quoted "0.1" as a string; both are refused as the config is read, naming the key. This holds for the seconds and metres (timeout_s, policy_timestep_s, warmup_s, route_horizon_m, route_resolution_m, rear_axle_offset_m, traffic_light_sight_distance_m, actor_horizon_m, t4_scene_history_s, t4_scene_wheel_base_m), for every number of a camera, its lens and a LiDAR, and for image_quality, a whole number (see below) in 1-100: 90, or 90.0 kept as 90. Each is refused out of its domain too, naming the key (driver.route_horizon_m must be positive, got 0.0): warmup_s, traffic_light_sight_distance_m and actor_horizon_m must not be negative (0 is no run-up, CARLA's own light trigger, and no road user reported); route_horizon_m, route_resolution_m and t4_scene_wheel_base_m (when not null) must be positive; timeout_s, policy_timestep_s positive as well, and t4_scene_history_s not negative.

The switches send_ground_truth, send_renderer_data, send_actor_ground_truth and t4_scene are each true or false, or one of their spellings as text in any case ("false", "no", "off", "0", and "true", "yes", "on", "1"), or the integer 0 or 1, as the run's other switches are (The switches). So a quoted driver.t4_scene='false' is false, not true; anything else (ture, 2, null) is refused as the config is read, naming the key (driver.send_ground_truth must be true or false, got 'ture'). driver.address must be a non-empty string (host:port, or unix:///path/policy.sock): null is refused too, naming the key, rather than dialled as None; leave it out for the default localhost:50051. driver.policy and driver.map_dir are each a non-empty string or null (no policy served, no map written): an empty or blank string, false or a number is refused as the config is read, naming the key (driver.policy must be a non-empty string, or null, got ''). driver.map_formats is a list of format names ([lanelet2, opendrive]), each one of those the map set is written in: a bare string (driver.map_formats=lanelet2, which would be read as its letters) and an unknown name are refused as the config is read, naming the key (driver.map_formats[0] must be one of [...], got 'lanelet3'), and so is t4 without driver.t4_scene, which alone writes that format.

The same holds for driver.control (and ros2.control): each gain and limit is a finite int or float (an int is read as a float), each enable_* switch a boolean and velocity_smoothing_window a whole number, and a *_deg angle must be a number too. A null setting is its default, as an absent one. Each angle is set in degrees (max_steer_angle_deg, yaw_rate_trim_limit_deg) or in radians (max_steer_angle_rad, yaw_rate_trim_limit_rad), not both: setting both is refused, naming the two keys (driver.control.max_steer_angle_deg and max_steer_angle_rad set the same angle, ...; set one of them to null), since neither may silently win. conf/driver/default.yaml leaves both of the full lock's null, so driver.control.max_steer_angle_rad=0.6 and driver.control.max_steer_angle_deg=35 each work as an override. What the controller divides by -- wheelbase_m, max_steer_angle_deg (or _rad), steer_exponent, min_lookahead_m and velocity_lookahead_time -- must be positive, and max_lookahead_m at least min_lookahead_m. So must max_steer_rate (a steering rate limit of 0 would not steer at all; a large one barely limits). The pedal values stop_brake and throttle_deadband (0: no floor) and the low-pass gains lpf_vel_error_gain, lpf_acc_error_gain and accel_lowpass_gain are in [0, 1], and lookahead_gain_s is not negative. A value out of these bounds is refused before the run starts, naming its key (driver.control.steer_exponent must be positive, got 0.0), not once the ego first steers.

Each driver.cameras entry needs its own logical_id, a non-empty string; two cameras under one id are refused. Its image_width and image_height must be positive whole numbers, its fps positive and finite, its fov in (0, 180) degrees -- CARLA renders the pinhole of the horizontal fov -- and its mount (position_x/y/z, roll, pitch, yaw) finite. A camera out of these bounds is refused as the config is read, not when it attaches after the ego spawned.

A null setting of a driver.cameras or driver.lidars entry is its default, as an absent one, the way a ROS 2 camera's is: logical_id: null is camera_front_wide_120fov (a LiDAR's lidar_top), fov: null 120 degrees, range_m: null 100 m; lens: null is CARLA's pinhole and a LiDAR's noise_seed: null the run's seed, as by default. An entry refused names itself by its place in the list and its logical id (driver.lidars[1] ('lidar_rear'): LiDAR channels must be positive), and driver.cameras and driver.lidars must each be a list of mappings.

A whole number is an integral value: 960, or 960.0 (or 9.6e2), which is kept as the int 960, so the policy's CameraSpec resolution (an integer protobuf field, which refuses any float) is given 960 and CARLA image_size_x="960". 960.5, NaN, infinity, a boolean and a string are refused, naming the field, as the config is read. Every whole-number setting of a camera, a LiDAR or the ROS 2 ego follows this rule, as a seed does (noise_seed, driver.random_seed).

A camera with a lens of its own: driver.cameras[].lens

A camera's frames are CARLA's pinhole of its fov unless it says otherwise. A carla_canvas renderer may draw a camera through another lens: the Cosmos3 world model (cosmos3-canvas) draws only its Hyperion 8 cameras, each through its F-theta fisheye lens. Such a camera carries its lens, so the policy and a T4 dataset are told what the frames are:

driver:
  cameras:
    # python -m cosmos3_canvas.camera camera_front_wide_120fov
    - logical_id: camera_front_wide_120fov
      image_width: 832
      image_height: 480
      fov: 120.0
      lens:
        model: ftheta
        principal_point: [412.641208, 242.790324]
        pixeldist_to_angle_poly: [0.0, 0.00243165382, -4.05625414e-08, 6.10818843e-10, -9.93181484e-13, 8.73787714e-16]
        angle_to_pixeldist_poly: [0.0, 411.286296, 2.40299018, -16.2530848, 10.3586782, -3.39986006]
        max_angle: 1.226783919
        linear_cde: [1.00234926, 0.0, 0.0]
Field Meaning
model ftheta, NVIDIA's F-theta model: the only one
principal_point [x, y] in pixels (the centre of the top-left pixel is (0, 0))
pixeldist_to_angle_poly Radius from the principal point (pixels) to the angle from the optical axis (radians); coefficients from the constant term up
angle_to_pixeldist_poly Its inverse, the angle to the radius
max_angle The largest angle the lens sees (radians)
linear_cde [c, d, e]: [[c, d], [e, 1]] maps the radial offset to pixels
reference_poly Which polynomial defines the lens: pixeldist_to_angle (default) or angle_to_pixeldist

The policy is sent the lens as alpasim's FthetaCameraParam, in place of the pinhole. The CARLA camera is still made with image_width, image_height and fov: what the renderer reads to know which of its cameras it is.

How the plan is tracked

TrajectoryFollower (driver/control.py) does what Autoware's control stack does behind an end-to-end planner:

  • Speeds on the plan. A plan is poses and times. The speeds Autoware's controller reads off a trajectory are put on it as autoware_tensorrt_e2e puts them on its own (autoware_diffusion_planner's get_trajectory_from_poses): each point's speed is its distance from the previous point -- the first point's from where the ego was as the plan arrived -- over the plan's time step; then a forward moving average over velocity_smoothing_window (8) points, the tail keeping the last average. While the ego moves, the first average to fall below stopping_threshold (0.3 m/s) stops the plan there. Each point's acceleration is the change of speed to the next.
  • Longitudinal. A port of autoware_pid_longitudinal_controller with the defaults of its param.yaml, under its parameter names. The plan is read spatially: the stop distance is the arc length from the ego to the plan's first zero-speed point (or its end). STOPPED departs once that is over drive_state_stop_dist (0.5 m), DRIVE stops through STOPPING (smooth stop) once it is under stopping_state_stop_dist, STOPPED holds stopped_acc, and an overshoot of emergency_state_overshoot_stop_dist brakes at emergency_acc (EMERGENCY). DRIVE follows the speed at the point the ego will be at after delay_compensation_time, with Autoware's velocity lookahead feedback (or, with enable_velocity_lookahead_feedback: false, its PID and feedforward), within the acceleration and jerk limits.
  • Lateral. Pure pursuit on the plan's points, with an integral trim on the measured yaw rate.
  • Vehicle interface. The asked acceleration becomes pedals through the ego's powertrain model, read from its CARLA physics control, as Autoware's raw_vehicle_cmd_converter does with its accel and brake maps. The model lets a standing car move at any throttle while CARLA 0.10's MKZ stays put below 0.25. A plan pulling away at a crawl asks a standing car for a throttle near 0.23, so throttle_deadband (0.3) floors the throttle of a standing car asked to speed up: it moves as a real car would creep.

What differs from Autoware, because CARLA or the plan has no counterpart:

  • The plan carries no pitch, so there is no slope compensation; CARLA's towns are flat where the scenarios drive.
  • The follower has no steering angle feedback: the steering counts as converged, so enable_keep_stopped_until_steer_convergence holds only the first tick of a departure, as Autoware's first check does.
  • The ego is always under Autoware's control (AUTONOMOUS), and always in drive: the manual-driving transitions and reverse gear are left out.
  • The ego's acceleration is estimated from its speed with autoware_twist2accel's low-pass gain (accel_lowpass_gain, 0.9).
  • The filters run per control step, at the simulation's 20 Hz rather than Autoware's 33 Hz.
  • A plan's times come from its timestamps rather than the node's fixed 0.1 s, and the smoothing window is capped at one less than the plan's points; a 0.1 s plan gets Autoware's numbers. carla-driver-interface leads each plan with the ego's pose at the planning time; that anchor is the base the first speed is measured from (the node's ego position), not a point of the tracked trajectory. A plan without one is measured from where the ego is as it arrives. Either is latched with the plan.
  • A plan of fewer than two points is ignored and the last one tracked on, as Autoware's setTrajectory does.
  • So is a plan that is not all finite (a NaN or an infinity in a pose, or in a ROS 2 trajectory's speeds or accelerations), with a warning, the first time and then once in a hundred; with no plan before it, the ego is held. A non-finite yaw rate is not fed back, and a non-finite speed or ego pose, or a command that comes out non-finite gives no throttle and the stop brake, the steering held. No controller state is updated with a non-finite value, so one bad inference cannot latch in it.
  • A follower taken over on a moving ego (the warm-up's hand-off, a spawn at speed, a new ROS 2 command source) starts in DRIVE rather than STOPPED, which would brake for a tick.
  • The smooth stop fits the speed history of the delay's control steps: 3 at 0.05 s, where Autoware keeps 5.

A ROS 2 autoware_planning_msgs/Trajectory is tracked on its own longitudinal_velocity_mps and acceleration_mps2, every point kept (a stop included) even when its time_from_start is not set.

A plan that dwells on the ego's spot -- the first eight speeds averaging to zero -- stops at its first point, and the ego stays STOPPED until a plan's stop point is more than 0.5 m away, as Autoware's would. longitudinal_controller: time_indexed selects the tracker this framework had before: the plan read on its own clock, a PI on the speed it has the ego at now and a pull towards where it has it (speed_kp, speed_ki, integral_limit, position_gain, max_position_correction_mps, speed_preview_s, stop_distance_m); RoutingAction in follow mode drives timed trajectories with it. Those keys are ignored, with a warning, for pid.

Warming up a policy

A policy that reads a history -- past LiDAR maps, the ego's past poses -- has none at the first step and plans as if the ego had stood still: an ego spawned moving brakes on those first plans. driver.warmup_s gives it a run-up instead: the warmup_s before the scenario's first frame are played by rule, so that everything arrives at its first-frame pose at its initial speed.

  • Every vehicle with an initial speed, the ego among them, is put back along its lane by its initial speed times warmup_s, keeping its offset from the lane centre and its heading relative to the lane, and driven onto its pose. One with no initial speed stands on its pose. One whose lane ends sooner than that waits at the lane's end until it is due to leave; the ego reports a speed of zero to the policy while it waits, and its initial speed once it moves.
  • Every pedestrian with an initial speed is moved in a straight line along its heading; one with none stands where it starts.
  • The traffic lights are frozen, so the run-up spends none of their phases, and thawed on arrival. A light the scenario (a TrafficSignalAction in register_init, say) or a signal controller already froze stays frozen and is not thawed on arrival.

The policy gets every observation and plans all the way, but does not drive; the scenario's clock starts, and the policy takes over, on arrival. Set warmup_s to the history the policy reads (3 s for ResWorld's ego past).

ego.entity selects the entity and accepts four values:

Value Behaviour
autopilot (default) CARLA's TrafficManager drives the ego.
autoware Nothing drives the ego; the actor is left for an external stack.
carla_driver An external policy drives the ego over the contract described here.
ros2 ROS 2 nodes drive the ego over DDS (ROS 2 over DDS).

A policy is usually built for one rig, so a preset can choose the vehicle and cameras along with the rest of the group. driver=vision_pilot is VisionPilot's: a Lincoln MKZ with one 1920×1280, 50° front camera at 10 Hz (its own CARLA rig), and it hands the ego to the policy (ego.entity: carla_driver):

uv run vision-pilot-driver --model-dir <weights> --port 50051   # in vision_pilot
uv run scenario driver=vision_pilot

A preset extends driver/default (defaults: [default, _self_]) and may set ego.*, since the driver group is composed after ego.

logical_id is the name the policy looks a camera up by, so it must match what the policy expects. carla-driver-interface's reference policies use camera_front_wide_120fov.

A stack serving the policy's world (driver.bridge)

A policy can drive in a world something else serves it -- splatsim rendering what its cameras see, say -- that has to come up before the policy can drive, and has to know where the run goes. driver.bridge gives the ego the same AutowareBridge handshake an ego.entity=autoware run uses: the framework hosts the bridge, the stack's interface node dials it, pulls the scenario's mission (initial pose, goal, waypoints, at its base_link) and reports when it is ready.

driver:
  bridge:
    address: 0.0.0.0:50052     # AutowareBridgeConfig's fields
    ready_timeout_ticks: 1200

With a bridge the scenario needs a goal (ego.goal_lanelet_id, or one the scenario derives), handed over when the scenario starts. Until the stack reports ready the policy is not queried and the scenario's clock does not start; the policy then starts -- its warm-up first, when warmup_s is set -- on the tick the stack turns ready. A stack that is not ready within ready_timeout_ticks, or exits, ends the run. null (the default) runs the policy alone, from the first tick.

From Python the same is CarlaDriverEntity(config, bridge=..., bridge_config=..., launcher=...); a launcher starts and stops the stack per scenario, as it does for Autoware.

Using it from Python

Scenarios that build their ego themselves can attach the entity directly:

from autoware_carla_scenario import CarlaDriverEntity, DriverClientConfig

scenario.ego_entity = CarlaDriverEntity(
    DriverClientConfig(address="localhost:50051", policy_timestep_s=0.1)
)

ScenarioRunner calls BaseScenario.create_ego(), which returns ego_entity when one is set and otherwise instantiates ego_type. Assigning ego_entity is the supported way to supply an entity that needs constructor arguments.

To test against a fake policy, pass a BaseEgoDriverClient implementation:

CarlaDriverEntity(config, client=MyFakeDriverClient(config))

CARLA ground truth

The alpasim contract has no field for a traffic light and none for other road users. Both ride inside DriveRequest.renderer_data, an upstream-sanctioned bytes extension point, as a serialized carla_driver.v0.CarlaRendererData:

Field Contents
ego_traffic_light State of the light governing the ego lane
ego_traffic_light_distance_m Distance to its stop line along the ego's heading; negative once crossed or when no line applies
speed_limit_mps Posted limit for the ego lane, 0 when unknown
actors[] Other vehicles, pedestrians and props (vehicle.*, walker.*, static.prop.*, told apart by type_id): pose, bounding box, and velocity, all in the local frame
weather, map_name, frame_id Scene context
lidar[] One sweep per driver.lidars entry, packed [N, 4] float32 (x, y, z, intensity) in the rig frame
map_id, traffic_lights[] With driver.map_dir: which map file set describes the world, and every light with its state and stop points

Turning this off is not a no-op

A policy reads the payload defensively — a missing one means "no light applies" and "no other road users" rather than an error. So send_renderer_data: false does not fail loudly. It silently disables every rule that depends on the world outside the ego. For a specification-driven policy such as stl_driver (in this repository), that is stop_on_red, collision_free and safe_headway — the car keeps driving and the rules simply stop applying.

Finding the light that governs us

CARLA's is_at_traffic_light() answers a different question: whether the ego is inside the light's trigger volume, which is about a metre thick along the road. A policy asking CARLA therefore learns of a red light at the moment it arrives at it, when stopping from any ordinary speed is already impossible.

So the lane graph is walked forward instead — up to traffic_light_sight_distance_m — and any light whose stop line lies on one of those lanes governs us. Inside a junction nothing governs us: having crossed the line, the thing to do is clear the box.

The point reported is not the stop waypoint itself but the mouth of the junction it governs, reached by walking forward from the waypoint. Upstream measured stop waypoints a median of 5.5 m short of their junction on Town10HD_Opt; a policy told to stop there hesitates most of a car-and-a-half before the line a driver aims at.

This logic is ported from carla_driver_interface's reference runtime (runtime/carla_world.py at af1dcd3) so that a policy tuned against that runtime sees the same numbers here.

LiDAR

The contract has no LiDAR submission RPC, so sweeps ride in renderer_data too. Each driver.lidars entry mounts a sensor.lidar.ray_cast (mount in CARLA's convention, like a camera) that sweeps once per policy step (policy_timestep_s), and each policy step sends that sweep, already converted into the rig frame. The sweep may come from an earlier simulation tick of the step than the one the step is sent on; it is stamped with the tick it was taken on, not the tick it is sent on. A sweep that does not arrive within a second is left out rather than replaced by an older one. CARLA first sweeps a whole interval after the LiDAR is attached (its sensor_tick counts from the spawn), so the first read waits only for a sweep that can have been taken by then, as a camera's first read does: with a sweep every tick, that tick's; with a longer policy_timestep_s, none (the first step may then send no sweep, and no warning is logged for it) (#2228).

A policy does not see that transport: the servicer records each sweep before drive as a LidarFrame in session.frame_history, beside the camera frames, and announces it through on_frame -- the one path every sensor takes. frame_history_length applies to both, and session.latest_frame("lidar_top").as_array() unpacks the [N, 4] points as latest_frame("camera_front").as_array() decodes an image.

driver:
  lidars:
    - {logical_id: lidar_top, channels: 64, range_m: 100.0, position_z: 2.0}

LiDAR logical ids must be unique non-empty strings, and every number a LiDAR takes must be finite: channels and points_per_second positive whole numbers (64, or 64.0 and 1.2e6, which YAML reads as floats, kept as the ints 64 and 1200000 for CARLA; not 64.5), range_m positive, lower_fov_deg below upper_fov_deg, dropoff_general_rate (the fraction of points CARLA drops at random) from 0 to 1, weather_noise not negative, noise_seed a whole number (3 or 3.0; not 1.9, a boolean or a string) from 0 to 2147483647, fog_density_scale positive.

CARLA's ray-cast LiDAR ignores the weather, so the sweep sent has the world's rain (precipitation) and fog (fog_density, beyond fog_distance) put on it, read when the sweep is first read (the frame it is first sent on, at most one sweep interval after the frame it was taken on: weather changed between the two is put on it) -- what the weather rules render a run's rainfall rate and fog visibility as, environment.precipitation and environment.fog_density, and an EnvironmentAction mid-run set it alike (sensor/lidar_weather.py):

  • attenuation: a return at range r keeps exp(-2 alpha r) of its intensity and survives with that probability, so far returns thin out first;
  • range noise: a normal error along the ray, growing with the distance travelled through the weather;
  • early returns: a few beams come back off a drop or the fog a few metres out, at low intensity.

Each grows with its intensity (0-100). Clear weather leaves a sweep as CARLA cast it.

Field Default
weather_noise 1.0 Scale of the weather noise: 0 off, 1 as tuned, more exaggerates it
noise_seed null Seed of the noise; with the sweep's frame, it repeats a run's sweeps. Null takes the run's seed, mixed with the LiDAR's logical_id, so that LiDARs alike draw different noise on the same frame and each the same in every run of a seed; a seed given is used as it is, so two LiDARs given the same one draw the same noise
fog_density_scale server.fog_density_scale, else 0.001 The server's carla.Weather.FogDensityScale; normally left to server

How thick a fog_density looks is the server's carla.Weather.FogDensityScale, and the LiDAR's fog follows it so that it matches the cameras': the model is tuned at 0.01, the scale that draws fog a camera sees, and CARLA's default 0.001 gives a tenth of its fog. server.fog_density_scale (a finite number above 0, as a LiDAR's own must be, or null; anything else is refused before the run starts, LiDARs or none) sets both: a server the run starts is started with -ExecCmds="carla.Weather.FogDensityScale <scale>", and every LiDAR that sets no scale of its own takes it. A server already running keeps the scale it was started with (the run warns), so start it with the same one. Built without the runner, pass fog_density_scale_args(scale) (autoware_carla_scenario.server) as the server's extra_args and the same scale to each LiDAR.

driver:
  lidars:
    - {logical_id: lidar_top, weather_noise: 1.5}
environment:
  precipitation: 80   # heavy rain: sparser, noisier sweeps
server:
  fog_density_scale: 0.01   # fog the cameras and the LiDARs both see

The map, as files

alpasim's services read a scene's map from its artifact, never off the wire; this is the CARLA counterpart. With driver.map_dir set (and the map extra installed), the world's OpenDRIVE is converted by roadgen at scenario start and written to <map_dir>/<map_id>/, in the formats driver.map_formats names (Lanelet2 by default; lanelet2, opendrive, osm, clipgt and commonroad, a CommonRoad 2020a scenario), beside the OpenDRIVE source, roadgen's IR and its traces. map_id is <map name>-<first 12 hex digits of the OpenDRIVE's SHA-256>, so a set is written once per map and a changed map gets a new id. The id does not depend on the formats: a run asking for a format the set lacks writes it again with the formats it already holds and the new ones, under the same id, so a set only grows -- except by a format this version or its roadgen cannot write, which is dropped with a warning. The new set is built beside the old one and renamed into its place whole. An old set a run could neither replace nor put back (another run's set took its place meanwhile, or the run was killed) stays in <map_dir>/.<map_id>.*/old: later runs read its formats into the set they write, and never delete it.

Each step then carries only what changes: every light's state, named by its OpenDRIVE signal id, with its stop points on OpenDRIVE lanes. A policy that sets map_dir (its own copy of the directory) gets the set opened as ctx.map, and ctx.stop_lines() resolves the lights through roadgen's traces into its format's own elements -- for Lanelet2, the approach lanelet, the traffic-light regulatory element and the light's line strings:

class MyPolicy(BaseDriver):
    map_dir = "/shared/maps"   # the runtime's driver.map_dir, as this process sees it

    def drive(self, ctx):
        for stop in ctx.stop_lines():
            stop.state, stop.lane_ids, stop.position_local

To hand the policy a Lanelet2 map of your own instead of roadgen's conversion, name it with driver.lanelet2_path:

driver:
  map_dir: /shared/maps
  lanelet2_path: /path/to/lanelet2_map.osm

The file is copied into the set as lanelet2_map.osm, as it is (with the map_projector_info.yaml beside it, if any), and its content is part of map_id, so another file gets another set. roadgen is not needed unless driver.map_formats names other formats too. The file carries no roadgen trace (the manifest marks the format "provided": true, its trace empty), so ctx.stop_lines() finds each light on the map instead (carla-driver-interface 2.3 and later):

  • lane_ids are the lanelets the stop point lies on;
  • rule_ids are the traffic-light regulatory elements of the first lanelets on or ahead of it that carry one, up to 50 m past its own lanelet, so a map may hang its lights off the approach or off the junction lanes past the line;
  • light_ids are the line strings those elements refer to.

The map's local_x / local_y must be the local frame (CARLA's world with y mirrored, as carla-ue5-maps' are). Nothing checks that the file and the world's OpenDRIVE describe the same roads in the same frame.

roadgen validates the OpenDRIVE before it writes any format and has no option to skip that: a map it refuses (ValueError: the map failed validation (N issues): ...) ends the run at scenario start, before the driver session opens. The nishishinjuku fixture map (data/nishishinjuku_carla.xodr) is one (#2065, metrics.md). On such a map, hand the policy its Lanelet2 map with driver.lanelet2_path: with the default driver.map_formats: [lanelet2] nothing is left for roadgen to convert, so it is not run at all (the set's IR and read trace are left empty). Any other format in driver.map_formats still goes through roadgen, and fails the same way.

Diagnostics coming back

DriveResponse.debug_info.unstructured_debug_info is decoded as carla_driver.v0.CarlaDriveDebugInfo when it parses as one, surfacing the policy's name, its inference time, and whatever scalars it chose to publish. They are logged every ten policy steps. Anything that does not parse is left as raw bytes — the field is deliberately unstructured, and a policy is free to use its own encoding.

Turn indicators

A policy commands the ego's turn indicators as well as its plan (DriveResult.turn_indicator on the policy side, carla-driver-interface 2.1+). The egodriver contract carries only the plan, so the command does not ride in the response: after each drive the client asks for it with carla_driver.v0.CarlaVehicleService's get_vehicle_command, which the policy's server serves on the same port. A server that predates it answers UNIMPLEMENTED, and the policy is then taken to command nothing.

The entity lights the ego's LeftBlinker / RightBlinker with the command, leaving its other lights as they are, and holds them until the policy asks for others: DISABLE turns both off, and NO_COMMAND -- the default, and what a policy that never sets it sends -- changes nothing. The numbers are Autoware's TurnIndicatorsCommand ones, so a planner that classifies Autoware's recorded signal (DISABLE / LEFT / RIGHT) passes its class plus one. CarlaDriverEntity.turn_indicator is what is lit now; a value the runtime does not know is logged and ignored.

Because the indicators are the CARLA vehicle's own lights, a scenario checks them with TurnIndicatorCondition -- the policy signals left before it turns, say -- and sets another vehicle's with TurnIndicatorAction (a car ahead signalling that it pulls out). An action aimed at the policy's ego holds only until the policy's next command, as one aimed at an Autoware (ego.entity=autoware) or ROS 2 (ego.entity=ros2) ego holds only until the stack commands other indicators.

from autoware_carla_scenario import (
    EntityRole,
    TurnIndicatorAction,
    TurnIndicatorCondition,
    TurnIndicatorState,
)

signals_left = TurnIndicatorCondition(
    EntityRole.ego(), TurnIndicatorState.LEFT, label="ego_signals_left"
)
npc_signals_right = TurnIndicatorAction(EntityRole.npc(1), TurnIndicatorState.RIGHT)

Perception faults

A perception stack gets road users wrong: it calls a pedestrian a car, it draws the box too small, it misses one. PerceptionFaultInjectionAction has the policy told so of one road user, from the tick the action fires on, while every other road user is reported as it is:

from autoware_carla_scenario import (
    EGO_ROLE_NAME,
    EntityDistanceCondition,
    PerceivedCategory,
    PerceptionFaultInjectionAction,
)

# The pedestrian is reported as a car of a car's size once the ego is 30 m off.
self.register_pre_tick(
    PerceptionFaultInjectionAction(
        "pedestrian1",
        PerceivedCategory.CAR,
        length_m=4.5,
        width_m=1.8,
        condition=EntityDistanceCondition(
            EGO_ROLE_NAME, "pedestrian1", 30.0, label="ego_near_pedestrian"
        ),
    )
)
Argument Reports Default: as it is
category the category: a T4Scene ObjectCategory (CAR, PEDESTRIAN, BICYCLE, ...) PerceivedCategory.ACTUAL
length_m, width_m, height_m the box's size, each on its own, m 0
visible False: nothing at all of it True

Where the policy is told of the road users, each takes the fault:

  • renderer_data's actors. A road user not seen is left out; the box's size is the fault's, about the same centre; the category is reported in type_id, under the prefix a policy tells road users apart by: walker.perception_fault.pedestrian, vehicle.perception_fault.car, and static.prop.perception_fault.<category> for UNKNOWN, ANIMAL, HAZARD, OVER_DRIVABLE and UNDER_DRIVABLE.
  • The live T4Scene's boxes (driver.t4_scene). The category is the box's label, and each row keeps what was reported on it, as a perception stack's history would: the rows before the fault show the road user as it was.

It stays so until another PerceptionFaultInjectionAction sets that road user again -- one with every argument at its default reports it as it is -- or the run ends; each run starts with no fault. The ego cannot be faulted (a ValueError): it is not reported to its own policy.

Nothing else changes. The world is not touched, and the trajectory log, the T4 dataset, the metrics and the conditions keep CARLA's ground truth: a collision with a road user the policy was told is not there is a collision.

An ego driven by a stack that perceives the road users itself -- Autoware (ego.entity=autoware), a ROS 2 stack, a HILS ECU -- is not told of them by the framework, so there is no report to alter. The action raises NotImplementedError as it fires: it is not implemented yet for those egos.

Coordinate frames

Two conversions sit between CARLA and the contract, both handled in autoware_carla_scenario.driver.observation:

  • Handedness. CARLA's world is left-handed (x=East, y=South, z=Up, yaw clockwise); the contract is right-handed (x forward, y left, z up). Positions flip y, yaw is negated, and angular velocity — a pseudovector — picks up an extra sign flip.
  • Rig origin. alpasim puts the rig origin at the rear-axle centre projected to the ground, while a CARLA actor's origin is at the vehicle centre. The offset is derived from the vehicle's wheels, else its bounding box, and logged at session start. Sensor mounts (driver.cameras, driver.lidars) are relative to the actor origin; the cameras' rig_to_camera and the LiDAR sweeps are moved by the offset into the rig.

Rear-axle offset on CARLA 0.9.x

CARLA 0.9.x reports wheel positions in world coordinates and centimetres, so the derived offset can land far outside a plausible range. When that happens the value falls back to half the bounding box's half-length (-0.5 * extent.x, a quarter of the box's length) behind the origin and a warning is logged. Set driver.rear_axle_offset_m (e.g. -1.4) to pin it explicitly.

Observations are submitted with the ego pose in the local frame and velocities in the rig frame; the route is submitted in the rig frame. The plan comes back in the local frame and is converted to the rig frame before the controller tracks it.

The route starts level with the rig: its first waypoint is the rear axle's projection onto the ego's lane, at rig x of about zero, not the actor origin's, which is rear_axle_offset_m further ahead. From there the waypoints follow the lane route_resolution_m apart, for route_horizon_m; where a lane splits between the rig and the actor origin, the route keeps to the lane the actor is on. At the very start of the road network, where no lane leads in, it starts at the actor origin's waypoint instead, so a policy should project the rig onto the route rather than assume it sits on the first point.

Early termination

A policy can set terminate_session on its response. The entity records it, and ScenarioRunner ends the tick loop — but only after evaluating the scenario's own pass and fail conditions, so a condition firing on the same tick still decides the outcome. On its own, an early stop is reported as a failure with the message Ego entity requested session termination: The driver policy requested session termination, because the scenario never satisfied its pass condition.

Writing a policy

A policy subclasses BaseDriver and returns a plan in the rig frame (x forward, y left, origin on the ground below the rear axle); the servicer handles sessions, frame retention, ego history and the rig/local conversion:

from carla_driver_interface.driver import BaseDriver, DriveContext, DriveResult
from carla_driver_interface.server import run_server


class MyPolicy(BaseDriver):
    name = "my_policy"

    def drive(self, ctx: DriveContext) -> DriveResult:
        ...


run_server(MyPolicy(), port=50051)

Without CARLA, carla_driver_interface.testing.FakeLoop drives a policy server over real gRPC on a straight road, rendering each declared pinhole camera, which is what a policy's CI runs (carla-driver-interface demo --driver localhost:50051 from the command line). The scenario framework is then the CARLA-backed runtime for the same server.

Protobuf definitions

The alpasim egodriver contract and its CARLA extension payloads (carla_driver.v0) are not compiled here: carla-driver-interface vendors their .proto files, generates the Python modules and checks them, and the framework imports them from carla_driver_interface.protocol. Its own documentation gives their provenance.

autoware_carla_scenario/scripts/compile_protos.py compiles only the contracts this repository owns or vendors itself -- AutowareBridge, tier4/e2e-devkit's T4Scene and PedestrianMotion -- and test_proto_generated.py fails if their committed modules drift from the .proto files (compile_protos.py --check runs the same check). See autoware_carla_scenario/proto/README.md.

Limitations

  • Only RGB cameras and ray-cast LiDARs are streamed.
  • CARLA has no recorded drive, so with send_ground_truth: true the reference sent through submit_recording_ground_truth is the route itself: its waypoints, headed along the path and spaced policy_timestep_s apart. This is the recorded trajectory channel, unrelated to the CARLA ground truth in renderer_data.
  • The route is a lane-following walk of CARLA's road graph. At a fork it follows the scenario's route when there is one (a RoutingAction, or CarlaDriverEntity.route_to), so the turn the scenario asserts is the one the policy is shown; without a route, or where no lane route reaches the goal, it takes the first continuation. It is a rolling window, re-sent on every policy step so the horizon stays ahead of the vehicle, not the whole route to the goal.
  • Actor reporting covers vehicles, pedestrians and props (vehicle.*, walker.*, static.prop.*) within actor_horizon_m of the ego, the ego itself excluded. Walker AI controllers, sensors, the spectator, traffic lights and signs are not reported, and neither is an actor parked under the map (hidden): one whose CARLA z is below the hidden threshold every other consumer of the world shares is excluded explicitly, whatever actor_horizon_m is. A vehicle's and a prop's velocity is the one CARLA reports, a prop's zero unless its physics has it moving. A walker's is measured from where it stood at the previous policy step, over the simulated time since (its average velocity over the step), as the trajectory log measures it: CARLA reports 0 for a walker walking on its controller. At the first step, and right after a walker is teleported (brought in from under the map, put where its walk starts), there is nothing to measure from and it is the one CARLA reports.
  • Actor dynamic_state carries linear velocity; angular velocity and acceleration are left zero, matching the reference runtime.