Skip to content

Software Architecture

This document describes the software architecture of the autoware-carla-scenario package, with a focus on the scenario execution system and the result viewer.

System Overview

The package consists of four major subsystems:

graph TB
    subgraph CLI["CLI Entry Points"]
        scenario["scenario command<br/>(Hydra)"]
        viewer_cmd["viewer command<br/>(FastAPI + Uvicorn)"]
        detect["detect-no-3d-model<br/>(argparse)"]
    end

    subgraph Execution["Scenario Execution System"]
        queue["ScenarioQueue"]
        runner["ScenarioRunner"]
        base["BaseScenario"]
        server["CarlaServerManager"]
    end

    subgraph Evaluation["Condition & Action System"]
        conditions["Conditions"]
        actions["Actions"]
    end

    subgraph Viewer["Result Viewer"]
        app["FastAPI App"]
        scan["Scanner"]
        ui_runner["Runner (subprocess)"]
        sweep_res["Sweep Resolver"]
    end

    subgraph Support["Support Modules"]
        coord["Coordinate Transforms"]
        entity["Entity Management"]
        kin["Kinematics"]
        sweeper["Lanelet Constraint Sweeper"]
        recorder["Camera Recorder"]
    end

    scenario --> queue
    viewer_cmd --> app
    queue --> runner
    queue --> server
    runner --> base
    runner --> conditions
    runner --> actions
    runner --> recorder
    base --> entity
    base --> coord
    app --> scan
    app --> ui_runner
    app --> sweep_res
    ui_runner -->|"uv run scenario<br/>(subprocess)"| scenario

Scenario Execution System

The scenario execution system is responsible for the full lifecycle of a CARLA scenario test: server management, map loading, actor spawning, tick-loop execution, condition evaluation, recording, and cleanup.

Architecture Diagram

graph TB
    subgraph Entry["Entry Layer"]
        main["main()<br/>examples/run.py"]
        hydra["Hydra Config<br/>Compose API"]
    end

    subgraph Orchestration["Orchestration Layer"]
        SQ["ScenarioQueue"]
        SR["ScenarioRunner"]
        CSM["CarlaServerManager"]
    end

    subgraph Scenario["Scenario Layer"]
        BS["BaseScenario (ABC)"]
        IP["IntersectionPassingScenario"]
        LC["LaneChangeScenario"]
        TLC["TrafficLightComplianceScenario"]
        TS["TemporaryStopScenario"]
    end

    subgraph Traffic["Traffic Backend"]
        tb["TrafficBackend<br/>(traffic_manager | none | third-party)"]
    end

    subgraph CARLA["CARLA Simulator"]
        world["carla.World"]
        tm["TrafficManager"]
        rec["Native Recorder"]
    end

    main --> hydra
    hydra --> SQ
    SQ --> CSM
    SQ --> SR
    SR --> BS
    BS --> IP & LC & TLC & TS
    SR --> world
    SR --> tb
    tb --> tm
    SR --> rec

Component Responsibilities

Component File Responsibility
CarlaServerManager server.py Start/stop/reuse the CARLA UE5 process. Manages lifecycle via process groups.
ScenarioQueue scenario_queue.py Batch execution of multiple scenarios. Owns server and runner lifecycle. Context manager.
ScenarioRunner scenario_runner.py Single-scenario execution: sync mode, tick loop, condition evaluation, recording, cleanup.
BaseScenario scenario_base.py Abstract base. Subclasses implement setup() and is_done(). Registers conditions, actions, entities.
TrafficBackend traffic/ Owns the vehicles the scenario did not author and answers their manoeuvre intents. traffic_manager by default; see Traffic Backends.

Execution Lifecycle

The following sequence diagram shows the complete lifecycle of a single scenario execution:

sequenceDiagram
    participant CLI as CLI (main)
    participant SQ as ScenarioQueue
    participant CSM as CarlaServerManager
    participant SR as ScenarioRunner
    participant BS as BaseScenario
    participant EGO as Ego entity
    participant W as carla.World
    participant TB as TrafficBackend
    participant TM as TrafficManager

    CLI->>SQ: add(scenario)
    CLI->>SQ: __enter__() / start()
    SQ->>CSM: start()
    Note over CSM: Reuse existing server<br/>or launch new process
    SQ->>SR: new ScenarioRunner(server, ...)
    SQ->>SR: load_map_by_name(), or<br/>load_map_by_overwriting_xodr()<br/>when map.overwrite_xodr is on
    SQ->>SR: Initialize MapManager<br/>(Lanelet2 + OpenDRIVE)

    CLI->>SQ: run_all()
    loop For each scenario (each attempt, up to 1 + cooldown_max_retries)
        Note over SQ: A dead server the queue owns is restarted first
        opt The world is dirty (a run, or a failed attempt, left it)
            SQ->>SR: reset_world(strict=True) (map reload, or soft reset)
            Note over SQ: A reset that fails is this attempt's failure:<br/>retried (reset again), or the case errors
        end
        SQ->>SR: run_scenario(scenario, reset_world=False)

        rect rgb(240, 248, 255)
            Note over SR: Phase 1: Setup
            SR->>W: Reload the runs' map if another client loaded a world
            SR->>W: Destroy leftover dynamic actors
            SR->>W: apply_settings(sync=True, dt=0.05)
            SR->>TB: prepare(TrafficContext)
            TB->>TM: set_synchronous_mode(True)
            TB->>TM: set_random_device_seed(seed)
            SR->>BS: set_client(client, tm_port)
            SR->>BS: set_traffic_backend(backend)
            SR->>BS: setup()
            Note over BS: Spawn NPCs, register<br/>conditions & actions
            SR->>W: Spawn ego vehicle
            SR->>BS: register_fail_condition(EntityExistence)
            SR->>W: start_recorder(path)
        end

        rect rgb(255, 248, 240)
            Note over SR: Phase 2: Warm-up and start-up
            loop STABILIZE_TICKS times (5 by default)
                SR->>W: hold the vehicles still, tick()
            end
            SR->>BS: run_init (each init action once, at elapsed=0)
            SR->>EGO: set_initial_speeds(...)
            SR->>EGO: on_scenario_start(world)
            SR->>EGO: _wait_for_ego (until the ego is ready)
            Note over SR: An ego that asks to terminate here<br/>ends the run (no tick loop)
            SR->>W: Release the vehicles' brakes
            SR->>TB: start(world, skip_actor_ids)
            TB->>W: set_autopilot(True) on the vehicles it owns
            SR->>BS: set_initial_speed(ego actor)
            SR->>BS: register_fail_condition(Timeout) (unless registered)
        end

        rect rgb(240, 255, 240)
            Note over SR: Phase 3: Tick Loop
            loop While not is_done(), until a result
                SR->>BS: pre_tick_actions.tick(world, elapsed)
                SR->>BS: pre_tick_callbacks(world)
                SR->>W: tick()
                SR->>EGO: on_tick(world, elapsed)
                SR->>TB: tick(world, elapsed)
                SR->>BS: post_tick_actions.tick(world, elapsed)
                SR->>BS: post_tick_callbacks(world)
                SR->>BS: Check pass_conditions
                Note over SR: If any pass condition<br/>returns ScenarioResult → PASS
                SR->>BS: Check fail_conditions (not MetricConditions)
                Note over SR: If any fail condition<br/>returns ScenarioResult → FAIL
                SR->>EGO: termination_requested?
                Note over SR: If True → not a pass (termination)
            end
            Note over SR: is_done() with no result → PASS
            SR->>BS: Judge MetricConditions on the recorded frames
        end

        rect rgb(255, 240, 240)
            Note over SR: Phase 4: Cleanup
            SR->>EGO: on_scenario_end(world)
            SR->>BS: close_conditions()
            SR->>W: Destroy ego
            SR->>W: stop_recorder()
            SR->>TB: close()
            TB->>TM: shut_down()
            SR->>SR: Write coverage, gate record and result JSON
        end

        SR-->>SQ: ScenarioResult
        Note over SQ: The world stays dirty until the next attempt
    end

    CLI->>SQ: __exit__() / stop()
    opt Server not stopped by the queue (given, or reused) and world dirty
        SQ->>SR: restore_world() (best-effort reset, settings restored)
    end
    SQ->>CSM: stop() (if owned)

run_scenario called on its own (reset_world=True, the default) resets the world at its end, best effort: a reset that fails is logged, not raised. The queue instead passes reset_world=False and resets strictly before the attempt that follows (reset_world(strict=True)), so that a world it could not reset fails that attempt -- retried, or erroring the case -- rather than being the world the next case runs on.

Tick Loop Detail

The tick loop runs at a fixed 20 Hz (0.05 s per tick) in CARLA synchronous mode. Each tick follows a strict evaluation order:

┌─────────────────────────────────────────────────────────────────┐
│                        Single Tick Cycle                        │
├─────────────────────────────────────────────────────────────────┤
│  0. is_done() check       → True → exit (PASS if no result)     │
│  1. Pre-tick actions      → action.tick(world, elapsed)         │
│  2. Pre-tick callbacks    → callback(world)                     │
│  3. world.tick()          → advance simulation by 0.05 s        │
│  4. Trajectory log        → record the new frame                │
│  5. Ego entity            → ego.on_tick(world, elapsed)         │
│  6. Traffic backend       → backend.tick(world, elapsed)        │
│  7. Post-tick actions     → action.tick(world, elapsed)         │
│  8. Post-tick callbacks   → callback(world)                     │
│  9. Action states, NPC    → logged (collisions, falls)          │
│     contacts                                                    │
│ 10. Ego progress, metric  → frames kept for MetricConditions    │
│     frames                                                      │
│ 11. Coverage              → sample TICK / condition items       │
│ 12. Periodic logging      → ego OpenDRIVE position (1/s)        │
│ 13. Pass conditions       → first satisfied → PASS & exit       │
│ 14. Fail conditions       → first triggered → FAIL & exit       │
│ 15. Ego termination       → requested → not a pass, & exit      │
└─────────────────────────────────────────────────────────────────┘
After the loop: MetricConditions are judged on the recorded frames.

Evaluation semantics:

  • Pass conditions are checked first. The first condition to return a non-None ScenarioResult terminates the loop with a pass.
  • Fail conditions are checked only if no pass condition was satisfied. The first triggered fail condition terminates the loop with a failure. A MetricCondition is not checked in the loop: it is judged on the recorded frames once the run is over, and can still fail a run the loop passed.
  • An ego termination request (e.g. the driver policy signalled one) is checked after the conditions, so a pass or fail on the same tick still decides the outcome; on its own it is not a pass.
  • If the loop exits via is_done() returning True with no result, the scenario is treated as passed. Every path that reaches the tick loop records one condition_statuses entry per registered condition; an ego that asks to terminate before it (its stack never became ready) records none, since no condition was ever checked.

The initialization phase

register_init registers what runs once, after the warm-up and before the ego is asked to start: run_init performs each action a single time at elapsed=0 and never ticks it again. That suits an action that is over once commanded (a light frozen, a speed handed over, a source's initial vehicles placed). An action that keeps acting after the tick it fired on, or whose run waits on an until the world has to reach, says so with BaseAction.needs_tick_loop -- by default, whether it reissues its command (reissues_while_running) -- and register_init puts it on the pre-tick loop instead, where it fires on the first tick: a trajectory or a stalled object, a rate-limited, until-bounded or reissue=True SetSpeedAction, a lane change waiting to settle, a walk that may turn back, a played or generated pedestrian motion, an adversarial manoeuvre or NADE traffic, a construction zone that appears late or is cleared away, and a RoutingAction that moves its entity itself. Left in the phase, such an action would sit in runningState for the whole run with nothing driving it. A TrafficSourceAction or TrafficSinkAction stays in the phase on purpose: registered there it acts once and stops, as does a construction zone that stands from the start for the whole run, placed before the ego starts.

run_init performs the actions in registration order and settles each one (complete, or under way for one that runs on) right after it fires, which also gives back the control domains it took. Two init actions on the same domain of one entity therefore both run, and the one registered last is what stands. The case that comes up in practice is an ego with a goal and an init Set Goal card for it: the ego's own goal is registered as it is spawned, before the document's cards, so the ego is handed both and keeps the card's (the validator warns about it).

The scenario clock

_ScenarioClock supplies the elapsed every action, condition, entity and backend receives, and the elapsed_seconds on a ScenarioResult. It reads world.get_snapshot().timestamp.elapsed_seconds — simulated time, and nothing else.

The world advances by fixed_delta_seconds per tick and the loop steps it as fast as the slowest client allows (see max_tick_rate_hz), so how much simulated time fits into a second of real time is a property of the host. A scenario whose durations came off the wall clock would fire its triggers in different places on a fast machine and a slow one — the determinism that synchronous mode exists to provide.

That holds for ScenarioRunner.timeout_seconds as much as for a condition an author wrote. It is registered as a default TimeoutCondition on every scenario that registers no timeout of its own, so a scenario given sixty seconds is given sixty of its own however fast the host runs.

A scenario that registers a timeout of its own has said how long it runs, and the runner adds no default to cut it short: a red light whose drawn red phase is 13.4 s, given seven seconds after the green, runs for 20.4 s even when the run's default is twelve. Exactly these fail conditions count as its own:

  • a bare TimeoutCondition;
  • an OrCondition with a child that counts (an OR fires as soon as any child does);
  • a StickyCondition whose child counts (it hands on the child's failure as it comes; it latches only a pass).

A timeout under an AndCondition or a NotCondition does not count — it fires only on some other state of the world, so it does not bound the run — and neither does one under a PersistentCondition, which resets on every failure its child reports and so never fires around a timeout.

The run's default — ScenarioRunner(timeout_seconds=...), run_scenario(timeout_seconds=...), ScenarioQueue.add(timeout_seconds=...), and with them a scenario.timeout_seconds= override — therefore does not apply to a scenario with a timeout of its own. A hand-written scenario that builds its timeout from cfg.timeout_seconds still follows the override, since that is the same key; one that builds it from another (traffic_light_compliance with timeout_after_green_seconds set) does not. A DeclarativeScenario always has one: it registers its document's timeout_seconds (or the Hydra override of it) itself, so a timeout node in the document is a second limit beside it, and whichever fires first fails the run. scenario.timeout_seconds=null, how a declarative scenario falls back to its document's timeout, leaves the run's default at 60 s, as an absent key does; a value that is not a number (a string, a bool, a list) is refused with a ValueError naming the key before the run starts. A sweep keeps the wall-clock sweep.job_timeout_seconds guard over every job either way.

Wall-clock protection against a stuck simulator lives outside the scenario clock, where it belongs:

Guard Where Covers
CARLA client RPC timeout (60 s) ScenarioRunner.__init__ world.tick() stops returning
sweep.job_timeout_seconds (120 s) sweeper config a whole job overrunning

The clock starts after the ego is ready, so an entity that needs an autonomy stack to come up does not spend the scenario's timeout booting. Simulation time does advance during that wait, which is why the start is measured rather than assumed to be zero.

An entity that gives up waiting (its stack never became ready, so it sets termination_requested) ends the run right there, before it begins: a failed result of the driver category ("Ego entity requested session termination: ", at 0 s, with no condition statuses), and no vehicle released, no traffic started, no condition checked. The cleanup and the result JSON go as for any other run, but the run writes no trajectory log and no coverage file, since its clock never started.

run_scenario(reset_world=True) (the default, for a direct call; a queue passes False and resets before its next attempt) resets the world at the end of a run, and also after a run that raises, before the error propagates: best effort, a reset that fails is logged and the run's own error is the one raised. A KeyboardInterrupt (Ctrl-C) propagates at once, with no reset.

ScenarioQueue: Batch Execution and Retry

ScenarioQueue wraps ScenarioRunner to support sequential execution of multiple scenarios:

graph LR
    subgraph ScenarioQueue
        direction TB
        A[Scenario 1] --> B[Cooldown]
        B --> C[Scenario 2]
        C --> D[Cooldown]
        D --> E[Scenario N]
    end

    subgraph Retry["Retry Logic"]
        direction TB
        R1["Attempt 1"] -->|fail| R2["Cooldown, reset_world()"]
        R2 --> R3["Attempt 2"]
        R3 -->|fail| R4["..."]
        R4 --> R5["Attempt N"]
        R5 -->|fail| R6["RuntimeError, or an errored result (continue_on_error)"]
    end

Key features:

  • Cooldown: Configurable wait time (cooldown_seconds) between consecutive scenarios to allow CARLA server cleanup.
  • Retry: A scenario whose run raises is retried up to cooldown_max_retries times before raising an error; one that runs and fails is not. The lanelet sweeper follows the same rule for its forked cases. A retry runs the same scenario object again, and so does a second run_all(): before each setup() the runner resets what an earlier attempt registered (BaseScenario._reset_registrations()) -- its conditions, actions, callbacks, entities, coverage and measures -- back to what the scenario's __init__ registered, and resets each of those (BaseCondition.reset(), BaseAction.reset()) to the state it was built in, so a condition that latched in a failed attempt cannot decide the retry, a once action that fired there fires again, and a closed CollisionCondition attaches a sensor of its own.
  • World reset: The queue owns every reset between runs (#1941). It runs each scenario with run_scenario(reset_world=False) and keeps a flag of whether the world holds what a run left; right before the next attempt -- the next scenario's first, a retry, or the first of a later run_all() (the SLURM worker calls run_all() once per case) -- it resets the world (ScenarioRunner.reset_world(strict=True)), so the weather, frozen traffic lights and light states one run set do not carry into the next. Between scenarios the reset comes first and the cooldown after it, so the server settles after the reload before the next scenario connects; a retry waits first (its recovery wait), then resets. The reset is part of that attempt, and strict: a map load that fails (a timed-out load_world, a server that died in it) is logged and raised rather than leaving the world as the last run left it (#2243), and the next reset loads the map. One that raises fails that attempt, which is retried (and resets again) or, with continue_on_error, errors that scenario, and never changes the verdict of the scenario that ran before it, nor ends the batch. An attempt that failed before its reset (in a retry's wait, or a server restart) leaves the flag as it was. After the last scenario the world is left as it is, for the next run_all() to reset, unless the server is restarted (a new world) in between. When the queue stops (stop(), __exit__, the end of an as_fixture() session) with the world dirty, a server it does not stop -- one it was given, or one its own manager found already running and reused -- would be left synchronous at 20 Hz with nothing ticking it, and with the last run's actors in it, frozen for every other client (#2067). So stop() first restores it (ScenarioRunner.restore_world()): it resets the world once more (reset_world(), which destroys the runs' actors) and applies the world settings the runner's first run found (asynchronous, no fixed time step, for a freshly loaded map; a queue that loads no map puts back whatever its first run found, synchronous settings included). It leaves the TrafficManager alone: each run's traffic backend already shut its own down at the end of the run, and asking the client for one would start a new one, or take over another client's on that port. A server that does not answer a quick ping (CarlaServerManager.is_alive(), a few seconds) is not restored, with a warning: each call would wait for the client's 60 s timeout, or abort the process on arm64, before the launcher is stopped. Each step is best effort: a failure is logged, the next step still runs, and stop() goes on to close the Autoware launcher and the backends without raising. A server the queue stops gets none of this. The single-scenario run (run_scenario_with_queue()), batch mode, the SLURM worker and the sweeper's jobs (which run batch mode) all stop their queue with with queue:, so each restores the world this way. A run_all() that raises (a scenario failing every attempt without continue_on_error, or Ctrl-C) leaves the world as it was too: the next run_all() resets it before its first scenario. With run_all(continue_on_error=True), which batch mode uses, a scenario that fails every attempt is recorded instead as a failed result naming the error (errored_indices() tells it from a scenario that ran and failed), and the queue goes on with the next one, whose first attempt resets the world.
  • Server restart: A CARLA server that dies mid-batch cannot recover by waiting, so before every attempt -- each case's first, and every retry -- a queue that owns its server checks it (CarlaServerManager.is_alive()) and, when it is gone, logs it, stops it and redoes the server part of start(): the server is started again (on the same port and arguments, so a SLURM task keeps its port slot), a new ScenarioRunner connects to it, and the map is loaded and the MapManager built again. The backends and the Autoware launcher are not prepared again: neither talks to the CARLA server. A restart that raises is the failure of the attempt it came before, retried or errored like any other, and the next attempt restarts again. A server the queue was given (ScenarioQueue(server=...)), or one its own manager found already running and reused (CarlaServerManager.reused), is never restarted, and a server that is alive is never touched. The SLURM worker, which calls run_all() once per case on one queue, gets the same restart before each case.
  • Server ownership: Accepts an external CarlaServerManager or creates and owns one internally. A start() (or __enter__) that fails -- the runner cannot connect, the map name is unknown, the Lanelet2 map does not load -- calls stop() before the error propagates, so the server the queue owns is stopped and the Autoware launcher closed even though __exit__ does not run.
  • Server start: CarlaServerManager.start() never takes another process's server for the one it launched (#2233). Before launching on a local host it tries a plain TCP connection to host:port: with reuse_if_running=False a port that accepts one is refused at once ("in use by another process"; stop that process, choose another server.port, or set reuse_if_running=True to use the server there), and with reuse_if_running=True a port that accepts a connection but did not answer the first ping as CARLA (a busy server can miss its 2 s deadline) is pinged again for up to CarlaServerManager.REUSE_PING_S (30 s) and reused once it answers, or refused -- never launched over, since a second server could not bind the port; a port that stops accepting connections during that wait (a server still shutting down, as after a queue's restart stopped it) is launched on. While a launched server starts, its own process is checked before each ping and after the one that answers: once it has exited, start() fails with its exit code and where CARLA's own log is (<launcher directory>/CarlaUnreal/Saved/Logs), even when something answers on the port.
  • Server lifetime: A CARLA that CarlaServerManager launched does not outlive the process that launched it. stop() (or with, or atexit) stops it on a normal exit; a SIGTERM to scenario raises KeyboardInterrupt, so the with blocks unwind as on Ctrl-C, and it exits 143; and CARLA is launched under a small watchdog that leads its process group and, once that process is gone (killed with SIGKILL, say, as the lanelet sweeper does a job past sweep.job_timeout_seconds), sends the group SIGTERM and, 10 s later, SIGKILL; the sweeper then waits until that CARLA has exited before its next attempt. The watchdog polls its parent process rather than using PR_SET_PDEATHSIG, which follows the thread that started it. A server that was already running and was reused is never stopped.
  • pytest integration: as_fixture() generates a session-scoped pytest fixture with automatic skip when the queue fails to start() -- among other reasons, when no server answers at its host and there is none to launch. Whether a CARLA is installed is not asked first: a remote server, or one already running, needs none. It runs the scenarios with run_all(continue_on_error=True), so a scenario that raises on every attempt fails its own tests rather than skipping the session: its result is a failed one naming the error, and a CarlaScenarioFixture for it fails each test that uses it.

Recordings of a run

A run leaves two recordings, neither of which renders anything:

  1. The CARLA recording ({stem}.log): CARLA's native recorder captures every actor's state from the moment the ego exists, warm-up included. CARLA's replayer (client.replay_file()) plays it back on a server running the same map.
  2. The trajectory log ({stem}.trajectory.jsonl, Trajectory Log): written by the runner itself after every tick of the scenario, in-process -- every vehicle and pedestrian, the lights, the map the run was on, the ego's debug markers, each condition's state and the verdict. A Rerun recording made from it with the autoware_carla_scenario_rerun package is how a run is seen (Seeing a run): with rerun.enabled=true the runner hands each closed log to a RerunConverter (rerun.py), which converts it on a background thread while the next case runs, and the run waits for it at the end.

Releases before 15.0.0 also rendered a video from the CARLA recording after each run ({stem}.mp4, a replay with an RGB camera and ffmpeg); that is gone, and so is the ffmpeg it needed.

Debug markers

The ego entity can show what the world does not: EgoVehicle.debug_markers() returns what it is following on the current tick (DebugMarkers, polylines in CARLA world coordinates, from autoware_carla_scenario.debug_markers). The base class shows nothing; the entities that follow a trajectory show it, as a green line with a dot at each point:

Entity What it shows
CarlaDriverEntity The driver policy's current plan
Ros2EgoEntity The trajectory it is tracking, while a trajectory command source drives

After every tick of the scenario the runner asks the ego for its markers and writes them into the trajectory log when they change (a markers record). An entity hands out the same DebugMarkers object until its plan changes (plan_markers() builds one per plan), so the log holds one per plan rather than one per tick. A viewer of the run draws them beside the path the ego drove, which the log's ticks give.

An entity of your own shows more by overriding debug_markers():

from autoware_carla_scenario.debug_markers import DebugMarkers, DebugPolyline

class MyEgo(EgoVehicle):
    def debug_markers(self) -> DebugMarkers:
        markers = DebugMarkers()
        markers.add_polyline(
            DebugPolyline(self._path_points, color=(255, 0, 0), point_size=0.1)
        )
        return markers

Condition and Action System

Condition Architecture

Conditions evaluate scenario pass/fail criteria. All conditions inherit from BaseCondition:

classDiagram
    class BaseCondition {
        <<abstract>>
        +label: str
        +check(world, elapsed) ScenarioResult | None
        +get_details() dict
        +to_summary_dict() dict
        +close()
    }

    class ScenarioResult {
        +passed: bool
        +message: str
        +elapsed_seconds: float
        +condition_statuses: list
        +failure: Failure
    }

    class TimeoutCondition
    class ElapsedTimeCondition
    class CollisionCondition
    class EntityExistenceCondition
    class OffRoadFractionCondition
    class PlaceClearCondition
    class EntityCountCondition
    class TrafficSignalCondition
    class TrafficSignalControllerCondition
    class RouteProgressCondition
    class ActionStateCondition
    class TrajectoryTimeCondition
    class LaneChangeSettledCondition
    class MetricCondition
    class AdversityFeasibleCondition
    class AdversityActiveCondition

    class CompositionCondition {
        <<abstract>>
        +_check(world, elapsed) ScenarioResult | None
    }
    class EntityLanePositionCondition
    class EntityDistanceCondition
    class EntityPositionDistanceCondition
    class EntityAreaDistanceCondition
    class TimeToCollisionCondition
    class TimeHeadwayCondition
    class RelativeSpeedCondition
    class SpeedCondition
    class AccelerationCondition
    class StandstillCondition
    class TemporaryStopCondition
    class TurnIndicatorCondition
    class WaypointCondition
    class EntityLaneOfCondition

    class AndCondition
    class OrCondition
    class NotCondition
    class StickyCondition
    class PersistentCondition
    class AlwaysTrueCondition

    BaseCondition <|-- TimeoutCondition
    BaseCondition <|-- ElapsedTimeCondition
    BaseCondition <|-- CollisionCondition
    BaseCondition <|-- EntityExistenceCondition
    BaseCondition <|-- OffRoadFractionCondition
    BaseCondition <|-- PlaceClearCondition
    BaseCondition <|-- EntityCountCondition
    BaseCondition <|-- TrafficSignalCondition
    BaseCondition <|-- TrafficSignalControllerCondition
    BaseCondition <|-- RouteProgressCondition
    BaseCondition <|-- ActionStateCondition
    BaseCondition <|-- TrajectoryTimeCondition
    BaseCondition <|-- LaneChangeSettledCondition
    BaseCondition <|-- MetricCondition
    BaseCondition <|-- AdversityFeasibleCondition
    BaseCondition <|-- AdversityActiveCondition
    BaseCondition <|-- CompositionCondition
    CompositionCondition <|-- EntityLanePositionCondition
    CompositionCondition <|-- EntityDistanceCondition
    CompositionCondition <|-- EntityPositionDistanceCondition
    CompositionCondition <|-- EntityAreaDistanceCondition
    CompositionCondition <|-- TimeToCollisionCondition
    CompositionCondition <|-- TimeHeadwayCondition
    CompositionCondition <|-- RelativeSpeedCondition
    CompositionCondition <|-- SpeedCondition
    CompositionCondition <|-- AccelerationCondition
    CompositionCondition <|-- StandstillCondition
    CompositionCondition <|-- TemporaryStopCondition
    CompositionCondition <|-- TurnIndicatorCondition
    CompositionCondition <|-- WaypointCondition
    EntityLanePositionCondition <|-- EntityLaneOfCondition
    BaseCondition <|-- AndCondition
    BaseCondition <|-- OrCondition
    BaseCondition <|-- NotCondition
    BaseCondition <|-- StickyCondition
    BaseCondition <|-- PersistentCondition
    BaseCondition <|-- AlwaysTrueCondition

    BaseCondition --> ScenarioResult : returns

Condition categories:

Category Conditions Description
Temporal TimeoutCondition, ElapsedTimeCondition Time-based triggers
Safety CollisionCondition, EntityExistenceCondition, OffRoadFractionCondition, PlaceClearCondition, EntityCountCondition Collision detection — with anything, or with a named entity or class of object — actor alive checks, leaving the driving lanes by more than a tolerance, a place clear of road users, and how many road users of a kind are where
Position EntityLanePositionCondition, EntityLaneOfCondition, WaypointCondition, RouteProgressCondition Road/lane position (a named lane, or the lane of a position), no road ahead within a distance (WaypointCheckType.IS_EMPTY), distance along the route
Relative EntityDistanceCondition, EntityPositionDistanceCondition, EntityAreaDistanceCondition, TimeToCollisionCondition, TimeHeadwayCondition, RelativeSpeedCondition Gap to another entity, to a place on the map, or from an entity's body to an area of the map chosen by its ground (along the road with measure: lane; signed there for ahead of the subject, the way it faces, positive, behind negative), time to collision, following headway and speed difference
Motion SpeedCondition, AccelerationCondition, StandstillCondition, TemporaryStopCondition, TurnIndicatorCondition Speed and acceleration thresholds, standstill detection, stop-and-go, turn indicators
Traffic TrafficSignalCondition, TrafficSignalControllerCondition One light's state, or a whole junction's phase
Actions ActionStateCondition, TrajectoryTimeCondition, LaneChangeSettledCondition Another action's lifecycle state, a trajectory's clock, a lane change settled
Adversity AdversityFeasibleCondition, AdversityActiveCondition Whether an adversity can be carried out here, and whether one is in progress
Metric MetricCondition A metric NG for longer than allowed
Composition AndCondition, OrCondition, NotCondition Logical combinators
Stateful StickyCondition, PersistentCondition Latch once satisfied / persist across ticks (a persistent condition counts only passing ticks, so it never fires around a condition that only reports a failure -- the collision, off-road, existence and timeout ones -- and warns when built around one: Usage)
Utility AlwaysTrueCondition Unconditional trigger (default for actions)

A collision with map geometry is a collision. CARLA gives no actor for the other side of a collision with a wall, a kerb or another part of the map that is not an actor (CollisionEvent.other_actor is None). An untargeted CollisionCondition (no target, target_type ANY) fires on it, and its details say other_actor_type: "not an actor". A targeted one ignores it, as it ignores any collision that is not the one named, and keeps waiting for that one (#277).

Distances carry a coordinate system. EntityDistanceCondition, TimeHeadwayCondition and TimeToCollisionCondition each take a coordinate_system, which is OpenSCENARIO's coordinateSystem attribute:

  • ENTITY (the default, so existing documents keep their meaning) is the straight-line separation in the world frame: on a curve, the chord. The directional measures project it onto the subject's axes: a LONGITUDINAL or LATERAL distance along or across its heading, a headway along its direction of travel. The chord and the projection along the subject equal the along-lane gap on a straight road and under-read on a curve: for a leader 20 m ahead along the lane, the chord (2R·sin(L/2R), a EUCLIDEAN distance or a TTC's range) reads 19.9 / 19.5 / 18.6 m at R = 50 / 25 / 15 m, the projection (R·sin(L/R), a headway or a LONGITUDINAL distance) 19.5 / 17.9 / 14.6 m. Past a half turn (an arc of π·R or more) the projection changes sign, as does a TTC's closing speed, taken along the chord: a headway and a TTC both stop firing for a leader that is ahead along the lane. A LONGITUDINAL distance takes the magnitude and does not.
  • LANE projects both entities onto the OpenDRIVE road they share and compares their s, which is what scenario_simulator_v2 measures. For TTC the closing speed is resolved onto the road too, so a vehicle's cornering is not counted as approach.

LANE follows the chain of connected roads between the two, which it has to: an OpenDRIVE map is cut into short roads — a median of 33 m on the converter's own fixture — so a leader at an ordinary following distance is usually on the next road, not the one behind it. Measuring within one road would answer "no measurement" to most following scenarios.

A junction is measured too. Its connecting roads are ordinary roads carrying junction="<id>", so the chain reaches them like any other, and two entities on one connecting road need no walk at all — which is the measurement an intersection scenario is usually about. What a junction cannot settle is which way out a vehicle will take, so that ambiguity is reported rather than hidden: where several chains reach the target the shortest answers, by distance rather than by which link the map happens to list first, and every measurement carries whether it ran through a junction. A caller that must not have a turn chosen for it passes across_junctions=False and gets no measurement instead.

Nothing falls back to the straight line: silently swapping one measure for another makes a scenario pass for the wrong reason. Bounded at 200 m, which covers a motorway-speed TTC over the seconds a threshold is set at. The measurement itself lives in autoware_carla_scenario.coordinate.lane_distance.

check() contract:

  • Returns ScenarioResult when the condition is triggered (satisfied or violated).
  • Returns None when not yet triggered.
  • __init_subclass__ auto-wraps check() to track the latest result for UI display.

Action Architecture

Actions execute side effects when their trigger condition is met:

classDiagram
    class BaseAction {
        <<abstract>>
        +label: str
        +timing: TickTiming
        +once: bool
        +state: ActionState
        +reissues_while_running: bool
        +execute(world) void
        +tick(world, elapsed) void
    }

    class BaseCondition {
        <<abstract>>
    }

    class TrafficSignalAction
    class TrafficSignalControllerAction
    class EnvironmentAction
    class TurnAction
    class WalkStraightAction
    class PlayMotionAction
    class SmartPedestrianControllerAction
    class LaneChangeAction
    class RoutingAction
    class SetSpeedAction

    BaseAction <|-- TrafficSignalAction
    BaseAction <|-- TrafficSignalControllerAction
    BaseAction <|-- EnvironmentAction
    BaseAction <|-- TurnAction
    BaseAction <|-- WalkStraightAction
    BaseAction <|-- PlayMotionAction
    BaseAction <|-- SmartPedestrianControllerAction
    BaseAction <|-- LaneChangeAction
    BaseAction <|-- RoutingAction
    BaseAction <|-- SetSpeedAction
    BaseAction --> BaseCondition : trigger condition

Action lifecycle:

Each tick the runner calls action.tick(world, elapsed), which walks the action through the OpenSCENARIO storyboard element states held in ActionState (standbyState -> startTransition -> runningState -> endTransition -> completeState).

  1. Standby. The trigger BaseCondition is evaluated via condition.check(world, elapsed). A non-None result fires the action: execute(world) runs and the action enters startTransition.
  2. Running. The trigger cannot fire, so one run cannot begin on top of another. A repeating action (once=False) still checks it every tick and drops the result, so a stateful trigger (PersistentCondition, StickyCondition) sees every tick, as the children of And / Or do. Two independent questions are then asked every tick, in this order:

Does execute repeat? — reissues_while_running. This is a fact about the command, not about the manoeuvre: a TrafficManager target persists until it is changed, while a command that walks a target towards a goal has to be re-sent on every tick to move at all. Because that depends on what drives the entity, it is resolved in the order the knowledge is available:

Source Who knows
reissue= constructor argument whoever knows the backend — the injection point
_reissues_by_default() override the action, from its own configuration or by asking its entity
REISSUES_BY_DEFAULT class attribute the action's class, when it has one answer

When does the run end? — until, a BaseCondition. This is the same kind of object the trigger is written in, so the two ends of a run read together. No until means there was nothing to wait for, and the run is over on the tick after the one it was commanded on. until is reset (BaseCondition.reset()) as each run of a repeating action after its first begins, before execute, so a run never ends on a latch or a hold timed in an earlier run. (The runner resets it before the first run.)

Keeping the two apart is what lets both shapes exist. LaneChangeAction hands work to the simulator and watches for it to land — the traffic backend's lane change (TrafficManager force_lane_change, SUMO changeLane) returns long before the vehicle is in the next lane, and must not be re-sent — so it supplies a LaneChangeSettledCondition as its until and does not repeat. A rate-limited speed change says both. 3. End. endTransition is held for one tick so a condition can watch for it. Then once decides: True (default) means completeState and no further evaluation; False returns the action to standby to be triggered again.

The exception: an instantaneous action. runningState and endTransition are only seen on an action that runs on -- one with an until, or one that reissues its command. An action with neither is over the moment it is commanded: on the tick after startTransition it goes straight to completeState (standbyState with once=False, from which it may fire again on that same tick), never showing runningState or endTransition. Pausing on a transition it passed straight through would only make every instantaneous action a tick slower to repeat.

Action startTransition runningState endTransition
No until, no reissue one tick never never
No until, reissues one tick never (the run is one reissued tick) one tick
until one tick every tick until has not yet held (none if it holds on the tick after firing) one tick
Stopped by an overrides=True action as above as above, until it is stopped never: straight to completeState, even with once=False (stopped later in the tick it fired, its startTransition is not recorded either)

So a condition waiting on runningState or endTransition of an instantaneous action never holds -- wait on completeState (or startTransition) instead. A repeating instantaneous action whose trigger still holds fires again on the very tick it would have gone back to standbyState, so it stays in startTransition tick after tick and standbyState is not seen until its trigger stops holding. An action fired in the init phase never holds endTransition either: it is settled there at once (_settle_start) -- to completeState (standbyState with once=False) if it does not reissue its command and either has no until or its until already holds at elapsed=0. One that reissues, or whose until does not yet hold, is left in runningState, and since the init phase never ticks it again it stays there for good, unless it was put on the tick loop instead: register_init does that for every action whose needs_tick_loop is true. An action can opt out, as the background-traffic source and sink (TrafficSourceAction, TrafficSinkAction) do (a ConstructionZoneAction with a later start or an end time does not): one registered for initialization reissues by default (REISSUES_BY_DEFAULT = True), so it places its vehicles (or clears its lanelets) once and holds runningState for the rest of the run, even with an until that already holds.

until, reissues_while_running and once are all independent — until says when this run ends, reissues_while_running says whether the command repeats while it does, and once says whether another run may begin.

The adversarial actions (TeraSim's adversities, one entity each), NADE (AdversarialTrafficAction, which picks which agent carries out which adversity, with its importance-sampling weight) and the scheduled stalled objects build on this lifecycle: see Adversarial Actions and NADE.

ActionStateCondition observes these states, which is what lets one actor react to another finishing a manoeuvre rather than to the command having gone out (done only reports the latter).

Control domains:

An action that commands an entity takes control of one or more of its control domains, OpenSCENARIO's four (ControlDomain, in autoware_carla_scenario.control_domain): longitudinal (how fast it goes), lateral (where across it goes), lighting and animation. Longitudinal and lateral together are its movement (MOVEMENT).

Action Domains of its entity
SetSpeedAction longitudinal
LaneChangeAction, TurnAction lateral
TurnIndicatorAction lighting
EmergencyAction lighting
WalkStraightAction longitudinal, lateral
RoutingAction, follow / position longitudinal, lateral
RoutingAction, driver lateral; longitudinal too with a speed, a timed waypoint, stop_at_goal, or for a pedestrian (which it walks)
The adversarial actions longitudinal, lateral
AdversarialTrafficAction (NADE) longitudinal, lateral of each road user it drives, from when its path starts until it is handed back
PlayMotionAction animation; longitudinal and lateral too with upper_body_only=False
SmartPedestrianControllerAction longitudinal, lateral and animation

The domains are taken as the action fires, before execute runs. An action that runs on (it has an until, or reissues its command) holds them until its run ends (endTransition, or close()); one that is over the moment it is commanded holds them for that tick alone. An action that fires to command a domain another action holds raises ControlConflictError, naming both, and commands nothing: the run ends in error. So two speed commands to one vehicle in the same tick, or a SetSpeedAction while a WalkStraightAction that turns back is still watching, are errors, while a LaneChangeAction beside a running SetSpeedAction is not.

Whether that is an error is the newer action's to say, one action at a time. An action built with overrides=True (in the Scenario Editor, its card's Override a running action) takes the domain over instead, as OpenSCENARIO has every action do (Actions at runtime): the action holding it is stopped -- it goes to completeState, OpenSCENARIO's stop transition -- and gives back every domain it held, and any entity it took from its driver: a follow or position RoutingAction brakes its vehicle (or stands its walker still) and hands the vehicle back to its traffic backend, an adversity hands its road user back, a routed walk is stopped. NADE is stopped only on the road user taken: that one is handed back and NADE runs on; it never picks a road user another action holds. A follow or position run that its until condition ends before the end of its trajectory gives its entity back the same way. The default is False: actions fighting over an entity say two things at once, and which one CARLA would end up doing is an accident of the tick order.

A post-tick action and the pre-tick action of the next loop are handed the same time and act on the same simulation step, so they share a tick. The holds are process-wide, like the entity registry, and the runner forgets them as each scenario is set up.

Tick timing:

Actions can be registered as pre-tick (before world.tick()) or post-tick (after world.tick()) via BaseScenario.register_pre_tick() / register_post_tick().

Traffic signals: one light, or a junction's cycle

Two pairs of primitives, answering two different questions.

TrafficSignalAction / TrafficSignalCondition set and read one light. That is enough to arrange a specific state and no more: a junction assembled a light at a time can be left with two conflicting approaches green, a state no real road reaches, and an amber has no name at all — it is just a colour one light happens to be showing.

TrafficSignalControllerAction / TrafficSignalControllerCondition work in phases. A phase names every signal its controller drives and how long it holds, so showing one puts the whole junction into a known state at once, and waiting for one is a statement about the junction rather than about a colour.

The one-light pair and the controllers find a Lanelet2 light's CARLA lights alike, coordinate.find_traffic_lights_for_lanelet2_id: through the OpenDRIVE controller named after it, or, on a map where none is (a roadgen map), by the CARLA light whose stop waypoint lies on its stop line. A light an action sets is therefore the light a condition reads and a controller's phases drive.

A junction is declared in two halves, split along what changes between scenarios and what does not.

map.signal_groups is the road. Which signals change as one, and which movements cross each other, are facts about the junction: every scenario that runs here inherits the same ones. Naming a group is also the only way a phase table stays writable — one approach of three lanes is three regulatory elements, and nishishinjuku declares 164 of them.

map.traffic_signal_controllers is this scenario's timing. A test of an ego creeping into a long amber and a test of the same crossing on a short cycle are two scenarios on one road, so the durations are the scenario's to choose. OpenSCENARIO reads the same way: RoadNetwork/TrafficSignals holds the controllers, and RoadNetwork is a section of the scenario file, beside the LogicFile that names the map rather than inside it — scenario_simulator_v2 builds its TrafficSignalController from that scenario XML too.

map:
  signal_groups:                        # the road: no colours, no durations
    - name: ns
      lanelet2_regulatory_element_ids: [1001, 1003]
      conflicts_with: [ew]              # read symmetrically
    - name: ew
      lanelet2_regulatory_element_ids: [1002]

  traffic_signal_controllers:           # this scenario's timing
    - name: crossing
      phases:
        - name: ns_green
          duration_seconds: 20.0
          states:
            - {group: ns, state: green}
            - {group: ew, state: red}
        - name: ns_amber
          duration_seconds: 3.0
          states:
            - {group: ns, state: yellow}
            - {group: ew, state: red}
        - name: all_red
          duration_seconds: 1.0
          states:
            - {group: ns, state: red}
            - {group: ew, state: red}
    - name: next_junction
      reference: crossing      # a green wave along a corridor is declared,
      delay_seconds: 8.0       # not timed by a chain of actions
      phases: [...]

A state may name a lanelet2_regulatory_element_id instead of a group, which is the escape hatch for a single light — including the case below, where the junction's impossible state is the thing under test.

autoware_carla_scenario.signals turns those declarations into running SignalController objects at scenario setup. Each is registered as a pre-tick callback and cycles on its own: a phase holds for its duration_seconds and then hands over to the next, looping. No runner change was needed — durations are differences, so the controller reads the world's own simulated elapsed seconds directly.

The action jumps to a named phase and the cycle carries on from there, so a junction forced green does not thereby stay green; holding it is a separate decision, made by declaring a phase long enough to hold. The condition reads the phase off the controller, not off the lights: the controller is what decided them, so a junction that some other actor happened to set to the same colours does not read as that phase.

A reference offsets one controller's start from another's, which is how a progressive system — a green wave — is written. The offsets are declared; nothing has to time them.

Groups are what make the original problem checkable. Two conflicting approaches green is the state a junction assembled one light at a time can reach and a real road cannot; while a phase was a list of regulatory element ids, nothing could tell. Now conflicts_with says which movements cross, so the validator rejects any phase that shows green or amber to both — amber included, because a junction showing amber to one movement still has vehicles in it, which is why a real one puts an all-red clearance between them. A scenario whose subject is a junction in that state names the regulatory elements directly and is left alone: naming a group is how an author invokes the map's statement about that movement, this rule included.

The validator also rejects a delay with no reference, a reference to a controller the document does not declare, a loop of them, duplicate controller, phase or group names, a signal in two groups, a group that drives nothing, a conflict naming an undeclared group, a phase state naming both a group and an element or neither, an unknown state, and any action or condition naming a controller or phase that does not exist — all while the document is being edited, because a misspelt phase name is invisible at runtime: the junction cycles normally and the scenario just waits forever.

Signal group, controller and phase names are read without surrounding whitespace, wherever they are written: a group's name and conflicts_with, a phase state's group, a controller's name and reference, a phase's name, and an action's or condition's controller and signal_phase. A hand-written "ns " is the group ns, so a name the validator accepts is the name the running controller is found by.


Result Viewer

The result viewer is a web application for browsing, inspecting, and triggering scenario test results.

Architecture Diagram

graph TB
    subgraph Browser["Browser"]
        HTML["HTML Pages<br/>(Jinja2 templates)"]
        SSE["SSE Client<br/>(EventSource)"]
        JS["JavaScript<br/>(htmx + vanilla)"]
    end

    subgraph FastAPI["FastAPI Application (app.py)"]
        direction TB
        pages["Page Routes<br/>GET /, /session/..., /scenario/..."]
        api["API Routes<br/>POST /api/run, /api/refresh<br/>GET /api/scenarios, /api/run/progress"]
        static["Static Files<br/>CSS, JavaScript"]
    end

    subgraph Services["Service Layer"]
        scanner_svc["Scanner<br/>(scanner.py)"]
        runner_svc["Runner<br/>(runner.py)"]
        sweep_svc["Sweep Resolver<br/>(sweep_resolver.py)"]
    end

    subgraph Storage["File System"]
        outputs["outputs/<br/>YYYY-MM-DD/HH-MM-SS/"]
        multirun["multirun/<br/>YYYY-MM-DD/HH-MM-SS/{0,1,2...}/"]
        conf["conf/scenario/<br/>*.yaml"]
    end

    subgraph External["External Process (uv run scenario)"]
        subprocess["subprocess.run()"]
        ext_main["main()<br/>(examples/run.py)"]
        ext_queue["ScenarioQueue"]
        ext_runner["ScenarioRunner"]
        ext_carla["CARLA Simulator"]
    end

    HTML --> pages
    JS --> api
    SSE --> api
    pages --> scanner_svc
    api --> scanner_svc
    api --> runner_svc
    api --> sweep_svc
    scanner_svc --> outputs
    scanner_svc --> multirun
    scanner_svc --> conf
    runner_svc --> subprocess
    subprocess --> ext_main
    ext_main --> ext_queue
    ext_queue --> ext_runner
    ext_runner --> ext_carla
    sweep_svc --> conf
    ext_runner -->|"*_result.json<br/>*.trajectory.jsonl"| outputs
    ext_runner -->|"*_result.json<br/>*.trajectory.jsonl"| multirun

Component Responsibilities

Component File Responsibility
app.py ui/app.py FastAPI routes, template rendering, SSE streaming
scanner.py ui/scanner.py Scan outputs/ and multirun/ directories for result JSON files. Build session lists and condition trees.
runner.py ui/runner.py Execute uv run scenario as subprocesses in a background thread. Track progress with thread-safe global state.
sweep_resolver.py ui/sweep_resolver.py Expand a scenario's sweep into concrete per-job override lists without launching CARLA, through the CLI's sweeper.expand.expand_config (constraints, route searches, ODD draws); a failure, or a scenario whose sweep expands to no case, is reported to the Run panel and nothing runs. The jobs run with the Run panel's extra overrides.
models.py ui/models.py Pydantic data models for the viewer UI.
cases.py ui/cases.py Which files are each case's: the stem its result, trajectory log, coverage and ODD archive share (a SLURM batch's manifest, or matched against batch_results.json). Its session_dir is the containment check every route and scanner function builds a session's directory with (resolved, it must stay under multirun/ or outputs/; else 404).
results.py ui/results.py A session's cases with their ODD case, coverage and metrics off the trajectory log; the cases in a cell of openodd's coverage view; sessions compared, cases matched by what they ran.
slurm_launch.py ui/slurm_launch.py Submit a batch through SLURM (launcher=slurm --no-wait) and follow it from its directory and squeue; resubmit, collect, cancel with scenario-slurm.
slurm_options.py ui/slurm_options.py The launcher.* keys the run panel sets: what each may hold, the Hydra overrides they become (quoted as Hydra reads them back) and the scenario ... --no-wait command line. Checked by the library check (docs/typecheck.md).
recordings.py ui/recordings.py A case's Rerun recording, made from its trajectory log on demand in a worker thread (the optional autoware_carla_scenario_rerun), and where the page loads Rerun's web viewer from (a copy it serves, or the CDN).

Data Models

classDiagram
    class SessionSummary {
        +date: str
        +time: str
        +session_type: str
        +scenario_name: str
        +passed_count: int
        +total_count: int
        +pending_count: int
    }

    class SessionItem {
        +index: int
        +scenario_name: str
        +passed: bool | None
        +elapsed_seconds: float | None
        +overrides: list[str]
        +message: str
    }

    class ScenarioResultView {
        +passed: bool | None
        +message: str
        +elapsed_seconds: float | None
        +condition_statuses: list[ConditionNode]
        +overrides: list[str]
        +raw_log: str
        +trajectory_path: str | None
    }

    class ConditionNode {
        +label: str
        +satisfied: bool
        +message: str
        +condition_type: str
        +role: str
        +details: dict
        +children: list[ConditionNode]
    }

    class RunProgress {
        +current: int
        +total: int
        +scenario_name: str
        +status: Status
    }

    SessionSummary "1" --> "*" SessionItem : contains
    SessionItem "1" --> "1" ScenarioResultView : details
    ScenarioResultView "1" --> "*" ConditionNode : condition tree
    ConditionNode "1" --> "*" ConditionNode : children (recursive)

Result Directory Structure

The scanner discovers results from two directory layouts:

base_path/
├── outputs/                          # Single and batch runs
│   └── YYYY-MM-DD/
│       └── HH-MM-SS/                 # HH-MM-SS-1, -2, ...: a later run in the same second
│           ├── ScenarioName_result.json
│           ├── ScenarioName_coverage.json  # Coverage hits (docs/coverage.md)
│           ├── ScenarioName.trajectory.jsonl  # Actor motion (docs/trajectory_log.md)
│           ├── ScenarioName.log        # CARLA native recording
│           ├── batch_results.json      # Batch summary (batch mode only)
│           └── .hydra/
│               └── overrides.yaml      # Hydra CLI overrides
│
└── multirun/                         # Sweep / multi-scenario runs
    └── YYYY-MM-DD/
        └── HH-MM-SS/
            ├── 0/                    # Job index 0
            │   ├── ScenarioName_result.json
            │   ├── ScenarioName.trajectory.jsonl
            │   ├── raw_output.log    # Subprocess stdout/stderr (a viewer run's)
            │   └── .hydra/
            │       └── overrides.yaml
            ├── 1/                    # Job index 1
            │   └── ...
            └── N/

Session types:

Type Source Identification
multirun multirun/ directory Numbered subdirectories (0, 1, 2, ...)
batch outputs/ directory Contains batch_results.json
single outputs/ directory Contains *_result.json (no batch file)

Scenario Execution from Viewer

The viewer runs scenarios via subprocess to isolate the CARLA Python API (which can cause SIGSEGV) from the web server process:

sequenceDiagram
    participant B as Browser
    participant A as FastAPI
    participant R as Runner (thread)
    participant S as Subprocess (uv run scenario)
    participant SQ as ScenarioQueue
    participant SR as ScenarioRunner
    participant C as CARLA Simulator

    B->>A: POST /api/run {scenario, extra_overrides, sweeper, launcher, slurm}
    opt The body is not JSON, not a JSON object, names no scenario, or a field has the wrong type
        A-->>B: 400 {"status": "error", "message": ...}
    end

    alt With sweeper
        A->>A: sweep_resolver.resolve_sweep()
        Note over A: expand_config, as the CLI: constraints,<br/>route searches, ODD draws
        opt A scenario fails to expand
            A-->>B: 400 {"status": "error", "message": ...}
        end
    else Without sweeper
        A->>A: Expand glob to scenario names
    end
    opt The pattern is refused or matches no scenario
        A-->>B: 400 {"status": "error", "message": ...}
    end

    A->>R: start_run(overrides_list, extra_overrides)
    Note over R: Marks the run as running and resets<br/>its progress under one lock
    alt Another run in progress
        A-->>B: 409 {"status": "busy"}
    else Started
        A-->>B: {"status": "started"}
    end

    B->>A: GET /api/run/progress (SSE)

    loop For each job
        R->>R: _set_progress(running)
        R->>S: Popen(["uv", "run", "scenario", ...overrides]), output to a file

        rect rgb(240, 248, 255)
            Note over S,C: Isolated process — same execution<br/>pipeline as the scenario CLI command
            S->>SQ: main() → build_scenario() → queue.add()
            SQ->>SR: run_scenario(scenario)
            SR->>C: Setup, tick loop, recording, cleanup
            C-->>SR: ScenarioResult
            SR->>SR: Write *_result.json + *.trajectory.jsonl
        end

        S-->>R: exit code (stdout/stderr written to raw_output.log as it ran)
        R->>R: _set_progress(passed/failed)
        A-->>B: SSE event {current, total, status}
    end

    R->>R: _set_progress(done)
    A-->>B: SSE event {status: "done"}

Key design decisions:

  • Process isolation: Each scenario runs in a separate uv run scenario subprocess. This prevents CARLA's C++ extension from crashing the web server.
  • Thread-safe progress: A global RunProgress object protected by threading.Lock provides progress state. start_run() checks for a run in progress and claims the run under that lock, so two runs never start at once and the stream never replays the previous run's done.
  • Job output on disk: a job's stdout and stderr go to a file as it runs, never held in memory: raw_output.log in its job directory when the run is grouped as a multirun (the scenario page shows its last 256 KB), a temporary file otherwise -- an ungrouped job's directory is Hydra's, not known to the runner -- removed once the job is over. A job directory left empty (a job stopped or not launched before Hydra wrote to it) is removed too, so it is not listed as a case. A failed job's warning logs the last 4 KB.
  • Checked run bodies: /api/run and /api/run/preview read their body through one parser (app.parse_run_body): it must be JSON (a body that does not parse, cut off or garbled, is refused rather than read as no body), a JSON object, scenario a string, extra_overrides a list of strings (null or absent: none), sweeper a string or null, launcher local, slurm or null, and slurm an object of text or number options. /api/run also needs scenario: an empty body, or one without it, would otherwise run */*. Anything else is answered with 400 before a run starts or a batch is submitted, so a string of overrides is never run one character per argument (#1500). So is a request the body is read from but that cannot run: a scenario pattern that is refused or matches nothing, a sweep that fails to expand, or a SLURM option value that is not valid (/api/run/preview too), each with the same JSON body as before; a SLURM submission is refused with 503 where SLURM is not available and 409 while another is still being made (#1964). The SLURM resubmit action refuses a body that is not JSON or not an object, or a failed that is not a boolean, the same way (an empty one resubmits with no flags).
  • SSE streaming: The /api/run/progress endpoint uses Server-Sent Events for real-time progress updates (polled at 0.5 s intervals).
  • Multirun grouping: When running multiple scenarios, jobs share a timestamped multirun/ directory with numbered subdirectories. The directory is the run's alone: one started in the same second as another run or a CLI sweep gets HH-MM-SS-1, then HH-MM-SS-2, ...

Condition Tree Display

The viewer reconstructs the condition hierarchy from the JSON result for display:

pass[0](all_roads_visited)         ← AndCondition (top-level)
├── StickyCondition(road_5)        ← Wrapper
│   └── EntityLanePositionCondition  ← Leaf
├── StickyCondition(road_12)
│   └── EntityLanePositionCondition
└── StickyCondition(road_8)
    └── EntityLanePositionCondition

fail[0](ego_existence)             ← EntityExistenceCondition
fail[1](default_timeout)           ← TimeoutCondition

The scenario's own fail conditions come first, then the runner's ego_existence, then default_timeout — which is absent when the scenario registered a timeout of its own.

The _build_condition_tree() function in scanner.py recursively parses:

  • children key for composite conditions (AndCondition, OrCondition)
  • child key for wrapper conditions (StickyCondition, PersistentCondition, NotCondition)

Caching Strategy

The scanner uses a module-level dictionary cache (_cache) keyed by path and session identifiers. The cache is cleared explicitly via the /api/refresh endpoint (triggered by the "Update" button in the UI). A batch's session and case pages are also kept only while the batch's files stay the same (cases.batch_stamp: the times of batch_results.json, the SLURM manifest, collected.json, and the slurm/submissions/ and slurm/results/ directories), so a SLURM batch resubmitted and collected again shows its new results on a reload; until it is collected again, its summary is older than its last submission or case outcome, and its cases are read from slurm/results/ instead. A multirun or single session is stamped too, since a run still in progress writes its files after its pages were first read (#1964): a multirun session's page by the modification times of its directory and of each job directory (one scandir and a stat per job, not per file: a job added, or a result, log or other file created, removed or renamed in a job changes one), and it is kept only once every job has a result that could be read (a result written in place changes no directory's time); a case page of a multirun job, and a single run's session and case pages, by the name, time and size of every entry of the run's directory (a handful of stat calls). This avoids repeated filesystem scans while keeping results fresh on demand.


Coordinate Transform System

Three coordinate systems are unified through MapManager:

graph LR
    L2["Lanelet2Pose<br/>(lanelet_id, s)"]
    OD["OpenDrivePose<br/>(road_id, lane_id, s)"]
    CW["CarlaWorldPose<br/>(x, y, z, yaw)"]

    L2 -->|"to_opendrive()"| OD
    L2 -->|"to_carla_world()"| CW
    OD -->|"to_carla_world()"| CW
    CW -->|"to_lanelet2()"| L2
    OD -->|"snap_to_carla_road()"| CW
    CW -->|"to_opendrive()"| OD

MapManager is a singleton that holds the LaneletMap, RoadNetwork, and a precomputed z-offset (averaged from CARLA spawn points, leaving out those matched to a lanelet on another level of stacked roads). It is initialized once per ScenarioQueue.start() when both xodr_path and lanelet2_path are provided.

to_carla_world() returns a CarlaWorldPose; to_carla_location() takes any of the three (or a carla.Location) and returns the carla.Location of its CarlaWorldPose.

snap_to_carla_road() uses CARLA's ray-cast API to project an OpenDRIVE position onto the actual 3D road surface, accounting for terrain height differences between the map definition and the rendered world.


Entity Management

classDiagram
    class VehicleEntityConfig {
        +role_name: str
        +spawn_location: SpawnLocation
        +vehicle_type: str
        +initial_speed_kmh: float
        +spawn_retry_max_count: int
    }

    class EgoConfig {
        +role_name = "Ego" (fixed)
        +goal_pose: Lanelet2Pose | None
    }

    class SpawnLocation {
        <<interface>>
    }

    class SpawnTransform {
        +transform: carla.Transform
    }

    class SpawnPointIndex {
        +index: int
    }

    class VehicleEntity {
        +actor: carla.Actor | None
        +spawn(world) carla.Actor
        +destroy()
    }

    class EgoVehicle {
        +use_autopilot = true
        +requires_goal = false
        +actor: carla.Actor | None
        +spawn(world, config) carla.Actor
        +on_scenario_start(world)
        +on_tick(world, elapsed)
        +on_scenario_end(world)
        +destroy()
    }

    class ExternallyDrivenEgo {
        +use_autopilot = false
        +stack_name: str
        +termination_requested: bool
        +termination_reason: str | None
        +change_lane() refuses
        +set_desired_speed() refuses
        +turn_at_junction() refuses
    }

    class CarlaDriverEntity {
        +handshake: BridgeHandshake | None
    }

    class BridgeHandshake {
        +bridge: AutowareBridge
        +launcher: AutowareLauncher | None
        +set_mission(initial_pose, goal_pose, waypoints)
        +route_to(world, goal)
        +start(actor)
        +poll() reason | None
        +ready: bool
    }

    class Ros2EgoEntity

    class BridgedEgoEntity {
        +requires_goal = true
        +handshake: BridgeHandshake
        +is_initialized: bool
    }

    class AutowareEgoEntity

    VehicleEntityConfig <|-- EgoConfig
    SpawnLocation <|.. SpawnTransform
    SpawnLocation <|.. SpawnPointIndex
    VehicleEntityConfig --> SpawnLocation
    VehicleEntity --> VehicleEntityConfig
    EgoVehicle <|-- ExternallyDrivenEgo
    ExternallyDrivenEgo <|-- CarlaDriverEntity
    ExternallyDrivenEgo <|-- Ros2EgoEntity
    ExternallyDrivenEgo <|-- BridgedEgoEntity
    BridgedEgoEntity <|-- AutowareEgoEntity
    BridgedEgoEntity *-- BridgeHandshake
    CarlaDriverEntity o-- BridgeHandshake

The ego hierarchy splits on who drives. EgoVehicle is driven by the traffic backend. ExternallyDrivenEgo is the base of every ego a stack of its own drives: it opts out of the TrafficManager, refuses TrafficManager manoeuvres instead of delegating them, and carries the termination request. BridgeHandshake is the splatsim-consistent AutowareBridge handshake — the mission handed over, readiness awaited, the stack launched and stopped per scenario — which depends only on the abstract bridge, so it is not particular to Autoware. An ego holds one rather than inheriting it, as the static check takes single inheritance only: BridgedEgoEntity is the ego that is nothing but the handshake (the stack drives), AutowareEgoEntity is that with Autoware on the other end, and a CarlaDriverEntity given a bridge holds one beside its policy loop, starting the policy once the stack is ready.

Ego lifecycle hooks: ScenarioRunner calls on_scenario_start() after the warm-up ticks, on_tick() on every simulation tick, and on_scenario_end() during teardown. They are no-ops on EgoVehicle, so a TrafficManager-driven ego costs nothing. An ego that drives itself overrides them — see External Driver Interface.

Who drives the traffic: everything the scenario did not author belongs to a traffic backend — traffic_manager by default, none for an empty road, or one from another package. ScenarioRunner calls prepare / start / tick / close on it and never names a traffic simulator itself, and an entity's manoeuvre intents (change_lane, turn_at_junction, follow_route) are delegated to whichever backend drives it. A goal given to a vehicle the backend drives, the ego or an NPC, becomes one of them: BackendDriven.route_to hands the backend the route from where the vehicle is, through any waypoints, to it. A pedestrian given one lays its walk out on the walkable lanelets (PedestrianEntity.route_to), and the RoutingAction walks it a tick at a time (PedestrianEntity.walk_on, steered as a RoutingAction steers a walker in follow mode) -- so on a pedestrian the action needs the tick loop (BaseAction.needs_tick_loop). The walk is searched on coordinate.walk_graph.WalkGraph, not on Lanelet2's routing graph: a crosswalk is usually drawn across the walkways it joins, reaching onto them without sharing a point, so Lanelet2 joins nothing across a road. The walk graph joins a walkable lanelet's end to any walkable lanelet it lies on (or within 0.5 m of), at that point along it, joins two walkable lanelets where their centrelines cross (a crosswalk drawn right across a walkway), and walks every walkway both ways whatever its one_way tag. A leg it has no walk for (a map with no walkways, as most Autoware maps, or a waypoint off them) is a straight line, with a warning.

Who drives the ego: use_autopilot decides whether the backend is given the ego actor to drive or told to keep its hands off it. EgoVehicle opts in (TrafficManager drives); every ExternallyDrivenEgo opts out: on its own it leaves the actor for something outside the scenario, CarlaDriverEntity drives it from an external policy's plan, Ros2EgoEntity from ROS 2 nodes' commands, and BridgedEgoEntity leaves it to the stack it handed the mission to.

Where the ego is going: requires_goal says whether an entity can start without a destination. A BridgedEgoEntity (AutowareEgoEntity) cannot — the goal is what makes it move at all — and BaseScenario.require_goal(), asked by register_route_to_goal and by ScenarioRunner once setup() returns, refuses one that has none. An ego driven by the TrafficManager or a driver policy reads no goal, so a scenario may name none and the routing action is simply not registered. The goal reaches the ego config from ego.goal_lanelet_id or from the scenario itself — IntersectionPassingScenario derives it from the route it asserts, through BaseScenario.derive_goal_from_route().

Spawn retry logic: When a vehicle fails to spawn (e.g., collision with existing geometry), the spawn system retries with lateral (t_step) and vertical (z_step) offsets, up to spawn_retry_max_count attempts.


Lanelet Constraint Sweeper

The sweeper enables automated parameter sweeps across lanelets that match specified constraints:

graph TB
    subgraph Input
        SC["Scenario Config<br/>(YAML)"]
        Constraints["Sweep Constraints<br/>(traffic_light, direction, ...)"]
        Bindings["Sweep Bindings<br/>(ego.spawn_s, npc.spawn_s, ...)"]
    end

    subgraph Processing
        Map["Load Lanelet2 Map"]
        RG["Create Routing Graph"]
        Match["Find Matching Lanelets"]
        Resolve["Resolve Bindings<br/>for each matched lanelet"]
    end

    subgraph Output
        Jobs["Override Lists<br/>[scenario=X, ego.spawn_lanelet_id=5, ...]"]
    end

    SC --> Map
    Constraints --> Match
    Map --> RG
    RG --> Match
    Match --> Resolve
    Bindings --> Resolve
    Resolve --> Jobs

The sweeper is registered as a Hydra plugin (hydra_plugins/autoware_scenario_sweeper/) and can also be invoked directly from the viewer via sweep_resolver.py (without CARLA, using only the Lanelet2 map).


Ports and Environment Variables

Network Ports

The system uses three TCP ports for inter-process communication with the CARLA simulator and the viewer web server:

graph LR
    subgraph Viewer["Viewer Process"]
        FastAPI["FastAPI<br/>(Uvicorn)"]
    end

    subgraph Scenario["Scenario Process"]
        Client["carla.Client"]
        TM_Client["TrafficManager<br/>Client"]
    end

    subgraph CARLA["CARLA Simulator"]
        RPC["RPC Server"]
        TM_Server["TrafficManager<br/>Server"]
    end

    subgraph Browser["Browser"]
        B["User"]
    end

    B -->|"TCP 9000<br/>(VIEWER_PORT)"| FastAPI
    Client -->|"TCP 2000<br/>(server.port)"| RPC
    TM_Client -->|"TCP 8100<br/>(traffic_manager.port)"| TM_Server
Port Default Component Protocol Configured via Description
2000 2000 CARLA RPC Server TCP server.port (Hydra override, e.g. scenario server.port=2001; scenario takes no --port) Main communication channel between carla.Client and the CARLA simulator. Used for world control, actor spawning, recording, etc.
8100 8100 CARLA TrafficManager TCP traffic_manager.port (Hydra), tm_port constructor arg RPC port for the TrafficManager, which controls NPC vehicle autopilot behavior. The CARLA default is 8000, but this project uses 8100 to avoid conflicts with other services (e.g., VS Code).
9000 9000 Viewer Web Server HTTP viewer --port, VIEWER_PORT env var FastAPI/Uvicorn server for the result viewer web UI. Serves HTML pages, REST API, and SSE progress stream.

Note

The CARLA RPC port and TrafficManager port are used within the scenario subprocess, not the viewer process. The viewer communicates with CARLA only indirectly through subprocess.run(["uv", "run", "scenario", ...]).

Environment Variables

Variable Required Default Used by Description
CARLA_EXECUTABLE No the launcher scenario-setup installed CarlaServerManager Absolute path to the CARLA UE5 launcher (CarlaUnreal.sh). Unset, the one scenario-setup installed under ~/.autoware_carla_scenario/bin/carla is launched. When a server is already running and reuse_if_running=True, neither is needed. Also a gate for pytest: tests are skipped when there is no launcher at all and no server answers at their host.
AUTOWARE_CARLA_SCENARIO_HOME No ~/.autoware_carla_scenario carla_install_paths Where scenario-setup installs CARLA (bin/carla).
NISHISHINJUKU_MAP_PATH Yes (if map.overwrite_xodr is on) — ScenarioRunner.load_map_by_overwriting_xodr() Path to the internal .xodr file inside the CARLA installation for the NishishinjukuMap. Used to overwrite the built-in OpenDRIVE file with a custom version while retaining full CARLA map assets (meshes, textures). The variable name is derived from the map name via CamelCase → UPPER_SNAKE_CASE_PATH conversion.
NISHISHINJUKU_XODR_PATH No data/nishishinjuku_carla.xodr Map config YAML (${oc.env:...}) Path to the custom OpenDRIVE file for the Nishishinjuku map. Resolved by OmegaConf's oc.env interpolation in conf/map/nishishinjuku.yaml.
NISHISHINJUKU_LANELET2_PATH No data/nishishinjuku.osm Map config YAML (${oc.env:...}) Path to the Lanelet2 .osm file for the Nishishinjuku map. Resolved by OmegaConf's oc.env interpolation.
VIEWER_BASE_PATH No Current working directory Viewer (ui/__init__.py) Base directory that the viewer scans for outputs/ and multirun/ result directories. viewer --base-path wins over it.
VIEWER_HOST No 127.0.0.1 Viewer (ui/__init__.py) Bind address for the Uvicorn HTTP server. viewer --host wins over it; 0.0.0.0 serves every interface, with no authentication.
VIEWER_PORT No 9000 Viewer (ui/__init__.py) Listen port for the Uvicorn HTTP server. viewer --port wins over it.
SWEEP_RESUME_FROM No 0 Sweeper (lanelet_constraint_sweeper.py reads it; sweeper/job_status.py's resume_from_env() checks its value) Internal variable set by the --resume-from N CLI flag. Passed via environment because Hydra's CLI parser rejects unknown overrides. Tells the sweeper to start at job N (1-indexed); set (to a whole number, 0 or empty for none), it wins over sweep.resume_from, and any other value is refused (exit 2). A batch (a glob, or launcher=slurm) reads the flag directly instead; a single run or Hydra's basic sweeper, which nothing would resume, refuses the flag (exit 2).

Dynamic Map Path Variables

ScenarioRunner.load_map_by_overwriting_xodr() runs only when the map config sets overwrite_xodr: true (or ScenarioQueue(overwrite_xodr=True) is passed directly). It is off by default: a map cooked for CARLA already ships the OpenDRIVE its assets were built from, and replacing that puts the road network and the meshes out of step. When it is on, the method derives the destination environment variable name from the CARLA map name using map_asset_env_var() (maps/opendrive.py):

Map Name Derived Variable
NishishinjukuMap NISHISHINJUKU_MAP_PATH
Town01 TOWN01_PATH
Town10HD_Opt TOWN10_HD_OPT_PATH

The conversion rule is: insert _ before each uppercase letter that follows a lowercase letter or digit, convert to UPPER_SNAKE_CASE, and append _PATH.


Key Design Decisions

Process Isolation for Viewer

The viewer deliberately avoids importing CARLA in its own process. All scenario execution happens via subprocess.run(["uv", "run", "scenario", ...]). This prevents CARLA's C++ extension from causing segmentation faults in the web server.

Synchronous Simulation Mode

All scenario execution uses CARLA's synchronous mode at 20 Hz. This ensures:

  • Deterministic tick ordering (actions, conditions, physics all advance together)
  • Reproducible results with fixed random seeds
  • No frame drops during recording

Nothing Rendered During a Run

A run renders no camera image for itself: the tick loop does no RGB processing beyond what the ego's own sensors need, which keeps condition evaluation timing accurate. What is needed to see the run afterwards is written as data (the trajectory log) and drawn by a viewer.

Condition Composition Pattern

Complex pass/fail criteria are built by composing atomic conditions:

# Example: pass when ego visits all three road segments
pass_condition = AndCondition(
    label="all_roads_visited",
    conditions=[
        StickyCondition(EntityLanePositionCondition.anywhere_on_road("ego", "5", ...)),
        StickyCondition(EntityLanePositionCondition.anywhere_on_road("ego", "12", ...)),
        StickyCondition(EntityLanePositionCondition.anywhere_on_road("ego", "8", ...)),
    ],
)

StickyCondition latches once its child is satisfied, so each road only needs to be visited once. AndCondition requires all children to be satisfied simultaneously. It checks every child on every tick, so each sticky child latches when its own road is reached and the roads may be visited in any order; to require an order, latch the later one on the earlier: StickyCondition(AndCondition([StickyCondition(on_5), on_12])).

World Reset for Clean State

Between two scenarios -- before the next one's attempt (and before its cooldown), and before a retry -- the queue has ScenarioRunner.reset_world() reset the world; the last scenario's world is reset by the next run_all(), or, on a server the queue does not stop, by stop() (ScenarioRunner.restore_world(), #2067). By default (server.world_reset: reload) it loads the batch's map again by name (client.reload_world() fails on CARLA UE5): a new episode, with the run's actors destroyed first so their sensor streams are released. server.world_reset: soft keeps the episode and puts back what a run changes -- destroying every actor the freshly loaded world did not have and restoring the settings, weather, traffic lights and spectator read right after the load (world_reset.py) -- and falls back to a reload when the world is another episode or anything is left over. See Resetting the world between cases for what a soft reset does and does not reset.