VisionPilot–Safety Island control contract
Status: draft for SI issue #62. The actuator boundary, initial camera/ego matching rule, applied-brake event and initial 500 ms gate have been agreed; the remaining wire, freshness and scenario limits below have not. This is not yet an implementation-ready contract. The current deployed interfaces are documented separately in DDS topics.
Purpose and boundary
No real vehicle or VCU interface has been selected. This document defines a vehicle-independent logical motion request, not a claim that a particular production vehicle accepts these fields directly. When a target vehicle is chosen, its actual DBW/CAN/Ethernet contract, enable/override rules and actuator feedback must be checked separately. CARLA’s API must not dictate the SI/VP motion-request semantics.
The first validation rig uses CARLA 0.9.16, its matching Python API, and no native CARLA ``–ros2``. The CARLA bridge transports camera and vehicle feedback to ROS 2; the DDS domain bridge forwards only SI-approved requests to a separate CARLA actuator component. That component is the only writer to the vehicle actor. It realizes SI’s normal steering/speed/signed-acceleration request as CARLA actuator controls and applies SI’s explicit stop/hold decision as braking. It does not choose a different normal driver or independently detect a fault. The CARLA bridge does not control speed, brake on a timeout, choose the normal driver, or decide whether a command is safe. The CAN–CARLA demo is a separate transport, not a second writer in this rig. On a future real vehicle, this CARLA-specific component is replaced by a separately verified vehicle/VCU-facing realization.
There are two SI control modes and three startup configurations:
Configuration |
Normal candidate |
SI decision |
|---|---|---|
|
Autoware Universe planning trajectory |
Select only Autoware input; follower proposes the command |
|
VP path and speed/stop intent from one camera cycle |
Select only VP reference; the same follower proposes the command |
|
One VP steering, VP-selected target-speed and acceleration request |
Check and pass the accepted request; do not run the follower or recompute VP’s normal decision |
Only SI’s output gate releases normal or stop commands. Faults latch a stop/hold response without silently changing the mode or planning source; resuming normal output requires an explicit re-enable action. While SI is running, missing/invalid selected input, bad normal output, and stale vehicle feedback must cause SI’s fault response. The stop path must not depend on a responsive follower/MPC. This design does not guarantee a stop if SI or the bridge itself stops producing output.
Wire and source-identity decisions
The existing Autoware autoware_planning_msgs/msg/Trajectory input
and follower algorithm remain supported without an Autoware-side adapter.
Its topic is /planning/trajectory (domain 1 to 2), the trajectory
Autoware 2.0’s own follower reads.
VP needs no adapter either: SI subscribes to VP’s own
visionpilot_msgs/msg/DrivingReference on /vehicle/driving_reference
and converts it to a follower trajectory itself (vp_reference.hpp). The
earlier E2E adapter and its safety_island_msgs/msg/TrajectoryCandidate on
/planning/visionpilot/trajectory_candidate are retired; the SI conversion
reproduces the adapter’s output (13 points, same speed profile) within
1e-9 on 3000 randomized references. The same SI binary reads
SI_SUPERVISION_MODE and SI_TRAJECTORY_SOURCE once at startup and
subscribes only to the selected input (or the VP command in VP_CONTROL).
Neither fault nor re-enable can change it; an invalid, unsupported or
half-set startup combination is fatal. With neither variable set (targets
without an environment, such as Zephyr), the build default applies:
SI_DEFAULT_SUPERVISION_MODE/SI_DEFAULT_TRAJECTORY_SOURCE, si/autoware
unless overridden at compile time, and the SI logs that it used it. Run both publishers concurrently
and inject selected-source loss to verify isolation.
SI_LEGACY_CONTROL_CMD (always | stop_only, build default
SI_DEFAULT_LEGACY_CONTROL_CMD = always) selects what reaches the
legacy DDS control_cmd topic. always publishes every approved
payload, as before. stop_only publishes only SI_STOP there, for a
consumer that arbitrates “a fresh SI command wins over my own driver” (the
X5H CES demo’s bench arbiter): SI is heard only when it overrides. It is a
transitional compatibility switch, not a second control path:
ApprovedRequest stays the only authoritative output and is never
filtered, and CAN output is not affected. An invalid value is fatal at
startup.
SI_OPERATION_MODE_DURABILITY (transient_local | volatile, build
default SI_DEFAULT_OPERATION_MODE_DURABILITY = transient_local)
selects the durability the SI requests on /system/operation_mode/state.
The operation mode is published only when it changes, so a reader that does
not request transient_local sees nothing if it joins after the last
change. The default matches the classic demo bridge, which retains the state
for late-joining SIs; a deployment whose bridge publishes volatile (the
E2E rig) selects volatile, because a transient_local request does not
match a volatile writer at all. An invalid value is fatal at startup.
The VP command must be one compound message, not independently arriving steering, acceleration and speed samples. It must carry steering in tire radians, VP’s target speed in m/s, acceleration/deceleration in m/s², source camera time and frame identity, a boot/session identity, cycle number, and validity. Sign conventions, optional fields, bounds, serialized size, QoS, the exact ROS/DDS type and topic must be frozen before implementing the VP and SI producers/consumers.
The VP path and speed/stop intent likewise come from one VP cycle. SI checks their identity on the reference itself, so old, mismatched or reordered data cannot become fresh because a DDS bridge republishes it. A reference VP marks unusable (invalid, no path, no source stamp, no speed horizon), one without a same-frame ego sample, or one that converts to fewer points than the follower can run is ignored and logged, never turned into a fabricated stop: only an accepted reference refreshes the selected-source watchdog, which owns the stop. Malformed values (non-finite, oversized horizon) latch SI_STOP. A genuine VP-chosen stop in a valid path/speed cycle remains a normal candidate. SI checks the original VP session/cycle and progress of the camera source stamp before refreshing its selected candidate watchdog. Equal source stamps (replayed frames) do not refresh it; regressions latch SI_STOP. The source stamp uses CARLA simulation time and is compared only with earlier source stamps, not SI wall time. The selected arrival watchdog is separate. Native Autoware trajectories have no session/cycle; SI checks progress of their header stamps instead, without inventing an Autoware producer identity, and latches SI_STOP for a trajectory shorter than the follower’s minimum point count or with any field the follower consumes non-finite.
SI publishes through one supervised DDS command output for all three configurations. The wire contract must carry a unique SI session and output sequence so the test rig can distinguish the normal request from SI’s stop response and correlate each output with a CARLA frame. The legacy raw follower output must not be routed to CARLA around this gate. The selected DDS/CAN output mode is configured per run; the CAN demo is not enabled simultaneously with the first DDS rig.
SI_STOP holds the last known steering and requests -stop_decel_mps2
throughout the stop. Its speed target ramps down from the ego speed captured
at the latch only when odometry is fresh; unseen, stale or invalid ego speed
uses a zero-speed stop instead. Ramp duration uses a monotonic clock, so a
wall-clock correction cannot restart a completed ramp. Output stamps and
source-arrival watchdogs still use the SI system clock.
Implementations and wire layout
The VP side is implemented on the fork branch
feat/vp-si-interface (visionpilot_msgs). SI’s supervised output
and VP-specific candidate ingress are implemented. The clean rig replay
and cross-DDS wire check are recorded in the E2E evidence separately.
Topic / direction |
Type |
State |
|---|---|---|
|
|
Implemented: steering tyre angle, VP-selected target speed, signed acceleration, source capture stamp, VP session + cycle |
|
|
Implemented: lane polynomial (a, b, c, x_max), speed horizon (20 × 0.05 s), same identity fields; SI’s SI_CONTROL + VP input |
|
|
Existing Autoware-only input |
|
|
Implemented (SI IDL): the only actuator-facing output; SI-only
authoring with SI |
The VP messages are published once per processed camera cycle and carry
the source stamp of the image that produced the decision; the legacy
Float64 command topics stay unchanged. target_speed_mps is the speed
at the end of the plan’s 1 s schedule: VP’s own statement of where the
current plan is heading, not a consumer re-derivation.
DrivingReference’s path and speed horizon always come from one cycle.
SI-facing sizes must stay within the SI runtime’s 1400 B DDS message
limit. DrivingCommand is a few hundred bytes and DrivingReference
(four path doubles plus a 20-sample horizon) under 300 B, and the SI rejects a horizon longer than 100 samples so a reference always fits the limit; the SI output
stays a Control-sized sample. The SI-side IDL mirror of
visionpilot_msgs was generated from the fork’s .msg definitions
with rosidl_adapter and compiles in the SI build (0.11 idlc; the
@verbatim comments are skipped with the usual warnings). The
ApprovedRequest IDL has a matching ROS-side
safety_island_msgs definition in the E2E rig. Build/replay must check
ROS-to-SI serialization interop; a generated type alone is not live proof.
Measured cadence and identity (VPS, 2026-09-25)
One 150 s run of the implemented VP image with the E2E camera bridge on Town04 (CARLA 20 Hz fixed ticks, 1920 × 1280 camera), measured with a Humble/CycloneDDS reader:
Topic |
Rate |
p50 |
p95 |
max |
p95 age |
|---|---|---|---|---|---|
|
8.82 Hz |
103 ms |
202 ms |
627 ms |
113 ms |
|
8.82 Hz |
103 ms |
201 ms |
627 ms |
114 ms |
|
9.55 Hz |
101 ms |
126 ms |
479 ms |
38 ms |
|
20.0 Hz |
50 ms |
75 ms |
100 ms |
22 ms |
|
20.0 Hz |
50 ms |
75 ms |
107 ms |
— |
Identity over 1324 command/reference pairs: every command had a reference from the same cycle with the same source stamp; no missing source stamp, one stable session, no cycle regression, the speed horizon was always 20 × 0.05 s, and no invalid path or reference was seen. The VP output rate (8.8 Hz) tracks the camera (9.6 Hz); VP processes most, not all, camera frames.
Consequences for the first thresholds (proposed values that also serve as the implemented SI supervisor defaults; approval pending before closing #62):
A 0.5 s source-age timeout on VP outputs would false-trip: the measured maximum gap is 627 ms (682 ms in the re-run below). Values: 1.0 s for the selected candidate, 0.4 s default for the 20 Hz ego/speed feedback (within the agreed 0.3-0.5 s band), 0.5 s for the 10 Hz steering report; the operation mode has no age limit (the rig republishes it at 10 Hz, but Autoware and the classic demo publish it only on change, so an age limit would latch a healthy system). These are per-source watchdogs, not the fault-to-brake gate.
The 500 ms gate stays as agreed: it starts at SI fault detection, the control tick that first observes the stale/invalid source, after the source has already been declared stale.
The first measurement used the bridge’s host-time image stamps. The E2E bridge now stamps camera and ego samples with the CARLA simulated time of the capture/frame and publishes one ego sample per world frame. Re-measured over 90 s: every one of 820 VP command cycles had an exact ego sample with the same source stamp (100%); 806/820 source stamps were also seen on the camera stream by the probe (loss of large image samples in transport, not a VP fault); all identity checks above still held. SI therefore requires an exact same-stamp ego sample (a bounded history of ~10 s of odometry) and ignores the reference when it is missing, without inventing a stop.
Camera, ego and deadline clocks
CARLA camera frames expose an episode frame number and a simulated capture time. CARLA world snapshots expose a frame number and simulated world time. The bridge must retain this source identity on camera and ego samples; using its publication time for both hides acquisition and sampling skew. VP must preserve the input frame identity through inference and planning. SI’s VP ingress uses only an exact-match ego sample, never just the latest available pose.
On two pinned 0.9.16 VPS smoke runs (Town04, 20 Hz fixed world ticks, 10 Hz RGB camera at 640 × 480 and at the rig’s 1920 × 1280 resolution), each run’s 35 camera frames matched an ego world snapshot with the same frame number and exactly the same CARLA source time (maximum measured source-time skew 0.0 s). For the first validation rig, require the same episode and frame number for camera and ego; reject the VP reference if the exact ego frame is missing. Do not match an identical frame number across a CARLA episode reset. Bounded interpolation may be designed later if full-rig evidence shows missing exact frames; it is not a permitted first-rig fallback. This tolerance is for CARLA source samples, not permission to replace their capture stamps with host arrival time. These short probes demonstrate source-frame joinability, not ROS/VP delivery latency or full E2E processing performance.
Simulation timestamps, ROS wall time and a host monotonic clock are different time bases. A CARLA episode time in seconds must not be subtracted from SI’s Unix/system clock. Specify the conversion or correlation used for source age, and the monotonic clock used for elapsed fault-to-applied time. The POSIX CARLA rig is on one host; later hardware needs its own verified clock-sync contract. A repeated camera frame or VP cycle is not fresh merely because it was received again.
CARLA actuation and measured application
The CARLA-facing interface is not frozen. A single-vehicle VPS probe
used the pinned 0.9.16 server digest from the E2E rig, its checksum-verified
Python 3.10 wheel, Town04, synchronous 0.05 s ticks paced at 20 Hz, and a
Tesla Model 3. Python apply_ackermann_control did eventually drive
and stop the car, but it did not directly honor a signed deceleration
request:
After changing from target speed 3 m/s to 0 m/s with
acceleration=-3 m/s², the actor continued to receive throttle for about 0.85 s of wall time. The first observed brake greater than 0.05 appeared after about 1.15 s, at CARLA frame 163 (request after frame 139). This is one smoke run, not a worst-case latency bound.CARLA 0.9.16
AckermannController.cpp::SetTargetPointstoresFMath::Abs(UserTargetPoint.Acceleration): negative and positive values with equal magnitude give the same acceleration limit to its internal speed controller. They do not directly specify braking intent.On the same VPS, a direct Python
VehicleControlwith throttle 0 and brake 0.6 appeared as those same actuator fields in the first CARLA world frame following the call (about 2.3 ms in that one probe; not a fault-to-actuation integration bound).
Thus the original preference for the Ackermann interface is superseded
for this rig: CARLA 0.9.16 Ackermann cannot be assumed to realize
signed VP acceleration/deceleration or prompt SI brake intent. The
agreed architecture places vehicle-specific actuator realization in a
separate single-writer CARLA component, after SI’s vehicle-independent
approved output, not inside SI and not in the camera/feedback bridge.
For CARLA this component can use VehicleControl as its final actor
API: normal speed/acceleration-to-pedal behavior and immediate mapping of
an explicit SI stop/hold to brake must be implemented and validated
before integration. The direct brake smoke does not validate normal
driving, a fixed brake strength or fault-to-applied timing. This adapter
must not invent a new timeout-brake policy or make a safety decision when
SI/bridge messages stop; that silence is measured separately and is not
claimed safe.
For each fault injection, record the SI fault-detection event, SI
decision and unique output sequence, actuator receipt, and the first
CARLA world frame whose actor control applies a brake command. The
initial candidate threshold is throttle at most 0.01 and brake greater
than 0.05 in the actor’s get_control() for that frame (to exclude
negligible controller chatter). This is evidence of a simulator-applied
control request, not measured wheel brake torque, deceleration or a
physical-vehicle result. Instrument the single final CARLA writer and
frame observer to correlate that control to the SI stop sequence;
pre-existing normal braking is not proof that the fault response arrived.
A publisher callback, bridge queue, Python RPC return or accepted
zero-speed setpoint is not application evidence. A missing applied frame
fails the run. This event is different from the later moment when the car
reaches zero speed.
Initial CARLA pass/fail gate, agreed for the first scenarios: while
SI is running, at most 500 ms of elapsed wall-monotonic time from
SI’s fault-detection event to the first CARLA world frame with the
above applied-brake condition. The bound is defined before the
integration run, not selected from a passing run. It is an initial
simulation acceptance gate, not a real-vehicle safety claim. The
SI POSIX detection timestamp and the CARLA frame observer must use
correlated monotonic clocks on the same test host, not SI’s existing
system_clock/ROS epoch stamp or CARLA simulated elapsed seconds.
Record and reject missing clock correlation. The
150 ms existing SI cycle plus one nominal 50 ms actuator loop and one
50 ms CARLA step leave approximately 250 ms for scheduling, DDS and test
instrumentation. These are budgeting assumptions to verify, not
guaranteed component response times. Also report fault onset to SI
detection separately, so a slow watchdog cannot be hidden by a fast
post-detection response.
Decision status
Item |
Decision / evidence required |
|---|---|
VP command and reference schema |
Implemented in fork branch |
SI candidate and output schema |
Output implemented as the |
Source-age watchdogs |
Implemented in the SI supervisor with the proposed defaults (candidate 1.0 s, ego 0.4 s, steering 0.5 s; no age limit for the operation mode, which is published on change only), per-source by design. Final-ingress rig runs report SI-stated selected-source ages of 1.01–1.10 s at detection; the injection→latch figure is reported separately because a multi-node source container adds its own shutdown time. |
Restart and re-enable rules |
Implemented and rig-validated: session change handling,
cycle-regression rejection, SI_STOP latch with |
CARLA actuator implementation |
Implemented in the E2E rig ( |
Follower-independent fault response (SI #64) |
Partly implemented. Supervision and the SI_STOP decision run at the start of the control tick, before the follower/MPC, and use per-source checks, the single output and the explicit re-enable. They still share the follower’s timer and thread, so a follower that hangs or overruns also stops the supervisor. A separately scheduled supervisor with a bounded execution path, and a test that a hung follower still yields SI_STOP, remain open. |
Per-scenario pass/fail gates |
Initially 500 ms from SI fault detection to first CARLA braking frame while SI is running; missing or late application fails. If scenarios need different limits, agree on them before integration, not after examining a failing run |
The 500 ms limit is not copied from the former 0.5 s all-inputs staleness gate: that gate was refreshed by any input and could not reliably detect loss of the selected source, which is why the supervisor replaced it with the per-source checks above. The 500 ms gate begins only after SI has actually detected a fault. End-to-end fault-onset-to-brake latency and physical stopping distance need separate reporting and scenario-specific limits.