Skip to content
Vision Pilot v1.2

Documentation

Everything you need to build, run, configure and contribute to Vision Pilot - the single source of truth for the project.

Vision Pilot is a free and fully open-source L2 ADAS system. It is designed to be integrated by automotive OEMs and Tier-1 suppliers into series-production passenger cars, and can optionally be adopted for transportation and logistics use cases in buses and trucks.

The complete codebase - including AI model weights - is available under the permissive Apache 2.0 licence, for commercial and research use alike.

New here? Start with these

1 · Install

Getting started →

Build from source, install the Debian package, or run the Docker image - then replay the sample dataset.

 2 · Understand

Architecture →

How camera frames become steering and throttle commands, and where the safety path sits.

 3 · Configure

Configuration →

Every key in the three .conf files, with defaults and valid values.

 4 · Go live

Hardware and calibration →

Choosing a camera, mounting it, and computing the homography that Vision Pilot needs.

What Vision Pilot does

Vision Pilot supports the entry-level L2 feature set for in-lane autonomous driving:

Feature Meaning Control axis
ACC Autonomous cruise control Longitudinal
FCW Forward collision warning Warning only
AEB Autonomous emergency braking Longitudinal
LKAS Lane keep assist Lateral
ALK Autonomous lane keep Lateral
LDW Lane departure warning Warning only
LDA Lane departure avoidance Lateral
ISA Intelligent speed assist (map-based) Longitudinal
Autopilot Single-lane hands-free highway autopilot Both

Hands-free autopilot has a defined operational domain. It is available only on highways and motorways with clearly marked lane lines, in fair weather and visibility, across the full range of highway driving speeds (0–70 mph), and away from construction zones and roadwork objects. A human driver is required to monitor and supervise the system at all times.

What it needs

Sensor specification. A single, front-facing, monocular RGB camera with a 50–55° horizontal field of view at 2 MP resolution. That is the whole sensor set. See hardware and calibration for mounting and camera selection.

No HD maps. Vision Pilot operates in a mapless mode and follows the road in real time. There is no localisation stack to run and no 3D map to keep current. (Intelligent speed assist does use speed-limit map data - that is a very different thing from an HD map.)

Compute. The stack is designed to run at 10 Hz within a budget of roughly 3–5 INT8 TOPs. It runs on CPU via ONNX Runtime, or on NVIDIA GPUs via the CUDA/TensorRT execution provider - see configuration.

For a 10-minute overview of the project’s goals and design, see the introductory presentation.

Map of the repository

Path What lives there
VisionPilot/app/ The VisionPilot executable entry point
VisionPilot/modules/ Sensing, engine, models, safety guardian, visualization, logging
VisionPilot/config/ vision_pilot.conf and friends, H.yaml, vehicle.dbc
VisionPilot/docker/ build.sh, run.sh, GPU and CPU Dockerfiles
Calibration/ Homography calibration script and printable checkerboard
Sensing/ Camera selection and mounting guide
Simulation/ CARLA (ROS 2 and Zenoh) and SODA.Sim integrations
Functional_Safety/ Safety plan, SEooC scope, software requirements, safety metrics
github-io/ This documentation site

The three AI models Vision Pilot runs are developed and released separately by the Autoware Foundation:

  • AutoSpeed - closest in-path object detection
  • AutoSteer - ego path future waypoint detection
  • AutoDrive - end-to-end distance, in-path object presence and road curvature

Getting help

Something out of date or missing? Edit this page. Report an issue ↗