Open source L2 ADAS,
from one camera.
A productionizable and safety-certifiable L2 ADAS stack — ACC, AEB, LKAS, LDW, ISA and single-lane highway autopilot — running on a single front-facing camera, with no HD maps. Complete codebase and AI model weights, free under Apache 2.0.
Entry-level L2, done properly
Vision Pilot targets the in-lane autonomous driving feature set that automotive OEMs and Tier-1 suppliers ship in series production passenger cars.
Hybrid end-to-end AI
Camera frames are processed in parallel by perception models for safety and by an end-to-end model for performance. A Safety Guardian fuses both paths before anything reaches the planner.
AutoSpeed ↗
Closest in-path object detection — what is in front of you, and how far away it is.
PerceptionAutoSteer ↗
Ego path future waypoint detection — the drivable corridor ahead of the vehicle.
End to endAutoDrive ↗
Distance and in-path object presence detection, plus road curvature estimation.
No HD maps
Vision Pilot operates fully mapless, following the road in real time. No localisation stack, no map maintenance, no geofence to keep current.
One monocular camera
A single front-facing RGB camera with 50–55° horizontal FoV at 2 MP. That is the entire sensor specification.
Three ways to run it
Sample OpenLane data is provided so you can see the full pipeline in action before touching a vehicle or a camera.
Best if you intend to modify the stack. Requires ONNX Runtime.
git clone https://github.com/autowarefoundation/vision_pilot.git
cd vision_pilot/VisionPilot
mkdir build && cd build
cmake -DONNXRUNTIME_ROOT=<ONNX_RUNTIME_ROOT_PATH> ../
make -j$(nproc)
./VisionPilot
Add -DENABLE_ROS2_INTERFACE=ON for ROS 2, or
-DENABLE_OCCUPANCY=ON for the bird's-eye Occupancy window.
Best on a fresh machine — the package pulls in the CUDA dependencies for you.
sudo apt install ./VisionPilot-1.0-x86_64.deb
# reboot, then:
VisionPilot
Config lives in /usr/share/visionpilot/config for package installs.
Best for a reproducible environment. GPU/CPU and ROS 2 variants are all built from the same scripts.
cd VisionPilot/docker
./build.sh --gpu --ros2 # or --cpu, and optionally --occupancy
./run.sh --gpu --ros2
Edit config/*.conf before building — the config is baked into the image.
Open-loop and closed-loop
Replay recorded drives for evaluation, or close the loop in CARLA to test how the stack actually behaves when its own commands move the vehicle.
Object detection, closest-in-path distance, ego path, lane departure and speed limit, rendered live alongside the heuristic Occupancy bird's-eye panel.
The same binary, fed by a ROS 2 bridge from CARLA, steering and braking a virtual vehicle. See the simulation guide.
Where the project is
Tagged releases land on GitHub with source, Debian packages and model weights.
- Occupancy bird's-eye view, with configurable colours
- Rerun logging: stream per-frame data straight to an
.rrdrecording
- Optional Occupancy BEV window beside the HUD
- Camera calibration guide and homography tooling
- First full L2 feature set on the hybrid end-to-end architecture
- Debian packaging, Docker images, CARLA closed-loop support
What is coming next
- Fusion between the front-facing camera and automotive RADAR
- 8 MP camera support at 120° horizontal field of view
- Safety verification and standards compliance (ISO 26262, ISO 8800)
- SODA.Sim simulator integration
Built in the open, weekly
Vision Pilot is developed by the Privately Owned Vehicle working group of the Autoware Foundation. Everything happens in public.
Discord ↗
Introduce yourself in the privately owned vehicles channel and ask anything.
Working group ↗
Weekly meetings in two time-zone-friendly slots. Minutes and recordings are public.
Contribute →
Onboarding steps, coding conventions and the pull request workflow.