Software installation

System Requirements

• Operating System: Ubuntu 22.04
• NVIDIA GPU

Prerequisites

Since VisionPilot requires AI packages, we need to install the following packages first.

• CUDA: Optional for GPU processing.
• OpenCV: For image and video processing.
  • Ubuntu: sudo apt install libopencv-dev
• ONNX Runtime: For model inference.
  • Download from the GitHub release
• LibTorch: For tensor manipulation capabilities.
  • Download from the PyTorch website
• TensorRT: Improve the inference performance.
  • Download from the website
  • Follow the tutorial to install.

Get the code & data

• Download the code

git clone https://github.com/evshary/autoware.privately-owned-vehicles.git -b zenoh_more_models
cd autoware.privately-owned-vehicles/VisionPilot

• Download the video: You can download any dashcam video on YouTube

# Create a folder for the video and models
mkdir -p data
cd data
# Put your video here, and name it as video.mp4

• Download the models

# Tool to download from Google Drive
pipx install gdown
# SceneSeg
gdown -O models/ 'https://docs.google.com/uc?export=download&id=1l-dniunvYyFKvLD7k16Png3AsVTuMl9f'
# Scene3D
gdown -O models/ 'https://docs.google.com/uc?export=download&id=19gMPt_1z4eujo4jm5XKuH-8eafh-wJC6'
# DomainSeg
gdown -O models/ 'https://docs.google.com/uc?export=download&id=1zCworKw4aQ9_hDBkHfj1-sXitAAebl5Y'