Skip to content

Planning Data Analyzer#

Overview#

This package provides offline evaluation tools for trajectory planning performance analysis from recorded rosbag data.

Evaluation Modes#

  1. Open Loop: Evaluate prediction accuracy against ground truth trajectories
  2. OR Scene: Evaluate override regression scenarios (LIVE vs HISTORICAL)

Quick Start#

Open Loop Evaluation#

ros2 run autoware_planning_data_analyzer autoware_planning_data_analyzer_node --ros-args \
  -p bag_path:=~/my_bag \
  -p evaluation.mode:=open_loop \
  -p trajectory_topic:=/planning/trajectory \
  -p json_output_path:=~/results.json

OR Scene Evaluation#

# Using the unified script
./install/autoware_planning_data_analyzer/share/autoware_planning_data_analyzer/scripts/run_evaluation.sh \
  -b ~/result_bag.mcap \
  -m or_scene \
  -t /planning/diffusion_planner/trajectory \
  -o ~/results \
  --live \
  --viz

Multi-Model Pipeline#

# Evaluate multiple models from DLR results
./install/autoware_planning_data_analyzer/share/autoware_planning_data_analyzer/scripts/multi_model_pipeline.sh \
  --scenario ~/dataset/scenario.yaml \
  --trajectory /planning/model_a/trajectory \
  --trajectory /planning/model_b/trajectory \
  --output ~/comparison \
  --viz

Output Files#

  • evaluation_result.json - Detailed metrics (ADE, FDE, TTC, etc.) and summary statistics
  • evaluation_output.bag/ - Evaluation metrics as rosbag for visualization
  • debug_images/*.png - Trajectory visualization (with --viz flag)

Parameters#

Key parameters in config/offline_evaluation.param.yaml:

  • evaluation.mode: Evaluation mode (open_loop or or_scene)
  • evaluation_interval_ms: Sampling interval (default: 100ms)
  • sync_tolerance_ms: Time synchronization tolerance (default: 100ms)
  • trajectory_topic: Trajectory topic to evaluate
  • or_scene_evaluation.*: OR scene specific settings

Utility Scripts#

Located in scripts/ directory:

  • Bag Processing: merge_bags.py, rename_bag_topic.py, add_prefix_to_bag.py
  • Ground Truth: add_gt_trajectory_to_bag.py
  • OR Analysis: detect_or_and_route.py, evaluate_or_segments.py
  • Visualization: generate_or_visualization.py
  • Comparison: compare_live_historical.py

See scripts/README.md for detailed documentation.

For More Information#