Perception

graph TD
    cmp_sen("Sensing"):::cls_sen
    cmp_loc("Localization"):::cls_loc
    cmp_per("Perception"):::cls_per
    cmp_plan("Planning"):::cls_plan

    msg_img("<font size=2><b>Camera Image</b></font size>
    <font size=1>sensor_msgs/Image</font size>"):::cls_sen

    msg_ldr("<font size=2><b>Lidar Point Cloud</b></font size>
    <font size=1>sensor_msgs/PointCloud2</font size>"):::cls_sen

    msg_lanenet("<font size=2><b>Lanelet2 Map</b></font size>
    <font size=1>autoware_auto_mapping_msgs/HADMapBin</font size>"):::cls_loc

    msg_vks("<font size=2><b>Vehicle Kinematic State</b></font size>
    <font size=1>nav_msgs/Odometry</font size>"):::cls_loc

    msg_obj("<font size=2><b>3D Object Predictions </b></font size>
    <font size=1>autoware_auto_perception_msgs/PredictedObjects</font size>"):::cls_per

    msg_tl("<font size=2><b>Traffic Light Response </b></font size>
    <font size=1>autoware_perception_msgs/TrafficSignalArray</font size>"):::cls_per

    msg_tq("<font size=2><b>Traffic Light Query </b></font size>
    <font size=1>TBD</font size>"):::cls_plan


    cmp_sen --> msg_img --> cmp_per
    cmp_sen --> msg_ldr --> cmp_per
    cmp_per --> msg_obj --> cmp_plan
    cmp_per --> msg_tl --> cmp_plan
    cmp_plan --> msg_tq -->cmp_per

    cmp_loc --> msg_vks --> cmp_per
    cmp_loc --> msg_lanenet --> cmp_per

classDef cmp_sen fill:#F8CECC,stroke:#999,stroke-width:1px;
classDef cls_loc fill:#D5E8D4,stroke:#999,stroke-width:1px;
classDef cls_per fill:#FFF2CC,stroke:#999,stroke-width:1px;
classDef cls_plan fill:#5AB8FF,stroke:#999,stroke-width:1px;

Inputs

PointCloud

PointCloud data published by Lidar.

• sensor_msgs/msg/PointCloud2

Image

Image frame captured by camera.

• sensor_msgs/msg/Image

Vehicle kinematic state

current position of ego, used in traffic signals recognition. See output of Localization.

Lanelet2 Map

map of the environment. See outputs of Map.

Output

3D Object Predictions

3D Objects detected, tracked and predicted by sensor fusing.

• autoware_auto_perception_msgs/msg/PredictedObjects
  • std_msgs/Header header
  • sequence<autoware_auto_perception_msgs::msg::PredictedObject> objects
    • unique_identifier_msgs::msg::UUID uuid
    • float existence_probability
    • sequence<autoware_auto_perception_msgs::msg::ObjectClassification> classification
      • uint8 classification
      • float probability
    • autoware_auto_perception_msgs::msg::PredictedObjectKinematics kinematics
      • geometry_msgs::msg::PoseWithCovariance initial_pose
      • geometry_msgs::msg::TwistWithCovariance
      • geometry_msgs::msg::AccelWithCovariance initial_acceleration
      • sequence<autoware_auto_perception_msgs::msg::PredictedPath, 10> predicted_paths
        • sequence<geometry_msgs::msg::Pose, 100> path
        • builtin_interfaces::msg::Duration time_step
        • float confidence
    • sequence<autoware_auto_perception_msgs::msg::Shape, 5> shape
      • geometry_msgs::msg::Polygon polygon
      • float height

Traffic Signals

traffic signals recognized by object detection model.

• autoware_perception_msgs::msg::TrafficSignalArray
  • autoware_perception_msgs::msg::TrafficSignal signals
    • autoware_perception_msgs::msg::TrafficSignalElement elements
      • uint8 UNKNOWN = 0
      • uint8 Red = 1
      • uint8 AMBER = 2
      • uint8 WHITE = 4
      • uint8 CIRCLE = 1
      • uint8 LEFT_ARROW = 2
      • uint8 RIGHT_ARROW = 3
      • uint8 UP_ARROW = 4
      • uint8 UP_LEFT_ARROW=5
      • uint8 UP_RIGHT_ARROW=6
      • uint8 DOWN_ARROW = 7
      • uint8 DOWN_LEFT_ARROW = 8
      • uint8 DOWN_RIGHT_ARROW = 9
      • uint8 CROSS = 10
      • uint8 SOLID_OFF = 1
      • uint8 SOLID_ON = 2
      • uint8 FLASHING = 3
      • uint8 color
      • uint8 shape
      • uint8 status
      • float32 confidence
    • int64 traffic_signal_id
  • builtin_interfaces::msg::Time stamp