Point cloud pre-processing design#

Overview#

Point cloud pre-processing is a collection of modules that apply some primitive pre-processing to the raw sensor data.

This pipeline covers the flow of data from drivers to the perception stack.

Recommended processing pipeline#

graph TD
    Driver["Lidar Driver"] -->|"Cloud XYZIRCAEDT"| FilterPR["Polygon Remover Filter / CropBox Filter"]

    subgraph "sensing"
    FilterPR -->|"Cloud XYZIRCAEDT"| FilterDC["Motion Distortion Corrector Filter"]
    FilterDC -->|"Cloud XYZIRCAEDT"| FilterOF["Outlier Remover Filter"]
    FilterOF -->|"Cloud XYZIRC"| FilterDS["Downsampler Filter"]
    FilterDS -->|"Cloud XYZIRC"| FilterTrans["Cloud Transformer"]
    FilterTrans -->|"Cloud XYZIRC"| FilterC

    FilterX["..."] -->|"Cloud XYZIRC (i)"| FilterC["Cloud Concatenator"]
    end

    FilterC -->|"Cloud XYZIRC"| SegGr["Ground Segmentation"]

List of modules#

The modules used here are from pointcloud_preprocessor package.

For details about the modules, see the following table.

It is recommended that these modules are used in a single container as components. For details see ROS 2 Composition

Point cloud fields#

The lidar driver is expected to output a point cloud with the PointXYZIRCAEDT point type.

name	datatype	derived	description
`X`	`FLOAT32`	`false`	X position
`Y`	`FLOAT32`	`false`	Y position
`Z`	`FLOAT32`	`false`	Z position
`I` (intensity)	`UINT8`	`false`	Measured reflectivity, intensity of the point
`R` (return type)	`UINT8`	`false`	Laser return type for dual return lidars
`C` (channel)	`UINT16`	`false`	Channel ID of the laser that measured the point
`A` (azimuth)	`FLOAT32`	`true`	`atan2(Y, X)`, Horizontal angle from the lidar origin to the point
`E` (elevation)	`FLOAT32`	`true`	`atan2(Z, D)`, Vertical angle from the lidar origin to the point
`D` (distance)	`FLOAT32`	`true`	`hypot(X, Y, Z)`, Euclidean distance from the lidar origin to the point
`T` (time)	`UINT32`	`false`	Nanoseconds passed since the time of the header when this point was measured

Note

A (azimuth), E (elevation), and D (distance) fields are derived fields. They are provided by the driver to reduce the computational load on some parts of the perception stack.

Warning

Autoware supports conversion from PointXYZI to PointXYZIRC (with channel and return type set to 0) for prototyping purposes. However, this conversion is not recommended for production use since it is not efficient.

Intensity#

We will use following ranges for intensity, compatible with the VLP16 User Manual:

Quoting from the VLP-16 User Manual:

For each laser measurement, a reflectivity byte is returned in addition to distance. Reflectivity byte values are segmented into two ranges, allowing software to distinguish diffuse reflectors (e.g. tree trunks, clothing) in the low range from retroreflectors (e.g. road signs, license plates) in the high range. A retroreflector reflects light back to its source with a minimum of scattering. The VLP-16 provides its own light, with negligible separation between transmitting laser and receiving detector, so retroreflecting surfaces pop with reflected IR light compared to diffuse reflectors that tend to scatter reflected energy.

Diffuse reflectors report values from 0 to 100 for reflectivities from 0% to 100%.

Retroreflectors report values from 101 to 255, where 255 represents an ideal reflection.

In a typical point cloud without retroreflectors, all intensity points will be between 0 and 100.

Retroreflective Gradient road sign, Image Source

But in a point cloud with retroreflectors, the intensity points will be between 0 and 255.

Intensity mapping for other lidar brands#

Hesai PandarXT16#

Hesai Pandar XT16 User Manual

This lidar has 2 modes for reporting reflectivity:

Linear mapping
Non-linear mapping

If you are using linear mapping mode, you should map from [0, 255] to [0, 100] when constructing the point cloud.

If you are using non-linear mapping mode, you should map (hesai to autoware)

[0, 251] to [0, 100] and
[252, 254] to [101, 255]

when constructing the point cloud.

Livox Mid-70#

Livox Mid-70 User Manual

This lidar has 2 modes for reporting reflectivity similar to Velodyne VLP-16, only the ranges are slightly different.

You should map (livox to autoware)

[0, 150] to [0, 100] and
[151, 255] to [101, 255]

when constructing the point cloud.

RoboSense RS-LiDAR-16#

RoboSense RS-LiDAR-16 User Manual

No mapping required, same as Velodyne VLP-16.

Ouster OS-1-64#

Software User Manual v2.0.0 for all Ouster sensors

In the manual it is stated:

Reflectivity [16 bit unsigned int] - sensor Signal Photons measurements are scaled based on measured range and sensor sensitivity at that range, providing an indication of target reflectivity. Calibration of this measurement has not currently been rigorously implemented, but this will be updated in a future firmware release.

So it is advised to map the 16 bit reflectivity to [0, 100] range.

Leishen CH64W#

I couldn't get the english user manual, link of website

In a user manual I was able to find it says:

Byte 7 represents echo strength, and the value range is 0-255. (Echo strength can reflect the energy reflection characteristics of the measured object in the actual measurement environment. Therefore, the echo strength can be used to distinguish objects with different reflection characteristics.)

So it is advised to map the [0, 255] to [0, 100] range.

Return type#

Various lidars support multiple return modes. Velodyne lidars support Strongest and Last return modes.

In the PointXYZIRC and PointXYZIRCAEDT types, the R field represents the return type with a UINT8. The return type is vendor-specific. The following table provides an example of return type definitions.

R (return type)	Description
`0`	Unknown / Not Marked
`1`	Strongest
`2`	Last

Channel#

The channel field is used to identify the vertical channel of the laser that measured the point. In various lidar manuals or literature, it can also be called laser id, ring, laser line.

For Velodyne VLP-16, there are 16 channels. Default order of channels in drivers are generally in firing order.

In the PointXYZIRC and PointXYZIRCAEDT types, the C field represents the vertical channel ID with a UINT16.

Azimuth#

The azimuth field gives the horizontal angle between the optical origin of the lidar and the point. Many lidar measure this with the angle of the rotary encoder when the laser was fired, and the driver typically corrects the value based on calibration data.

In the PointXYZIRCAEDT type, the A field represents the azimuth angle in radians (clockwise) with a FLOAT32.

Elevation#

The elevation field gives the vertical angle between the optical origin of the lidar and the point. In the PointXYZIRCAEDT type, the E field represents the elevation angle in radians (clockwise) with a FLOAT32.

Solid state and petal pattern lidars#

Warning

This section is subject to change. Following are suggestions and open for discussion.

For solid state lidars that have lines, assign row number as the channel id.

For petal pattern lidars, you can keep channel 0.

Time stamp#

In lidar point clouds, each point measurement can have its individual time stamp. This information can be used to eliminate the motion blur that is caused by the movement of the lidar during the scan.

Point cloud header time#

The header contains a Time field. The time field has 2 components:

Field	Type	Description
`sec`	`int32`	Unix time (seconds elapsed since January 1, 1970)
`nanosec`	`uint32`	Nanoseconds elapsed since the `sec` field

The header of the point cloud message is expected to have the time of the earliest point it has.

Note

The sec field is int32 in ROS 2 humble. The largest value it can represent is 2^31 seconds, it is subject to year 2038 problems. We will wait for actions on ROS 2 community side.

More info at: https://github.com/ros2/rcl_interfaces/issues/85

Individual point time stamp#

Each PointXYZIRCAEDT point type has the T field for representing the nanoseconds passed since the first-shot point of the point cloud.

To calculate exact time each point was shot, the T nanoseconds are added to the header time.

Note

The T field is uint32 type. The largest value it can represent is 2^32 nanoseconds, which equates to roughly 4.29 seconds. Usual point clouds don't last more than 100ms for full cycle. So this field should be enough.