YAML Specification#

Complete reference for the thread configuration file used by agnocast_cie_thread_configurator.

Top-level Structure#

hardware_info:       # Optional: validated at startup (auto-generated by the prerun mode)
  model_name: "..."
  cpu_family: "..."

rt_throttling:       # Optional: validated against kernel settings
  period_us: 1000000
  runtime_us: 950000

callback_groups:     # Required: list of CallbackGroup configurations
  - id: ...
    ...

non_ros_threads:     # Optional: list of non-ROS thread configurations
  - name: ...
    ...

kernel_threads:      # Optional: list of kernel thread configurations (auto-generated by the prerun mode)
  - comm: ...
    ...

irqs:                # Optional: list of interrupt affinity configurations (auto-generated by the prerun mode)
  - irq: ...
    ...

callback_groups#

Each entry configures one CallbackGroup's thread:

Field	Required	Description
`id`	Yes	CallbackGroup identifier (auto-generated by the prerun mode)
`policy`	Yes	Scheduling policy
`priority`	`SCHED_FIFO` / `SCHED_RR` only	Real-time priority (1–99, 99 = highest)
`nice`	`SCHED_OTHER` / `SCHED_BATCH` / `SCHED_IDLE` only	Nice value (-20–19, lower = stronger)
`affinity`	No	List of CPU cores to pin the thread to
`domain_id`	No	ROS domain ID (defaults to the current domain)
`runtime`	`SCHED_DEADLINE` only	Maximum execution time per period (ns)
`period`	`SCHED_DEADLINE` only	Period length (ns)
`deadline`	`SCHED_DEADLINE` only	Relative deadline from start of period (ns)

CallbackGroup ID Format#

The id is auto-generated and composed of the node name followed by callback identifiers separated by @:

/<node_name>@<Callback1>@<Callback2>...

Callback identifiers include: Subscription(<topic>), Timer(<period_ns>), Service(<name>), Client(<name>).

Tip

Use the prerun mode to discover the exact IDs for your application rather than constructing them manually.

Warning

If a node has multiple CallbackGroups with timers of the same period, their auto-generated IDs will collide. To avoid this, slightly offset the timer periods (e.g., 100ms and 100ms + 1ns).

Scheduling Policies#

Policy	Parameter	Description
`SCHED_FIFO`	`priority`: 1–99 (99 = highest)	Real-time, first-in-first-out
`SCHED_RR`	`priority`: 1–99 (99 = highest)	Real-time, round-robin
`SCHED_OTHER`	`nice`: -20–19 (lower = stronger)	Default Linux scheduler
`SCHED_BATCH`	`nice`: -20–19 (lower = stronger)	Batch processing
`SCHED_IDLE`	`nice`: -20–19 (no practical effect)	Lowest priority
`SCHED_DEADLINE`	`runtime`, `period`, `deadline`	Deadline-based

The scheduling-parameter key is policy-dependent: real-time policies (SCHED_FIFO / SCHED_RR) require priority, while CFS policies (SCHED_OTHER / SCHED_BATCH / SCHED_IDLE) require nice. A missing required key is rejected at startup with an error naming the required one; a key belonging to the other policy class is ignored like any other unknown key. Values outside the valid range are also rejected.

SCHED_DEADLINE#

For SCHED_DEADLINE, specify runtime, period, and deadline (in nanoseconds) instead of priority or nice:

callback_groups:
  - id: /my_node/my_deadline_group@Timer
    policy: SCHED_DEADLINE
    runtime: 1000000      # 1 ms
    period: 10000000       # 10 ms
    deadline: 10000000     # 10 ms
    affinity: [2]

Note

SCHED_DEADLINE uses cgroup cpuset for CPU affinity, which requires cgroup v1. See Kernel boot parameter for setup.

non_ros_threads#

Threads not managed by the executor can also be configured. See Non-ROS Threads for how to use this feature in your application.

Field	Required	Description
`name`	Yes	Thread name string
`policy`	Yes	Scheduling policy (same options as callback_groups)
`priority`	`SCHED_FIFO` / `SCHED_RR` only	Real-time priority (1–99, 99 = highest)
`nice`	`SCHED_OTHER` / `SCHED_BATCH` / `SCHED_IDLE` only	Nice value (-20–19, lower = stronger)
`affinity`	No	List of CPU cores

kernel_threads#

Scheduling attributes for kernel threads. A kernel thread is a thread that the kernel itself runs, such as ksoftirqd/0 or a threaded interrupt handler.

The prerun mode lists every kernel thread it finds, one entry per comm, and fills each entry with the values it observed. An unedited entry therefore asks for the state the thread was in at generation time. The configurator applies nothing while the thread is still in that state. For a pool that shares one comm, the recorded values are those of the lowest thread id.

Field	Required	Description
`comm`	Yes	Thread name from `/proc/<pid>/comm`, for example `irq/103-nvidia`
`policy`	No	Scheduling policy (same options as callback_groups)
`priority`	`SCHED_FIFO` / `SCHED_RR` only	Real-time priority (1–99, 99 = highest)
`nice`	`SCHED_OTHER` / `SCHED_BATCH` / `SCHED_IDLE` only	Nice value (-20–19, lower = stronger)
`affinity`	No	List of CPU cores
`runtime`, `period`, `deadline`	`SCHED_DEADLINE` only	Same meaning as in callback_groups, in nanoseconds

policy and its scheduling parameter go together. An entry that carries nice, priority, runtime, period or deadline without policy is rejected at startup and on every reload.

Thread names are not unique. One entry configures every running kernel thread whose comm matches. A pool of eight nfsd threads therefore takes eight sets of syscalls from one entry.

Each comm appears at most once in kernel_threads, and each irq at most once in irqs. A duplicate is rejected.

kworker/* threads stay out of the template, and the configurator rejects an entry for one. Those names carry worker-pool and CPU state that changes at runtime, so they never match a thread reliably.

If no running kernel thread matches a comm, the configurator logs a warning and skips the entry. Call the reload service once the thread appears.

Warning

The kernel fixes the affinity of a per-CPU kernel thread, which carries the PF_NO_SETAFFINITY flag. The prerun mode writes UNMANAGEABLE for such a thread. A different CPU list in its place fails on every apply pass.

Note

The prerun mode never writes SCHED_DEADLINE, because /proc does not report runtime, period and deadline. A thread whose policy the template cannot represent appears as policy: UNMANAGEABLE and priority: UNMANAGEABLE. A hand-written SCHED_DEADLINE entry works, but the configurator cannot tell whether it is already in effect, so it re-applies the entry on every pass.

Unset values#

Three forms tell the configurator to leave an attribute alone. They apply to the scheduling attributes of kernel_threads and to affinity in irqs.

Form	Meaning
Omitted key, or `~` (YAML null)	You chose not to manage the attribute
`UNMANAGEABLE`	The kernel or the configurator cannot manage the attribute
`[]` (`affinity` only)	Same as null

The prerun mode writes UNMANAGEABLE. It is valid only in kernel_threads and irqs, and an entry whose attributes are all unset appears in no reload-service array.

irqs#

CPU affinity for hardware interrupts, which is the only attribute a hardware interrupt has. A threaded interrupt handler is an ordinary kernel thread, so its policy and priority belong in kernel_threads under the matching irq/<N>-<device> name.

Field	Required	Description
`irq`	Yes	Interrupt number in decimal, as listed under `/proc/irq/`
`name`	No	Expected content of `/sys/kernel/irq/<N>/actions`, compared before any write. Omit the key or write `~` to skip the check
`affinity`	No	List of CPU cores

The prerun mode lists every interrupt that a device claims, read from /sys/kernel/irq/, and fills affinity with the value it observed. An unedited entry causes no write while the interrupt still has that affinity.

Interrupt numbers are assigned at boot and change when hardware or drivers change. name guards against that. If the current actions content differs from name, the configurator reports an error and leaves the interrupt alone. Re-run the prerun mode. Then update the file.

Some interrupts reject every affinity write from user space, such as interrupt 0 (the timer) and kernel-managed MSI-X vectors. Set affinity to UNMANAGEABLE for those entries.

Writing /proc/irq/<N>/smp_affinity_list needs CAP_DAC_OVERRIDE on top of CAP_SYS_NICE. See Grant capabilities.

Warning

Stop irqbalance before you set interrupt affinity. It rewrites interrupt affinity periodically and undoes what the configurator writes. While it runs, the configurator fails every entry that needs a write instead of writing it. See Stop irqbalance.

rt_throttling#

Optional section to validate that the kernel's RT throttling settings match expectations:

Field	Description
`period_us`	Expected value of `kernel.sched_rt_period_us`
`runtime_us`	Expected value of `kernel.sched_rt_runtime_us`

The configurator reports an error at startup if the current kernel values don't match. These values must be pre-configured via /etc/sysctl.d/ (see RT Throttling).

hardware_info#

Optional section for hardware validation. If present, the configurator compares these values against the current system (via lscpu) and reports an error on mismatch. The prerun mode automatically populates this section.

Field	Source (`lscpu` field)	Example
`model_name`	Model name	`"Intel(R) Core(TM) i9-12900K"`
`cpu_family`	CPU family	`"6"`
`model`	Model	`"151"`
`threads_per_core`	Thread(s) per core	`"2"`
`frequency_boost`	Frequency boost	`"enabled"` or `"disabled"` (not all CPUs)
`cpu_max_mhz`	CPU max MHz	`"5200.0000"`
`cpu_min_mhz`	CPU min MHz	`"800.0000"`

Full Example#

hardware_info:
  model_name: "Intel(R) Core(TM) i9-12900K"
  cpu_family: "6"
  model: "151"
  threads_per_core: "2"
  cpu_max_mhz: "5200.0000"
  cpu_min_mhz: "800.0000"

rt_throttling:
  period_us: 1000000
  runtime_us: 950000

callback_groups:
  - id: /camera_node/image_cb_group@Subscription(/camera/image)
    policy: SCHED_FIFO
    priority: 90
    affinity: [0, 1]

  - id: /lidar_node/pointcloud_cb_group@Subscription(/lidar/points)
    policy: SCHED_FIFO
    priority: 85
    affinity: [2, 3]

  - id: /planning_node/planner_cb_group@Timer(100000000)
    policy: SCHED_DEADLINE
    runtime: 5000000
    period: 100000000
    deadline: 50000000
    affinity: [4]

  - id: /logging_node/default_cb_group@Subscription(/rosout)
    policy: SCHED_OTHER
    nice: 10

non_ros_threads:
  - name: sensor_driver
    policy: SCHED_FIFO
    priority: 95
    affinity: [5]

kernel_threads:
  - comm: irq/103-nvidia
    policy: SCHED_FIFO
    priority: 50
    affinity: [6]

  - comm: ksoftirqd/0
    policy: SCHED_OTHER
    nice: 0
    affinity: UNMANAGEABLE

irqs:
  - irq: 103
    name: nvidia
    affinity: [6]

  - irq: 0
    name: timer
    affinity: UNMANAGEABLE