Static Check (Codon)¶
Before a scenario runs, the runner compiles it with
Codon, a statically typed compiler for
Python syntax, and refuses it when it does not compile. A misspelled method, a
str where a float belongs, a condition missing its label, a turn direction
passed where a lane-change direction is expected, or a YAML value of the wrong
type is reported with the file and line it is on, before CARLA is started:
$ uv run scenario scenario=intersection_passing/straight scenario.timeout_seconds=fast
autoware_carla_scenario.examples.intersection_passing.IntersectionPassingScenario: static check failed:
scenario.timeout_seconds: error: 'str' does not match expected type 'float'
The scenario still runs as Python, exactly as before: the check compiles the
same source, never runs what it compiled, and changes nothing at run time.
The CARLA API a scenario calls is
typesafe_carla, the statically
typed CARLA client this framework runs on. A scenario imports it as
import typesafe_carla.carla as carla: at run time that is typesafe_carla's
CPython package, and for the check it is typesafe_carla's Codon library, so
the same names are checked that run.
Running it¶
The runner checks every scenario registered with register_scenario() when
it builds it, so a run, a batch (scenario='lane_change/*') and every job of a
sweep are checked before anything starts. To check without running anything:
uv run scenario-check # every scenario config
uv run scenario-check scenario=intersection_passing/straight
uv run scenario-check scenario='lane_change/*' scenario.timeout_seconds=20
scenario-check composes each config exactly as the runner does, so the same
overrides apply. scenario='*' is every config, like no scenario=; a
package author checks only theirs (scenario='my_package/*', about 15 s per
config), as every built-in config takes minutes. It exits with 0 when every
scenario passed, 1 when one failed, and 2 when it could not run: there is no
Codon compiler, no config matches scenario= (the message lists the
configs), or an override names a key the config does not have
(scenario.timout_seconds=3 is reported with the closest key,
scenario.timeout_seconds; +key=value adds a new one; HYDRA_FULL_ERROR=1
shows the traceback). Every config of a glob that does not compose is
reported, each headed by its name, before it exits with 2. A config the check
could not compile at all ([SKIPPED] <config>: ..., such as a scenario with
no source file) is no pass either: with none failed, the exit status is 2
(--library likewise prints [SKIPPED] ... and exits with 2). A check that
raises (an error of the checker itself, not a verdict on the scenario) fails that config only: it is reported as
[FAILED] <config>: checker error: <exception>, its traceback goes to
stderr, the other configs are still checked and reported, and the exit
status is 1, as for any failed config (--library likewise reports
[FAILED] library check: checker error: ... and exits with 1). Its flags
are -h/--help and --library: any other argument that starts with -
(a typo such as --libary) is refused with
Error: scenario-check does not take <flag> and exit 2, rather than read as
an override, as scenario-expand refuses one; overrides (key=value,
+key=value, ++key=value, ~key) are passed on. Ctrl-C prints
scenario-check: interrupted and exits 130. A scenario package
can make the same check in its own tests, with no simulator (the template and
scenario-new packages have one in tests/test_typecheck.py):
from autoware_carla_scenario.typecheck import typecheck_scenario
def test_my_scenario_compiles() -> None:
result = typecheck_scenario(MyScenario, MyScenarioConfig, {"timeout_seconds": 10.0})
assert result.ok, result.format()
When it runs: the typecheck key¶
| Value | Effect |
|---|---|
auto (default) |
Check when a Codon compiler is installed; warn and run unchecked when none is. |
required |
Refuse to run without a Codon compiler. |
off |
Do not check. |
Set it like any other key (uv run scenario ... typecheck=required) or with
AUTOWARE_CARLA_SCENARIO_TYPECHECK, which takes precedence. A scenario
registered with register_scenario_builder() (a custom builder, such as the
packages the Scenario Editor exports) is not checked: only its builder knows
how it is constructed.
Keeping verdicts between runs¶
A process compiles a program once and keeps the verdict for as long as it
runs. The program is keyed by its shape: the driver with each config value
replaced by its type (a bool, a float, an int, a string, an enum's class),
since Codon types it statically and reads no value at compile time. The
cases of a sweep that differ only in their values are therefore one program,
compiled for the first case and reused, verdict and diagnostics, for the
others (logged as "scenario-check checks
configs side by side, and one whose program is being compiled when it
starts waits for that compile rather than starting its own. What changes the shape
compiles again: another key or order of keys (a mapping's keys too where
its values are dataclasses, whose config paths an error names), a value of another type (an
int where the first case had a float in a field not typed float, None
where it had a value), another length of a list, a mapping the config
converts in one case and not in another. A value whose literal Codon could
refuse on its own (an int beyond 64 bits, a string with an escape) is kept
as it is.
Verdicts are kept beyond the process, one file per program, in
$XDG_CACHE_HOME/autoware_carla_scenario/typecheck (~/.cache/... without
an absolute XDG_CACHE_HOME): a later process with the same inputs reads the
verdict instead of compiling. Set AUTOWARE_CARLA_SCENARIO_TYPECHECK_CACHE
to use another directory (an absolute path, ~ expanded; a relative one is
ignored with a warning, as it would put the cache wherever the check runs
from), or to an empty string to keep each verdict for its
process only (as happens anyway without a home directory, or with one that
cannot be written). A verdict read back is the program's, and is reported
for the check that read it (#652): a scenario check's line names that
scenario, and a library check's names the modules this run was given and
says cached, with the time this run took rather than the compile's.
A Codon crash is kept for its process only. A compile that timed out, or a
Codon killed with SIGKILL (the out-of-memory killer), is no verdict and is
kept nowhere: the next check of the same program, such as the next case of
a sweep, compiles it again (#694).
Nothing prunes the directory; delete it, or its files
not accessed for a while, to reclaim the space. The key covers
everything the verdict depends on. For the compile, that is the scenario's
(or the framework's) modules, the generated driver's shape, the model,
typesafe_carla's Codon library, and the compiler's version and standard
library. For the None check, which the scenario, ODD and library checks all
run, it is every Python source and stub of the framework, the checker
included (mypy reads them for the types of what a scenario calls, and the
CARLA stubs are written by its carla_stubs.py from typesafe_carla's Codon
library, already in the key), the name and version of every other installed distribution
(mypy follows a scenario's imports into numpy, lanelet2 or typesafe_carla),
the mypy version, and the Python version. Where a version says nothing of
what mypy reads, the files are in it by path, size and modification time:
the .py, .pyi and py.typed files of each distribution installed
editable and of each directory on MYPYPATH and PYTHONPATH, which mypy
searches (a relative entry is taken from the directory the check runs in;
symlinked directories are followed). Of the packages being checked, whose
modules the key has by their text already, only their stubs count there, so
an edit to one scenario of an editable or PYTHONPATH package does not
stale the verdicts on the others. And it includes what the checks made in
Python found before the compile: an import the framework's Python lacks, or
an abstract class that is built. These digests are taken once a process
(about 100 ms, more with a large PYTHONPATH such as a ROS workspace's).
mypy runs with --config-file= and in an empty directory of its own, so
neither a configuration file nor a module of the directory the check runs
from reaches the verdict. A scenario package is therefore found as an
installed or editable distribution, or through MYPYPATH: give a namespace
package (one without __init__.py) an __init__.py, as the scaffolded
packages have. What the key still leaves out: a distribution
installed in development mode without PEP 610's direct_url.json (an old
setup.py develop, a .pth file written by hand); clear the directory after
editing one.
A crash is never kept on disk (only for its process, as above), and an entry
that cannot be read counts as a miss. A verdict read from the directory has
its file's times touched, so the directory can be pruned by access time.
The test suite shares such a directory between its xdist workers. CI keeps one from run to run, so a run compiles only the programs whose inputs changed.
Builds wait for their memory¶
Codon's peak memory grows with what it compiles: about 1.9 GB for a scenario or a module of the framework, and 2.8 GB more for each million characters of source, about 14 GB for the whole library check (4.3 million characters). Builds started side by side -- the xdist workers of the test suite, the checker's own pool of four -- would otherwise together outgrow a 16 GB machine, and the out-of-memory killer would take a build (a SIGKILL, above) or, on a CI runner, the runner itself ("The runner has received a shutdown signal").
So each build first waits for its share (typecheck/memory_gate.py). The
builds of a machine record what they hold in one directory
($TMPDIR/acs-codon-gate-<uid>), and one is started when what the running
builds hold plus its own estimate fits the machine's memory (or its cgroup's
limit) less 1.5 GB, or when none is running: a build too large for the machine
still runs, alone. The holds of a process that died are forgotten, and so
are those whose pid now names another process (a hold records its process's
start time, from /proc/<pid>/stat; without /proc, only the pid is
checked). Files in the directory that are not holds are ignored. The
directory is per user; set AUTOWARE_CARLA_SCENARIO_CODON_GATE to another
directory of yours to use that one instead, or to 0 to start every build at
once.
The directory's name is predictable and sits in a world-writable one, so the
gate trusts it only if it is private: it is created with mode 0o700, opened
without following a symlink, and used only if it is a real directory owned by
you that neither its group nor others can write to. Every later use goes
through that open directory, so it cannot be swapped once checked;
gate.lock and the holds are opened without following symlinks, a hold is
created only if it does not exist, and a file that is not a regular one (a
FIFO) is ignored. A hold is counted only if it is a regular file of yours with
a single link (a hard link could make it one of your files elsewhere): one
another user owns is removed. gate.lock is trusted only if it is a regular
file of yours with a single link that no one else may open (an older gate made
it 0o644, and someone may have kept a descriptor of it). A build tries for
it for at most two minutes (LOCK_TIMEOUT_S; the gate's own builds hold it
for milliseconds), so no one holding it can stall the checks, and a lock found
replaced while it is checked or waited for is let go and the one in place
tried instead.
Only the gate's own directories (the default and the fallbacks; a configured
path that is one of them counts as one) are ever changed, and every removal
of a lock or of another user's file from one is made under an exclusive lock on the directory itself, after
checking again what it removes, so no two builds change it at once. The
gate's builds hold that lock for microseconds. No one else can open the
directory once it is 0o700, but a descriptor someone opened while it was
open to others survives; if the lock cannot be taken within a second, the
process gives that directory up for good and falls back to the next one, so
its builds stay gated. One that its group or others can use at all (as an
older gate left it, 0o755, or 0o775 under a umask of 002) is made
0o700, if it holds nothing but gate.lock and holds. On every use, the
gate's files another user owns are removed from it, and it is used only if
nothing of another user's remains and its gate.lock, if any, is a regular
file of yours, so a repair that failed once keeps falling back. A gate.lock
there that is not trusted (0o644, or a hard link, whose other file stays as
it is) is removed and made afresh by the next build, so a descriptor kept on
the old one no longer contends; the gate first tries for it for a second, so
that a build of its own inside it finishes first. So is, once per build, a
lock still held after two minutes (a descriptor opened before an earlier gate
made it 0o600 in place). A build that finds the lock replaced while it
waits waits afresh for the new one, at most twice, so no build waits more
than about six minutes in all. A build of an older gate, or one stalled
inside the old lock for over that second, may then decide while the next
build does in the fresh one: once per machine.
A directory set with AUTOWARE_CARLA_SCENARIO_CODON_GATE must be private in
the same way (yours, mode 0o700 or at least not group- or other-writable),
or it is skipped. The gate never changes it, but for gate.lock and the holds
it creates there and the holds it removes (of builds that died, of other
users): a gate.lock there that is not trusted starts the build ungated, and
so does one held for two minutes, after which the process's later builds try
for it for only a second (enough for the gate's own builds) until one takes it
again. When the directory fails these rules (or there is no temporary
directory), the gate warns and falls back to $XDG_RUNTIME_DIR/acs-codon-gate,
then to ~/.cache/autoware_carla_scenario/codon-gate; when none is usable, or
the gate's own files or the memory budget fail (a lock refused on NFS, a full
disk), it warns and starts the build ungated. The gate never holds up or fails
a check on its own account, and never hides what a build raised.
Installing Codon and typesafe_carla¶
The check needs typesafe_carla (the typesafe-carla package, from PyPI) and
the Codon compiler it pins (typesafe-carla-toolchain, 0.19, which
typesafe-carla depends on). Both ship for Linux x86_64 and aarch64, and both are
run-time dependencies of the framework anyway: typesafe_carla is its CARLA
client (see installation).
The checker finds Codon the way typesafe_carla's typesafe-codon launcher
does (it calls typesafe_carla's own lookup), so one setup serves both:
$TYPESAFE_CODON (a codon executable), the typesafe-carla-toolchain
package, $CODON_DIR/bin/codon, ~/.codon/bin/codon, then codon on PATH.
It runs it with the launcher's environment (CODON_DIR, and LD_LIBRARY_PATH
for the bundled runtime).
The check is written for Codon 0.19: a Codon of another release series counts
as no Codon at all, so auto warns and required refuses. So does a missing
typesafe-carla, and a codon that cannot be run (not executable, gone) or
whose codon --version does not answer within 60 seconds.
What is checked¶
The checker copies the scenario's own modules (the module defining the scenario class, its config class, and every module of the same package they import) into a scratch directory, next to a typed model of the framework, and compiles a small program that does what the runner does:
config = MyScenarioConfig() # then one line per YAML value:
config.timeout_seconds = 10.0 # scenario.timeout_seconds
scenario = MyScenario(ego, config=config, spawn_pose=..., ground_projection=...)
scenario.setup()
scenario.is_done()
A value for a bool, int, float or str field, at the top level as in a
nested dataclass, is first passed to a function that takes that type only,
on a line of its own (#598): Codon converts any value with __bool__ that is
assigned to a bool field, so scenario.enabled=no (which Hydra, like a
quoted "false" in YAML, reads as the str "no") would otherwise pass, and
the feature would be on. It is refused at scenario.enabled, whether the
field has a default or not. An int for a float field still passes.
Codon checks a function only when something calls it. The program checks
everything setup() and is_done() reach: the calls into the framework, the
scenario's own helpers, and, through register_pass_condition() and the
other register_* methods, the check() of a custom condition and the
execute() of a custom action.
Then it compiles the rest of the scenario's own modules too (#192). Each module the checker copies gets the library check's function appended (What is compiled), and the program calls it after the lines above:
def _acs_library_check(): # appended to my_scenario.py
MyScenario(_acs_value(EgoConfig), _acs_value(Lanelet2Pose), ...)
_acs_value(MyScenario)._goal_ids(_acs_value(int)) # a helper setup() never calls
_acs_value(MyScenario).initial_speeds_mps(_acs_value(Optional[carla.Actor]))
unused_probe(_acs_value(carla.World)) # a module-level function
_acs_cls_default_9: int = "three" # MyScenario.RETRIES: int = "three"
So an error is reported wherever it is in those modules, not only on the path
setup() takes: in a private helper or a public method nothing calls yet, in
a hook override the runner calls later (initial_speeds_mps()), in a
module-level function, in a class-level default, or in the config class's
methods. Each call passes a value of each parameter's annotated type, and a
defaulted parameter's default is compiled by one more call that leaves it out.
The result of a hook override is checked against the framework's type
(#238). Codon compiles an override against its own annotation only, and the
framework code that calls it is a stub of the model, so check() -> bool,
initial_speeds_mps() -> list[float] or is_done() -> str compiled, and
failed or misbehaved when the scenario ran. So each appended call of a method
passes its result to the model, which checks it strictly against the return
type of the method it overrides, when the class derives from BaseScenario,
BaseCondition, BaseAction or EgoVehicle (directly, or through a built-in
condition or another class of the scenario's modules):
_acs_hook_return(MyScenario, "is_done", _acs_value(MyScenario).is_done())
_acs_hook_return(GapCondition, "check",
_acs_value(GapCondition).check(_acs_value(carla.World), _acs_value(float)))
Every method and property each of the four classes declares is such a hook,
with the return type the model declares for it (test_typecheck_model.py
keeps the two in step): setup() returns None, is_done() a bool,
initial_speeds_mps() a Dict[int, float], check() an
Optional[ScenarioResult], execute() None, debug_markers() a
DebugMarkers, and so on. The check is strict where an annotated assignment
is not (Codon 0.19 converts any value with __bool__ in done: bool = x):
an override must return the type itself or what Python's typing takes for
it -- a subclass of a class, an int for a float, a bool for an int,
and, for an Optional[X], an X, an Optional of one, or only None (a
condition that never decides). A str, an int or an Optional[bool] for a
bool is refused, at the override's def line:
scenario.py:62: error: BaseScenario.is_done() returns bool, and this override returns another type
reached by <scenario check: MyScenario.is_done()>
during the realization of _acs_expect_return(r: str, T: bool, ...) [scenario_base.codon:230]
A method of a class deriving from none of them, or one its base does not
declare (a helper of the scenario), is not checked this way, and neither is
an abstract one (@abstractmethod) or one that only raises
(raise NotImplementedError()): it returns no value, which Codon types
None whatever its annotation says. The program
also checks what the scenario's own setup() and is_done() return, after
the module checks, for one the appended calls leave out.
A wrong call or return is reported in the author's terms (#527, #528,
529, #530). Codon says `no function 'SetSpeedAction.init' with arguments¶
(SetSpeedAction, EntityRole, str, str)for a call it cannot match, and'float' does not match expected type 'str'(the two types swapped) for areturnof the wrong type. The check reads the call or thereturn` in your
file, binds it to the real Python signature of the class, method or function,
and says what is wrong, at the argument:
scenario.py:28:62: error: SetSpeedAction(): target_speed_kmh: expected float, got str
scenario.py:28:60: error: TimeoutCondition(): 'lable' is an invalid keyword argument (did you mean 'label'?); TimeoutCondition takes (timeout_seconds: float = 60.0, *, label: str)
scenario.py:36:9: error: val() returns str, but the function is declared -> float
A message that cannot be tied to a parameter is X() takes (...); got
(EntityRole, str, label=str), with self dropped and a keyword argument
marked. A static string type prints as 'str' (not ''3''), the frames of
the model's own helpers (_keyword_only, _required_str, actions.codon)
are left out of the trace, and a literal None given where a type is
expected is told that the parameter does not take None rather than to test
the value.
Some common slips get a hint after Codon's message (#599), each only where both the message and your source at the line say it is that slip:
| You write | Codon says | The hint |
|---|---|---|
range(len(ids) / 2) |
range() cannot be called with (float) |
range() takes ints, and / always gives a float: divide with //, or convert with int() |
ids[len(ids) / 2] |
__getitem__() cannot be called with (float) |
an index must be an int: divide with // |
",".join(ids) of a list[int] |
'Generator[int]' does not match expected type 'Generator[str]' |
",".join(map(str, ids)) or ",".join(str(v) for v in ids) |
self.register_pre_tick(lambda: None) (also register_post_tick, register_init, a register_measure read function, a register_cover expression, or a def of no parameter) |
a lambda() takes 0 arguments (1 given) |
the callback is called with the world: lambda world: ... |
self.register_measure("flag", lambda w: True) |
expected a float (or an int) |
names the measure and what its read function returns (a bool is no number to a measure: 1.0 if ... else 0.0, or a cover) |
A range() or index of a float without a / written there, and a join of
something that is not a call of join at that column, get none.
Two declared return types Codon does not enforce are refused from the syntax,
before Codon runs (#529): a return of a literal, an f-string or a display in
a function declared -> bool (Codon converts it to a bool; Python returns
the str), and a function declared to return a type other than None,
Optional[...], Any or NoReturn whose body can run off its end. The flow
analysis follows if, while, for, try, with and match; a while
True without a break, a raise, assert False, sys.exit() and a call of
a function declared NoReturn end a path, and a body that is only ...,
pass or a docstring (an abstract method, an overload) is not checked. A
returned expression whose type is not obvious from its syntax
(return some_str) is not judged.
Then mypy checks that an Optional is tested for None before it is
used (#239). Codon unwraps an Optional[T] implicitly wherever a T is
needed, so cfg.min_speed_kmh / 3.6 of a float | None field compiled, and
raised TypeError in Python when the field was None. After the compile,
the checker runs mypy (a dependency of the framework) over the scenario's own
modules, following imports silently, checking the bodies of unannotated
functions too, and with no configuration file of the package's (so the
verdict does not depend on how strict the package's own mypy settings are),
and keeps only what it reports of a value that may be None used where it
may not be: an attribute of it, an operator, an argument, an assignment, a
return, an index, or an item of a list or dict display (mypy's
union-attr, operator, arg-type, assignment, return-value, index,
call-overload, list-item and dict-item; a set display's item is an
arg-type to mypy), each only where
None is the offending type (#267): the None item of a union an attribute
is missing on (Item "None" of "Waypoint | None"), an operand, the value
indexed or an argument that is None or a union holding it, or a value whose
type holds None given where the type expected does not (incompatible type
"float | None"; expected "float"), the key or the value of a dict entry
included (Dict entry 0 has incompatible type "str": "float | None"; expected
"str": "float"). An item matters because Codon unwraps it where the model
iterates the sequence (#308): register_cover(..., buckets=[0.0, 10.0,
cfg.min_speed_kmh]) and register_cross("c", ["speed", maybe_name])
compiled, and failed in Python on the None; [0.0, x or 0.0] passes. A message that only mentions None
inside another type or in the type expected is not one: a reader returning
Optional[str] given to register_measure(), which takes one returning
Optional[float], is a str for a float, Codon's error to report, and a
list[float] for a list[float | None] is mypy's invariance, which Codon
accepts. An assignment that only declares is not one either: a None default for
a non-Optional annotation (speed: float = None), or a value assigned to an
attribute mypy inferred as None from its first assignment. Each finding
fails the check at its line, after Codon's errors:
scenario.py:67:28: error: Unsupported operand types for / ("None" and "float"): Codon unwraps an Optional implicitly, so test the value for None before this use, or Python fails here when it is None (mypy: operator)
Left operand is of type "float | None"
The same pass refuses a framework union used as one of its members
(#309). Codon holds TrajectoryVertex.position (a TrajectoryPosition, the
union of the ten poses) as a Union, and converts it to whichever member a
use needs at run time (No attribute of a union, below), so
v.position.lanelet_id of a MapPose vertex, or v.position passed to a
parameter typed Lanelet2Pose, compiled, and raised AttributeError in
Python. mypy reports both (union-attr, arg-type), and the pass keeps
them where the union mypy names is made of the members of one of the unions
the model holds: each module-level Name = Union[...] alias of
typecheck/codon/autoware_carla_scenario/ (TrajectoryPosition, AnyPose)
and each property it declares to return a Union[...]
(TrajectoryVertex.position, CoverItem.event), read from the model, so
the list follows it (none_check.framework_unions();
test_typecheck_none_check.py checks each against the union Python
declares). mypy reports a missing attribute once per member lacking it; the
check reports it once, naming them all, with its own advice:
scenario.py:71:27: error: Items "CarlaWorldPose", "OpenDrivePose", "MapPose", ... of "CarlaWorldPose | Lanelet2Pose | ... | RouteRoadsidePose" have no attribute "lanelet_id": Codon holds this union of the framework's (TrajectoryPosition) and converts it to the member this use needs at run time, so narrow it with isinstance first, or Python fails here on any other member (mypy: union-attr)
Narrowed code passes (if isinstance(p, Lanelet2Pose): p.lanelet_id), and
so does the union passed on where the union is taken
(TrajectoryVertex(v.position, advance=v.advance), a TrajectoryPosition
parameter).
A field of a frozen dataclass of your own is read-only (#567). Codon compiles
a frozen dataclass as an ordinary class, so mypy's verdict is used: p.x = 2.0,
p.x += 1.0, self.x = ... in a method of the class, a loop variable's field
and a frozen subclass's field are refused as Pt is a frozen dataclass; use
dataclasses.replace(x, x=...), where the receiver's type is known to mypy (a
value of unknown type, Any, is not). dataclasses.replace and
object.__setattr__(self, ...) in __post_init__ stay valid.
The class is found by its name: a name is treated as a frozen dataclass only
where every class of that name in the scenario's modules is one (a same-named
plain class with a read-only property is left alone, at the cost of a missed
frozen assignment). The decorator may be @dataclass(frozen=True, ...),
@dataclasses.dataclass(frozen=True) or an aliased import. This rests on
mypy, as the None check does: without it the check warns the None check did
not run and a frozen assignment is not reported. setattr(p, "x", 1) is not
caught.
bytes and str do not mix (#574). Codon's bytes is a str, so
b"x" + "y", "sent " + name.encode(), a bytes returned for a -> str,
passed to a str parameter or assigned to a str, "x" in raw and
raw == "ab" compile, and Python raises TypeError (or the comparison is
always False). They are refused with mypy's verdict, only where the offending
pair is exactly bytes and str: bytes and str do not mix in Python: decode
with .decode() / encode with .encode(). A str | bytes union, a bytearray
and every other incompatible type are left alone, and b + b, s + s and
.encode() used as bytes stay valid.
A comparison that is always False is refused (#575). The None check runs
mypy with --strict-equality and keeps its comparison-overlap findings: an
Enum member compared with its raw value (Color.RED == "red", mode == 1,
mode in (1, 2)) or with a member of another enum is False in Python, and
Codon compiles it. The advice is an Enum member never equals its raw value:
compare with Color.RED, or use Color.RED.value. Only the findings with an
enum as an operand are kept, not every comparison-overlap: mypy also reports
two different literals (MODE: Final = 1 and if MODE == 2), a deliberate
switch. The enum is found among the classes of the scenario's own modules (an
enum imported from elsewhere is not recognised, so its non-literal comparisons
are missed). mypy skips a class that defines __eq__, and IntEnum, StrEnum and (str, Enum) overlap their raw
value in Python as well, so Level.LOW == 1 and Phase.GO == "go" are valid.
Color.RED.value == "red" is the valid spelling. A member written out as a
literal (Level.LOW == 1) is judged by mypy on identity, so it is kept only for
a plain Enum / Flag defined in the scenario's own modules, and two members
of one enum (Color.RED != Color.BLUE) are not a comparison with a raw value. The flag adds no other
error, so the None check's other findings are unchanged; the framework's
modules and the built-in scenarios have none of these comparisons. Like the
rest of this section it rests on mypy.
A string formatting Python fails on is refused (#589). The None check
keeps mypy's str-format findings of a str.format call or a % on a
literal format string where Python raises at the call: a conversion the value
does not take ("{:d} km/h".format(speed) for a float and "{:.2f}".format(name)
for a str raise ValueError, "%d km/h" % name and "%x" % speed raise
TypeError), too few or too many arguments ("%s %s" % (name,),
"{} {}".format(name)), a missing named field or mapping key, an unknown
format character or conversion ("%q", "{!x}"), a malformed spec
("{:abc}", an unmatched {), and mixing automatic and manual field
numbering. Codon formats with its own rules and compiles all of them. The
advice is Python raises ValueError, TypeError, IndexError or KeyError on this
format string with these values (...); convert the value first (int(x),
str(x)) or use "{}" / "%s". "%d" % speed (a float) and "{:.2f}".format(n)
(an int) are valid, as in Python. A conversion the value does not take is
kept only for a built-in type that fails for every value (str, float,
None, bytes, complex, a list, ...): mypy does not count a class's
__index__ ("%x" % obj runs) or a __getitem__ without keys
("%(k)s" % obj runs), and refuses "{:.2f}" of a complex or a
Fraction, which run. Too many arguments is kept only for a % whose right
operand is a tuple display or a constant: "{}".format(a, b) ignores the
extra argument, and "abc" % d formats no key of a dict (or of a list, or
bytes). An unknown spec is kept where the Python running the check refuses
it ("{:z.1f}" runs on 3.11). Three more of mypy's str-format messages are
not kept, because mypy gives them to valid code as well: Numeric flags are
only allowed for numeric types ("{:05}".format(name) runs; "{:+}".format(name)
does not, and the message does not say which flag), Syntax error in format
specifier and the field-accessor ones ("{0[my key]}".format(d) and
"{0['k']}".format(d) index d with "my key" and "'k'"). An f-string's
spec (f"{speed:d}") is not checked: mypy does not check it, and rewriting it
into a .format call for mypy would move the columns mypy reports. Like the
rest of this section it rests on mypy.
Typing contracts you declared must hold (#577). Python mostly does not
enforce these and Codon does not read them, so mypy's verdict is used, each
kept only on its exact code and message (no other misc, override, index
or attr-defined error is a finding; Codon reports attribute errors itself):
Finalassigned again (MAX_SPEED: Final = 30.0, thenMAX_SPEED = 40.0,self.LIMIT = 40.0,Cls.LIMIT = 1.0or aFinallocal rebound):a name declared Final is bound once; .... Bind a new name or dropFinal.- A
ClassVarassigned through an instance (self.count += 1,c.count = 5):a ClassVar is shared by the class; .... Python creates a shadowing instance attribute and Codon treats the annotated name as an instance field, so what runs can differ. Assign through the class (Cls.count += 1) or dropClassVar. - A method incompatible with the one it overrides: a narrowed argument
type, a different return type, a required parameter added, or a parameter
removed (
the override must accept what the overridden method accepts ...). This covers the framework's hooks too:BaseScenario.is_done(self) -> booloverridden asis_done(self, strict: bool)is refused, since the framework calls it with the base signature. An extra parameter with a default, or a wider argument type, is fine, and# type: ignore[override]on thedefline (mypy's own) silences it. The dunder methods are not checked (__eq__(self, other: Pt)is common and benign), nor is a method overridingobject's. The built-in scenarios and the template produce none. - A
Mapping,SequenceorAbstractSetmutated (m["k"] = 1,del m[k],s.append(3),s.sort(),m.update(...),a.add(1)):... is read-only by its annotation; copy it first .... Only an attribute that mutates (append,extend,insert,pop,remove,sort,reverse,clear,update,setdefault,popitem,add,discard, the set*_updatemethods,__delitem__) on a receiver typed exactly as one of those three counts; a typo (s.appnd) or a missing attribute of another type is not reported here. Import them fromtypingorcollections.abc(read astyping's, #586; itsSetisAbstractSet): the check is the same.
Not checked: a name bound on one path only (#576). Python raises
UnboundLocalError for if c: y = 1 followed by use(y), and mypy reports it
only with --enable-error-code possibly-undefined. That code is not enabled:
mypy's flow analysis cannot see that the branch that binds a name and the
branch that reads it test the same condition (if x is not None: v = ...
then if x is not None: use(v)), nor that a for over a non-empty constant
runs, so valid code is refused and the cure is to restructure correct code.
Run over the framework's 313 modules it reports 15 times in 11 modules: 9 are
a condition tested twice or a try/with that binds on every path that
reads, 4 are a loop mypy cannot see always binds (a for over a non-empty
constant, a for/break/else), and 2 are latent edge cases in the
framework (an unbound exit_code when server.cooldown_max_retries is
negative, and an unbound port_file in slurm/worker.py's finally when
layout.task_port raises after the port slot was taken), not scenario
patterns. Probes of
scenario-style branches showed the same: if/else, an exhaustive enum
if/elif, try/except that raises or returns, match with a default and
for/else pass, but a loop variable read after a for and a guard tested
twice are refused. Bind the name before the branch when you mean to read it
later.
The CARLA API's Optional results are checked too (#267).
typesafe_carla's CPython package is a prebuilt extension with no .pyi and
no py.typed, so mypy alone reads typesafe_carla.carla as Any. The
checker gives mypy stubs of it instead, written from typesafe_carla's own
inventory of its Codon library (typesafe_carla.pycarla.Inventory, which the
CPython package is generated from; typecheck/carla_stubs.py) and put on
MYPYPATH: every public class with its bases, and every method and property
with the return type the library declares. So Map.get_waypoint(),
ActorList.find(), World.get_actor(), World.try_spawn_actor(),
Actor.parent, Vehicle.get_traffic_light(), Waypoint.get_left_lane(),
Sensor.poll() and the library's other Optional results are Optional to
mypy, and a use without a test for None is refused like a float | None
config field:
scenario.py:76:16: error: Item "None" of "Waypoint | None" has no attribute "transform": Codon unwraps an Optional implicitly, so test the value for None before this use, or Python fails here when it is None (mypy: union-attr)
get_waypoint() is the one exception to the library's declarations: it
returns None when no lane of lane_type matches, and projected onto the
road (project_to_road=True, its default) that is only on a map with no
such lane at all, so the stubs give that call a Waypoint (the framework
uses it unguarded too). With project_to_road=False, or a flag mypy cannot
read as True, it is Optional. The stubs type only what the None check needs: parameters are
*args: Any, **kwargs: Any (the arguments are Codon's to check), and a
return type mypy cannot be given (a pointer, a type parameter, overloads
returning different types), an enumeration, carla.command and any name not
written are Any, so they add no error of their own. They are written once
per library, under ~/.cache/autoware_carla_scenario/carla-stubs/ (named by
a digest of the library's Codon sources), or into a temporary directory where
that cannot be written; where typesafe_carla's library cannot be read, mypy
runs with CARLA untyped, as before.
Narrow the value first, as mypy reads it: if x is not None:, assert x is
not None, an early return when it is None, or a default (x or 0.0).
mypy keeps a cache in ~/.cache/autoware_carla_scenario/mypy (under
$XDG_CACHE_HOME when it is set), so only the first check reads the
framework (about ten seconds); where that directory cannot be written, as in
a container with a read-only home, mypy runs without a cache. A scenario
package installed as a wheel (not editable) is checked too: mypy would take
site-packages for its source root, so the check gives it the package
through a link in a directory of its own. When mypy
gives no verdict (it ran out of time or crashed), the check keeps Codon's
verdict and says so in a warning:
warning: the None check did not run (mypy did not finish within 600s), so an Optional used without a test for None is not reported
The error is reported at the scenario file's own line. One in a function's body is at that line, with the call that reached it in the trace:
scenario.py:88:16: error: 'MyConfig' object has no attribute 'timeout_secs'
during the realization of _helper(self: MyScenario) [<scenario check: MyScenario._helper()>, scenario.py:87]
One in what the appended line compiles itself is at the line of the
class-level default, or, for a parameter's default, at the def of its
function, and the trace names the call
(reached by <scenario check: class-level default of MyScenario.RETRIES>).
A function the check cannot call, because a parameter has no annotation, one
Codon cannot express (Any, object, Callable[..., R]) or one naming a
TypeVar (a generic function), is not refused: it is left out of the
appended calls with a warning, and compiled where setup() and
is_done() reach it, generically, as before. So is one whose return
annotation names Any, such as a custom condition's
get_details() -> dict[str, Any] of mixed value types, which Codon cannot
build (the library check lists the framework's own as uncalled):
my_scenario.py:40: warning: MyScenario._scaled: parameter `x` has no annotation; the check does not call it on its own, so it is compiled only where the scenario's constructor, setup() and is_done() reach it
A helper with a **kwargs or a *args that the scenario calls is
compiled through those calls (#266). The appended calls give a *args: T
one T and a **kwargs no keyword at all, and Codon types each from the
call: a Tuple[T], an empty NamedTuple. So def opts(**kw: float) reading
kw["speed"] was refused ('NamedTuple' object has no attribute 'speed',
reached by <scenario check: opts()>) although setup() calls
opts(speed=2.0), and xs[1] of a *xs: float likewise. Such a function
whose body reads what they hold (kw["speed"], xs[1], a, b = xs,
g(*xs), g(**kw)) and that a function, a class body or the module's top
level of the scenario's modules calls (by its name: opts(...), self.scale(...),
helpers.opts(...), Tuner(...) for its __init__; its own recursive call
does not count) is not called again: its callers' calls compile it with what
they pass. One whose body does not read them (kw.get("speed", 1.0),
sum(xs)) is called as before: a call is known by its name only, so a
gains.update(p=1.0) would otherwise leave a helper update to a call of
dict's, and nothing would compile it. One nobody calls is still called, so
an error in it is found,
unless it subscripts its **kwargs, which no call without a keyword
compiles: that is left out with the warning above (MyScenario._opts: reads**kwby key, and nothing in the scenario calls it: the check has no keyword
to call it with).
A warning does not fail the check (result.ok stays true; they are in
result.warnings), and scenario-check and the runner print them. Annotate
the parameter to have the function checked on its own. Refusing such a
function would have refused a scenario that passed before, for a helper that
compiles wherever it is called. No built-in example, the scenario package
template or a scaffolded package has one.
An ODD written in Python (ODD) is checked the same way: the program
calls its builder (odd: OddDefinition = urban_odd()). Every attribute's
probe is then called with a world, and every condition must come from an
attribute. The ODD's own modules have every function compiled as a
scenario's do, with the same warning for one the check cannot call (compiled
where the builder reaches it). The runner checks the ODD named by the odd key under the same
typecheck mode, and scenario-odd check does it without running anything:
it exits with 0 when every ODD passed, 1 when one failed (a checker error is
reported as [FAILED] <odd>: checker error: ..., its traceback on stderr, and
the other ODDs are still checked), and 2 when one could not be compiled, for
want of Codon or because the check cannot compile it ([SKIPPED] ...).
The model¶
autoware_carla_scenario/typecheck/codon/ holds Codon declarations of
everything a scenario imports from autoware_carla_scenario: the conditions,
actions, poses and coordinate functions, entities, BaseScenario, the shared
config dataclasses, with their static types. The CARLA API is not modelled:
the checker puts typesafe_carla's Codon library next to the model, the way
typesafe-codon puts it on CODON_PATH, and import typesafe_carla.carla as
carla (the import the runtime uses) resolves to it, so a scenario is checked
against the whole typed CARLA API.
typesafe_carla's Python-API compatibility shortcuts (such as calling a
Vehicle method on a plain Actor) compile, as they do outside strict mode.
Nothing in the model runs; test_typecheck_model.py keeps each declaration
it compares in step with the Python definition of the same name (parameter
names, order, defaults, keyword-only parameters, methods, enum members): every
name __init__.codon exports, the names the checked modules or a scenario
import from the model beside those (_BOUNDARY), and every name of the model
modules that stand for one framework module each (_MODULE_MODELS), which
must declare exactly the names compared, beside model-only helpers named
_acs_* (the list below). metrics.codon and measures.codon are two of
those: each declares the whole of its module's
__all__, and the test also compares by value their constants (the keys,
VEHICLE_AHEAD_VERSION, the distances and speeds the vehicle-ahead measures
read with), each built-in entry of BUILT_IN_METRICS (title, unit, text,
aggregation) and of BUILT_IN_MEASURES (unit, text, version), the field types
and defaults of Metric, Measure and Bumpers (frozen dataclasses, so
@tuples: assigning to a field fails the check, as it fails the run), and
the parameters of Metric's and Measure's __new__. Only a built-in's
read differs: Python's is a private function of the module, which no
scenario can import, and the model's a stub (_acs_unread), since nothing in
the model is read. Metric(), Measure() and
BaseScenario.register_measure() take any read a world can be passed to
that returns a float, an int, None, or an Optional of a float or an
int -- a function returning float, lambda world: None -- as Python's
do, and refuse any other return type (a str, say) at the line that passes
read, with "expected a float (or an int)". The three share that check
(_reader.codon, outside measures.codon, which imports scenario_base),
and Measure is defined there too, so BaseScenario.measures can return a
Dict[str, Measure]; measures.codon re-exports it.
Metric() and Measure() store the stub: a Codon Callable field takes a
function of its exact type only, and Codon 0.19 crashes on any other. set_measured_scenario() takes a scenario of any
BaseScenario subclass or None, which Python annotates
Optional[BaseScenario]: Codon's Optional takes no subclass.
register_pre_tick(), register_post_tick() and register_init() take an
Optional action or callable (#537): Codon does not narrow an Optional
after if a is not None:, so the value registered there is still one, and the
model unwraps it (None itself is refused). A reader that is maybe_float or
0.0 is a Union for Codon and passes the reader check (#538). An Optional
of a subclass held where its base is declared (c: Optional[BaseCondition] =
ElapsedTimeCondition(...), or an Optional[BaseCondition] helper parameter)
is Codon's to refuse, in the scenario's own code (#541): the message says to
declare it without Optional. to_lanelet2() is one untyped definition, as
to_carla_world() is, because Codon tries only the last of two typed
overloads; test_typecheck_model.py fails on a model function defined twice.
An actor is checked by kind for the same reason (#278).
follow_with_spectator() and log_actor_position() take a getter Python
types Callable[[], Optional[carla.Actor]], and initial_speeds_mps() an
Optional[carla.Actor]. Each takes a carla.Actor, any subclass of it
(carla.Vehicle, carla.Walker, carla.Sensor, ...), an Optional of
one, or None. A getter returning a carla.Vehicle is the natural one to
pass, because typesafe_carla keeps a vehicle's controls on carla.Vehicle.
Codon 0.19 refused an Optional[carla.Vehicle] for an
Optional[carla.Actor], so these parameters are generic in the model. The
model's _expect_optional_actor unwraps the Optional and upcasts a class
of a hierarchy to carla.Actor, the way a hook result is checked. Anything
else is refused at the call, the framework's EgoVehicle (lambda:
self.ego_entity) included:
scenario.py:70:9: error: BaseScenario.follow_with_spectator(): the getter must return a carla.Actor (or a subclass of it, such as carla.Vehicle), or None
register_cover()'s values= and ignore=, and OddAttribute(values=),
are checked as openodd uses them (#279). values= must be None, an
iterable (a list, a tuple, an empty one too, a set, a str, a range) or an
enum class, the framework's (values=TurnDirection) or the scenario's own.
Nothing is iterated: the enum class is read as a value of the type, not
looped over (see Iterating an enum).
ignore= must be None or a callable that takes one value of the type
expression returns, unwrapped from its Optional, since the collector never
passes None. The model calls it with such a value, so its body is compiled
too. A wrong ignore was the worse gap. The collector calls it inside a
try, and counts an exception as "cannot bucket ... counted as no sample",
so a wrong ignore emptied the item for the whole run, with one warning
line. What ignore returns is tested for truth, which Python takes of any
value, so the result is not checked:
scenario.py:68:9: error: BaseScenario.register_cover(): ignore must take one argument, a value the expression returns
scenario.py:68:9: error: BaseScenario.register_cover(): values must be an iterable of the bucket values (a list, a tuple or an enum class), or None
What test_typecheck_model.py compares, model by model:
- Every name
__init__.codonexports, with the object of the same name in the package. coordinate.codon: beside its exports,lanelet_length,on_lanelet,project_onto_road,road_traffic_rule,project_onto_lanelet,get_stop_line_linestrings,get_stop_line_linestrings_with_following,find_traffic_lights_for_lanelet2_id,get_signal_ids_for_controller,to_map_frame,MapManager,map_origin,resolve_projector,_parse_geo_reference, and pyxodr'sRoadNetworkand lxml's element (below).conditions.codon: beside its exports,ActionStateCondition,LaneChangeSettledCondition,compare,beside_lane_id, andConditionStatus, all but itsdetails(adict[str, Any], which Codon cannot express:_OMITTED_PARAMETERS), and no other name it defines (find_actor_in_listit imports fromentity_role.codon, as Python'sconditions.basedoes).ActionStateCondition'sstateis anActionStateor its OpenSCENARIO name, and itsactionsany dict of action ids to actions:actions.codonimportsconditions.codon, so an action is checked by whatBaseActiondeclares.entity_role.codon: besideEntityRole,find_actor_in_list(acarla.ActorListor a list of actors, as Python's) andfind_actor_in_world.kinematics.codon: beside its exports,normalize_angle_deg, andframe_ofandFrameMismatchErrorascoordinate.codondeclares them. Each class declares the arithmetic operators Python defines, each with Python's result for that operand (AbsoluteVelocity - AbsoluteVelocity -> RelativeVelocity, #2259); an operator whose operand is a class defined further down the model (AbsoluteVelocity + RelativeVelocity) takes it untyped, dispatches onisinstance()and refuses any other operand with acompile_error.test_typecheck_model.pycompares the operators and their results with Python's.coordinate.codon:AnyPose, the Union alias, by its members' names, in order, for a scenario's own annotations (no model function takes it: a Union parameter crashes Codon 0.19, so each is generic or overloaded).constants.codon: every constant, by value (EGO_ROLE_NAMEasEntityRole.ego()), and no nameconstants.pylacks.route.codon: every name it declares --parse_anchor,scenario_route,scenario_route_frame,RouteFrame,RouteMatch, andAPPROACHES,CROSSWALK_LEGSandSIDESby value.RouteMatch's constructor is generic: Python's is a frozen dataclass of variadic tuples (tuple[int, ...]), which Codon cannot express, but the model's takes the same parameters, and an overload of none is acompile_error(Codon builds a class without fields with no arguments otherwise; see What Codon builds by the fields below), so a call Python refuses fails the check too.trajectory.codon: beside its exports,ResolvedTrajectory,TrajectorySampleand theTrajectoryPositionUnion, by its members' names, in order, and no other name.traffic.codon: every name it declares --TurnDirection,LaneChangeDirection,TrafficContextand traffic.registry's functions andTRAFFIC_BACKEND_ENTRY_POINT_GROUP(by value).TrafficBackendFactorydiffers on purpose (_DIFFERING): aCallablealias in Python of a mapping of any values (Hydra's options), which Codon has no type for, and an empty class in the model, there only so the checkedtrafficpackage's re-export resolves; a test checks that it stays empty.utils/traffic_light.codon: its three functions, and no other name.coverage.codon,entity.codon,odd.codon: the names the checked modules import (_BOUNDARY),COVERAGE_SCHEMAandCoverGroupby value.- The models of one framework module each (
_MODULE_MODELS):_bridge_pose,action_state,debug_markers,trajectory_recorder,environment,autoware_bridge/grpc_server,autoware_stack/launcher,autoware_stack/devcontainer(and its values),driver/egodriver_client,_map_source,maps/cache,maps/z_offset_cache,maps/road_network_cache,pedestrian/soma,ui/models,_ros2,metrics,measures,scaffold/generator,commonroad/session,commonroad/worker,hils/configandutils/traffic_light, each declaring exactly the names compared;ros2/cyclonewhere cyclonedds is installed. - Every public name of a modelled module: each name a module the model
has exports in Python (its
__all__, or the public names it defines when it has none) is declared by the model or listed, with the reason, intypecheck/not_modelled.py(NOT_MODELLED), and a name listed there must be neither declared nor gone from Python. Each declared one is compared as above, re-exports included (kinematics.frame_of).OddSampleandOddCasediffer on purpose (_DIFFERING): an opaque, unbuildable stand-in for openodd's sample, and the@tupleof the pair Python's alias names. - Every public member of a compared class (#265): the comparisons above
check that what a model class declares exists in Python; this checks the
converse. For every class they compare, enums included, each public
method, property, class attribute, dataclass field and instance attribute
the package's Python, or openodd's, defines on the class or a class it
derives from
(
VehicleEntity.set_desired_speed()isBackendDriven's) is declared by the model class or one of its model bases, or listed, with the reason, inNOT_MODELLED_MEMBERS(typecheck/not_modelled.py), under the class or the Python base that defines it (BaseCondition.get_detailsfor every condition). openodd's classes, which the package re-exports (OddAttribute,OddModule,OddDefinition,OddCondition,CoverItem,CrossItem,CoverageReport), are compared as the package's are (#298), each listed under the name the package gives it (openodd'sConditionisOddCondition). A listed member must still be public in Python and absent from the model. A dataclass's public field is a member too, beside the constructor's parameter it is compared as (#298): the model declares it as a field or, where a field would change the constructor Codon generates (What Codon builds by the fields, below), as a read-only property (RouteMatch.lanelet_ids, aList[int]for Python'stuple[int, ...];TrajectoryVertex.position, the union of the ten poses, which narrows withisinstanceand passes on to a newTrajectoryVertex, and which the None check's pass refuses un-narrowed where one pose is expected, #309;CoverItem's fields). A frozen dataclass's field is read-only in Python too; writing aCoverItem's is refused, which Python allows, and a scenario, which registers its items withregister_cover(), has none to write. An attribute a method sets on the instance (self.<name> = ...orsetattr(self, "<name>", ...), read from the class's source,__init__'s and the others', openodd's included; one set under a name computed at run time is not found) is a member too (#307), as Python reads it like any other: the model declares it as a read-only property, since a field would change the constructor Codon generates (TimeoutCondition.timeout_seconds,OddDefinition.attributes,OddAttribute.targetanditem,TrafficSinkAction.removed,RouteFrame'smatch,map,graphandlanelets). Writing one is refused, which Python allows ('TimeoutCondition' object has no attribute 'timeout_seconds'), except in the__init__of a scenario's own subclass, where the assignment declares the subclass's own attribute. An enum is compared beyond its members (#307): a model enum is a@tupleof its value, so Codon gives it noEnum, and it declares Enum'snameas a property, a chain ofif self == Rule.MEMBER: return "MEMBER"thattest_typecheck_model.pycompares with each member's Python name, in order (it also compares each member's value), and its own methods and properties (ComparisonRule.text,CollisionTargetType.matches(),LaneChangeDirection.to_carla_bool()). A scenario's own enum gets itsnamefrom the check's rewrite (below). The scenario check adds the reason to Codon's error:
scenario.py:68:19: error: 'CaseScenario' object has no attribute 'traffic_backend': BaseScenario.traffic_backend is not part of the Codon model, on purpose: takes or returns the run's TrafficBackend, ...
Left out are what returns or takes a value Codon has no type for (a
dict[str, Any]: a condition's get_details() and to_summary_dict(),
to_dict(), describe(), RouteMatch.to_config() and from_config()),
the run's traffic backend (set_traffic_backend(), traffic_backend),
RouteMatch.junction(), junctions and segments (a
RouteSegmentMatch), a field of Any (ConditionStatus.details,
RouteLanelet.values, OddModule.metadata, SweepConfig's Hydra
mappings, CoverItem's expression, ignore and values,
OddAttribute.probe), a class held as a value (BaseScenario.ego_type:
a scenario reads ego_requires_goal and calls create_ego()),
RouteFrame.features (route.geometry's queries, which the library check
compiles from their source), openodd's
evaluation, sampling and archive of an ODD (OddDefinition.evaluate(),
sample(), archive(), ..., OddCondition.evaluate()) and its report's
rows (CoverageReport.entries, ...), and the members of the classes the
model declares for the library check alone (CarlaDriverEntity, the
egodriver and bridge servers, the ROS 2 transport and its config, the map
cache's MapSource, RunEnvironment). An ODD's members compile as they
run:
roads = OddModule("roads", include_and=[location.is_in(["urban"])], labels=["fair"])
roads.as_situation(cover_by="meters", target=100.0)
text = roads.conditions()[0].describe() + roads.labels[0] + location.cover_by
for attribute in odd.attributes:
text += attribute.name + str(attribute.target)
So do an instance attribute and an enum's name and methods:
timeout = TimeoutCondition(cfg.timeout_seconds, label="timeout")
logger.info("limit %f, %s", timeout.timeout_seconds, ComparisonRule.LESS_THAN.name)
logger.info("%s %s", ComparisonRule.LESS_THAN.text, LaneChangeDirection.LEFT.to_carla_bool())
.codon of the package's model is in one of
the tables above or exempt with the reason (_UNCOMPARED_MODEL_FILES: the
model's helpers, the modules coordinate.codon and entity.codon
re-export, and the __init__ of a checked subpackage, which models
nothing), so a model file no test compares fails the suite.
- Third-party models: carla_driver_interface's geometry, python-dotenv's
find_dotenv and load_dotenv, packaging's Version and InvalidVersion, eclipse-sumo's
SUMO_HOME (where it is installed), roadgen's map, trace and
read_opendrive() (where it is installed), pyxodr's RoadNetwork and lxml's
element.
A constant whose model value is a literal (a str, int, float or a tuple
of them) is compared by its value and type (a tuple's items' too: 1 is no
1.0 or True) wherever it is compared, and a Literal alias
(ui.models.Status) as the type of its values. Any other constant fails the
comparison unless a test of its own compares it (EGO_ROLE_NAME,
BUILT_IN_METRICS, BUILT_IN_MEASURES).
What Codon builds by the fields. Codon 0.19 gives every class a
constructor of its fields, its bases' first, by position or by name, beside
the __init__ it declares: a class without fields is built with no arguments
(VehicleEntity(), #225), a condition or an action with the label it
inherits alone (TimeoutCondition("10"), TurnAction("npc1"), #236), and
any class by fields its constructor does not take (DebugMarkers([]),
CoverItem("speed"), RunProgress(1, 2, "case", "running"), #244), where
Python refuses each. A class that derives from one of Codon's exceptions
(Static[Exception]) also takes its bases' constructors, and its fields
start with BaseException's eight: FrameMismatchError("x") passed by
Exception(typename, msg="") (#244). A class that derives from a model class
plainly does not take its base's __init__.
Codon tries the __init__ overloads a class declares from the last one up,
and those it generates or inherits only after them, so the model declares,
ahead of the constructor, an __init__ of as many arguments whose body is a
compile_error:
class TimeoutCondition(BaseCondition):
def __init__(self, label):
compile_error("TimeoutCondition(): no constructor takes these arguments "
"(required: keyword label)")
def __init__(self, timeout_seconds: float = 60.0, *_kw, label = _Required()):
...
The constructor, declared last, still wins every call it takes
(TimeoutCondition(label="t"), TimeoutCondition(10.0, label="t")); the
refusing overload only takes what would reach Codon's own. The one of as many
arguments as the class has fields takes the fields' names, so a call by them
(CoverItem(name="speed")) is refused too.
test_typecheck_model.py compares the last __init__ with Python's and
finds the overloads needed. For every compared class that declares an
__init__, it counts each number of positional arguments the constructor
Codon generates (the fields, _codon_fields) or an exception base's
(_inherited_inits, Codon's own read from _CODON_EXCEPTIONS, which a test
compares with Codon's standard library) would take, and asks Python whether
its constructor binds as many placeholders (inspect.signature().bind).
Where Python refuses and the model's constructor does not take them either,
the model declares the refusing overload of that many; where the model's
constructor takes them through *_kw, its _keyword_only(_kw, ...) refuses
them (OddModule, OddAttribute, BaseScenario). The classes it finds:
| Arguments | Classes |
|---|---|
| none | RouteMatch, RouteFrame, VehicleEntity, PedestrianEntity, CoverageCollector, CoverageReport, EgoDriverGrpcClient, Ros2EgoTransport (and CycloneRos2Transport, compared apart) |
| the label | the conditions and actions whose constructor Python refuses one argument (SpeedCondition, TurnAction) |
| the fields | DebugMarkers, CoverItem, OddCondition (one), SomaMotion (two), RunProgress (four) |
| an exception's | FrameMismatchError (one, two, ten), DevContainerError, MapCacheError (none, two, eight) |
RouteCoverage declares Python's fields in Python's order (#298), so the
constructor Codon generates of them takes no call Python's refuses, and the
one it declares refuses each.
Every condition and action declares the overload of one, its label, even
where Python takes one argument by position: a call the constructor refuses
by its type (TimeoutCondition("10")) would still reach the one Codon
generates.
Those constructors take their parameters by keyword too, and a call by
keyword Python refuses passes where no guard takes those names: an
exception's base parameters (DevContainerError(typename="x"),
FrameMismatchError("x", message="y")), or a class's field names
(OddAttribute(name="speed", unit="m", text="t"), EntityRole(_value="x")).
So the test also enumerates every call, by position and by keyword, the
constructor Codon generates or an exception base's would bind, and requires,
for each one Python and the model's constructor refuse, a refusing overload
that binds it too: an __init__ of the fields' names (OddAttribute,
OddDefinition, EntityRole, TrajectoryVertex, EgoConfig,
BaseScenario, RunEnvironment), and of typename and
typename, message (DevContainerError, MapCacheError;
FrameMismatchError takes one positional typename already). A class
whose __init__ takes its fields by position and by name needs none: the
constructor Codon generates is its own.
A scenario that imports a module with no model (lanelet2's matching, say,
or a framework module outside the public API) is refused with a message
naming the module. For a framework module, the message says where the model
declares the names imported as Python exports them, and only where Python's
module has the same object, so the import it suggests runs too; a name the
model leaves out on purpose is refused with its reason, whichever module it
is imported from (#237):
scenario.py:3: error: autoware_carla_scenario.actions.base is not part of the Codon model; import ActionState from autoware_carla_scenario.action_state instead
scenario.py:4: error: autoware_carla_scenario.route.parse_route_search is not part of the Codon model, on purpose: the route search, parsed from a document's YAML ...
Third-party packages the framework reads are modelled as far as it reads them (lanelet2, pyxodr, carla_driver_interface, python-dotenv, packaging, eclipse-sumo, roadgen; see Making a module compile), and a scenario may use what those models declare.
Writing a scenario that type-checks¶
A scenario is ordinary typed Python. Codon is stricter than Python in a few
places, and the checker smooths over most of them (it rewrites X | None into
Optional[X], drops @dataclass (and appends the ==, ordering and hash it
generates) and resolves field(...) defaults, accepts
keyword-only parameters, required ones after a default included (a call
that passes one by position is refused at the call, as Python refuses it:
gap() takes 1 positional argument but more were given: scale is keyword-only,
622), gives a list of different conditions the type¶
list[BaseCondition], checks a tuple[X, ...] return annotation as a
List[X], drops a parameter or return annotation naming a
class the module defines further down, which Codon cannot name yet, gives a
subclass that keeps its base's __init__ that __init__, and types the
return None of a function returning Optional[X]; see below). What a
scenario has to do itself:
- Import its own modules from its own package. A module of the
scenario's package imported as a module works (#310): Codon 0.19 resolves
from X import nameonly as a nameX/__init__defines, and an absoluteimport a.bonly from the importing file's directory, so the checker rewrites, on the same line,from . import helpers,from pkg import helpersandimport pkg.helpers as helpersinpkg/scenario.pyintoimport helpers as helpers, andfrom .sub import modintoimport sub.mod as mod. A module outside the importing module's package (from .. import helpersinpkg/sub/scenario.py, orimport pkg as p) has no such form, and neither has one imported without a name (import pkg.helpers, which bindspkg): each is refused at its line, naming the form that works, which is to import the names (from ..helpers import speed_kph) or to name the module (import pkg.helpers as helpers):
scenario.py:3: error: `from .. import helpers` imports the module pkg.helpers as a module, which Codon 0.19 resolves only inside the importing module's own package (pkg.sub): import the names instead (`from ..helpers import name`)
-
from .helpers import *bindshelpers.__all__. When the helper module of the scenario's package sets__all__as a literal list or tuple of strings, the checker rewrites the star import into an import of those names, on the same line (#506), so a scenario that uses a public name__all__leaves out is refused (name 'SPEED' is not defined) as Python raisesNameError. An__all__built any other way is not read: the star import then binds every public name of the helper. -
Import a name once. Python accepts
from autoware_carla_scenario import ComparisonRulein the import header and again further down, and so does the check (#354): Codon 0.19 leaves a module-level name imported twice undefined in every module but the one it compiles first, so a use of it failed the whole function with a bare "cannot typecheck". The checker drops a module-level import that repeats an identical earlier one (the same module and name, under the same alias), on its line: the repeat becomespass, or loses just the repeated names. An import in a function or anifis left as written. A name imported from two different places (from a import X, thenfrom b import X) means the later one in Python: the checker drops the earlier import when nothing reads the name in between, and refuses the later import, naming the first, when something does:
scenario.py:12: error: `ComparisonRule` is imported again from somewhere else (first on line 5) after it was used, which Codon 0.19 cannot compile: import each name once, under its own name (`import ... as other` to tell two apart)
- Declare the attributes it assigns on
self, at class level, with their type:
class MyScenario(BaseScenario):
_config: MyScenarioConfig
def __init__(self, ego_config, spawn_pose, config=None, ground_projection=None):
super().__init__(ego_config, spawn_pose=spawn_pose, ground_projection=ground_projection)
self._config = config or MyScenarioConfig()
A bare annotation is ignored by Python, so this changes nothing when the scenario runs; Codon needs it to type an attribute of a subclass. The check names each undeclared attribute and suggests its type.
Declare them in every class of the scenario's modules, helper classes
included (#218). The check refuses an undeclared attribute of a class with
a base, a subclass, or at least one class-level annotation: once a class
declares one attribute, Codon gives it only those it declares. A plain
class that declares none has its attributes inferred from its __init__,
which Codon cannot always do: not for self._samples = [] (a list of
nothing yet), and not in a method the scenario check calls on its own
(What is checked), on a value its __init__ did not
build ('int' does not match expected type 'NoneType', during the
realization of get(self: Counter[NoneType])). Codon's error is then at
a line of the helper, or names the attribute holding the helper, not the
helper (type of attribute '_tracker' of object 'GapCondition' cannot be
inferred), so the check adds, for that helper, each attribute it assigns
without declaring it, at its line, with the type to declare:
scenario.py:34: error: GapTracker._samples is assigned without a type: declare it in the class body, e.g. `_samples: list[...]` ...
And assign each attribute you declare (#588). Codon gives every field
its zero value when the object is built, so a declared attribute nothing
assigns reads as 0 (or "", an empty list) in the check, and Python
raises AttributeError at its first read. The check refuses a bare
class-level annotation (no default; not a dataclass, NamedTuple,
TypedDict or Protocol field, nor a ClassVar) that a method of the
class or of a subclass reads (self._count, self._count += 1: an
augmented assignment reads first) when nothing assigns it, at the
annotation:
scenario.py:41: error: CountCondition._count is declared but never assigned: Codon zero-initialises it, and Python raises AttributeError where a method reads self._count; assign self._count in CountCondition.__init__, or give it a default (_count: ... = <value>)
The rule is conservative: the attribute counts as assigned when any
module of the scenario stores an attribute of that name, on any object
and in any form (self._count = 0, self._count: int = 0, self.a,
self._count = ..., setattr(self, "_count", 0), object.__setattr__(self,
"_count", 0), other._count = 0), when a framework base assigns it (the
base's source, or the framework assigning it from outside; for a mixin,
when the framework bases of every scenario class deriving from it assign
it: class VehicleInfoMixin: ego_config: EgoConfig mixed into a
BaseScenario, but not when a class Plain(VehicleInfoMixin) derives
from it too), and when the
class or a base gives it a value (a class attribute, a property). A class
that assigns attributes by a computed name (setattr(self, name, v),
self.__dict__, vars(self)), defines __getattr__, or derives from a
class outside the scenario, the framework and the built-ins is left alone.
-
Give a local its type at its first assignment. Codon fixes the type of an unannotated local where it is first assigned; Python and mypy let it widen later. Two common patterns are refused:
-
A float accumulator started at an int literal (
total = 0, thentotal += vwith a floatv), or a conditional expression with an int literal branch (s = v if v is not None else 0withv: float | None). Write0.0. Codon names neither the first assignment nor the literal, so the check adds them (#583):scenario.py:74:9: error: 'float' does not match expected type 'int': Codon fixes the type of the local `total` at its first assignment, `total = 0` on line 72, which makes it an int: write `total = 0.0` to store a float in it scenario.py:72:9: error: 'Optional[float]' does not match expected type 'Optional[int]': Codon fixes the type of the local `s` at its first assignment (here), a conditional expression whose int literal branch `0` makes it an int: write `0.0`Codon reports a float passed to an
intparameter in the same words, so when the stored value holds a call (k = half(y)), the hint begins "if the float is the value stored inkhere". -
A local first assigned
Noneand rebound later without an annotation, as in a running minimum (best = None, thenbest = xin a loop). Annotate it:best: float | None = None. Without the annotation, Codon 0.19 either refuses the rebinding ('Optional[float]' does not match expected type 'Optional[NoneType]', which the check completes with the= Noneline and the annotation) or crashes with no location at all (#584). After such a crash the check keeps the crash and adds, at its= Noneline, each unannotated local of the checked modules that is first bound toNoneand rebound, as a likely cause. The type comes from what the local is rebound to where the source says it plainly (a literal, an annotated parameter, the element of an annotated list it loops over); otherwise it is written<T>:error: Codon failed without a diagnostic: exit status -11 scenario.py:72: error: Codon crashed (exit status -11); a likely cause: Codon fixes the type of the local `best` at its first assignment, `best = None` on line 72, and cannot widen it when it is rebound (line 75): annotate it: `best: float | None = None`Codon compiles some of these as written (a
Nonefirst binding read only after anis Nonetest), so a local named after a crash is a candidate, not a verdict: the crash may have another cause. A Codon killed with SIGKILL (exit status -9: the out-of-memory killer, as when two library checks of about 14 GB each run at once) is no crash of the code: the check says so, names no local, and should be run again with more memory free. Like a timeout, it is not kept as the verdict, so the next check of the same program in the process compiles it again (#694).The same holds where a conditional expression tests the local before it is rebound, the previous value of a loop (
prev = None, thend = v - prev if prev is not None else Noneandprev = v, #629): Codon decides the test while the local's type is still the unresolvedOptionalof its= None, completes it asOptional[int], its default, and refuses'Optional[int]' does not match expected type 'Optional[float]'further on. The check adds the local's= Noneline and its annotation (prev: float | None = None). A local that is only everNoneand that a conditional expression tests (z = None, thenz + 1 if z is not None else None, even1 if z is None else 2) crashes Codon 0.19 with no diagnostic; the check names it after the crash, at the conditional expression. Annotate it, or drop the branch that cannot run. - Write the element types of a class attribute (#482):xs: list[int] = field(default_factory=list),d: dict[str, float],ids: set[int],xy: tuple[float, float] = (0.0, 0.0)ortuple[X, ...]. Codon types a dataclass field, a NamedTuple field and any other attribute declared in a class body (xs: list = [], or a barexs: list) from its annotation and cannot infer the elements of a barelist,dict,setortuple("type of attribute 'xs' of object 'Cfg' cannot be inferred", or a bareTuplethat matches no tuple), whatever the default is. The check refuses the field at its line and names what to write; a barelistordictin a function's parameter or return annotation compiles, and a baretuplethere gets a hint with the error. -
Use only the public API from
autoware_carla_scenarioand its listed subpackages, and theBaseScenarioattributes meant for subclasses (ego_config,ego_entity,world,_ground_projection, ...). An import from a modelled framework module must also work in Python (#214): the model declares a few names the Python module does not have (coordinate.RoadNetwork,entity.EgoConfig,actions.ActionState: import it fromautoware_carla_scenario.action_state, as Python does) and model-only helper modules (autoware_carla_scenario._tc,_lists), and an import of one inside a function would compile and raiseImportErrorwhen the function first runs. So the check imports each modelled module a scenario imports from, in Python, and refuses each name it lacks, as an attribute and as a submodule, at the import's line:
scenario.py:118: error: autoware_carla_scenario.coordinate has no RoadNetwork in Python, whatever its Codon model declares: `from autoware_carla_scenario.coordinate import RoadNetwork` raises ImportError when the scenario runs
An import under if TYPE_CHECKING: is exempt: Python never runs it.
One in its else branch, or in the body of if not TYPE_CHECKING:, is
checked this way only: Codon never compiles it, so it needs no Codon model
and what it imports is not compiled (#2288).
- Define every abstract method of what you build (#215). Codon has no
@abstractmethod, and the model gives BaseScenario.is_done(),
BaseCondition.check() and BaseAction.execute() bodies (its built-in
conditions and actions inherit them), so a scenario whose is_done was
renamed, or a condition that defines only _check, compiles, and Python
refuses to build it (TypeError: Can't instantiate abstract class). The
check asks Python (inspect.isabstract) about each class of the
scenario's modules that is built, the scenario class (the runner builds
it) or a class the modules call by name (GapCondition(...)), and
refuses one still abstract at its class line:
scenario.py:46: error: GapCondition is built (at scenario.py:99) but leaves abstract check undefined: define it in GapCondition, or Python refuses to build it (TypeError: Can't instantiate abstract class)
A class only derived from, such as an abstract base of the scenario's own
conditions with an @abstractmethod of its own, is never called and may
stay abstract.
- Call the framework base's __init__ from yours (#587). A scenario,
condition, action or entity whose __init__ never calls
super().__init__(...) compiles, since Codon zero-initialises what the
base would set, and Python raises AttributeError when the runner reads
condition.label or the scenario's ego_config. The check refuses such an
__init__ at its def line, naming the base and what its __init__ sets:
scenario.py:52: error: LateCondition.__init__ never calls BaseCondition.__init__, which sets label, _last_satisfied, _last_message on self: Codon zero-initialises them, and Python raises AttributeError where the framework reads them; call super().__init__(...) in LateCondition.__init__
The bases are the framework's classes after yours in the method resolution
order whose own __init__ (written, or the one @dataclass generates)
assigns an attribute of self; a base whose __init__ assigns nothing,
a class with no framework base, and an __init__ that itself assigns
every attribute those bases' would, are left alone. So is a subclass of
your own condition that defines no __init__; one that defines its own
and does not call super().__init__ skips the framework's too, through
yours, and is refused. Any mention of __init__ in the body counts as the
call, on any path (super().__init__(...), super(Cls, self).__init__(...),
BaseCondition.__init__(self, ...), inside an if or a try), as does a
call of a method of self when a method of the class calls the base's
__init__: the check does not follow paths. With @overload stubs, the
last def __init__, the one Python binds, is the one checked.
- Pass numbers of the declared type. An int is accepted for a float,
but not inside a container: a list[float] parameter wants [1.0, 2.0].
YAML values are converted for you.
- An empty [] or () is any sequence (#264). Codon gives an empty
display no element type ([] is a List[NoneType], () a tuple of no
items), so the model takes one wherever Python takes a Sequence[X]
(RoutingAction(EGO_ROLE_NAME, goal), whose waypoints default to (),
waypoints=[], self.waypoint_poses = [], derive_goal_from_route(()),
AndCondition([], ...), an ODD's all_of([])) through _acs_sequence
(_tc.codon), and checks the elements of any other: waypoints=[460, 265]
and waypoints=[None] are refused, in EgoVehicle.route_to() too.
- A sequence of names is a list or a tuple, not a str (#299). Python's
typing takes a str as a Sequence[str], and the framework then reads
each of its characters as a name (roots="roads" fails at run time with
"no root module ['r', 'o', 'a', 'd', 's']"). Here the check is stricter
than mypy, on purpose: _expect_str_sequence (_tc.codon) refuses a str
(or an Optional[str]) where a sequence of names is taken, the items of
BaseScenario.register_cross(), OddModule's labels and
OddDefinition's roots, and takes a list or a tuple of str, an empty
one too:
scenario.py:41: error: BaseScenario.register_cross(): items: expected a sequence of names (a list or a tuple of str), not a str: wrap it in a list
OddAttribute.is_in("urban") still passes: it takes an Iterable[Any] of
values, and CPython takes a str's characters as values.
- Call a manoeuvre your own entity class overrides with every argument, by
position. VehicleEntity and EgoVehicle declare the manoeuvres they
take from traffic.driven.BackendDriven (set_desired_speed(),
change_lane(), lane_change_finished(), turn_at_junction(),
release_from_traffic(), ...), and an EgoVehicle subclass's override of
one is checked as a hook. Codon 0.19 calls a method a subclass overrides
through a dispatch thunk, which takes no keyword argument, fills in no
default (set_client(client) needs its tm_port then), and cannot call
turn_at_junction(world, direction, **kwargs) at all: once a class of the
scenario overrides it, a call of it does not compile ("cannot
typecheck"), on the subclass as on its base; turn with a TurnAction
instead. A class that only defines the override, and never calls it,
compiles. (Declaring Python's **kwargs as TurnAction's three keywords
would not help: the thunk fills in none of their defaults, so even
turn_at_junction(world, direction) would fail, and a backend's own
keywords would be refused.)
- A dict of different actions (or conditions) is a dict of their base.
Codon types a dict display by its first item, so the check rewrites one
whose values are not evidently of one type, as it does a list display:
{"turn": turn, "lane_change": lc} (a TurnAction and a
LaneChangeAction) becomes {"turn": _acs_base(turn), "lane_change":
_acs_base(lc)}, which makes each action a BaseAction (each condition a
BaseCondition): a dict[str, BaseAction] as in Python, which passes as
ActionStateCondition's actions. Assigned to an annotated variable,
actions: dict[str, BaseAction] = {...}, it becomes a dict of the
annotated type (_acs_dict_into, when the key and value types are plain
names). A value of another type is still refused at the display's line:
Any other value is left to Codon's display, which types it as before
({"a": 1, "b": speed} is a dict[str, float], a value with None is
optional). Values evidently of one type (all literals, all calls of the
same name, all members of one enum) are left as written. Building the dict
one item at a time into a dict[str, BaseAction] stays valid.
- Convert nested mappings in the config. The runner passes a YAML mapping
to the config class as a dict, so a field annotated with a dataclass
(goal: Goal) holds one at run time unless the config's __post_init__
turns it into a Goal, as the built-in configs do for their NPC lists.
The check builds the config the same way and refuses an unconverted
mapping at its scenario.<key>. The same goes for an Enum field (YAML
gives its name or value as a str) and a tuple field (YAML gives a
list): they compile when the __post_init__ converts them
(self.mode = Mode(self.mode)), and are refused at scenario.<key>
when it does not. A bad value inside a nested list is refused at its own
field, e.g. scenario.legs[1].speed.
A config that raises when built (a __post_init__ that rejects a value, a
name Mode(...) does not know) fails the check with the exception, as it
stops the run.
- Annotate every parameter of every function and method, helpers and
hook overrides included. Every function of the scenario's modules is
compiled, whether
setup()reaches it or not, with a value of each parameter's annotated type (What is checked). A function with a parameter that has no annotation, one Codon cannot express or one naming aTypeVar, is only compiled wheresetup()andis_done()call it, and the check warns about it, at the parameter's line; so is one returningAny, with the warning at itsdef. - Give each class-level default a value of its annotated type
(
RETRIES: int = 3): it is compiled against the annotation. - Return what the framework reads from a hook you override: a
boolfromis_done(), aScenarioResultorNonefrom a condition'scheck(), adict[int, float]frominitial_speeds_mps(),Nonefromsetup()and an action'sexecute()(What is checked). - Test an
OptionalforNonebefore using it: afloat | Noneconfig field,goal_pose,find_actor_by_role_name(...), and CARLA'sOptionalresults (get_waypoint(...),get_actors().find(...),world.get_actor(...),actor.parent). - Read
**kwargsby name withkw.get(), and test*args, like any tuple, withlen()(#266). Codon makes**kwargsaNamedTupleof the keywords passed, not a dict, and*argsaTuple, and gives no tuple a truth value (#278). Codon refuses this valid Python, and the message says what to write instead:
| Python | Codon's error | Write instead |
|---|---|---|
for k, v in kw.items(): (.keys(), .values(), for k in kw, len(kw)) |
'NamedTuple[a=float]' object has no attribute 'items' |
kw.get("a", default) for each name, or keyword parameters (def f(*, a: float = 1.0)) |
if args: / x if args else y, or if window: of any tuple |
'Tuple[float,float]' object has no attribute '__bool__' |
if len(args) > 0: (if len(window) > 0:) |
kw["speed"] and "speed" in kw compile for a keyword the call passes;
kw.get("speed", 1.0) compiles whether it does or not. A helper taking
**kwargs or *args that the scenario calls and that reads them by key
or position is compiled through its callers' calls, so it is checked with
the keywords and values they pass
(What is checked).
- Give your own helper an Optional of the class it gets, not of its
base. Codon 0.19 does not convert an Optional of a subclass to an
Optional of its base (#278). A helper def gap(actor: carla.Actor | None)
refuses an Optional[carla.Vehicle], and a plain carla.Vehicle too, which
Codon wraps in an Optional first ('Optional[Vehicle]' does not match
expected type 'Optional[Actor]'). Python accepts both. A parameter of
carla.Actor alone takes a carla.Vehicle, and the framework's own
Optional[carla.Actor] parameters take either
(What is checked). Annotate the helper with the
subclass (carla.Vehicle | None), or test for None before the call and
pass the unwrapped value to a carla.Actor parameter.
- Build an object of a class you hold in an Optional, when the class has
subclasses and nothing else builds one. Codon 0.19 checks
x: X | None = None (or a parameter defaulting to None) of such a
class, then generates code that LLVM refuses ("LLVM: error: value doesn't
match function result type 'ptr'"), when the program builds no X nor
any subclass of it, or defines a subclass after Codon compiled X (#263).
The model builds its own such classes once, after their subclasses:
BaseCondition, BaseAction and lanelet2's ConstLanelet,
ConstLineString2d, ConstLineString3d, ConstPoint3d and
RegulatoryElement (the others are built by the scenario check anyway).
So a scenario that builds actions but no condition compiles, though each
action's condition = None is an Optional[BaseCondition], whether it
registers the actions or keeps them in a list or dict of BaseAction.
A class of your own, or your subclass of a framework class that nothing
builds (class MyResult(ScenarioResult)), can still meet it; the check
then names the class and what to do, at the class line of it or of its
subclass, and keeps no verdict, as for any crash:
scenario.py:73: error: Codon 0.19 generated code it cannot compile for autoware_carla_scenario.conditions.ScenarioResult, a class with subclasses: a bug of Codon's rather than an error of the program (#263). It does so for an Optional of such a class (`x: ScenarioResult | None = None`, a parameter that defaults to None) when the program builds no object of the class or of a subclass, or defines a subclass after Codon compiled the class. Build a MyResult where setup() reaches it, or hold the value without Optional
LLVM: error: '%obj' defined with type 'ptr' but expected '{ ptr, ptr }'
Annotations Codon cannot express (Union of two types, Any, Callable,
Sequence, type[...]) are dropped from parameters, which become generic:
Codon then checks each call with the arguments it is given. The appended
calls give a union parameter (Lanelet2Pose | OpenDrivePose) each of its
members in turn, and a Callable[[A], R] of types Codon can express a value
of that signature; Any, object and Callable[..., R] give them nothing to
call with, hence the warning.
The checker also rewrites what Codon spells differently: an Enum class (its
members become instances with a name and a typed value; a lookup by value,
ActionState("completeState"), takes a value of that type and is typed but
not resolved to the member; looking a member up by name and iterating over the
class work for a scenario's own enum, see
Iterating an enum), a class deriving from an exception, a @classmethod
(a static method of its class: cls names the class it is defined in), and an
annotation naming one of CARLA's integer enumerations
(carla.TrafficLightState), which typesafe_carla's Codon library has as a
value whose members are int, not as a type: it becomes int.
Two things Python does that Codon 0.19 does not are filled in for you:
- A subclass that inherits
__init__. Codon 0.19 inherits no__init__: it gives a class without one a constructor taking each of its fields, which refused a scenario deriving from another scenario and keeping its constructor ("'config' is an invalid keyword argument for__init__()", #201). The check appends one to such a class, after the module, so no line moves:
@extend
class MyDerivedScenario:
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
It is added only where a base really defines an __init__ (a class of the
scenario's modules, its own or one it inherits, or one of the model's, such
as BaseScenario's), and never to a dataclass, an Enum or an exception,
which are built as before. When the __init__ is a later base's
(class MyDerivedScenario(MyMixin, MyScenario), the mixin having none),
it is called through that base, MyScenario.__init__(self, *args,
**kwargs), as Python does: Codon's super() is the first base's. The
base's __init__ still checks every argument: a wrong one is reported at
the class line of the subclass, with the call that passed it in the
trace:
scenario.py:52:9: error: no function 'MyScenario.__init__' with arguments (Super[MyScenario], EgoConfig, Lanelet2Pose, str)
during the realization of __init__(self: MyDerivedScenario, ...) [scenario.py:58]
A literal None passed through it is refused, though Python takes it
(MyDerivedScenario(ego, pose, None), or config=None): forwarded
through *args or **kwargs, it reaches the base's __init__ as an
Optional[NoneType], which no Optional[X] parameter takes. Leave the
argument to its default, pass a variable declared Optional[X], or give
the subclass an __init__ of its own.
A subclass with an __init__ of its own that lists a mixin before its
framework base (class CaseScenario(VehicleInfoMixin, BaseScenario))
calls super().__init__(ego_config, spawn_pose=..., ...), which Python
follows through the MRO past the mixin to BaseScenario.__init__. Codon's
super() is the mixin's constructor of its fields, which refused the
base's arguments ('ground_projection' is an invalid keyword argument for
__init__()), so the check rewrites the call to
BaseScenario.__init__(self, ego_config, spawn_pose=..., ...) (#626),
which checks every argument against the base's __init__ as before. It
does so only where the MRO makes the two the same method: every base
before it is a plain mixin of the scenario's modules (no __init__ of its
own or inherited, no @dataclass, and only such mixins, object, ABC
or Generic[...] as bases). A class whose first base has an
__init__, or whose earlier bases the check cannot see, keeps super();
listing the framework base first (class CaseScenario(BaseScenario,
VehicleInfoMixin)) always works.
return Nonefrom a function returningOptional[X]. A bareNoneis Codon'sNoneType, so a function whose only return isreturn Nonecould not be typed ("cannot typecheck 'None'"), and, when nothing else in the program usedX(carla.Actorin a scenario),Optional[X]failed in code generation ("value doesn't match function result type 'ptr'", #202). The check makes eachreturn None(and barereturn) of the function itselfreturn Optional[X](), theNoneof its type (a nested function is rewritten for its own annotation), and the appended calls realizeX(_acs_value(X)) before they call it. So
compiles as written, and so does a local found: Optional[carla.Actor] =
None returned at the end.
- Your own dataclasses, enums and named tuples keep what Python gives them (#365), with these limits:
- A
@dataclasscompares, orders and hashes as Python's does: the check appends==and!=(eq=True, the default),<,<=,>and>=(order=True) andhash()(frozen=True, orunsafe_hash=True) after the module, comparing the fields in order (a dataclass base's first;field(compare=False)andClassVarare left out;InitVarandKW_ONLYcannot be imported in the check, see "Standard library names Codon lacks"). A method the class defines itself is kept. This holds for a dataclass whose bases are dataclasses of the same module (or none): one deriving from a class the check cannot see the fields of, such as a framework dataclass, gets none, and==on it is refused.eq=Falsecompares and hashes by identity, as a plain class does (below). Python refuses<withoutorder=Trueandhash()of a non-frozen dataclass, and so does the check, with a hint naming the flag. - A plain class (one that does not define
__eq__) compares and hashes by identity, as Python'sobjectdoes (#596):a in npcs,npcs.remove(a),a == b,{t},{t: 1}anddict[Npc, float]compile. The check gives each top-level class of the checked modules whose bases are plain (none,object,ABC)__eq__/__ne__(self is other) and__hash__(id(self)), with@extendright after the class, so module-level code and the module's subclasses have them; a__hash__(or__ne__) the class defines is kept, and a@dataclass(eq=False, unsafe_hash=True)keeps the hash of its fields. A class deriving from a framework class gets the framework model's:BaseCondition,BaseAction,VehicleEntity,PedestrianEntityandEgoVehiclecompare and hash by identity in the model, as in Python ({TimeoutCondition(1.0, label="a")}compiles). Enums, named tuples, exceptions, protocols and a class with a metaclass get nothing. A class that defines__eq__and not__hash__is unhashable in Python (it sets__hash__ = None), and the check refuses a set of them, or a dict keyed by one, with a hint to define__hash__. The gap: such a class deriving from a plain class of the module (or from a framework class above) inherits the identity hash in the check, so a set of them compiles there and raisesTypeErrorin Python. dataclasses.replace(obj, a=x, b=y)is checked as the typed copy it is (#487): the check appends to each dataclass of the module one_acs_with__<field>method per field the constructor takes, and writes the call asreplace(obj)._acs_with__a(x)._acs_with__b(y). An unknown field, a value of the wrong type (replace(w, speed="fast")) and a field declaredfield(init=False)(which Python refuses too) are refused at the call, andreplace(plan, best=None)of anOptionalfield works. A field a base dataclass declares can be replaced on the base and on any subclass, however many levels (replace(derived, a=5)), also one a subclass redeclares and in a frozen hierarchy (#494): the value goes by position, as Codon cannot pass a keyword to a method a subclass overrides. A wrong value for it is reported asdataclasses.replace: field 'a' of Derived is declared int, not str. Any other error raised inside the_acs_with__<field>method is reported at thereplace(...)call, not at the class (#614), also when the call is in another module than the class (replace(cfg.lanes, ids=...)inscenario.pyof a dataclass ofconfigs.py, #623). Codon names that module by its file name alone, so when two checked modules share it (configs.pyandsub/configs.py) the error stays at the class. It holds for a dataclass the scenario package defines, in any of its modules, also one nested in a class or a function (Plans.Leg, #623: its methods, and its==, hash and ordering, go at the end of its body, as@extendcannot name it; a nested dataclass whose body is on theclassline,class Leg: x: int = 0, gets none), whose bases are dataclasses of the same module (of the same body, for a nested one) or none, and which has no__init__of its own; for any other class (a framework or library dataclass) the call is refused with a hint to build the object yourself (OpenDrivePoseandDevContainerhave the methods in their models, as the framework usesreplaceon them). The object goes by position and the fields by keyword:replace(obj, **changes)andreplace(obj, 3.0)are refused. The fields change one after the other, so a__post_init__is not run between them, andreplace(obj)with no change returnsobjitself, not a copy (both only matter to the types). Areplacethat is notdataclasses' (a parameter, a local or another import of that name,str.replace) is left alone.- An
IntEnum(alsoIntFlag,class M(int, Enum)) member compares with its value (M.A == 1), orders against a member or anint(M.A < M.B) and converts withint()andfloat(). AStrEnumorclass M(str, Enum)member compares with its string (M.A == "a") and does nothing more: ordering andint()are refused, as in Python for a plainEnum, aFlagand a(str, Enum). AFlagand anIntFlagalso combine with|,&,^,~andin(Flag and IntFlag). A plainEnummember equals only members (M.A == 1isFalse, as in Python). An enum member is not anintor astrotherwise: passing it to anint/strparameter or doing arithmetic on it (M.A + 1) is refused with a hint. WriteM.A.value. - An enum is typed by its members' values, which must be literals of one
type: all
int(a signed one,LEFT = -1, too), allfloat, allstr, orauto(), unless the base gives the type (IntEnum,class M(str, Enum)). An enum whose values are tuples (LEFT = (-3.5, "left"), unpacked into an__init__that setsself.offset), of different types (A = 1,B = "b"), or not literals (A = WIDTH,A = 2 * WIDTH,A = True) is refused at its class line (#600), with what to write instead: for a record per member, a@dataclass(frozen=True)or aNamedTupleand a module-level dict keyed by a plain enum (LANES = {Lane.LEFT: LaneInfo(-3.5, "left")}), or one enum per field. An enum with no member (a base for others) is not refused. - A
class Pt(typing.NamedTuple)is checked as a Codon@tupleclass: its fields, defaults and methods,==, ordering,hash(), iteration andlen()work. Codon indexes and unpacks it (p[0],x, y = p) only when all its fields have one type: otherwise readp.x,p.y. It has no_replace(),_asdict(),_fieldsor_make(): build a newPt(...). Only the class syntax is covered (notNamedTuple("Pt", [...])orcollections.namedtuple).
Python the check rewrites for Codon (#376)¶
Each of these is valid Python that Codon 0.19 refuses or crashes on. The check rewrites the checked copy of your module (on the same lines; what runs is your code as written), so you write the Python you would anyway. A form the rewrite cannot express is refused at its line, with what to write instead.
| You write | The check compiles | Refused when |
|---|---|---|
class MyError(Exception): ... |
the same with pass for ... |
|
{**a, "k": v, **b} |
(a \| {"k": v} \| b); {**a} is dict(a) |
the values of the parts have other types (Dict[str, int] and Dict[str, str]) |
"{:>5}\|{a}".format(1, a=x) on a string literal |
the f-string f"{1:>5}\|{x}" (a complex argument is bound once by a lambda) |
*args / **kwargs, a field inside a format spec ({:>{}}: write the f-string, which is rewritten), a missing argument, mixed auto and manual numbering, a template that is not a literal (write an f-string) |
f"{x:+.2f}", f"{n:>{w}d}", a spec Codon's f-string parser does not know |
the value's own __format__, the f-string a + of its parts: (x).__format__("+.2f"), (n).__format__(">" + str(w) + "d"). A nested field is str(w) (repr(w) for !r and !a); the result is a str, as before. In an f-string rewritten this way (or for a literal brace or !a), a field whose expression holds a double quote or a backslash is a call too, str(e), repr(e) or its __format__ (#704), where it was refused. "{:+.2f}".format(x) is rewritten to the f-string first, so it gets the same |
a brace or a nested f-string inside an expression of the f-string (below) |
An f-string (or "{}".format(...), {x=}) in the body of a lambda that reads a name the lambda does not bind: lambda v: f"{v}{p}", sorted(xs, key=lambda e: f"{e}{p}"), lambda v, lab=cfg.name: f"{v}{lab}" (#470) |
str() parts joined by +: str(v) + str(p); repr(p) for !r, (p).__format__(spec) for a spec, "p=" + repr(p) for {p=}, repr(p) for !a (as for ascii()). Each value is evaluated once, in order, and the result is a str. Codon looks the names of an f-string up in the lambda's own scope ("name 'p' is not defined"); a def or a comprehension does not have that limit. An f-string of the lambda's own parameters is left as written |
a brace or a nested f-string inside an expression (as above) |
f"{x!a}", f"{x!a:>6}", f"{{{x!a}}}", "{!a}".format(x) (#701) |
the same field with !r: f"{x!r}", f"{x!r:>6}"; a field rewritten to a call (a spec Codon does not parse, an expression holding a quote) is repr(x), as ascii(x) is. Codon 0.19 has no ascii() ("name 'ascii' is not defined"); repr gives the same str type, and only the checked copy changes, so the escaping of non-ASCII characters is unchanged at run time |
|
zip(a, b, strict=True) |
zip(a, b): the check types the loop, it does not run it |
|
s.startswith((a, b)), s.endswith((a, b), start) with the tuple written in the call (#439) |
(s.startswith(a) or s.startswith(b)); a receiver or item that is not a name, attribute or constant is bound once by a lambda |
the tuple is held in a name (s.startswith(PREFIXES)), is empty, or the call has *args / keywords; each item is still checked as a str |
format(x, spec), format(x) (#439) |
x.__format__(spec), str(x) |
format is bound where the call is: by the module (an assignment, import, def or class at module level) or by an enclosing function or lambda (a parameter or local); a method named format or a class attribute binds nothing there, so it does not disable the rewrite (#471) |
ascii(x) (#439) |
repr(x): the same str type; the check does not run it, so the escaping of non-ASCII characters is unchanged at run time |
ascii is bound where the call is, as for format (#471) |
getattr(x, "name", default) with a literal name, x a name or an attribute chain, the default on one line (#590) |
((default), x.name if hasattr(x, "name") else (default))[1]: Codon's getattr takes no default, and Codon decides hasattr when it compiles, so only the branch x's type takes is compiled, typed as that branch (the attribute's type where the class declares it, the default's where it does not). The first item keeps the default compiled either way, as Python evaluates it either way (getattr(x, "name", x.typo) is refused). hasattr on an Optional is decided on the Optional (see "Making a module compile"), so the default's branch is the one compiled there. Inside an f-string field the name is quoted with the quote the f-string does not use (f'{getattr(o, "a", 1)}' gives hasattr(o, "a")), as is every string a rewrite writes into a field (#609): Codon's parser ends the f-string at its own quote |
a name built at run time, another receiver (a call, a subscript), a default over several lines or hasattr bound where the call is: refused with getattr() takes 2 arguments (3 given) and a hint to write the hasattr test |
t.count(x), t.index(x[, start[, stop]]) with t a name or an attribute chain (#601) |
(t.count(x) if hasattr(t, "count") else _acs_count(t, x)): Codon's tuple has no count / index and cannot be extended, and only Codon knows t's type, so a list, a str or a class of yours keeps its own call, compiled as written, and only a tuple goes to the helper (an item of another type than x never equals it). A tuple display, (1, 2).index(2), goes to the helper directly. A tuple narrowed from an Optional (t.count(1) if t is not None else 0) is still Optional to Codon: the helper unwraps it (#609). Where Codon crashes with no diagnostic in a module that has these rewrites (an earlier error left the helper a wrong type, as lst = [None, a] then lst.index(a)), the module is compiled once more without them, and that compile's error is reported; a rerun that passes, crashes too or refuses a tuple's count / index (the crash had another cause) leaves the crash reported |
the receiver is a module (itertools.count), a builtin type (str.index(s, "a")), a call or a subscript (f().count(x), rows[0].index(x): bind it to a name), or the call spans lines; then a tuple's is refused ('Tuple[int,int]' object has no attribute 'count') |
d.keys() & e.keys(), \|, -, ^; d.items() <= e.items(), <, >, >=, ==, != (#601) |
set(d.keys()) & set(e.keys()), the other operand of &, \|, -, ^ a set too (a view takes any iterable there, and gives a set); a comparison makes a set of each view and leaves the other side as it is (a view orders only against a set) |
the view is held in a name (k = d.keys(), then k & e): write set(d) & set(e) |
super(Derived, self).__init__(k) in a method of Derived whose first parameter is self (#601) |
super().__init__(k): Codon's super takes no arguments |
another class (super(Base, self), which skips Derived's bases up to Base), a @classmethod / @staticmethod, the class or self rebound in the method, or the call inside a nested function, lambda or comprehension: Codon's super() takes 0 arguments (2 given) |
round(x), round(x, None), round(number=x, ndigits=None) (#585) |
int(round(x)): Codon's round(x, n=0) always returns a float, Python's an int when ndigits is left out or None, so xs[round(x)], n: int = round(x) and range(round(x)) check. round(x, ndigits=2) is round(x, 2) (Codon's has no Python keywords). round(x, 2) stays a float, also for an int x (Python returns an int there). Being an int, round(x) is refused inside a list[float] or dict[str, float] display ([round(x), 1.5]), as any int is: write float(round(x)) there |
round is bound where the call is, as for format (#471); an ndigits that is a variable is passed as it is: an int \| None one is read as a float, one bound to None only fails in Codon's round (__pow__() cannot be called with (None)) |
sorted(xs), xs.sort(), min / max of a class that defines __lt__ only |
the class gets __gt__ (other < self) and __ge__ (not self < other), which Codon's sort calls |
the class defines no ordering (use @dataclass(order=True) or key=) |
a if x else None, a if x is not None else None, None if x is None else a (also not x), where a reads the narrowed x: m.group(1) if m else None, npc.speed if npc is not None else None (#597, #621) |
_acs_some_if(lambda: a, x) if not isinstance(x, Optional[NoneType]) else None. Codon cannot unify the type of a with the None ('Optional[str]' does not match expected type 'str'), and where that type is known only later (an attribute or a method of an Optional) it left the expression an unresolved Optional[T], so a later str(), round() or format spec of it was refused or crashed Codon. Codon decides the isinstance test from the type, so it compiles only the branch it picks (the None where x is always None, as Python never evaluates a there) and unifies nothing; _acs_some_if returns a as an Optional (one that already is one is kept), the type Python gives the expression. In a class body, or where a cannot be in a lambda (a walrus, yield, await, super() or super(Cls, self)), the branch is _acs_some(a) |
|
a.__eq__(b), a.__ne__(b) (#596) |
(a == (b)), (a != (b)): Codon cannot call by name an __eq__ that overrides an inherited one (the identity __eq__ the check gives a plain class, or the model's BaseCondition), as a __ne__ written return not self.__eq__(other) does; == is how Python calls it, too |
super().__eq__(b), A.__eq__(a, b) and a parenthesized receiver are left alone |
a class that defines __eq__ but not __ne__ (a value object in a set or as a dict key) |
the class gets __ne__ (not (self == other)): Python derives it, Codon does not and its hash table calls it |
(a @dataclass already has one) |
namedtuple("P", "x y") / "x, y" |
the string becomes the list ["x", "y"] (Codon crashes on a string) |
field names that are not a literal (a variable, .split()), and rename= / defaults= / field_names= / module= as keywords, and field names Python rejects (a duplicate, a keyword, a leading underscore); a namedtuple the module defines or imports from elsewhere is left alone; use a list written in the call or a typing.NamedTuple class |
@functools.total_ordering |
the decorator and its import are dropped, and the class gets the comparisons it would add from its __lt__, __le__, __gt__ or __ge__ (and ==, or identity without __eq__) |
the class is not at module level, or defines none of the four |
lambda x, k=k: x + k (a default that is not a literal) |
(lambda k: lambda x: x + k)(k): the default is bound by a call, once, as in Python |
|
a def in a function whose parameter default is a local of an enclosing function (#612): for name in names: def log_it(world, n: str = name), scale = 2; def f(k: int = scale) |
(name,)[0]: Codon refuses the bare name ("name 'name' cannot be captured") but evaluates any other expression once, when the def runs, as Python does. The tuple holds the same value and type, and the parameter can still be passed |
a default that is no bare name is left as written (Codon takes it); a method of a class defined in a function is not rewritten. Another cannot be captured is hinted: bind the value with functools.partial(f, n=value), or read the local in the body |
an f-string in a decorator (#616): @tag(f"lane-{N}"), @tag(f"plain") |
str() parts joined by +, as in a lambda above: @tag(("lane-" + str(N))). Codon reads an f-string there as an attribute of the string ("'str' object has no attribute 'prefix_f'"), with or without fields. An f-string in a default, an argument or a body is no decorator and is left as written |
a brace or a nested f-string inside an expression (as above) |
def __exit__(self, exc_type, exc, tb) (annotated type[BaseException] \| None, BaseException \| None, object, ... or not) |
each parameter without a default gets =None: Codon calls __exit__() with no arguments, even when the block raised (#435). An annotation of BaseException or Exception (alone or \| None, Optional[...]) is kept as X \| None; any other annotation is dropped |
|
fs: list[Callable[[int], int]] = [inc, lambda x: x * 2], table: dict[str, Callable[[int], int]] = {"a": inc, "b": dbl}, or a return of such a display from a function annotated so (a dispatch table) |
each function, module function (mod.f) or lambda element is wrapped in Codon's Callable[Tuple[int], int](...), so they share one type (#436) |
the element is a bound method, a lambda using a local variable of its function, or a call (not wrapped, then Codon's own refusal and a hint); the table is a set, an attribute of a class, is unannotated, or is built another way than a display (a comprehension) |
a class constant read in its own class body (#433): DOUBLE = LIMIT * 2, x: int = SIZE (a dataclass default), def f(self, n=LIMIT), @register(LIMIT) |
Class.LIMIT (Outer.Inner.LIMIT in a nested class) |
the constant is annotated and not a single-line literal (LIMIT: int = 2 * 3), it is assigned again later in the body, the class is defined in a function, the read is a NamedTuple field default, or it is an Enum member with a non-literal value |
a comprehension or generator in a class body (#469): SQ = [i * i for i in range(3)], field(default_factory=lambda: [d.name for d in Mode]), lambda: tuple(gen), lambda: list(Mode) |
a module-level function above the class (def _acs_hoist_Q_2(): return (...)) that the attribute or field default calls, on the same lines (Codon does not bind a comprehension's variable in a class body, and crashes on an Enum iterated there) |
the expression reads its own class, or an enclosing one (range(LIMIT) with LIMIT a constant of the class): build it in a module-level function. A class nested in a class is hoisted too (#482, _acs_hoist_Outer_Inner_5, above the outermost class); one inside a function is not |
a lambda assigned in a class body (#482): f = lambda self, v: v * 2, key = lambda v: str(v) + "!" |
def f(self, v): return (v * 2) on the same line, as Python makes it a method. A lambda whose first parameter is not self / cls is a @staticmethod (a decorator line is added, mapped to the lambda's line): it is called as C.key(3), and Codon would otherwise bind v to the instance (Codon cannot call a method through the class with an explicit instance for this, T.f(3) is refused for a plain def). Python binds the instance to v on T().f(3) and raises a TypeError, which the check accepts: write self when the lambda is meant as a method. In a nested class too |
a lambda on the line of its class (class C: f = lambda: 1) or a multi-target assignment is left as written |
staticmethod(lambda a, b=2: a + b), classmethod(lambda cls: ...) in a class body (#482) |
an @staticmethod / @classmethod def of the same name |
|
a lambda default of a method (#482): def sort(self, xs: list[int], key=lambda v: -v) |
a module-level function above the outermost class (def _acs_lambda_P_3_27(v): return (-v)) that the default names; Python evaluates a default once and the lambda reads only globals, so it is the same value |
the class is inside a function or a block, or the lambda has defaults that are not literals: write a module-level def and name it as the default |
dict(a=1) |
{"a": 1} (Codon's dict takes no keywords). In an f-string field, f"{dict(a=1)}", the f-string becomes a + of its parts first (str({"a": 1})): a brace ends the field for Codon's parser (#609) |
|
dict(base, a=1) (#613) |
(dict(base) \| {"a": 1}): the same dict, the keywords winning (in an f-string field, the f-string is lowered first, as for dict(a=1)) |
dict(base, **more), or a dict the module binds itself |
filter(None, xs) (#613) |
(_acs_v for _acs_v in xs if _acs_v): the truthy items, as Python's filter yields them (Codon's takes a function) |
a filter the module binds itself; the items still need a truth value (an enum member has none, see Iterating an enum) |
items: list[int] = [], tags: dict[str, int] = {}, seen: set[str] = set() (annotated, in a class body, a function or the module, #611) |
an empty one of the annotated type, List[int](): Codon types a display from its elements, and an empty one is a container of None (a class with no __init__ of its own was refused at its constructor call) |
an annotation Codon cannot name there (Any, a TypeVar, a class defined further down outside a function) |
pairs: list[tuple[int, ...]] = [] (a local of a function, #625), then tuple displays of more than one length appended or inserted in that function: pairs.append((1, 2)), pairs.insert(0, (3,)) |
pairs = List[List[int]](), pairs.append(List[int]([1, 2])): the tuple[X, ...] inside is the List[X] a class-level one is checked as (Codon's tuples have a fixed length), so tuples of different lengths go in, and the elements are still checked. Dropped, the annotation left Codon to type the list from its first append. Given tuples of one length only, the annotation is dropped as before (the list of fixed-length tuples Codon types then takes a tuple held in a name, extend, in with a tuple display), and so is a tuple[X, ...] in a dict, a set, a nested list or an Optional |
a list given tuples of more than one length that is also given a tuple from a name (pairs.append(t)), or another way (+=, extend), or tested with (1, 2) in pairs: refused with a hint to build it with tuple([...]); a module-level one, or one a nested function appends to, is typed from its first append as before |
lst: list[Npc \| None] = [None, a], d: dict[str, Npc \| None] = {"x": None, "y": a} (a display whose first element or value is None, #611) |
the elements put into a list (dict) of the annotated type, each None the Optional's (_acs_list_into(List[Optional[Npc]](), Optional[Npc](), a)): Codon types a display by its first element |
as above |
Plan(lanes=(1, 2, 1)), replace(Plan(), lanes=(1, 2)), replace(self, lanes=()), for a dataclass field lanes: tuple[int, ...] (or Sequence[int]) of the module (#614) |
List[int]([1, 2, 1]) / List[int](), the list the field is checked as, as for its default |
a tuple held in a name, a positional argument of a dataclass with a base, ClassVar or init=False field, or replace() of an object whose class is not told by a constructor call or self: refused with 'Tuple[int,int,int]' does not match expected type 'List[int]' and a hint to write tuple([1, 2, 1]) |
Notes:
- Codon calls
__exit__()with no arguments, so a parameter isNonein the check. A branch on a parameter annotatedtype[...],objectorTracebackType(if exc_type is not None:,if tb is not None:) is therefore dead code to Codon, not compiled and not type-checked: Python runs it as written. Annotate the exception parameterexc: BaseException | None(orException | None) and test that one (if exc is not None:): its annotation is kept, so the branch is compiled and checked (str(exc)).from types import TracebackTypeis refused (typeshas no model): annotatetbasobject.__enter__returningself(-> "Ctx", or no annotation) compiles as it is, and so does-> bool(suppression).__exit__(self, *args: object)needs no rewrite. The canonical signature__exit__(self, exc_type: type[BaseException] | None, exc: BaseException | None, tb: object) -> boolraises no "Codon cannot express" warning: the dropped parameters are the ones the check leaves to theirNonedefault (#482). - Codon gives every function and lambda a type of its own, so a list or dict
of different ones does not compile without the annotation, even if you
annotate nothing else. Annotate where the table is assigned or returned; an
element of another signature than the annotation is refused, a
setof functions is refused whatever the annotation (Codon cannot hash a function), as is a dict keyed by functions (a dict with functions as values is fine): the error carries a hint to use a list or a dict keyed by name. An unannotatedfs = [inc, dbl]or a class attribute is refused with a hint: write anif/elifchain or amatchon the key instead (if key == "a": return inc(x)). A bound method (self.run) or a lambda using a local variable cannot be converted: use a module-level function that takes the object, or the chain. __ge__of a class that defines__lt__only is more than Python gives it (a >= bis aTypeErrorthere): the check accepts it because Codon's sort needs it. It does not change what runs.- A rewritten lambda no longer takes the captured parameter:
fs[0](1, 5), which passesk, is refused (with a hint naming this). A default after a non-literal one is bound the same way, so positional calls keep their meaning. Usefunctools.partialwhere you pass the captured value. - Codon's f-strings have no
{{/}}: an f-string with a literal brace becomes a+of its parts, the brace a string of its own (f"{name} {{x}}"is(f"{name} " + "{" + "x" + "}")), whether or not another of its fields is rewritten (#696). - Codon has no
ascii(), so the!aconversion of a field is checked as!r(#701): the type is the samestr; only the checked copy changes. - An f-string that is rewritten (a literal brace, a spec Codon does not
parse,
!a) is written back as double-quoted strings, so a field whose expression holds a double quote or a backslash (d["it's"],d['a\\b']) cannot stay a field. It becomes a call instead, whatever its conversion (#704):str(e)with no conversion or!s,repr(e)with!ror!a, and with a spec that call's__format__, or(e).__format__(spec)with no conversion:f"{x:+d} {d['a\\b']!r:>9} {{"is((x).__format__("+d") + " " + repr(d['a\\b']).__format__(">9") + " " + "{"). The value and itsstrtype are the same as Python's. - F-string format specs (#434). Codon's parser takes a fill and align,
#,0, a width,,/_, a precision and a type (f"{x:>8.3f}",f"{n:#010x}",f"{n:,}",f"{x!r:^10}",f"{x=:.2f}") as they are. It has no sign (+, space,-), no field inside the spec ({x:>{w}}), and it refuses a spec it does not know ({x:q}). Those fields are rewritten as above; the other fields of the same f-string stay f-string fields. The rewrite calls__format__onint,float,strand a class that defines it: aboolor a class without__format__has none in Codon (the error is hinted:int(flag)). The spec is read at run time, so a spec Python rejects ({x:q}on a float, a sign on astr) compiles in the check, which does not run; Python raises itsValueErrorwhen the scenario runs. Only the checked copy changes. - A brace or a nested f-string inside an f-string expression is refused, at the
line of the f-string: a dict or set display or comprehension
(
f"{len({1, 2})}"), a string holding{or}(f"{'{}'.format(x)}") and an f-string inside an f-string. Codon's parser ends the field there and reports a position that is no line of your file. Bind the value to a local first (label = "{:>3}".format(x), thenf"{label}"). A syntax error Codon still reports inside an f-string field is shown at the line of the f-string that has it (the first f-string with a format spec, else with a literal brace, else the file's only f-string line), with no column, and never at a line before the file: with no one f-string to blame, it is shown with no line rather than at a guessed one. - A type error inside an f-string field is shown at the f-string's line,
with no column (#610). Codon reports it on the line after the line the
f-string starts on, at the faulting expression's column plus the length of
the field's text: a column past the end of that line, or no line of the
file when the f-string is on its last line. The check moves such an error
back when nothing on the reported line starts at the reported column, of
any width (Codon locates an error on that line itself there; its width of
a call on several lines counts the characters across the line breaks,
that of a
withorforstatement is one more than Python's), and an f-string that starts on the line before has a field holding an expression of that width that ends at or before the column (and, on a one-line f-string, no further from it than the f-string's end). The line shown is the expression's own (a field on a later line of a triple-quoted f-string is shown on its line). An error of the line itself always has such an expression (a comprehension'sfor c in nclause, which Codon locates from itsfor, is matched by where it ends), so it stays where it is, column and all. An implicitly concatenated f-string ("a" f"b{n + 'x'}", also on several lines, #616) is reported on the line after its first literal's line too, at a column the bound above does not hold for: the first literal's 0-based column, plus the offset of the field's{in its own literal, plus the field's length, plus the expression's offset from the{. The check computes that column for each field of an f-string that starts on the line before (a triple-quoted literal, a field on several lines and a line that is not ASCII excepted) and moves an error that matches it, column and width, to the field's line, with the same guard (anything that starts on the reported line at that column keeps it). - A class constant read in the class body (#433): Python resolves the name in
the class scope, Codon in the module's (
name 'LIMIT' is not defined), so the check writesClass.LIMIT. It rewrites exactly what Python resolves there: a name an earlier assignment of the same body bound, read by another statement of the body, a dataclass field default, a method's parameter default or a decorator. A method body, a lambda body and a comprehension (but for its first iterable) do not see the class scope, so a bareLIMITthere is the module's and stays; a nested class sees only its own body; a parameter or comprehension variable namedLIMITshadows nothing at the default, which is read in the class scope. In an Enum a member read by a later statement is its value in Python (B = Amakes an alias), so the check writes the literal; any other name of the body is qualified as above. - The constant has to be a class attribute in Codon. A name annotated at class
level (
LIMIT: int = 5,Final[int],ClassVar[int]) is a typed field of the instance, not an attribute of the class, so a read of it cannot be qualified: when its value is a single-line literal the check writes that literal where the name is (n: int = 5; Python reads the same value), and otherwise the read is refused with a hint: writeLIMIT: Final = 5(a bareFinalis dropped, leaving a class attribute) or no annotation, or use a module-level constant. ANamedTuplefield default cannot read its own class constant either (x: int = LIMis "no module named"): its other statements and methods can.
The names of typing a scenario may import¶
Codon has no typing module of its own beyond a few names, so the check
brings a shim (typecheck/codon/typing.codon). A scenario may import these
names from typing (#364); test_typing_names keeps this list and the shim
in step:
TYPE_CHECKING, Sequence, MutableSequence, Iterable, Collection,
Mapping, MutableMapping, AbstractSet, FrozenSet, Any, TypeVar,
Protocol, runtime_checkable, Literal, IO, Iterator, Deque,
DefaultDict, Counter, Final, Annotated, NoReturn, Type, Sized,
Hashable, cast, Optional, List, Dict, Set, Tuple, Callable,
Union, ClassVar, Generator, overload, Self, override, TypeAlias,
NewType, Never, assert_never, TypeGuard, TypeIs, Container
from collections.abc import Sequence, Callable(what ruff's UP035 suggests) is read asfrom typing import Sequence, Callable(#586), on the same lines. The names ofcollections.abcthe check has areCallable,Collection,Container,Generator,Hashable,Iterable,Iterator,Mapping,MutableMapping,MutableSequence,Sequence,SizedandSet(typing'sAbstractSet); any other (KeysView,MutableSet,Awaitable) is refused at its import line with that list.import collections.abcandfrom collections import abc(annotations spelledcollections.abc.Sequence,abc.Sequence) are refused too, with the hint to import the names.typing_extensionshas the same names as far astypinghas them, so a scenario for Python 3.10 importsSelf,override,Never,assert_neverandTypeIs(whichtypinggained later) from there; any other name of it (deprecated,ReadOnly) is refused with a hint to this list.- The type-only names are rewritten before Codon reads the code (#514), and
none of them is checked:
@overrideis dropped (not that the method overrides one);X: TypeAlias = TisX = T;Selfin an annotation of a class is the class (-> Selfis-> Counter, so a subclass's method is checked as returning its base, and a call ofTwice(1).inc().twice_only()is refused although Python accepts it);NewType("LaneId", int)isLaneId = intandLaneId(v)is(v), so the new type is checked as its base (aLaneIdis anyint, as Python runs it);assert_never(x), as a statement or areturn, israise AssertionError(...), and anywhere else (a ternary, a lambda body, an argument) it is refused, since Codon types the call asNone;NeverandContainerannotations are dropped, likeNoReturnandSized;TypeGuard[X]andTypeIs[X]arebool, which they are at run time. -
A
TypeVarwith abound=or constraints (TypeVar("B", bound=Base),TypeVar("N", int, float)) is rewritten toTypeVar("B")(#513): Codon's takes a name only, and makes a function generic over its unannotated parameters anyway, so the bound or the constraints are not checked. -
FinalandAnnotatedare read as what they wrap (Final[int]andAnnotated[int, "m"]areint; a bareFinalis dropped).NoReturn,Type,Sized,Hashable,AnyandCallableannotations are dropped, so they are not checked. The exception is aCallable[[A], R]parameter whose function body stores it (self._cb = fn,self._cbs[key] = fn,self._cbs.append(fn), or hands it to a method of the same class that stores it,self._add(fn)), with argument and result types Codon can express: it is kept, written as Codon'sCallable[Tuple[A], R], so a callback registry (a class-level_cbs: list[Callable[[int], int]]thatregister()appends to) has the same type on both sides. A generic parameter put in a typedCallablecontainer made Codon abort withstd::bad_allocand no location (#488). A named function or a lambda passed to it converts, and one of the wrong signature is refused at the call. Two limits, both Codon's: a closure reading a local variable of its function cannot be converted (Codon's own refusal, with the dispatch-table hint), and neither can a lambda that takes no arguments (lambda: None; it is refused at its line after the crash, with the hint to pass a named function). A parameter the body only calls, or passes on, stays dropped, so a closure still checks. An optional callback (Optional[Callable[[A], R]],Callable[[A], R] | None) is kept the same way when the body stores it, written as the plainCallable[Tuple[A], R](Codon converts a function, a lambda andNoneto it, but a function or a lambda to anOptionalof it not), the= Nonedefault staying. The class-level attribute is writtenOptional[Callable[...]](the|between aCallableandNonecannot be evaluated by Codon when it imports the class, #495), soif self.cb is not None: self.cb(v)checks. Codon accepts a call of it without that test; the check does not refuse it. A lambda's type, which Codon names%_lambda_N, shows in a message as "a lambda". - A Codon abort with
std::bad_allocnames theCallableattributes of the checked modules at their lines; a function passed where aCallableof another signature is expected is named at the call that passes it (Codon 0.19 reports no type error for that, only a code generation failure). cast(T, x)isxtyped asT(as in Python, nothing checks thatxis aT); whenTcannot be named in Codon (aCallable, a class defined further down) it isxas it is.- A class-level
Y: Final[int] = 4stays a typed field, not a class attribute, as any annotated class-level name does; writeY: Final = 4for a class attribute read asC.Y. A read from outside the class (Q.LIMITwithLIMIT: int = 5orClassVar[int]) gets Codon's'Q' object has no attribute 'LIMIT'with a hint to writeLIMIT: Final = 5(#471); the check does not rewrite it, since in a @dataclass the annotated name is an instance field in Python too. Iteratoris Codon's generator type, andDeque,DefaultDictandCounterarecollections.deque,defaultdictandCounter.- Any other name is refused at its import line.
GenericandTypedDictcannot be modelled: leave the type parameters of a class or function unannotated (Codon makes it generic over them) instead of subclassingGeneric, and write a@dataclass(or aDict[str, X]) instead of aTypedDict.
Python 3.12+ syntax (#624)¶
The package runs on Python 3.10 to 3.14. Codon 0.19 parses Python 3.11's syntax, so the check rewrites the newer syntax first, with every line and column kept:
| Python 3.12+ | What the check makes of it | Refused (with a hint, at its line) |
|---|---|---|
A string in an f-string's field quoted with the f-string's own quote (PEP 701): f"{cfg.name.replace("_", "-")} {d["k"]}" |
the string quoted with the other quote, f"{cfg.name.replace('_', '-')} {d['k']}" (Codon's parser, as Python's before 3.12, ends the f-string at its own quote, and reported unexpected dedent a line later) |
a string that holds the other quote too (f"{d["it's"]}"), a string of either quote that holds the quote of a single-quoted f-string around it (f'{d["it's"]}', which Codon's parser ends at that quote too), and a backslash anywhere in a field (f"{"\n".join(names)}"), which Codon's parser refuses whatever the quotes: bind the value to a local first. In an f-string that is rewritten anyway (a literal brace, a spec Codon does not parse, !a), such a field becomes a call and is not refused (#704). A nested f-string is refused as before ("A brace or a nested f-string") |
type LaneIds = list[int] (PEP 695), at any level |
LaneIds = list[int], as X: TypeAlias = T is |
a generic alias, type Pair[T] = tuple[T, T]: Codon has no generic alias; write the type out, or make the alias concrete |
Type parameters of a function, a method or a class (PEP 695): def first[T](xs: list[T]) -> T, class Box[K, V]: |
kept: they are Codon's own generics syntax. A bound, constraints or a default ([B: Base], [N: (int, float)], [T = int]) are dropped, as a TypeVar's are, since Codon reads [T: int] as a static integer parameter |
*Ts and **P (PEP 646, PEP 612): Codon has no variadic generics |
A generic function or class is compiled, and checked, where the scenario
uses it (Box("a").get() + 1 is refused): like a function annotated with a
module-level TypeVar, it is not called on its own, and a warning says so
("parameter xs is annotated with a TypeVar", "the class is generic").
On Python 3.10 and 3.11 this syntax does not parse, so a scenario using it
needs Python 3.12 or later to run at all.
Python constructs Codon refuses (#379)¶
Each of these is valid, mypy-clean Python that Codon 0.19 cannot compile and that the check cannot cheaply rewrite. The check refuses it at its line with a hint, and the working spelling is below. What runs is your code as written; only the check needs the other spelling.
| You write | Codon says | Write instead |
|---|---|---|
case int(n):, case Pt(x=x, y=y):, case Pt(a, b): (a class pattern with arguments), or case Pt(): of a class of your own |
name 'n' is not defined, or it reads the pattern as a call of Pt (__init__() missing 1 required positional argument) |
if isinstance(v, Pt):, then read the attributes (v.x, v.y); case int():, case str(): and the other builtin types compile |
case {"k": v}:, case {"k": _}:, case {**rest}: (any name or _ inside the mapping) |
name 'v' is not defined |
if "k" in d: and read d["k"]; a mapping pattern whose values are literals, values or or-patterns (case {"k": 1 \| 2}:) compiles |
case [a, *rest]:, case [*_]: |
unexpected star expression |
a fixed length (case [a, b]:), or test len(xs) and slice |
case [a, b] as whole: |
unexpected indentation |
match without as and use the name the subject already has |
def show(h: HasArea) called with a class that has area() but does not derive from the Protocol HasArea |
'Sq[float]' does not match expected type 'HasArea' (also List[Sq] against List[HasArea]) |
derive from it (class Sq(HasArea):), give the classes a common base class, or annotate Union[Sq, Circle]; Codon lists are invariant: annotate a list with the class it holds (list[Sq]) |
f(*xs) with a list, f(**d) with a dict |
argument after * must be a tuple, not 'List[int]' / argument after ** must be a named tuple, not 'Dict[str,int]' |
pass the elements (f(xs[0], xs[1]), f(a=d["a"], b=d["b"])), or make the parameter a list or dict; a tuple (f(*(1, 2))) is accepted |
try: ... except* ValueError as eg: (exception groups, #439) |
scenario.py:70:1: error: unexpected dedent, at the line after the handler |
except ValueError: (one handler per exception type); Codon has no syntax for except* |
s.startswith(PREFIXES) with a tuple held in a name (#439) |
'Tuple[str,str]' does not match expected type 'str' |
any(s.startswith(p) for p in PREFIXES); a tuple written in the call is rewritten |
d.update(c=3) (#439) |
no function 'Dict.update' with arguments (Dict[str,int], int) |
d.update({"c": 3}) (the check cannot rewrite it: it does not know d is a dict, and obj.update(dt=1) of your own class is valid) |
int.from_bytes(b, "big"), n.to_bytes(2, "big") (#439) |
'int' object has no attribute 'from_bytes' |
shifts and masks (n = (n << 8) \| byte); Codon's bytes is a str |
"{a}".format_map(d) (#439) |
'{a}' object has no attribute 'format_map' |
an f-string, or "{a}".format(a=d["a"]) |
b.decode(), b.hex() on a bytes value (#439) |
'str' object has no attribute 'decode' |
Codon's bytes is a str: it is already text |
A class decorator other than those the check drops (@register, adding the class to a registry, #603) |
unsupported class decorator |
drop it, and choose the class with an if/elif chain (or a match) on the name: Codon cannot hold a class as a value. @dataclass, @enum.unique, @typing.runtime_checkable and @functools.total_ordering are handled by the check |
async def, await, async for, async with anywhere in a checked module, called or not (#603) |
async not yet supported |
a plain def: the runner calls the scenario synchronously; or move the coroutine to a module the scenario does not import |
if obj: (not obj, bool(obj)) on an object of a class with no __bool__: a plain object, or one with only __len__ (#603) |
'Bag[List[int]]' object has no attribute '__bool__' |
say what is meant: obj is not None for a value that may be missing, len(obj) > 0 for a container, or define __bool__ on the class |
class Registry(dict[str, int]), class Bag(list), class Meters(int): a class deriving from a builtin type alone (dict, list, set, frozenset, tuple, int, float, str, bytes, or their typing names), #613 |
__init__() missing 1 required positional argument: '_n_buckets', at the call (Codon's builtins are its own classes); refused at the class line |
hold one as an attribute (items: dict[str, int]) and give the class the methods it needs; a class with another base besides (class Mode(str, Enum)) and exceptions are left alone |
x = a if c else 3 with branches of different types (#603) |
'str' does not match expected type 'int', at the conditional expression |
one type for both branches (str(3), float(n)), or an if/else statement that uses each value in its own branch |
lambda a, *, k=1: a * k, a lambda with keyword-only parameters (#629) |
a syntax error (unexpected dedent); refused at its line before Codon runs |
a nested def f(a, *, k=1): return a * k, whose keyword-only parameters the check compiles (#622) |
A call of a method that a subclass of yours overrides, passing a keyword (b.m(1.0, s=3.0)), or leaving out a parameter with a default (b.m(1.0)), keyword-only parameters included (#629) |
's' is an invalid keyword argument for class_thunk_dispatch(), or cannot typecheck 'b.m(1.0)', with a hint |
pass every argument by position: Codon calls an overridden method through a dispatch thunk, which takes no keyword and fills in no default. A keyword-only parameter cannot be passed by position, so in an overridden method make it a positional one with its default, in the base and in each override |
A function with keyword-only parameters in a table annotated Callable (fs: list[Callable[[int], int]] = [key_of, dbl] with def key_of(x, *, rev=False), #629) |
no function 'Callable.__new__' with arguments (key_of[int,T2,bool]), with a hint naming it |
a lambda that calls it, [lambda x: key_of(x), dbl]. An annotated variable (g: Callable[[int], int] = key_of) and a Callable parameter take it as it is |
A nonlocal variable that may be None used as its value, even narrowed (out = y + 1 if y is not None else None in the inner function, #629) |
unsupported operand type(s) for +: 'Capsule[Optional[int]]' and '1', with a hint |
copy it into an annotated local of the inner function (cur: int \| None = y), and test and use that; an unannotated copy is a Capsule too |
Plain value, or-pattern, guard, sequence (without *) and enum-member
(case Mode.A:) patterns compile, as does case int(): of a builtin type.
A Protocol is not structural in the check: Codon compares class names, so a
class is a HasArea only if it lists HasArea among its bases. Python runs
the same code either way, because a Protocol base is only a marker there.
What Python raises for that Codon compiles (#602, #603)¶
Codon checks types, so a call it compiles may still raise in CPython. Three
kinds are decided from the syntax and refused at the call, before Codon
runs (typecheck/static_raises.py):
-
sum()with astrstart (#625):sum(["a", "b"], ""),sum(parts, start="-"), a start that is an f-string orstr(...), and likewise abytesorbytearrayone. CPython refuses it before it adds anything ("sum() can't sum strings [use ''.join(seq) instead]"); Codon concatenates. Use"".join(parts). A start held in a variable is left to the run. -
isinstance/issubclasswith a subscripted generic (#603):isinstance(v, list[int]),issubclass(C, dict[str, int]),isinstance(v, typing.List[int]), also inside a tuple of classes ((str, list[int])), a|union (int | list[int]) or atyping.Union/Optional. Python raisesTypeError("isinstance() argument 2 cannot be a parameterized generic"; fortyping.List[int]and the other aliases oftyping, "Subscripted generics cannot be used with class and instance checks"); in a tuple or a union, once the value is none of the classes before it (a|union on Python 3.10: at once). Test against the bare class (isinstance(v, list)) and check the elements separately; for aLiteral[...], compare the value.isinstance(x, int | float), an unsubscriptedtyping.List,Optional[int]and an index into a tuple of classes (TYPES[0]) are left alone, and so is a name the module binds itself (a parameter calledlist). - A framework class built with literal arguments it refuses (#602):
MetricCondition("no_such_metric", label="m"),ElapsedTimeCondition(1.0, label=""),RouteCrossingPose(approach="Left"),RouteRoadsidePose(side="kerb"). When every argument of a call is a literal (a constant, or a tuple, list, set or dict of them, by position or by keyword) and the callee is imported from the framework, the check builds the class with those literals and reports theValueErrorit raises, in the framework's own words:
scenario.py:71:34: error: RouteCrossingPose(...) with these literal arguments raises ValueError when the scenario runs: RouteCrossingPose.approach must be one of ['left', 'right', 'opposite'], got 'Left'
The validation is the constructor's own, so the check refuses what the run
would and nothing else. Only classes whose constructor just validates and
stores its arguments (no CARLA, world or map) are built:
AlwaysTrueCondition, ElapsedTimeCondition, MetricCondition,
OffRoadFractionCondition, RouteProgressCondition, TimeoutCondition,
RouteCrossingPose, RouteCrosswalkPose, RouteLanePose,
RouteOppositePose and RouteRoadsidePose (LITERAL_CHECKED). A call
with any argument that is not a literal (a config field, a constant's name
such as OFFROAD, an enum member such as ComparisonRule.LESS_THAN,
*args) is left to the run, as is a TypeError (Codon reports the type)
and a literal of another type than its parameter declares (a str for a
float, an int for a str: Codon reports the mismatch, though the constructor
would raise ValueError for it; an int for a float fits).
The check imports these classes only for a module that calls one of them
by name. The conditions import typesafe_carla.carla: where its first
import would build the CPython package (no prebuilt one applies, 15-50
min), the check does not build it, and leaves the conditions to the run.
Standard library names Codon lacks¶
Codon ships its own functools (only partial), itertools, collections,
math, random and statistics, and a name it lacks fails with a bare
"cannot import name". The check supplies these (#378, #515, #516); test_stdlib_names keeps
this list, the check's additions and the tests in step:
functools.reduce, functools.lru_cache, functools.cache,
functools.cached_property, itertools.pairwise, collections.OrderedDict,
collections.deque, math.dist, math.isqrt, math.comb, math.perm,
math.cbrt, math.exp2, math.hypot, random.Random, random.choices,
random.triangular, statistics.fmean, re.Match, re.Pattern
functools.reduce(with or without an initial value) is a function of the model, which the check puts in place of the name.lru_cache(with or without arguments),cacheandcached_propertyare decorators the check drops, as caching changes no type;cached_propertybecomes aproperty. Nothing is cached, sof.cache_clear()andf.cache_info()are refused, and so is assigning to acached_propertyattribute (self.area = 3is legal in Python, not on aproperty). A name the module binds again (a parameterreduce, adef cache) is left alone. The names are handled where the module imports them, not in a function.- A decorator the check drops (
lru_cache,cache,@abstractmethod,@override) may sit under one it keeps (@property,@staticmethod,@classmethod), as in@propertyover@abstractmethod(#595). Codon's parser refuses a blank or comment-only line inside a decorator list, so the check removes the dropped decorators' lines, and any comment or blank line, between the first kept decorator and thedef, and moves the kept decorators down to thedef: thedefand every line after it keep their line numbers, and an error on a kept decorator is reported at the line it was written on. A decorator is its whole logical line, from its@to its end (@(/name/)on three lines). - The initial value of
reducemust have the type of the accumulator the function returns:reduce(add, [1.5, 2.5], 0)is refused (write0.0). itertools.pairwiseis added to Codon'sitertools, andcollections.OrderedDictis adict, which keeps its insertion order;move_to_end()andpopitem(last=...)are not declared, and are refused when called.collections.dequegets what Codon's lacks:deque(iterable, maxlen),extend,extendleft,rotate,count,index,insert,remove,reverse, item assignment and deletion and==/!=, all as Python's (maxlendrops items from the other end,rotate(-n)turns left). Codon's owndeque(maxlen=n)kept one item too many whenn + 1was a power of two; the check's is fixed.deque(iterable, maxlen=None)andreversed(d)are not declared, andd.maxlenis-1, notNone, for a deque with no limit (Codon's).math.dist,isqrt,combandpermare Python's (the sameValueErrors); anintis 64 bits in Codon, socombandpermoverflow where Python's would not.math.cbrtandmath.exp2are Python 3.11 functions: the check accepts them on 3.10 too (run time is where 3.10 refuses them), so a scenario meant for 3.10 has to avoid them itself.nextafterandulpare not declared.math.hypottakes any number of coordinates (Python 3.8's, #590),hypot(x, y, z)included; Codon's took exactly two, and a call with two keeps Codon's (and itsfloat32one).statistics.fmean(data)is Python's mean of the values as floats (#590); Codon's own is commented out.fmean(data, weights)is the weighted mean of Python 3.11, accepted on 3.10 too, likemath.cbrt. The rest ofstatisticsis Codon's (mean,median,stdev, ...), which takes alist[float].re.Match.group(n),group(a, b),m[n],groups()andgroupdict()arestr(#597), as typeshed declares them (str | Any), som.group(1) if m else ""compiles; Codon's areOptional[str], and theirOptional[str]and the""did not unify. The caveat is typeshed's too: a group that did not take part in the match ((-x)?) isNonein Python, which the check does not see: test an optional group withis not Nonebefore using it as astr(the check compiles the test), or read it throughgroups(default)/groupdict(default), which keep Codon'sOptional[str].re.Pattern.splitkeeps itsOptional[str]items, a group that did not take part splitting off asNone, as in Python. The rest ofreis Codon's.random.Random.choices(andrandom.choices) takeweightsof ints or floats, orcum_weights=, andk=, as Python's do, andtriangular(low, high)(the method and the function) takes the midpoint as the mode. Codon'schoiceswanted int weights and all four arguments.cum_weightsandkcan be passed by position here, where Python makes them keyword-only.
These are refused at the import line, with a hint saying what to write instead:
dataclasses.asdict, dataclasses.InitVar, dataclasses.KW_ONLY, functools.wraps, math.ulp, contextlib.contextmanager, warnings,
json.JSONEncoder, types, textwrap, traceback, io, csv
dataclasses.asdictreturns adict[str, Any]: build the dict from the fields you need. Thedataclassesshim declares onlydataclass,fieldandreplace(a call is rewritten and checked per field, see "Your own dataclasses, enums and named tuples").dataclasses.InitVarandKW_ONLY(#590): Codon's constructor of a dataclass takes every annotated field and does not call__post_init__, so an init-only value has nowhere to go and theKW_ONLYmarker would be a field. Pass anInitVar's value to a@classmethod(or a function) that computes the fields and builds the instance, or keep it as an ordinary field; dropKW_ONLYand pass the fields by keyword (a field without a default then goes before the defaulted ones).functools.wraps(and any other name offunctools,itertools,collections,mathorrandomnot listed above, such asmath.ulp) is not modelled.contextlibdeclaressuppressonly: write a class with__enter__and__exit__(self, exc_type, exc, tb)forcontextmanager(the check rewrites the signature, see "Python the check rewrites for Codon").warnings: log withlogger.warning.jsondeclaresdumps,dump,loads,loadandJSONDecodeErroronly (see the shims below), notJSONEncoderor the rest.types: name the type itself.textwrap: build the string withstrmethods.traceback: log withlogger.exception.io: use astr, or apathlib.Pathopened withopen().csv: read the file withPath.read_text().splitlines()andsplit(",")each line, and write one with",".join(fields)andPath.write_text().time.strftime()anddatetime.strftime()are not modelled (Codon'sdatetimehas neither, nor__format__): the error says so; format the fields instead,f"{now.year:04d}{now.month:02d}{now.day:02d}"fornow = datetime.datetime.now().- A name of
os,os.path,shutilor a member ofpathlib.Paththat the shims below do not declare is refused with Codon'scannot import nameor'Path' object has no attribute, to which the check adds that it is not modelled (the shim declares only the names listed below), so it does not read as invalid Python.
Iterating an enum¶
For an Enum class, a scenario's own (any of Enum, IntEnum, StrEnum,
Flag, IntFlag, (str, Enum)) or one of the framework's (TurnDirection,
ComparisonRule, TickTiming, ...), the check rewrites, as Python reads it:
for m in Modeand a comprehension overMode, and*Mode;list(Mode),tuple,set,frozenset,sorted,len,reversed,enumerate,iter,min,max,zip,mapandfilter(with a function) ofMode;x in Modeandx not in Mode;Mode["A"], a member looked up by name.
Inside the enum's own methods the class has other spellings, and the check
reads them as the class (#481). In a @classmethod, the first parameter
(cls, or whatever it is called) is the class, so [g.name for g in cls],
list(cls), len(cls), x in cls and cls["A"] work. In an instance
method, type(self) and self.__class__ are the class too
(for m in type(self)). These hold only in a method written directly in the
class body: a @staticmethod, a function outside the class and a class that
is no enum are left alone, and so is a parameter the method rebinds (an
assignment, a loop or comprehension variable, an import, a nested def), a
nested function or lambda that declares a parameter of the same name, and a
type the module or function binds.
Iteration yields the members in definition order. A member whose value repeats
an earlier one is an alias, and is skipped, as in Python. A Flag or IntFlag
yields its single-bit members only, as Python 3.11 and later do: a composite
(RW = R | W, BOTH = 3), a zero member and an alias are skipped, though
Perm["RW"] finds them. Only the name of the class (and the spellings above) is rewritten, so a
parameter or local called like the class is left alone, and the enum may be
defined in another module of the package.
The framework's enums are @tuple classes in the model
(typecheck/codon/). Each declares _acs_members() and _acs_by_name(), and
the check takes the enums to rewrite from the model files, so a new model enum
is iterable once it declares them. test_typecheck_model.py keeps both equal
to the Python enum's members, in its order. A scenario class of the same name
as a model enum that is no enum is left alone, and so is a module-level
variable, function or non-framework import of that name (#471); a parameter or
local of the name shadows it only inside its function.
What does not work:
- the class under another name or as a value:
k = clsand thenfor m in k, orf(cls), andcls.__members__. They stay as written (the first is refused with a hint to write the class's name:'Mode' object has no attribute '__iter__'); - the class passed as a value,
f(Mode), orMode.__members__: both stay as written, which Codon does not resolve (no function 'List.__init__' with arguments (List[T], type[Mode])forlist(e)inf, and'Mode' object has no attribute '__members__'); sum,anyandallof the class, andfilter(None, Mode): Codon needs__bool__or+on the member (sum(int(m) for m in Mode),any(m == x for m in Mode));- an enum with no member (
'Empty' object has no attribute '_acs_members').
Flag and IntFlag¶
A scenario's own Flag or IntFlag (also class M(int, Flag)) combines as in
Python: A | B, &, ^, ~ and A in x (every bit of A is set in x)
make a value of the same class, an IntFlag takes an int on either side
too (P.R | 4, 4 | P.R), and bool(x) is false for no bit set. Two
combinations of the same members are equal, hash alike and compare equal to
the member that has their value (P.R | P.W == P.RW). A Flag does no
other arithmetic, order or int(); an IntFlag compares, orders and converts
as an IntEnum does, and P.R + P.W is refused (use .value). ~x flips the
bits of the single-bit members for a Flag, and every bit up to the highest
of any member for an IntFlag, as Python 3.11+ does. The operators are usable
at module level and in a class body or default (RW = P.R | P.W).
A combination is no member: the check builds it from the combined int, so it
has an empty name, and P.R | P.W is P.RW is not modelled (compare with
==). A value written as an expression of the class's own members, even a
composite or an auto() one, is worked out when the check reads the class
(RW = R | W, ALL = A | B | C, HIGH = 1 << 4, ALL = ~0); auto() is the
power of two above the highest bit so far. Operators of two different flag
classes, and of a framework enum such as TurnDirection.LEFT |
TurnDirection.RIGHT (not a Flag, so Python refuses it too), are refused.
What the check does not catch¶
The check compiles the scenario and never runs it, so it says nothing of what a value is at run time (a division by zero, a missing key, an actor that is not spawned yet), and it accepts some code that fails in Python. The known gaps:
- An
Optionalunwrapped implicitly, where mypy cannot see it. Codon unwraps anOptional[T]wherever aTis needed (arithmetic, an attribute, an argument), and mypy's pass (#239, What is checked) catches what mypy can type, CARLA's results included (from stubs of typesafe_carla's Codon library, #267). It does not see a value mypy reads asAny: one from a package with no type information (an import it cannot follow, which it ignores, such aslanelet2), a CARLA result whose type the stubs leaveAny(an enumeration,carla.command, a pointer, a method whose overloads return different types), or the result of a function with no return annotation. Neither does it check the arguments of a CARLA call: anOptionalpassed to one is Codon's to unwrap (world.get_actor(maybe_id)compiles). - A union of the scenario's own classes used as one of its members.
The None check's pass refuses only a union made of the members of one of
the framework's (#309, What is checked): a union of
classes the scenario defines (a class-level
tuple[A | B, ...], held asList[Union[A, B]]) is converted at run time as Codon converts any union, and an un-narrowed use of one member's attribute compiles. - An attribute set on some paths only. Codon zero-initialises every
field, and the class rules (#587, #588) follow no paths: an
__init__that callssuper().__init__(...)in one branch only, or an attribute assigned in a method that may run after its first read, compiles and can raiseAttributeErrorin Python. A class that is not a module-level name (one defined in a function) is not given the rules at all. - A helper with
**kwargsor*argsthat nothing calls and that reads**kwargsby key is not compiled, with a warning: the check has no keyword to call it with (#266). One the scenario calls that reads them is compiled only with the keywords and the number of values its callers pass, and, as a call is known by its name only, not at all when the only calls of its name are another's (d.update(...)for a helperupdate) or are in a function the check does not compile. - A value converted by an annotated assignment. Codon 0.19 converts any
value with
__bool__in an annotated assignment:done: bool = "no"compiles, and isTrue. (A literal returned from a function declared-> boolis refused, #529.) The check binds hook results strictly (#238) and class-levelbooldefaults through_expect_bool(#190), but an annotated local of the scenario's own code is converted as written. A localx: bool = some_stris noboolin Python. - An
intwhere afloatis expected is accepted, in a call and a hook result alike, as Python's typing takes it; not inside a container ([1, 2]for alist[float]is refused). - A function the check cannot call (a parameter with no annotation, or
one of
Any,object,Callable[..., R]or aTypeVar, or a return annotation namingAny) is compiled only wheresetup()andis_done()reach it, with a warning (What is checked). - A hook a class takes from a mixin listed before its framework base
(
class Gap(CheckMixin, BaseCondition), withcheck()defined inCheckMixin) is compiled as the mixin's own method, not checked against the hook's return type: define the override in the class itself. - A
Nonedefault is not checked against its annotation:speed: float = Noneis anOptional[float]to Codon (mypy, in the scenario package's own pre-commit, reports it; the None check reports only its uses). - What only a tuple does: a
tuple[X, ...]is checked as aList[X], so its hash, its equality with another tuple and its immutability are not checked. - An
intis 64 bits, and no value is computed (#613): Codon'sintis a 64-bit integer, and the check compiles the code without running it, so an overflow (2**64,math.factorial(25)) and2 ** -1(afloatin Python) assigned to anintare not caught. bytesis read asstr:path.read_bytes() + "text"compiles, and CPython refuses it.- A
values=that only Codon iterates (register_cover(),OddAttribute, #279). Codon 0.19 cannot tell a class passed as an argument from one of its objects, so an enum member passes where an enum class is meant (values=TurnDirection.LEFT). A frozen dataclass of the framework (a@tuple, so iterable to Codon) passes too, and so does an element typeexpressionnever returns. - An enum member looked up by value (
ActionState("completeState")) is typed, but not resolved to the member. Lookup by name and iteration work for a scenario's own enum, within the limits of Iterating an enum. - typesafe_carla's Python-API compatibility shortcuts (a
Vehiclemethod called on a plainActor) compile, as they do outside its strict mode. - A scenario registered with
register_scenario_builder()is not checked at all: only its builder knows how it is built. - The
Nonebranch of a conditional expression chosen statically (#629). The check makes Codon decidea if x else None(andNone if x else a) fromx's type (#597): wherexis alwaysNoneto Codon, only theNonebranch is compiled. When the test does not narrow as Python does (None if x else x.a, which evaluatesx.aexactly whenxisNone), the error in the other branch is not compiled. mypy's pass reports it wherexis annotated (Item "None" of "Npc | None" has no attribute "a"); an unannotated parameter only ever givenNonepasses. - A
sum()whosestrstart is a variable (sum(parts, sep)), which CPython refuses: only a start written as a string, an f-string orstr(...)is decided from the syntax (#625). - A value a framework constructor refuses, unless every argument of
the call is a literal and the class is one of those
built with literal arguments
(#602):
MetricCondition(cfg.metric, label="m")with an unknown metric, or any other framework class, raises atsetup().
Checking the framework itself¶
The scenario check trusts the model: it never compiles the framework's own source. The library check does, a module at a time, as each is made to compile:
uv run scenario-check --library # every checked module
uv run scenario-check --library autoware_carla_scenario.kinematics.vector
from autoware_carla_scenario.typecheck import typecheck_library
result = typecheck_library() # the modules typecheck/library.py lists as checked
assert result.ok, result.format()
test_typecheck_library.py makes the same check in the test suite.
A name after --library that is not a module of the package, or that
typecheck/library.py lists as EXCLUDED, is refused ([FAILED] ..., exit
1) before Codon is looked for, so the refusal is the same whether Codon is
installed or not.
The checked modules get the None check too (#277), as a scenario's
modules do: after the compile, mypy runs over every module the run compiled,
with the CARLA stubs, and an Optional used without a test for None fails
the check at its line. Codon unwraps an Optional in the framework as it does
in a scenario, so the compile alone accepted CollisionCondition reading
CollisionEvent.other_actor (None for a collision with map geometry) as a
carla.Actor, and the sensor callback raised in Python:
conditions/collision.py:158:32: error: Incompatible types in assignment (expression has type "Actor | None", variable has type "Actor"): Codon unwraps an Optional implicitly, so test the value for None before this use, or Python fails here when it is None (mypy: assignment)
It adds about 11 seconds to a run with mypy's cache cold, and under a second
with it warm, to a compile of about a minute. A verdict kept between runs
(below) holds the None check's too: the key already covers what it reads.
test_typecheck_none_check.py runs the same None check over CHECKED
without Codon, so a machine without a Codon compiler catches it as well.
So do the class rules (#587, #588): before the compile, the library
check imports each module it compiles and refuses an __init__ that skips
a base's (any class of the package counts as the framework here) and an
attribute declared at class level that nothing assigns, as the scenario
check does (Writing a scenario that type-checks).
A module that does not import in Python has its classes left alone.
test_typecheck_class_rules.py runs both rules over every module of the
package without Codon.
A run that names modules is a slice of the full run. With the named
modules, it compiles from their source the CHECKED modules they import,
transitively. It does not follow an import that MODELLED_IMPORTS routes
through the model (an import cycle Codon cannot resolve, or a call of an
UNCALLED function of the imported module, see A call of another checked
module's UNCALLED function below), or one inside an UNCALLED function. It also brings the
other modules of a REPLACED_MODELS entry once one of them is in, so that the
model re-exports the checked definitions, as it does in the full run.
scenario_base, for example, imports register_vehicle_entity from the
checked entity.registry, which the model does not declare, and
entity._spawn holds the checked GroundProjectionConfig, which the model
VehicleEntityConfig must share. The modules it imports are checked as
fully as the named ones: their functions and methods are called too, and errors
are reported for every compiled module, not only the named ones. An error
in a module the named one imports is an error the full run reports as well,
and it would stop the named module from compiling anyway. Hiding it would
only make the run fail without a reason. The result's title gives both
counts (1 checked modules, 44 more they import for scenario_base, at the
time of writing). Run without names, the check
compiles exactly CHECKED, as before. Every checked module passes on its own.
The check's driver imports the model before any checked module, whichever
module it names first. The model reaches the checked coordinate.frames
through _poses.codon, and a run that started at coordinate.frames would
reach the model, and come back to coordinate.frames, before that module
defines anything.
The manifest: typecheck/library.py¶
CHECKED lists the modules compiled from their source; EXCLUDED maps every
other module to the reason it is not. "not yet checked (#45)" marks a module
nobody has made compile yet; any other reason names what keeps a module out
("imports omegaconf, hydra": a package or a standard module Codon cannot
compile). numpy is not one: Codon ships its own, and a module's numpy calls
compile. Every module of the package except typecheck/ is in exactly one of
the two, which the test suite checks, so a new module needs an entry.
UNCALLED names the few functions and methods of a checked module the
check leaves out (module.Class.method, or module._helper for a protected
one), each with its reason: one whose job Codon cannot express, in a module
that otherwise compiles, such as what builds a condition's details (a
dict[str, Any] of mixed value types) or reads a config node of any type
(What is checked of the framework below lists every entry). Codon compiles a function
only when something calls it, so neither its body nor an import made inside
it is compiled: a module that needs a package or a standard module Codon
lacks for one function imports it inside that function (import yaml in
constraints_from_text()) and lists the function here. (json is not one of
them: its shim, typecheck/codon/json.codon, writes any one-typed value and
reads opaquely; what keeps to_json() out is the mixed-type dict it writes.) A test checks that every entry names a function or
method of a checked module.
To check one more module:
- Move its entry from
EXCLUDEDtoCHECKED.CHECKEDholds one module a line, sorted (a test keeps it so), so the move is one line removed and one added, each where no other pull request's lines are. - Run
uv run scenario-check --libraryand fix what it reports, in the module. Only annotations and Codon-friendly rewrites: the module must behave exactly as it did. A function that cannot compile gets anUNCALLEDentry, one line, with its reason. - Run
uv run pytest autoware_carla_scenario/test/carla_scenario/test_typecheck_library.py.
This page needs no edit: the counts and the lists of What is checked of the
framework are generated from typecheck/library.py when the docs are built
(mkdocs_hooks/typecheck_library.py, #789). A checked module is listed with
the first paragraph of its docstring, so give it one that says what the
module is.
What is checked of the framework¶
Every module now has an entry other than "not yet checked (#45)".
348 of the package's 479
modules (73%; about 110,800 of
186,700 lines) are CHECKED: the kinematics, the
coordinate frames and poses and the conversions between Lanelet2, OpenDRIVE
and CARLA world poses, the map manager and the map sources, the route search,
the trajectories, the conditions, the actions and the entities, the scenario
base, its runner and its queue, the traffic backends' interface, the
plain-typed parts of the SUMO backend (its geometry, its physics control, the
signal states it sets, and its command-line options, mode bits, vehicle types
and type files, with the command line of randomTrips.py and the options of
each demand it generates: traffic.sumo.options, #1679, and where SUMO is,
the network's files and the demand randomTrips.py generates for it:
traffic.sumo.network, #1967, and the crossings read off the network
with xml.etree.ElementTree, the conversion of the map with roadgen and
the signal table loaded from roadgen's traces, #2312; not what the table asks
the traces, which the check leaves uncalled), the signal
controllers, the adversity math, the measures, the metrics (and which of the
CommonRoad metrics are heavy, the environment that turns them off and the
checks of their periods: heavy_metrics, #1689, and the commonroad
package's __init__, which re-exports them; not commonroad.session,
which runs the workers and re-exports it too) and the ODD's
probes (and the names, knob keys, limits and ranges of the traffic
parameters' probes: odd.traffic_ranges, #1680; not the probes themselves,
odd.traffic_probes, which builds openodd's knobs), the gate package's
__init__, its per-run record and probe, finding and writing the markers
of the --multirun jobs given up, and finding and writing the plans of
the sweeps (gate.errored and gate.sweep_plan, #1967; a plan written a
member at a time, as a marker is, #2265; not reading either, JSON objects
of mixed value types), what a set of runs
left unrecorded (gate.unrecorded, #2124: the batches under a directory, a
local batch's summary read for its errored and pending cases, and the
--multirun jobs given up or never started; not a SLURM batch's manifest,
which gate.evaluate reads; it re-exports them),
and the typed criteria its evaluator judges (gate.criteria: the criterion
results, the trajectory consistency, the case coverage and the changes from a
baseline; and gate.report, #1738: a verdict's and a comparison's Markdown,
the criteria and the cases compared from a baseline's runs to a candidate's,
and the records grouped by scenario; not the evaluator itself, which reads the
gate's pydantic model and openodd's merged coverage), the HILS bridge's wire format (a command's values included)
and its configuration's sensor spec and the check of a field's signal name
(not the YAML reading around it, hils.config; the hils package's
__init__, which re-exports them, is checked), the virtual devices
scenario-hils --setup creates for a configuration's devices, each validated
in turn (hils.host_devices, #2124; not the playbooks' variables, a dict of
mixed value types, or running them with ansible-runner, hils.host_setup,
which re-exports it), how detect-no-3d-model writes the map configuration
it edits, through a temporary file or in place, and reports the error of
either (tools.file_writes, #2135; not the edit of the YAML text, the CARLA
ray casts or the command line, tools.detect_no_3d_model_lanelets, which
re-exports it), how an error shows an integer too long for
repr() (value_text.huge_integer(), which hils.config re-exports, #1704), the
example scenarios and their configs, the SLURM launcher's job scripts (and
how one is written, #2124), where
a batch's files are and which submissions it holds (slurm.scripts, #1648),
and its calls of SLURM's command-line tools (sbatch, squeue, scancel,
scontrol, sinfo: slurm.queue, #1634), the naming rules of
scenario-new (scaffold.names, #1654) and those of the Scenario Editor's
export, which ids it refuses and the framework pin's lines in
the package's README (authoring.package_names, #1695; the rule against the
framework's own name is scaffold.names', which both apply), the whole
numbers the Scenario Editor's forms post, a list field's ids and a record
list's row count (editor.form_values, #2135; not the parsing of a form
against a field's spec, editor.forms, which re-exports them), where the viewer keeps its
recordings and its copy of Rerun's web viewer, and which web viewer it picks
(ui.web_viewer_paths, #1672), and how it puts them in place on disk
(ui.recording_files, #2124), what the viewer reads off a request and links
a session with (ui.request_values, #2135: whether a body is sent as JSON, a
session's reference, the ODD attribute a query names; not ui.app, which
serves the requests and re-exports it), what a run names its outputs, the path
rule of a trajectory log and its recordings among them (run_outputs, #1678:
the stem a log is of, its recording and its sensor recording, which rerun,
rerun_lidar and ui.web_viewer_paths all name them with; ui.case_paths
names a case's log and recording with it too; the suffixes of a run's result
and coverage file are its too, which scenario_runner, ui.case_paths,
ui.scanner and examples/run.py name them with, #1704; so are those of a
run's environment and lock and of its T4Scene and map, which environment and
t4_scene.convert re-export, #1713; so are those of the metrics' scratch
recording, which rerun_metrics re-exports, and of the ODD archive, which
ui.case_paths names, #1748), the typed core of the sensors
in a run's recording (a camera's mount, the colours of a LiDAR sweep, which
entities are the sensors', the parse of rerun rrd print and the scan of a
recording's bytes for those entities: rerun_sensor_mount, #1688, #1697; not rerun_lidar, which holds the Rerun SDK as
Any and re-exports it), how converting a run's logs into Rerun recordings
went and the run's summary of it (rerun_report, #2124; not rerun, which
runs the converter on a thread pool and re-exports it), the frame math
of the T4Scene converter (CARLA's world frame to Autoware's map frame, and the rotations
and quaternions of its angles, and the check of the ego's rear-axle offset:
t4_scene.frames, #1665, #1697; the converter
itself, which reads the log's mixed-type records and builds protobuf
messages, re-exports it), the OSM bytes of a T4Scene's map resource and the
digest that names them (t4_scene.osm, #2116; t4_scene.map_resource,
which builds the protobuf MapResource, re-exports them), the file names of a driver policy's map set and the
id it is written under (driver.hdmap_files, #1728; driver.hdmap_export,
which writes the set with roadgen, re-exports them), what the weather rules
read and which vehicles entry of environment.rules a vehicle follows
(weather_inputs, #1732; weather_rules, which reads the rules with openodd,
re-exports it), the fields of a control and of sweep.odd_sample and the keys
a sweep sets (odd.control_fields, #1733; odd.controls and odd.sampler
re-export them), what scenario-odd route reads off its command line and the
metres per bucket it prints (odd.route_text, #2135; not odd.cli, which
expands the scenario configs' globs and re-exports it), which sections of a
run's config a scenario's digest reads and which of their settings it leaves
out as endpoints (scenario_hash; not the digest itself, which walks a
config of any shape and is left uncalled), and the parts of the
tooling around a run that need nothing Codon lacks. A checked package's
__init__ re-exports what is not checked through models of it (see A class
a checked package only re-exports below). Each checked module is listed here
with the first paragraph of its docstring:
The 348 CHECKED modules
| Module | What it is |
|---|---|
action_state |
The lifecycle state of a scenario action. |
actions |
Actions that execute side effects in response to conditions during scenarios. |
actions._adversarial_execution |
Drive a planned adversarial command, then hand the vehicle back (#391). |
actions._departures |
When an entity following a trajectory is where: the departures it knows of. |
actions._road_threat |
When a pedestrian on (or at the edge of) a road has a vehicle too close. |
actions._route_pacing |
Pace an entity's own driver to its waypoints' times, and log the arrivals. |
actions._trajectory_run |
Move an entity along a trajectory: the engine behind RoutingAction. |
actions.adversarial |
TeraSim's adversities as actions (#393, #398, #399, #400). |
actions.adversarial_traffic |
NADE as an action: adversarial traffic around the ego, importance sampled (#395). |
actions.background_traffic |
Background traffic: vehicles that come and go without a scenario entity behind them. |
actions.base |
Base class for conditional actions executed during scenario tick loops. |
actions.communication_loss |
Communication loss action: stop data between the HILS bridge and its ECUs. |
actions.construction_zone |
TeraSim's construction zone as an action: cones, barriers and signs on a lane (#402). |
actions.despawn |
Take a road user out of the world for good. |
actions.emergency |
Emergency action: put an emergency vehicle on a call, or take it off one. |
actions.environment |
Environment action: set the world's weather and sun position, the road's grip and how far drivers see. |
actions.lane_change |
Lane-change action: ask a vehicle's traffic backend for a lane change. |
actions.perception_fault |
Perception fault injection: report one road user to the ego's driver as something it is not. |
actions.play_motion |
Play-motion action: pose a pedestrian after a SOMA motion file. |
actions.routing |
Send an entity along a route: through its waypoints, on time, to a goal. |
actions.set_speed |
Set-speed action: command a vehicle's target speed during a run. |
actions.spawn |
Put something into the scene mid-run -- the counterpart of despawn. |
actions.stalled_object |
TeraSim's static adversities: a stalled object and collision objects (#401). |
actions.traffic_parameters |
Change the traffic's parameters during a run. |
actions.traffic_signal |
Traffic signal action: control traffic lights via Lanelet2 regulatory element IDs. |
actions.traffic_signal_controller |
Traffic signal controller action: put a junction into a named phase. |
actions.turn |
Turn action: have a vehicle take a left, right or straight path through the next junction. |
actions.turn_indicator |
Turn indicator action: light a vehicle's turn indicators. |
actions.walk_straight |
Walk-straight action: send a pedestrian forward with no destination. |
adversity |
TeraSim's adversarial behaviour generation, ported (#385). |
adversity.avoidance |
Who an adversity challenges, whether they can avoid it, and how they respond (#396). |
adversity.checks |
The checks an adversity's settings make of their numbers (#2318). |
adversity.collision |
Conflicts between two predicted trajectories (TeraSim's collision_check_cy). |
adversity.command |
Adversarial commands and the paths they drive (TeraSim's NDECommand). |
adversity.construction |
Where a construction zone's cones, barriers and signs go (#402). |
adversity.geometry |
Footprints and predicted states (TeraSim's geometry_utils_cy). |
adversity.idm |
The car-following model the triggers compare accelerations with. |
adversity.location |
Where an agent is, for an adversity's location gate (TeraSim's get_location). |
adversity.nade |
TeraSim's NADE: which adversity to sample, with an unbiased weight (#395). |
adversity.observation |
What an agent sees, for the adversity triggers (TeraSim's NDEEgoSensor). |
adversity.prediction |
Kinematic prediction (TeraSim's trajectory_predictor and trajectory utils). |
adversity.snapshot |
Read the agents and the signals of a CARLA world for a observation.Scene. |
adversity.triggers |
When each phase-1 adversity fires, and what it does then (#392). |
adversity_activity |
Which adversities are under way now, for the conditions that ask. |
adversity_event |
What an action reports for the run's log, beside its lifecycle (#397). |
atomic_file |
Writing a file that others read while it is rewritten: in one rename. |
authoring.framework_pin |
Work out how an exported scenario package should depend on this framework. |
authoring.package_names |
The names an exported Scenario Package takes from its document's id. |
authoring.requirements_pin |
Pin a wheelhouse's direct-reference requirements to the wheels built for them. |
authoring.uv_tool |
Running uv on behalf of an export. |
autoware_bridge |
minimal readiness contract with autoware_carla_interface. |
autoware_bridge._proto |
Generated protobuf modules for the AutowareBridge contract. |
autoware_bridge.base |
Minimal contract for delegating Autoware startup to autoware_carla_interface. |
autoware_bridge.fake |
In-memory fake AutowareBridge for unit tests. |
autoware_bridge.ros_bridge |
The scenario_bridge ROS 2 node: Autoware's end of the AutowareBridge. |
autoware_bridge.ros_bridge.ad_api |
Autoware AD API names and the readiness aggregation logic. |
autoware_stack |
start a fresh Autoware for every scenario, and remove it after. |
autoware_stack.bridge_node |
How a launcher runs the scenario_bridge node next to the stack it starts. |
autoware_stack.devcontainer_text |
The text a dev container's configuration is written in. |
autoware_stack.docker |
Run a local Autoware workspace in its dev container, one container per scenario. |
autoware_stack.launcher |
Bring an Autoware stack up for one scenario and tear it down after it. |
batch_report |
A batch's summary: the table it prints, and batch_results.json. |
batch_resume |
Resuming a local batch into its own directory: --resume-into DIR (#1957). |
canvas |
Sensors rendered by carla_canvas instead of the CARLA server (canvas.config). |
carla_install_paths |
Where scenario-setup installs CARLA, and the paths it swaps an install by. |
commonroad |
The CommonRoad metrics: CriMe's criticality measures and CommonRoad-Reach's drivable area (autoware_carla_scenario.metrics.COMMONROAD_METRICS), computed once a run is over. |
commonroad.frame |
A frame of the world as the CommonRoad worker reads it. |
commonroad.map |
The world's map as a CommonRoad scenario, converted by roadgen and cached. |
commonroad.worker_start |
The command that starts the CommonRoad worker, and the worker's error. |
conditions |
Scenario pass/fail conditions for CARLA scenario testing. |
conditions.action_state |
Condition on the lifecycle state of a named action. |
conditions.adversity_active |
Condition that holds while a road user is carrying out an adversity. |
conditions.adversity_feasible |
Whether an entity may carry out an adversity now (#392). |
conditions.always_true |
Condition that is always satisfied. |
conditions.and_condition |
AND composite condition that requires all child conditions to pass. |
conditions.base |
Base classes and helpers for scenario pass/fail conditions. |
conditions.collision |
Collision-based scenario fail condition using CARLA sensor.other.collision. |
conditions.comparison |
Shared comparison rule for condition evaluation. |
conditions.composition |
Composition conditions built from multiple base conditions. |
conditions.composition.acceleration |
Acceleration condition for scenario evaluation. |
conditions.composition.base |
Base class for composition conditions. |
conditions.composition.distance_measure |
How far apart two actors are, and what "apart" is being asked about. |
conditions.composition.entity_area_distance |
Distance from an entity's box to an area of the map, and the closest it came. |
conditions.composition.entity_distance |
Relative-distance condition between two entities. |
conditions.composition.entity_lane_position |
Entity-lane-position scenario pass condition. |
conditions.composition.entity_position_distance |
Distance from an entity to a fixed place on the map. |
conditions.composition.relative_speed |
Relative speed condition for scenario evaluation. |
conditions.composition.speed |
Speed condition for scenario evaluation. |
conditions.composition.standstill |
Standstill condition composed from PersistentCondition + SpeedCondition. |
conditions.composition.temporary_stop |
Temporary stop condition for detecting standstill at specified positions. |
conditions.composition.time_headway |
Time-headway condition between two entities. |
conditions.composition.time_to_collision |
Time-to-collision (TTC) condition between two entities. |
conditions.composition.turn_indicator |
Turn indicator condition: which of a vehicle's turn indicators are lit. |
conditions.composition.waypoint |
Waypoint condition for scenario evaluation. |
conditions.elapsed_time |
Elapsed-time-based scenario pass condition. |
conditions.entity_count |
Condition on how many road users an EntityFilter takes. |
conditions.entity_existence |
Fail condition that triggers when a named actor does not exist. |
conditions.entity_filter |
Which road users a condition looks at: an EntityFilter. |
conditions.lane_change_settled |
Condition satisfied once a commanded lane change has settled. |
conditions.map_area |
Areas of the map chosen by what the ground is: crosswalks, walkways, the lanes through a junction, Autoware's intersection areas, the raised and grooved parts of a road's surface. |
conditions.metric |
A condition on a metric: how the ego drove, judged once the run is over. |
conditions.not_condition |
NOT condition that inverts the satisfied result of a child condition. |
conditions.off_road_fraction |
Fail condition on the fraction of an entity's box that is off the driving lanes. |
conditions.or_condition |
OR composite condition that requires any child condition to pass. |
conditions.persistent |
Persistent condition that requires a child to remain True for a duration. |
conditions.place_clear |
Pass condition that holds while a place on the map is clear of road users. |
conditions.route_progress |
How far an entity has come along the scenario's route. |
conditions.sticky |
Sticky condition that latches after the first passing result. |
conditions.timeout |
Timeout-based scenario fail condition. |
conditions.traffic_signal |
Traffic signal condition: verify traffic light state by Lanelet2 regulatory element ID. |
conditions.traffic_signal_controller |
Traffic signal controller condition: is a junction showing a named phase? |
conditions.trajectory_time |
The time on a trajectory's clock: the condition a vertex time stands for. |
config_flags |
A config's true/false switch, read strictly: a boolean or its spelling. |
constants |
Package-wide constants for autoware_carla_scenario. |
control_domain |
Which actions control an entity: its control domains and who holds each. |
coordinate |
mutual conversion between Lanelet2, OpenDRIVE, and CARLA world poses. |
coordinate.frames |
Coordinate frame definitions shared across coordinate and kinematics modules. |
coordinate.lane_distance |
Distance measured along the road, rather than across the map. |
coordinate.map_manager |
Singleton MapManager for holding map data used by coordinate transforms. |
coordinate.poses |
Pose dataclasses for coordinate transformation between map systems. |
coordinate.projection |
Which projection a Lanelet2 map wants, and the projector that reads it. |
coordinate.snap |
Snap a pose onto the actual CARLA road surface. |
coordinate.stacked |
Which of the lanelets under a pose it is on: one rule for every caller. |
coordinate.stop_line |
Stop line pose computation from Lanelet2 map data. |
coordinate.traffic_light |
Traffic lights, placed and looked up in the ambient run. |
coordinate.transform |
6-direction mutual conversion between Lanelet2, OpenDRIVE, and CARLA world coordinates. |
coordinate.walk_graph |
Where a pedestrian can walk on the Lanelet2 map, and the shortest way there. |
coverage |
ODD and scenario coverage, after cover() in ASAM OpenSCENARIO DSL. |
coverage.item_names |
The names a versioned measure's cover items are written under, and what a merged item's name splits into. |
debug_markers |
Debug markers: what an ego entity knows that the world does not show. |
driver |
connect the ego vehicle to an external driving policy. |
driver.base |
Configuration and the abstract client for an external ego driver. |
driver.control |
Turn a planned trajectory into CARLA vehicle controls. |
driver.geometry |
Rigid-body primitives for the egodriver contract. |
driver.hdmap_files |
The names of a map set's files, and the id a map set is written under. |
driver.observation |
Build driver observations from CARLA state. |
driver.stop_lines |
Which traffic light governs the ego, and where it stops for it. |
editor.form_values |
The whole numbers the editor's forms post: a list field's ids and a record list's row count. |
editor.gate_fields |
The gate's thresholds as the editor's inspector edits them. |
editor.naming |
The names the editor gives the rows it adds. |
ego_base_link |
Where the running scenario's ego has its base_link, as the run uses it. |
ego_plan |
A driver's plan as the run records it: the trajectory, timed. |
entity |
vehicle entity types for CARLA scenarios. |
entity._policy_warmup |
A run-up that fills a driver policy's history before the scenario starts. |
entity._spawn |
Shared spawn helpers for vehicle entities. |
entity._walker_ai |
Walk a routed pedestrian with CARLA's own walker controller (#521). |
entity.autoware_entity |
The Autoware ego vehicle. |
entity.bridged_ego |
Egos whose driving stack reaches the scenario over the AutowareBridge. |
entity.ego |
Ego vehicle spawning. |
entity.externally_driven |
The base of every ego that something other than the TrafficManager drives. |
entity.fit |
The vehicle blueprint nearest a size, chosen on the server the run is on. |
entity.pedestrian_entity |
Pedestrian entity for CARLA scenarios. |
entity.registry |
The live entities of a run, addressable by role name. |
entity.ros2_entity |
Ego vehicle driven by ROS 2 nodes, over DDS, with the framework in the loop. |
entity.vehicle_entity |
NPC vehicle entity for CARLA scenarios. |
entity_role |
Validated entity role names for CARLA actors. |
environment |
What a run ran with: this framework, the scenario's package, every dependency. |
environment_pin |
The requirement a run's lock pins a distribution to, from where it was installed from. |
examples.adversity |
Example scenarios: TeraSim's adversities carried out in CARLA (#404). |
examples.argv |
What the scenario command reads off its command line and environment. |
examples.conf |
- |
examples.configs |
Hydra-compatible dataclass definitions for scenario parameters. |
examples.cut_in |
Example scenario: a vehicle in the next lane cuts in just ahead of the ego. |
examples.intersection_passing |
Example scenario: Ego vehicle passes through an intersection on expected roads. |
examples.lane_change |
Example scenario: Ego vehicle performs a lane change on a straight road. |
examples.pedestrian_dart_out |
Example scenario: a pedestrian darts out into the road in front of the ego. |
examples.temporary_stop |
Example scenario: the ego stops at a stop line, then drives on. |
examples.traffic_light_compliance |
Example scenario: Ego vehicle stops at a red traffic light and proceeds when green. |
failure |
Why a run failed: the category of the fail condition that ended it. |
gate |
A logical scenario's gate: when it passes over all its runs. |
gate.criteria |
The typed core of a gate's criteria: what scenario-gate measures. |
gate.errored |
The marker a --multirun job leaves when it is given up: case_errored.json. |
gate.probe |
Sample the ego in the route's frame on every tick, for the gate. |
gate.record |
What one run of a logical scenario leaves for its gate: {stem}_gate.json. |
gate.report |
The typed core of scenario-gate's report: its Markdown, and what changed. |
gate.sweep_plan |
The jobs a --multirun sweep is going to run: sweep_plan.json. |
gate.unrecorded |
What a set of runs left unrecorded, file by file: the scans behind scenario-gate's refusal (unrecorded_cases). |
heavy_metrics |
Which CommonRoad metrics are heavy, and whether a run evaluates them. |
hils |
HILS: stream a scenario run's ego to devices under test (the hils extra). |
hils.host_devices |
The virtual devices a HILS configuration's devices need: what scenario-hils --setup (host_setup) creates. |
hils.lidar_revolution |
A LiDAR's per-tick arcs, gathered into the whole revolutions a sensor sends. |
hils.lidar_spin |
A LiDAR revolution's points dated and re-observed as a spinning sensor's. |
hils.sensor_spec |
The plain-typed half of config: a device's sensor, and the names a configuration chooses from. |
hils.signals |
The ego's state as named signals, the values a device's message fields are filled from. |
hils.wire |
The socket-free half of protocol: its constants and message lines. |
hils.yaml11 |
What YAML 1.1 makes of a plain scalar that does not mean what it looks like. |
kinematics |
velocity and acceleration types with coordinate-frame safety. |
kinematics.acceleration |
Acceleration types with coordinate-frame safety and affine-space semantics. |
kinematics.angle |
Angle wrapping shared by anything that compares two headings. |
kinematics.frames |
Coordinate frame definitions for kinematic quantities. |
kinematics.vector |
Immutable 3-D vector with full arithmetic support. |
kinematics.velocity |
Velocity types with coordinate-frame safety and affine-space semantics. |
kinematics.walker_motion |
A walker's velocity, measured from where it stands. |
lanelet_junction |
Whether a Lanelet2 lanelet lies in a junction: one judgment for every reader. |
maps |
HD maps that live in a git repository rather than beside the scenario. |
maps.cache |
A local cache of HD map repositories, filled by git. |
maps.catalogue |
What maps a repository offers. |
maps.config |
Turn a map config into the paths the rest of the framework reads. |
maps.opencrg |
The raised and grooved parts of a road's surface, read from the ASAM OpenCRG grids an OpenDRIVE road refers to. |
maps.opendrive |
Where the OpenDRIVE for a remote Lanelet2 map comes from: CARLA. |
maps.resolver |
From a map source to the files on disk a scenario actually reads. |
maps.source |
Where an HD map comes from, written as one string. |
maps.z_offset_cache |
Remember a map's Lanelet2-to-CARLA vertical offset across runs. |
measures |
Scenario measures: what a run sets up, read off the world under fixed keys. |
metric_frames |
The frames a run records for its metrics, one a tick, and the series those metrics are read into once the run is over. |
metrics |
Metrics: how the ego drove, read off the world per frame. |
npc_collisions |
Log the collisions of the road users other than the ego, and walkers that fall. |
odd.control_diagnostics |
A problem with the controls of a scenario config, and its one-line text. |
odd.control_fields |
The fields of controls and sweep.odd_sample, and the keys a sweep sets. |
odd.probes |
Probes: functions that read one ODD attribute from the world. |
odd.route_plan |
Plan a scenario's route on the Lanelet2 map before it runs. |
odd.route_text |
What scenario-odd route reads off its command line, and the metres it prints. |
odd.scenario_measure |
Map an ODD attribute onto a scenario measure. |
odd.traffic_ranges |
The traffic parameters' probes by name: their knob keys and their ranges. |
odd.unit_names |
How units are spelled: the canonical spelling of an alias (kph -> km/h). |
pedestrian |
Posing a pedestrian's skeleton: the core the pedestrian-motion actions share. |
pedestrian._proto |
Generated protobuf modules for the PedestrianMotion contract. |
pedestrian.contract |
The pedestrian motion contract's frame and values, without grpc. |
pedestrian.poser |
A pedestrian taken over by a motion: found, posed, placed, handed back. |
pedestrian.skeleton |
A walker's skeleton, posed by the directions of a motion's bones. |
pedestrian.soma |
A SOMA motion (NVlabs SOMA-X) replayed on a CARLA walker's skeleton. |
perception_fault |
What a driver policy is told of a road user, where a scenario overrides it. |
placement |
Place a road user so a scenario measure reads a given value at the start. |
prepare |
Backends a run needs started before it: declared, found, started, stopped. |
prepare.docker |
A backend run as a docker container on this host. |
prepare.host |
What the host has free for backends: GPU memory and host memory. |
rerun_report |
How converting a run's trajectory logs into Rerun recordings went. |
rerun_sensor_mount |
The typed core of the sensors in a run's Rerun recording: a camera's mount on the ego, the colours of a LiDAR sweep, and which entities of a recording are the sensors'. |
ros2 |
drive the ego from ROS 2 nodes, over DDS, with no bridge in between. |
ros2.commands |
What crosses between the ego entity and its DDS transport, and who drives. |
ros2.config |
Settings for the ego entity that talks to ROS 2 nodes over DDS. |
ros2.transport |
The DDS transport the ROS 2 ego entity talks through. |
route |
Logical scenarios: a drive described as a pattern of road, found on any map. |
route.active |
The route the running scenario is about. |
route.frame |
A route match laid on its map: route s to lanelet poses and back. |
route.geometry |
What a route search asks of a Lanelet2 map: lanes, junctions, crosswalks. |
route.model |
A route search and its matches, as data: no map, no lanelet2, no CARLA. |
route.positions |
Place a route pose (RouteLanePose and its kin) on the map. |
route.search |
Find the routes of a map that match a route search. |
run_dirs |
What a run's Hydra command line asks for, and the run directory it gets. |
run_outputs |
What one run leaves under its file stem: the single list of them. |
run_timeout |
scenario.timeout_seconds, the run's own fail-safe, read and checked. |
scaffold |
Scaffolding for standalone scenario packages. |
scaffold.names |
The naming rules of scenario-new: package, scenario and class names. |
scaffold.placement |
How scenario-new puts a rendered package in place of its target directory. |
scenario_base |
Base class for CARLA scenarios. |
scenario_config |
Public, reusable configuration dataclasses for scenario packages. |
scenario_glob |
Expand a scenario pattern to the config names it matches. |
scenario_hash |
The digest of a concrete scenario: what a run is, its seed aside (docs/wosac.md). |
scenario_queue |
Scenario queue bound to a CarlaServerManager. |
scenario_runner |
Top-level scenario runner integrating all sub-components. |
seed |
The run's seed: the one number every random draw of a run follows. |
sensor |
Sensor abstractions for CARLA scenario testing. |
sensor.base |
Abstract base class and configuration dataclass for camera sensors. |
sensor.carla_camera |
Concrete CARLA RGB camera sensor implementation. |
sensor.carla_lidar |
CARLA ray-cast LiDAR: one full revolution per sweep, a sweep every so many ticks. |
sensor.lidar_weather |
What rain and fog do to a LiDAR sweep: CARLA's ray-cast LiDAR ignores the weather. |
serve_bind |
Where the web applications (viewer, scenario-editor) listen: the default host, the TCP port range, which hosts are loopback ones, and the warning when they listen beyond this machine, which the HILS bridge's run link (hils.link) gives too. |
server |
CARLA UE5 server lifecycle management. |
signals |
Traffic signal controllers: a junction's cycle, declared and run. |
signals.controller |
A traffic signal controller: the thing that cycles a junction's phases. |
signals.registry |
Where a running scenario's signal controllers can be found by name. |
slurm |
Running a batch's cases as SLURM array jobs: launcher=slurm (docs/slurm.md). |
slurm.collect |
Collecting a SLURM batch: its summary and batch_results.json. |
slurm.follow |
Following a SLURM submission until it ends, and what a submitter says of it once it is submitted (docs/slurm.md). |
slurm.manifest |
What a SLURM batch leaves in its output directory, and reads back. |
slurm.port_slots |
The arithmetic of a SLURM task's port slots (docs/slurm.md). |
slurm.ports |
The ports a SLURM task's CARLA and servers listen on, one slot per task. |
slurm.queue |
SLURM's command-line tools, as the submitting side and the viewer run them: sbatch, squeue, scancel, scontrol and sinfo (docs/slurm.md). |
slurm.records |
A SLURM batch's JSON record as the text of each of its members. |
slurm.same_files |
Whether a string of a case's config that differs on a node names the same thing. |
slurm.scripts |
The text a SLURM batch hands to SLURM, and reads back from it (docs/slurm.md). |
slurm.settings |
The launcher config: who runs a batch's cases (docs/slurm.md). |
slurm.start_lock |
The files of the SLURM worker's start lock where a file system has no flock. |
slurm.status |
What scenario-slurm status prints of a SLURM batch, what cancel cancels of it, whether it is busy, so that resubmit refuses it, and which of its cases resubmit --cases names. |
slurm.stems |
The names a SLURM batch gives its cases' files and the keys of its configs. |
slurm.task_timing |
The arithmetic of a SLURM task's start and its memory report (docs/slurm.md). |
sweeper.bindings |
Binding resolution for auto-deriving parameter values per lanelet. |
sweeper.case_choice |
Which base case of a sweep each case drawn from the ODD is put on. |
sweeper.constraints |
Constraint engine for filtering lanelets by declarative conditions. |
sweeper.isolation |
How the lanelet sweeper runs each case: in a forked child of its own. |
sweeper.job_status |
The numbers of a lanelet sweep's case: its timeout and its exit status. |
sweeper.map_loader |
Lightweight Lanelet2 map loader for pre-simulation use. |
t4_scene |
Trajectory logs as T4Scenes, the exchange format of tier4/e2e-devkit. |
t4_scene._proto |
Generated protobuf modules for the T4Scene exchange contract (tier4/e2e-devkit). |
t4_scene.exporter |
Each run's trajectory log as a T4Scene, as the run ends (t4scene.enabled). |
t4_scene.frames |
The frame math of a T4Scene: CARLA's left-handed world frame to Autoware's right-handed map frame (docs/trajectory_log.md, coordinate.transform.to_map_frame), and the rotations and quaternions of map-frame roll, pitch and yaw. |
t4_scene.grid |
The pure pieces of a T4Scene's time grid: the step in ticks, the scene's stem, who the ego is, who is on the scene, the ego's turn indicator and how far from it a road user is still one of the scene's objects. |
t4_scene.osm |
A Lanelet2 map as the OSM XML of the T4Scene contract's LOCAL_METERS profile, and the SHA-256 that names it (docs/t4_scene.md). |
t4_settings |
What a run's T4 dataset holds: its settings, names and categories, and the OpenCV model of an F-theta camera. |
text_escape |
Escape free-form text for the TOML and Python files the generators write. |
tools |
- |
tools.file_writes |
How detect-no-3d-model writes the map configuration it edits. |
traffic |
what drives the vehicles a scenario did not author. |
traffic.authoring |
Traffic parameters out of a scenario document's flat fields. |
traffic.base |
What drives the traffic of a run, and the vocabulary it answers. |
traffic.config |
Configuration for traffic backends. |
traffic.context |
The run a traffic backend joins: TrafficContext. |
traffic.driven |
The vehicle side of the traffic seam. |
traffic.fidelity |
Which vehicles a traffic backend simulates mesoscopically, and when. |
traffic.final_approach |
Drive the last metres of a given path where the TrafficManager would not. |
traffic.mesoscopic |
A mesoscopic traffic model a backend can run its far traffic on. |
traffic.parameters |
How a run's traffic behaves, in words every traffic backend reads. |
traffic.route_match |
Match a recorded track to the lanes it was driven on (#521). |
traffic.sumo |
SUMO as a traffic backend (traffic.backend=sumo). |
traffic.sumo.config |
Options of the SUMO traffic backend. |
traffic.sumo.geometry |
Poses between SUMO's network frame and CARLA's world frame. |
traffic.sumo.network |
The SUMO network of the road network a CARLA world runs. |
traffic.sumo.options |
The SUMO backend's command-line options, mode bits, vehicle types and type files. |
traffic.sumo.physics_control |
Driving a CARLA vehicle along its SUMO vehicle's trajectory, with physics. |
traffic.sumo.signal_states |
The signal states the SUMO backend sets, from CARLA's lights and SUMO's programs. |
traffic.traffic_manager |
CARLA's TrafficManager as a traffic backend. |
trajectory |
Trajectories for entities to follow: the waypoints of a RoutingAction. |
trajectory.authored |
The typed builders of a *Set Goal* card: goal, via lanelets, timing, path. |
trajectory.model |
The trajectory an entity is told to follow, after OpenSCENARIO's. |
trajectory.relative_lane |
Resolve a RelativeLanePose to an absolute lanelet pose. |
trajectory.resolve |
Place a Trajectory in CARLA world coordinates. |
trajectory.vertex_text |
The editor's text of a trajectory's vertices, and what is read off typed rows. |
ui.case_paths |
Where a session's cases are on disk: the viewer's path guard and case stems. |
ui.comparison |
Sessions compared case by case and ODD cell by cell: each case's change from the first session that ran it to the last, its metric deltas, and the pass rate of each cell of two ODD attributes, per session. |
ui.drive_metrics |
How the ego drove: the gaps between road users' boxes, and the metrics the viewer reads off a trajectory log, fed one tick at a time. |
ui.recording_files |
How the viewer keeps its recordings and its copy of Rerun's web viewer on disk. |
ui.request_values |
What the viewer reads off a request and links a session with. |
ui.runner |
Scenario execution bridge using subprocess + progress tracking. |
ui.slurm_case_state |
The state of one case of a SLURM batch, and the message the viewer shows for it. |
ui.slurm_launch |
Running a batch from the viewer through SLURM (launcher=slurm, docs/slurm.md), and following it. |
ui.slurm_options |
The launcher.* keys the viewer sets on a SLURM batch (docs/slurm.md): what each may hold, the Hydra overrides they become, and the scenario command line that submits the batch with them. |
ui.web_viewer_paths |
Where the viewer keeps recordings and Rerun's web viewer, and which viewer it picks. |
utils |
Pure helpers, imported by module path. |
utils.cache_dir |
The user's cache directory, by the XDG Base Directory rules. |
utils.card_target |
The module an editor card's "..module:Name" target names. |
utils.clearance |
What stands in a road user's way: walkers ahead of a vehicle, vehicles ahead of a walker (#521). |
utils.light_lanes |
Which traffic light regulatory elements a CARLA light shows, on the Lanelet2 map. |
utils.opendrive |
CARLA's OpenDRIVE, made acceptable to roadgen. |
utils.owner_label |
The <host>:<pid> owner label on the containers a run starts. |
utils.polygon |
Shapes on the ground: a road user's box and the areas of a map, and how far apart two of them are. |
utils.powertrain |
The longitudinal response of a CARLA 0.10 vehicle, worked out from its physics control. |
utils.stop_line |
Stop line linestring extraction from Lanelet2 regulatory elements. |
utils.traffic_light |
Traffic light lookups in an OpenDRIVE road network. |
utils.turn_indicator |
A vehicle's turn indicators, as the blinker bits of its CARLA light state. |
utils.vehicles |
Vehicle-wide helpers that read nothing but the world they are handed. |
value_text |
A config value as an error message shows it, a huge integer included. |
visibility |
How far a driver sees: the visibility that night, rain and fog leave. |
waypoint_arrival |
When an entity reached a waypoint, against when it was meant to (#521). |
weather_inputs |
What the weather rules read and which vehicle entry a vehicle follows. |
world_reset |
Reset the CARLA world between scenarios. |
wosac._proto |
Generated protobuf modules for the Waymo Open Sim Agents Challenge submission contract. |
wosac.ids |
The ids of a WOSAC rollout and the counts its writers take (docs/wosac.md). |
202 functions and methods of those modules are UNCALLED
(64 of them protected helpers, see What is
compiled below). Most of them build or read values of mixed or unknown
types: a condition's get_details() or a backend's describe() (a
dict[str, Any]), what reads a Hydra config node or a parsed YAML or JSON
document (Mapping[str, Any], Any), what writes a manifest or a result
document of mixed values, and protected helpers of the same kinds. The rest
need what Codon has nothing for (ctypes, cv2, a signal handler's frame), hold
a class the check has no value of, or take a parameter
Codon 0.19 cannot call them with (it fails on a Union in a parameter
annotation, union already sealed, so the checker drops one). Some are
compiled through their callers all the same. Each is listed with its reason:
The 202 UNCALLED functions and methods, and why
actions.background_traffic.LaneletRegion.__init__: takes lanelet constraints asConstraints = Union[Sequence[ConstraintSpec], Mapping[str, Any]]: a sequence of Hydra nodes (Mapping[str, Any]) or parsed constraints (ConstraintSpec = Union[Mapping[str, Any], Any]), or a single mapping, which Codon cannot express; LaneletRegion.__init__ parses them (parse_constraints, itself uncalled), and LaneletRegion.lanelet_ids reads the parsed list in theelsebranch ofif TYPE_CHECKING:actions.background_traffic.TrafficSinkAction.__init__: takes lanelet constraints asConstraints = Union[Sequence[ConstraintSpec], Mapping[str, Any]]: a sequence of Hydra nodes (Mapping[str, Any]) or parsed constraints (ConstraintSpec = Union[Mapping[str, Any], Any]), or a single mapping, which Codon cannot express; LaneletRegion.__init__ parses them (parse_constraints, itself uncalled), and LaneletRegion.lanelet_ids reads the parsed list in theelsebranch ofif TYPE_CHECKING:actions.background_traffic.TrafficSourceAction.__init__: takes lanelet constraints asConstraints = Union[Sequence[ConstraintSpec], Mapping[str, Any]]: a sequence of Hydra nodes (Mapping[str, Any]) or parsed constraints (ConstraintSpec = Union[Mapping[str, Any], Any]), or a single mapping, which Codon cannot express; LaneletRegion.__init__ parses them (parse_constraints, itself uncalled), and LaneletRegion.lanelet_ids reads the parsed list in theelsebranch ofif TYPE_CHECKING:actions.background_traffic.constraints_from_text: reads YAML (yaml, imported inside it) into a list ofdict[str, Any]constraint mappings of mixed value types, which Codon cannot expressactions.background_traffic.parse_constraints: takes lanelet constraints of any type (Any: a list or a Hydra ListConfig of mappings or parsed constraints, a single mapping, or a value it refuses) and returns them parsed aslist[Any], which Codon cannot express; only the uncalled LaneletRegion.__init__ and constraints_from_text, and the excluded examples.run (the batch check of background_traffic constraints), call itauthoring.framework_pin.Pin.uv_source: builds the[tool.uv.sources]entry, a dict[str, Any] of str and bool values ({"path": ..., "editable": True}), which Codon cannot expressauthoring.framework_pin._project_name: imports tomllib or tomli inside it, which Codon has nothing for, and reads a TOML document of mixed value types; _is_framework_project() calls it in theelsebranch ofif TYPE_CHECKING:autoware_stack.bridge_node._seconds_parameter: takes the scenario_bridge node's ROS parameters asMapping[str, Any](bool, int, float or str values), which Codon cannot express: Codon 0.19 fails on a Union in a parameter annotation (union already sealed), so the checker drops it and the check has nothing to call withautoware_stack.bridge_node.bridge_command: takes the scenario_bridge node's ROS parameters asMapping[str, Any](bool, int, float or str values), which Codon cannot express: Codon 0.19 fails on a Union in a parameter annotation (union already sealed), so the checker drops it and the check has nothing to call with; CommandAutowareLauncher.start() and DockerAutowareLauncher.start() call it in theelsebranch ofif TYPE_CHECKING:autoware_stack.bridge_node.bridge_ros_args: takes the scenario_bridge node's ROS parameters asMapping[str, Any](bool, int, float or str values), which Codon cannot express: Codon 0.19 fails on a Union in a parameter annotation (union already sealed), so the checker drops it and the check has nothing to call withautoware_stack.bridge_node.launcher_bridge_parameters: takes the scenario_bridge node's ROS parameters asMapping[str, Any](bool, int, float or str values), which Codon cannot express: Codon 0.19 fails on a Union in a parameter annotation (union already sealed), so the checker drops it and the check has nothing to call with, and reads them off any launcher (object) with getattr(); BridgeHandshake.__init__() calls it in theelsebranch ofif TYPE_CHECKING:autoware_stack.bridge_node.liveness_timeout_problem: takes the scenario_bridge node's ROS parameters asMapping[str, Any](bool, int, float or str values), which Codon cannot express: Codon 0.19 fails on a Union in a parameter annotation (union already sealed), so the checker drops it and the check has nothing to call with; BridgeHandshake.__init__() calls it in theelsebranch ofif TYPE_CHECKING:; the comparison itself is the checked min_liveness_timeout_s()'scommonroad.frame._state: builds a road user's state as a JSON object of mixed value types (dict[str, Any]) for the worker, which Codon cannot express; the road user it is made of, its id, kind, geometry and motion, is the checked _road_user()'s (#2266)commonroad.frame.read_frame: builds the ego's and the other road users' states with _state(), JSON objects of mixed value types (dict[str, Any]), which Codon cannot express; the ego and the road users near it, typed, are the checked _read_road_users()'s (#2266)conditions.adversity_active.AdversityActiveCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.and_condition.AndCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.base.BaseCondition.__init_subclass__: wraps check() when a subclass is created (cls.__dict__, functools.wraps), which Codon has no hook forconditions.base.BaseCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.base.BaseCondition.to_summary_dict: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.base.ConditionStatus.to_dict: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.base.ScenarioResult.to_dict: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.base.ScenarioResult.to_json: serialises to_dict(), adict[str, Any]of mixed value types (see_DETAILS), and forwards**kwargs: Any, which Codon cannot expressconditions.collision.CollisionCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.comparison.ScalarComparisonRule.to_dict: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.composition.acceleration.AccelerationCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.composition.base.CompositionCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.composition.entity_area_distance.EntityAreaDistanceCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.composition.entity_distance.EntityDistanceCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.composition.entity_lane_position.EntityLaneOfCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.composition.entity_lane_position.EntityLanePositionCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.composition.entity_position_distance.EntityPositionDistanceCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.composition.relative_speed.RelativeSpeedCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.composition.speed.SpeedCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.composition.standstill.StandstillCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.composition.time_headway.TimeHeadwayCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.composition.time_to_collision.TimeToCollisionCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.composition.turn_indicator.TurnIndicatorCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.composition.waypoint.WaypointCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.elapsed_time.ElapsedTimeCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.entity_count.EntityCountCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.metric.MetricCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.not_condition.NotCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.off_road_fraction.OffRoadFractionCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.or_condition.OrCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.persistent.PersistentCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.place_clear.PlaceClearCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.route_progress.RouteProgressCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.sticky.StickyCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expressconditions.traffic_signal.TrafficSignalCondition.get_details: builds a dict[str, Any] of mixed value types, which Codon cannot expresscoordinate.projection._projector_info: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot express (map_projector_info.yaml, with yaml, imported inside it, which Codon does not have); only read_map_origin and read_projector_type call itcoordinate.projection.read_map_origin: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot express (map_projector_info.yaml); map_origin calls it in theelsebranch ofif TYPE_CHECKING:coordinate.projection.read_projector_type: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot express (map_projector_info.yaml); resolve_projector calls it in theelsebranch ofif TYPE_CHECKING:driver.base.DriverCameraConfig.from_mapping: takes a Hydra node (Mapping[str, Any]) of mixed value types, which Codon cannot express, checks its keys with utils.config, which reads a dataclass's fields from its class (type), and builds the config with**of a dict; utils.config is imported inside it, so it is not compileddriver.base.DriverCameraLens.from_mapping: takes a Hydra node (Mapping[str, Any]) of mixed value types, which Codon cannot express, checks its keys with utils.config, which reads a dataclass's fields from its class (type), and builds the config with**of a dict; utils.config is imported inside it, so it is not compileddriver.base.DriverClientConfig.from_mapping: takes a Hydra node (Mapping[str, Any]) of mixed value types, which Codon cannot express, checks its keys with utils.config, which reads a dataclass's fields from its class (type), and builds the config with**of a dict; utils.config is imported inside it, so it is not compileddriver.base.DriverLidarConfig.from_mapping: takes a Hydra node (Mapping[str, Any]) of mixed value types, which Codon cannot express, checks its keys with utils.config, which reads a dataclass's fields from its class (type), and builds the config with**of a dict; utils.config is imported inside it, so it is not compileddriver.control.ControlConfig.from_mapping: takes a Hydra node (Mapping[str, Any]) of mixed value types, which Codon cannot express, reads the dataclass's fields from its class (dataclasses.fields(cls)) and builds the config with**of a dictdriver.control._config_number: takes a config value of any type (Any: a Hydra node's, a bool, a string or an integer too large for a float among them), which Codon cannot express; only ControlConfig.from_mapping, itself uncalled, calls itdriver.observation.encode_frame_jpeg: encodes with cv2 (OpenCV), which Codon cannot compile; cv2 is imported inside it, so it is not compiledentity.registry.find_entity_by_role_name: returns the entity of any kind registered under a role (Optional[Any]), which Codon cannot express; find_vehicle_entity and find_pedestrian_entity are its typed lookupsentity.registry.register_entity: takes an entity of any kind (Any: a vehicle, a pedestrian or a duck-typed object); Codon resolves its isinstance() dispatch by the static type rather than the class, so a subclass of BackendDriven takes the wrong branch; register_vehicle_entity and register_pedestrian_entity are its typed counterparts, which scenario_base and scenario_runner callenvironment.RunEnvironment.__init__: takes the installed distributions as RunEnvironment holds and the environment file writes them,dict[str, Any]records of mixed value types (a name, a version or None, and a source whose values direct_url.json gives of any JSON type), which Codon cannot express; what Python and the platform are is the checked _interpreter()'senvironment.RunEnvironment.id: reads the installed distributions as RunEnvironment holds and the environment file writes them,dict[str, Any]records of mixed value types (a name, a version or None, and a source whose values direct_url.json gives of any JSON type), which Codon cannot express through pins; the hash is the checked _environment_id()'s; the checked scenario_runner reads it through environment.codon (MODELLED_IMPORTS)environment.RunEnvironment.lock: takes a scenario of any class (Any) and reads the installed distributions as RunEnvironment holds and the environment file writes them,dict[str, Any]records of mixed value types (a name, a version or None, and a source whose values direct_url.json gives of any JSON type), which Codon cannot express with _package_pin(); the lock's text is the checked _lock_text()'senvironment.RunEnvironment.pins: reads the installed distributions as RunEnvironment holds and the environment file writes them,dict[str, Any]records of mixed value types (a name, a version or None, and a source whose values direct_url.json gives of any JSON type), which Codon cannot express with _package_pin()environment.RunEnvironment.scenario_package: takes a scenario of any class (Any: its class's module names the package) and returns one of the installed distributions as RunEnvironment holds and the environment file writes them,dict[str, Any]records of mixed value types (a name, a version or None, and a source whose values direct_url.json gives of any JSON type), which Codon cannot express; which distributions provide the package is the checked _distribution_names()'s, and which editable or local checkout holds the module's file (#2226) the checked _deepest_directory()'senvironment.RunEnvironment.summary: takes a scenario of any class (Any) and builds the summary, adict[str, Any]of mixed value types, from the installed distributions as RunEnvironment holds and the environment file writes them,dict[str, Any]records of mixed value types (a name, a version or None, and a source whose values direct_url.json gives of any JSON type), which Codon cannot expressenvironment.RunEnvironment.to_dict: takes a scenario of any class (Any) and builds the environment file's document, adict[str, Any]of mixed value types, from the installed distributions as RunEnvironment holds and the environment file writes them,dict[str, Any]records of mixed value types (a name, a version or None, and a source whose values direct_url.json gives of any JSON type), which Codon cannot expressenvironment._module_file: reads the module a scenario's class comes from out ofsys.modules, and its__file__withgetattron a module object, neither of which the sys shim or Codon has; which checkout holds that file is the checked _deepest_directory()'senvironment._package_pin: reads a distribution as RunEnvironment holds it (dict[str, Any]), which Codon cannot express; the requirement it pins to is the checked environment_pin.pin()'senvironment._pin_source: reads a source as the environment file records it (dict[str, Any], of any JSON type), which Codon cannot express; the requirement it pins to is the checked environment_pin.pin()'senvironment._record: reads a source's members (_source_members()) back into the record the environment file writes, adict[str, Any]of values of any JSON type, which Codon cannot expressenvironment._source: builds a PinSource of the values a direct_url.json gives, as it gives them (of any JSON type:vcs,commit,requestedandsubdirectoryare no str in a malformed one), which PinSource's typed fields cannot hold in Codon; reading the file is the checked _direct_url()'s and _source_members()'s, which holds each value as its JSON textenvironment._source_record: builds the record the environment file writes of a PinSource, adict[str, Any]of mixed value types (str, bool, a dict of hashes), which Codon cannot express; capture() writes the checked _source_members()'senvironment.capture: builds the RunEnvironment of the installed distributions as RunEnvironment holds and the environment file writes them,dict[str, Any]records of mixed value types (a name, a version or None, and a source whose values direct_url.json gives of any JSON type), which Codon cannot express (with _record()); reading the distributions is the checked _installed()'s; the checked scenario_runner reads it through environment.codon (MODELLED_IMPORTS)environment.write: takes a scenario of any class (Any) and writes the environment file's document, adict[str, Any]of mixed value types, and the lock, both of the installed distributions as RunEnvironment holds and the environment file writes them,dict[str, Any]records of mixed value types (a name, a version or None, and a source whose values direct_url.json gives of any JSON type), which Codon cannot expressfailure.Failure.from_dict: takes the object to_dict() wrote asMapping[str, Any], of mixed value types (see_DETAILS), which Codon cannot express, so the check has nothing to call it with; batch_resume's KeptResult.failure, slurm.collect's RecordedResult.failure and slurm.manifest's readable_result call it with the object json.loads() read, so its body is compiled through themfailure.Failure.to_dict: builds a dict[str, Any] of mixed value types, which Codon cannot expressgate.criteria.CaseChange.to_dict: builds a dict[str, Any] of mixed value types, which Codon cannot expressgate.criteria.CriterionChange.to_dict: builds a dict[str, Any] of mixed value types, which Codon cannot expressgate.criteria.CriterionResult.to_dict: returns the criterion as the report stores it (dataclasses.asdict()), adict[str, Any]of mixed value types, which Codon cannot expressgate.record.GateCase.from_dict: reads the dict to_dict() wrote, aMapping[str, Any]of mixed value types (see_DETAILS), which Codon cannot expressgate.record.GateCase.to_dict: builds a dict[str, Any] of mixed value types, which Codon cannot expressgate.record.GateRecord.__init__: the generated constructor of a dataclass whose requiredgatefield is the gate as the document stores it, adict[str, Any]of mixed value types (see_DETAILS), which Codon cannot express; its other methods are called on a value of the classgate.record.GateRecord.from_dict: reads the dict to_dict() wrote, aMapping[str, Any]of mixed value types (see_DETAILS), which Codon cannot expressgate.record.GateRecord.to_dict: builds a dict[str, Any] of mixed value types, which Codon cannot expressgate.record._array: reads an array of a parsed JSON object (Mapping[str, Any], returninglist[Any]) of mixed value types, which Codon cannot express; only the uncalled from_dict() methods call itgate.record._object: reads an object of a parsed JSON object (Mapping[str, Any]) of mixed value types, which Codon cannot express; only the uncalled GateRecord.from_dict() calls itgate.record._whole: reads a whole number of a parsed JSON object (Mapping[str, Any]) of mixed value types, which Codon cannot express; only the uncalled GateCase.from_dict() calls itgate.record._wholes: reads an array of whole numbers of a parsed JSON object (Mapping[str, Any]) of mixed value types, which Codon cannot express; only the uncalled GateCase.from_dict() calls itgate.record.read_gate_record: parses a JSON document (Any) and reads it with GateRecord.from_dict(), aMapping[str, Any]of mixed value types, which Codon cannot expressgate.record.write_gate_record: serialises GateRecord.to_dict(), adict[str, Any]of mixed value types (see_DETAILS), which Codon cannot expressgate.unrecorded._recorded_at_stem: takes a batch summary's entry as json.loads() reads it (dict[str, Any]), of mixed value types, which Codon cannot express, so the check has nothing to call it with; _summary_cases() calls it with the entry json.loads() read, so its body is compiled through ithils.host_devices.HostSetup.playbooks: builds each playbook's variables as aDict[str, Any]of mixed value types (a list of interfaces beside a user and booleans), which Codon cannot express; what it derives them from is the checked host_devices()'shils.wire.encode: takes a message of any shape (Mapping[str, Any]: JSON values of mixed types), which Codon cannot expressmaps.config._flag: reads a booleanmapsetting with _value, off a config of any shape (map_config: Any), which Codon cannot express; only resolve_map_paths, itself uncalled, calls it, and passes what it reads to the checked _resolve()maps.config._value: reads onemapsetting off a config of any shape (map_config: Any: a MapConfig, an OmegaConf node or the editor's MapRef, read with hasattr/getattr and indexing), which Codon cannot express; only map_setting and _flag, both themselves uncalled, call it, and what they read is resolved by the checked _resolve()maps.config.map_setting: reads onemapsetting off a config of any shape (map_config: Any) with _value, which Codon cannot express; only resolve_map_paths, itself uncalled, and the excluded examples.run (_map_setting(), the batch map check) call it, and what it reads is resolved by the checked _resolve()maps.config.resolve_map_paths: takes themapconfig node asAny(a MapConfig, an OmegaConf node or the editor's MapRef, read with hasattr/getattr and indexing), which Codon cannot express; it only reads the settings, and the checked _resolve() resolves thempedestrian.soma._as_matrices: returns a motion's poses as they come when they are(N, J, 3, 3)matrices, or converts(N, J, 3)axis-angle ones with _rotvec_to_matrix: an array of 3 or 4 dimensions (np.ndarray[Any, ...]), which Codon, typing an array by its number of dimensions, cannot express; only load_soma_motion, itself uncalled, calls itpedestrian.soma.load_soma_motion: reads a SOMA motion's .npz withnumpy.load(..., allow_pickle=False): Codon's numpy cannot read an .npz, and its load() has noallow_pickle; the checked actions.play_motion, which calls it, imports pedestrian.soma through its model (MODELLED_IMPORTS), so it is not compiled through that call eitherprepare.docker.positive_seconds: takes a value of a backend's block of the run's config of any type (Any: a Hydra node's), which Codon cannot express (a timeout); only read_docker_settings, itself uncalled, and the excluded pedestrian.ardy call itprepare.docker.read_docker_settings: takes a backend's block of the run's config (Mapping[str, Any]: a Hydra node of mixed value types), which Codon cannot express; only the excluded pedestrian.ardy calls itprepare.docker.string_list: takes a value of a backend's block of the run's config of any type (Any: a Hydra node's), which Codon cannot express (a list setting,docker_args); only read_docker_settings, itself uncalled, calls itprepare.docker.whole_mib: takes a value of a backend's block of the run's config of any type (Any: a Hydra node's), which Codon cannot express (an amount of memory); only read_docker_settings, itself uncalled, calls itros2.config.CommandSourceConfig.from_mapping: takes a Hydra node (Mapping[str, Any]) of mixed value types, which Codon cannot express, checks its keys with utils.config, which reads a dataclass's fields from its class (type), and builds the config with**of a dict; utils.config is imported inside it, so it is not compiledros2.config.Ros2CameraConfig.from_mapping: takes a Hydra node (Mapping[str, Any]) of mixed value types, which Codon cannot express, checks its keys with utils.config, which reads a dataclass's fields from its class (type), and builds the config with**of a dict; utils.config is imported inside it, so it is not compiledros2.config.Ros2EgoConfig.from_mapping: takes a Hydra node (Mapping[str, Any]) of mixed value types, which Codon cannot express, checks its keys with utils.config, which reads a dataclass's fields from its class (type), and builds the config with**of a dict; utils.config is imported inside it, so it is not compiledroute.model.Range.parse: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot expressroute.model.RouteMatch.from_config: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot expressroute.model.RouteMatch.to_config: builds a dict[str, Any] of mixed value types, which Codon cannot expressroute.model._blank: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot express (whether a route search left a value unset); only _whole_number, itself uncalled, calls itroute.model._choice: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot express (a segment's shape or turn, a path's approach or turn: text, null, or a value of another type to refuse); only _parse_segment and _parse_paths, themselves uncalled, call itroute.model._distance: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot express (a route search's ego_spawn_s or ego_goal_margin); only parse_route_search, itself uncalled, calls itroute.model._parse_paths: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot express (a junction segment's paths); only _parse_segment, itself uncalled, calls itroute.model._parse_segment: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot express (one segment of a route search); only parse_route_search, itself uncalled, calls itroute.model._search_number: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot express (a route search's match_index, max_matches or seed: null, or a value to refuse when blank, #2266); only parse_route_search, itself uncalled, calls itroute.model._tristate: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot express (a segment's opposite_lane, stop_line, ... flags); only _parse_segment, itself uncalled, calls itroute.model._whole_number: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot express (a route search's match_index, max_matches or seed, the ODD sample count, and every whole-number setting a run reads through odd.sampler's whole_number_setting(): the run, ODD sample and background_traffic seeds and the server and background_traffic counts); only parse_route_search and RouteMatch.from_config, themselves uncalled, and the excluded odd.sampler call itroute.model.parse_route_search: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot expressrun_timeout.run_timeout_seconds: takes the run's config (Mapping[str, Any]: a Hydra node of mixed value types), which Codon cannot express; the check it applies is check_run_timeout'sscenario_base.BaseScenario.register_cover: builds a CoverItem of values of any type (expression: Callable[[carla.World], Any],values: Iterable[Any],ignore: Callable[[Any], bool],event: SamplingEvent | BaseCondition), which Codon cannot express; coverage.items is not checkedscenario_hash._without_endpoints: walks a run's config of any shape (Any: mappings, lists and scalars of mixed types at any depth) and returns its copy asAny, which Codon cannot express; only the uncalled scenario_hash calls itscenario_hash.scenario_hash: takes a run's config of any shape (Mapping[str, Any], a mapping of mixed value types at any depth), which Codon cannot express; only the excluded examples.run calls itscenario_queue.ScenarioQueue.as_fixture: builds a pytest fixture (pytest, imported inside it), a generator function returned as aCallable[[], Generator[ScenarioQueue, None, None]]scenario_runner._build_condition_status: builds a ConditionStatus, which holds the condition's details (dict[str, Any]), which Codon cannot express; only _collect_condition_statuses, itself uncalled, calls itscenario_runner._collect_condition_statuses: builds ConditionStatus values, which hold a condition's details (dict[str, Any]), which Codon cannot express; _condition_statuses calls it in theelsebranch ofif TYPE_CHECKING:scenario_runner._follow_weather_rules: follows the scenario's weather rules with weather_rules.use_weather_rules() (imported inside it), which reads rule files with openodd (a Rust extension) and takesenvironment.rulesasMapping[str, Any], which Codon cannot express, and which the weather_rules model does not declare; _run_scenario calls it in theelsebranch ofif TYPE_CHECKING:(#1936)scenario_runner._judge_metrics: evaluates the recorded frames with metric_batch (the frames replayed as a world of duck-typed actors, and the CommonRoad workers, processes spoken to in JSON), which is not checked; _run_scenario calls it in theelsebranch ofif TYPE_CHECKING:. How its verdicts merge into the run's result is checked, in _merge_metric_verdictsscenario_runner._record_metrics: writes the metric series evaluate_metrics returns into the trajectory log, both held asAny, in rows of mixed float and bool values (dict[str, Optional[Union[float, bool]]]); only _judge_metrics, itself uncalled, calls itscenario_runner._traffic_record: reads the traffic backend's describe() and fidelity_census(), dicts of mixed value types, which Codon cannot express; _run_scenario calls it in theelsebranch ofif TYPE_CHECKING:scenario_runner._write_gate_record: calls a scenario's gate_record() found with getattr(), a method of no declared type (make(result, file_stem)returnsAny), and writes its GateRecord with gate.record's write_gate_record(), itself uncalled (adict[str, Any]of mixed value types); _run_scenario calls it in theelsebranch ofif TYPE_CHECKING:scenario_runner._write_result: merges the traffic record (Any) and environment.write()'s record (adict[str, Any]of mixed value types, of the uncalled write()) into the result document, which Codon cannot express; _run_scenario calls it in theelsebranch ofif TYPE_CHECKING:sensor.base.drain: generic over what the queue holds (a TypeVar parameter, a camera's frames or a LiDAR's sweeps), so the check has nothing to call it with; the camera's and the LiDAR's destroy() call it with their queues, so its body is compiled through themsensor.base.put_dropping_oldest: generic over what the queue holds (a TypeVar parameter, a camera's frames or a LiDAR's sweeps), so the check has nothing to call it with; the camera's and the LiDAR's listeners call it with their queues, so its body is compiled through themsignals.controller._group_members: reads the authoring package's pydantic signal groups (list[Any], so the runtime does not import authoring); only build_controllers, itself uncalled, calls itsignals.controller._phase_states: reads one of the authoring package's pydantic phases (Any, so the runtime does not import authoring), with getattr defaults for fields older documents lack; only build_controllers, itself uncalled, calls itsignals.controller.build_controllers: reads the authoring package's pydantic specs (MapRef.traffic_signal_controllers and signal_groups), typedlist[Any]so the runtime does not import authoringslurm.manifest.BatchLayout.reserve_submission: takes the submission record its submitter builds and fills in after sbatch (slurm.submit's submit_cases), adict[str, Any]of mixed value types, which Codon cannot express, and sets itsindexin place; creating the record exclusively is the checked _link_exclusive()'s, and reading it the checked SubmissionRecord'sslurm.manifest.Manifest.__getitem__: returns a value of a SLURM batch's manifest asjson.loads()read it, for the readers the check leaves out (slurm.submit, slurm.worker, slurm.cli, the gate, the viewer), of any type (Any), which Codon cannot express; what the checked readers ask of the manifest is the checked Manifest's and ManifestCase'sslurm.manifest.Manifest.get: returns a value of a SLURM batch's manifest asjson.loads()read it, for the readers the check leaves out (slurm.submit, slurm.worker, slurm.cli, the gate, the viewer), of any type (Any), which Codon cannot express; what the checked readers ask of the manifest is the checked Manifest's and ManifestCase'sslurm.manifest._create_exclusive: writes a submission record of any value types (data: Any) as JSON, which Codon cannot express; creating the file exclusively from that text is the checked _link_exclusive()'sslurm.manifest.config_files: walks a resolved config of any shape (config: Any: the dicts and lists of an OmegaConf container), which Codon cannot expressslurm.manifest.read_json: returns the JSON document it reads, of any type (Any), which Codon cannot expressslurm.manifest.verdict_of_document: takes a case's outcome asjson.loads()reads it, a document of any type (Any: an object of mixed value types when it is one), which Codon cannot express; it writes its result back as JSON text for a JsonMembers, and the verdict is the checked recorded_verdict()'sslurm.manifest.write_json: writes a JSON document of any value types (data: Any: a manifest, a submission, a task's status), which Codon cannot expressslurm.records.JsonMembers.from_value: takes the object as Python holds it, adictof keys and values of any types (value: Any), which Codon cannot express; reading the members from an object's text is the checked parse_object()'sslurm.settings.SlurmSettings.from_mapping: takes thelauncherconfig (Mapping[str, Any]: a Hydra node of mixed value types), which Codon cannot express, reads the dataclass's fields from its class (dataclasses.fields(cls)) and builds the settings with**of adict[str, Any]slurm.settings.SlurmSettings.replace: takes**changes: Anyand reads them, merged into to_dict()'sdict[str, Any], back with from_mapping, which Codon cannot expressslurm.settings.SlurmSettings.to_dict: returns the settings as the manifest stores them (dataclasses.asdict()), adict[str, Any]of mixed value types, which Codon cannot expressslurm.settings._flag: takes a value of thelauncherconfig of any type (Any: a Hydra node's), which Codon cannot express; only SlurmSettings.from_mapping, itself uncalled, calls it (or another of its readers)slurm.settings._optional_int: takes a value of thelauncherconfig of any type (Any: a Hydra node's), which Codon cannot express; only SlurmSettings.from_mapping, itself uncalled, calls it (or another of its readers)slurm.settings._optional_str: takes a value of thelauncherconfig of any type (Any: a Hydra node's), which Codon cannot express; only SlurmSettings.from_mapping, itself uncalled, calls it (or another of its readers)slurm.settings._seconds: takes a value of thelauncherconfig of any type (Any: a Hydra node's), which Codon cannot express; only SlurmSettings.from_mapping, itself uncalled, calls it (or another of its readers)slurm.settings._str_list: takes a value of thelauncherconfig of any type (Any: a Hydra node's), which Codon cannot express; only SlurmSettings.from_mapping, itself uncalled, calls it (or another of its readers)slurm.settings._whole_number: takes a value of thelauncherconfig of any type (Any: a Hydra node's), which Codon cannot express; only SlurmSettings.from_mapping, itself uncalled, calls it (or another of its readers)slurm.settings.launcher_type: takes the run's config (Mapping[str, Any]: a Hydra node of mixed value types), which Codon cannot expresssweeper.bindings._check_offset: checks a stop-line offset as a YAML document or the editor wrote it (value: Any: an int, a float or numpy's numbers), which Codon cannot express; StopLineOffsetBinding's and StopLineApproachBinding's __post_init__ call it with their own field, so its body is compiled through themsweeper.bindings.parse_binding: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot expresssweeper.constraints._bind_children: reads the children of a constraint of any kind with hasattr (constraint: Any), which Codon cannot express; the constraints' _bind_set_level() call it with themselves, so its body is compiled through themsweeper.constraints._bind_set_level_constraints: walks a constraint tree of any node (constraint: Any: a Constraint, or any object withconstraintsorconstraint), which Codon cannot express; find_matching_lanelets calls it with each Constraint, so its body is compiled through itsweeper.constraints._cache_ids: sets_cached_idson a frozen constraint of any kind withobject.__setattr__(constraint: Any), which Codon cannot express; the set-level constraints' _bind_set_level() call it with themselves, so its body is compiled through themsweeper.constraints._check_length: checks a lanelet_length value as a YAML document or the editor wrote it (value: Any: an int, a float or numpy's numbers), which Codon cannot express; LaneletLengthConstraint's __post_init__ calls it with its own field, so its body is compiled through itsweeper.constraints._lanelet_id: reads one lanelet id as a YAML document or the editor wrote it (value: Any: an int, a whole float or a string of digits), which Codon cannot express; EqualsConstraint's __post_init__ calls it with its own field, and _lanelet_ids, which InSetConstraint and OnRouteConstraint call, with each of theirs, so its body is compiled through themsweeper.constraints._lanelet_ids: reads a list of lanelet ids as a YAML document or the editor wrote it (values: Any), which Codon cannot express; InSetConstraint's and OnRouteConstraint's __post_init__ call it with their own fields, so its body is compiled through themsweeper.constraints._real_number: reads a number of metres as a YAML document or the editor wrote it (value: Any: an int, a float or numpy's numbers), which Codon cannot express; _check_length() and sweeper.bindings._check_offset() call it with the value they check, so its body is compiled through theirssweeper.constraints._unresolved: asks whether a lanelet id, or a list of them, as a YAML document or the editor wrote it (values: Any) is still an unresolved${...}string, which Codon cannot express; EqualsConstraint's, InSetConstraint's and OnRouteConstraint's __post_init__ call it with their own fields, so its body is compiled through themsweeper.constraints.parse_constraint: reads a YAML mapping of values of any type (Any: numbers, strings, booleans, lists, mappings or None), which Codon cannot expresssweeper.isolation._raise_interrupted: a signal handler, whose frame is CPython's frame object (_frame: Any), which Codon has no type for; _run_in_child installs it with signal.signal(), whose shim calls it as CPython calls a handler, so its body is compiled through thatsweeper.isolation._request_pdeathsig: calls Linux's prctl through ctypes (imported inside it), which Codon has nothing for; _die_with_parent calls it in theelsebranch ofif TYPE_CHECKING:sweeper.isolation._run_in_child: takes the signal mask signal.pthread_sigmask() returns, aset[int | signal.Signals], which Codon cannot express (a set of a union); _launch_isolated calls it with the mask it holds, so its body is compiled through itt4_settings.T4Settings.from_driver: takes thet4config section (Mapping[str, Any]: a Hydra node of mixed value types), which Codon cannot express, and builds the settings with**of adict[str, Any]traffic.base.TrafficBackend.describe: builds a dict[str, Any] of mixed value types, which Codon cannot expresstraffic.build_hybrid: takes a backend's options asMapping[str, Any], a Hydra node of mixed value types, which Codon cannot expresstraffic.build_none: takes a backend's options asMapping[str, Any], a Hydra node of mixed value types, which Codon cannot expresstraffic.build_sumo: takes a backend's options asMapping[str, Any], a Hydra node of mixed value types, which Codon cannot expresstraffic.build_traffic_manager: takes a backend's options asMapping[str, Any], a Hydra node of mixed value types, which Codon cannot expresstraffic.config.TrafficConfig.from_mapping: takes a backend's options asMapping[str, Any], a Hydra node of mixed value types, which Codon cannot express, and checks its keys with utils.config, which reads a dataclass's fields from its class (type); utils.config is imported inside it, so it is not compiledtraffic.config.TrafficManagerBackendConfig.from_mapping: takes a backend's options asMapping[str, Any], a Hydra node of mixed value types, which Codon cannot express, and checks its keys with utils.config, which reads a dataclass's fields from its class (type); utils.config is imported inside it, so it is not compiledtraffic.config.traffic_backend_name: takes the traffic.backend setting as a value of any type (Any: None or a backend's name; a bool, a number, empty text or a list to refuse), which Codon cannot express; only TrafficConfig.from_mapping, itself uncalled, and the unchecked slurm.worker and examples.run call ittraffic.fidelity.MesoscopicConfig.from_mapping: takes a backend's options asMapping[str, Any], a Hydra node of mixed value types, which Codon cannot express, and checks its keys with utils.config, which reads a dataclass's fields from its class (type); utils.config is imported inside it, so it is not compiledtraffic.parameters.BicycleParameters.from_mapping: takes a traffic.vehicles, traffic.pedestrians or traffic.bicycles node asMapping[str, Any], a Hydra node of mixed value types, which Codon cannot express, and builds the parameters with**of thedict[str, Any]that _spreads_from, itself uncalled, returnstraffic.parameters.PedestrianParameters.from_mapping: takes a traffic.vehicles, traffic.pedestrians or traffic.bicycles node asMapping[str, Any], a Hydra node of mixed value types, which Codon cannot express, and builds the parameters with**of thedict[str, Any]that _spreads_from, itself uncalled, returnstraffic.parameters.VehicleParameters.describe: builds a dict[str, Any] of mixed value types, which Codon cannot expresstraffic.parameters.VehicleParameters.from_mapping: takes a traffic.vehicles, traffic.pedestrians or traffic.bicycles node asMapping[str, Any], a Hydra node of mixed value types, which Codon cannot express, and builds the parameters with**of thedict[str, Any]that _spreads_from, itself uncalled, returnstraffic.parameters._described: takes and returns a value of any type (Any: a Spread's describe() or the value unchanged), which Codon cannot express; only VehicleParameters.describe(), itself uncalled, calls ittraffic.parameters._number_of: takes one config value of any type (Any: a number or numeric text,'1.5'read as 1.5; a bool, other text, None, a list or a mapping to refuse), which Codon cannot express; only _spread_of, itself uncalled, calls ittraffic.parameters._spread_of: takes one config value of any type (Any: None, a Spread, a{mean, std}mapping, a number or numeric text; a bool, other text, a list or a mapping with other keys to refuse), which Codon cannot express; only _spreads_from, itself uncalled, calls ittraffic.parameters._spreads_from: takes a traffic.vehicles, traffic.pedestrians or traffic.bicycles node asMapping[str, Any], a Hydra node of mixed value types, which Codon cannot express, and the parameter class as a value (cls: type); checks the node's keys with utils.config, which reads a dataclass's fields from its class (type) and is imported inside it, and returns adict[str, Any]of Spreads and the values of the other fields, so it is not compiledtraffic.sumo.config.AmbientTrafficConfig.from_mapping: takes a backend's options asMapping[str, Any], a Hydra node of mixed value types, which Codon cannot express, and checks its keys with utils.config, which reads a dataclass's fields from its class (type); utils.config is imported inside it, so it is not compiledtraffic.sumo.config.BicycleConfig.from_mapping: takes a backend's options asMapping[str, Any], a Hydra node of mixed value types, which Codon cannot express, and checks its keys with utils.config, which reads a dataclass's fields from its class (type); utils.config is imported inside it, so it is not compiledtraffic.sumo.config.PedestrianConfig.from_mapping: takes a backend's options asMapping[str, Any], a Hydra node of mixed value types, which Codon cannot express, and checks its keys with utils.config, which reads a dataclass's fields from its class (type); utils.config is imported inside it, so it is not compiledtraffic.sumo.config.SumoBackendConfig.from_mapping: takes a backend's options asMapping[str, Any], a Hydra node of mixed value types, which Codon cannot express, and checks its keys with utils.config, which reads a dataclass's fields from its class (type); utils.config is imported inside it, so it is not compiledtraffic.sumo.network.SignalTable.links: asks roadgen's trace what a signal became in SUMO (Trace.translate()), answers of mixed value types (dict[str, Any]), which Codon cannot express, so the roadgen model does not declare ittraffic.traffic_manager.TrafficManagerBackend.describe: builds a dict[str, Any] of mixed value types, which Codon cannot expresstrajectory.model._refusing_vertex_times: the run-time wrapper of TrajectoryVertex.__init__ that refuses the calls written for a vertex with a time (*args: Any, **kwargs: Any, functools.wraps, an inspect.Signature); the check drops the decorator and compiles __init__ as writtentrajectory.model._vertex_advance: takes a TrajectoryVertex call's*argsand**kwargsas written (tuple[Any, ...],dict[str, Any]), which Codon cannot express; only _refusing_vertex_times, itself uncalled, calls itui.slurm_launch.SlurmOptions.from_request: reads the run panel's fields (dict[str, Any] | None: the JSON body of a request, of values of any type), which Codon cannot express, through a helper that returns a number of either type it is given (number(key, kind: type) -> Any, an int or a float); the checks of each field are the checked ui.slurm_options'ui.slurm_launch.Submission.to_dict: returns the submission as the viewer's API sends it, adict[str, Any]of mixed value types (text, an optional directory and an optional exit status), which Codon cannot expressui.slurm_launch.batch_status: returns the batch's progress as the viewer's API sends it, adict[str, Any]of mixed value types: the values of the batch's records asjson.loads()read them (the manifest's through the checked slurm.manifest's uncalled Manifest.get(), the submissions' and the tasks' through get(), of any type), which Codon cannot express; each case's state, and the values it shows as their JSON text, are the checked _case_states()'s (CaseState)utils.stop_line._road_marking_stop_lines: returns a road marking'srefersparameters as line strings, and arefersparameter may be any of lanelet2's primitives (a polygon, a lanelet, ...), which the lanelet2 model types as what they all have (id, attributes); regulatory_element_stop_lines calls it in theelsebranch ofif TYPE_CHECKING:value_text.shown: takes a config value of any type (Any: the value a validator refuses, an integer too long for repr() among them), which Codon cannot express; validators that takeAnythemselves, uncalled or in excluded modules, call it, and the checked traffic.fidelity._number (the traffic options), traffic.sumo.config's option checks (the SUMO backend's), traffic.authoring._number and _spread (the traffic-parameter form's fields), heavy_metrics._check_period (a heavy metric's period) and sweeper.bindings.RouteThroughBinding.__post_init__ (its depth) call it with the float or int the check types the value with; how it describes an integer too long for repr() is the checked huge_integer()'s
The 131 EXCLUDED modules fall in three groups. Many fit
more than one, so each belongs to the first group that fits it; most are in
the first:
- Those that import what Codon cannot compile: a package (pydantic,
pickle, grpc, protobuf, omegaconf and hydra, yaml, jinja2, fastapi, cv2,
cyclonedds, or openodd and hilspy, Rust extensions) or a standard module
Codon lacks (
inspect,concurrent.futures,importlibbeyondimportlib.metadata, or more ofos,pathlib,shutil,signal,socket,subprocess,tempfile,threading,dataclasses,getpass,importlib.metadataorurllib.parsethan their shims declare), at module level or inside a function the check calls. This is most of the tooling around a run (the editor, the UI, authoring, the sweeper's expansion and Hydra sweeper, the SLURM launcher's submitting side, the recordings and exports of a run, the HILS bridge), the ODD and coverage modules that wrap openodd, and the rest of what reads Lanelet2. A reason namespathlibonly where the module uses more of it than the shim declares, and the other shimmed modules likewise;test_typecheck_library.pyholds every reason to that. - Those that build on an excluded module, and import nothing Codon lacks themselves: they import a module that is neither checked nor modelled, or names that its package's model does not declare. Among them are packages that only re-export such a module. Some of them also hold values Codon has no type for.
- Those that hold values Codon has no type for, and nothing else keeps
them out:
Any, or a class as a value.
Each is listed with its reason:
The 131 EXCLUDED modules, and why
autoware_carla_scenario: loads its public names lazily with importlib (a PEP 562__getattr__returningobject), and Codon reads itsTYPE_CHECKINGre-exports as imports of every subpackage, two of which are neither checked nor modelled: pytest_fixtures (pytest) and registry (omegaconf)actions.smart_pedestrian: imports pedestrian.motion_client (grpc), pedestrian.ardy, prepare.declare (need_prepare()) and prepare.pool (running_backend()), which are neither checked nor modelled; applies@need_prepare(ArdyBackend)to its class, a class decorator the check does not support, and hands a class as a value (running_backend(ArdyBackend)), which Codon cannot hold; reads a keyframes file with json.loads and uses its values untypedauthoring: re-exports authoring.models (pydantic), authoring.compiler, authoring.hydra_config, authoring.starter and authoring.validator (pydantic, through authoring.models), authoring.persistence (yaml), authoring.package_export and authoring.wheelhouse (jinja2, packaging.specifiers) and authoring.registry (Anyfield defaults), none of which is checked or modelledauthoring._builders_generated: its builders take authoring.compiler's CompiledCondition, CompiledAction and BuildContext, which reach pydantic models through authoring.modelsauthoring.builders: dispatches to builders by name through vars() and globals() as Callable[..., Any]; its parameters reach pydantic models through authoring.compiler, and it imports authoring._builders_generated and authoring.models (pydantic) inside functions, which are not checkedauthoring.codegen: imports argparse, importlib, inspect, difflib, and templating (jinja2) and authoring.registry (Anyfield defaults), which are not checkedauthoring.compiler: reaches pydantic models through authoring.modelsauthoring.hydra_config: reaches pydantic models through authoring.models; imports yaml, authoring.persistence (yaml) and, inside scenario_map_refusal() and _map_group_file(), the registry (omegaconf, importlib), which are neither checked nor modelled, and, inside _shown() and _same_source(), the checked maps.source; builds config trees asdict[str, Any]of mixed value types (ROUTE_CONFIG_DEFAULTS, build_scenario_config()), which Codon cannot expressauthoring.models: imports pydanticauthoring.package_export: imports more of importlib.metadata than its shim declares (metadata()), annotates a field and a parameterdict[str, Any], which Codon cannot express, and, at module level, authoring.models (pydantic), authoring.persistence (yaml), authoring.hydra_config and authoring.validator (pydantic, through authoring.models), templating (jinja2) and authoring.wheelhouse (jinja2, packaging.specifiers), which are not checked and have no model; its naming rules and the framework pin's README lines are the checked authoring.package_names'authoring.persistence: imports yaml, and authoring.models (pydantic), which is not checked and has no modelauthoring.registry: FieldSpec.default is Any and holds str, int, float, bool, list or None across the spec table, which Codon has no type for; rule_symbol takes object, and searchable_lanelet_fields a union of spec classes; the spec table is built when the module is imported (243 FieldSpec calls), so no UNCALLED entry can leave the Any-typed defaults out, and typing FieldSpec.default would change what the editor, validator and codegen readauthoring.starter: reaches pydantic models through authoring.modelsauthoring.validator: imports authoring.models (pydantic) and authoring.hydra_config (yaml, and pydantic through authoring.models), which are not checked and have no model; the id rules it reports are the checked authoring.package_names'authoring.wheelhouse: imports jinja2, packaging.specifiers (which the packaging model does not declare), tomllib, tomli and, at module level,ThreadPoolExecutorfrom concurrent.futures; its requirements pinning is checked as authoring.requirements_pinautoware_bridge._proto.autoware_bridge_pb2: imports google.protobufautoware_bridge._proto.autoware_bridge_pb2_grpc: imports grpcautoware_bridge.grpc_server: imports grpc, concurrent.futuresautoware_bridge.ros_bridge.__main__: runs the scenario_bridge node: imports autoware_bridge.ros_bridge.node (rclpy), which is neither checked nor modelledautoware_bridge.ros_bridge.client: imports grpcautoware_bridge.ros_bridge.node: imports rclpy, autoware_adapi_v1_msgs, autoware_vehicle_msgs, geometry_msgs, grpcautoware_stack.devcontainer: imports yaml; reads a JSONC and a Compose document of values of any type (Any), which Codon cannot express. The checked autoware_stack.docker and the package take what they use of it (DevContainer,DevContainerError,read_devcontainer(),DEFAULT_WORKSPACE_MOUNT) from its model (autoware_stack/devcontainer.codon); its JSONC,${VAR}interpolation and workspaceMount text parsers are the checked autoware_stack.devcontainer_text'scarla_install: imports tqdm, argparse, urllib.request, http, email, tarfile, and more of pathlib and shutil than their shims declare (PurePosixPath,shutil.disk_usage()), and reads the install's stamp from JSON of any value types; its paths, lock file and the opening of it (_open_lock(),_UNWRITABLE_ERRNOS, #2144), needed room, swap, the decision whether an install is current and the conditional request that asks so, and the error of a run that can check an install but cannot write it (_cannot_install()), are the checked carla_install_paths's, which it re-exports, and the errors of a flock the file system does not support (_NO_FLOCK_ERRNOS) the checked slurm.start_lock's (#2318); holding the lock (_locked()) stays: it is acontextlib.contextmanagergenerator and tells a read-only lock without flock byerrno.EBADF, neither of which their shims declarecommonroad.session: imports concurrent.futures, and commonroad.worker and, inside a function, metric_batch, which are not checked; holds the workers' settings as values of any type (dict[str, Any]); which metrics are heavy, the environment that turns them off and the period checks are the checked heavy_metrics' (#1689), which it re-exports; commonroad/session.codon declares what the checked commonroad package re-exports of itcommonroad.worker: imports select (select.poll(),select.POLLIN), which Codon has nothing for; starts the worker with a keyword the model'sPopen()does not declare (bufsize=1) and, in close(), tries its write lock withacquire(blocking=False), which the model's Lock (Codon's, whose keyword isblock) refuses; decodes the bytes it reads off the worker's output (.decode()), which the check, reading bytes asstr, refuses; takes the worker's output stream typedAnyand speaks to the worker in JSON objects of mixed value types (dict[str, Any]), which Codon cannot express; the command that starts the worker, the worker's error and the excerpt an error quotes (worker_command(),CommonRoadWorkerError,WORKER_ENV,WORKER_SCRIPT,_checkout_project(),_excerpt()) are the checked commonroad.worker_start's (#2116), which it re-exports; what is left is CommonRoadWorker, the running worker; commonroad/worker.codon declares it, which the checked commonroad package re-exports, and, for a scenario importing them from commonroad.worker,WORKER_ENV,CommonRoadWorkerErrorandworker_command()coverage.cod: re-exports openodd's COD export (a Rust extension)coverage.collector: derives openodd's collector (a Rust extension); importstypes.SimpleNamespace, which Codon has nothing for; the names a versioned measure's cover items are written under are the checked coverage.item_names'scoverage.items: re-exports openodd's cover items (a Rust extension)coverage.report: re-exports openodd's coverage merge and report (a Rust extension)crash_report: imports sentry_sdk and faulthandler, which are neither checked nor modelled; subclasses a log handler and sentry_sdk's transport, and walks an exception's__cause__and__context__, which Codon cannot expressdeclarative: imports authoring.builders, authoring.compiler, authoring.models (pydantic) and authoring.persistence (yaml), which are not checked and have no modeldriver.egodriver_client: imports grpc, google.protobuf, numpy.typing (NDArray), which Codon's numpy lacks, and carla_driver_interface's contract, policies, server, service, driver, inprocess and protocol, which its model (geometry only) does not declaredriver.hdmap_export: hands the converter's module around as a value (_import_roadgen()returns it, typedAny, orNonebeside it) and the map it reads asworld_map: Any, which Codon cannot express (the model ofroadgen, #2312, declares read_opendrive() and the map's read_warnings(), export_ir() and write_read_trace(), which it calls, but not export_clipgt()); export_map() names its scratch directorywith tempfile.TemporaryDirectory(...) as scratch, which Codon binds to the manager, not to the directory's name; _can_export() and _export() test for and call the map's exporter by a name held in a variable, through hasattr() and getattr() (export_{name}), which Codon does not compile; _file_digest() reads the file in chunks with the two-argumentiter(callable, sentinel), and Codon'siter()takes one argument; builds the manifest and its format records asDict[str, Any]of mixed value types, and _known() reads them asMapping[str, Any], which Codon cannot express; the file names of a map set, where each format is written (MAP_FORMATS) and map_id_for() are the checked driver.hdmap_files's (#1728), which it re-exportsdriver.renderer: imports carla_driver_interface.protocol (the protobuf messages it builds and reads: CarlaRendererData, CarlaActorState, CarlaWeather, TrafficLight, StopPoint, LidarSweep, the TrafficLightState values, and common_pb2's AABB, DynamicState and Vec3), which its model (geometry only) does not declare; and _weather() reads the weather's fields by a name held in a variable, through getattr(), and passes them to CarlaWeather as ** a dict, neither of which Codon compiles; and the velocity it reports for an actor is a measured Vector3 or CARLA's Vector3D in one conditional expression, which Codon cannot type; which light governs the ego and where it stops for it are the checked driver.stop_lines'editor: imports argparse and serve_cli (argparse), which is not checked; imports uvicorn and editor.app (fastapi) inside main()editor.app: imports fastapi and web_guard (fastapi), and authoring.models (pydantic), authoring.hydra_config, authoring.persistence, authoring._builders_generated and editor.service (pydantic, through authoring.models), authoring.registry, trajectory.authoring and editor.forms, and, inside create_app(), editor.routes (fastapi), which are neither checked nor modellededitor.forms: imports trajectory.authoring, authoring.hydra_config (yaml, pydantic) and authoring.registry, which are not checked, and authoring.registry has no model; parses form values typed Any against a FieldSpec (parse_int_list(), parse_value(), parse_params(), parse_records() and _apply_row_op() stay here, and the bools through authoring.registry's read_bool()); the whole numbers it parses, a list field's items and a record list's row count, are the checked editor.form_values's (_parse_list_int(), the split at _LIST_SEPARATORS and the _MAX_RECORDS bound, #2135)editor.map_preview: imports authoring.models (pydantic) and authoring.hydra_config (yaml, and pydantic through authoring.models), which are not checked and have no modeleditor.routes: imports fastapi, starlette (starlette.types), pydantic (ValidationError) and web_guard (fastapi), which is not checkededitor.scenario_map: imports authoring.models (pydantic), authoring.hydra_config (yaml, and pydantic through authoring.models) and editor.map_preview, which are not checked and have no modeleditor.service: imports yaml, zipfile, stat and weakref (WeakValueDictionary), which Codon has nothing for, andcontextlib.contextmanager; reaches pydantic models through authoring.models; takes form fields asMapping[str, Any], which Codon cannot express; the gate fields the inspector edits and how they are shown are the checked editor.gate_fields's, and the name a new row gets the checked editor.naming'sentity.carla_driver_entity: imports carla_driver_interface.driver (TurnIndicator) and carla_driver_interface.protocol, which its model (geometry only) does not declare, driver.renderer (the protocol's messages) and rerun_lidar (active_recorder(); the Rerun SDK;CameraMountis the checked rerun_sensor_mount's, #1688), and, inside functions, driver.hdmap_export (itself excluded) and t4_scene.live (t4_scene.convert; also underTYPE_CHECKING), which are neither checked nor modelled, andactive_recording(),offer_driver_sensors()andwithdraw_driver_sensors()from t4, which the t4 model does not declare; builds an EgoDriverGrpcClient (grpc), which its model refuses to build, and encodes camera frames with encode_frame_jpeg (cv2)entity.hils_entity: imports hils.protocol (socket), which is neither checked nor modelled, for itsLineReader, whoseread()returnsList[Dict[str, Any]]; uses more of socket than its shim declares (settimeout(),sendall()); takes and sends messages of any shape (Any,Mapping[str, Any]), and builds its config from a mapping throughcls.__dataclass_fields__andcls(**values), which Codon cannot expressexamples: re-exports examples.run (omegaconf, hydra) and the registry (omegaconf, importlib)examples.compose_errors: imports difflib, and, inside functions, hydra and omegaconf (their errors), the registry (omegaconf, importlib) and examples.run (hydra, omegaconf), which are neither checked nor modelled; reads a composed config of any shape (Any), which Codon cannot expressexamples.expand: imports the registry (omegaconf, importlib) and sweeper.expand (omegaconf), and, inside main(), authoring.hydra_config (pydantic, through authoring.models), odd.controls (omegaconf, difflib) and examples.compose_errors (difflib, and hydra, omegaconf and examples.run inside its functions), which are not checked, andOddDrawsfrom odd.sampler, which the odd model does not declare; holds the composed config asAny, which Codon cannot expressexamples.run: imports omegaconf, hydra, and typecheck and typecheck.mode (the checker, outside the library check), registry (omegaconf, importlib), prepare.pool (BackendPool) and hydra_compat (argparse, hydra), crash_report (sentry_sdk) and, inside functions, pedestrian.ardy (grpc;ArdyBackend, inside _check_backend_settings()), authoring.hydra_config (pydantic, through authoring.models; inside check_scenario_maps()), sweeper.expand (omegaconf), commonroad.session (concurrent.futures), odd.controls (omegaconf, difflib), slurm.submit (omegaconf), utils.config (a dataclass's fields read from its class) and entity.hils_entity (hils.protocol), which are neither checked nor modelled, and, inside _cap_malloc_arenas(), ctypes and ctypes.util, which Codon has nothing for, and more of platform than its shim declares (platform.libc_ver()); importsScenarioQueue(and, inside build_ego_entity() and _autoware_bridge_config(),AutowareBridgeConfigandAutowareEgoEntity) from the package,PlannedRoute,RouteError,ODD_CASE_KEY,check_sample_settings(),decode_case(),seed_setting()andwhole_number_setting()(and, inside a function,OddDraws) from odd.route and odd.sampler, which the package and odd models do not declare,RerunConversion,RerunUnavailableErrorandreport()from rerun, which the rerun model does not declare,T4SettingsandT4UnavailableErrorfrom t4, which the t4 model does not declare, and, inside functions,use_weather_rules()andcheck_weather_rules()from weather_rules, which the weather_rules model does not declare; imports more of contextlib and threading than their shims declare (contextlib.contextmanager,threading.main_thread(),threading.current_thread()); reads the fields ofDockerAutowareConfigfrom its class (__dataclass_fields__), and those ofAutowareBridgeConfigthrough utils.config, and builds them,EnvironmentActionandRerunConverterwith**of a dict; holds a registered scenario's config class as a value (config_cls, fromget_scenario_classes()) and builds it withconfig_cls(**...)of the settings among itsdataclasses.fields()in _check_scenario_config(); none of which Codon compiles; takes a launcher setting of any type (value: object) in _launcher_str_list() and _launcher_str_mapping(), which Codon cannot express; holds settings asdict[str, Any]of mixed value types (a case's settings, the video, sensor and environment settings,_to_dict(),_rules_locations(spec: Any) -> Any), which Codon cannot express; what it reads off the command line and the environment is the checked examples.argv's, and what its Hydra argv asks for (whether it selects the lanelet sweeper too,lanelet_sweeper_requested_in_argv()) and the run directory it reserves are the checked run_dirs', as are why a directory a batch is told to write to cannot be its own and the removal of a reserved directory left empty (_batch_output_dir_refusal()and_remove_if_empty(), #2306), and the suffix of a run's result and the line naming it the checked run_outputs's (RESULT_SUFFIX, #1704, andresult_json_line())gate.evaluate: imports argparse, authoring.models (pydantic) and, inside functions, openodd's coverage merge (a Rust extension), and authoring.persistence (yaml), which are neither checked nor modelled, and reads a batch's manifest as adictreads it (the checked slurm.manifest's uncalled Manifest.get() andmanifest[key], values of any type), to find its cases that left no gate record; the batches under a directory, a local batch's summary read for its cases that left no gate record, and the--multirunjobs given up or never started are the checked gate.unrecorded's (#2124), which it re-exports; the criteria it judges, the trajectory consistency, the case coverage, finding the records and the changes from a baseline are the checked gate.criteria's, which it re-exports; a verdict's and a comparison's Markdown, the criteria and cases compared from a baseline and the records grouped by scenario are the checked gate.report's (#1738)hils.bridge: imports hilspy (a Rust extension) at module level under TYPE_CHECKING, for itsChannel,PeriodicandClockannotations, and inside _require_hilspy(), which hands the module on as atypes.ModuleTypevalue, and typesafe_carla's CPython package inside HilsBridge, which it holds as_carla: Any; uses more of threading than its shim declares (Event); holds hilspy's messages, CARLA's measurements and the queues that carry them typedAny; and builds on hils.link and hils.protocol (socket), hils.measurements (cv2) and hils.config (yaml), which are neither checked nor modelledhils.cli: imports argparse and importlib.resources, and serve_cli (argparse) and hils.config (yaml) and, inside main(), hils.bridge (hilspy) and, inside _setup(), hils.host_setup (ansible_runner), which are neither checked nor modelledhils.config: imports yaml inside load_config(), and reads a parsed YAML document of any shape (Any), holds a constant field value asConstant = Union[str, int, float, bool]and a device's codec options asMapping[str, Any], none of which Codon can express; imports at module level hils.protocol (socket) and hils.measurements (cv2), which are neither checked nor modelled; what YAML 1.1 makes of a plain scalar is the checked hils.yaml11's, and SensorSpec, the LiDAR attribute checks, the relative files, the check of a field's signal name (checked_signal(), its_signal()), the decimal text of an integer (_decimal()), CLOCKS, PHASES and HilsConfigError the checked hils.sensor_spec's, which it re-exports, as it does the checked value_text'shuge_integer()(its_huge(), #1704); hils/config.codon declares what the checked hils package re-exports of ithils.host_setup: imports importlib.resources, and, inside AnsibleRunner, ansible_runner, which Codon has nothing for; uses more of getpass than its shim declares (getpass.getpass(), the sudo password prompt); AnsibleRunner names its data directorywith tempfile.TemporaryDirectory(...) as data, which Codon binds to the manager, not to the directory's name; builds on hils.config (yaml), which is neither checked nor modelled; hands the playbooks' variables to a runner asDict[str, Any]of mixed value types, which Codon cannot express; the virtual devices a configuration's devices need, HostSetup and its description are the checked hils.host_devices's (#2124), which it re-exports, and host_setup() hands it each device's name and URLhils.link: imports more of socket than its shim declares (socket.create_server(),socket.has_dualstack_ipv6(),socket.timeout, and a socket'ssettimeout(),accept(),sendall()andshutdown(), andsocket.SHUT_RDWR), and builds on hils.protocol (socket), which is neither checked nor modelledhils.measurements: imports cv2 inside camera_frame(), and takes a buffer that is bytes, a memoryview or a numpy array, which Codon cannot express, and reads it withnp.frombuffer(), which takes only a str in Codon; declaresFloatPointsat module level asnp.ndarray[tuple[int, int], np.dtype[np.float32]], which is not Codon'sndarray[dtype, ndim]; returns a LiDAR scan's fields as aDict[str, np.ndarray]of float32 and uint8 arrays, which Codon cannot express; the gathering of a LiDAR's arcs into revolutions is the checked hils.lidar_revolution's, and the dating of a revolution's rays and its start and the re-observing of its points as a spinning sensor's (ray_times(),steady_revolution_start(),spin_lidar_points()) the checked hils.lidar_spin's (#2116), which it re-exportshils.protocol: imports selectors, which Codon has nothing for, and more of socket than its shim declares (socket.timeout, and LineReader reads the bridge's socket withrecv()), and reads JSON messages of any shape, which LineReader.read() returns asList[Dict[str, Any]]and Codon cannot express; what needs none of these, command_value() and the quote of a refused line in an error (_preview()) included, is hils.wire, which it re-exportshydra_compat: imports argparse and hydra (hydra._internal.utils), which is neither checked nor modelled; derivesargparse.ArgumentParserandtypes.ModuleType, and replaces a module global ofhydra._internal.utilsat importmaps.pack: hands CARLA content packs to carla-pack-registry (a Rust extension imported inside functions and held asAny) and the CARLA client, and reads its releases and map formats,dict[str, Any]of mixed value types, which Codon cannot express; maps/pack.codon models what scenario_queue uses of itmaps.road_network_cache: imports io (BytesIO,BufferedIOBase), pickle (Pickler,Unpickler) and weakref (WeakKeyDictionary); pickles pyxodr's objects typedAny, which Codon cannot express; maps/road_network_cache.codon models what MapManager uses of itmetric_batch: replays the frames metric_frames records as a world of duck-typed actors (SimpleNamespace transforms, a list subclass for ActorList) through the checked metrics module, and evaluates the CommonRoad metrics with commonroad.session (worker processes), which is not checkedodd:from . import backend, probesfails first: Codon reads it as an import ofoddfrom the top-level package's__init__, which the model does not declare ("cannot import name 'odd'"); behind that, it re-exports odd.model, odd.sampler, odd.openodd, odd.sources and odd.backend (openodd, a Rust extension), and odd.route and odd.registry, which are excluded (odd.probes is checked)odd.backend: derives openodd's Backend (a Rust extension), a dataclass of probe callables and knobs (Mapping[Any, OddKnob])odd.cli: imports argparse, openodd (inside a function), andodd_builder()andodd_names()from odd.registry, which the odd model does not declare; inside its commands, it importscombine_route_coverage()from odd.route, which the odd model does not declare either (it declares onlyplan_route_coverage()), and typecheck (the checker, outside the library check), examples.run (omegaconf, hydra) and registry (omegaconf, importlib), which are neither checked nor modelled; the split ofscenario-odd route's arguments into scenario configs and Hydra overrides (_split_items(), in _scenario_configs(), which expands the configs' globs) and the metres per bucket it prints (_format_buckets(), which it re-exports) are the checked odd.route_text's (#2135)odd.controls: imports omegaconf inside_key_exists()andvalidate_controls()(the config it checks the keys of is aDictConfig), difflib (for its did-you-mean hints), andcontrols_from_mapping()from odd.sampler (openodd, a Rust extension), insidevalidate_controls(); it reads thecontrolsof a config asAnyandMapping[Any, Any](a control is a key, or a mapping of any values:isinstanceonAny,spec.get()of mixed types), which Codon cannot express; the diagnostic it reports and its one-line text are the checked odd.control_diagnostics', and the fields and targets of a control the checked odd.control_fields' (#1733)odd.model: re-exports openodd's model (a Rust extension)odd.openodd: re-exports openodd's reader (a Rust extension)odd.registry: wraps openodd's registry (a Rust extension) and imports odd.model (openodd's model) at module levelodd.route: the route-coverage half of checking a planned route against an ODD (it imports the planning,PlannedRoute,RouteErrorandplan_route(), from odd.route_plan, which is checked, and re-exports it), built on openodd's model: each lanelet's attribute values are adict[str, Any](whatever a probe reads, a bool, int, float or str,None, or theUNDECIDEDsentinel, compared by identity) that it hands to openodd'sOddDefinition.complete()andevaluate()(a Rust extension, with no model) and to a cover item'sbucket_of(), so a typed record of them would be converted back to that mapping at the boundary, which the check cannot see past; a probe'son_laneletreader is an attribute of any callable (getattrwith a default), which Codon has no attributes of on a function; thedescribe()reports aredict[str, Any]of mixed values; and it names what the odd and coverage models do not declare (coverage.items.value_label(), and the probe, cover item and derivation ofOddAttributeand the attributes,complete(),evaluate(),situations()andoutside_buckets()ofOddDefinition)odd.sampler: wraps openodd's sampler and openodd.flaky's search (a Rust extension), imports base64 and yaml, and readsMapping[str, Any]config nodes (sampler_from_config()), which Codon cannot express; the fields of a control and ofsweep.odd_sample, the keys a sweep sets and whether a case's overrides set one (sets_any(), in _with_quantile()) are the checked odd.control_fields' (#1733), which it re-exportsodd.sources: re-exports openodd's git sources (a Rust extension)odd.traffic_probes: builds openodd's OddKnob (a Rust extension) for each probe, and each probe reads a traffic parameter by name (getattron the recorded parameters) and is a callable object keyed in adict[Callable[..., Any], OddKnob], which Codon cannot express; the probes' names, their knob keys and limits and the parameters' ranges are the checked odd.traffic_ranges'sodd.units: re-exports openodd's units (a Rust extension)pedestrian._proto.pedestrian_motion_pb2: imports google.protobufpedestrian._proto.pedestrian_motion_pb2_grpc: imports grpcpedestrian.ardy: imports grpc and pedestrian.motion_client (grpc), which is neither checked nor modelled; ArdyBackend.from_settings() reads the backend's settings asMapping[str, Any], which Codon cannot expresspedestrian.motion_client: imports grpc and the generated protobuf modules (google.protobuf); the contract's frame helpers and values it re-exports are the checked pedestrian.contract'sprepare.declare: sets and reads a class attribute by name (setattr,type.__dict__,__mro__) and imports a module by name (importlib), which Codon cannotprepare.pool: builds backends from the classes it is handed, by attribute name (getattr(cls, ...)), which Codon cannotpytest_fixtures: imports pytest, and takes the scenario's class (Type[BaseScenario]), which Codon cannot hold as a valueregistry: imports omegaconf (DictConfig) and authoring.models (ScenarioDocument; pydantic), both underTYPE_CHECKING, which are neither checked nor modelled, and more of importlib.metadata than its shim declares (entry_points()); holds scenario classes (type[BaseScenario]) and their config classes as values, which Codon cannot hold, and document loaders (Callable[[], ScenarioDocument]) that return a pydantic model as valuesrerun: imports concurrent.futures and types, and, inside functions, importlib (importlib.import_module(), in require_converter(), which its shim does not declare) and argparse (in main()), which are neither checked nor modelled; raisesFileNotFoundError(errno.ENOENT, os.strerror(errno.ENOENT), str(path))in convert_outputs(), of OSError's(errno, strerror, filename), which the model's FileNotFoundError refuses (it declares them of OSError itself and of a class a checked module derives from it); holds the converter package (imported at run time) asAny; the path rule of a log and its recording (TRAJECTORY_SUFFIX,RECORDING_SUFFIX,_label()andrecording_path()) is the checked run_outputs's (#1678), which it re-exports; how a conversion went, which recordings' conversions count and the run's summary of them (RerunConversion,latest(),report()) and the converter's keyword for a draw radius (_draw_radius_argument()) are the checked rerun_report's (#2124), which it re-exports; rerun.codon models what scenario_runner, scenario_queue and t4_scene.grid use of itrerun_lidar: imports the Rerun SDK (at run time), which Codon has no model of, and holds it asAny; the name and place of a run's sensor recording (SENSORS_SUFFIXandlidar_recording_path()) are the checked run_outputs's (#1678), which it re-exports; so are a camera's mount, the colours of a sweep, the sensors' entities and the parse ofrerun rrd print(CameraMount,_height_colours(),SENSOR_ENTITIES,_is_sensor_entity(),_CHUNK_LINEandchunk_entities()) the checked rerun_sensor_mount's (#1688), as is the scan of a recording's bytes for the sensors' entity paths (_SCAN_BLOCKandcarries_sensors(), #1697), which it re-exports too; entity.carla_driver_entity (itself excluded), rerun and rerun_metrics (itself excluded) use itrerun_metrics: imports the Rerun SDK (at run time), which Codon has no model of, and holds it asAny; reads the log's metric records as JSON objects of mixed value types (dict[str, Any]); the name of its scratch recording (METRICS_SUFFIX) is the checked run_outputs's (#1748), which it re-exportsros2.cyclone: imports cyclonedds (its domain participant, readers, writers, topics and QoS policies) and ros2.messages, which imports it too; ros2/cyclone.codon models the CycloneRos2Transport that entity.ros2_entity buildsros2.messages: imports cyclonedds (the IdlStruct base of every message, and the IDL field typesarray[float64, 36]andsequence[TrajectoryPoint], which Codon cannot express)scaffold.generator: imports argparse, ast, functools, importlib.metadata, templating (jinja2) and, inside functions, packaging.requirements (which the packaging model does not declare), typecheck.transform and authoring.package_export (pydantic, yaml, jinja2, through authoring.models, authoring.persistence and templating), which are not checked; its naming rules, and the framework-name rule it shares with authoring.package_names, are the checked scaffold.names'; putting a rendered package in place and the error of a target that exists (_move_into_place(),_occupied(),_RENAME_ONTO_OCCUPIED,PackageExistsError) are the checked scaffold.placement's (#2144), which it re-exports; scaffold/generator.codon declares what the checked scaffold package re-exports of it,PackageExistsErrorincluded, which the package still imports from scaffold.generator, so a scenario importing it from either finds the model'scompile_errors as beforeserve_cli: imports argparse: port() raisesargparse.ArgumentTypeErrorfor a bad --port value and add_bind_arguments() adds options to anargparse.ArgumentParser; the default host, the port range, which hosts are loopback ones and the warning are the checked serve_bind's, which it re-exportsslurm.cli: imports argparse, and slurm.submit, which are not checked, and reads the manifest as adictreads it (the checked slurm.manifest's uncalled Manifest.get() andmanifest[key], values of any type); what it collects, and which unfinished cases it resubmits, are the checked slurm.collect's, and whatstatusprints, whether a batch is busy (the resubmit's check), which casesresubmit --casesnames and whatcancelcancels the checked slurm.status'slurm.submit: imports omegaconf (inside functions), which is not checked, and takes the manifest as it builds it (dict[str, Any]) or as it is read back (the checked slurm.manifest's Manifest, through its uncalled get() andmanifest[key], values of any type) and a function asCallable[..., Any], which Codon cannot express; the job scripts and sbatch command lines it writes and runs are built, and the scripts written (_write_script(), #2124, which it re-exports), by the checked slurm.scripts, and run, as squeue, scancel and scontrol are, by the checked slurm.queue; following a submission until it ends and what a submitter says of it once submitted (follow, after_submit) are the checked slurm.follow's, which it re-exportsslurm.worker: imports crash_report (sentry_sdk), traceback,contextlib.contextmanager, more of threading and importlib than their shims declare (threading.Event,importlib.import_module()), omegaconf (inside functions), and, inside functions, commonroad.session (concurrent.futures) and examples.run (hydra, omegaconf), which are neither checked nor modelled, and, inside run_task(), typecheck.mode (the checker, outside the library check); importsRerunConverter(underTYPE_CHECKING) and, inside a function,report()from rerun, which is not checked (report() is the checked rerun_report's, #2124, which rerun re-exports), and the rerun model declares neitherreport()norRerunConverter.close(), and itsRerunConverter.wait()returns None, not the RerunConversion records the worker passes torerun_errors(); compares config values of any type (Any); the files of the start lock where the file system has no flock (creating, reading, releasing and removing a stale one, the file server's clock,_NO_FLOCK_ERRNOS) are the checked slurm.start_lock's (#2116), and so is taking a stale one over, one waiter at a time (_take_over(),_TAKEOVER_STALE_AFTER, #2144), which it re-exports; reading a ROS domain id an environment holds and taking a task's domain for one (_env_domain_id(),_take_domain()) are the checked slurm.ports' (#2124), which it re-exports; the start-turn wait, the reading of a process's status, the sum of a process tree's resident memory, the memory report and the bridges' addresses are the checked slurm.task_timing's, and whether a differing config string names the same file the checked slurm.same_files' (#1720)sweeper: re-exports, besides the checked sweeper.bindings, sweeper.constraints and sweeper.map_loader, sweeper.expand (omegaconf) and sweeper.lanelet_constraint_sweeper (hydra)sweeper.expand: imports omegaconf, and openodd inside a function; importsOddDraws(underTYPE_CHECKING) andODD_CASE_KEY,QUANTILE_DRAWS,case_settings(),check_sample_settings(),controls_from_mapping(),encode_case(),sample_draws(),sample_knobs(),sample_odd(),scenario_backend(),search_from_config(),seed_setting()andwhole_number_setting()from odd.sampler (inside _with_odd_samples(), _refuse_overridden(), _config_seed(), _case_index() and expand_config()),lanelet_readings()from odd.route (inside _case_readings()) andControlsErrorandvalidate_controls()from odd.controls (inside scenario_controls()), which the odd model does not declare; expand_sweep() and expand_route() take the sweep, the route search, the map and the ODD asAnyandMapping[..., Any], which Codon cannot express; the lanelet a base case is on, whether a drawn case fits a base case and which base case each drawn case is put on are the checked sweeper.case_choice's, as are the config key an override names and a binding's write of an override (#2279), and the lanelet key a walk-back moves (_walk_back_lanelet_key()) is the checked sweeper.bindings's (#1697), which it re-exportssweeper.lanelet_constraint_sweeper: imports omegaconf, hydra, tqdm, sweeper.expand (omegaconf) and hydra_compat (argparse, hydra), and, inside functions, examples.run (hydra, omegaconf) and odd.controls (omegaconf, difflib;ControlsErrorandformat_diagnostics()), which are neither checked nor modelled; derives hydra'sSweeperandCompactHydraException; importsRerunUnavailableErrorfrom rerun, which the rerun model does not declare, and, inside a function,T4UnavailableErrorfrom t4, which the t4 model does not declare; its cases are forked, stopped and waited for by the checked sweeper.isolation, a case's timeout is checked as a whole number of seconds and its verdict read as an exit status by the checked sweeper.job_status (#1712), as are why a job was given up, the text ofSWEEP_RESUME_FROMand the job directories no other job shares (#2279), and the stems its case runs left in a directory are worked out by the checked run_outputs, and what a job's directory is cleared of and given (an earlier run's or a failed attempt's outputs, the markers:_remove_earlier_outputs(),_remove_run_stems(),_write_unfinished_marker(),_write_marker(),_remove_marker()) is the checked sweeper.job_status's (#2306), which it re-exportst4: hands the world and the sensors to the T4 writer, a Rust extension imported at run time with importlib and held as a module (Any), and keeps NumPy arrays, datetime and threading.Condition waits Codon has no model of; t4.codon models what scenario_runner and scenario_queue use of itt4_scene.__main__: the scenario-t4scene command: imports argparse and t4_scene.convert, which is not checked; the logs it converts and the two of one stem it refuses to write into one directory are the checked t4_scene.grid's (trajectory_logs(),stem_collision())t4_scene._proto.t4_scene_pb2: imports google.protobuft4_scene.convert: reads JSON-lines records of mixed value types (dict[str, Any], which Codon cannot express) with orjson, imports numpy.typing (NDArray), which Codon's numpy lacks, and builds protobuf messages (t4_scene._proto); its frame math (the map frame's positions and angles, the rotations and quaternions) is the checked t4_scene.frames (#1665), and so is the check of a rear-axle offset (checked_rear_axle_offset(), #1697), which it re-exports for t4_scene.live and t4_scene.__main__; the suffixes of a T4Scene and its map (SCENE_SUFFIX,MAP_SUFFIX) are the checked run_outputs's (#1713); t4_scene.codon models the write_t4_scene() the checked t4_scene.exporter callst4_scene.live: builds the run's scenes with t4_scene.convert, which is not checked, from a subclass of trajectory_recorder's TrajectoryRecorder (excluded), and imports numpy.typing (NDArray); entity.carla_driver_entity (excluded) is all that uses itt4_scene.map_resource: builds protobuf messages (t4_scene._proto); the OSM bytes it carries and their digest (local_meters_osm(),resource_digest(),CENTERLINE_INVERTED_TAG) are the checked t4_scene.osm's (#2116), which it re-exportst4_scene.route: Codon crashes compiling RouteGraph.of(), which builds the lane graph with lanelet2's routing graph (a segmentation fault, or an assertion at translate.cpp:193); imports numpy.typing (NDArray), which Codon's numpy lacks; holds the lane graph asMapping[int, tuple[int, ...]]attributes; and moves a map's points withLanelet.resetCache()andLineString3d.__setitem__(), which the lanelet2 model does not declare; t4_scene.convert and t4_scene.live, which are not checked, are all that use ittemplating: imports jinja2; its filters escape with the checked text_escapetools.detect_no_3d_model_lanelets: imports omegaconf, yaml, tqdm, argparse, and serve_cli (argparse), which is not checked; writing the map configuration in place or through a temporary file, and describing the error of either (_write_in_place(),_write_atomically(),_PartialWriteError,_describe_os_error()), are the checked tools.file_writes' (#2135), which it re-exportstraffic.registry: holds backend factories behind theTrafficBackendFactoryalias,Callable[[Mapping[str, Any]], TrafficBackend]("expected type expression"), and takes their options asMapping[str, Any], which Codon cannot express; load_traffic_backend_plugins and the _walk_traffic_backend_plugins it calls import, inside them, autoware_carla_scenario.registry (_PLUGIN_LOAD_LOCK,load_entry_point_plugins()), which is neither checked nor modelled and is the first error the check reports (UNCALLED entries for both would skip those imports, but not the alias); traffic.codon models it for the traffic packagetraffic.sumo.backend: imports traci, sumolib and libsumo inside its functions, and builds its network and reads its signals and crossings with build_network(), SignalTable and crossing_edges() of the checked traffic.sumo.network, which the check leaves SignalTable.links() uncalled of, for roadgen's trace queries; holds SUMO's connection as_traci: Any(traci or libsumo, whichever _load_traci() finds) and sumolib's network as_net: Any, and takes CARLA's world, its actors and the scenario's entities typedAny, which Codon cannot express; the signal states it sets, SUMO's step per CARLA tick and the links a program lets go together are the checked traffic.sumo.signal_states', and its command-line options, mode bits, vehicle types and type files the checked traffic.sumo.options'; its pure helpers (_sight_cap(),_safe_weights_prefix(),_sumo_velocity(),_lane_step()andworld_opendrive()) are the checked traffic.sumo.geometry's (#2306), which it re-exportstraffic.sumo.hybrid: reads SUMO's network with sumolib (imported inside its functions) asAnyvalues, which Codon cannot express, and holds the mesoscopic model as a union of the checked traffic.mesoscopic's and the excluded traffic.sumo.mesosim's; the TrafficManager side it extends is the checked traffic.traffic_managertraffic.sumo.mesosim: starts SUMO as a process of its own (subprocess, socket) and holds a TraCI connection to it as_conn: Any(traci, imported inside its functions), which Codon cannot express; the vehicles it holds are the checked traffic.mesoscopic's MesoVehicle and its roads its MesoLink, and its pure helpers (segment_count(),_height_at()and_angle_between()) the checked traffic.sumo.geometry's (#2306), which it re-exportstrajectory.authoring: parses document values ofAny(YAML rows, editor fields) intodict[str, Any], which Codon cannot express; what it does with the editor's text and with parsed rows is the checked trajectory.vertex_text's, and the typed builders it re-exports (the goal, via lanelets, timing and lanelet path) are the checked trajectory.authored'strajectory_recorder: writes JSON-lines records of mixed value types (dict[str, Any], which Codon cannot express) serialised with orjson (module level;orjson.JSONEncodeError), reads the projector's settings with yaml (inside_projector_info), and neither has a Codon model; trajectory_recorder.codon models what scenario_runner uses of itui: imports argparse and serve_cli (argparse), which is not checked; imports uvicorn, ui.app (fastapi) and the registry (omegaconf, importlib) inside main(), and ui.recordings (tarfile) inside a functionui.app: imports fastapi, starlette (starlette.convertors), web_guard (fastapi), which is not checked, asyncio, more of contextlib than its shim declares (asynccontextmanager, for _lifespan()), and openodd (a Rust extension) inside _openodd(); whether a request body is sent as JSON, a session's reference and the ODD attribute a query names (_is_json_type(),_session_ref(),_attribute()) are the checked ui.request_values's (#2135), which it re-exports;_parse_ref()stays, reading the session's directory under this module's_base_path(), which tests patchui.cases: reads JSON documents, and the verdicts and durations in them, of any type (Any), and a run's overrides with yaml (imported inside the function); the path guard, the case stems and a batch's stamp (batch_stamp,_summary_is_current) are the checked ui.case_paths's, the overrides every job of a session shares (_shared_overrides(), session_overrides()'s fallback) the checked ui.case_paths's too, and whether a SLURM case's outcome is stale the checked slurm.collect's (stale_reason(), and_stale(), which reads it without letting an unreadable record break the page, #2285); every function left here reads a batch's JSON documents (Any,dict[str, Any]), so checking it would leave all but case_files UNCALLEDui.models: imports pydanticui.recordings: imports base64, io, tarfile, urllib.request and zlib, and, inside functions, the converter (autoware_carla_scenario_rerun, a Rust extension), which it holds as a module of any type (Any); the web viewer's selection and the path rules of the recordings and of the web viewer's copy are the checked ui.web_viewer_paths's, and whether a recording is fresh, putting a recording in place and swapping a copy of the web viewer into place (_fresh(),_publish(),_swap_into_place()) the checked ui.recording_files' (#2124), which it re-exports, as is the name of a recording kept in the cache (cached_recording_name()), which it imports:_cached_recording()stays, joining that name to this module's_cache_root()so that a patch ofrecordings._cache_rootreaches itui.results: imports ui.cases, which is not checked and has no model (and whose readers of a JSON document, a verdict, a duration and a run's overrides it re-exports), and openodd (a Rust extension) inside CaseResult.coverage_viewable; reads JSON documents and trajectory log records of any type (Any); the comparison of sessions (fed the buckets and key read here), a case's outcome and scenario, and its key by overrides are the checked ui.comparison's, only the key's fallback (the JSON of the ODD values the case was drawn at) is built here, and a session's job indices and a case'sscenario=override are the checked ui.case_paths's; what is left reads coverage documents and trajectory records of any type, and its typed logic is already checked in ui.comparison, ui.drive_metrics and ui.case_pathsui.scanner: imports the registry (omegaconf, importlib), which is not checked, ui.models (pydantic), whose model declares none of the view models it builds, and ui.cases, which is not checked and has no model (a verdict, a duration and a run's overrides are read with its readers); reads JSON documents of any type (Any); a run's result suffix is the checked run_outputs's (RESULT_SUFFIX, #1704)ui.sweep_resolver: imports hydra, the registry (omegaconf, importlib) and ui.scanner (the built-in config directory,BUILTIN_CONF_DIR), which is not checked, and, inside resolve_sweep(), authoring.hydra_config (pydantic, through authoring.models), odd.controls and sweeper.expand (omegaconf), which are not checked; importsOddDrawsfrom odd.sampler (underTYPE_CHECKING), which the odd model does not declareutils.config: reads the fields of a dataclass it is handed as a class (type, dataclasses.fields) and, in null_as_default(), their annotations, takes the options asMapping[str, Any]and, in checked_str_list(), checked_section() and checked_mapping_list(), a list setting or a nested options section of any type (value: Any), which Codon cannot expressweather_rules: imports openodd (a Rust extension), reads rule files andenvironment.rulesasMapping[str, Any], and imports odd.backend inside a function, which Codon cannot express; weather_rules.codon models what visibility and actions.environment use of it; the quantities, the weather inputs,rule_values()and whichvehiclesentry a vehicle follows (vehicle_entry()) are the checked weather_inputs's (#1732)web_guard: an ASGI middleware: takes ASGI scopes and messages asMutableMapping[str, Any]and calls the application andsendas async callables, which Codon cannot express; imports fastapiwosac: re-exports wosac.convert and wosac.pack, which are not checked, and the generated protobuf modules (wosac._proto), which import google.protobufwosac.__main__: the scenario-wosac command: imports argparse, yaml and orjson, and wosac.convert and wosac.pack, which are not checked; the logs it exports and the two of one stem it refuses to write into one directory are the checked t4_scene.grid's (trajectory_logs(),stem_collision())wosac._proto.camera_tokens_pb2: imports google.protobufwosac._proto.compressed_lidar_pb2: imports google.protobufwosac._proto.dataset_pb2: imports google.protobufwosac._proto.keypoint_pb2: imports google.protobufwosac._proto.label_pb2: imports google.protobufwosac._proto.map_pb2: imports google.protobufwosac._proto.scenario_pb2: imports google.protobufwosac._proto.sim_agents_submission_pb2: imports google.protobufwosac._proto.vector_pb2: imports google.protobufwosac.convert: reads trajectory logs with t4_scene.convert, which is not checked (JSON-lines records of mixed value types,dict[str, Any], which Codon cannot express), with its presence check and box poses (_Log.present(),box_poses()), writes them with orjson and builds protobuf messages (wosac._proto); the suffixes it writes are the checked run_outputs's, and the blueprints of a cyclist the checked constants's (CARLA_BICYCLE_BLUEPRINTS), and the ids and counts it takes the checked wosac.ids's (object_id_of(),scenario_id_of(),checked_count())wosac.pack: imports tarfile and google.protobuf (json_format,message.DecodeError), and builds protobuf messages (wosac._proto); takes the submission's metadata asMapping[str, Any], which Codon cannot express; the counts it takes are the checked wosac.ids's (checked_count())
The scenario check is not limited by this: it compiles every scenario against
the model. The model does not declare the whole public API: what
test_typecheck_model.py guarantees is that every public name of a modelled
module, and every public method, property and class attribute of a class the
model declares, is either declared or listed with the reason it is left out
in typecheck/not_modelled.py (NOT_MODELLED, NOT_MODELLED_MEMBERS), and
the check refuses a use of a listed one with that reason (#249, #265).
Attributes a class only annotates (set on the instance) are outside that
guarantee.
What is compiled¶
Codon checks a function only when something calls it, so the check appends a
function to each checked module that calls every function and every
method of every class (constructors, static and class methods,
properties and their setters, dunder methods, protected _ ones and those
of _ classes included) with a value of
each parameter's annotated type:
def _acs_library_check():
normalize_angle_deg(_acs_value(float))
Vector3(_acs_value(float), _acs_value(float), _acs_value(float))
_acs_value(Vector3).dot(_acs_value(Vector3))
_acs_value(T) is a T Codon cannot tell from a real one; nothing compiled
is ever run. A parameter annotated with a union (Lanelet2Pose |
OpenDrivePose, Union[BaseAction, Callable[[carla.World], None]], or an
Optional of one, Optional[Union[EntityRole, str]]) is
called once with each member, so the body is checked for every type it
accepts. A parameter with no annotation, or one Codon cannot
express (Any, object, Callable[..., R], Callable[[Any], Any], ...),
gives the check nothing to call with and is reported as a problem of the
module, at its line: annotate it with the type its callers pass. A
container of a Callable (List[Callable[[carla.World], None]]), which a
parameter annotation cannot spell, can be named with a module-level
TypeAlias (_TickCallbacks in scenario_base.py): the check passes the
alias through and Codon resolves it. The
exceptions are other: object in __eq__ and __ne__, which Python
requires, called with the class itself, and a parameter Codon cannot express
that has a default: a function whose every such parameter has one is called
only leaving the defaults out (Defaults are compiled too, below), so its body
is compiled with them (#199). A Callable whose argument and result
types Codon can express (Callable[[carla.World], Optional[float]], a
dataclass field too, or an Optional of one) is called with a value of
Codon's Callable type, which a function of that signature converts to. One
with a union among its argument types (Callable[[Optional[Union[EntityRole,
str]]], CarlaWorldPose], the reference lookup of trajectory.resolve) is no
type Codon 0.19 can hold: it crashes on it ("union already sealed"). Such a
parameter is called with _acs_Returning[R, N] (_tc.codon), a callable
that takes N arguments (as many as the Callable does) of any type and
returns an R, so the body is compiled with the number of arguments it passes
checked, their types unchecked, and with what it gets back checked.
A *args: T is given one T (run_uv(cwd, *args: str, ...) is called
with one str), so the body is compiled for it; one with no annotation, or
of object or Any (__exit__(self, *args: object)), is given nothing.
A method with keyword-only parameters or **kwargs is called through its
class (TrafficBackend.start(_acs_value(TrafficBackend), ...,
skip_actor_ids=...)): Codon 0.19 calls a method a subclass may override
through a dispatch thunk, which takes neither a keyword argument nor
**kwargs, and the call through the class checks the same body.
Defaults are compiled too (#183). The calls above pass every
argument, so no default was ever compiled: find_nearest_traffic_light(...,
max_distance: float = "far") passed the check. A function or method with a
defaulted parameter is now called once more, with only the arguments that
have no default, each of its first type:
find_nearest_traffic_light(_acs_value(carla.World), _acs_value(Lanelet2Pose))
AutowareBridge.configure(_acs_value(AutowareBridge), _acs_value(BridgePose), _acs_value(BridgePose))
A method is called through its class (AutowareBridge.configure(...), not
_acs_value(AutowareBridge).configure(...)): a dispatch thunk fills in no
default, and through it AutowareBridge.configure() and
FakeAutowareBridge.configure() do not compile. A *args: T is still given
one T (run_uv(cwd, *args: str, timeout: int, env=None) with one str
and timeout), so the positional parameters before it are passed, defaulted
or not: without it run_uv() does not compile. A function with no default gets no
such call. An error in a default is reported at the call, which names it
(<library check: find_nearest_traffic_light(world: carla.World, location:
Lanelet2Pose), with the default ofmax_distance>: error: 'str' does not
match expected type 'float').
That was 169 calls more, beside the 1525 others (1694 in all), when it came in. Measured before any fix, 13 of them failed, and all 13 are fixed by the transform, with no change to a module:
- 11 had a
()default on aSequence[X]orCollection[X]parameter (RoutingAction.__init__'swaypoints,set_mission()/route_to()ofBridgeHandshakeandBridgedEgoEntity,EgoVehicle.route_to(), theskip_actor_idsof the three traffic backends'start(),utils.vehicles.hold_vehicles_still()andscenario_runner._hold_vehicles_still()). The parameter's annotation is dropped, so the()made it Codon's empty tuple ("'Tuple' does not match expected type 'List[Lanelet2Pose]'"). The transform makes such a defaultList[X](), as it does a dataclass'sfield(default=()). - 2 had a
Nonedefault on anOptionalparameter whose annotation the transform drops:EgoConfig.__init__'swaypoint_poses: Optional[Sequence[Lanelet2Pose]], and theCallableparameters oftrajectory.resolve.resolve_trajectory().Nonealone is Codon'sNoneType, which has no elements and no call, and a body that tests it forNonefirst is compiled with it all the same. The transform makes such a default theNoneof theOptionalthe library check calls with:Optional[List[Lanelet2Pose]](),Optional[Callable[[float, float, float], Optional[float]]](), and for aCallablewith a union among its argumentsOptional[_acs_Returning[CarlaWorldPose, 1]]()(the transform's prelude imports_acs_Returning; the check's own function does not import it again, which Codon 0.19 cannot type: "cannot typecheck").
So are the field defaults of a dataclass (#188). The check drops
@dataclass, and Codon gives the class a constructor that takes each field,
with its default; every constructor call above passes every field, so no
field default was compiled: stop_timeout_s: float = "far" on
CommandAutowareLauncher passed the check. A dataclass with a defaulted
field is now constructed once more, with only the fields that have no
default, each of its first type:
A field without a default after one with (a field(kw_only=True)) is passed
by keyword. A field with no default of its own (field(), or
field(init=False), which the constructor leaves out anyway) is not one
with a default, and neither is a field "defaulted" to OmegaConf's
mandatory-value marker (spawn_lanelet_id: int = MISSING, the string
"???"): the marker is no value of the field but a placeholder for one the
config must supply, so the call passes the field, and the marker is not
compiled against the field's type. Nothing enforces that in Python: reading
the field from Hydra's config raises MissingMandatoryValue, but
OmegaConf.to_container() returns "???" for it, and a dataclass built
without it (NpcVehicleConfig()) holds the string, which the check does not
see. A field the call leaves out because its annotation names Any (A
dataclass field may hold a union or a free-form mapping, below) is left out
of this call too. An error is
reported at the call, which names the defaults it leaves out
(<library check: CommandAutowareLauncher.__init__(argv: Sequence[str]),
with the default ofenv,log_dir,stop_timeout_s>: error: 'str' does
not match expected type 'float').
That is 96 calls more, beside the 1694 others (1790 in all, 265 of them
leaving defaults out), at the time of writing. Measured before any fix, 5
of the 96 failed, at the first error of each. Those 5 classes have 10
defaults Codon refused, of four causes; all are fixed in the check, with no
change to a module and nothing UNCALLED (9 by the transform, whose output
changes at those 9 defaults alone, and 1 by the call):
- 3 had a tuple display default on a
Sequence[X]ortuple[X, ...]field (DockerAutowareConfig.colcon_args,DriverClientConfig.camerasandmap_formats). The field is declared as aList[X](thetypingshim'sSequenceisList), and the display is a fixed-length Codon tuple ("'Tuple[str,str,str]' does not match expected type 'List[str]'"). The transform makes itList[X]([...]), a list of the field's element type, replacing only the brackets, so the elements are rewritten as anywhere else. - 4 had a plain
()default on such a field (DockerAutowareConfig.ros_domain_idsanddocker_args,DriverClientConfig.lidars,PlannedRoute.via), which only afield(default=())of atuple[X, ...]was typed for. The transform makes itList[X](), written directly or throughfield(default=()), for aSequence[X]as for atuple[X, ...]. - 2 had
field(default_factory=dict)on aMapping[str, str]field (envofDockerAutowareConfigand ofCommandAutowareLauncher): the factory is typed only for adict[K, V]field, so it was a bare{}, a dict ofNone("'Dict[NoneType,NoneType]' does not match expected type 'Dict[str,str]'"). The transform makes alist,dictorsetfactory of an abstract collection the concrete one thetypingshim maps it onto, of the field's types (Dict[str, str]()). - 1 was the OmegaConf marker on an
intfield (NpcVehicleConfig.spawn_lanelet_id: int = MISSING, "'str' does not match expected type 'int'"), which the call now passes, as above.
So are class-level defaults (#190). On a class with a constructor, its
own or a base's, Codon makes a class-level x: T = v a field and drops v:
no call compiles it, and BackendDriven._tm_port: int = "x",
BaseScenario.STABILIZE_TICKS: int = "a" and AutowareEgoEntity.stack_name:
str = 1 passed the check. The check's function now assigns each such default
to a local of its type, which compiles the value against the annotation:
Each local is named after its line in the function, and an error is reported
at it, labelled with the member (<library check: class-level default of
BackendDriven._tm_port>: error: 'int' does not match expected type 'str';
Codon names the two types the other way round in an annotated assignment). A
bool default goes through _expect_bool() (_tc.codon): Codon converts
any value assigned to a bool local with its __bool__, so ready: bool =
"x" would compile, and a call converts nothing. A default written without
an annotation, overriding one a base declares (REISSUES_BY_DEFAULT = True
on TrajectoryRun (RoutingAction's engine), whose base declares REISSUES_BY_DEFAULT:
ClassVar[bool] = False), is checked against the base's annotation, a
ClassVar[T]'s T; the base is found by name among the classes of its
module and the other checked modules, depth first. Left out:
- the defaults of a dataclass, compiled by the constructor call that leaves them out (above), and the members of an enum, which are its values;
- a
ClassVar, which Codon keeps as a class variable, with its value; - every default of a class that has no
__init__, its own or a base's, and that its module's top level constructs with no argument (_CACHE = _Cache()inmeasures): Codon's generated constructor takes each field with its default, and that call compiles them already (an error is reported at the call); - a default without an annotation that no base declares (a base outside the
checked modules), a dunder (
__slots__), and one over a bareClassVar; - an annotation Codon cannot express or naming a
TypeVar, and a value or annotation naming anything else the class body binds (TWICE: float = LIMIT * 2; a member unpacked,LOW, HIGH = 0.0, 1.0; a method; a nested class), which at the function's scope names something else, or nothing.
That is 22 defaults, at the time of writing: 20 annotated ones, in 10
classes (BaseScenario, the five ego entities, TurnAction,
CollisionCondition, DrivenEntity and BackendDriven), and the 2
unannotated overrides of BaseAction.REISSUES_BY_DEFAULT
(TrajectoryRun, _BackendAction). Of the 34 non-ClassVar
class-level defaults outside dataclasses and enums, the other 14 are those
of measures' _Cache, _Lines and _FrameMemo, which the module
constructs; the 7 ClassVars are left out. None failed when it came in.
So are the values a function returns as a tuple[X, ...] (#191). Codon's
tuples have a fixed length, and the transform dropped such a return
annotation, so nothing checked what the function returns:
signals.registry.registered_signal_controllers() returning 1 passed the
check. A class-level tuple[X, ...] is a List[X], and tuple(x) of a list
is one, so the transform now writes the return annotation as List[X]
(Optional[List[X]], dict[str, List[X]] inside another type), and the
error is reported at the return. It stays dropped when it names an abstract
collection (a parameter's is dropped, so the function may return it as it
got it), a class defined further down the module or a TypeVar, and so does
a Union, Any or Callable return (a Callable return kept fails for a
nested function). That is 10 return annotations, in 6 modules, at the time
of writing: actions.routing, maps.resolver, route.model,
signals.registry, 3 in ros2.config and 3 in signals.controller
(one of them dict[str, tuple[int, ...]]). None failed when it came in.
Protected functions and methods are called too (#175). A _ method is
often a hook a subclass or another module uses: six example scenarios call
BaseScenario._setup_ego_spawn(), and the model's BaseScenario is what a
scenario compiles against, so before the driver called it, nothing compiled
its real body, and an error injected into it passed the check. The check now
calls every _ function of a checked module and every _ method of its
public classes, as it calls a public one: about 340 of them, beside about
920 public ones. Measured before any fix, 25 of them failed: 15 had a
parameter the check cannot call with, 8 a bare np.ndarray parameter, and
_setup_ego_spawn() and scenario_runner._build_condition_status() did not
compile. 12 were fixed in the module (shape-typed arrays, an annotated
local, an lxml element type for coordinate.transform._cubic(), and the
driver taking Optional[Union[A, B]]), and 13 are UNCALLED, each with the
reason, most of them because they take Any and only an UNCALLED function
calls them.
The methods of _ classes are called too (#182). A _ class is a
module's own helper, and was left to be compiled through what uses it, but
what uses it does not always compile it. Of the 42 methods of the 20 _
classes of the checked modules, an error injected into each found 4 that
nothing compiled: _Never.__init__ and _BackendAction.__init__
(actions.background_traffic), reached only from constructors that are
UNCALLED; _BackendAction.set_traffic_backend(), which
BaseScenario._hand_backend_to() calls behind a hasattr() on a
BaseAction, which Codon decides statically as false; and
route.search._Search.__post_init__(), which Codon's constructor does not
call. The check now calls them as it calls a public class's: 53 calls
(for the 48 methods of the 21 _ classes there are since #180), beside
about 1470 others, none with a parameter the check cannot call with.
Measured before any fix, 3 of the first 47 failed, and all 3 were fixed in the module:
_ScenarioClock declares the _world and _simulated_start it assigns on
self, and _BackendAction.__init__ branches into a BaseCondition local
instead of choosing between until and _Never() with a conditional
expression, which Codon 0.19 cannot type. Each of the 4 now reports an
injected error. Two things are left out:
- a module's dunders (PEP 562's
__getattr__), which are Python's hooks, not the module's code; a class's (__eq__,__enter__) are called, as before; - what
UNCALLEDnames, as for a public function.
A protected member that calls an UNCALLED one calls it in the else
branch of if TYPE_CHECKING:, as a public one does (see A call into a
module Codon cannot compile, from a function it must below).
_setup_ego_spawn() needs no such branch: the register_route_to_goal() it
ends with is checked (#174).
A constructor is compiled with the defaults of what Codon cannot
express (#199). BaseScenario.__init__ takes the ego's class (ego_type:
type[EgoVehicle] | None = None), and Codon cannot hold a class as a value, so
the whole constructor was UNCALLED, though only one line stores it: its
other ~25 lines were compiled nowhere (the example scenarios compile against
the model's BaseScenario), and an injected self._tm_port = "x" passed
every run. A function some of whose parameters Codon cannot express, each
with a default, is now called once, leaving out every defaulted argument,
instead of being a problem:
and the one line that stores the class is under if not TYPE_CHECKING:,
which Python runs and Codon does not (ego_type stays declared at class
level, so ego_requires_goal still reads it). The error is reported at its
line, through the call that leaves the defaults out (<library check:
BaseScenario.__init__(ego_config: EgoConfig), with the default ofspawn_pose,ground_projection,random_seed,ego_type,ego_entity>). A parameter Codon cannot express with no default is a
problem as before, and so is a defaulted one that call would still pass (one
before a *args: T, which is given one T). ConditionStatus.__init__
(its details: dict[str, Any]) and TrafficConfig.__init__ (a backend's
options, dict[str, Any]) are compiled the same way, with no UNCALLED
entry, so a wrong default of one of their other parameters is reported
(#217).
Property setters are called too (#200). Codon compiles an @x.setter
only where something assigns the property, so an error injected into the
setter of ScenarioRunner.xodr_path passed a run of scenario_runner alone:
only the full run found it, through ScenarioQueue.start(). The check now
assigns each property that has a setter, once with each member of the
annotation of the value it takes:
_acs_value(BaseScenario).goal_pose = _acs_value(Optional[Lanelet2Pose])
_acs_value(ScenarioRunner).xodr_path = _acs_value(Optional[Path])
An error is reported at its line, through <library check:
ScenarioRunner.xodr_path (setter)>. That is 3 assignments, for the 3
setters of 2 modules (scenario_base and scenario_runner), at the time of
writing; none failed when it came in. A deleter is not called. The scenario
check appends the same calls to a scenario's own modules (#197), so a setter
of a scenario nothing assigns is compiled too.
Checked modules and the model¶
The checked modules are compiled next to the same model the scenario check uses. Every import of the framework in a checked module is pointed at what stands in for it in the check's workspace:
- another checked module: its own (rewritten) source;
- any other module: its model, the
codon/autoware_carla_scenario/module of the public package it belongs to (coordinate.posesis modelled bycoordinate.codon), or, for a module of a package that is itself checked, a model of that module alone (utils/traffic_light.codon,autoware_bridge/grpc_server.codon,driver/egodriver_client.codon,maps/cache.codon,autoware_stack/devcontainer.codon,ros2/cyclone.codon,pedestrian/soma.codon,scaffold/generator.codon,commonroad/session.codon,commonroad/worker.codon,hils/config.codon), or, for a module of the top-level package, a model of that module (trajectory_recorder.codon, forscenario_runner, theTrajectoryRecorderit starts, records and closes, whose JSON lines hold values of mixed types;environment.codon, forscenario_runner,capture()and the id of theRunEnvironmentit returns;weather_rules.codon, forvisibilityandactions.environment, theweather_range(),weather_at(),vehicle_at(),vehicle_rules_in_use()andcustom_rules_in_use()they read the weather rules through). A module with no model (utils.config) cannot be imported by a checked module until it is checked itself.
A module that is excluded for most of what it does can have the part a
checked module needs split into a checked module of its own, which the
excluded one re-exports, instead of a model: server imports
installed_executable() from carla_install_paths, which holds the path
logic of the installer carla_install (tqdm, urllib.request, tarfile) and is
compiled from its source (#1635). It holds the decision whether an install is
current and the headers of the conditional request too (_current_of(),
_conditions_of(), #1671), over the stamp as a dict[str, Optional[str]]:
carla_install reads the stamp's JSON and converts it at that boundary, so
that a value of another type, in a stamp edited by hand, decides as it did. A
constant a test replaces in the excluded module (_UNPACKED_PER_PACKED) is
passed to the checked function (_room_needed()) rather than read there.
The opening of the lock file is there too (_open_lock() and the errnos of a
lock file that cannot be written, _UNWRITABLE_ERRNOS, #2144): it opens the
file with lock.open(mode), the same file object as open(lock, mode) in
Python, since Codon's open() takes no Path and gives no IO; a test that
makes the lock read-only patches Path.open, as carla_install no longer
calls open(). Holding the lock (_locked(), a contextmanager) stays in
carla_install. scaffold.placement is one too: how scenario-new puts a
rendered package in place (_move_into_place(), _occupied()) and the error
of a target that exists (PackageExistsError, a FileExistsError whose
root is declared at class level), split out of scaffold.generator
(argparse, jinja2), which re-exports them (#2144). A test that replaces
generator._occupied to reach the rename's own refusal replaces
placement._occupied too, which the move reads. slurm.start_lock, split
out of slurm.worker for the files of the start lock (#2116), holds the
takeover of a stale lock too, one waiter at a time (_take_over() and the
age of a dead waiter's mutex, _TAKEOVER_STALE_AFTER, #2144); the worker
re-exports them and keeps taking the lock (_exclusive_file(), a
contextmanager). A test that shortens that age or watches the takeover's
_create_lock() and _remove_stale() replaces start_lock's, which
_take_over() reads (and, for _create_lock(), the worker's too, which
_exclusive_file() reads).
Each checked module gets the __file__ Python sets on it (its source's path,
a str), which Codon 0.19 does not, bound on the last line of the prelude so
it exists before any code of the module and no line moves (#515), so
Path(__file__).resolve().parent / "data" and
os.path.dirname(os.path.abspath(__file__)) compile at module level and in a
function: autoware_stack.bridge_node finds the installed
autoware_bridge package with from ..autoware_bridge import __file__.
Codon has no module object, so from .. import autoware_bridge and
autoware_bridge.__file__ do not compile.
A constant imported from another checked module is read through that
module (#1693): Codon 0.19 can report a constant imported with from M import
NAME and read at module level as not defined, depending on what precedes it
in M (heavy_metrics reading metrics.CRIME_TTR_S). So, on the import's
own line, from .metrics import CRIME_TTR_S, ego_progress becomes
from _acs_lib.autoware_carla_scenario__metrics import ego_progress; import _acs_lib.autoware_carla_scenario__metrics as _acs_const_12; CRIME_TTR_S = _acs_const_12.CRIME_TTR_S
(from M import X as NAME binds NAME). A constant is an UPPER_CASE name of
more than one letter (not a T TypeVar) that the imported module binds with a
plain assignment at module level (NAME = ... or NAME: type = ..., not
NAME: TypeAlias = ...), or re-exports from another checked module that has
it as a constant (from .metrics import CRIME_TTR_S as CRIME_TTR_S);
module_constants() in typecheck/transform.py finds them. Every other name,
and an import through the model (MODELLED_IMPORTS), stays a from M import.
An UPPER_CASE assignment that holds a type (JSON = dict[str, Any]) would
read as a variable, not a type, through the module: annotate it TypeAlias.
Two checked modules that import each other can each need the other's types
the moment Codon reads it: BaseScenario declares _measures: dict[str,
Measure] at class level, and measures declares the running scenario
(_MEASURED: dict[str, Optional[BaseScenario]]) as a global, so neither can
be read first. MODELLED_IMPORTS in typecheck/library.py breaks such a
cycle: the module it names (measures, odd.route_plan) imports the other
(scenario_base) through its model, as it did before that one was checked.
Its types are the model's, which never meet the checked module's.
A cycle that only a definition in the wrong module made is better broken by
moving the definition: EgoConfig is defined in entity.ego, next to the
EgoVehicle.spawn() that takes it, and scenario_base re-exports it, so
entity.ego imports nothing from scenario_base and needs no entry, and the
runner's ego.spawn(world, scenario.ego_config) is checked as it runs.
The egos derived from EgoVehicle (entity.bridged_ego,
entity.autoware_entity) do not override EgoVehicle.spawn(): Codon 0.19
cannot call super().spawn() through an intermediate class ("'EgoVehicle'
does not match expected type" in its dispatch thunk). Through the checked
scenario_base, a run with an example scenario in it failed in Codon 0.19's
code generation (LLVM: error: value doesn't match function result type
'{ ptr, ptr }', in opt_ref_new of the model's BaseScenario).
A model module that stands in for unchecked code declares only what the
checked modules and scenarios use of it. coordinate.codon still declares,
of coordinate.projection, the map_origin(), resolve_projector() and
_parse_geo_reference() that coordinate.map_manager and
sweeper.map_loader import, though the library check now compiles all three
from their source. Its MapManager is now the scenario check's
alone: the library check compiles map_manager.py from its source, which
every checked module imports, and nothing of the model takes or returns a
MapManager, so it needs no REPLACED_MODELS entry.
A checked module is compiled under a name of its own
(_acs_lib.autoware_carla_scenario__coordinate__frames), since Codon reads
the workspace's packages only from .codon files and names a file in its
errors by its base name alone; errors are reported at the line of the real
source file. A type a checked module defines is not the type the model
declares under the same name, so a checked module cannot hand its own value to
a model function that wants the model's: check modules bottom-up, before the
modules that use them.
Where the model's functions take the types a checked module defines (the
poses, which to_opendrive takes and returns), the model's own declarations
of them live in a model module of their own (_poses.codon), which
REPLACED_MODELS in typecheck/library.py names with the modules it
models. Once those are all checked, the library check compiles that model
module as a re-export of their definitions, so the model and the checked
modules share one Lanelet2Pose. A name the model declares that none of the
checked modules defines is reported.
Conditions: the checked base and the model¶
conditions.base is checked, so the checked conditions derive from the real
BaseCondition (_acs_lib.autoware_carla_scenario__conditions__base), not
from the model's. The two are different types that never meet:
- the scenario check compiles a scenario against the model only; no
checked module is in its workspace, and the model's
BaseCondition,_expect_conditionand_acs_list(which recognises a condition by the model-only_acs_conditionmarker) are unchanged; - the library check compiles the checked conditions against the checked
base. A checked module that hands a condition to a model function (an
unchecked action) cannot do so until that module is checked too
(
scenario_baseis checked, and importsBaseConditionfromconditions.basefor that);_acs_listgives a list display of checked conditions the type of its first element (_acs_baseleaves a checked condition in a dict display as it is), so a display mixing two checked condition classes waits for the same.
BaseCondition.__init_subclass__, which wraps every subclass's check() to
record its last result for the UI, is in UNCALLED: the check sees each
check() as written.
actions.base is checked the same way: a checked action takes the checked
BaseCondition, so it imports it from conditions.base rather than from the
conditions package, which the model stands in for until it is checked. Its
BaseAction and ActionState are again not the model's, so
conditions.action_state watches checked actions. An action that hands its
base a condition of its own making imports that condition from its checked
module too (SetSpeedAction's default SpeedCondition from
conditions.composition.speed); one whose module is not checked waits for it.
Making a module compile¶
What Codon 0.19 needs that Python does not, beyond the rewrites above:
- Write an iterator class's
__iter__as a generator (#504): Codon has no__next__protocol, sofor k in Countdown(3)over a class whose__iter__returnsselfis refused (expected iterable expression, with a hint).def __iter__(self): while self.n > 0: self.n -= 1; yield self.ncompiles, and is valid Python too. -
A dict display holds one value type (#505):
{"a": 1, "b": "x"}is refused even annotateddict[str, Any](Codon has no usableAnyorUnionvalue). Convert the values tostr, use a dataclass with typed fields, or use one dict per type. The base-class annotation of #489 is for the scenario's own subclasses only. -
Annotate what the check calls: every parameter of a function or method it calls, protected (
_) ones included, with a type Codon can express. Spell a union out (Lanelet2Pose | OpenDrivePose) rather than through an alias (AnyPose); a union withobjectorAnyin it gives nothing to call with. - A default must have the parameter's type (#183): the check compiles
each function once with its defaults. A
()default of a sequence parameter is checked as an emptyList[X], soXmust be a type Codon can express and name there: a class defined further down the module is not (the()is left as it is, and fails). ANonedefault of a droppedOptionalis typed only for a sequence or aCallablethe check can call with;Optional[Callable[..., R]]andOptional[Mapping[K, V]]stay a bareNone, so a body that uses them past anis Nonetest fails. A tuple display default (= ("a", "b")) of a sequence parameter is checked as a list of its elements,List[X](["a", "b"]), as a dataclass field's is (#188); a fixed-length Codon tuple is noList[X]. An empty[]/{}default of alist[X]/dict[K, V]parameter (or[]of a sequence) is checked as an empty one of that type; Codon would make it a list ofNone. ANonedefault is not checked against its annotation: Codon makesspeed: float = NoneanOptional[float], which passes (mypy, in pre-commit, reports it). - Declare attributes on any class in a hierarchy (exceptions included)
and on a class with
__slots__, at class level (_value: str). - A class-level default is compiled as written (#190): the library check
assigns it to a local of its annotation in its own function, which the
transform does not rewrite. So a default the transform would rewrite in a
dataclass (a tuple display or
()of atuple[X, ...]member,frozenset()) is reported: assign such a value in__init__instead. A default naming another member of the class (TWICE: float = LIMIT * 2), or anything else its body binds, is not checked, as at the function's scope the name is not the member; neither is aNonedefault, which Codon makes anOptionalof any annotation (x: float = Nonecompiles). An override without an annotation (REISSUES_BY_DEFAULT = True) is checked against its base's annotation only when the base is a class of its module or of a checked one. hasattr()instead ofgetattr(x, name, default): Codon decideshasattrwhen it compiles, so the branch for a type without the attribute is never compiled. The check rewritesgetattr(x, "name", default)so itself where the name is a literal andxa name or an attribute chain (#590, "Python the check rewrites for Codon").- An exception's
__init__passes one message string tosuper().__init__. An exception class with nothing in its body but a docstring needs that__init__written out (def __init__(self, message: str) -> None:). d.get(key, default)takes a default of the value's type, as Codon'sdict.getdoes;d.get(key)is anOptionalof it, as in Python (the model's_tc.codondeclares it). So doesd.pop(key, default)(#590, Codon'spoptakes the key only), andd.pop(key, None)is anOptionalof the value's type.dict.fromkeys(xs)of one argument (#625) is a dict whose values are allNone, as in Python (Dict[K, Optional[NoneType]], so it holds onlyNone: give the value in the call,dict.fromkeys(xs, 0), for a dict to fill with other values). Likewiseif not x: continueinstead of(x or {}),s.startswith(a) or s.startswith(b)instead ofs.startswith((a, b)), andif x is None: return None/return x.attrinstead ofNone if x is None else x.attr.- A dict of values of different types (
{"at": None, "around": None}read back as a tuple and a list) becomes a small class with one declared attribute per key. A cache that told "not read yet" from a read that found nothing with anobject()sentinel (cache.get(key, _UNREAD)) keeps the names it has read in aset[str]beside them:odd.probes._Frameholds the frame's waypoint, weather, lanelet tags, ... eachOptional, andread, so a read that failed is still kept asNonefor the frame, as before. - Standard modules Codon does not have (
inspect,concurrent.futures, ...) are not available to a checked module; the shims ofcodon/are. They are these (test_shim_listkeeps the list and the files ofcodon/in step):
__future__, abc, array, atexit, contextlib, dataclasses, enum, errno, fcntl, fnmatch, getpass, hashlib,
importlib.metadata, ipaddress, json, keyword, logging, os, os.path, pathlib, platform, queue,
shlex, shutil, signal, socket, subprocess, sys, tempfile, threading,
typing, typing_extensions, urllib.parse, uuid and xml.etree.ElementTree
tempfile declares mkstemp(suffix=None, prefix=None, dir=None,
text=False), mkdtemp(suffix=None, prefix=None, dir=None) and
TemporaryDirectory(suffix=None, prefix=None, dir=None,
ignore_cleanup_errors=False) only, with CPython 3.10's signatures (a str or an
os.PathLike directory; bytes affixes are not declared). The check only
compiles it, so its body is a stub that makes no file (it is typed to give
(int, str), the descriptor and the name; mkdtemp() gives the str
name, and a TemporaryDirectory its name). Codon binds a with
statement's target to the manager itself, not to what __enter__()
returns, so with TemporaryDirectory() as d makes d the manager in the
check (CPython's is the name), and a use of it as a str or a path fails
the check: hold the manager and read its name (scratch =
TemporaryDirectory(); with scratch: ... scratch.name, as t4_scene.osm
does). NamedTemporaryFile(), gettempdir() and the
rest are not declared and fail the check. Two more modules are not
shims but extensions of Codon's own collections and itertools, in
codon/_stdlib/ (#422): they add
collections.OrderedDict and itertools.pairwise, as listed under
"Standard library names Codon lacks".
array declares what metric_frames keeps a recorded frame's motion in
(#2338): array("d") (an array of doubles, which reads back the floats it
was given bit for bit; another typecode raises ValueError, and no
initializer is declared), its typecode, len(), indexing by an int,
iteration, append(x) and extend(iterable). Slicing, the other
typecodes, the arithmetic and the byte methods are not declared.
xml.etree.ElementTree declares what traffic.sumo.network reads a
network's crossings with: iterparse(source) of a file name (no file
object, no events or parser), which gives no element in the check, of
(event, element) pairs, and an element's tag, get(key, default=None)
(a str or None default, an Optional[str]) and clear(); and what
t4_scene.osm rewrites the OSM file Lanelet2 wrote with:
fromstring(text), an element's attrib (a Dict[str, str]),
find(path) (an Optional[Element]), findall(path) and
set(key, value) (by position only, as in CPython),
SubElement(parent, tag, attrib={}, **extra) (the three by position only,
as CPython's C implementation takes them; str attribute values) and
tostring(element, encoding=None, method=None, *, xml_declaration=None).
The check reads bytes as str, so fromstring() takes and tostring()
gives a str; a path is any str (ElementPath's syntax is not checked).
parse(), ParseError, an element's text, its children and iteration,
and the rest are not declared.
ipaddress declares what serve_bind.is_loopback() reads a host with:
ip_address(address) of a str (no int or bytes), giving an
IPv4Address or an IPv6Address as a Union Codon decides isinstance
on when the program runs, and of either its is_loopback, and of an
IPv6Address its ipv4_mapped (an Optional[IPv4Address]). Codon reads
no attribute of one member off the union, so narrow it with isinstance
and assign it to a local annotated with the member (v6:
ipaddress.IPv6Address = address), which Python reads as a plain
assignment; an attribute no member has, read off the union, is not
refused. ip_network(), ip_interface(), the other is_*
properties and the rest are not declared.
Each shim takes the
arguments CPython takes, so the check accepts what CPython accepts and
refuses the rest; where Python versions differ, it takes what Python 3.10,
the oldest the package supports, takes. pathlib declares Path(*parts),
/, str(), ==, hash(), expanduser(), resolve() and
Path.home() and Path.cwd(); like Python's, it builds and joins a Path with a str or
an os.PathLike only (a Path, or what shutil.copy2() returns for one;
an Optional of one is unwrapped, as Codon unwraps one passed anywhere). It declares the file-system queries too: exists(),
is_file(), is_dir(), glob(pattern), rglob(pattern) and
absolute(), with Python
3.10's signatures (3.12 and 3.13 added the keyword-only follow_symlinks,
case_sensitive and recurse_symlinks, which are refused), and name,
suffix, parent, parents (indexed, iterated, len() and in; not
sliced), stem, suffixes, parts (indexed, sliced, iterated, len() and in),
root, drive, anchor, as_uri(), with_name(name),
with_suffix(suffix) and with_stem(stem) (of a str), joinpath(*other),
as_posix(), relative_to(other) (one path, by position:
Python 3.10 also joins more parts, but 3.14 refuses them, so the check
does too; 3.12's walk_up is refused), is_relative_to(other) (one path,
by position, likewise), is_absolute() and the ordering of two Paths, so
sorted() takes them. parts is not a tuple here: parts == ("a", "b"),
a, *rest = p.parts and tuple(p.parts) are not supported (use list(p.parts)
or index it). A Path member, or an os, os.path or shutil name, that
CPython lacks too is refused as a typo, with the closest declared name. Nothing
touches the file system: a query is False and a glob, or iterdir(),
yields nothing; samefile(other_path) and match(path_pattern) are False.
is_symlink() takes no argument, as in every supported Python. Of what
reads, read_text(encoding=None, errors=None) (3.13's newline is
refused) and read_bytes() are declared (autoware_stack.docker reads a
launch file and hashes the package.xml files, coordinate.map_manager
an OpenDRIVE file); both return "", and
read_bytes() returns a str, since the check reads bytes as str
(so it takes path.read_bytes() + "text", which CPython refuses, and
refuses path.read_bytes().decode(), which CPython takes). Of what
writes, mkdir(mode=0o777, parents=False, exist_ok=False) and
write_text(data, encoding=None, errors=None, newline=None) are declared
(maps.opendrive caches what it fetches), and so are unlink(missing_ok=False),
symlink_to(target, target_is_directory=False) (a target that is a str or
an os.PathLike) and open(mode="r", buffering=-1, encoding=None,
errors=None, newline=None), which returns a file object, typing.IO[str]
whatever the mode, since the check reads bytes as str (so IO[bytes]
and IO[str] are one type). typing.IO declares name, mode,
closed, close(), fileno(), flush(), read(), readline(),
seek(offset, whence=0), which returns the new position (ui.runner reads
the tail of a job's log, #1647; it takes whence by keyword, as typing.IO
declares it, where a CPython file object takes it by position only),
write() (of a str, read as bytes too), iteration over the lines and
with. stat() (no argument) returns a result of the seven fields the
framework reads (st_dev, st_ino, st_uid, st_gid, st_size,
st_mtime_ns, st_mtime), all zero (lstat() too), and replace(target) and rename(target) (a
str or an os.PathLike) return the new Path; nothing is read or
renamed. touch(mode=0o666, exist_ok=True), rmdir(),
chmod(mode, *, follow_symlinks=True), write_bytes(data) (a str here, as
bytes are), readlink(), owner() and group() do nothing. The rest
(is_socket(), PurePath, ...) is not declared. os replaces Codon's own (whose
environ has no get() and iterates over pairs), keeping its getenv(),
which Codon's runtime and typesafe_carla's library call: environ is a
mapping of str to str ([key], assignment, del, in, iteration
over the keys, len(), get(key), get(key, default), pop(),
setdefault(), keys(), values(), items() and copy(), a dict, as
CPython's returns), getenv(key, default=None) (a str default gives a
str, None an Optional[str]; a default of another type, which CPython
takes, is refused), fspath(), getpid(), getppid(), getpgid(pid),
getuid(), geteuid(), getgid(), readlink(path, *, dir_fd=None)
(of a str or an os.PathLike, a str; a bytes path is not declared),
kill(pid, signal, /), killpg(pgid, signal, /) (of an int or a
signal.Signals), system(command), what a forking parent and its child
call (#719): fork() (0: nothing is forked), setpgrp(),
setpgid(pid, pgrp, /), waitpid(pid, options, /) (a (pid, status)
tuple of ints, (0, 0)) with WNOHANG, _exit(status) and the
wait-status macros WIFEXITED(), WEXITSTATUS(), WIFSIGNALED() and
WTERMSIG() (of an int, by position or as status),
mkdir() (Codon's had one), getcwd(), listdir(path=".") (empty),
makedirs(name, mode=0o777, exist_ok=False), remove(), unlink(),
rmdir(), rename() and replace() (each with its keyword-only
dir_fd), walk(top, topdown=True, onerror=None, followlinks=False) (a
generator that yields nothing; the callable onerror is not declared), the
separators (sep, pathsep, linesep,
devnull, ...), access(path, mode, *, dir_fd=None, effective_ids=False,
follow_symlinks=True) (False: nothing is asked) and its modes, and
open(path, flags, mode=0o777, *, dir_fd=None) (a descriptor, 0:
nothing is opened) with the flags O_RDONLY, O_WRONLY, O_RDWR,
O_CREAT, O_EXCL, O_TRUNC and O_CLOEXEC (Linux's values) and
O_NOFOLLOW (x86_64's; aarch64's differs, and the check only reads its
type) and fdopen(fd, mode="r", buffering=-1, encoding=None,
errors=None, newline=None, closefd=True), a file object (typing.IO, as
Path.open() gives), and what slurm.manifest creates a file
exclusively with: link(src, dst, *, src_dir_fd=None, dst_dir_fd=None,
follow_symlinks=True) (nothing is linked) and stat(path, *,
dir_fd=None, follow_symlinks=True), and what slurm.start_lock creates
and reads a lock file with: fstat(fd) and write(fd, data, /) (of bytes,
a str in the check, which reads bytes as str, or a memoryview of them;
0: nothing is written), and what tools.file_writes writes a file
with: ftruncate(fd, length, /), fsync(fd) (of an int; the object
with a fileno() CPython also takes is not declared) and chown(path,
uid, gid, *, dir_fd=None, follow_symlinks=True); nothing is cut, synced
or given away, and what commonroad.worker reads the worker's output
with, read(fd, length, /) (bytes, a str in the check; "": nothing is
read), and makes its input non-blocking with, set_blocking(fd, blocking,
/) (of an int and a bool; nothing is changed), and what rerun
builds a FileNotFoundError with, strerror(code, /) (a str, "" in
the check), and what slurm.ports checks its lock directory with,
lstat(path, *, dir_fd=None). memoryview(obj) of bytes is declared as far as
tools.file_writes uses it: a view is true while it is not empty and
sliced into another (view[n:]), which os.write() takes; indexing an
item, tobytes(), release() and the rest are not declared.
The result of stat(), lstat() and fstat() has st_dev, st_ino,
st_mode, st_nlink, st_uid, st_size, st_mtime and st_mtime_ns
(all zero) and nothing else. errno declares EPERM, ENOENT, EACCES, EEXIST,
ENOTDIR, EINVAL, EROFS, ENOLCK, ENOSYS, ENOTEMPTY, ENOTSUP and
EOPNOTSUPP (Linux's values) and no other code. uuid
declares uuid4(), of no argument, and its UUID's hex (a str; the
nil UUID's in the check, as nothing runs), and UUID(hex) of a str, the
first of CPython's parameters; the rest of UUID (int, bytes,
version, ...), uuid1(), uuid3(), uuid5() and the namespaces are
not declared. getpass declares getuser(), of no argument, as in
CPython 3.10, giving a str (empty in the check, as nothing runs), the user
hils.host_devices sets the devices up for when SUDO_USER is unset;
getpass(), the prompt for a password, and GetPassWarning are not
declared. fcntl declares flock(fd, operation, /), of
an int descriptor or a file object (what has fileno()), and LOCK_SH,
LOCK_EX, LOCK_NB and LOCK_UN (Linux's values); nothing is locked.
os.path declares join(),
basename(), dirname(), splitext(), isabs(), exists(), isfile(),
isdir(), expanduser() and abspath(), and getsize(), getmtime(),
getatime(), getctime(), islink(), lexists(), realpath(),
relpath(), expandvars(), split() and normpath(), of a str or a
Path, with CPython's parameter names (nothing is read: a size is 0,
realpath(), relpath() and expandvars() return the path as it is).
shutil declares copy(src, dst, *, follow_symlinks=True), copyfile()
(likewise), copytree(src, dst, symlinks=False, ignore=None,
copy_function=None, ignore_dangling_symlinks=False, dirs_exist_ok=False),
copy2(src, dst, *,
follow_symlinks=True), copymode(src, dst, *, follow_symlinks=True)
(which returns None), move(src, dst), rmtree(path,
ignore_errors=False) and which(cmd, mode=1, path=None) of a str
command; the callables CPython also takes (onerror, copy_function)
are not declared (nor copytree()'s ignore and copy_function).
subprocess declares run(args, *, input=None,
capture_output=False, timeout=None, check=False, ...) and Popen(args,
*, ...) with the keywords of CPython's Popen the framework passes
(stdin, stdout, stderr, cwd, env, text, encoding, errors,
start_new_session, and pass_fds, a list or tuple of descriptors), a standard stream being None, one of the constants,
a file descriptor or a file object (typing.IO, what Path.open()
returns), CompletedProcess(args, returncode, stdout=None,
stderr=None) and its check_returncode(), Popen's poll(), wait(),
communicate(), send_signal() (of an int or a signal.Signals), terminate(), kill(), pid,
returncode and standard streams, PIPE, STDOUT, DEVNULL, and
SubprocessError, CalledProcessError(returncode, cmd, output=None,
stderr=None) and TimeoutExpired(cmd, timeout, output=None,
stderr=None). The command is a list of str, a working directory a
str or an os.PathLike, a stream None, one of the constants, a file
descriptor or a file object, an environment a dict of str, a timeout
a number of seconds or None. What CPython also takes is refused: a str
command, shell=True and the other keywords, os.environ itself as the environment (pass os.environ.copy(), which
is the dict CPython's returns, as authoring.uv_tool does), and
arguments after the command by position (run(args, -1), which run()
hands to Popen as bufsize). Since the check reads bytes as str, a
process's output is a str with or without text=True, and
CompletedProcess[bytes] and Popen[bytes] are [str]; it is None
when it was not captured, as in CPython, so stdout and stderr are an
Optional. A stream iterates over its lines, each a str
(for line in process.stdout:, after assert process.stdout is not
None). A stream's write() takes what a binary stream takes in
CPython, bytes (a str) or a buffer such as an array's data (a Ptr in
Codon's numpy), and since the check
cannot tell a text stream from a binary one, it takes either for a text
one too, where CPython refuses a buffer. importlib.metadata declares
version(distribution_name) and PackageNotFoundError, a ModuleNotFoundError built from the
distribution's name; from importlib import metadata and import
importlib.metadata compile, and nothing else of importlib does (the
package imports its metadata, as Codon 0.19 finds a submodule no other
way, so the check also takes importlib.metadata after a bare import
importlib, which CPython takes only once something has imported the
submodule). signal declares Linux's signal numbers (SIGHUP, SIGINT,
SIGQUIT, SIGKILL, SIGUSR1, SIGUSR2, SIGALRM, SIGTERM,
SIGCHLD) as members of Signals, CPython's IntEnum of them, here a
class of its own: name ("SIGTERM"), value and int() (the number),
==, != and hash() with a Signals or an int, str() and repr()
as Python 3.10 writes them, and Signals(number), which looks a member up.
What takes a signal number (os.kill(), os.killpg(), signal(),
Popen.send_signal()) takes an int or a Signals, so
os.killpg(pgid, signal.SIGTERM) compiles; the rest of an int's
operations (arithmetic, ordering) are not declared for a Signals, which
CPython takes: take int(sig) first. SIG_DFL and SIG_IGN, which
are no ints here, since CPython takes no other int as a handler; and
signal(signalnum, handler), whose handler is one of those two or a
function the check calls as CPython calls a handler, with the signal's
number (an int) and the current frame (None in the check, which has no frame
type: annotate that parameter Any), so a function that cannot take them
is refused where CPython fails only when the signal arrives. What it
returns, the previous handler, and what getsignal(signalnum) returns,
the handler in place, is a value nothing is declared of (a Handlers
member, a function or None in CPython) but that signal() takes it back,
so a handler is restored as CPython restores one (CPython refuses None,
a handler not installed from Python, which the check cannot tell).
alarm(seconds, /) takes an int, and pthread_sigmask(how, mask) one of
SIG_BLOCK, SIG_UNBLOCK and SIG_SETMASK (ints here; CPython's
Sigmasks is an IntEnum of them) and an iterable of signal numbers, and
returns the previous mask, a set of Signals (#719). The rest of the
module is not declared. contextlib
declares suppress(*exceptions) only, a class with __enter__ and
__exit__ that does nothing (the exceptions are not checked); a
contextmanager is not declared, as the scenario check already tells a
user to write a class. json writes generically and reads opaquely
(#557): dumps(obj, **kwargs) and dump(obj, fp, **kwargs) take a value of
any one type (a dict of one value type, a list, a float), with
CPython's options accepted and not checked, but a mapping of mixed value
types is refused at the argument, as for any dict (the Lanelet2 z offset
cache dumps its strings as one object and writes the number in front of
them, so the file is what CPython writes for the whole mapping). Codon
has no Any, so loads(s, **kwargs) and load(fp, **kwargs) return a
JSONValue, which stands for any part of a document: the check holds a
caller to its members only ([key] by str or int, get(), items(),
keys(), values(), iteration, len(), in, str(), int(),
float(), bool()), each returning a JSONValue again, and never what
the document holds; the ValueError or TypeError CPython raises for a
value of the wrong kind is invisible to it. get(key, default) takes a
default of any type and returns a JSONValue (float(doc.get("speed", 5.0))),
a list takes a slice (doc["wps"][1:3]), and load(fp) / dump(obj, fp)
take any file, a builtin open() among them (untyped: Codon 0.19 crashes on a
Union[File, IO[str]] parameter, so json.load(5) passes the check and
CPython refuses it; its encoding=, errors=
and newline=, which Codon's open lacks, are dropped from the checked
copy, #562). A scenario that reads a document converts what it takes
explicitly: speed = float(cfg["v"]) passes, speed: float = cfg["v"] and
cfg["v"] + 1.0 are refused, with a hint to convert the value (speed is
assigned a JSONValue, but is declared float, #564; assigning into a
document is refused as read-only); whether the key exists, or holds a
number, is not checked. isinstance(v, dict) of a JSONValue is
compiled as a run-time test (#563): Codon decides isinstance by the static
type and a JSONValue is none of dict, list, str, so it would not
compile the branch the test guards, and wrong code in it would pass. In a
module that imports json, the python rewrites turn isinstance(x, T) into
isinstance(x, T) or (isinstance(x, JSONValue) and x._acs_isjson()), which
changes nothing for any other type; both branches are then checked against
the members of JSONValue. (Codon does the same for any value whose static type is not the class: after
x = 5, the body of if isinstance(x, str): is not compiled, as dead code
is not; that is Codon's, and only JSONValue, whose type is opaque by
design, is rewritten. The check warns, without failing, when the static
type of the subject is certain from the syntax (an annotated parameter or
local, a local bound once to a literal, an annotated self attribute) and
cannot be the class: isinstance(x, str): x is statically int here, so this
branch is not checked by the static check; an error in it is not reported,
#565.) (A module that never imports json but gets a
JSONValue from another module's function is not rewritten, and a call
that spans lines is not either.) JSONDecodeError is a
ValueError with msg, doc and pos. shlex declares
quote(s) and join(split_command), CPython 3.10's on Codon's re; a
quote() of a false value that is no str (None, which CPython quotes
as '') is refused. It declares split(s, comments=False, posix=True)
too, which takes a str (CPython 3.10 reads standard input for None, and
3.12 refuses it) and gives a list[str]; the check only compiles it, so
its body splits nothing. fnmatch declares fnmatch(name, pat) and
fnmatchcase(name, pat), CPython 3.10's shell-pattern match written
without a regular expression (CPython's translation uses a lookahead,
which Codon's re has no syntax for); fnmatch() takes a str or an
os.PathLike for either, read through os.fspath() as CPython's
normcase() reads it on POSIX, and fnmatchcase() two strs, as
CPython's does. filter() and translate() are not declared (#728).
keyword declares kwlist, CPython's hard keywords (the same on every
supported Python, generated from keyword.kwlist), and iskeyword(s),
which takes one value by position, as CPython's
frozenset(kwlist).__contains__ does, and is False for any value that is
no str; softkwlist and issoftkeyword(), which differ between
versions, are not declared (#1654).
socket declares create_connection(address, timeout=None),
gethostname(), AF_INET, AF_INET6, SOCK_STREAM and
socket(family=AF_INET, type=SOCK_STREAM, proto=0), whose socket has
bind(address, /) and getsockname() of a (host, port) pair (an AF_INET6
socket's bind takes the same pair), setsockopt(level, optname, value, /)
of an int or bytes value, with the options SOL_SOCKET, SO_REUSEADDR,
IPPROTO_IPV6 and IPV6_V6ONLY (Linux's values), close() and with;
nothing connects, binds or sets an option, and fileno and the rest of the module are not
declared (#1498). sys replaces Codon's, keeping all of it (argv,
the standard streams, exit(), maxsize), and adds platform
("linux"), executable (a str) and a standard stream's buffer
(the stream itself: the check reads bytes as str). atexit
declares register(func, *args, **kwargs), which takes the function by
position only and returns it, and which calls it with the arguments it is
given, so a function that cannot take them is refused where CPython
reports it only at exit, and unregister(func), of one value by position.
So do CPython's undocumented _clear() and _run_exitfuncs(), of no
argument, which sweeper.isolation calls in a forked case to drop the
handlers it inherited from the sweep and to run its own before os._exit
(#1647).
copy2() and move() return a str for a str
destination, as CPython does; for a Path one CPython returns the Path,
or a str when it names a directory, so the check returns a path that
str(), os.fspath() and Path() take and nothing else does. Python's
built-in exceptions that Codon lacks (ImportError,
ModuleNotFoundError, FileNotFoundError, FileExistsError, BlockingIOError, PermissionError,
ProcessLookupError, ChildProcessError, TimeoutError,
ConnectionError, BrokenPipeError, RecursionError, and
KeyboardInterrupt, derived from
BaseException, so except Exception: does not catch it) are declared in
the model's _exceptions.codon, each
derived from its CPython base, and the prelude of every rewritten module
imports them, so except ImportError: compiles as Python reads it.
SystemExit.code is an int here, the status Codon's SystemExit holds;
CPython's is whatever it was raised with (an int, a str or None), so
test it with isinstance() before reading it as an int. Each
takes no message or one value by position, as CPython does; a keyword
(TimeoutError(message=...)) and OSError's (errno, strerror) are
refused. OSError(errno, strerror, filename) is declared of an
Optional[int], an Optional[str] and a str (CPython takes any value
for each), with which tools.file_writes raises another's error as its
own, and so is it for a class a checked module derives from OSError;
the subclasses above still take one message (FileNotFoundError(errno,
strerror, filename) is refused). OSError's (and IOError's, and
each subclass's) errno is declared, an Optional[int], as CPython's is
None for an OSError of one message and the code for one the system
raised; CPython's is whatever the first of two to five values was
(OSError("a", "b").errno is "a"), a form the check refuses, so code
catching another's error tests it as CPython needs
(exc.errno in (errno.EPERM, ...), which is False for None or a
str) rather than doing arithmetic on it;
slurm.manifest tells a file system without hard links by it. Its
strerror and filename are declared too, each an Optional[str]:
CPython's are None for an OSError of one message, and whatever the
second and the third of three to five values were, and the system's
filename is the path as it was given (a Path, or bytes), so these are
narrower than CPython's; tools.file_writes passes the filename to
os.fspath() before comparing it with a path. Nothing in these
shims runs: no environment is read, no file is copied, no process is
signalled, no signal handler is installed and no exit function is
registered. hashlib declares sha256(string=b"", *,
usedforsecurity=True) and its hash object (update(data, /), digest(),
hexdigest(), copy(), name, digest_size, block_size); no other
hash. str.encode() takes (encoding="utf-8", errors="strict"). Since
the check reads bytes as str (below), it cannot tell the two apart:
it takes hashlib.sha256("text"), which CPython refuses (a str must be
encoded first). bytes.decode() is not declared, since it would be a
str's too and pass "text".decode(); the check refuses
data.decode(), which CPython takes. str.removeprefix(prefix, /) and
str.removesuffix(suffix, /) (Python 3.9) are declared too, and so are
str.maketrans(x, y[, z]) and str.maketrans(dict_of_str) (#601), whose
table str.translate() takes (Python's also maps to ints, which Codon's
translate does not). list.index(value, start[, stop]), xs += (4, 5)
(a list += any iterable, as in Python; xs + (4, 5) stays refused, as
there) and reversed(d) of a dict (its keys, last first) are declared in
_tc.codon (#601); reversed() of a dict view is not. sys.version_info,
sys.version and sys.hexversion are not modelled, and refused with a
hint: the check compiles every branch of a version test, so the code must
be valid on every supported Python anyway. urllib.parse
declares urlsplit(), urlunsplit(), unquote() and SplitResult (its five parts,
as attributes and as a tuple, hostname, port, username, password
and geturl()), urlparse() and ParseResult (its six parts, likewise;
not geturl()), and parse_qsl(qs, keep_blank_values=False), the first
two of CPython 3.10's parameters, of a str query, giving a
list[tuple[str, str]] of its pairs (none in the check: nothing is
parsed), with which hils.host_devices reads a device's fd option;
strict_parsing, encoding, errors, max_num_fields and separator,
a bytes query and parse_qs() are not declared. A scenario sees the same shims: it may import pathlib,
hashlib, os, shutil, subprocess, importlib.metadata,
urllib.parse, signal, atexit, shlex, fnmatch, keyword and sys and call what they
declare,
file-system queries and the two writes included (Path(...).exists(), Path(...).write_text(...)), read
and set the environment (os.environ.get()), copy, move and remove files
with shutil, start and wait for a process with subprocess.run() or
subprocess.Popen, signal one with os.kill(), install a signal handler
with signal.signal(), register a function to run at exit with
atexit.register(), read an installed
distribution's version with importlib.metadata.version(), and catch the
exceptions above. Any other name of them (Path.is_socket(), hashlib.md5(),
shutil.disk_usage(), subprocess.check_output(), os.scandir(),
importlib.import_module(), urllib.request, signal.setitimer()) fails
its check wherever Codon compiles it. The model's top-level package declares __file__ (a
str, as CPython sets it on a regular package), which
authoring.framework_pin imports (from .. import __file__).
collections.abc has no model of its own: from collections.abc import
Sequence is read as an import from typing (#586), for the names listed
under "The names of typing a scenario may import", and import
collections.abc is refused. @abstractmethod is dropped, since
Codon 0.19 cannot decorate a method; ABC is an empty base, and so is
typing.Protocol, whose @runtime_checkable is dropped too.
typing.Any imports, and every annotation naming it is dropped.
typing.TYPE_CHECKING is a compile-time true, so Codon compiles only the
first branch of if TYPE_CHECKING: ... else: ..., and the check reports
no import in the else branch, nor in the body of if not TYPE_CHECKING:
(the same branch, written the other way round): scenario_queue imports
tqdm in an else branch, at module level, and waits without the progress
bar in the first branch. The logging shim covers, with CPython's
signatures: getLogger, getLevelName, the level constants (NOTSET to CRITICAL,
WARN, FATAL) and BASIC_FORMAT; Logger.debug, info, warning,
warn (the deprecated alias), error, exception, critical, fatal and
log(level, msg, ...), and the module-level functions of the same names;
setLevel, isEnabledFor, getEffectiveLevel, addHandler,
removeHandler, hasHandlers, getChild and the handlers list; Handler, StreamHandler,
FileHandler, NullHandler and Formatter; and basicConfig with its
keywords (filename, filemode, format, datefmt, style, level,
stream, handlers, force, encoding, errors). Every logging call
takes exc_info and stack_info as a bool, stacklevel as an int
and extra untyped: the check reads no more of it than that it is passed.
The message may be any object (logger.info(123)), and log() takes its
level as an int, as CPython does (a str raises TypeError there), while
setLevel() and basicConfig(level=...) take an int or a str
("INFO"). extra={"k": 1} and extra={"k": "v"} pass; a dict with values
of two classes ({"k": "v", "n": 2}) is valid Python that Codon cannot type
as a dict display (even annotated dict[str, Any]), so convert
the values to str (extra: dict[str, str]). exc_info takes a bool only (not an exception or a
tuple). Left out: LogRecord, Filter, LoggerAdapter, disable,
shutdown, captureWarnings, addLevelName, the record factory and logger class
functions, and the internals of Logger (handle, makeRecord,
callHandlers, findCaller); test_typecheck_model.py lists them.
- pyxodr's road network is the model's: codon/pyxodr/ names the
RoadNetwork, Road, LaneSection and Lane that coordinate.codon
declares (what the framework reads of them, down to the XML elements, and
how MapManager loads one: RoadNetwork(xodr_file_path, resolution) and
get_roads()), so a checked module annotates with pyxodr's own classes
under TYPE_CHECKING (utils/traffic_light.py, which may not import
coordinate; coordinate/transform.py) and takes the network
MapManager returns. Nothing else of pyxodr is there. codon/lxml/ names
the XML element coordinate.codon declares likewise, as
lxml.etree._Element, the type of pyxodr's lane_xml and road_xml
(coordinate.transform._cubic(); import lxml.etree under
TYPE_CHECKING, since Codon does not find etree with
from lxml import etree).
test_typecheck_model.py compares the constructor and get_roads() with
pyxodr's and loads the fixture map's OpenDRIVE with them.
- lanelet2 is modelled: codon/lanelet2/ and
codon/autoware_lanelet2_extension_python/ declare the classes and
functions of simple-lanelet2's Python API the checked modules use, with
its parameter names: lanelet2.core's map, lanelet layer, lanelets, line
strings, points, attribute maps and regulatory elements (their ids,
attributes, centrelines, bounds, rule parameters and stop lines), the
map's other layers, polygons and areas, and what
coordinate.projection._read_local_tags() edits a Local map with (a
Point3d's x/y/z setters, LaneletMap() and an add() of each
primitive),
BasicPoint2d, BasicPoint3d (built from x, y and z), GPSPoint,
a point layer's points by id (pointLayer[id]), and the lanelets whose bound a
line string is
(LaneletLayer.findUsages(), of a line string only); lanelet2.geometry's
to2D(), length(), length2d() (of a lanelet), toArcCoordinates()
(an ArcCoordinates), findNearest() and findWithin2d() (of the
lanelet layer, around a point) and inside(), with the points a 2D line
string yields (ConstPoint2d) and a lanelet's outline
(ConstLanelet.polygon2d(), a CompoundPolygon2d of them); lanelet2.routing.RoutingGraph(laneletMap, trafficRules) with
following(), previous(), left(), right(), adjacentLeft(),
adjacentRight() and conflicting() (a list of ConstLanelets, as
simple-lanelet2 hands them out: lanelet2's own binding wraps each in a
ConstLaneletOrArea, which route.geometry still unwraps at run time);
lanelet2.traffic_rules.create()
with its Locations and Participants; lanelet2.io.Origin,
lanelet2.io.load(filename, projector) and
lanelet2.io.writeRobust(filename, map, projector) (the errors, a list of
str), which t4_scene.osm writes a map with; lanelet2.projection's
UtmProjector, LocalCartesianProjector and MercatorProjector (with
forward() of a GPSPoint, and MercatorProjector's reverse() of a
BasicPoint3d); and Autoware's MGRSProjector, TransverseMercatorProjector,
AutowareTrafficLight and RoadMarking. Each module's header says what
it leaves out; parts of the API the framework does not use yet (write(),
routes, matching, any other editing) are not declared, and the check refuses
them. MapManager (coordinate.map_manager, checked) and
sweeper.map_loader load the map with them and hand it out as these
classes, and MapManager its routing_graph too. Python's
lanelet2 has no signatures to introspect (it is a compiled extension), so
test_typecheck_lanelet2_model.py checks that every declared name exists in
the installed package, that its parameters are the ones the package's
stubs (lanelet2/*.pyi) name, and that each declared call, made on the
fixture map, returns what the model says. What the model has to say about
lanelet2 that Python does not:
- A mutable class derives from its Const twin (Lanelet from
ConstLanelet, LineString3d from ConstLineString3d, Point3d from
ConstPoint3d): in Python they are unrelated classes, but every function
that takes the Const one takes the mutable one, which has all its
members. So a layer's Lanelet goes where the routing graph takes a
ConstLanelet, and a variable that holds either is annotated with the
Const class (lanelet: ConstLanelet = lanelet_map.laneletLayer[...], as
trajectory/relative_lane.py walks from a lanelet to the ones that
follow it). lanelet2's stubs, which mypy reads, keep the two unrelated,
so a function that takes either spells the union out
(Union[ConstLanelet, Lanelet]), which the check drops, as it drops any
union parameter. A member the mutable class narrows (Lanelet.leftBound
and TrafficLight.stopLine are LineString3ds) is declared with the
Const type, so the check reads less than Python hands out, never more.
The projectors likewise derive from lanelet2's Projector, as they do in
C++: Python registers none of them as derived from it, but
lanelet2.io.load() takes any, so the projector
coordinate.projection.resolve_projector() hands out (one of four) is a
Projector to the check, which load() takes.
(Codon 0.19 does not convert an Optional of a subclass to its base in
one step, Optional[LineString3d] into a list[ConstLineString3d]:
that would take an annotated local of the subclass first.)
- An overloaded function dispatches on its argument: Codon 0.19 cannot
overload a module-level function, so to2D() and length() are generic
and declare the overloads the framework calls (of a line string); any
other argument is a compile error. Codon decides isinstance by the
static type, and isinstance(Lanelet, ConstLanelet) is false there as
in Python, so each overload names both twins.
- A regulatory element is its own class: lanelet2 hands each one out as
the class of what it is (TrafficLight, RightOfWay, TrafficSign, or
Autoware's), all derived from RegulatoryElement, as the model declares
them, and a lanelet's regulatoryElements is a list of the base. Codon
decides hasattr(element, "stopLine") by that static type, so the
stop-line branch of code that duck-types the element would never be
compiled. utils/stop_line.py therefore reads one element in a public
function whose parameter is annotated with the union of the classes it
reads (Union[TrafficLight, AutowareTrafficLight, RightOfWay,
TrafficSign, RoadMarking, RegulatoryElement]), which the library check
calls once per member, so each branch is compiled for each class that
takes it. At run time it still asks the element, as before.
- A rule parameter is declared by its role: lanelet2 holds a point, a
line string, a polygon, a lanelet or an area in a regulatory element's
parameters[role], and Codon has no type for "one of five classes".
ref_line, cancel_line and light_bulbs hold line strings
(ConstLineString3d), right_of_way and yield lanelets
(ConstLanelet); any other role is a compile error. refers holds a
different class per element (line strings or polygons for a traffic
light, polygons for Autoware's detection areas, lanelets for its
crosswalks), so the model declares it as a class of its own with only
what all five have, id and attributes. The stop line a road marking
refers to is therefore not checked: utils/stop_line.py reads it in a
function called in the else branch of if TYPE_CHECKING: (UNCALLED).
test_typecheck_lanelet2_model.py reads the roles off the model and
checks each against what the fixture map's elements hold.
- carla_driver_interface's geometry is modelled:
codon/carla_driver_interface/geometry.codon declares the Pose and
Trajectory the driver modules use (and waypoints_to_proto, which
driver.geometry re-exports, with its protobuf result left untyped), with
the package's fields, signatures and types, the arrays as Codon's
(position: np.ndarray[np.float64, 1]). Its scipy-typed constructor
(from_rotation) and the protobuf conversions (from_proto, to_proto,
to_proto_at_time) are not there. test_typecheck_model.py compares the
model with the installed package. Like every top-level directory of
codon/ (pyxodr, pathlib, ...), it is a model of a module that is not
the framework's: _shims() lists it, so the library check and the scenario
check let import carla_driver_interface... through, and Codon compiles
the import against the model. MODELLED_IMPORTS, which redirects one
framework module's import of another, does not concern it. A scenario may
therefore use Pose and Trajectory of carla_driver_interface.geometry
now; a name the model lacks fails its check (Pose.to_proto() has no
attribute), and so does any other module of the package (no module named
'carla_driver_interface.protocol').
- python-dotenv is modelled as far as server uses it, with the
installed package's signatures, which test_typecheck_model.py compares:
codon/dotenv/ declares
find_dotenv(filename=".env", raise_error_if_not_found=False,
usecwd=False), returning a str, and
load_dotenv(dotenv_path=None, stream=None, verbose=False, override=False,
interpolate=True, encoding="utf-8"), of a path that is None, a str or
an os.PathLike (a stream, an IO[str] Codon has no type for, is not
taken). Nothing else of the package is declared (dotenv_values()), and
a scenario may use what is, as it may the shims.
- packaging is modelled as far as authoring.framework_pin uses it,
with the installed package's signatures, which test_typecheck_model.py
compares: codon/packaging/version.codon declares Version(version),
whose local is a str or None, and InvalidVersion, a ValueError
built from nothing or one message. Nothing else of a Version (its
comparisons, release, public, ...) or of the package (parse(),
packaging.specifiers) is declared.
- eclipse-sumo is modelled as far as traffic.sumo.network uses it to
find SUMO: codon/sumo/__init__.codon declares the sumo module's
SUMO_HOME and __file__, both str, and nothing else. eclipse-sumo is
optional (the sumo extra), so test_typecheck_model.py compares the model
with the package only where it is installed.
- roadgen is modelled as far as commonroad.map and
traffic.sumo.network convert the map with it (#2312):
codon/roadgen/__init__.codon declares __version__ (a str),
read_opendrive(path, sampling=2.0, origin=None,
projection="local_cartesian"), a Map's read_warnings(),
write_read_trace(), export_ir(), export_commonroad() and
commonroad_warnings() (a light cycle, which Codon has no one type for,
only as None), export_sumo() (the files' prefix) and
sumo_warnings(), and Trace.load(*paths, check_files=True) of str
paths, with the installed package's signatures, which
test_typecheck_model.py compares where roadgen (the map and sumo
extras) is installed. A trace's queries (translate(), whose answers are
dicts of mixed value types) and the rest of the package are not declared,
so SignalTable.links() stays UNCALLED. The probes compile as they are:
getattr(roadgen, "__version__", "0") is the rewrite of a getattr with a
default, and hasattr(roadgen.Map, "export_commonroad") is decided on the
model's Map. commonroad_map() compares the version as lists, as Python
compares the tuples (Codon's tuple[int, ...] is a List[int]), and
SignalTable.load() passes the three trace files by name, since Codon does
not unpack a generator into arguments.
- carla_canvas is modelled for canvas.use_canvas(): codon/carla_canvas/plugin.codon
declares plugin_path() -> Path and nothing else. carla_canvas is not a
dependency of this package, so test_typecheck_model.py cannot compare it with
the installed one.
- No bytes: Codon 0.19 has none, and reads a b"..." literal as a
str, so the check reads an annotation naming bytes as str
(DriveOutcome.debug_info: bytes = b""). What a process writes is a str
in the check, which has no decode(): decode it in the else branch of
if TYPE_CHECKING: and take the str in the first.
- dataclasses.fields() is not in the shim: a function that reads a
dataclass's fields from its class takes a free-form mapping anyway, so it
is UNCALLED and imports fields inside it (ControlConfig.from_mapping).
- Codon's numpy appends an array of the same dimensions:
np.append(yaws, yaws[-1:]), not np.append(yaws, yaws[-1]), which numpy
answers the same. Nor does Codon unpack an array into arguments
(f(*row)): pass row[0], row[1], row[2].
- Codon's numpy has no numpy.polynomial: evaluate a polynomial by
Horner's scheme from its highest coefficient, starting at 0.0 (so at
c[-1] + 0.0 * x, as polyval() starts at c[-1] + x * 0), and read
polyder(c)[0] as c[1] (as c[0] * 0.0 of a constant c, which
numpy answers with c[:1] * 0): every operation is numpy's, so the result is
too, bit for bit (t4_settings._polyval_at(), _polyval_on() and
_polyder_at_zero(), #731,
which test_t4_settings.py compares with numpy). Nor does Codon unpack a
generator (c, d, e = (float(v) for v in cde)): read cde[0], cde[1],
cde[2] (hils.lidar_spin.spin_lidar_points() reads its motion's six
columns so, one product at a time and in the generator's order, #2116).
- numpy is Codon's own, so import numpy as np compiles
(utils/powertrain.py). Codon compares no np.float32 with a Python
float: convert the float (x <= np.float32(max_rpm)), which is what
numpy 2 does anyway (NEP 50), so nothing changes at run time. Codon's
ndarray needs its dtype and dimension count, so a parameter annotated
with a bare np.ndarray or numpy.typing's NDArray[...] does not
translate; numpy's shape-typed array does (see An array is typed by its
shape below). A bare np.ndarray parameter compiles while only its
callers reach it, but the check has no value of it to call with
("cannot typecheck '_reach_m(_acs_value(np.ndarray))'"): the protected
geometry helpers of coordinate.transform and driver.control take
np.ndarray[tuple[int, int], np.dtype[np.float64]] and their points
np.ndarray[tuple[int], np.dtype[np.float64]].
- An overridden method takes concrete types: Codon 0.19 cannot call a
method a subclass overrides when one of its parameters is generic, which an
abstract collection (Sequence[X]) or an unannotated parameter is.
AutowareBridge.configure() takes waypoints: Optional[list[BridgePose]]
= None in the base class and in the checked FakeAutowareBridge, and its
caller passes a list. (None, not []: a mutable default is shared by
every call.)
- A call to an overridden method passes every argument, by position, of
the parameter's own type: Codon 0.19 calls it through a dispatch thunk,
which fills in no default, takes no keyword argument and no **kwargs,
and does not upcast. Hand a subclass over through a local of the
parameter's type (entity: DrivenEntity = self, as traffic/driven.py
does). A thunk cannot call a method that takes **kwargs at all:
BackendDriven.turn_at_junction(), which actions.turn calls, hands the
backend its **kwargs only in the else branch of if TYPE_CHECKING:
(see below), and the check calls each backend's turn_at_junction()
through its class instead.
- A list of a subclass is not a list of its base: Codon 0.19 refuses a
list[EntityLanePositionCondition] where a list[BaseCondition] is
expected, and extend() does not upcast either. Build the list in a local
declared list[BaseCondition], appending each element
(composition/temporary_stop.py). The same holds for a scenario's own
classes, and for the display itself, not just extend()
(rules: list[Rule] = [Dbl(), Neg()]). The check makes a list display a
list of the base wherever the base is known, as _acs_list_into(List[Rule](),
Dbl(), Neg()) appends each element: an annotated target (a module-level,
local or class-level x: list[Rule] = [...], a dataclass
field(default_factory=lambda: [...]), a -> list[Rule] return, an
argument of a parameter annotated list[Rule] of a function or class
defined in the same module), or, with no annotation, the nearest class of
the module that the constructed elements ([Dbl(), Neg()]) share, when
that class and its bases are all defined in the module (a base imported from
elsewhere may be a condition or an action, left to _acs_list). A
display whose target is not known and whose classes share no base, or an
argument of an imported function, is refused with a hint: annotate the
target list[Base], or declare a local list[Base] and append() each
element. An element that is no subclass of the base is refused, and so is a
method only a subclass has, as in Python. A set or a tuple display is not
rewritten.
The same holds for the neighbouring shapes (#489). A dict display of
constructor calls, {"a": Circle(), "b": Square()}, is a dict of the nearest
shared class when its keys are all str or all int literals
(_acs_dict_into(Dict[str, Shape](), ...)), and of the annotation for an
annotated target or a dict[str, Shape] parameter of a function or class of
the module; any other dict display of different classes is refused with a
hint to annotate it. A list display added to a local or parameter annotated
list[Shape] (a + [Square()], [Square()] + a, a += [Square()],
a.extend([Square()])) is a list of that class; added to a list whose type
the check does not know (a call, an attribute), it is refused with a hint to
put the display in a list[Shape] local first. A class defined below the
scenario class may be named inside a function body (Codon types a body
when it is first called, after the whole module), so the displays above
work with it; a class body, a default or a module-level statement runs at
once and cannot name it (name 'Rule' is not defined): define the class
above, as the hint says. The check is more permissive than Python here: a
function that names a class defined further down passes, although calling it
at module level before that class is defined raises NameError. A name
rebound by a loop, a with, a comprehension or a lambda is no longer the
tracked list[Base]. An element of a Sequence[AnyPose] is
a Union of poses; the model's _expect_pose takes one, reading its
members with static.vars_types(pose, with_index=True) (without
with_index, Codon 0.19 does not bind the loop variable).
- A class cannot be named before it is defined or imported, in a
signature or a class-level declaration, and an import cycle resolves only
with the names already defined. Two classes whose signatures name each
other across modules (a backend's methods take the entity, the entity holds
the backend) need a base class the first module defines: the backends take
a traffic.base.DrivenEntity, which traffic.driven.BackendDriven derives
from.
- A part of a module that needs what Codon lacks moves out: the
backend interface of traffic/base.py is checked, and TrafficContext,
which holds pathlib.Paths, lives in traffic/context.py (re-exported by
base), which is checked on its own with the pathlib shim.
- A carla.Actor where the CARLA API takes a carla.Vehicle
(TrafficManager.set_desired_speed()): typesafe_carla's Python API takes
the actor as it is, and its Codon library wants the checked conversion.
Convert under if TYPE_CHECKING:, which Codon reads as true and Python as
false, so nothing changes at run time (traffic_manager._vehicle()):
traffic.config imports utils.config in
from_mapping()).
- A CARLA container has no truth value in Codon: if not found: on a
carla.BlueprintLibrary is if len(found) == 0:.
- No ** in a dict display ({"a": 1, **other}): Codon's parser fails on
it, even in a function it never compiles. Build the dict, then
update() it.
- dict.get(key, default) takes a default of the value type in Codon.
Where "absent" has no value of that type, take d.get(k), an Optional
of the value type, as the check declares it with Python's meaning.
- CARLA's containers are not lists. world.get_actors() is a
carla.ActorList; a parameter that takes it or a list is
Union[carla.ActorList, list[carla.Actor]], and the check calls it with
each.
- Codon cannot hash a class: hash((MyClass, x)) names the class by its
name instead.
- An attribute holds one type: one assigned a union (EntityRole | str)
is stored converted where every use converts it anyway
(self._entity_name = str(entity_name)). Codon 0.19 can assign nothing
to an attribute declared with an optional union (Optional[Union[EntityRole,
str]]): neither a member nor None matches it ("'Optional[EntityRole]'
does not match expected type 'Optional[Union[EntityRole | str]]'"), and
Union[EntityRole, str, None] takes no None. A public dataclass field
is converted in __post_init__ (VehicleEntityConfig.role_name: str,
self.role_name = str(self.role_name)), which still takes an EntityRole
at run time; or
by a constructor written out, which keeps the union on the parameter for
mypy (RelativeLanePose(entity_ref: Union[EntityRole, str, None]), stored
as entity_ref: Optional[str]). Without None the union is held as it
is: RoutingAction._entity_name: Union[EntityRole, str] keeps what it was
given, which its public entity_name returns.
- A union of classes without None is an attribute type: Codon 0.19
holds Union[Lanelet2Pose, MapPose, ...] (TrajectoryVertex.position, any
of ten poses), declared whole or through an alias (position:
TrajectoryPosition), and decides isinstance on it when the program
runs. A union is no parameter type: Codon 0.19 crashes on one ("union
already sealed"), so the check drops it and the library check calls the
function once per member. Spell the parameter's union out, not through an
alias, which the library check would call with the union itself and so
catch less. A module-level alias whose members are strings
(Union["Lanelet2Pose", MapPose], for names imported under
TYPE_CHECKING) has them unquoted by the check.
- Codon does not run __post_init__: its constructor of a class with
declared fields only assigns them. The check calls __post_init__ itself,
as a dunder method, so its body is checked. A field(init=False) set there
is given an empty value of its type (list[float]()) by the check, so the
constructor leaves it out as Python's does.
- A dataclass field may hold a union or a free-form mapping: the library
check constructs a dataclass once per member of a union field
(ScenarioRunConfig.scenario, one of seven scenario configs), as it calls a
function once per member of a union parameter. A field with a default whose
annotation names Any (SweepConfig.constraints: dict[str, Any], a Hydra
mapping of any values) keeps its declaration, its empty default is built as
that type (dict[str, Any]()), and the check's constructor call leaves it
out, passing the fields after it by keyword. A default_factory=list on a
List[int] field is built as a List[int] as on a list[int] one, and a
list, dict or set factory of an abstract collection as the one the
typing shim maps it onto (env: Mapping[str, str] =
field(default_factory=dict) as a Dict[str, str], #188).
- object.__setattr__(self, "name", value), as a frozen dataclass sets
its fields, is checked as self.name = value: Codon 0.19 has no
object.__setattr__.
- No **kwargs iterated as a dict: Codon's **kwargs is a named tuple,
with no items(). Pass the names and values as parameters
(_finite(owner, "ds", self.ds)), one call each.
- A run-time-only wrapper of a method: Codon 0.19 cannot decorate a
method, and a decorator UNCALLED names (a function of the same module) is
dropped by the library check, which compiles the method as written.
TrajectoryVertex.__init__ is typed (position, advance=None);
_refusing_vertex_times wraps it at run time to refuse the calls written
for a vertex with a time, with inspect and functools imported inside
it, where Codon never compiles them. The wrapper is typed with a
TypeVar ((init: _Init) -> _Init), so mypy still sees the typed
__init__; the check drops an annotation that names a module-level
TypeVar.
- A class chosen by a conditional expression (type(entity) if entity is
not None else self.ego_type) does not compile: Codon cannot hold a class
as a value ("'type[Optional[EgoVehicle]]' does not match expected type
'EgoVehicle'"). Read a class attribute off each in its own branch, the
instance or the class (BaseScenario.ego_requires_goal, which reads
requires_goal, a class attribute no instance assigns).
- A call to a register_*() method that takes a Union (register_init,
Union[BaseAction, Callable[[carla.World], None]]) compiles both
isinstance branches of its dispatch with the caller's type. Where the
caller has an action, do in the if TYPE_CHECKING: branch what the method
does for one, and call it in the else branch
(BaseScenario.register_route_to_goal).
- isinstance(x, (A, *OTHERS)) does not compile: Codon 0.19 cannot
unpack a tuple of classes into isinstance. Spell the classes out
(Trajectory.is_relative).
- callable(x) is checked as hasattr(x, "__call__"), which Codon
decides when it compiles: Codon 0.19 has no callable.
- memoryview(x), which Codon 0.19 lacks too, is checked as a view of
bytes (_acs_memoryview): true while it is not empty, sliced into another
view, and taken by os.write(), which is what a loop writing all of a
buffer needs (tools.file_writes). A memoryview (or callable, or
tuple) the code binds itself, a parameter, an import or a function of
that name, is its own and is left as it is.
- Annotate a local that starts as None (spawn_points:
Optional[list[carla.Transform]] = None): Codon types it by its first
assignment, and on a branch it then compiles alone it can crash without a
diagnostic. A function that imports a name locally (import
typesafe_carla.carla as carla) makes it local to the whole function, so an
annotation naming it needs that import above it.
- Operators take the operand types they accept, not object. Annotate
other with the class (def __add__(self, other: FrenetVelocity)) and a
scalar with float, and keep the isinstance guard that returns
NotImplemented: Python still runs it, and Codon, which decides
isinstance when it compiles, drops it.
- An operator whose result depends on the operand (Absolute - Absolute
-> Relative, Absolute - Relative -> Absolute) keeps its @overload
stubs, for mypy, and spells the union out on the implementation:
@overload
def __sub__(self, other: AbsoluteVelocity) -> RelativeVelocity: ...
@overload
def __sub__(self, other: RelativeVelocity) -> AbsoluteVelocity: ...
def __sub__(
self, other: AbsoluteVelocity | RelativeVelocity
) -> Union[RelativeVelocity, AbsoluteVelocity]:
if isinstance(other, AbsoluteVelocity):
return RelativeVelocity(...)
if isinstance(other, RelativeVelocity):
return AbsoluteVelocity(...)
return NotImplemented
The check calls the implementation once per member, and Codon compiles
only the isinstance branch of that member, so each call has one result
type; the Union return annotation is dropped. Two classes whose methods
take each other are fine: an annotation naming the later one is dropped
from the earlier one's signatures, and the check still calls with it.
- Two checked modules that import each other: Codon reads
TYPE_CHECKING as true, so an import made only for annotations is still
an import cycle, and Codon resolves a cycle only with the names already
defined. Put that if TYPE_CHECKING: import at the end of its module
(# noqa: E402), after the names the other module imports, as
coordinate/frames.py does for poses.py; a parameter or return
annotation naming a class imported there is dropped, as for a class
defined further down (#507). That covers signatures only: a class-body
attribute declaration (peer: Optional["B"]) is read when the class is
defined, so a class cannot declare an attribute typed by a class of a
module that imports its own, and Codon 0.19 refuses even a class defined
further down the same module. Declare the attribute in one module of the
pair, and in the other name the class in signatures or import it inside
the function that uses it, or put both classes in one module. The check
says so at the attribute (or at the import) instead of Codon's bare
"expected type expression" / "cannot import name". The scenario imports
the module with the late import first (from .a import A before from
.b import B), since Codon runs the cycle from the first module it reaches.
Set class attributes in the class body (FRAME: ClassVar[CoordinateFrame]
= CoordinateFrame.LANELET2) rather than from a function run after the
class.
- A global of a module whose top level runs a statement (server, which
calls load_dotenv() on import) could not be read at another module's top
level: Codon 0.19 reported "name '_WATCHDOG_GRACE_S' is not defined" for
_CARLA_GONE_TIMEOUT_S = _WATCHDOG_GRACE_S + 5.0, and sweeper.isolation
gave the check a copy of its value under if TYPE_CHECKING: (#719). A
constant imported from another checked module is now read through that
module (#1693, Checked modules and the model above), so the module
computes it once, as Python does (#1704). Any other name is still imported
with from M import, which Codon 0.19 may report as not defined at the
importing module's top level; a function of the module, or a parameter
default, reads it.
- Codon does not narrow an Optional after if x is None: return. Where
the narrowed value goes into something typed (a tuple the function
returns), assign it to an annotated local first (od: OpenDrivePose = x),
which Codon unwraps. Likewise, a function returning
tuple[Optional[Path], bool] returns a Path it found through an
Optional[Path] local (shipped: Optional[Path] = candidate, as
maps.resolver._find_xodr() does): Codon types a tuple by its members,
and a bare annotation without a value (found: Optional[Path]) does not
change the type of what is assigned to it later. Nor does a value: an
int assigned to an Optional[int] local is an int in the tuple
returned right after it, so sweeper.isolation._launch_isolated() sets
its (succeeded, exit_code) in each branch and returns them once, after
the branches join. An attribute is not
narrowed either: after if self._spawn_pose is None: raise ...,
self._spawn_pose is still an Optional[Lanelet2Pose], and
snap_to_carla_road()'s isinstance dispatch takes its CARLA branch for
it ("'Lanelet2Pose' does not match expected type 'CarlaWorldPose'").
BaseScenario._setup_ego_spawn() reads it into
ll2_pose: Lanelet2Pose = self._spawn_pose first.
- Codon does not narrow a union either, with is or isinstance, and an
attribute holding a union (Union[Sequence[int], TrafficLightTarget,
None]) reaches every use as the union. Store it split by kind
(_all_lights: bool, _lanelet2_ids: Optional[Sequence[int]]), deciding
the kind once in __init__, where isinstance on the parameter is decided
when Codon compiles.
- A default of a subclass for an Optional of its base goes through an
if statement, not a conditional expression: x if x is not None else
Sub() is refused (Optional[Base] is not Optional[Sub]).
- A conditional expression of two types (log if log is not None else
subprocess.DEVNULL, a file or an int) is typed by its first branch.
Neither can an if statement assign the two to one name: call a function
once with each (autoware_stack.launcher._start_session(), whose
Union[IO[bytes], int] parameter the checker drops, so Codon compiles it
for each).
- A dict has no truth value in Codon: if not d: is if len(d) == 0:.
- No setattr(obj, name, value) with a name known only at run time:
spell the assignments out (actions/environment.py).
- No attribute on a function: a Codon function has none, so neither
setattr(probe, "on_lanelet", read) in a decorator nor
probe.on_lanelet = read compiles. odd.probes defines each lanelet
reader as a plain function and attaches it at the end of the module, in
the body of if not TYPE_CHECKING:, which Python alone runs: the
probes have the same on_lanelet attribute as before, and the check
compiles each reader on its own, as it does every function of the module.
- A union a function returns goes through a module-level alias: the
check drops a Union written out in any annotation (Codon 0.19 crashes on
one in a parameter), a local's and a return's included, and Codon then
types the function by its first return. An alias is kept, and Codon
converts each return to it: odd.probes._speed_limit_on_lanelet()
returns _SpeedLimitOnLanelet = Union[float, "Undecided"], a number or
UNDECIDED. Codon 0.19 converts a value of any other type too, failing
only when the program runs (internal.new_union), so the check compiles
the body but does not check which member each return is.
- An exception class spells out its constructor: Codon 0.19 cannot
raise a subclass of a built-in exception that has none of its own
('MapSourceError' object has no attribute '_pytype'), so it takes the
message and passes it on: def __init__(self, message: str) -> None:
super().__init__(message) (maps/source.py). One that hands on whatever
it is given, as Exception's own does, takes *args: str, **kwargs: str
and passes them on (hils/host_devices.py's HostSetupError): Python
builds it as before, with the same arguments, message and repr, and the
same TypeError for a keyword.
- except (A, B): is split by the check: Codon 0.19 takes one class
per except clause ("expected type expression"), so the check rewrites a
tuple written out into one clause per class, in the tuple's order, each
with the same body (and the same as name), where the tuple's clause was:
Python tries a tuple's classes in order, so the same exceptions reach the
same body and the others the clauses after it. The copies are lines the
source does not have; an error in one is reported at the line it copies,
and the lines after them where they are. A tuple that is not written out
(except errors:, except (*errors,):) is left as it is, and Codon
refuses it. raise X from None is rewritten to raise X by the check;
raise X from exc compiles as it is. A builtin exception class raised
as it is (raise NotImplementedError, an abstract stub) is rewritten to
an instance (raise NotImplementedError()), which Python raises anyway:
Codon 0.19 crashes on the class.
- A caught exception kept as an Exception (#512): Codon does not
convert an exception to its base class ('ValueError' does not match
expected type 'Exception'), so last: Exception = ...; except ValueError
as e: last = e (also Exception | None, BaseException, list[Exception]
and dict[K, Exception] with append, add, an item assignment or a
display, a return of -> Exception | None, and an argument of a function,
method or class of the module whose parameter is annotated so) is
rewritten to last = Exception(e.typename, e.message): the stored value is
a plain Exception with the caught one's type name and message, so
str(last) and logger show the same, but raise last is not caught by
except ValueError: in the check, and the fields of a subclass are not
kept. Only a handler variable (except X as e, X not Exception) is
rebuilt: any other value (last: Exception = ValueError("x"), an
exception held in a local first) is refused with the hint to store str(e)
or catch Exception. raise last of an Exception | None is refused by
Codon (exceptions must derive from BaseException): test last is not
None and raise a local Exception instead.
- e.args (#512): Codon's exceptions have no args; the model declares
it on Codon's own exception classes (_exceptions.codon) as the one-value
tuple of the message, which CPython's is for an exception raised with one
message. An exception raised with none or several values has other args
there (len(e.args) is 1 here, 0 for ValueError()).
- operator.itemgetter(i) and attrgetter("n") are lambdas (#517):
Codon's pass the argument wrapped in a tuple (tuple index out of range,
'Tuple[A]' object has no attribute 'n'), so they cannot be a sort key.
With literal arguments (an int or str key, a string of dotted attribute
names, several of each for a tuple) the check writes the lambda
(key=lambda v: v[1], lambda v: v.n, lambda v: (v[0], v[2])), however
it is imported (operator.itemgetter, from operator import itemgetter as
ig); with a variable it refuses the call and says to write the lambda.
- copy.copy and copy.deepcopy of a class of the module (#517): Codon
calls the class's __copy__ and __deepcopy__, which a class has not. In a
module that imports copy the check writes both, with @extend, for each
top-level class whose bases it can see (none, object, ABC, or classes
of the module): __copy__ makes a new instance holding the same field
values, __deepcopy__ one holding a deep copy of each, over the class-level
annotated fields (a base's first, ClassVar left out), as Python copies the
attributes. A method the class defines itself is kept. A class with another
base (a framework class), an exception, an Enum and a NamedTuple get none,
and a class defined in a module that does not import copy has none:
copying it from another module is refused with the hint to define the two
methods on the class (def __copy__(self) -> Self) or to build the copy
from its fields. copy.copy of a list, dict or set compiles as it is.
- frozenset is checked as a set: Codon 0.19 has none, so the check
rewrites frozenset(...) to set(...), frozenset[X](...) to
Set[X](...) and frozenset[X] / FrozenSet[X] to Set[X]; the source
keeps its frozenset. An empty one is frozenset[str](): a bare
frozenset() is a set of no element type in Codon
(scenario_runner._destroy_all_dynamic_actors()). A parameter that takes
either (exclude: set[str] | frozenset[str] = frozenset()) is a union,
which Codon cannot express: take Optional[set[str]] = None and read
None as the empty set (sweeper.job_status._remove_run_stems(), #2306).
- Whether a directory has an entry is any(True for _ in p.iterdir()),
not any(p.iterdir()): Codon's any() asks each item's __bool__, which
a Path has not (CPython's neither: every path is true), so the two read
the same (run_dirs._batch_output_dir_refusal(), #2306).
- A variable-length tuple attribute is checked as a list: a class-level
tuple[X, ...] / Tuple[X, ...] becomes List[X], and tuple(x) of a
list or a generator becomes a list (a tuple passed to tuple() stays
itself), so self.vertices = tuple(vertices) compiles; the source keeps
its tuple. A dataclass field's () default, written directly or as
field(default=()), is an empty List[X] (Pin.warnings,
PlannedRoute.via), for a Sequence[X] field as for a tuple[X, ...]
one, and its tuple display default (("a", "b")) a list of its elements,
List[X](["a", "b"]) (DockerAutowareConfig.colcon_args, #188). Any
other tuple display stays a fixed-length Codon tuple, which is no
List[X]: pass tuple([...]) of a list instead (ros2.config's
_default_sources(); CommandSourceConfig.topics, which returns a one-
or a two-element tuple, likewise, so its two returns have one type). A
parameter tuple[X, ...] is dropped (a caller may pass a tuple), and the
library check calls it with a List[X]. Its () default, and that of a
Sequence[X], MutableSequence[X], Iterable[X] or Collection[X]
parameter, is an empty List[X] too (RoutingAction(..., waypoints:
Sequence[Lanelet2Pose] = ())), and its tuple display default a list of
its elements, as a field's; see Defaults are compiled too above.
Phase.states and SignalController.phases (signals.controller) are
checked this way. What only a tuple does is not checked: its hash (a
frozen dataclass's __hash__), its equality with another tuple and its
immutability; nothing compiled is run, so nothing can mutate the list.
Of a union of classes (tuple[A | B, ...], tuple["A | B", ...]) it
becomes List[Union[A, B]], which Codon 0.19 holds as it holds an
attribute of the union (above): route.model's
RouteSearchSpec.segments, each a lane or a junction segment. A union with
None in it is not translated.
- No star-unpacking into a tuple display ((*a, *b)): Codon wants a
tuple after *. Concatenate the lists (a + b).
- No * of a list in a call (f("rm", *ids) with ids a list):
Codon 0.19 unpacks only a tuple into arguments ("argument after * must be
a tuple"). Give the function a twin that takes the list
(DockerAutowareLauncher._run_docker(["rm", "--force"] + ids, ...), which
_docker(*args) calls with list(args)).
- An empty dict display for a Mapping parameter (extra_env={}) is a
dict of no element type: the check drops the abstract Mapping[str, str]
from a parameter, so nothing gives it one. A private function that is
handed dicts takes a dict[str, str] (DockerAutowareLauncher._run_argv()).
- A default does not name a class attribute (random_seed: int =
DEFAULT_RANDOM_SEED in a method of the class defining it): Python
evaluates it once, when the class is created, and Codon cannot name it
there. Write the value, with a comment naming the attribute.
- No del of a parameter (del world, goal to mark it unused): Codon
reads it as a call on the value. Leave the parameter unused.
- An Optional attribute is assigned without an annotation: with one
declared at class level (_lane_id: Optional[int]), Codon refuses
self._lane_id: Optional[int] = address.lane_id (an int "does not match"
Optional[int]) but takes self._lane_id = address.lane_id, and
X(...) if c else None likewise only without the annotation.
- x or [] with x: Optional[list[T]] has no one type in Codon: start
from an annotated empty list and assign x under if x:. Likewise
stated or DEFAULT with stated: Optional[str]: branch on the cases
into an annotated str local (coordinate.projection.resolve_projector).
- A regex group used as a str (m.group(1) handed to re.search, or
to a str local) compiles now that the check's Match.group() is a str
(#597); str(m.group(1)), the same str in Python, is still found in
older modules (coordinate.projection._parse_geo_reference).
- Projectors of different classes return through one -> Projector
(coordinate.projection.make_projector): the lanelet2 model derives each
from Projector, and Codon 0.19 converts a subclass to it at return.
- A subclass goes into an Optional base parameter from a base-typed
variable: Codon does not upcast and wrap in one step, so
super().__init__(child=PersistentCondition(...)) against
child: BaseCondition | None fails, and
child: BaseCondition = PersistentCondition(...) passed on compiles.
- getattr(x, name, None) for an attribute a value may lack at run time
(a mock in a test) becomes try: v = x.name / except AttributeError:,
which reads the attribute once, as getattr did.
- hasattr on an Optional is decided on the Optional, which has
none of its value's attributes: assign the value to an annotated local
after the None test (walker: PedestrianEntity = entity) and ask that.
- An Optional local tested in a loop (assert x is not None, or if x
is None: return) is left the unwrapped type for the next lap, whose
assignment of the Optional then fails ("'BaseCondition' does not match
expected type 'Optional[BaseCondition]'"). Read the value through a function
that returns it unwrapped (_trajectory_run._gate_at()), and test the
Optional expression itself where the loop needs the test.
- No attribute assigned through an Optional local (#649):
driven = _DRIVEN.get(key), then if driven is None: driven = _Driven(...)
and driven.stretches = ... in the branch and driven.turn_back = ...
after it, crashed Codon without a diagnostic. Build the new value in a
local of its own type (created = _Driven(...), assigned through and
stored), and read the stored one back after the branch
(driven = _DRIVEN[key]), as entity._walker_ai.drive() does.
- A call into a module Codon cannot compile, from a function it must: a
function UNCALLED names is still compiled when a checked function calls
it. Call it in the else branch of if TYPE_CHECKING: and check in the
first branch what it can (ScenarioRunner calls _judge_metrics(),
which evaluates the recorded frames with the unchecked metric_batch, in
the else branch, and the check leaves the run's result as it is; how
the verdicts merge into that result is a helper of its own,
_merge_metric_verdicts(), which the check compiles). A
private helper that imports such a module inside it needs an UNCALLED
entry of its own: its import is rewritten, and refused, even when nothing
compiled calls it.
- A type alias imported from another module, inside an alias of this one
(Received = Tuple[str, Command], with Command a Union alias of
ros2.commands) is "not defined" to Codon 0.19, though the imported alias
works as an annotation. Spell the union out in the alias
(ros2.transport.Received; test_ros2_commands.py checks the two are
equal).
- An entity looked up by role is looked up by kind:
find_entity_by_role_name returns an entity of any kind (Optional[Any]),
so a checked module asks entity.registry for the kind it acts on,
find_vehicle_entity (a BackendDriven: an NPC vehicle or the ego) or
find_pedestrian_entity. An object of neither kind (a test double) is
registered as both, so at run time every lookup still finds it; keep a
hasattr test where the code relied on getattr(entity, name, None).
Where the code called a method the other kind lacks, look the other kind
up when the lookup misses and raise the AttributeError the call raised
(LaneChangeAction, TurnAction and SetSpeedAction on a pedestrian),
so a wrong kind still fails rather than reading as an entity not found.
A narrower kind is a typed lookup of its own:
isinstance(find_vehicle_entity(name), HilsEgoEntity) is decided by the
static type (BackendDriven), so CommunicationLossAction calls
entity.hils_entity.find_hils_entity (an Optional[HilsEgoEntity]), which
makes that test in Python and is declared in the entity model.
- An entity is registered by kind, too: register_entity picks the kind
with isinstance(entity, BackendDriven), which Codon decides by the
static type and does not match for a subclass (a VehicleEntity), so a
checked caller calls register_vehicle_entity (through a BackendDriven
local, which upcasts) or register_pedestrian_entity. BaseScenario
calls them under an isinstance test on its parameter's own class, which
Codon decides as true, and hands anything else (a test double) to
register_entity, so at run time it is registered as before.
- carla.TrafficLightState is a value in the check, not a type: its
members are ints, as get_state() returns. The check reads an annotation
naming one of typesafe_carla's integer enumerations
(carla.TrafficLightState, carla.LaneType, ...) as int, so keep the
CARLA name (the authoring code generator reads it), and compare with
get_state() rather than look a member up by value. A member's name is
found through the enumeration's names (for name, member in
carla.TrafficLightState.names.items()), which both APIs have, as
utils/traffic_light.py does.
- A sensor callback gets a carla.SensorData in the check, where the
CPython API hands it the measurement itself (a carla.CollisionEvent).
Listen through a method that converts only when it has to
(if hasattr(data, "as_collision"): data = data.as_collision(), as
conditions/collision.py does): Codon decides hasattr when it compiles,
and the CPython measurement has no as_<kind>().
The camera and the LiDAR (sensor/) listen the same way, with
as_image() and as_lidar(). A bound method of another object (a
queue's put) is no listener: wrap it in a method of the sensor.
- An array is typed by its shape: Codon's numpy types an array by its
dtype and its number of dimensions (np.ndarray[np.float32, 2]), which
numpy.typing's NDArray[np.float32] does not say, and Codon has no
numpy.typing. Annotate with numpy's shape-typed array instead,
np.ndarray[tuple[int, int], np.dtype[np.float32]] (two dimensions),
which mypy reads as precisely and the check translates; a shape of
tuple[int, ...] or Any, and NDArray, are dropped (pedestrian.soma
holds a motion's bone directions as np.ndarray[tuple[int, int, int],
np.dtype[np.float64]] and its rotations with four dimensions; its
_as_matrices(), whose poses have three dimensions or four, takes
np.ndarray[Any, np.dtype[np.float64]] and is UNCALLED). An annotation
broken over lines keeps its line breaks inside its brackets, so
newest: Optional[\n ...\n] = None still compiles.
- A measurement's raw_data is a view in typesafe_carla's Codon library,
and Codon's np.frombuffer takes the bytes: read
np.frombuffer(image.raw_data.tobytes(), ...) under if TYPE_CHECKING:
and the CPython raw_data in the else branch, which copies nothing.
- queue.Queue[T] is the queue shim's, with CPython's signatures
(Queue(maxsize=0), put(item, block=True, timeout=None), get(block=True,
timeout=None), put_nowait, get_nowait, empty, full, qsize, and
the Empty and Full exceptions). Declare the queue's item type at class
level (_frame_queue: queue.Queue[carla.Image]); queue.Queue() in
__init__ takes it from there.
- threading.Thread is the threading shim's, with CPython 3.10's
signature (Thread(group=None, target=None, name=None, args=(),
kwargs=None, *, daemon=None), start(), join(timeout=None),
is_alive()); the check compiles the call the thread makes,
target(*args), so a target that does not take its arguments fails it.
args is a tuple, kwargs None and group None. The shim replaces the
threading module Codon ships, which Codon's numpy and typesafe_carla's
Codon library import for Lock, so it keeps all of that module
(Lock, RLock, active_count(), get_native_id(), get_ident()), as
Codon has it; a test compares the two.
- A datetime is formatted from its fields: Codon's datetime has no
__format__, so f"{now:%Y-%m-%d}" is
f"{now.year:04d}-{now.month:02d}-{now.day:02d}", the same string for
every year from 1000 on (run_dirs.timestamped_dir(), which
examples.run and ui.runner share, #1653).
- FileExistsError is OSError and a test of the path: Codon does not
name FileExistsError, so except FileExistsError: around a mkdir() is
except OSError: that raises again unless the path exists, or is a
symbolic link (one that resolves to nothing fails mkdir() with
FileExistsError too) (run_dirs.reserve_timestamped_dir()).
- A dataclass derived from a dataclass is constructed with its base's
fields first, as Python's constructor and Codon's take them, wherever the
base is checked (CarlaCameraSensorConfig(CameraSensorConfig)). Of two
bases that both bring fields, Python takes the last one's first and Codon
0.19 the first one's, so such a constructor is a problem: list it as
UNCALLED.
- No variable-length tuples beyond attributes: Codon's tuples have a
fixed length. A class-level tuple[X, ...] and tuple(x) are checked as
lists (above), but a tuple built otherwise (a display grown in a loop,
a + (b,)) is not, and turning the source's tuple into a list changes what
the class does (a frozen dataclass's hash, its equality with a tuple). A
function that returns one is checked as returning a List[X] (#191):
return tuple(x) of a list (signals.registry.registered_signal_controllers(),
ui.case_paths.batch_stamp()), not a tuple display, which is a
fixed-length Codon tuple, nor ().
- A lookup with a default is a loop: next((f(x) for x in xs if p(x)),
None) is a for loop that returns f(x) at the first x that passes
and None after it, and sorted(f(x) for x in xs if p(x)) sorts a list
built in a loop (ui.case_paths._override_label() and _job_indices(),
#1637). Both do what the generator did, in the same order.
- No ** of a dict in a call (f(**extra) with extra a dict):
Codon unpacks only a named tuple into keyword arguments. Write one call
per set of keywords (RoutingAction passes waypoints= only when there
are some, in a call of its own).
- Literal["a", "b"] is checked as the type of its values (str), an
annotation and a module-level alias alike (autoware_stack.docker's
BuildPolicy, the type of DockerAutowareConfig.build):
Codon's own Literal types a compile-time value of int, bool or str
only, and the typing shim's Literal is str (its TYPE_CHECKING is
declared with Static[bool], Codon's other name for its Literal). A
Literal inside a Union/Optional alias is translated the same way. A
Literal of values of several types is dropped as an annotation; as an
alias it is left as written, which the check rejects.
- No getattr(obj, name) with a name built at run time: spell the
attributes out (JunctionSegmentSpec.crossing()), and keep the getattr
for any other name in the else branch of if TYPE_CHECKING:, so Python
still raises the AttributeError it raised.
- A method only some entities have (route_to(), which an entity found
as a BackendDriven has and a pedestrian does not): ask
hasattr(entity, "route_to") and call it under that, and keep the call
that raises the AttributeError for an entity without it in the else
branch of if TYPE_CHECKING:. Codon decides hasattr by the static type,
which lacks the method, so it compiles none of the calls: the check covers
the lookup around them, not the calls (RoutingAction.execute()).
- lanelet2's map at the model boundary: the checked MapManager hands
out its lanelet_map and routing_graph as the lanelet2 model's classes
(above), which the checked RouteFrame (route.frame) takes, so the
checked scenario_route_frame() (route.active) builds it from them.
- A model shared with checked modules is split out: the model's
EgoConfig derives from VehicleEntityConfig, and the checked
EgoVehicle.spawn() hands its spawn location to the checked
spawn_vehicle_actor(). So the spawn locations and the NPC vehicle live
in a model module of their own (_vehicle_entity.codon), which
REPLACED_MODELS replaces with entity._spawn and
entity.vehicle_entity: the model's EgoConfig then holds the checked
SpawnLocation. maps.cache is itself checked, and the library check compiles its source; its
model (maps/cache.codon) stays for the scenario check, which imports models
and not sources, and takes and returns a MapSource from a model module
of its own (_map_source.codon), which REPLACED_MODELS replaces with the
checked maps.source, so the checked maps.resolver hands the model's
GitMapCache its own MapSource. Likewise GroundProjectionConfig (_ground_projection.codon,
replaced with coordinate.snap): the example scenarios import it from the
top-level package, which is the model, and hand it both to the model's
BaseScenario and to the checked VehicleEntityConfig (#99). The
top-level package model itself is not replaced: the examples derive from
the model's BaseScenario, so their constructors' calls to
super().__init__() compile the model's constructor. The checked
BaseScenario.__init__ is compiled by the library check's own call, with
its defaults; the one line that stores the ego's class (ego_type) is
under if not TYPE_CHECKING: (#199). The model of
ros2.cyclone (ros2/cyclone.codon, cyclonedds) declares the
CycloneRos2Transport constructor that the checked
entity.ros2_entity._cyclone_transport() calls; its base Ros2EgoTransport
and its Ros2EgoConfig come from a model module of their own
(_ros2.codon), which REPLACED_MODELS replaces with the checked
ros2.transport and ros2.config, so the transport _cyclone_transport()
returns is the checked Ros2EgoTransport, and its body is compiled. For the
scenario check, which has no ROS 2 package, the two are stand-ins whose
constructors are a compile_error, as for a class a checked package only
re-exports (below).
- A class a checked package only re-exports is declared in the model by
name and constructor: GrpcAutowareBridgeServer
(autoware_bridge/grpc_server.codon, grpc), EgoDriverGrpcClient
(driver/egodriver_client.codon, grpc) and CarlaDriverEntity
(entity.codon, grpc and carla_driver_interface's protocol), which the
checked autoware_bridge, driver and entity packages re-export and no
checked module builds or uses, and CoverageReport (coverage.codon,
openodd's), which the checked coverage package re-exports with
merge_coverage(documents), the COVERAGE_SCHEMA string and the
CoverGroup enum (ODD, SITUATION, SCENARIO), and T4SceneExport
(t4_scene.codon, Protobuf messages), which the checked t4_scene package
re-exports with the suffix strings and map_resource() and
trajectory_log_to_t4_scene(), functions whose bodies are a
compile_error too, and PackageExistsError and ScaffoldResult
(scaffold/generator.codon, argparse and jinja2), which the checked
scaffold package re-exports with create_scenario_package(),
resolve_names() and main(), compile_errors as well: they write a new
scenario package (scenario-new), not a scenario (PackageExistsError is
the checked scaffold.placement's, #2144, which scaffold.generator
re-exports; the package still imports it from scaffold.generator, and
scaffold/generator.codon still declares it, so a scenario importing it
from either finds the compile_error it did), and
CommonRoadSettings (commonroad/session.codon) and CommonRoadWorker
and CommonRoadWorkerError (commonroad/worker.codon, worker processes
spoken to in JSON), which the checked commonroad package re-exports with
configure_commonroad_metrics(), commonroad_settings(),
evaluate_commonroad(), close_commonroad_workers(), worker_command()
and the WORKER_ENV string, compile_errors too: the runner evaluates
the CommonRoad metrics after a run, not a scenario
(CommonRoadSettings leaves out workers and heavy_metrics, whose
defaults the machine and the environment decide: _OMITTED_PARAMETERS;
CommonRoadWorkerError takes *args and **kwargs, as Python's hands
both on to RuntimeError's; it, worker_command() and WORKER_ENV are
the checked commonroad.worker_start's, which commonroad.worker
re-exports and the package imports from there; commonroad/worker.codon
still declares them, for a scenario importing them from
commonroad.worker), and DeviceSpec, FieldSource, HilsConfig and
MessageSpec (hils/config.codon, yaml and a document
of any shape), which the checked hils package re-exports with
load_config() and parse_config(), compile_errors too: scenario-hils
reads the HILS bridge's configuration on the HILS PC, not a scenario (the
package imports SensorSpec and HilsConfigError from the checked
hils.sensor_spec, which defines them). Each constructor takes
Python's parameters and is a compile_error: its config
(AutowareBridgeConfig, DriverClientConfig) is a checked class the model
does not have, the class is only ever built by openodd's merge, its
fields are messages the model has no type for, or only the scaffolder
builds it, so a
scenario that builds one fails its check rather than passing it with
arguments the run would refuse. Such a class derives from nothing in the
model: a subclass of the model's EgoVehicle that nothing builds breaks
Codon 0.19's code generation for every scenario (LLVM: error: value
doesn't match function result type). test_typecheck_model.py compares
each with the Python class, and checks that its constructor is the
compile_error. Of CoverageReport, which merge_coverage() returns, the
model also declares what Codon has types for (runs, scenarios,
groups(), group_grade(), to_markdown(), #298).
- A Protocol a scenario may take is modelled with what is called of it:
the library check compiles autoware_stack.launcher from its source, and
for the scenario check autoware_stack/launcher.codon declares the AutowareLauncher that
AutowareEgoEntity starts, polls and stops (start(*, name,
bridge_address), poll(), stop()), with the prepare() and close()
that ScenarioQueue calls (below), and the AutowareEpisode that
start() returns. Python cannot instantiate a Protocol, so the model's
constructor is a compile_error.
- A model function returning a checked class: to_map_frame
(coordinate.codon, which the scenario check and the checked modules that
import the coordinate package compile against) returns
autoware_bridge's BridgePose, which _bridge_pose.codon models
(BridgePose, its Vector3 and Quaternion, as @tuples, since they are
frozen, and moved_forward()); REPLACED_MODELS replaces it with the
checked autoware_bridge.base, so AutowareEgoEntity hands the pose
to_map_frame returns to the checked AutowareBridge.configure()
(it imports coordinate.transform itself, which is checked, so the library
check compiles the real to_map_frame there).
- math.hypot of the differences where CPython's math.dist would do
(AutowareEgoEntity, actions.background_traffic, where
test_background_traffic.py pins the two to the same number): Codon's
math had no math.dist when these were written; the check adds it now
(see "Standard library names Codon lacks").
- A function whose only return is return None: with an Optional[X]
return annotation, the check returns Optional[X]() and realizes X
before calling it (Writing a scenario that
type-checks, #202), so it compiles
as written, as does a local declared Optional[X] = None and returned
(_Never.check in actions.background_traffic). Without an Optional
annotation a bare None is still Codon's NoneType ("cannot typecheck
'None'"), and del of a parameter is refused.
- A method that takes **kwargs, called on a value whose class a checked
subclass overrides it in, is called through the base class under if
TYPE_CHECKING:, and as written in the else branch: actions.turn calls
BackendDriven.turn_at_junction(entity, ...) in the check, since
AutowareEgoEntity overrides it and Codon 0.19 calls an overridden method
through a dispatch thunk, which takes no keyword argument. The check calls
each override through its class.
Codon never compiles the else copy, so test_typecheck_else_branch.py
guards it: wherever the then branch calls a method through its class and
the else branch calls it through an instance, both must pass the same
arguments, the receiver aside (#193).
- A name the model lacks at the boundary: when a checked module imports a
name of a modelled package that the model does not declare
(trajectory/__init__.py re-exports ResolvedTrajectory), declare it in
the model module. When the package does not export it, add it to
_BOUNDARY in test_typecheck_model.py, so it is compared with the Python
class as an exported name is.
- A Protocol that checked code dispatches through becomes a base class:
Codon has no structural typing, and decides isinstance and hasattr by
the static type. sweeper.constraints.Constraint was a Protocol; it is
an ABC whose evaluate() is abstract (so, as before, it cannot be
instantiated itself), and every constraint of the module derives from it.
What walks a constraint tree asks each class what to bind
(_bind_set_level(), which the composite and set-level constraints
override) instead of testing the value with isinstance and hasattr. A
constraint of someone else's that does not derive from it still works at
run time: the walk takes the duck-typed path for it, in the else branch
of isinstance(constraint, Constraint), which Codon decides as true. What
a set-level constraint caches (_cached_ids) is a field(init=False,
repr=False, compare=False): declared, as Codon needs, but neither a
constructor parameter nor part of the repr or the comparisons, and unset
until the constraint is bound, as before (dataclasses.fields() now lists
it, so dataclasses.asdict() of a set-level constraint includes it, and
raises AttributeError before the constraint is bound; nothing in the
framework calls either on a constraint).
- A Protocol that is only read through (batch_report's _Result and
_Status, the results it summarises): each property and method has a
typed stub body (return "", return False, return 0.0, return
None, return list[_Status](), return dict[str, Any]()) instead of
..., which Codon types as returning None ('NoneType' object has no
attribute '__iter__'); a bare [] or {} would be a List[NoneType] or
Dict[NoneType,NoneType]. Nothing instantiates or calls the Protocol at
run time, so the bodies change nothing there; an abc.abstractmethod on a
@property is a Codon syntax error, so an ABC is no alternative. An
attribute a result may lack (rerun_recording) is a property of the
Protocol, read in try: / except AttributeError:, since Codon's
getattr() takes no default. The module imports Collection, Mapping
and Sequence from typing (with # noqa: UP035), since Codon has no
collections.abc, and writes batch_results.json as text a member at a
time, each member through json.dumps({key: value}, indent=2), since an
entry was a ** merge of the result's to_dict() (dict[str, Any]) with
its own keys. A key both set is written once, where the merge put it and
with the value it kept; test_batch_report.py holds the text to what
json.dumps of the merged entries gives.
- A registry of classes at module level (sweeper.constraints'
_LEAF_REGISTRY, sweeper.bindings' _BINDING_REGISTRY): Codon holds no
class as a value, and cannot type dict[str, type]. Declare it
dict[str, Any] and assign it in the else branch of if TYPE_CHECKING:;
only the uncalled parser reads it.
- An enum member looked up by name, or an enum iterated over
(ComparisonRule[name], list(ComparisonRule)): write it as in Python.
The check rewrites both for an enum of the package being checked and for the
framework's own enums (Iterating an enum), so
sweeper.constraints has no spelled-out member lists left.
- An exception class raised with no arguments (raise _Exhausted() in
route.search): Codon 0.19 raises no exception class without its own
__init__, and its Exception takes a message where Python's takes none.
The constructor calls super().__init__("") under if TYPE_CHECKING: and
super().__init__() in the else branch.
- An Optional chosen by a conditional expression, whose value goes to a
union field (beside = graph.left(l) if left else graph.right(l), then
BindingResult(value=beside.id), a float | int | list[int]): Codon 0.19
generated invalid LLVM IR and crashed without a diagnostic ("Call
parameter type does not match function signature", optimize.cpp:1070).
Choose with an if statement into a local annotated Optional[...].
- A function that returns a tuple with an Optional member, None in
one return and a value in another (route.frame.ego_placement()):
annotate the local in each return's own assignment (no_goal:
Optional[tuple[int, float]] = None, goal_at: Optional[tuple[int, float]]
= frame.locate(s)): one local annotated Optional where it is assigned
None, and assigned the value later, was not enough.
- A loop variable assigned a subclass's value is retyped: in a function
that already declared other: ConstLanelet, other = near (a Lanelet)
makes other a Lanelet, and a list of (float, ConstLanelet) then
refuses it. Give the upcast a name of its own (nearby: ConstLanelet =
near, route.geometry).
- A call of another checked module's UNCALLED function: Codon
compiles a function when something calls it, so a checked module that
calls an UNCALLED function of another compiles that function's body
through the call, and its UNCALLED entry does not keep it out. When the
other module has a model that declares the function, the caller imports
the module through the model (MODELLED_IMPORTS), and is compiled against
that declaration instead: actions.play_motion imports pedestrian.soma
through pedestrian/soma.codon, so its call of load_soma_motion(), whose
numpy.load of an .npz Codon cannot compile, compiles against the model,
while pedestrian.soma itself is checked from its source. The caller then
holds the model's types (SomaMotion), as below.
- A value of a type the model holds: a checked module that hands a checked
value to a checked module that imports it through the model
(MODELLED_IMPORTS) hands the wrong type. scenario_runner hands the
checked BaseScenario to measures.set_measured_scenario(), which takes
the model's: it makes that call in the else branch of if TYPE_CHECKING:.
- No module object to assign a global through
(vehicle_entity._warmup_done = True): Codon 0.19 imports no module as a
value. The module sets its global in a function of its own
(entity.vehicle_entity._set_warmup_done()).
- The runner's boundary is modelled: coverage.codon declares the
CoverageCollector (its constructor, start(), tick(), end(),
write(path, scenario), and the run's case and outcome,
record_case(case, space=None) and
record_outcome(passed, *, failed_at=None, environment=None); openodd's
backend and generator are refused), odd.codon resolve_odd(spec),
whose union parameter (str, Path, OddDefinition or None) is
generic and checked in its body, reset_probes(), the ODD's modules,
the case a run records (OddCase, a @tuple of openodd's OddSample and
ParameterSpace, which only the sampler builds) and what the runner reads
of odd.route's verdict on the ego's planned route (plan_route_coverage(),
RouteCoverage and RouteLanelet, which only it builds), and openodd's
UNDECIDED, which the checked odd.probes returns: the one instance of a
class openodd does not export (odd.model names it Undecided, for
annotations), whose constructor is a compile error, as Python cannot build
another, so the model allocates its one with Undecided.__new__();
environment.codon the id of the run's environment (capture() and
RunEnvironment.id), and autoware_stack/launcher.codon the launcher's
prepare() and close(), which the queue calls.
test_typecheck_model.py compares each with the Python one.
- No attribute of a union: Codon 0.19 reads pose.yaw of a value typed
as a union (a vertex's position) as no type, even when every member has
it with one type. After isinstance, name the value as the class
(relative: RelativeLanePose = position) and read that; a union reaches a
parameter typed with one of its members the same way. A generic function
handed the union compiles every branch for it, so one that returns a
different class per branch (route.positions.resolve_route_pose()) is
called with each class on its own (trajectory.resolve._place_on_route()).
- A callable of a union: Callable[[Union[A, B]], R] crashes Codon 0.19
("union already sealed"), whether a parameter's, a variable's or a
return's. Spell the callable out in the annotation (the checker drops a
Callable parameter or return annotation, so the function is generic,
and calls it with _acs_Returning[R, N]); a module-level alias
(PositionResolver = Callable[...]) stays for Python and is None to the
check, which refuses the alias as an expression ("expected type
expression"). A function that takes either a resolver it is given or one
it builds hands each to a helper of its own
(trajectory.resolve._resolve_vertices()): one variable holds one type.
- A closure calling another closure does not see that one's captures
("name 'x' is not defined"): make the inner one a module-level function
taking what it read (trajectory.resolve._ego_route_s()).
- A tuple display does not make a float the Optional[float] its
annotation wants (return world.x, world.y, world.z, world.yaw from a
function returning Tuple[float, float, Optional[float], Optional[float]]):
a call does (trajectory.resolve._resolved()).
- A singleton's __new__: Codon's constructor allocates and calls
__init__, never a class's __new__, and its object.__new__ takes no
class. The check rewrites super().__new__(cls) to C.__new__(), which
allocates a C, and a class without __init__ is built by Codon from
every attribute it declares: give it an __init__ that does nothing
(MapManager; Python calls it after __new__).
- Path.read_text() is in the pathlib shim (CPython 3.10's
encoding and errors; 3.13's newline is refused); it reads an empty
text.