Standard ASR

standard_asr.compliance

The compliance test suite. Engine authors run these checks to verify their plugin before publishing; applications can also run check_entrypoints() at startup to catch broken installations early.

from standard_asr.compliance import check_entrypoints, check_streaming_param_gating

Compliance helpers for Standard ASR plugin authors.

ComplianceIssue#classSource

class ComplianceIssue

ComplianceIssue(
    level: Literal['error', 'warning'],
    code: str,
    message: str,
    model: str | None = None,
)

Single compliance issue detected during validation.

Mirrors the runtime Diagnostic shape: every issue carries a stable, machine-readable code so a CI pipeline can assert against (or whitelist) a specific category without string-matching the human-readable message -- the message is for humans and MAY be reworded, the code is the programmatic contract (the same reasoning that gives Diagnostic a code).

Attributes
NameTypeDescription
levelLiteral['error', 'warning']

Issue severity ("error" or "warning").

codestr

Stable machine-readable category identifier (for example, "entrypoint_factory_failed", "streaming_invariant:<guard_code>"). Safe to match in CI; never reworded within a major version.

messagestr

Human-readable description (for display; MAY be reworded).

model
= None
str | None

The model key the issue is attributed to, or None for registry-/environment-level issues.

ComplianceReport#classSource

class ComplianceReport

ComplianceReport(
    registry: ModelRegistry | None,
    issues: list[ComplianceIssue],
)

Aggregate result returned by the compliance check functions.

Attributes
NameTypeDescription
registryModelRegistry | None

Model registry the entry-point check ran against. The behavioral checks (check_event_sequence, check_streaming_param_gating, check_recommended_wire_format, check_sync_bridge) do not operate on a registry and pass None.

issueslist[ComplianceIssue]

Collected compliance issues.

passedbool

Return True when no errors were encountered.

iter_level#methodSource

def ComplianceReport.iter_level(
    level: Literal['error', 'warning'],
) -> Iterable[ComplianceIssue]

Yield issues matching level.

Parameters
NameTypeDescription
levelLiteral['error', 'warning']

Severity level to filter.

Returns

DEFAULT_SYNC_BRIDGE_TIMEOUT#attributeSource

DEFAULT_SYNC_BRIDGE_TIMEOUT = 5.0

SupportsCapabilities#classSource

class SupportsCapabilities(Protocol)

The one-method surface check_sync_bridge needs from an engine.

A deliberately minimal protocol instead of StandardASR: the check consults nothing but supports(), and demanding the full surface would force every caller with a partial test double (or a wrapper) to fake members the check never touches. (StandardASR itself is now strict-assignable from real plugins -- config is a read-only protocol property -- so this narrowing is about least-surface, not a typing workaround.)

supports#methodSource

def SupportsCapabilities.supports(dot_path: str) -> bool

Return whether the capability at dot_path is supported.

Parameters
NameTypeDescription
dot_pathstr

A capability dot-path.

Returns
  • bool

    True if supported.

SupportsWireRecommendation#classSource

class SupportsWireRecommendation(Protocol)

The two-member surface check_recommended_wire_format needs.

A deliberately minimal protocol instead of EngineBase: the check's subjects include structural (non-EngineBase) engines -- the standard's own promise -- and the previous EngineBase-typed signature invited calling EngineBase-only members on them (the check once called ensure_stream_format_supported, not a StandardASR member, so a fully compliant structural engine failed with a false recommended_wire_format_self_inconsistent verdict on an AttributeError).

Attributes
NameTypeDescription
propertiesBaseProperties

recommended_wire_format#methodSource

def SupportsWireRecommendation.recommended_wire_format() -> AudioFormat | None

Return the engine's recommended minimal wire format.

Returns
  • AudioFormat | None

    The recommended format, or None when none is derivable.

assert_prefix_invariant#functionSource

def assert_prefix_invariant(events: Iterable[TranscriptionEvent]) -> None

Assert a recorded stream's partials honor the frozen-prefix invariant.

Test helper for engine authors. Partials are lossy under backpressure: the base coalesces pending partials when the consumer is slow, so the partial count is non-deterministic -- the same engine may surface five partials or none purely by consumer timing. Asserting a count is therefore flaky; assert the invariant instead. This checks only the prefix invariant -- a segment's frozen prefix (text[:stable_until]) is never rewritten and stable_until never regresses -- across however many partials survived coalescing, and (unlike check_event_sequence) does NOT require a terminal event, so it also applies to a mid-stream slice. It replays events through the same runtime _LifecycleGuard the runtime uses, so the assertion cannot drift from enforcement.

Parameters
NameTypeDescription
eventsIterable[TranscriptionEvent]

The recorded events, in emission order.

Raises
  • AssertionError

    If any segment's frozen prefix was rewritten or its stable_until regressed.

check_entrypoints#functionSource

def check_entrypoints(
    registry: ModelRegistry | None = None,
    *,
    strict_discovery: bool = False,
    instantiate: bool = True,
    names: Iterable[str] | None = None,
) -> ComplianceReport

Validate that discovered entry points conform to expectations.

Environment-level invariants are reported as issues alongside per-engine ones -- never as raised exceptions -- so a single command yields one report even when discovery itself found problems:

  • Engine-identity collisions (an engine_id provided by more than one distribution, making config.engine routing ambiguous) are reported as errors here. The discovery layer only marks them (registry.shadowed_engine_ids); the compliance suite is the fail-loud layer the standard mandates for them, so a collision is a compliance failure even on a default (non-strict) run rather than a log line.
  • Invalid entry-point names surface as an error too. When strict_discovery is True, discover_models would normally raise on them; this function catches that and converts it into an error issue (so its Raises: None contract holds and a report is always returned), then re-discovers leniently so the valid engines are still checked.
Parameters
NameTypeDescription
registry
= None
ModelRegistry | None

Optional pre-discovered registry. When provided, discovery is skipped and strict_discovery is ignored (collisions are still reported from registry.shadowed_engine_ids).

strict_discovery
= False
bool

Treat invalid entry-point names as a hard discovery error (reported as an error issue here, never raised). Default False. Engine-identity collisions are reported as errors regardless.

instantiate
= True
bool

If True, instantiate zero-arg factories and verify the instance surface -- including one BEHAVIORAL probe: on an engine declaring no streaming axis, start_transcription() is called once with no arguments and MUST raise UnsupportedFeatureError (a compliant engine refuses at the capability gate before constructing anything; a returned session is never entered, but a non-compliant implementation may still run arbitrary author code in the method body). False skips instantiation and the probe with it.

names
= None
Iterable[str] | None

Restrict the PER-ENGINE checks to these model keys; None checks every discovered engine. The registry-global invariants (RuntimeParams closedness, engine-identity collisions, no-entry-points) always evaluate the whole environment -- they are environment facts, not per-engine verdicts. Pass the user's named subset here rather than filtering the report afterwards: the instance checks EXECUTE engine code (construction, a supports() sweep, the start_transcription() refusal probe -- a model load, for a cloud engine potentially a billable call), a side effect that must not be paid on a co-installed plugin the caller never named, for a verdict they are never shown.

Returns

check_event_sequence#functionSource

def check_event_sequence(
    events: Iterable[TranscriptionEvent],
    *,
    allow_empty: bool = False,
    capabilities: DeclaredCapabilities | None = None,
) -> ComplianceReport

Validate a recorded streaming event sequence against the invariants.

Behavioral check for streaming engines that is pure: it replays an already-captured event stream through the standard lifecycle/frontier guard and reports every invariant it violates, without ever instantiating or calling an engine. (Behavioral checks that would require running a model -- strict sample-rate, the input-conversion matrix, language membership -- are deliberately left to unit tests, because invoking a cloud engine from a compliance run would be a billable side effect; this one only inspects data the author already produced.)

Detected violations (each an error): an illegal lifecycle transition (partial/final after a segment is finalized/superseded; a non-closed final after final; superseding a closed segment), a non-monotonic stable_until or audio_processed_until, a rewritten frozen prefix, the full supersede invariants -- frozen-prefix preservation across a replacement (the concatenated frozen text of old_ids MUST be preserved by new_ids), old_ids that were never announced, a new_id that reintroduces an already-known segment, and an empty new_ids (pure deletion) that would destroy frozen text -- an event stream that never reaches a terminal (done / non-recoverable error) event, an empty sequence (unless allow_empty=True), and any event emitted after the session-terminal event (a terminal MUST be the last event).

The per-segment lifecycle / frozen-prefix / supersede checks are obtained by replaying the events through the same _LifecycleGuard the runtime uses, so the compliance verdict cannot drift from the runtime's enforcement. Events after the session-terminal are flagged and not replayed (they do not exist in a well-formed stream, so they MUST NOT mutate segment state).

Parameters
NameTypeDescription
eventsIterable[TranscriptionEvent]

The recorded events to validate, in emission order.

allow_empty
= False
bool

When True, an empty sequence is accepted (the rare intentional case). Default False -- an empty sequence is a violation, because a real session always emits at least a terminal event.

capabilities
= None
DeclaredCapabilities | None

When provided, additionally cross-check each event against the engine's declared streaming capabilities: a stream MUST NOT exceed what it declares -- for example, emit a non-zero stable_until while word_stability is unsupported, an audio_processed_until cursor while timestamps mode is none, words while word_timestamps is unsupported, or a speaker label (event- or word-level) while diarization is unsupported. Pass engine.declared_capabilities to catch a declaration that disagrees with the engine's actual output. None skips the cross-check.

Returns

check_provider_params_swap_safety#functionSource

def check_provider_params_swap_safety(engine: StandardASR) -> ComplianceReport

Assert an engine always rejects another engine's provider_params.

The standard makes provider_params swap-safety an unconditional MUST: a wrong-typed provider_params (the classic "switched engines, forgot to change the params model" bug) MUST raise InvalidProviderParamError independent of strict / best_effort -- it is a code bug, not a capability negotiation, so it is never silently dropped. The EngineBase template enforces this in gate_params before any audio is decoded or the model is touched, so an engine that bypassed the template and forgot the check is the gap this probe closes -- the same "bypassed the template must show up here" reasoning behind check_streaming_param_gating.

The probe calls the engine's public transcribe with a foreign ProviderParams subclass private to this module (so it can never be the engine's own declared type) and a one-sample silent input. Because provider-params validation precedes audio decoding and inference, the probe incurs no billable side effect under either policy. The contract is the same for a strict and a best_effort engine: it MUST raise InvalidProviderParamError.

Parameters
NameTypeDescription
engineStandardASR

The engine instance to exercise (any policy).

Returns
def check_recommended_wire_format(
    engine: SupportsWireRecommendation,
    *,
    model: str | None = None,
) -> ComplianceReport

Assert an engine's recommended wire format is one it would itself accept.

recommended_wire_format is the single source of truth for the minimal wire AudioFormat the standard layer opens a streaming_input session with when the application chose none -- the CLI sync-bridge runner and the streaming gating probe both rely on it. A self-inconsistent engine, whose recommended format the standard session-establishment rule rejects for its own declared Properties, would make those paths fail-loud on a format the standard layer chose rather than the application -- a silent-looking compliance trap. This closes that loop: when a format is recommended it MUST pass ensure_wire_format_supported -- the pure (Properties, AudioFormat) rule that ensure_stream_format_supported itself implements. Validating via the pure rule (never the EngineBase method) keeps the verdict correct for structural engines, which have no such method.

Parameters
NameTypeDescription
engineSupportsWireRecommendation

The engine under test. Deliberately NOT required to declare streaming_input (or any capability): the recommendation is Properties-pure and capability-blind (see recommended_wire_format), so the self-consistency round-trip holds for every engine — the protocol member is unconditionally required (spec §3.1) and the entrypoint-layer instance checks run this round-trip for EVERY successfully constructed engine (batch-only included; an output-only engine passes trivially).

model
= None
str | None

The model key (engine/model) to attribute issues to, or None for a single-engine run. In a multi-model run an unattributed issue renders as <registry> and the user cannot tell which engine failed.

Returns

check_streaming_param_gating#functionSource

def check_streaming_param_gating(engine: StandardASR) -> ComplianceReport

Assert a streaming engine gates an unsupported standard parameter.

Closes the streaming-gating bypass gap as a compliance failure rather than a silent one: the base start_transcription template runs gate_params(mode="streaming") for every engine, so a "forgot to gate" engine (one that bypassed the template) must show up here.

The check establishes a streaming session (via start_transcription, which constructs the session but does not enter its context, so no wire connection is opened) for the first standard parameter the engine does not support in streaming mode and asserts the standard contract:

  • strict policy -- the call MUST raise UnsupportedFeatureError whose param identifies the gated field;
  • best_effort policy -- the call MUST succeed, drop (or degrade) the parameter, and surface the probe's expected diagnostic (for example, unsupported_parameter_ignored) via session.diagnostics().

When the engine supports every probed parameter at the feature level, the check falls back to violating a declared sub-constraint of a supported feature (a prompt over its max_tokens budget, or a word-timestamp granularity outside the declared granularities; see _pick_sub_constraint_probe) and asserts the same strict-raise / best_effort-diagnose contract.

Legal session context. A streaming_input engine is probed with a valid wire AudioFormat taken from the engine's own recommended_wire_format (guarded like every other sync-member call), so an engine that legitimately fail-louds on a missing audio_format is not misjudged as non-compliant for obeying the standard. A streaming_output-only engine is probed with a one-sample silent audio input, but only under the strict policy: strict gating raises before the audio is decoded or the model is touched (gate order: params first, then audio), so the probe is free of the billable side effect a best_effort probe would incur by reaching the engine. A best_effort streaming_output-only engine is therefore reported as a warning skip (inconclusive) rather than driven into real inference.

Distinguishing a gating raise from "streaming unsupported". The strict contract is satisfied only by an UnsupportedFeatureError whose param equals the probed field. An engine that declares a streaming axis but never implements the hook raises an UnsupportedFeatureError with no (or a different) param; that is a capability lie, not a gating success, and is recorded as a distinct error instead of being mistaken for a clean pass.

An engine that declared streaming support yet accepts the violating parameter -- the "forgot to gate" engine that bypassed the base template -- is a compliance failure here, so the gap is loud rather than silent. An engine that raises anything other than UnsupportedFeatureError from the probe is likewise recorded as a compliance error (never re-raised), so one crashing engine cannot abort the run.

Engines that declare no streaming support, or that support every probed parameter and declare no violable sub-constraint, yield a clean (no-op) pass -- there is nothing to gate.

Parameters
NameTypeDescription
engineStandardASR

The engine instance to exercise. Its config.strict selects which branch (strict raise / best_effort drop) is asserted.

Returns

check_sync_bridge#functionSource

def check_sync_bridge(
    session_factory: Callable[[], TranscriptionSession],
    *,
    timeout: float = DEFAULT_SYNC_BRIDGE_TIMEOUT,
    model: str | None = None,
    engine: SupportsCapabilities | None = None,
) -> ComplianceReport

Drive an async engine's SyncSession from an external thread.

Implements the standard's sync-bridge mandate: a no-deadlock / no-leak test. A fresh session is created and driven synchronously from a different thread than the one that built it, feeding no audio and immediately ending input. The test asserts the session terminates (emits a terminal event and tears down) within timeout -- a deadlock or a leaked background loop/ thread shows up as a timeout.

Parameters
NameTypeDescription
session_factoryCallable[[], TranscriptionSession]

A zero-argument callable returning a fresh async TranscriptionSession (for example, engine.start_transcription bound with its arguments). The return crosses the same sync-call boundary as every protocol member: a factory handing back an awaitable (an async def start_transcription behind the CLI's canonical factory) or any non-TranscriptionSession object is reported as sync_bridge_invalid_session -- with a stray coroutine closed -- instead of being driven into SyncSession and misreported as a bridge lifecycle fault.

timeout
= DEFAULT_SYNC_BRIDGE_TIMEOUT
float

Seconds granted to EACH phase of the check independently: session establishment (session_factory() plus, on an unsupported refusal, the supports() classification probe) and the bridged drive (open, end-of-audio, drain, close combined -- also each bridged lifecycle call's submit_timeout). Both phases run under bounded daemon workers, so a hanging start_transcription or supports() is reported instead of hanging the check; worst case the check takes about twice this value. Per-phase (not a shared total) so a slow-but-successful establishment can never starve the drive join into a false "did not terminate" verdict. MUST be finite and strictly positive: <= 0 would make the wait return immediately (a false "did not terminate" verdict against a compliant engine) and inf/nan would hang the check on the very deadlock it exists to diagnose, so both are rejected loudly (the same rule the CLI's --bridge-timeout enforces at parse time). It also caps each bridged lifecycle call (forwarded as the SyncSession submit_timeout), so granting a larger budget genuinely extends slow-but-compliant _open/_close phases. This MUST exceed the engine's real _open + _close cost: a slow but compliant engine (a cloud session doing a real network handshake) is not a deadlock, so when a run reports a timeout, re-run with a larger value to tell "slow" from "stuck". The driver thread is a daemon, so a false positive (or a real deadlock) never blocks interpreter exit -- the process is not held hostage by the fault this check diagnoses.

model
= None
str | None

The model key (engine/model) to attribute issues to, or None for a single-engine run (a multi-model run needs the attribution to name the failing engine).

engine
= None
SupportsCapabilities | None

The engine the factory drives, if available (anything with a supports() method -- see SupportsCapabilities; the full StandardASR protocol is deliberately not required, so a real plugin passes without casts). Used for exactly one thing: classifying an UnsupportedFeatureError raised by session_factory() itself (session establishment). Only an engine that does NOT declare streaming_input earns the passing sync_bridge_not_applicable verdict -- the bridge feeds bare frames, which such an engine genuinely cannot accept. An engine that DECLARES streaming_input yet refuses establishment is a capability lie (a declared-but-unimplemented hook, or a recommended wire format its own guard rejects) and FAILS. Without engine the classification is fail-closed: an establishment refusal is reported as a failure, with a hint to pass engine= when the engine is genuinely output-only.

Returns
Raises
  • ValueError

    If timeout is not finite or not strictly positive (a caller code bug, rejected independent of any policy).

check_transcription_result#functionSource

def check_transcription_result(
    result: TranscriptionResult,
    *,
    capabilities: DeclaredCapabilities,
) -> ComplianceReport

Cross-check a recorded batch result against the declared capabilities.

Behavioral check for batch engines that is pure, mirroring check_event_sequence: it inspects a result the author already produced -- it never instantiates or calls an engine (invoking a cloud engine from a compliance run would be a billable side effect). In v1 it verifies the diarization couple: a result carrying speaker labels anywhere (top-level segments[] / words[], a segment's words, or any channels[i] view) while batch.diarization is unsupported (or the batch domain is absent -- fail-closed) is a capability⇄result desync, reported as an error with code result_exceeds_diarization.

Honest scoping note: this is the only diarization coverage the suite can offer. Positive diarization behavior -- labels present when requested, correctly attributed, consistent across views -- is unverifiable without multi-speaker audio fixtures, and the standard probes feed silence. The check is therefore an opportunistic negative cross-check an author runs over their own recorded results; an always_on engine necessarily declares supported=True (the model validator forbids the contradictory pair) and is never flagged here.

Parameters
NameTypeDescription
resultTranscriptionResult

The recorded result to validate.

capabilitiesDeclaredCapabilities

The engine's declared capabilities (pass engine.declared_capabilities).

Returns

prepare_requires_arguments#functionSource

def prepare_requires_arguments(prepare: Callable[..., object]) -> bool

Return whether a prepare() warm-up hook needs caller-supplied arguments.

A warm-up hook MUST be invocable with no arguments. A parameter makes the hook non-conforming only when it is required: it has no default and is positional-or-keyword, positional-only, or keyword-only. *args and **kwargs impose no required argument, and a bound method's self is already supplied, so neither counts.

This is the single definition of the zero-argument half of the contract, shared by _check_prepare_hook and the standard-asr prepare CLI command so the compliance verdict and the runtime behavior cannot drift.

Parameters
NameTypeDescription
prepareCallable[..., object]

An engine's prepare attribute, already confirmed callable.

Returns
  • bool

    True when calling prepare() with no arguments would fail because a

  • bool

    required parameter is unfilled; False for a valid zero-argument hook

  • bool

    (or one whose signature cannot be introspected).

validate_bridge_timeout#functionSource

def validate_bridge_timeout(timeout: float) -> float

Validate a sync-bridge timeout: MUST be finite and strictly positive.

The single owner of the rule, shared by check_sync_bridge and the CLI's --bridge-timeout parser (which wraps the ValueError into an argparse usage error) so the two layers can never drift: <= 0 yields an instant false "did not terminate" verdict against a compliant engine, inf/nan hangs the check on the very deadlock it diagnoses, and a finite value above threading.TIMEOUT_MAX would blow up as an OverflowError out of Thread.join / Future.result mid-check -- a validated timeout MUST be one the bridge's waits can actually take. No clamping: silently shortening a caller's timeout would be an implicit rewrite of an explicit value.

Parameters
NameTypeDescription
timeoutfloat

The candidate timeout in seconds.

Returns
  • float

    timeout unchanged.

Raises
  • ValueError

    If timeout is not finite, not strictly positive, or exceeds this platform's threading.TIMEOUT_MAX.

On this page