Define the offline preflight boundary

This commit is contained in:
2026-11-18 15:28:38 +00:00
parent 63e26dc203
commit d1d2394729

View File

@@ -133,13 +133,6 @@ the existing operator disclosure boundary, including redaction of
credential-shaped provider parameters. Do not recreate backend default
resolution to produce the former `effective_model_params` view.
Distinguish configuration and resource preflight from request readiness.
Use `InspectRequirements` to explain recognized dependencies when helpful,
not as proof that a request is executable. Use `Prepare` or
`ValidateRequest` when concrete request inputs are available and their
validation is actually needed. A routine config-only check should remain
credential-free and offline, without synthetic requests.
Retain a private, versioned Notarius checkpoint identity based on the engine's
captured state. Use native resource enumeration and prompt hashes, keeping
the effective execution inputs needed to distinguish reuse safely. A diff of
@@ -148,6 +141,27 @@ Version the fingerprint for the cutover so obsolete identity inputs cannot
cause a false hit; do not reread mutable sources or expose the digest inputs.
Keep run manifests based on the profiles actually used, not the inventory.
### Offline Preflight
Keep `config validate` focused on validating the resolved Notarius application
configuration. With `--pipeline`, also resolve the selected stages and lanes
against the module catalog. For an LLM-backed selection, load the same
effective PromptKit definitions used by `run`, even when the pipeline does
not explicitly select a profile. Let PromptKit validate and capture those
definitions, including companion sources and referenced message files.
Check the prompt and profile IDs known from pipeline resolution, including a
prompt's default profile when the prompt is known and its default is the
effective selection. Do not simulate stages to discover runtime selections.
Preflight requires no transcript, credential values, or provider calls. Its
success confirms configuration and known resource selection, not that every
concrete request is runnable. A valid PromptKit profile can be partial. Let
PromptKit check the completed target, required credentials, inputs, and schema
during request preparation. Use `InspectRequirements` to explain recognized
dependencies when useful; use `Prepare` or `ValidateRequest` for an actual
request when no-generation validation is needed. Do not construct synthetic
requests or reimplement target resolution for offline preflight.
### Native Execution And Retained Pipeline Policy
Use ordinary `Engine.Run` for structured completion and use `OnPrepared`
@@ -201,9 +215,12 @@ focused behavior tests for the new format and intentional rejection of the
old one, main-relative companion discovery, and native relative imports.
Verify that a companion can replace a qualified embedded prompt or profile
without changing unrelated defaults, and that preflight, inspection, and
runtime use the same effective definitions. Exercise preparation,
generation, structural repair, and semantic retries through the maintained
fake-provider integration. Check that real message or target changes affect
runtime use the same effective definitions. Verify that pipeline preflight
catches invalid effective definitions and unknown selected IDs without
requiring inputs or credentials, and does not reject a profile merely for
being partial. Exercise preparation, generation, structural repair, and
semantic retries through the maintained fake-provider integration. Check
that real message or target changes affect
checkpoint reuse, and that session routing, cache controls, diagnostics, and
durable outputs keep their existing meaning. Live provider checks are useful
for deployment confidence, not a requirement of the offline suite.
@@ -215,21 +232,6 @@ for deployment confidence, not a requirement of the offline suite.
## Open Decisions
### How Far Offline Preflight Goes
**Recommendation:** Check that the captured configuration is valid and that
selected resource IDs resolve. Make clear that a valid profile can be partial
and that concrete request readiness is checked during preparation. This
preserves a useful offline preflight without duplicating target resolution or
requiring credentials or source data.
**Alternative:** Preserve offline checking of completed targets by pursuing a
dedicated upstream API that resolves targets without artifacts or credentials.
That would catch more missing endpoint/model settings before a run and could
serve other consumers, but it would add an upstream dependency to this work or
require a subsequent release. Reimplementing completion rules in Notarius
would risk a discrepancy with actual execution.
### Operator Diagnostics In This Upgrade
**Recommendation:** Migrate existing inspection commands and use native