Define the offline preflight boundary
This commit is contained in:
@@ -133,13 +133,6 @@ the existing operator disclosure boundary, including redaction of
|
|||||||
credential-shaped provider parameters. Do not recreate backend default
|
credential-shaped provider parameters. Do not recreate backend default
|
||||||
resolution to produce the former `effective_model_params` view.
|
resolution to produce the former `effective_model_params` view.
|
||||||
|
|
||||||
Distinguish configuration and resource preflight from request readiness.
|
|
||||||
Use `InspectRequirements` to explain recognized dependencies when helpful,
|
|
||||||
not as proof that a request is executable. Use `Prepare` or
|
|
||||||
`ValidateRequest` when concrete request inputs are available and their
|
|
||||||
validation is actually needed. A routine config-only check should remain
|
|
||||||
credential-free and offline, without synthetic requests.
|
|
||||||
|
|
||||||
Retain a private, versioned Notarius checkpoint identity based on the engine's
|
Retain a private, versioned Notarius checkpoint identity based on the engine's
|
||||||
captured state. Use native resource enumeration and prompt hashes, keeping
|
captured state. Use native resource enumeration and prompt hashes, keeping
|
||||||
the effective execution inputs needed to distinguish reuse safely. A diff of
|
the effective execution inputs needed to distinguish reuse safely. A diff of
|
||||||
@@ -148,6 +141,27 @@ Version the fingerprint for the cutover so obsolete identity inputs cannot
|
|||||||
cause a false hit; do not reread mutable sources or expose the digest inputs.
|
cause a false hit; do not reread mutable sources or expose the digest inputs.
|
||||||
Keep run manifests based on the profiles actually used, not the inventory.
|
Keep run manifests based on the profiles actually used, not the inventory.
|
||||||
|
|
||||||
|
### Offline Preflight
|
||||||
|
|
||||||
|
Keep `config validate` focused on validating the resolved Notarius application
|
||||||
|
configuration. With `--pipeline`, also resolve the selected stages and lanes
|
||||||
|
against the module catalog. For an LLM-backed selection, load the same
|
||||||
|
effective PromptKit definitions used by `run`, even when the pipeline does
|
||||||
|
not explicitly select a profile. Let PromptKit validate and capture those
|
||||||
|
definitions, including companion sources and referenced message files.
|
||||||
|
Check the prompt and profile IDs known from pipeline resolution, including a
|
||||||
|
prompt's default profile when the prompt is known and its default is the
|
||||||
|
effective selection. Do not simulate stages to discover runtime selections.
|
||||||
|
|
||||||
|
Preflight requires no transcript, credential values, or provider calls. Its
|
||||||
|
success confirms configuration and known resource selection, not that every
|
||||||
|
concrete request is runnable. A valid PromptKit profile can be partial. Let
|
||||||
|
PromptKit check the completed target, required credentials, inputs, and schema
|
||||||
|
during request preparation. Use `InspectRequirements` to explain recognized
|
||||||
|
dependencies when useful; use `Prepare` or `ValidateRequest` for an actual
|
||||||
|
request when no-generation validation is needed. Do not construct synthetic
|
||||||
|
requests or reimplement target resolution for offline preflight.
|
||||||
|
|
||||||
### Native Execution And Retained Pipeline Policy
|
### Native Execution And Retained Pipeline Policy
|
||||||
|
|
||||||
Use ordinary `Engine.Run` for structured completion and use `OnPrepared`
|
Use ordinary `Engine.Run` for structured completion and use `OnPrepared`
|
||||||
@@ -201,9 +215,12 @@ focused behavior tests for the new format and intentional rejection of the
|
|||||||
old one, main-relative companion discovery, and native relative imports.
|
old one, main-relative companion discovery, and native relative imports.
|
||||||
Verify that a companion can replace a qualified embedded prompt or profile
|
Verify that a companion can replace a qualified embedded prompt or profile
|
||||||
without changing unrelated defaults, and that preflight, inspection, and
|
without changing unrelated defaults, and that preflight, inspection, and
|
||||||
runtime use the same effective definitions. Exercise preparation,
|
runtime use the same effective definitions. Verify that pipeline preflight
|
||||||
generation, structural repair, and semantic retries through the maintained
|
catches invalid effective definitions and unknown selected IDs without
|
||||||
fake-provider integration. Check that real message or target changes affect
|
requiring inputs or credentials, and does not reject a profile merely for
|
||||||
|
being partial. Exercise preparation, generation, structural repair, and
|
||||||
|
semantic retries through the maintained fake-provider integration. Check
|
||||||
|
that real message or target changes affect
|
||||||
checkpoint reuse, and that session routing, cache controls, diagnostics, and
|
checkpoint reuse, and that session routing, cache controls, diagnostics, and
|
||||||
durable outputs keep their existing meaning. Live provider checks are useful
|
durable outputs keep their existing meaning. Live provider checks are useful
|
||||||
for deployment confidence, not a requirement of the offline suite.
|
for deployment confidence, not a requirement of the offline suite.
|
||||||
@@ -215,21 +232,6 @@ for deployment confidence, not a requirement of the offline suite.
|
|||||||
|
|
||||||
## Open Decisions
|
## Open Decisions
|
||||||
|
|
||||||
### How Far Offline Preflight Goes
|
|
||||||
|
|
||||||
**Recommendation:** Check that the captured configuration is valid and that
|
|
||||||
selected resource IDs resolve. Make clear that a valid profile can be partial
|
|
||||||
and that concrete request readiness is checked during preparation. This
|
|
||||||
preserves a useful offline preflight without duplicating target resolution or
|
|
||||||
requiring credentials or source data.
|
|
||||||
|
|
||||||
**Alternative:** Preserve offline checking of completed targets by pursuing a
|
|
||||||
dedicated upstream API that resolves targets without artifacts or credentials.
|
|
||||||
That would catch more missing endpoint/model settings before a run and could
|
|
||||||
serve other consumers, but it would add an upstream dependency to this work or
|
|
||||||
require a subsequent release. Reimplementing completion rules in Notarius
|
|
||||||
would risk a discrepancy with actual execution.
|
|
||||||
|
|
||||||
### Operator Diagnostics In This Upgrade
|
### Operator Diagnostics In This Upgrade
|
||||||
|
|
||||||
**Recommendation:** Migrate existing inspection commands and use native
|
**Recommendation:** Migrate existing inspection commands and use native
|
||||||
|
|||||||
Reference in New Issue
Block a user