Define the offline preflight boundary
This commit is contained in:
@@ -133,13 +133,6 @@ the existing operator disclosure boundary, including redaction of
|
||||
credential-shaped provider parameters. Do not recreate backend default
|
||||
resolution to produce the former `effective_model_params` view.
|
||||
|
||||
Distinguish configuration and resource preflight from request readiness.
|
||||
Use `InspectRequirements` to explain recognized dependencies when helpful,
|
||||
not as proof that a request is executable. Use `Prepare` or
|
||||
`ValidateRequest` when concrete request inputs are available and their
|
||||
validation is actually needed. A routine config-only check should remain
|
||||
credential-free and offline, without synthetic requests.
|
||||
|
||||
Retain a private, versioned Notarius checkpoint identity based on the engine's
|
||||
captured state. Use native resource enumeration and prompt hashes, keeping
|
||||
the effective execution inputs needed to distinguish reuse safely. A diff of
|
||||
@@ -148,6 +141,27 @@ Version the fingerprint for the cutover so obsolete identity inputs cannot
|
||||
cause a false hit; do not reread mutable sources or expose the digest inputs.
|
||||
Keep run manifests based on the profiles actually used, not the inventory.
|
||||
|
||||
### Offline Preflight
|
||||
|
||||
Keep `config validate` focused on validating the resolved Notarius application
|
||||
configuration. With `--pipeline`, also resolve the selected stages and lanes
|
||||
against the module catalog. For an LLM-backed selection, load the same
|
||||
effective PromptKit definitions used by `run`, even when the pipeline does
|
||||
not explicitly select a profile. Let PromptKit validate and capture those
|
||||
definitions, including companion sources and referenced message files.
|
||||
Check the prompt and profile IDs known from pipeline resolution, including a
|
||||
prompt's default profile when the prompt is known and its default is the
|
||||
effective selection. Do not simulate stages to discover runtime selections.
|
||||
|
||||
Preflight requires no transcript, credential values, or provider calls. Its
|
||||
success confirms configuration and known resource selection, not that every
|
||||
concrete request is runnable. A valid PromptKit profile can be partial. Let
|
||||
PromptKit check the completed target, required credentials, inputs, and schema
|
||||
during request preparation. Use `InspectRequirements` to explain recognized
|
||||
dependencies when useful; use `Prepare` or `ValidateRequest` for an actual
|
||||
request when no-generation validation is needed. Do not construct synthetic
|
||||
requests or reimplement target resolution for offline preflight.
|
||||
|
||||
### Native Execution And Retained Pipeline Policy
|
||||
|
||||
Use ordinary `Engine.Run` for structured completion and use `OnPrepared`
|
||||
@@ -201,9 +215,12 @@ focused behavior tests for the new format and intentional rejection of the
|
||||
old one, main-relative companion discovery, and native relative imports.
|
||||
Verify that a companion can replace a qualified embedded prompt or profile
|
||||
without changing unrelated defaults, and that preflight, inspection, and
|
||||
runtime use the same effective definitions. Exercise preparation,
|
||||
generation, structural repair, and semantic retries through the maintained
|
||||
fake-provider integration. Check that real message or target changes affect
|
||||
runtime use the same effective definitions. Verify that pipeline preflight
|
||||
catches invalid effective definitions and unknown selected IDs without
|
||||
requiring inputs or credentials, and does not reject a profile merely for
|
||||
being partial. Exercise preparation, generation, structural repair, and
|
||||
semantic retries through the maintained fake-provider integration. Check
|
||||
that real message or target changes affect
|
||||
checkpoint reuse, and that session routing, cache controls, diagnostics, and
|
||||
durable outputs keep their existing meaning. Live provider checks are useful
|
||||
for deployment confidence, not a requirement of the offline suite.
|
||||
@@ -215,21 +232,6 @@ for deployment confidence, not a requirement of the offline suite.
|
||||
|
||||
## Open Decisions
|
||||
|
||||
### How Far Offline Preflight Goes
|
||||
|
||||
**Recommendation:** Check that the captured configuration is valid and that
|
||||
selected resource IDs resolve. Make clear that a valid profile can be partial
|
||||
and that concrete request readiness is checked during preparation. This
|
||||
preserves a useful offline preflight without duplicating target resolution or
|
||||
requiring credentials or source data.
|
||||
|
||||
**Alternative:** Preserve offline checking of completed targets by pursuing a
|
||||
dedicated upstream API that resolves targets without artifacts or credentials.
|
||||
That would catch more missing endpoint/model settings before a run and could
|
||||
serve other consumers, but it would add an upstream dependency to this work or
|
||||
require a subsequent release. Reimplementing completion rules in Notarius
|
||||
would risk a discrepancy with actual execution.
|
||||
|
||||
### Operator Diagnostics In This Upgrade
|
||||
|
||||
**Recommendation:** Migrate existing inspection commands and use native
|
||||
|
||||
Reference in New Issue
Block a user