Clarify PromptKit capture and preparation behavior

This commit is contained in:
2026-11-18 00:30:24 +00:00
parent 0663fe566a
commit 3f9727d5e8
5 changed files with 25 additions and 13 deletions

View File

@@ -358,8 +358,10 @@ profiles:
service_tier: null service_tier: null
~~~ ~~~
The `promptkit.local_backend` configuration registers that conventional The backend can be declared as `backends.local` in the operator source or
backend. Prompt overlays use the same recursive source composition; for registered through the optional `promptkit.local_backend` convenience object.
The [source precedence](#native-promptkit-sources-and-migration) applies when
both contribute. Prompt overlays use the same recursive source composition; for
example, this changes one output setting while retaining the other embedded example, this changes one output setting while retaining the other embedded
definition fields: definition fields:
@@ -372,8 +374,10 @@ prompts:
~~~ ~~~
Sequences such as prompt messages are replaced as a whole when overridden. Sequences such as prompt messages are replaced as a whole when overridden.
Prompt and schema definitions remain subject to final effective validation at Engine construction validates final effective prompt definitions and captures
engine construction. message files and static templates. Request preparation loads response schemas
and resolves request variables and input artifacts. Profile preflight needs no
transcript, credential value, or provider call, but can read prompt-content files.
`notarius config validate --pipeline <id>` resolves the selected pipeline and `notarius config validate --pipeline <id>` resolves the selected pipeline and
inspects every explicit effective profile without contacting a provider or inspects every explicit effective profile without contacting a provider or

View File

@@ -60,8 +60,9 @@ configuration only. Validation with a selected pipeline also builds the
effective catalog, resolves the pipeline, and verifies every explicit effective effective catalog, resolves the pipeline, and verifies every explicit effective
PromptKit profile. Selected LLM-backed input, chunk, lane, output, and validator PromptKit profile. Selected LLM-backed input, chunk, lane, output, and validator
profiles are inspected against the configured PromptKit source and backend profiles are inspected against the configured PromptKit source and backend
registrations without loading a prompt or performing generation, so an unknown registrations without preparing a request or performing generation. Engine
or invalid final effective profile fails before pipeline preparation. construction validates effective prompt definitions and captures message content;
an unknown or invalid profile fails before pipeline preparation.
Credential availability remains an execution-time concern. Pipeline listing Credential availability remains an execution-time concern. Pipeline listing
validates configuration before returning normalized, sorted identifiers. validates configuration before returning normalized, sorted identifiers.

View File

@@ -72,8 +72,9 @@ An empty request profile lets the prompt select its configured default. Before a
run begins, the CLI asks the adapter to inspect every explicit profile on the run begins, the CLI asks the adapter to inspect every explicit profile on the
resolved selected LLM-backed bindings and validators, including inherited resolved selected LLM-backed bindings and validators, including inherited
pipeline profiles. Inspection resolves the profile and its selected backend and pipeline profiles. Inspection resolves the profile and its selected backend and
target without loading a prompt, reading credentials, admitting capacity, or target without preparing a request, reading credentials, admitting capacity, or
contacting a provider, so a missing or invalid explicit profile fails before contacting a provider. Engine construction validates effective prompt definitions
and captures message files and static templates. A missing or invalid profile fails before
stage execution while a valid `api_key_env` may remain unset. Calls record the stage execution while a valid `api_key_env` may remain unset. Calls record the
profile actually selected by PromptKit. The recorder trims and deduplicates profile actually selected by PromptKit. The recorder trims and deduplicates
non-secret profile identity, provider, model, selected backend ID, and non-secret profile identity, provider, model, selected backend ID, and
@@ -95,8 +96,9 @@ relative resource roots. PromptKit validates final effective definitions at
engine construction. It does not run a separate audit that rejects a malformed engine construction. It does not run a separate audit that rejects a malformed
definition superseded by a later overlay. The D&D registrar contributes its definition superseded by a later overlay. The D&D registrar contributes its
`dnd-extraction` fallback, and maintained D&D prompts select that logical ID `dnd-extraction` fallback, and maintained D&D prompts select that logical ID
by default. A profile selecting `backend: local` requires the optional local by default. A profile selecting `backend: local` can use an operator
registration. `backends.local` declaration or the optional Notarius registration; see
[native source rules](../config.md#native-promptkit-sources-and-migration).
This PromptKit integration uses the old-path ConfigKit v0.5.0 types exposed by This PromptKit integration uses the old-path ConfigKit v0.5.0 types exposed by
PromptKit v0.12.0. Application configuration uses the distinct new-path PromptKit v0.12.0. Application configuration uses the distinct new-path

View File

@@ -117,7 +117,12 @@ Validation: `GOWORK=off go test ./internal/cli ./internal/framework/llm`.
## Finding 3 — Correct Lifecycle And Fallback Documentation ## Finding 3 — Correct Lifecycle And Fallback Documentation
Priority: P3. Category: documentation accuracy. Status: open. Priority: P3. Category: documentation accuracy. Status: complete.
Completed 2026-11-18: corrected construction/preparation timing, fallback overlay
semantics, and operator/local-backend alternatives in comments and canonical docs.
Checked against current source construction and existing passing inspection tests;
affected links and whitespace checks pass. No runtime behavior changed.
Evidence: Evidence:

View File

@@ -58,8 +58,8 @@ func (r *AssetRegistry) RegisterSchemaFS(fsys fs.FS, root string) error {
return nil return nil
} }
// RegisterFallbackProfileFS registers profile assets that PromptKit uses only // RegisterFallbackProfileFS registers lower-precedence application profiles.
// when an operator-configured source does not provide a matching profile. // Operator entries recursively overlay them, retaining omitted fields.
func (r *AssetRegistry) RegisterFallbackProfileFS(fsys fs.FS, root string) error { func (r *AssetRegistry) RegisterFallbackProfileFS(fsys fs.FS, root string) error {
if r == nil { if r == nil {
return fmt.Errorf("asset registry must not be nil") return fmt.Errorf("asset registry must not be nil")