Refresh native configuration, CLI, and LLM internal guides

This commit is contained in:
2026-11-22 06:41:22 +00:00
parent 2c27695df9
commit 5db09d0203
4 changed files with 428 additions and 642 deletions

View File

@@ -1,204 +1,157 @@
# CLI Internals
This document describes **internal/cli**, Notarius's production composition
root. The [CLI reference](../cli.md) owns command syntax and exit statuses;
[Configuration](../config.md) owns configuration values; and
[Operations](../operations.md) owns filesystem layout, recovery, and operator
procedures.
`internal/cli` is the production composition root. [CLI Reference](../cli.md)
owns syntax, reports, and outcome codes; [Configuration](../config.md) owns
settings; [Operations](../operations.md) owns physical state and recovery.
This guide explains declaration, loading, and orchestration owners, not another
public command manual.
## Inputs, Outputs, And Boundaries
## Native Application Construction
The CLI accepts process arguments, standard streams, and injectable options
used by tests and embedding code. It writes command results to the supplied
streams and returns a process exit status. For a run, it also creates the
production catalog and runtime collaborators, hands a prepared pipeline and
source bytes to the framework, and places the logical files returned by the
runner.
[NewApplication](../../internal/cli/application.go) returns a native
`*commandkit.App` from application identity and injectable options. Construction
declares commands without configuration discovery, engine creation, provider
calls, or run-storage allocation. The executable supplies build identity;
`RunWithOptions` is a convenience execution entry point, not a manual dispatcher.
It is the only boundary allowed to compose concrete registries, LLM clients,
cache/checkpoint collaborators, debug recorders, and physical output paths.
Pipeline modules receive interfaces and request data rather than CLI streams or
filesystem roots. The [Architecture](../policy/architecture.md) defines this
composition-root boundary; [Pipeline Internals](pipeline.md) owns resolution,
preparation, and runner mechanics after their inputs are supplied.
The constructor wires:
## Dispatch And Configuration Handoff
- native help/version and installed shell completion;
- application pipeline declarations and a native run alias;
- ckconfig whole-application and focused PromptKit scopes, sharing one loader;
- ckprompt resource/requirements inspection, concrete request validation,
prepared preview, namespace selection, and explicit smoke testing.
The root dispatcher handles help, version reporting, configuration validation,
effective-configuration inspection, explanation inspection, pipeline listing,
and a pipeline run. Version reporting resolves build
information through `internal/buildinfo` before production composition, so the
diagnostic remains available without configuration or runtime collaborators.
The public syntax, streams, exit classes, and version semantics are defined by
the [CLI reference](../cli.md). Other root commands normalize injectable
options before dispatch so that a missing production dependency fails as a
command error rather than reaching execution.
ckconfig adapters use the original application result and focused native subtree
view. ckprompt uses an engine loader through its standard builders rather than
application-owned copies of inspection DTOs. Ordinary resource inspection does
not become request preparation or an implicit probe. The
[native CLI reference][native-cli] owns parser/report mechanics.
Commands that need configuration use one shared ConfigKit-backed loader. The
CLI constructs an exclusive file source for an explicit flag or environment
selection, or a native required search source, then passes it to `LoadSource`
with the requested overlay and only explicit configuration-valued run overrides.
It receives a **LoadedConfig** and uses its retained main-source path for
invocation provenance without a second search. The
configured discovery and precedence contract is in
[Configuration](../config.md), while the loading and resolution mechanics are
in [Configuration Internals](configuration.md).
## Invocation-Local Loading
`config effective` and `config explain` render the loaded ConfigKit view by
default and do not resolve pipelines or construct providers. With explicit
`--scope promptkit`, they construct the PromptKit inspection engine from the
same captured source groups used by execution. Effective output is PromptKit's
redacted YAML; explanation output shows its source history. `config profile`
uses that engine to emit one neutral Notarius-owned JSON summary for the exact
requested profile ID. These inspection commands load no run pipeline, prepare
no request, resolve no credential value, and make no provider call. Ordinary
prompt and provider-parameter content is not generally secret-free even though
credential-shaped provider parameter keys are redacted.
[nativeLoaders](../../internal/cli/configkit_load.go) belongs to one invocation.
It selects the main source and assembles one layered ConfigKit application
result. Engine construction calls `promptkit.NewFromConfig` on that result,
with runtime-only client options; it does not load YAML again.
Configuration validation without a selected pipeline checks structural
configuration only. Validation with a selected pipeline also builds the
effective catalog, resolves the pipeline, and verifies every explicit effective
PromptKit profile. Selected LLM-backed input, chunk, lane, output, and validator
profiles are inspected against the configured PromptKit source and backend
registrations without preparing a request or performing generation. Engine
construction validates effective prompt definitions and captures message content;
an unknown or invalid profile fails before pipeline preparation.
Credential availability remains an execution-time concern. Pipeline listing
validates configuration before returning normalized, sorted identifiers.
The retained selection includes supplied-value state, overlay, and override
values. Repeated identical selections share a result/engine, while distinct
comparison selections cannot be memoized into one side. There is no
process-global loader cache. The selected main's native context anchors companion
imports; source edits do not mutate an existing captured engine.
## Production Composition
[Configuration Internals](configuration.md) owns definition, normalization,
origins, and conversion. Shared loaders allocate no output/cache/debug data.
The production composition helper allocates every framework registry and the
prompt-asset registry, then registers the generic, Seriatim, and D&D module
families in that order. The resulting registries provide both the module
catalog used for resolution and the concrete constructors used for preparation.
Tests may provide a catalog or registries instead; production code must not
silently merge an injected partial catalog with production registrations.
## Catalog And Preflight Boundaries
The production LLM factory builds one PromptKit-backed client from the native
source groups assembled from PromptKit catalogs, registered application prompt
and fallback assets, the optional operator file or recursively selected
directory, and the optional local backend registration. The same construction
path serves runtime, preflight, and configuration inspection. It attaches the
profile-provenance recorder, captures one immutable PromptKit configuration
identity, creates one scheduler from the effective global LLM limit, and wraps
the client before it reaches modules. Registration and LLM construction errors
are returned before a pipeline is prepared. Configuration fields remain in
[Configuration](../config.md#promptkit-profiles); source ownership, capture,
identity, and adapter mechanics remain in [LLM Runtime](llm.md).
[catalog.go](../../internal/cli/catalog.go) composes generic, Seriatim, and D&D
registrars into typed registries, codecs, evidence projectors, and validator
defaults. Registration does not build an asset/schema registry or an LLM service.
Injected registries/catalogs are alternative sources, not implicitly augmented
with production entries.
The factory also accepts `LLMRuntimeOverrides`, whose reasoning pointer
preserves inherit, replace, and clear states across the composition boundary.
Run orchestration constructs this value from the mutually exclusive
`--reasoning-effort` and `--clear-reasoning-effort` controls. Absence preserves
a nil pointer, replacement is trimmed, and clear uses a non-nil empty string.
The same override reaches the one shared production client, checkpoint
identity, and debug invocation metadata. Persistent reasoning configuration
remains owned by PromptKit profiles; Notarius configuration has no reasoning
field.
[pipeline_commands.go](../../internal/cli/pipeline_commands.go) separates:
- listing: enumerate captured configured IDs without a catalog/engine;
- showing: domain resolution and composition description, without profile expansion;
- validating: domain resolution plus known-resource preflight, supplying the
same captured engine only when model roles require it.
[Preflight](../../internal/preflight/preflight.go) accepts collaborators/data;
it does not load configuration, register/construct modules, read execution
inputs, render CLI reports, or allocate state. Partial profiles pass native
inspection, with concrete request readiness deferred until preparation.
## Run Inputs And Native Overrides
[run_input.go](../../internal/cli/run_input.go) binds typed application inputs
and repeated literal selections. [execution_overrides.go](../../internal/cli/execution_overrides.go)
uses reusable ckprompt resource/execution inputs, preserving omission versus
explicit empty/false/zero. Only profile/session and execution controls needed
by pipeline runs are selected; prompt inputs stay module-owned.
The resolved run-wide profile selection reaches every LLM-backed role. Native
execution overrides travel through operation requests to producers, validators,
and continuation, without mutating loaded settings. The
[public reference](../cli.md#run-wide-llm-overrides) owns exact spellings/semantics.
Reference selectors are expanded at the CLI boundary using registered target
specifications. Pipeline/lane actions become exact binding changes after
specificity/conflict handling. Generic resolution receives exact reference
bindings/unbinds, not CLI selector grammar. External/generated conflicts and
required-slot/type checks remain with domain resolution; materialization follows.
[session.go](../../internal/cli/session.go) resolves the effective session from
explicit supplied input or the input-module/raw-byte policy before runner work.
Framework/model code transports it rather than deriving another session.
## Run Orchestration
After parsing and validating a run invocation, the CLI performs this ordered
handoff:
[pipeline_run.go](../../internal/cli/pipeline_run.go) performs the ordered handoff:
1. load and validate configuration with ConfigKit command overrides;
2. create and validate a safe run identity, then allocate a debug bundle only
when requested;
3. build the effective catalog, resolve requested reference changes, resolve
the effective pipeline, and inspect its explicit effective PromptKit
profiles;
4. materialize external or generated references and record redacted invocation
and resolution provenance when debug capture is enabled;
5. construct registries, the scheduled LLM client, and prepared modules;
6. read the source input once, resolve its effective session from the explicit
override or resolved input module and raw bytes, then construct requested
checkpoint collaborators and invoke the framework runner with that same
value; and
7. write the runner's logical output files only after a successful run, then
complete the command report and user-facing result.
1. Load/validate the selected application result and derive runtime configuration.
2. Generate a safe run identity and allocate requested capture.
3. Capture selected configuration, compose registries, expand reference actions,
and resolve the effective pipeline.
4. Obtain the shared engine when needed and perform offline preflight.
5. Materialize references, construct prepared typed modules, read input bytes,
and resolve the effective session.
6. Construct the accepted-result cache and optional recorder collaborator,
create one native execution group, and invoke the runner with independent
worker and execution selections.
7. Report process diagnostics, publish returned logical files only after runner
success, finalize requested capture, then submit the native completion report.
Preparation happens before source parsing, so module construction and
dependency failures cannot begin stage execution. The CLI also preserves the
framework's result and warning information when it writes summaries and the
final command result. Detailed state lifecycle, resume handling, and physical
path confinement are maintained in [Run State Internals](state.md) and
[Operations](../operations.md).
Modules receive the shared engine, operation data, and explicit collaborators,
not physical roots or CommandKit reporters. Detailed worker, assessment,
eligibility, and cache algorithms belong in [Pipeline Internals](pipeline.md);
physical publication/capture collaborators belong in [Run State Internals](state.md).
The CLI owns the versioned generated-session policy and resolves the sole
effective value before checkpoint construction. It records that value in the
final debug invocation summary when capture is enabled and passes it unchanged
to checkpoint identity and `pipeline.RunInput`. The public flag and stability
contract are defined by the [CLI reference](../cli.md#run); framework and LLM
packages only transport the supplied value.
Requested capture allocates before engine/preflight and finalizes once.
Primary operation causes and additional capture failures remain available.
Later report delivery failure does not rerun model work, remove valid artifacts,
or terminalize capture again.
The CLI also owns the scope grammar for reference flags. It enumerates the
selected chunk and lane targets from registered module specifications, expands
pipeline- and lane-scoped actions into exact stage-and-lane bindings, and
resolves overlapping bind and unbind actions by specificity before calling
configuration resolution. The generic pipeline therefore receives only exact
`ReferenceBinding` and `ReferenceUnbind` values and has no knowledge of CLI
selector syntax. Configuration resolution retains ownership of configured
external/generated conflicts, required slots, and module compatibility; file
materialization still occurs afterward.
## Native Reporting And Outcomes
For `run --json`, the CLI constructs and encodes its private run-result receipt
after a successful runner result is available, before it publishes logical
output files. It writes the prepared receipt to standard output only after
output publication and requested debug terminalization succeed. A receipt-write
failure exits with runtime status 1 and may leave partial standard-output bytes,
but the already-published output bundle remains complete and requested debug
reporting remains successfully terminalized. The CLI reports a bounded
command-owned error and does not repeat terminal reporting. The receipt remains
a CLI reporting concern rather than a framework or output-module responsibility;
its public contract is the
[native completion report](../cli.md#completion-report).
[run_result.go](../../internal/cli/run_result.go) submits one small application
value with a native text template. Native serializers render that same value.
Domain files, not a persisted report or ledger, are the deliverables.
Fallback qualification is independent of optional diagnostic visibility.
## Failure Mapping And Terminal Reporting
Warnings/advisories keep their domain classification and use the reporter's
diagnostic channel. Outcome helpers retain native configuration/preflight causes
and distinguish identifiable invalid input from operational/mixed failures.
CommandKit retains rendering/output/diagnostic delivery evidence separately from
the operation cause; quiet mode does not mask such failures.
Argument, flag, and invocation-combination failures are reported to standard
error before runtime composition and use the syntax error class. Once an
invocation is syntactically valid, configuration loading and validation,
resolution, registration, profile checks, reference materialization, module
construction, input reads, runner failures, output publication, and requested
debug handling use the runtime failure class. The public status numbers and
stream contract are defined in the [CLI reference](../cli.md#output-streams-and-exit-statuses).
The [completion report](../cli.md#completion-report) owns fields and
[stream/outcome contract](../cli.md#output-streams-and-exit-statuses) owns process
behavior. There is no universal Notarius envelope or parallel serializer.
When debug capture has been allocated, one command-state value records the
known run result. Guarded terminalization writes a success report once, or
attempts a failure report and error record once. A persistence failure is
reported in addition to the original failure and never replaces it. If a debug
path exists, failure output includes that path so the retained diagnostic data
is discoverable.
## Focused Verification
## Invariants To Preserve
- [Application tests](../../internal/cli/application_test.go) cover lazy built-ins,
shared scopes, selection evidence, and native diff outcomes.
- [Loader tests](../../internal/cli/native_load_test.go) and
[schema tests](../../internal/cli/native_schema_load_test.go) cover native
composition, captured contexts, and operator contracts.
- [Pipeline command tests](../../internal/cli/pipeline_commands_test.go) cover
listing/showing/preflight without execution inputs or state.
- [Override tests](../../internal/cli/execution_overrides_test.go) and
[run tests](../../internal/cli/pipeline_run_test.go) cover supplied values,
routing, references, sessions, and run ordering.
- [Reference tests](../../internal/cli/reference_contract_test.go) cover selector
specificity, generated conflicts, and materialization boundaries.
- [Native executable tests](../../internal/cli/native_process_test.go) cover the
real command routes and fake-provider presentations.
- [Completion tests](../../internal/cli/run_result_command_test.go) and
[recovery contracts](../../internal/cli/recovery_contract_test.go) cover retained
publication, capture/delivery failures, reuse, fallback, and native facts.
- Only the CLI composes production implementations and physical runtime roots.
- Configuration and resolved composition failures occur before module
preparation or source parsing.
- A runner's logical files are published only after a successful run.
- Production registries and a caller-supplied catalog or registries are
alternative composition sources, not an implicit mixture.
- A requested debug bundle has one terminal report attempt; its persistence
errors supplement rather than obscure the primary command error.
- User-facing flags, paths, exit codes, and configuration fields are defined
by their public documentation, not duplicated here.
Use focused CLI tests and pair with configuration/preflight tests when those
boundaries change. [Development](../development.md#validation) owns wider checks.
## Focused Tests
- **internal/cli/command_contract_test.go** covers dispatch, help, syntax and
runtime error classes, discovery, validation, and listing.
- **internal/cli/run_contract_test.go** covers the run handoff, publication,
debug reporting, and command-owned state collaborators.
- **internal/cli/production_contract_test.go** covers registrar composition,
production catalog contents, assets, and representative configuration
validation.
- **internal/cli/reference_contract_test.go** covers scoped CLI reference
expansion, specificity, bind/unbind conflicts, generated-reference
protection, origin separation, and materialization boundaries.
- **internal/cli/state_hardening_test.go** covers safe run identity, state
roots, and failure ordering.
Run **go test ./internal/cli** after changing command composition or command
behavior. Pair it with **go test ./internal/core/config** when the configuration
handoff changes.
[native-cli]: https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.7.0/docs/cli.md

View File

@@ -1,161 +1,148 @@
# Configuration Internals
This document describes the maintainer-facing configuration boundary in
**internal/core/config**. The [Configuration](../config.md) reference owns the
file format, fields, defaults, precedence contract, and selectable keys. The
[CLI reference](../cli.md) owns command syntax; this document does not redefine
either interface.
This guide explains the maintained configuration boundary and its composition-root
handoff. [Configuration](../config.md) owns source selection, syntax, fields,
defaults, and selectable modules; [CLI Reference](../cli.md) owns invocation.
[Architecture](../policy/architecture.md#configuration-and-provenance) defines
ownership. Do not infer operator contracts from private helper choices here.
## Boundary
## Owners And Handoffs
The configuration package turns a selected ConfigKit-composed YAML document
into a validated, independently owned configuration. It then resolves a
requested pipeline against a module catalog before the framework prepares or
runs anything.
| Owner | Responsibility |
| --- | --- |
| [core/config definition](../../internal/core/config/load.go) | Typed application document, native PromptKit configuration rules, application resolution/validation. |
| [CLI loaders](../../internal/cli/configkit_load.go) | Main-source selection, native underlays, invocation-local result/engine reuse. |
| [LoadedConfig](../../internal/core/config/loaded_config.go) | Retain the original result and derive independently owned runtime pipeline configuration. |
| [Config.Resolve](../../internal/core/config/effective_config.go) | Resolve a selected pipeline against the supplied catalog. |
| [preflight](../../internal/preflight/preflight.go) | Check known native resources from resolved composition and supplied engine. |
| Pipeline preparation | Construct typed execution collaborators from resolved, materialized inputs; see [Pipeline Internals](pipeline.md). |
| Boundary | Inputs | Outputs | Does not own |
| --- | --- | --- | --- |
| Loading | Selected ConfigKit source, overlay, overrides, and invocation directory | **LoadedConfig** with ConfigKit inspection metadata and a populated **Config** | Choosing the source or reporting a command result. |
| Validation | **Config** | Structural configuration errors with pipeline, lane, or binding context | Module availability, capabilities, or construction. |
| Resolution | Valid **Config**, selected pipeline and lanes, runtime reference changes, LLM override, and module catalog | **EffectiveConfig** with a **ResolvedPipeline** | Materializing reference bytes, preparing modules, execution, or filesystem state. |
| Inspection | **LoadedConfig** and an optional JSON Pointer | Redacted effective YAML or explanation text | Pipeline selection, module preparation, or provider work. |
The production flow keeps one application-owned
`*configkit.Result[config.Document]`. That result feeds runtime conversion,
whole-app/focused inspection, and `promptkit.NewFromConfig`. Rendered YAML is
an operator view, not an interchange format for constructing the engine.
The CLI describes the selected source, invokes this package, and supplies the
result to the framework. Configuration never reads an input file, constructs a
module, or creates output, cache, or debug paths. Those responsibilities remain
at their respective [CLI](cli.md), [pipeline](pipeline.md), and
[run-state](state.md) boundaries.
## Definition And Application Meaning
## Loading And Validation
`Document` contains native `promptkit.Settings` at its PromptKit field.
`Definition` composes application defaults, version policy, sensitivity,
identified step lists, resolution, and validation with `promptkit.Configure`.
Native resource rules therefore participate in the same load rather than being
validated by a separate application parser.
`Load` selects an exclusive file; `LoadSource` accepts the CLI's source and
enables imports through the same definition and callback path. ConfigKit composes
the main input, imports, and optional overlay, then expands native inheritance
before typed decoding. Explicit CLI assignments follow decoding and precede
application resolution. The document definition supplies typed
defaults and structural validation; Notarius's recorded resolution changes
normalize configured paths after decode. Reference paths resolve before key
normalization moves their source coordinates. Ordinary list leaf edits are
recorded at the leaf; a changed step identity requires a covering list record.
Actual map-key moves retain structural records. Path normalization follows the source
of the surviving value, including a value copied through inheritance, rather
than the inheritance directive's location. Inherited input remains explicit
for presence-sensitive behavior, including zero, false, and empty values;
existing field validation still governs whether the resulting values are valid.
ConfigKit's inspection history retains inheritance transfers alongside later
normalization, and effective configuration renders the normalized values used
by runtime conversion. The private typed document consumes the directive, so
no upstream inheritance type or unresolved directive reaches runtime pipeline
or module contracts. See [Configuration](../config.md#sibling-inheritance)
for the operator syntax and field rules. Only
`NOTARIUS_CONFIG` participates in discovery; retired field-level environment
variables do not participate in loading. Version 5 decoding rejects scalar
module bindings and scalar external references. ConfigKit retains redacted,
historical inspection metadata while **LoadedConfig** keeps that implementation
detail private behind effective and explanation writers. `MainSourcePath` reads
the non-imported main file from retained sources, and runtime conversion stays detached.
Application callbacks normalize pipeline identifiers and operational paths while
retaining presence/source history. The per-load state tracks original coordinates
as map keys move. Paths are resolved before normalization loses their source
coordinates. Inherited values retain their contributing origins; CLI path
assignments use invocation origin. See
[document paths](../../internal/core/config/document_paths.go) and
[pipeline document normalization](../../internal/core/config/pipeline_document.go).
**Config.Validate** checks configuration-only invariants before resolution. It
rejects incompatible profile sources, invalid state-surface values, unsupported
concurrency settings, malformed bindings and references, invalid retries, and
invalid pipeline, step, or lane structure. PromptKit local-backend validation
accepts only an absolute HTTP or HTTPS endpoint with a host and no user
information, query, or fragment, and rejects a negative local concurrency
limit. It also rejects a negative `promptkit.transient_retries` value; decoded
omission resolves to two, while explicit zero remains a distinct opt-out. Its
errors retain the closest known pipeline, lane, and binding context.
It deliberately does not require modules to be registered: that requires a
catalog and belongs to resolution.
ConfigKit owns parsing, imports, overlays, inheritance, typed composition, and
provenance. The application supplies meaning through supported rules. It does
not implement another inheritance resolver or translate retired inputs.
[The public reference](../config.md#imports-and-sibling-inheritance) owns exact
presence/null/list rules and processing order.
The exact user-selectable values and validation rules are defined in
[Configuration](../config.md). Keep document schema, source-aware normalization,
runtime conversion, and that reference aligned in the same change.
`Config.Validate` checks application-only state and pipeline structure without
requiring registered modules. Catalog-dependent compatibility belongs to
resolution. Configuration does not parse a source transcript, materialize
references, construct modules, or allocate run storage.
## Effective Resolution
## Production Source Assembly
**Config.Resolve** derives an absent extraction-worker value in an owned copy
and validates the configuration. A programmatically supplied worker value is
always explicit. It normalizes the requested pipeline ID, copies the selected
profile, and passes the non-empty command-level LLM profile override, requested
lane selection, and reference changes to the framework resolver.
`nativeLoaders.application` selects the main source and uses
`configkit.LoadLayered` with the application definition, native underlays,
selected overlay, and explicit overrides. Underlays come from bundled provider
resources, ordinary embedded application content, and main-relative operator
companions. Native source/filesystem contexts survive the load.
After module and validator selection, the resolver applies the effective
profile policy to LLM-backed bindings only: command override, binding profile,
pipeline profile, then the prompt default. Deterministic bindings remain
profile-free, and no second inheritance decision occurs during execution. The
public field definitions and precedence are owned by
[Configuration](../config.md#pipelines).
The embedded assets package supplies content, not registration behavior.
The operator companion uses a native main-relative context; it follows the
selected main rather than rediscovering the working directory. Strict peers,
optional absence, imports/destinations, and whole-record resource replacement
remain native composition. Source grammar and precedence belong in
[Configuration](../config.md#configuration-discovery-and-precedence).
The resolver retains configured `validation_policy` overrides and derives one
detached concrete terminal policy for the chunk producer and every lane's
extract, merge, and normalize producers. That field-by-field inheritance is
complete before preparation, and the effective values contribute to pipeline
and checkpoint identity; execution does not interpret configuration defaults.
Core `Load`/`LoadSource` entry points support selected sources and the same
definition/conversion for direct consumers and tests. They do not replace the
production composition root's underlay assembly. Do not accidentally introduce
a second production load by combining those paths.
The framework resolver supplies defaults, selects lanes, resolves validator
chains, checks registered module and artifact compatibility, validates module
options, and returns the fixed ordered pipeline shape. Positive validator retry
budgets require an LLM-backed selected validator; deterministic validators are
rejected during resolution. Eligible LLM-backed producer specifications also
contribute their declared correction protocol to the resolved metadata. The
resulting **EffectiveConfig** retains the selected ID, requested selection and reference
changes, a clone of the input configuration, and the resolved pipeline.
Callers may therefore retain or modify their input slices and maps without
changing the resolved result, and later consumers cannot mutate the original
configuration through the effective value. This ownership includes the nested
PromptKit local-backend value.
## Result Lifetime And Runtime Conversion
Resolution failures stop before module construction and source parsing. They
include an error path for an unconfigured pipeline, missing module, missing
capability, incompatible artifact variant, invalid option, invalid reference,
or invalid lane selection. CLI code maps these valid-invocation failures to the
runtime error class described in the [CLI reference](../cli.md#output-streams-and-exit-statuses).
`LoadedConfig.Result()` exposes the original typed result for intentional native
handoff. `FromResult` derives pipeline runtime settings without loading sources
or serializing settings again. `RuntimeConfig` returns a detached copy;
the native PromptKit configuration stays with its owning result/engine rather
than being copied into a private runtime wrapper.
## Resolved Identity And Redaction
`MainSourcePath` reads the retained non-imported main-file evidence. It does not
repeat discovery or assume the first acquired file is the main.
The framework assigns the resolved pipeline a deterministic SHA-256 digest
after defaults, lane selection, module bindings, reference bindings, validator
chains, selected correction protocols, effective LLM profiles, and artifact
schema identity have been resolved. The digest excludes
its own stored value. It identifies resolved composition rather than raw YAML
bytes, a debug payload, or all runtime state. The CLI records it as invocation
provenance before execution; cache and checkpoint identity have additional
owners in [Run State Internals](state.md).
Within one invocation, `nativeLoaders` retains results by complete
`ckconfig.Selection`, including overlay and override presence/values. Captured
override containers are copied. A repeated identical selection returns the
same result; distinct diff sides remain distinct selections. The engine is
constructed lazily once for its retained result. This is invocation state,
not a process-wide persistent loader cache.
Configuration publication uses ConfigKit's redacted effective YAML. Resolved
pipeline publication uses **RedactedResolvedPipelinePayload**, which copies
bindings and nested option containers, replaces credential-shaped values with
**[REDACTED]**, and omits materialized reference content while retaining safe
provenance. This redaction is deliberately narrow and does not authorize
recording arbitrary environment values or provider requests.
`NewFromConfig` consumes the completed result, preserving native contexts and
inspection evidence without rerunning callbacks or adding underlays. Runtime-only
HTTP/client settings and test provider options stay at engine construction.
[PromptKit's application-owned-load contract][handoff] owns the library API.
## Invariants To Preserve
## Effective Resolution And Preflight
- ConfigKit composition applies defaults, imports, an optional overlay, and
explicit overrides in one direction with retained origin metadata.
- A configuration is structurally valid before it is resolved, and a resolved
pipeline is compatible with the supplied catalog before preparation begins.
- Whitespace-normalized identifiers are unique wherever they identify a
pipeline, step, lane, worker, or reference slot.
- Runtime conversion and resolved-pipeline publication return detached data.
- The resolved PromptKit transient-retry budget is always passed to the shared
built-in client; document presence preserves an explicit zero.
- The resolved digest changes when resolved composition changes and never
includes itself.
`Config.Resolve` clones configuration, resolves an empty programmatic cache-root
selection, validates structure, and passes selected pipeline/lanes, profile/cache
overrides, and exact reference changes to `pipeline.ResolvePipeline`.
The framework resolves module/default/validator/type/reference compatibility and
concrete producer controls before execution. Input slices/maps and the original
configuration do not become mutable aliases of the effective result.
## Focused Tests
Presence-aware selections preserve explicit invocation profile clearing, cache
overrides, and zero semantic-turn controls rather than collapsing into omission.
Deterministic bindings remain
separate from LLM roles. A resolved-composition digest is not an accepted-artifact
cache key; [state ownership](state.md) is a different boundary.
- **internal/core/config/document_test.go** and
**operational_document_test.go** cover ConfigKit document loading,
normalization, imports, overlays, and overrides.
- **internal/core/config/validation_contract_test.go** covers configuration
invariants and contextual failures.
- **internal/core/config/effective_config_contract_test.go** covers defaults,
selections, overrides, resolution context, digest changes, and ownership.
- **internal/core/config/loaded_config_test.go** covers safe effective and
explanation rendering; **redaction_test.go** covers resolved-pipeline
redaction and reference-content exclusion.
Preflight consumes resolved data, the catalog, and the captured engine when model
roles are selected. It checks static initial/correction/validator dependencies and
applicable explicit/default profiles. It does not load configuration, construct
modules, read reference/input bytes, or manufacture a request to prove readiness.
Partial profiles remain valid; dynamic request selections are runtime checks.
Run **go test ./internal/core/config** after changing this boundary. Changes to
the handoff or resolved-composition semantics also need the focused framework
pipeline tests.
## Inspection And Sensitive Values
Native whole-app and focused views use retained ConfigKit evidence. Existing
effective/explanation writers delegate to that result rather than walking YAML.
Application sensitivity rules and native resource sensitivity participate in the
same definition. Source/runtime option containers must not be independently
reserialized as an alternative secret-handling mechanism.
Do not infer that arbitrary source, prompt, provider, or module-option strings
are safe to disclose. Capture uses native inspection and exact marked-secret
paths, not heuristic application-text rewriting. See [CLI composition](cli.md)
and [LLM integration](llm.md) for those consumers.
## Focused Verification
- [Document/native settings tests](../../internal/core/config/native_settings_test.go)
prove typed handoff, native rules, source-relative messages, and secret-aware views.
- [Path tests](../../internal/core/config/document_paths_test.go) and
[inheritance tests](../../internal/core/config/inheritance_test.go) cover origins,
explicit presence, overlays, and source/runtime ownership.
- [Control tests](../../internal/core/config/validation_policy_contract_test.go)
cover loaded omission/zero and resolved controls.
- [Effective resolution tests](../../internal/core/config/effective_config_contract_test.go)
cover catalog checks, selection, and detached results.
- [CLI loader tests](../../internal/cli/native_load_test.go) cover actual production
underlays, strict companions, imports, frozen messages, and distinct selections.
[Schema-load tests](../../internal/cli/native_schema_load_test.go) exercise native
operator schema preparation/execution.
Use focused configuration and CLI tests when changing this handoff. The
[development guide](../development.md#validation) owns repository-wide checks.
[handoff]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/consumers/pkg-promptkit.md#reuse-an-application-owned-load

View File

@@ -1,363 +1,186 @@
# LLM Runtime Internals
# Native LLM Integration Internals
`internal/framework/llm` is Notarius’s provider-independent structured
completion boundary. It adapts framework requests to PromptKit, bounds
provider calls, assembles registered prompt and schema assets, records selected
profiles, and redacts provider errors. The architectural boundary is defined in
[Architecture](../policy/architecture.md#llm-boundary); profile sources,
credentials, and concurrency settings belong in
[Configuration](../config.md#promptkit-profiles) and
[Configuration](../config.md#concurrency-output-cache-and-debug).
Model-executing code uses native PromptKit types directly. This guide explains
Notarius's integration points, not a private completion service or a copied
library manual. [Architecture](../policy/architecture.md#native-execution-boundary)
defines the boundary; [Configuration](../config.md#promptkit-profiles) owns
resources/settings; [CLI Reference](../cli.md#run-wide-llm-overrides) owns request
controls; [Operations](../operations.md) owns state and handling.
## Structured Completion Boundary
## Construction And Ownership
Modules and LLM-backed validators depend only on
`contracts.StructuredLLMClient`. A completion request supplies a prompt ID and
version, optional profile and session IDs, named input material, variables, and
a caller-owned decode target. The successful response returns the validated raw
structured bytes together with non-secret provider, model, profile, and token
metadata.
The [CLI loader](../../internal/cli/configkit_load.go) hands the original completed
ConfigKit application result to `promptkit.NewFromConfig`. One invocation shares
the resulting engine across initial producers, corrections, normalization, and
LLM validators. Runtime-only HTTP/client settings and an injected native
`promptkit.LLMClient` remain construction options, not application YAML.
There is no provider-neutral completion conversion or second engine per role.
The caller owns the domain behavior: it chooses the prompt, prepares inputs,
selects the private response schema, and interprets the decoded result. The
adapter does not own source evidence, artifact conversion, normalization, or
durable schemas. Those responsibilities remain with the module and its
[integration contract](../integrations/).
[Configuration Internals](configuration.md) owns native definition/source assembly
and invocation selection reuse. PromptKit owns captured resources, request
preparation, native validation/repair, provider transport, conversation state,
and backend admission. See the [upstream Go consumer contract][consumer] for those
APIs rather than reproducing their mechanics here.
The calling module also resolves contextual entity selections and attaches any
application identity; PromptKit and this adapter do not own entity identity.
Application contracts remain typed production, candidate assessment, supported
fallback, and artifact encoding. Pure domain models, codecs, source resolution,
and deterministic transforms do not need PromptKit merely because a surrounding
operation invokes a model.
`PromptKitClient` validates the request target and prompt identity, maps each
named material to a PromptKit inline artifact while preserving its origin URI,
passes the supplied request session through to PromptKit's direct per-run
session field, retains the same value as the `session_id` prompt variable for
maintained prompt compatibility, and forwards profile selection. It does not
derive or replace session values; the CLI owns that policy. It then creates one
frozen prepared execution, captures its caller-owned credential-redacted
details for debug material, and executes that exact snapshot through
PromptKit's prepared-execution boundary. The direct field
is authoritative for provider session behavior. A session ID is a stable,
non-secret correlation identifier and may be exposed to providers and provider
observability. The adapter returns PromptKit’s validated raw bytes rather than
re-encoding the decoded target. An empty optional material is represented as
one space so its named input is retained by PromptKit.
## Requests And Fixed Operation Inputs
When a request includes semantic correction material, the adapter validates and
defensively copies it before preparation, then appends exactly two messages
after the ordinarily rendered prompt: the prior response as an assistant
message and the correction guidance as a user message. Requests without a
correction do not add messages or introduce caller roles. Ordinary request
summaries record correction byte counts and digests only; complete messages are
available solely in an explicitly requested debug trace.
Model modules receive the shared engine during typed preparation. A module
prepares transcript, references, contextual identity projections, and any
normalization candidate mapping once. It constructs a `promptkit.RunRequest`
with its selected prompt, profile/session, native artifact inputs, and variables.
Response interpretation consumes `*promptkit.RunResult` and produces the
application's typed candidate.
The adapter leaves the ordinary rendered message prefix, named inputs,
variables, session, profile, execution overrides, prepared-execution path, and
PromptKit repair policy unchanged for a corrected request. It never imports a
PromptKit message type into a module or pipeline contract. PromptKit reports
actual structural repair count and cumulative token usage per completion; the
pipeline's safe terminal debug record projects those values without copying
message content.
[Spell extraction](../../internal/modules/dnd/extract/spells/extractor.go) is a
representative path: fixed transcript/catalog/NPC projections feed an initial
request; native output is decoded and canonically mapped into its durable type.
The correction builder selects a paired prompt and supplies current guidance,
not another copy of original source/reference material.
Client construction may also receive a run-wide reasoning-effort override from
the CLI factory boundary. The adapter copies the caller-owned pointer and
creates a fresh PromptKit execution override for each request: a nil pointer
inherits the selected profile, a non-empty value replaces it, and an empty
value clears inherited reasoning. The CLI's mutually exclusive
`--reasoning-effort` and `--clear-reasoning-effort` controls select those
states. With neither flag, profile behavior remains unchanged. Because
production constructs one shared client, the selected state applies uniformly
to module calls, retries, and LLM-backed validators for the whole run.
Domain identity stays deterministic. Contextual selectors and request-local
normalization handles resolve against fixed supplied records; models do not
acquire authority over durable IDs, citation meaning, or typed artifact shape.
[Module Internals](modules.md) and [D&D Internals](dnd.md) own extension/domain
recipes.
An empty request profile lets the prompt select its configured default. Before a
run begins, the CLI asks the adapter to inspect every explicit profile on the
resolved selected LLM-backed bindings and validators, including inherited
pipeline profiles. Inspection resolves the profile and its selected backend and
target without preparing a request, reading credentials, admitting capacity, or
contacting a provider. Engine construction validates effective prompt definitions
and captures message files and static templates. A missing or invalid profile fails before
stage execution while a valid `api_key_env` may remain unset. Calls record the
profile actually selected by PromptKit. The recorder trims and deduplicates
non-secret profile identity, provider, model, selected backend ID, and
effective reasoning values for manifest use. Entries that differ in backend or
reasoning remain distinct and deterministically ordered. Endpoint-only profiles
retain an empty backend ID, which the published JSON omits. Successful
completion responses and recorded profile manifests identify the adapter
provider as `promptkit`.
Native request overrides preserve supplied values without rewriting configured
profiles. The producer helper's request composition preserves operation-owned
prompt/input/capture fields and existing callbacks; validators pass native
execution options explicitly on their own requests.
The CLI's profile-inspection engine and production adapter use the same source
assembly: PromptKit catalogs, one application group containing registered
prompts and fallback profiles, the selected operator file or directory, and
the optional final `local` backend registration. Execution, preflight, and
inspection therefore capture the same effective definitions. PromptKit and
ConfigKit compose recursively across groups while retaining each input's file
context; inputs within a group compose strictly. Operator imports are enabled
for the selected operator input, and imports retain their declaring file's
relative resource roots. PromptKit validates final effective definitions at
engine construction. It does not run a separate audit that rejects a malformed
definition superseded by a later overlay. The D&D registrar contributes its
`dnd-extraction` fallback, and maintained D&D prompts select that logical ID
by default. A profile selecting `backend: local` can use an operator
`backends.local` declaration or the optional Notarius registration; see
[native source rules](../config.md#native-promptkit-sources-and-migration).
## Native Production And Application Assessment
This PromptKit integration uses the old-path ConfigKit v0.5.0 types exposed by
PromptKit v0.12.0. Application configuration uses the distinct new-path
ConfigKit package; those named types do not cross the adapter boundary.
[RunProducer](../../internal/framework/pipeline/producer.go) connects ordinary
module interpretation/correction/fallback functions to the pipeline assessor.
It uses native `Run`, or retained prepared execution on the cache path, then
checks errors and public validation status before interpretation. Failed native
validation can accompany nil Go error and is not a usable semantic candidate.
Each engine captures its definitions at construction. A later source-file edit
does not mutate an existing engine; constructing a new engine captures the
updated view. Before execution, the adapter computes one immutable,
versioned `promptkit_configuration` checkpoint identity from public captured
inspection APIs. The private digest covers captured effective configuration,
all effective profile targets and credential-policy metadata (including
provider parameters), and every effective prompt ID/version and captured
content hash. The identity also includes PromptKit and the selected OpenRouter
and Rakestrawhome catalog versions. Local-backend concurrency settings remain
excluded because they affect scheduling; its endpoint is represented only in
the private digest, never as checkpoint metadata. Neither the private digest
payload, raw sources, paths, endpoints, provider parameters, nor prompt bodies
are published. An existing engine retains its identity after files change; a
new engine gets the identity of its newly captured definitions. This replaces
the former live-file profile and local-endpoint fingerprints. Run manifests
remain independent and list only profile choices actually observed during LLM
calls.
Every usable initial/revised candidate receives the full applicable configured
assessment chain once, including unchanged candidates and the final permitted
response. Native coordination owns history and additional-turn iteration.
Its initial callback uses already collected findings instead of reassessing the
initial result. Module templates own correction assignments; detecting modules/
validators own actionable guidance, separately from diagnostics.
## Shared Provider-Call Limit
Native execution success is not domain acceptance. Stopped/exhausted outcomes
are control facts. Result-less failed continuation does not recover an earlier
candidate, and an undelivered response is not assessed or accepted. Caller
cancellation stops without fallback. Native timeout/provider/capacity failures
remain distinguishable through context state and typed causes.
Production construction creates one PromptKit client and wraps it in one
scheduled client. The scheduler has a fixed, positive permit limit, serves
queued calls in FIFO order, and removes a queued call when its context is
cancelled. It rechecks the caller context after admission and before dispatch.
A granted permit is released exactly once on every completion path.
Resolved producer mode chooses terminal handling. A supported permissive fallback
must preserve intrinsic/source/downstream invariants without repeating ordinary
assessment or claiming semantic approval. Preparation/capture/encoding failures
are not reasons to extend model budgets. Detailed assessment/disposition and
handoff eligibility belong in [Pipeline Internals](pipeline.md).
The scheduled wrapper surrounds every `CompleteStructured` call, so concurrent
lanes, pipeline retries, and LLM-backed validators share the same provider-call
ceiling. This ceiling is independent of pipeline worker concurrency; changing
worker counts cannot exceed the configured LLM limit. The configuration field
and its effective default are owned by
[Configuration](../config.md#concurrency-output-cache-and-debug).
## LLM Validators
PromptKit applies a second, independent admission limit when the selected
profile names a limited backend. It sits beneath the Notarius scheduled client,
so it may narrow but cannot expand the application-wide limit. Built-in
OpenRouter profiles select PromptKit's reserved backend and its upstream
capacity policy. A positive configured local-backend limit bounds active local
generations inside PromptKit; zero leaves that backend unlimited there.
Endpoint-only profiles do not select a PromptKit backend and remain limited
only by the Notarius scheduler.
The [combat-semantics validator](../../internal/modules/dnd/validate/scenedescriptions/combat_semantics/validator.go)
builds one native request with its own profile and scene-local input. It uses
the same engine and run-wide native execution/session selections, then checks
the native error/status before decoding.
Successful interpretation returns a domain judgment and actionable rejection
guidance where appropriate. Execution or interpretation failure is unavailable
assessment with retained causes, not a semantic defect or invented instructions.
Cancellation delivers no judgment. Validators do not coordinate another semantic
conversation or rerun themselves to extend structural repair.
A completed producer/continuation releases native admission before assessment,
so a nested validator can use that same limited backend. Application workers
and native admission remain independent; there is no second scheduler or queue.
Capacity handling follows producer mode, not an additional application retry.
## Prompt And Schema Assets
An `AssetRegistry` collects prompt, schema, and optional fallback-profile
filesystems from production module families. It retains each registered root's
filesystem context when constructing PromptKit's application source group.
Schema files are supplied separately through the engine's schema filesystem;
PromptKit reads and validates a selected schema while preparing that request,
not while constructing the configuration snapshot. Prompt and fallback
definitions are validated after native source composition, and referenced
prompt content is captured with the engine. Checkpoint identity uses captured
prompt content hashes rather than rescanning live files or hashing only
content-file paths. Raw paths and bytes are never included. The framework’s
`promptfs` helper combines
module-selected prompt files with reusable domain fragments without making the
framework depend on D&D content.
[assets/notarius.yml](../../assets/notarius.yml) imports ordinary embedded
initial/correction/validator resources with retained filesystem contexts.
The assets package remains a content-only leaf. Prompt declarations own default
output format, validation, and schema selections; modules own prompt meaning
and small static resource-ID dependencies, not behavioral asset registration.
LLM-facing content is embedded once by the root `assets` package. Each consumer
uses only its scoped subtree, while the module retains ownership of its prompt
declaration, ordered manifest, private response-schema identity, and
registration. Shared D&D fragments are selected by D&D's shared implementation;
the detailed convention is in [D&D Module Internals](dnd.md). This physical
arrangement and its data-only boundary are defined by
[Architecture](../policy/architecture.md) and
[ADR-0011](../adr/0011-centralize-llm-assets.md), rather than by this runtime
guide.
Each producer pair requests a complete replacement response using its shared
schema. Enduring principles live in initial system context; the follow-up
supplies current findings and its domain assignment. Normalization applies
each complete proposal afresh to fixed originals, not a union of prior groups.
The generic registrar is the sole production registration owner for the
semantic-reconciliation default prompt and private response schema. The
domain-neutral reconciliation package also exposes only its mandatory protocol
and candidate/transcript presentation files for domain prompt manifests. D&D
registry normalizers mount those files while retaining ownership and hashing
of their D&D system message, semantic instructions, and complete prompt
declaration. The response schema is therefore registered once even though
several typed normalizers select it.
Native preparation loads selected inline/file/root schemas. Intrinsic durable
artifact schemas remain application-owned codec contracts and are not removed
with duplicate model-response schema machinery. Full source/root/presence rules
belong in [Configuration](../config.md#prompt-output-contracts-and-schemas) and
[PromptKit formats][formats].
Mounted prompt assets determine a module's fingerprint. The fingerprint hashes
only the module and shared files explicitly selected by its manifest, so an
unrelated asset does not invalidate a checkpoint. Schema loaders validate JSON,
attach identity and digest metadata, make defensive copies, and expose
diagnostics without raw schema bytes.
Native structural repair and transient retries remain upstream-owned. Notarius
does not pin omitted counts or wrap them in blanket producer/validator retries.
Application semantic allowance is a separate resolved control, not another
counter around native coordination.
Semantic-reconciliation normalizers extend this identity with the shared
response-schema digest, framework policy version, and complete limit-policy
digest. Their manifest metadata records the same content-free prompt, schema,
policy, and limit identities together with domain identity and normalization
policies. Request-local handles, source material, proposal content, and raw
asset bytes are not checkpoint metadata.
## Prepared Execution And Reuse
Private response schemas validate a model transport envelope. They are not the
durable artifact schema and should not be documented as an external wire
contract. Durable formats and compatibility rules remain in the
[integration contracts](../integrations/).
For eligible cacheable producers, the helper prepares the initial request once,
retains its details and opaque message-content hash, and either discards the
handle on a usable hit or executes that same handle with `RunPrepared`.
Unused handles are discarded on exits. Continuation retains the initial key.
## Prompt Maintenance And Backend Caching
Hits decode the accepted application artifact/plan without semantic reassessment,
exchange restoration, native execution, or historical usage. Normalizers cache
applied artifacts, not proposals. Provider-side prompt-cache hints remain native
request metadata, distinct from accepted-result identity. State layout and
eligibility mechanics belong in [Run State Internals](state.md), not another
LLM cache abstraction. The [prepared execution contract][prepared] owns native
handle lifetime and admission behavior.
Prompt message order and shared asset bytes are runtime behavior. Backend cache
reuse depends on identical preceding roles, rendered bytes, and cache-control
metadata—not merely equivalent meaning. Keep reusable shared assets
byte-identical and preserve each prompt’s declared ordering and cache controls
when editing it.
## Native Evidence And Sensitive Data
For sibling prompts that can reuse the same source material, order universal
shared context first, request source material next, and module-specific
suffixes last. Put a cache boundary at a reusable prefix that is useful to the
backend. Redundant intermediate cache boundaries do not extend that reusable
prefix and add no value.
The runner supplies one invocation execution group. Actual initial, continuation,
and nested validator executions enroll once; group observations preserve native
identity, terminal coverage, and unknown usage without copied totals or a
durable profile inventory.
Prompt-family owners may choose a different sequence when their inputs and
reuse pattern differ. The D&D family’s extraction, scene-chunking, and NPC
normalization policies are maintained in [D&D Module Internals](dnd.md#prompt-construction).
Do not add tests that enforce prompt prose; prompt tests should verify the
meaningful input placement and cache controls of the prompt being changed.
[Native capture](../../internal/framework/pipeline/native_capture.go) composes
observers/preparation callbacks rather than replacing them. Retain prepared
details cheaply during callbacks, then write at normal boundaries. Cache paths
reuse existing details; debug never triggers a second preparation or model call.
Shared callback state is synchronized. Available result-less failure facts and
unassessed undelivered responses remain distinguishable.
## Validation, Repair, And Retries
PromptKit performs prompt rendering, provider execution, and the prompt’s
structured-output validation. The adapter reports an empty result, validation
failure, empty structured body, or decode failure as
`ErrInvalidStructuredOutput`, while retaining the returned raw bytes and debug
material when they exist. Provider failures remain operational errors rather
than output-validation failures. Apart from documented context, capacity, and
invalid-output categories, provider error values and types do not cross the
adapter error chain; callers receive only a credential-redacted diagnostic.
`LLMGenerationError` preserves the provider-neutral failure kind (`transport`,
`provider_status`, `malformed_response`, or generic `generation`) and any HTTP
status. When PromptKit exhausts an eligible transient-status retry budget, it
also records that exhaustion and the attempts in that generation. Those fields
do not include repair generations, and zero attempts means unavailable rather
than proof that no request was sent.
Every execution that reaches PromptKit returns an optional request-local
`execution` summary. It contains only terminal aggregate attempts, retry and
repair counts, durations, outcome and failure classification, status, and
presence-aware token usage. Preparation failures have no summary; later local
decode failures retain the upstream terminal summary without changing it.
When PromptKit rejects backend admission before generation, the adapter maps
`promptkit.ErrCapacityExceeded` to
`contracts.ErrLLMCapacityExceeded`, retaining prompt context and a redacted
upstream diagnostic without exposing the PromptKit sentinel or capacity-error
type as a framework contract. When supplied, the normalized selected backend
ID appears only in that safe application-owned diagnostic context. A canceled
caller context takes precedence. The adapter does not retry capacity failures;
the pipeline's existing binding attempt policy sees the operational error and
decides whether to rerun the complete operation.
PromptKit executes structural repair within its structured-output flow. The
maintained production prompt manifests declare one additional repair attempt.
When a resolved binding supplies a repair value, the adapter inspects the
prompt, copies its complete output contract, changes only the repair limit, and
passes that complete replacement contract to PromptKit. This preserves the
prompt's output format, validation mode, schema, and provider structured-output
settings.
A successful repair is an ordinary successful completion, not a warning. The
adapter reports PromptKit's actual repair count and its cumulative usage
directly, without adding the initial and corrective counts again. Debug prompt
material records the configured complete contract; debug response material
records the repaired response and actual validation result. If the repair
budget is exhausted, the adapter retains the final raw bytes and debug material
and reports `ErrInvalidStructuredOutput`. Generation failures during an initial
or corrective call remain provider-neutral operational errors with the same
redaction boundary.
Structural repair does not replace pipeline retry behavior: a binding's
configured retry count reruns its complete stage attempt after an operational
or structural error, module-requested retry, or actionable semantic rejection.
An actionable module-requested retry and a validator rejection both use the
same correction payload when the producer exposes an exact latest response;
feedback-free module retries reconstruct the ordinary request without appended
messages. The pipeline owns attempt lifecycle, validation chains, and retry
diagnostics; see
[Pipeline Internals](pipeline.md#validation-retries-and-output) and the
[binding reference](../config.md#module-bindings-and-validators).
## Timeout Ownership
The caller context remains the outer cancellation authority. PromptKit applies
a positive effective generation timeout as an inner request deadline for each
built-in generation attempt; an explicit zero disables only that generation
deadline. The HTTP client timeout is a separate transport-wide cap. Notarius
forwards the caller context and does not install another timeout wrapper around
PromptKit.
The selected PromptKit profile owns generation settings. The resolved global
`promptkit.transient_retries` configuration is passed to the shared PromptKit
engine and controls additional built-in attempts for eligible transient HTTP
failures in one generation. It defaults to two; explicit zero disables those
attempts. Notarius binding retries remain outside the adapter and repeat the
complete module operation and validation chain.
Operator-facing behavior is summarized in
[Operations](../operations.md#operational-limits), and the pinned upstream
contract is identified in
[PromptKit Integration](../integrations/pkg-promptkit.md).
## Observability And Redaction
When debug recording is enabled, the pipeline decorates the shared client. The
wrapper records prepared prompt and response material, timing, selected profile
and backend, effective model parameters, and call identifiers in the run’s
debug bundle, including material available from a failed structured completion.
The profile-inspection CLI projects copied values into its Notarius-owned
neutral JSON DTO; effective configuration and explanation remain PromptKit's
redacted YAML and source-history views. Credential names and `APIKeyRequired`
metadata are configuration metadata, not credential-availability checks.
Effective parameters use PromptKit's stable lower-case JSON field names and may
include `backend_id`. For a successful completion, a debug-write failure is
surfaced; when the completion already failed, its call error remains the
result. Debug-bundle location, retention, and handling are operational concerns
documented in [Operations](../operations.md#debug-bundles).
The attempt-terminal summary is a separate safe trace record: it contains
attempt kinds, validator outcome counts and reason codes, effective policy,
terminal action, and repair/usage references. It excludes raw assistant
responses and correction text. Those values can appear only in the explicitly
requested detailed prompt and response artifacts, which require sensitive-data
handling.
Run manifests receive selected profile summaries, including optional effective
backend and reasoning provenance, and component identities—not prompt, schema,
source, reference, or response content. The published field semantics belong
to the [JSON output contract](../integrations/json-output.md#manifestjson).
Provider error text is wrapped with prompt context and bearer credentials are
redacted before it crosses the runtime boundary. Known-secret redaction is
available to other runtime collaborators; it does not make prompt or response
contents safe for general logging.
Generation failures expose an application-owned category and optional HTTP
status. Provider code, type, and message remain debug-only, after redaction.
## Failure Boundaries
- Construction fails for missing asset registries, mutually exclusive profile
sources, invalid asset registration, or a non-positive scheduler limit.
- Preparation failures, unavailable explicit profiles, provider failures, and
context cancellation propagate to the calling stage with context.
- Backend admission exhaustion is a provider-neutral operational error and is
not classified as invalid structured output or validator rejection.
- Malformed or schema-invalid provider output is classified separately as
invalid structured output so the module or pipeline can apply its own retry
and rejection policy.
- Domain semantic checks, evidence decisions, and deterministic normalization
run outside the provider adapter.
[Run capture](../../internal/cli/run_capture.go) uses secret-aware configuration
views and exact marked-secret omissions, while native snapshots preserve credential
omissions. Do not heuristically scrub arbitrary source/prompt/response strings.
Selected full evidence can be sensitive; private debug representations are not
a consumer serialization contract. Recording failures reach application boundaries
without retrying model work. [Operations](../operations.md#debug-bundles) owns
handling and [State Internals](state.md) owns recorder lifecycle.
## Focused Verification
Read the LLM adapter, scheduler, asset registry, schema loader, and redaction
tests when changing this boundary. Prompt changes also require the owning
module’s asset tests, and retry or debug changes require focused pipeline or
CLI coverage. The focused runtime and D&D checks are:
Use real kits with a fake native provider or local fake HTTP server, not a
replacement completion interface. Relevant owners include:
~~~sh
go test ./internal/framework/llm/... ./internal/modules/dnd/...
~~~
- [Native loading](../../internal/cli/native_load_test.go) and
[schema overrides](../../internal/cli/native_schema_load_test.go).
- [Producer/continuation tests](../../internal/framework/pipeline/producer_test.go),
[ordered assessment](../../internal/framework/pipeline/candidate_assessment_test.go),
and [nested native validation](../../internal/framework/pipeline/native_assessment_test.go).
- [Accepted-result reuse](../../internal/framework/pipeline/producer_cache_test.go)
and [native capture](../../internal/framework/pipeline/native_capture_test.go).
- [Representative extraction](../../internal/modules/dnd/extract/spells/extractor_test.go)
and [validator execution](../../internal/modules/dnd/validate/scenedescriptions/combat_semantics/native_execution_test.go).
- [Assembled recovery](../../internal/cli/recovery_contract_test.go) and
[real executable checks](../../internal/cli/native_process_test.go).
These verify native status/causes, complete assessment, fixed inputs/history,
override propagation, cancellation/admission, reuse, and evidence. They do not
claim production model quality. Wider checks belong in
[Development](../development.md#validation).
[consumer]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/consumers/pkg-promptkit.md
[formats]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/formats.md
[prepared]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/consumers/pkg-promptkit.md#prepare-now-and-execute-the-same-snapshot-later