Document PromptKit configuration ownership and capture

This commit is contained in:
2026-10-04 14:12:11 +00:00
parent 169a94f7ea
commit 4aba4f5672
6 changed files with 215 additions and 62 deletions

View File

@@ -1,6 +1,7 @@
# ADR-0010: Use workload-oriented LLM profile defaults
**Status:** Accepted
**Status:** Accepted; profile source replacement behavior superseded by
[ADR-0019](0019-adopt-native-promptkit-configuration.md)
**Date:** 2026-08-03
## Context
@@ -23,11 +24,13 @@ intentional exceptions, and the run-wide CLI profile override has highest
precedence.
The D&D family owns an embedded fallback profile named `dnd-extraction`.
Operators may provide a complete profile with the same ID through a PromptKit
filesystem source. PromptKit selects the higher-precedence matching definition;
Notarius does not merge profile documents. Production, development, and local
deployments can therefore use different execution policy behind one unchanged
pipeline ID.
Operators may configure the same ID through a PromptKit filesystem source. The
original decision specified that an operator definition replaced the fallback
as a complete profile; that source-composition clause has been superseded by
[ADR-0019](0019-adopt-native-promptkit-configuration.md), which adopts native
recursive overlays. The workload-oriented identity and D&D ownership decision
remain in force. Production, development, and local deployments can therefore
use different execution policy behind one unchanged pipeline ID.
## Alternatives considered
@@ -42,8 +45,8 @@ pipeline ID.
## Consequences
Pipeline configuration expresses workload intent rather than a specific
provider or model. Operators can replace the complete execution policy without
editing bindings, while binding-level and run-wide exceptions remain available.
Profile changes affect resolved pipeline and checkpoint identity, so they may
provider or model. Operators can configure execution policy without editing
bindings, while binding-level and run-wide exceptions remain available. Profile
changes affect resolved pipeline and checkpoint identity, so they may
intentionally cause work to be recomputed. The D&D fallback becomes a
maintained application execution-policy asset.

View File

@@ -0,0 +1,92 @@
# ADR-0019: Adopt native PromptKit configuration sources
**Status:** Accepted
**Date:** 2026-10-04
## Context
PromptKit v0.12 replaces its former source options with ConfigKit-backed
configuration sources, keyed documents, recursive overlays, and native
inheritance. Notarius must choose how application-owned prompt and fallback
assets, operator configuration, and the optional local backend reach PromptKit
without recreating source parsing or inheritance in the adapter. Runtime,
preflight, and operator inspection must agree about the effective definitions.
Notarius also captures checkpoint identity for the PromptKit configuration
used by an execution engine. That identity must follow the engine's captured
definition state rather than re-reading mutable source files, and must not
publish credentials, prompt bodies, provider parameters, or source paths.
## Decision
Use PromptKit's public v0.12 native source APIs as the only PromptKit
configuration boundary. Assemble PromptKit catalogs, registered application
prompt and fallback inputs, the optional operator file or recursively selected
directory, and the optional Notarius local backend registration in that order.
Use native ConfigKit imports and PromptKit/ConfigKit composition behavior;
retain each filesystem input's resource context. Inputs within a group compose
strictly, and later groups recursively overlay earlier ones. PromptKit owns
definition parsing, exact-key `extends` inheritance, and final effective-value
validation. Notarius does not provide a legacy parser, compatibility alias,
filename-priority merger, whole-profile replacement shim, or alternate
inheritance engine.
Execution, explicit-profile preflight, and configuration inspection use the
same source construction. Each constructed engine captures its effective
definitions. Edits to source files take effect for a newly constructed engine,
not an existing one. Derive the private versioned
`promptkit_configuration` checkpoint identity from public captured inspection
and definition APIs, including effective configuration, profile targets and
credential-policy metadata, captured prompt hashes, and the PromptKit and
selected catalog identities. Retain only the digest as checkpoint metadata;
keep all digest inputs private. Exclude local-backend concurrency settings
because they affect scheduling rather than execution semantics.
Keep PromptKit types and source assembly inside the LLM integration boundary.
The CLI may project copied profile fields into a Notarius-owned neutral JSON
contract. PromptKit's effective and explanation reports remain upstream
redacted configuration and source-history views. Neither inspection path reads
credential values or contacts a provider.
## Compatibility
The migration to v0.12 keyed configuration is an intentional external file
format change. Operators must migrate `id`/`version` embedded fields and
`base_profile` to native keyed documents and `extends`. Notarius does not
silently accept the former option-based API or file format. Notarius's own
version 5 configuration contract and loader decision in
[ADR-0018](0018-adopt-configkit.md) remain in force; this decision concerns
PromptKit definitions and source composition.
The workload-oriented profile identity and D&D fallback ownership in
[ADR-0010](0010-workload-oriented-llm-profile-defaults.md) remain in force.
Only that ADR's former claim that a matching operator profile completely
replaces the fallback has been superseded: native recursive overlays retain
unspecified inherited fields.
## Alternatives considered
- Keep the removed PromptKit option-based configuration API or add a
compatibility parser. This would fork the supported upstream contract and
preserve legacy format machinery.
- Merge profile documents inside Notarius or restore whole-profile
replacement. This would duplicate upstream composition behavior and make
inspection, preflight, and runtime easier to diverge.
- Re-read and hash source files for checkpoint identity. This could describe
mutable files rather than the definitions captured by the engine and would
duplicate upstream discovery and composition.
- Expose upstream inspection structs directly as CLI output. This would bind
the public Notarius contract to dependency-owned Go types.
## Consequences
PromptKit remains the owner of its native configuration semantics, while
Notarius owns which registered and operator sources are assembled and when an
engine is reconstructed. A valid later overlay can supersede an earlier
definition; validation applies to the final effective definitions rather than
as a separate audit of every superseded value. Configuration reports can
contain ordinary prompt and provider parameters, so operators own the
destination and retention of explicitly requested reports even though
credential-shaped fields are redacted. PromptKit upgrades require review of
the source adapter, captured identity, reports, and the pinned upstream
contracts.

View File

@@ -93,16 +93,24 @@ application fallback profile assets, and optional backend configuration
through the same construction path for inspection and execution. PromptKit
owns source precedence, strict group composition, and recursive field overlay:
omitted operator fields retain lower-layer values, while supplied fields
replace them. A malformed final effective definition fails construction. The
operator configuration and deployment workflow are defined in
replace them. The application group retains registered filesystem roots; the
operator may be one file with native imports or a recursively discovered
directory. Imports resolve resources from their declaring file. Prompt and
profile definitions are validated after source composition, so a malformed
lower-layer value superseded by a valid overlay is not separately audited. A
malformed final effective definition fails construction. The operator
configuration and deployment workflow are defined in
[Configuration](../config.md#promptkit-profiles) and
[Operations](../operations.md#promptkit-profile-deployment).
PromptKit and ConfigKit own `extends` inheritance under the
[pinned format rules](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/formats.md).
Notarius records the selected leaf identity and resolved target without parsing
or merging inheritance. An unset filesystem `api_key_env` is optional and may
reach the provider without authorization, which can result in a 401 or 403.
or merging inheritance. `api_key_env` names the environment variable; inspection
does not check whether it is set. PromptKit's effective `api_key_required`
metadata is supported in file-backed profiles and reported as metadata, not
as an availability check. Whether a usable credential reaches the provider
remains an execution concern.
PromptKit accepts only the `developer`, `system`, `user`, and
`assistant` text-chat roles after normalizing case and surrounding whitespace.
@@ -129,6 +137,11 @@ PromptKit obtains its maintained OpenRouter and Rakestrawhome backend and
profile catalogs from independently versioned transitive modules. Notarius
does not import or register either catalog; PromptKit retains catalog source,
identity, precedence, credential, and capacity ownership.
The selected integration modules are
[`promptkit-backend-openrouter/v3`](../oss/attribution.md#promptkit-openrouter-catalog)
and
[`promptkit-backend-rakestrawhome/v3`](../oss/attribution.md#promptkit-rakestrawhome-catalog),
both pinned at `v3.20260930.0` for this compatibility boundary.
External catalog profile names and settings can change between dependency
releases, including removal without aliases. Review directly selected IDs and
@@ -153,6 +166,16 @@ implicitly supported. A dependency upgrade requires reviewing the adapter,
profile-source construction, and this compatibility statement against the
pinned upstream documentation.
Captured inspection describes one engine's immutable effective view. Source
edits require constructing a new engine to become visible. Effective and
explanation reports retain PromptKit's redacted configuration and source
history contracts; the profile command is a separate Notarius-owned projection
of copied, provider-neutral metadata. None of these views reads credential
values or predicts provider defaults. See the public [CLI reference](../cli.md#config-effective),
[explanation](../cli.md#config-explain), and
[profile](../cli.md#config-profile) entries for their syntax and output
contracts.
## Notarius Ownership
[LLM Runtime Internals](../internal/llm.md) describes how Notarius mounts

View File

@@ -41,17 +41,26 @@ configured discovery and precedence contract is in
[Configuration](../config.md), while the loading and resolution mechanics are
in [Configuration Internals](configuration.md).
`config effective` and `config explain` only render the loaded ConfigKit view;
they do not resolve pipelines or construct providers. Configuration validation without a selected pipeline checks structural
`config effective` and `config explain` render the loaded ConfigKit view by
default and do not resolve pipelines or construct providers. With explicit
`--scope promptkit`, they construct the PromptKit inspection engine from the
same captured source groups used by execution. Effective output is PromptKit's
redacted YAML; explanation output shows its source history. `config profile`
uses that engine to emit one neutral Notarius-owned JSON summary for the exact
requested profile ID. These inspection commands load no run pipeline, prepare
no request, resolve no credential value, and make no provider call. Ordinary
prompt and provider-parameter content is not generally secret-free even though
credential-shaped provider parameter keys are redacted.
Configuration validation without a selected pipeline checks structural
configuration only. Validation with a selected pipeline also builds the
effective catalog, resolves the pipeline, and verifies every explicit effective
PromptKit profile. Selected LLM-backed input, chunk, lane, output, and validator
profiles are inspected
against the configured PromptKit source and backend registrations without
loading a prompt or performing generation, so an unknown or invalid profile
fails before pipeline preparation. Credential availability remains an
execution-time concern. Pipeline listing validates configuration before
returning normalized, sorted identifiers.
profiles are inspected against the configured PromptKit source and backend
registrations without loading a prompt or performing generation, so an unknown
or invalid final effective profile fails before pipeline preparation.
Credential availability remains an execution-time concern. Pipeline listing
validates configuration before returning normalized, sorted identifiers.
## Production Composition
@@ -62,14 +71,17 @@ catalog used for resolution and the concrete constructors used for preparation.
Tests may provide a catalog or registries instead; production code must not
silently merge an injected partial catalog with production registrations.
The production LLM factory builds one PromptKit-backed client from the resolved
**promptkit.profile_dir** or **promptkit.profile_file** source, attaches the
profile-provenance recorder, creates one scheduler from the effective global
LLM limit, and wraps the client before it reaches modules. Registration and LLM
construction errors are returned before a pipeline is prepared. Configuration
field definitions remain in [Configuration](../config.md#promptkit-profiles);
the D&D registrar's fallback profile assets and the adapter mechanics remain in
[LLM Runtime](llm.md).
The production LLM factory builds one PromptKit-backed client from the native
source groups assembled from PromptKit catalogs, registered application prompt
and fallback assets, the optional operator file or recursively selected
directory, and the optional local backend registration. The same construction
path serves runtime, preflight, and configuration inspection. It attaches the
profile-provenance recorder, captures one immutable PromptKit configuration
identity, creates one scheduler from the effective global LLM limit, and wraps
the client before it reaches modules. Registration and LLM construction errors
are returned before a pipeline is prepared. Configuration fields remain in
[Configuration](../config.md#promptkit-profiles); source ownership, capture,
identity, and adapter mechanics remain in [LLM Runtime](llm.md).
The factory also accepts `LLMRuntimeOverrides`, whose reasoning pointer
preserves inherit, replace, and clear states across the composition boundary.

View File

@@ -83,35 +83,38 @@ retain an empty backend ID, which the published JSON omits. Successful
completion responses and recorded profile manifests identify the adapter
provider as `promptkit`.
The CLI's profile-inspection engine and the production adapter use the same
profile-source construction to apply the configured profile directory or file,
the optional registered fallback profile assets, and the optional conventional
`local` backend. Preflight therefore resolves the same profile sources and
backend membership as runtime without performing generation. Fallback assets
are mounted only when at least one source is registered. The production D&D
registrar contributes its `dnd-extraction` fallback, and the maintained D&D
prompts select that logical ID by default. PromptKit owns source precedence and
profile parsing and inheritance: an operator-provided matching profile takes precedence over a
fallback profile without Notarius merging either document.
When the registration is absent, a profile selecting `backend: local` fails
inspection instead of falling back to a built-in or endpoint-only target.
The CLI's profile-inspection engine and production adapter use the same source
assembly: PromptKit catalogs, one application group containing registered
prompts and fallback profiles, the selected operator file or directory, and
the optional final `local` backend registration. Execution, preflight, and
inspection therefore capture the same effective definitions. PromptKit and
ConfigKit compose recursively across groups while retaining each input's file
context; inputs within a group compose strictly. Operator imports are enabled
for the selected operator input, and imports retain their declaring file's
relative resource roots. PromptKit validates final effective definitions at
engine construction. It does not run a separate audit that rejects a malformed
definition superseded by a later overlay. The D&D registrar contributes its
`dnd-extraction` fallback, and maintained D&D prompts select that logical ID
by default. A profile selecting `backend: local` requires the optional local
registration.
Before execution, the adapter also contributes a non-secret checkpoint
fingerprint for the effective PromptKit profile source. It combines the
identity of PromptKit's compiled-in profile catalog with a deterministic digest
of every YAML profile in the configured profile directory, or of the configured
profile file, and a deterministic digest of the flattened fallback profile
assets. The fingerprint contains neither profile content nor source paths. It
covers inherited pipeline profiles, explicit binding profiles, and
prompt-selected defaults, so changing a model or other profile setting cannot
reuse checkpoints created under the
prior profile source. This cache identity is independent of durable
profile provenance: run manifests continue to list only profiles actually
observed during LLM calls. When the local backend is registered, a second
fingerprint hashes its trimmed endpoint behind a stable marker. Changing that
semantic execution target invalidates checkpoint reuse. The raw endpoint is not
stored in checkpoint identity, and the local concurrency limit is excluded
because it changes scheduling rather than execution semantics.
Each engine captures its definitions at construction. A later source-file edit
does not mutate an existing engine; constructing a new engine captures the
updated view. Before execution, the adapter computes one immutable,
versioned `promptkit_configuration` checkpoint identity from public captured
inspection APIs. The private digest covers captured effective configuration,
all effective profile targets and credential-policy metadata (including
provider parameters), and every effective prompt ID/version and captured
content hash. The identity also includes PromptKit and the selected OpenRouter
and Rakestrawhome catalog versions. Local-backend concurrency settings remain
excluded because they affect scheduling; its endpoint is represented only in
the private digest, never as checkpoint metadata. Neither the private digest
payload, raw sources, paths, endpoints, provider parameters, nor prompt bodies
are published. An existing engine retains its identity after files change; a
new engine gets the identity of its newly captured definitions. This replaces
the former live-file profile and local-endpoint fingerprints. Run manifests
remain independent and list only profile choices actually observed during LLM
calls.
## Shared Provider-Call Limit
@@ -140,11 +143,16 @@ only by the Notarius scheduler.
## Prompt And Schema Assets
An `AssetRegistry` collects prompt, schema, and optional fallback-profile
filesystems from production module families. It flattens registered roots into
the corresponding PromptKit filesystems and rejects invalid roots, unreadable
assets, duplicate paths, and missing prompt or schema files during preparation.
Fallback assets receive a safe content digest for checkpoint identity; raw
paths and bytes are never included. The framework’s `promptfs` helper combines
filesystems from production module families. It retains each registered root's
filesystem context when constructing PromptKit's application source group.
Schema files are supplied separately through the engine's schema filesystem;
PromptKit reads and validates a selected schema while preparing that request,
not while constructing the configuration snapshot. Prompt and fallback
definitions are validated after native source composition, and referenced
prompt content is captured with the engine. Checkpoint identity uses captured
prompt content hashes rather than rescanning live files or hashing only
content-file paths. Raw paths and bytes are never included. The framework’s
`promptfs` helper combines
module-selected prompt files with reusable domain fragments without making the
framework depend on D&D content.
@@ -294,6 +302,10 @@ When debug recording is enabled, the pipeline decorates the shared client. The
wrapper records prepared prompt and response material, timing, selected profile
and backend, effective model parameters, and call identifiers in the run’s
debug bundle, including material available from a failed structured completion.
The profile-inspection CLI projects copied values into its Notarius-owned
neutral JSON DTO; effective configuration and explanation remain PromptKit's
redacted YAML and source-history views. Credential names and `APIKeyRequired`
metadata are configuration metadata, not credential-availability checks.
Effective parameters use PromptKit's stable lower-case JSON field names and may
include `backend_id`. For a successful completion, a debug-write failure is
surfaced; when the completion already failed, its call error remains the

View File

@@ -705,6 +705,17 @@ Validation: documentation/link review, scoped stale-term search, and
`git diff --check`. Reuse code/example checks when unchanged; no new scanner
or paid inference is required for prose.
> Completion: complete. Updated the internal LLM, CLI, and PromptKit integration
> guides to describe unified native sources, final-effective validation,
> captured engine lifetime, schema preparation timing, immutable configuration
> identity, inspection output ownership, and credential metadata. Added
> ADR-0019 and linked it from ADR-0010 while preserving the workload and fallback
> ownership decision; ADR-0018 remains unchanged. Scoped stale-claim review and
> local/pinned source and license-link review are complete; the browser could
> not access the private Gitea links, so pinned attribution targets and license
> identifiers were cross-checked against the locally cached pinned modules.
> `git diff --check` passes.
Prompt size: focused internal/integration references and one short ADR.
## Stage 15 — Verify Completeness And Remove Only Obsolete Machinery