Document PromptKit configuration ownership and capture
This commit is contained in:
@@ -1,6 +1,7 @@
|
||||
# ADR-0010: Use workload-oriented LLM profile defaults
|
||||
|
||||
**Status:** Accepted
|
||||
**Status:** Accepted; profile source replacement behavior superseded by
|
||||
[ADR-0019](0019-adopt-native-promptkit-configuration.md)
|
||||
**Date:** 2026-08-03
|
||||
|
||||
## Context
|
||||
@@ -23,11 +24,13 @@ intentional exceptions, and the run-wide CLI profile override has highest
|
||||
precedence.
|
||||
|
||||
The D&D family owns an embedded fallback profile named `dnd-extraction`.
|
||||
Operators may provide a complete profile with the same ID through a PromptKit
|
||||
filesystem source. PromptKit selects the higher-precedence matching definition;
|
||||
Notarius does not merge profile documents. Production, development, and local
|
||||
deployments can therefore use different execution policy behind one unchanged
|
||||
pipeline ID.
|
||||
Operators may configure the same ID through a PromptKit filesystem source. The
|
||||
original decision specified that an operator definition replaced the fallback
|
||||
as a complete profile; that source-composition clause has been superseded by
|
||||
[ADR-0019](0019-adopt-native-promptkit-configuration.md), which adopts native
|
||||
recursive overlays. The workload-oriented identity and D&D ownership decision
|
||||
remain in force. Production, development, and local deployments can therefore
|
||||
use different execution policy behind one unchanged pipeline ID.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
@@ -42,8 +45,8 @@ pipeline ID.
|
||||
## Consequences
|
||||
|
||||
Pipeline configuration expresses workload intent rather than a specific
|
||||
provider or model. Operators can replace the complete execution policy without
|
||||
editing bindings, while binding-level and run-wide exceptions remain available.
|
||||
Profile changes affect resolved pipeline and checkpoint identity, so they may
|
||||
provider or model. Operators can configure execution policy without editing
|
||||
bindings, while binding-level and run-wide exceptions remain available. Profile
|
||||
changes affect resolved pipeline and checkpoint identity, so they may
|
||||
intentionally cause work to be recomputed. The D&D fallback becomes a
|
||||
maintained application execution-policy asset.
|
||||
|
||||
92
docs/adr/0019-adopt-native-promptkit-configuration.md
Normal file
92
docs/adr/0019-adopt-native-promptkit-configuration.md
Normal file
@@ -0,0 +1,92 @@
|
||||
# ADR-0019: Adopt native PromptKit configuration sources
|
||||
|
||||
**Status:** Accepted
|
||||
**Date:** 2026-10-04
|
||||
|
||||
## Context
|
||||
|
||||
PromptKit v0.12 replaces its former source options with ConfigKit-backed
|
||||
configuration sources, keyed documents, recursive overlays, and native
|
||||
inheritance. Notarius must choose how application-owned prompt and fallback
|
||||
assets, operator configuration, and the optional local backend reach PromptKit
|
||||
without recreating source parsing or inheritance in the adapter. Runtime,
|
||||
preflight, and operator inspection must agree about the effective definitions.
|
||||
|
||||
Notarius also captures checkpoint identity for the PromptKit configuration
|
||||
used by an execution engine. That identity must follow the engine's captured
|
||||
definition state rather than re-reading mutable source files, and must not
|
||||
publish credentials, prompt bodies, provider parameters, or source paths.
|
||||
|
||||
## Decision
|
||||
|
||||
Use PromptKit's public v0.12 native source APIs as the only PromptKit
|
||||
configuration boundary. Assemble PromptKit catalogs, registered application
|
||||
prompt and fallback inputs, the optional operator file or recursively selected
|
||||
directory, and the optional Notarius local backend registration in that order.
|
||||
Use native ConfigKit imports and PromptKit/ConfigKit composition behavior;
|
||||
retain each filesystem input's resource context. Inputs within a group compose
|
||||
strictly, and later groups recursively overlay earlier ones. PromptKit owns
|
||||
definition parsing, exact-key `extends` inheritance, and final effective-value
|
||||
validation. Notarius does not provide a legacy parser, compatibility alias,
|
||||
filename-priority merger, whole-profile replacement shim, or alternate
|
||||
inheritance engine.
|
||||
|
||||
Execution, explicit-profile preflight, and configuration inspection use the
|
||||
same source construction. Each constructed engine captures its effective
|
||||
definitions. Edits to source files take effect for a newly constructed engine,
|
||||
not an existing one. Derive the private versioned
|
||||
`promptkit_configuration` checkpoint identity from public captured inspection
|
||||
and definition APIs, including effective configuration, profile targets and
|
||||
credential-policy metadata, captured prompt hashes, and the PromptKit and
|
||||
selected catalog identities. Retain only the digest as checkpoint metadata;
|
||||
keep all digest inputs private. Exclude local-backend concurrency settings
|
||||
because they affect scheduling rather than execution semantics.
|
||||
|
||||
Keep PromptKit types and source assembly inside the LLM integration boundary.
|
||||
The CLI may project copied profile fields into a Notarius-owned neutral JSON
|
||||
contract. PromptKit's effective and explanation reports remain upstream
|
||||
redacted configuration and source-history views. Neither inspection path reads
|
||||
credential values or contacts a provider.
|
||||
|
||||
## Compatibility
|
||||
|
||||
The migration to v0.12 keyed configuration is an intentional external file
|
||||
format change. Operators must migrate `id`/`version` embedded fields and
|
||||
`base_profile` to native keyed documents and `extends`. Notarius does not
|
||||
silently accept the former option-based API or file format. Notarius's own
|
||||
version 5 configuration contract and loader decision in
|
||||
[ADR-0018](0018-adopt-configkit.md) remain in force; this decision concerns
|
||||
PromptKit definitions and source composition.
|
||||
|
||||
The workload-oriented profile identity and D&D fallback ownership in
|
||||
[ADR-0010](0010-workload-oriented-llm-profile-defaults.md) remain in force.
|
||||
Only that ADR's former claim that a matching operator profile completely
|
||||
replaces the fallback has been superseded: native recursive overlays retain
|
||||
unspecified inherited fields.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
- Keep the removed PromptKit option-based configuration API or add a
|
||||
compatibility parser. This would fork the supported upstream contract and
|
||||
preserve legacy format machinery.
|
||||
- Merge profile documents inside Notarius or restore whole-profile
|
||||
replacement. This would duplicate upstream composition behavior and make
|
||||
inspection, preflight, and runtime easier to diverge.
|
||||
- Re-read and hash source files for checkpoint identity. This could describe
|
||||
mutable files rather than the definitions captured by the engine and would
|
||||
duplicate upstream discovery and composition.
|
||||
- Expose upstream inspection structs directly as CLI output. This would bind
|
||||
the public Notarius contract to dependency-owned Go types.
|
||||
|
||||
## Consequences
|
||||
|
||||
PromptKit remains the owner of its native configuration semantics, while
|
||||
Notarius owns which registered and operator sources are assembled and when an
|
||||
engine is reconstructed. A valid later overlay can supersede an earlier
|
||||
definition; validation applies to the final effective definitions rather than
|
||||
as a separate audit of every superseded value. Configuration reports can
|
||||
contain ordinary prompt and provider parameters, so operators own the
|
||||
destination and retention of explicitly requested reports even though
|
||||
credential-shaped fields are redacted. PromptKit upgrades require review of
|
||||
the source adapter, captured identity, reports, and the pinned upstream
|
||||
contracts.
|
||||
@@ -93,16 +93,24 @@ application fallback profile assets, and optional backend configuration
|
||||
through the same construction path for inspection and execution. PromptKit
|
||||
owns source precedence, strict group composition, and recursive field overlay:
|
||||
omitted operator fields retain lower-layer values, while supplied fields
|
||||
replace them. A malformed final effective definition fails construction. The
|
||||
operator configuration and deployment workflow are defined in
|
||||
replace them. The application group retains registered filesystem roots; the
|
||||
operator may be one file with native imports or a recursively discovered
|
||||
directory. Imports resolve resources from their declaring file. Prompt and
|
||||
profile definitions are validated after source composition, so a malformed
|
||||
lower-layer value superseded by a valid overlay is not separately audited. A
|
||||
malformed final effective definition fails construction. The operator
|
||||
configuration and deployment workflow are defined in
|
||||
[Configuration](../config.md#promptkit-profiles) and
|
||||
[Operations](../operations.md#promptkit-profile-deployment).
|
||||
|
||||
PromptKit and ConfigKit own `extends` inheritance under the
|
||||
[pinned format rules](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/formats.md).
|
||||
Notarius records the selected leaf identity and resolved target without parsing
|
||||
or merging inheritance. An unset filesystem `api_key_env` is optional and may
|
||||
reach the provider without authorization, which can result in a 401 or 403.
|
||||
or merging inheritance. `api_key_env` names the environment variable; inspection
|
||||
does not check whether it is set. PromptKit's effective `api_key_required`
|
||||
metadata is supported in file-backed profiles and reported as metadata, not
|
||||
as an availability check. Whether a usable credential reaches the provider
|
||||
remains an execution concern.
|
||||
|
||||
PromptKit accepts only the `developer`, `system`, `user`, and
|
||||
`assistant` text-chat roles after normalizing case and surrounding whitespace.
|
||||
@@ -129,6 +137,11 @@ PromptKit obtains its maintained OpenRouter and Rakestrawhome backend and
|
||||
profile catalogs from independently versioned transitive modules. Notarius
|
||||
does not import or register either catalog; PromptKit retains catalog source,
|
||||
identity, precedence, credential, and capacity ownership.
|
||||
The selected integration modules are
|
||||
[`promptkit-backend-openrouter/v3`](../oss/attribution.md#promptkit-openrouter-catalog)
|
||||
and
|
||||
[`promptkit-backend-rakestrawhome/v3`](../oss/attribution.md#promptkit-rakestrawhome-catalog),
|
||||
both pinned at `v3.20260930.0` for this compatibility boundary.
|
||||
|
||||
External catalog profile names and settings can change between dependency
|
||||
releases, including removal without aliases. Review directly selected IDs and
|
||||
@@ -153,6 +166,16 @@ implicitly supported. A dependency upgrade requires reviewing the adapter,
|
||||
profile-source construction, and this compatibility statement against the
|
||||
pinned upstream documentation.
|
||||
|
||||
Captured inspection describes one engine's immutable effective view. Source
|
||||
edits require constructing a new engine to become visible. Effective and
|
||||
explanation reports retain PromptKit's redacted configuration and source
|
||||
history contracts; the profile command is a separate Notarius-owned projection
|
||||
of copied, provider-neutral metadata. None of these views reads credential
|
||||
values or predicts provider defaults. See the public [CLI reference](../cli.md#config-effective),
|
||||
[explanation](../cli.md#config-explain), and
|
||||
[profile](../cli.md#config-profile) entries for their syntax and output
|
||||
contracts.
|
||||
|
||||
## Notarius Ownership
|
||||
|
||||
[LLM Runtime Internals](../internal/llm.md) describes how Notarius mounts
|
||||
|
||||
@@ -41,17 +41,26 @@ configured discovery and precedence contract is in
|
||||
[Configuration](../config.md), while the loading and resolution mechanics are
|
||||
in [Configuration Internals](configuration.md).
|
||||
|
||||
`config effective` and `config explain` only render the loaded ConfigKit view;
|
||||
they do not resolve pipelines or construct providers. Configuration validation without a selected pipeline checks structural
|
||||
`config effective` and `config explain` render the loaded ConfigKit view by
|
||||
default and do not resolve pipelines or construct providers. With explicit
|
||||
`--scope promptkit`, they construct the PromptKit inspection engine from the
|
||||
same captured source groups used by execution. Effective output is PromptKit's
|
||||
redacted YAML; explanation output shows its source history. `config profile`
|
||||
uses that engine to emit one neutral Notarius-owned JSON summary for the exact
|
||||
requested profile ID. These inspection commands load no run pipeline, prepare
|
||||
no request, resolve no credential value, and make no provider call. Ordinary
|
||||
prompt and provider-parameter content is not generally secret-free even though
|
||||
credential-shaped provider parameter keys are redacted.
|
||||
|
||||
Configuration validation without a selected pipeline checks structural
|
||||
configuration only. Validation with a selected pipeline also builds the
|
||||
effective catalog, resolves the pipeline, and verifies every explicit effective
|
||||
PromptKit profile. Selected LLM-backed input, chunk, lane, output, and validator
|
||||
profiles are inspected
|
||||
against the configured PromptKit source and backend registrations without
|
||||
loading a prompt or performing generation, so an unknown or invalid profile
|
||||
fails before pipeline preparation. Credential availability remains an
|
||||
execution-time concern. Pipeline listing validates configuration before
|
||||
returning normalized, sorted identifiers.
|
||||
profiles are inspected against the configured PromptKit source and backend
|
||||
registrations without loading a prompt or performing generation, so an unknown
|
||||
or invalid final effective profile fails before pipeline preparation.
|
||||
Credential availability remains an execution-time concern. Pipeline listing
|
||||
validates configuration before returning normalized, sorted identifiers.
|
||||
|
||||
## Production Composition
|
||||
|
||||
@@ -62,14 +71,17 @@ catalog used for resolution and the concrete constructors used for preparation.
|
||||
Tests may provide a catalog or registries instead; production code must not
|
||||
silently merge an injected partial catalog with production registrations.
|
||||
|
||||
The production LLM factory builds one PromptKit-backed client from the resolved
|
||||
**promptkit.profile_dir** or **promptkit.profile_file** source, attaches the
|
||||
profile-provenance recorder, creates one scheduler from the effective global
|
||||
LLM limit, and wraps the client before it reaches modules. Registration and LLM
|
||||
construction errors are returned before a pipeline is prepared. Configuration
|
||||
field definitions remain in [Configuration](../config.md#promptkit-profiles);
|
||||
the D&D registrar's fallback profile assets and the adapter mechanics remain in
|
||||
[LLM Runtime](llm.md).
|
||||
The production LLM factory builds one PromptKit-backed client from the native
|
||||
source groups assembled from PromptKit catalogs, registered application prompt
|
||||
and fallback assets, the optional operator file or recursively selected
|
||||
directory, and the optional local backend registration. The same construction
|
||||
path serves runtime, preflight, and configuration inspection. It attaches the
|
||||
profile-provenance recorder, captures one immutable PromptKit configuration
|
||||
identity, creates one scheduler from the effective global LLM limit, and wraps
|
||||
the client before it reaches modules. Registration and LLM construction errors
|
||||
are returned before a pipeline is prepared. Configuration fields remain in
|
||||
[Configuration](../config.md#promptkit-profiles); source ownership, capture,
|
||||
identity, and adapter mechanics remain in [LLM Runtime](llm.md).
|
||||
|
||||
The factory also accepts `LLMRuntimeOverrides`, whose reasoning pointer
|
||||
preserves inherit, replace, and clear states across the composition boundary.
|
||||
|
||||
@@ -83,35 +83,38 @@ retain an empty backend ID, which the published JSON omits. Successful
|
||||
completion responses and recorded profile manifests identify the adapter
|
||||
provider as `promptkit`.
|
||||
|
||||
The CLI's profile-inspection engine and the production adapter use the same
|
||||
profile-source construction to apply the configured profile directory or file,
|
||||
the optional registered fallback profile assets, and the optional conventional
|
||||
`local` backend. Preflight therefore resolves the same profile sources and
|
||||
backend membership as runtime without performing generation. Fallback assets
|
||||
are mounted only when at least one source is registered. The production D&D
|
||||
registrar contributes its `dnd-extraction` fallback, and the maintained D&D
|
||||
prompts select that logical ID by default. PromptKit owns source precedence and
|
||||
profile parsing and inheritance: an operator-provided matching profile takes precedence over a
|
||||
fallback profile without Notarius merging either document.
|
||||
When the registration is absent, a profile selecting `backend: local` fails
|
||||
inspection instead of falling back to a built-in or endpoint-only target.
|
||||
The CLI's profile-inspection engine and production adapter use the same source
|
||||
assembly: PromptKit catalogs, one application group containing registered
|
||||
prompts and fallback profiles, the selected operator file or directory, and
|
||||
the optional final `local` backend registration. Execution, preflight, and
|
||||
inspection therefore capture the same effective definitions. PromptKit and
|
||||
ConfigKit compose recursively across groups while retaining each input's file
|
||||
context; inputs within a group compose strictly. Operator imports are enabled
|
||||
for the selected operator input, and imports retain their declaring file's
|
||||
relative resource roots. PromptKit validates final effective definitions at
|
||||
engine construction. It does not run a separate audit that rejects a malformed
|
||||
definition superseded by a later overlay. The D&D registrar contributes its
|
||||
`dnd-extraction` fallback, and maintained D&D prompts select that logical ID
|
||||
by default. A profile selecting `backend: local` requires the optional local
|
||||
registration.
|
||||
|
||||
Before execution, the adapter also contributes a non-secret checkpoint
|
||||
fingerprint for the effective PromptKit profile source. It combines the
|
||||
identity of PromptKit's compiled-in profile catalog with a deterministic digest
|
||||
of every YAML profile in the configured profile directory, or of the configured
|
||||
profile file, and a deterministic digest of the flattened fallback profile
|
||||
assets. The fingerprint contains neither profile content nor source paths. It
|
||||
covers inherited pipeline profiles, explicit binding profiles, and
|
||||
prompt-selected defaults, so changing a model or other profile setting cannot
|
||||
reuse checkpoints created under the
|
||||
prior profile source. This cache identity is independent of durable
|
||||
profile provenance: run manifests continue to list only profiles actually
|
||||
observed during LLM calls. When the local backend is registered, a second
|
||||
fingerprint hashes its trimmed endpoint behind a stable marker. Changing that
|
||||
semantic execution target invalidates checkpoint reuse. The raw endpoint is not
|
||||
stored in checkpoint identity, and the local concurrency limit is excluded
|
||||
because it changes scheduling rather than execution semantics.
|
||||
Each engine captures its definitions at construction. A later source-file edit
|
||||
does not mutate an existing engine; constructing a new engine captures the
|
||||
updated view. Before execution, the adapter computes one immutable,
|
||||
versioned `promptkit_configuration` checkpoint identity from public captured
|
||||
inspection APIs. The private digest covers captured effective configuration,
|
||||
all effective profile targets and credential-policy metadata (including
|
||||
provider parameters), and every effective prompt ID/version and captured
|
||||
content hash. The identity also includes PromptKit and the selected OpenRouter
|
||||
and Rakestrawhome catalog versions. Local-backend concurrency settings remain
|
||||
excluded because they affect scheduling; its endpoint is represented only in
|
||||
the private digest, never as checkpoint metadata. Neither the private digest
|
||||
payload, raw sources, paths, endpoints, provider parameters, nor prompt bodies
|
||||
are published. An existing engine retains its identity after files change; a
|
||||
new engine gets the identity of its newly captured definitions. This replaces
|
||||
the former live-file profile and local-endpoint fingerprints. Run manifests
|
||||
remain independent and list only profile choices actually observed during LLM
|
||||
calls.
|
||||
|
||||
## Shared Provider-Call Limit
|
||||
|
||||
@@ -140,11 +143,16 @@ only by the Notarius scheduler.
|
||||
## Prompt And Schema Assets
|
||||
|
||||
An `AssetRegistry` collects prompt, schema, and optional fallback-profile
|
||||
filesystems from production module families. It flattens registered roots into
|
||||
the corresponding PromptKit filesystems and rejects invalid roots, unreadable
|
||||
assets, duplicate paths, and missing prompt or schema files during preparation.
|
||||
Fallback assets receive a safe content digest for checkpoint identity; raw
|
||||
paths and bytes are never included. The framework’s `promptfs` helper combines
|
||||
filesystems from production module families. It retains each registered root's
|
||||
filesystem context when constructing PromptKit's application source group.
|
||||
Schema files are supplied separately through the engine's schema filesystem;
|
||||
PromptKit reads and validates a selected schema while preparing that request,
|
||||
not while constructing the configuration snapshot. Prompt and fallback
|
||||
definitions are validated after native source composition, and referenced
|
||||
prompt content is captured with the engine. Checkpoint identity uses captured
|
||||
prompt content hashes rather than rescanning live files or hashing only
|
||||
content-file paths. Raw paths and bytes are never included. The framework’s
|
||||
`promptfs` helper combines
|
||||
module-selected prompt files with reusable domain fragments without making the
|
||||
framework depend on D&D content.
|
||||
|
||||
@@ -294,6 +302,10 @@ When debug recording is enabled, the pipeline decorates the shared client. The
|
||||
wrapper records prepared prompt and response material, timing, selected profile
|
||||
and backend, effective model parameters, and call identifiers in the run’s
|
||||
debug bundle, including material available from a failed structured completion.
|
||||
The profile-inspection CLI projects copied values into its Notarius-owned
|
||||
neutral JSON DTO; effective configuration and explanation remain PromptKit's
|
||||
redacted YAML and source-history views. Credential names and `APIKeyRequired`
|
||||
metadata are configuration metadata, not credential-availability checks.
|
||||
Effective parameters use PromptKit's stable lower-case JSON field names and may
|
||||
include `backend_id`. For a successful completion, a debug-write failure is
|
||||
surfaced; when the completion already failed, its call error remains the
|
||||
|
||||
@@ -705,6 +705,17 @@ Validation: documentation/link review, scoped stale-term search, and
|
||||
`git diff --check`. Reuse code/example checks when unchanged; no new scanner
|
||||
or paid inference is required for prose.
|
||||
|
||||
> Completion: complete. Updated the internal LLM, CLI, and PromptKit integration
|
||||
> guides to describe unified native sources, final-effective validation,
|
||||
> captured engine lifetime, schema preparation timing, immutable configuration
|
||||
> identity, inspection output ownership, and credential metadata. Added
|
||||
> ADR-0019 and linked it from ADR-0010 while preserving the workload and fallback
|
||||
> ownership decision; ADR-0018 remains unchanged. Scoped stale-claim review and
|
||||
> local/pinned source and license-link review are complete; the browser could
|
||||
> not access the private Gitea links, so pinned attribution targets and license
|
||||
> identifiers were cross-checked against the locally cached pinned modules.
|
||||
> `git diff --check` passes.
|
||||
|
||||
Prompt size: focused internal/integration references and one short ADR.
|
||||
|
||||
## Stage 15 — Verify Completeness And Remove Only Obsolete Machinery
|
||||
|
||||
Reference in New Issue
Block a user