721 lines
37 KiB
Markdown
721 lines
37 KiB
Markdown
# PromptKit 0.12 Implementation Plan
|
||
|
||
## Purpose And Baseline
|
||
|
||
Implement [PromptKit 0.12 Configuration Migration](promptkit.md). That roadmap
|
||
owns the accepted target state; this document owns implementation decisions,
|
||
ordered steps, and verification. It replaces the completed ConfigKit plan.
|
||
Each numbered stage is one bounded prompt for a Luna coding agent. Execute in
|
||
order, inspecting the named code and tests before editing.
|
||
|
||
The starting revision is `aae01a5`: ConfigKit v0.5.0, PromptKit v0.11.2, and
|
||
Notarius configuration version 5. Target PromptKit v0.12.0 at full commit
|
||
`4b8169da286917fde7875425bb32b799b12c6af3`, with its selected OpenRouter and
|
||
Rakestrawhome `/v3` catalogs at `v3.20260930.0`. ConfigKit and Go remain at the
|
||
existing v0.5.0 and 1.25.5 baselines. Inspect released source, not a moving
|
||
sibling branch. Use `git show v0.12.0:<path>` in `../promptkit` where helpful;
|
||
the annotated tag's source commit is `v0.12.0^{commit}`.
|
||
|
||
The focused LLM, CLI, and module-integration baseline tests passed during
|
||
planning. This is not evidence that the upgraded implementation passes.
|
||
|
||
## Instructions For Every Stage
|
||
|
||
- Read `docs/development.md`, `docs/policy/agents.md`, and the feature roadmap.
|
||
Select applicable architecture, testing, documentation, and OSS policies.
|
||
Prefer the code knowledge graph for code discovery. Reuse unchanged policy
|
||
context; do not read unrelated policies merely to fill a checklist.
|
||
- Use `GOWORK=off` for Go commands. No local dependency `replace`, fork,
|
||
legacy-format parser, whole-profile replacement shim, section whitelist,
|
||
custom inheritance engine, or filename-priority merger belongs here.
|
||
- Keep upstream configuration types in the LLM integration boundary. Modules
|
||
retain transport-neutral requests, typed outputs, and asset ownership. The
|
||
root `assets` Go package remains content-only.
|
||
- Preserve prompt text/order, identities, schema contents, retry budgets,
|
||
sessions, reasoning overrides, warnings, scheduling, and artifact/subprocess
|
||
contracts. Deliberately update tests for accepted configuration-format,
|
||
recursive-overlay, and eager-validation changes, not unrelated expectations.
|
||
- Use real loading and offline injected clients or local HTTP fixtures. Add
|
||
narrowly owned behavioral tests for actual risks; reuse existing coverage.
|
||
Do not mirror the complete upstream suite, snapshot long errors/reports, or
|
||
introduce exact shared-prefix-length detector tests.
|
||
- Keep credentials and live inference out of default tests. Do not add new
|
||
permission enforcement, credential requirements, publication gates, or source
|
||
restrictions. Fix actual regressions rather than weakening coverage.
|
||
- Review the diff, validate, and commit each nonempty coherent stage with a
|
||
plain-English message under agent policy. No empty commits. Record actual
|
||
checks and remaining work; do not call the upgrade complete before Stage 15.
|
||
Do not push, tag, retire other roadmaps, or publish.
|
||
|
||
### Bounded Migration Window
|
||
|
||
The new format and removed APIs are incompatible with the old integration.
|
||
Stages 1–5 divide that migration into small prompts. Intermediate revisions
|
||
can have expected format/test-compilation failures. Record them precisely;
|
||
do not add dual-version machinery to keep every intermediate revision green.
|
||
Native application compilation must be restored by Stage 3, the LLM suite by
|
||
Stage 4, and the full default suite by Stage 5. Do not deploy intermediate
|
||
revisions. Later stages use focused passing checks before committing.
|
||
|
||
## Settled Implementation Decisions
|
||
|
||
### Sources And Engine Ownership
|
||
|
||
Build the same source stack for execution, preflight, and inspection:
|
||
|
||
1. Upstream catalogs and PromptKit-owned definitions, supplied by PromptKit.
|
||
2. One application underlay containing registered prompt and fallback-profile
|
||
inputs, each retaining its own filesystem context.
|
||
3. One operator underlay from the selected file or recursively selected
|
||
directory. Set `Input.Imports = true` on operator inputs only. Accept native
|
||
combined `profiles`, `backends`, and `prompts` documents.
|
||
4. A final programmatic underlay for `backends.local` when Notarius's
|
||
`local_backend` is configured. Use `BackendSource` with
|
||
`LocalBackend(endpoint, concurrencyLimit)` to preserve explicit zero.
|
||
|
||
Leave `Sources.Main` nil. Use `FilesystemInputs` for embedded views,
|
||
`DirectoryInputs` for a directory, and
|
||
`configkit.Input{Source: configkit.File(path), Imports: true}` for one file.
|
||
Provide native filesystem resource context where necessary. File sources
|
||
derive their declaring directory; directory inputs retain the helper's root.
|
||
Do not make host files and embedded resources share an artificial root.
|
||
Native imports, root rules, and exact-key inheritance remain upstream behavior.
|
||
|
||
Sources in a group compose strictly; later groups overlay fields recursively.
|
||
Missing selected sources fail; empty selected directories contribute nothing.
|
||
No operator source means no operator group. Nil optional assets on the existing
|
||
profile inspector mean no application group; production supplies registered
|
||
assets. Preserve the client's required-assets check. Keep `WithSchemaFS` and
|
||
injected runtime options separate from definition sources. Register no catalog
|
||
backend manually.
|
||
|
||
### CLI Inspection Contract
|
||
|
||
Implement these commands, with flags before positional selectors as in the
|
||
current Go flag parser:
|
||
|
||
```sh
|
||
notarius config effective --scope promptkit [--config path] [--config-overlay name]
|
||
notarius config explain --scope promptkit [--config path] [--config-overlay name] <json-pointer>
|
||
notarius config profile [--config path] [--config-overlay name] <profile-id>
|
||
```
|
||
|
||
For effective/explain, omitted `--scope` or `--scope notarius` preserves the
|
||
current report and avoids constructing a PromptKit engine. Accept only
|
||
`notarius` and `promptkit`; explicit blank/unknown scope is a syntax error.
|
||
`config profile` inspects PromptKit only and needs no scope flag. Add no input,
|
||
pipeline selection, model override, or second source flag to these commands.
|
||
Load the selected Notarius configuration, use its normalized PromptKit sources
|
||
plus production assets, and inspect one engine without resolving a pipeline or
|
||
materializing references.
|
||
|
||
Effective output is upstream redacted YAML; explanation output is upstream
|
||
human-readable history. Profile output is one Notarius-owned JSON object plus
|
||
newline with `profile_id`, `api_key_required`, and `effective_model_params`.
|
||
The last object uses the upstream target's lower-case field vocabulary:
|
||
`backend_id`, `endpoint`, `model`, `temperature`, `max_tokens`, `top_p`,
|
||
`timeout_seconds`, `service_tier`, `reasoning_effort`, `api_key_env`, and
|
||
`extra_params`. Include empty `backend_id` for endpoint-only profiles. Copy
|
||
fields into a local DTO; do not publish the upstream `ProfileInspection` Go
|
||
struct as the CLI contract. These are inspected settings, not provider-default
|
||
predictions. Zero sampling values alone do not distinguish omission from
|
||
explicit zero; effective/explanation reports supply presence context.
|
||
|
||
Preserve exact profile IDs. Pointers use canonical JSON Pointer escaping;
|
||
an explicitly supplied empty pointer selects root. Use existing exit classes:
|
||
0 success, 2 malformed syntax/pointer/scope or wrong argument count, 1 load/
|
||
inspection failure or unknown path/profile. Known historical-only paths are
|
||
valid. Check `Explain`'s found result before writing: the upstream writer emits
|
||
nothing for an unknown path. Stdout contains only the selected report;
|
||
diagnostics go to stderr. Writer errors can leave partial output; add no
|
||
transactional-output or byte-stability guarantee.
|
||
|
||
Inspection never resolves credential values or contacts a provider. Report
|
||
credential names and actual upstream `APIKeyRequired` metadata without inferring
|
||
availability. Redact credential-shaped provider-parameter keys as upstream
|
||
configuration inspection does. Keep reporting projection separate from private
|
||
checkpoint hashing. Do not read unrelated environment values to sanitize
|
||
reports. Deliberately requested ordinary prompt/provider data remains visible;
|
||
add no automatic persistence, confirmation, or permission enforcement.
|
||
|
||
### Captured Checkpoint Identity
|
||
|
||
Compute one immutable identity at execution-engine construction, using public
|
||
captured inspection/definition APIs. Retain it on the client; return defensive
|
||
copies through `LLMCheckpointFingerprints`. Use name `promptkit_configuration`
|
||
and a versioned snapshot marker incorporating PromptKit and both catalog
|
||
versions. It supersedes separate profile-file/local-endpoint fingerprints; old
|
||
execution checkpoints naturally miss without migration/deletion. Source-
|
||
addressed chunk plans are unchanged.
|
||
|
||
The private digest input contains:
|
||
|
||
- The captured effective definition tree from `InspectConfiguration`.
|
||
- Every effective profile ID and its `InspectProfile` target and credential-
|
||
policy metadata, covering resolved backend defaults/provider parameters.
|
||
- Every effective prompt ID/version and `InspectPrompt.PromptHash`, covering
|
||
captured message content rather than just content-file paths.
|
||
|
||
Obtain the first tree by writing captured effective YAML to a private buffer
|
||
and decoding that report into a generic map with the existing YAML v4
|
||
dependency. This is inspection projection, not a second authoritative loader:
|
||
it performs no discovery, imports, inheritance, defaults, or validation. Use
|
||
its keys to enumerate profiles and prompt versions; sort entries and encode
|
||
the digest payload with `encoding/json`. Do not enumerate historical-only
|
||
definitions or hash history. Preserve null versus zero/omission in the tree.
|
||
Remove only `backends.local.concurrency_limit` and `.queue_capacity` from the
|
||
private tree to preserve that backend's established scheduling-only exclusion.
|
||
Other identity choices stay conservative; selected-profile optimization is
|
||
not required.
|
||
|
||
The redacted effective report is insufficient alone: include copied public
|
||
profile targets with their actual JSON-compatible provider parameters in the
|
||
private payload. This covers execution-relevant values whose names the report
|
||
redacts. Never publish the payload, raw sources, paths, endpoint, parameter
|
||
values, or prompt bodies as checkpoint metadata. Only its digest is retained.
|
||
Public profile inspection can describe partial definitions; do not turn an
|
||
incomplete unused profile into a construction failure while fingerprinting.
|
||
Constructor validation remains PromptKit's.
|
||
|
||
A new engine captures edits; an existing engine/fingerprint remains unchanged.
|
||
Preserve module/schema and CLI execution-override fingerprints, actual-used
|
||
profile provenance, and reusable-state eligibility. Imported definitions and
|
||
operator message edits participate through captured definitions/hashes, not a
|
||
separate filesystem scan.
|
||
|
||
## Stage 1 — Migrate Embedded And Maintained Definition Assets
|
||
|
||
> Completion: complete.
|
||
|
||
Goal: convert data without rewriting prompt semantics.
|
||
|
||
Inspect `assets/**/prompts/prompt.yaml`,
|
||
`assets/dnd/profiles/dnd-extraction.yaml`,
|
||
`examples/llm-profiles/dnd-extraction.yml`, their ordered manifests, and the
|
||
pinned upstream format reference.
|
||
|
||
Work:
|
||
|
||
1. Wrap each prompt body under `prompts: <existing-id>: v1:`; remove embedded
|
||
`id`/`version`. Keep every other field and content path.
|
||
2. Wrap both profile files under `profiles: dnd-extraction:`. Preserve
|
||
`openrouter`, `openai/gpt-5.6-luna`, timeout 240, `service_tier: flex`, and
|
||
unset reasoning. Keep filenames/root embedding unchanged.
|
||
3. Review identities against manifests and schema registrations. Change no
|
||
Markdown fragment, schema, cache control, or message ordering. Leave test
|
||
fixture migration to Stages 4–5.
|
||
|
||
Acceptance: all production definitions are keyed; non-identity bodies preserve
|
||
meaning and inventories select the same files. Only intended data changes land.
|
||
|
||
Validation: `git diff --check` and YAML-aware/manual comparison with prior
|
||
bodies. Legacy-loader failures are expected here, not runtime validation.
|
||
|
||
Prompt size: one mechanical conversion, no constructor work.
|
||
|
||
## Stage 2 — Upgrade Dependencies And Assemble Native Source Groups
|
||
|
||
> Completion: complete. The native compile pass confirms the expected follow-up
|
||
> work: migrate client and inspector constructors in Stage 3, then migrate LLM
|
||
> and CLI test fixtures in Stages 4–5; no compatibility wrappers were added.
|
||
|
||
Goal: establish v0.12 and shared native source construction.
|
||
|
||
Inspect `go.mod`, `go.sum`, `asset_registry.go`, `promptkit_profiles.go`,
|
||
`promptkit_client.go`, and upstream source/backend configuration APIs.
|
||
|
||
Work:
|
||
|
||
1. Upgrade PromptKit to v0.12.0 and tidy. Keep ConfigKit v0.5.0 and unrelated
|
||
pins; remove obsolete `/v2` catalogs. Promote YAML v4 to a direct dependency
|
||
when used for inspection projection without upgrading its version needlessly.
|
||
2. Implement source assembly specified above inside `internal/framework/llm`.
|
||
Replace source options with filesystem inputs, and backend registration with
|
||
keyed `BackendSource`. A registry-facing helper can return configuration
|
||
sources plus schema options; migrate direct callers in following stages.
|
||
3. Enable imports on operator inputs only. Preserve optional selection, resource
|
||
context, exact IDs, group order, and local concurrency zero. Do not auto-
|
||
clear retained queue settings if a later limit makes them invalid; native
|
||
validation decides.
|
||
4. Update PromptKit/catalog attribution, verifying released notices and full
|
||
commits: PromptKit `4b8169da286917fde7875425bb32b799b12c6af3`, OpenRouter
|
||
`f17d208cb7ad1fb043cda2266cd3011d11b40aa0`, Rakestrawhome
|
||
`993443d88c451901f4c917fa00c204fd3b54d28d`. Reuse unchanged standard texts;
|
||
change no project grant or unrelated notice.
|
||
|
||
Acceptance: selected graph and notices identify released revisions; shared
|
||
source construction uses native APIs. Name remaining constructor/test compile
|
||
failures for Stages 3–5 rather than hiding them with wrappers.
|
||
|
||
Validation: `GOWORK=off go list -m all`, `GOWORK=off go mod tidy -diff`,
|
||
`git diff --check`, and a native compilation attempt to locate removed APIs.
|
||
|
||
Prompt size: dependency/notices and one source-construction boundary.
|
||
|
||
## Stage 3 — Switch Runtime And Preflight To Shared Engine Configuration
|
||
|
||
> Completion: complete. The native command build passes. Test compilation is
|
||
> still blocked by removed PromptKit setup APIs and legacy fixtures in
|
||
> `asset_registry_test.go`, `promptkit_client_test.go`, and
|
||
> `production_contract_test.go`; the fixture work remains in Stages 4–5.
|
||
|
||
Goal: restore application compilation and consistent sources.
|
||
|
||
Inspect `NewPromptKitClient`, `NewPromptKitProfileInspector`,
|
||
`promptKitProfileSourceEngineOptions`, `PromptKitLocalBackendOption`,
|
||
`internal/cli/promptkit_profiles.go`, and production LLM composition.
|
||
|
||
Work:
|
||
|
||
1. Construct both engines with shared `Config.Sources`; remove obsolete
|
||
directory fields, source options, and `WithBackend`. Production preflight
|
||
includes application prompts, not just profiles, for partial overlays.
|
||
2. Retain timeout, HTTP client, transient retries, runtime injection options,
|
||
recorder, reasoning-pointer copies, and required client assets. Keep
|
||
`CompleteStructured` preparation/execution unchanged.
|
||
3. Retain safe errors and established `errors.Is`/`errors.As` behavior. Eager
|
||
constructor failures must not dump raw configuration/credentials into CLI
|
||
diagnostics. Do not classify by parsing upstream error wording.
|
||
4. Keep explicit-profile selection and credential-free inspection. Nil optional
|
||
inspector assets remain usable in tests; compose no modules in the framework.
|
||
5. Remove dead option helpers/fields, except old fingerprint inputs still used
|
||
until Stage 7. A new catalog marker is not a substitute for captured identity.
|
||
|
||
Acceptance: `cmd/notarius` builds; both construction paths use the same policy.
|
||
No upstream type leaks into domain/pipeline contracts. Report remaining test-
|
||
only compile failures for fixture migration.
|
||
|
||
Validation: native `GOWORK=off go build` of `./cmd/notarius` to an executable
|
||
beneath a `mktemp -d` directory outside the repository; `git diff --check`.
|
||
|
||
Prompt size: constructor/error call sites only.
|
||
|
||
## Stage 4 — Migrate LLM Fixtures And Restore The Adapter Suite
|
||
|
||
> Completion: complete. `GOWORK=off go test ./internal/framework/llm` passes
|
||
> offline; legacy keyed-format and removed-API fixtures are cleared from the
|
||
> LLM adapter suite. CLI and registered-module fixtures remain for Stage 5.
|
||
|
||
Goal: exercise released configuration through existing adapter coverage.
|
||
|
||
Inspect `promptkit_client_test.go`, `promptkit_observation_test.go`,
|
||
`asset_registry_test.go`, and helpers under `internal/framework/llm`.
|
||
|
||
Work:
|
||
|
||
1. Convert literal files to keyed documents. Replace `WithProfiles` with
|
||
`ProfileSource` inputs and pointer fields preserving omission/explicitness.
|
||
Update engine setup to consume registry sources plus schema/runtime options.
|
||
2. Translate `base_profile` to `extends`. Deliberately adjust tests of whole-
|
||
profile replacement and selection-time failures only. Invalid unused final
|
||
entries fail construction; superseded contributions are not independently
|
||
domain-validated.
|
||
3. Preserve prepared execution, repairs, correction pairs, session/reasoning,
|
||
admission, transient errors, observations, cancellation, schemas, and safe
|
||
debug coverage. Fix owning adapter regressions instead of weakening tests.
|
||
4. Reuse small setup helpers; no legacy-format test converter. Fingerprint
|
||
fixtures become keyed now, with behavior changed later in Stage 7.
|
||
|
||
Acceptance: the full LLM suite compiles/passes offline, with no removed API
|
||
fixtures. Intentional semantic expectation changes are clear in the diff.
|
||
|
||
Validation: `GOWORK=off go test ./internal/framework/llm`; `git diff --check`.
|
||
|
||
Prompt size: mechanical adapter fixture migration and restoring its suite.
|
||
|
||
## Stage 5 — Restore CLI And Registered-Module Fixture Coverage
|
||
|
||
> Completion: complete. CLI and registered-module fixtures now use keyed
|
||
> sources, `GOWORK=off go test ./internal/cli ./internal/modules/...` and
|
||
> `GOWORK=off go test ./...` pass offline, and the integration reference
|
||
> describes the pinned v0.12 source API. Captured fingerprint migration remains
|
||
> scheduled for Stage 7.
|
||
|
||
Goal: close the initial format/API migration window.
|
||
|
||
Inspect `internal/cli/promptkit_profiles_test.go`, `production_contract_test.go`,
|
||
D&D `register/prompt_cache_test.go`, and direct PromptKit setup in chunk,
|
||
extract, normalize, and validator prompt asset tests. Discover remaining
|
||
legacy fixtures across tests, not just this initial inventory.
|
||
|
||
Work:
|
||
|
||
1. Migrate direct construction and keyed fixtures mechanically through the
|
||
new registry boundary. Keep fake outputs and domain expectations intact.
|
||
2. Change CLI expectations only for eager definition/source failures; preserve
|
||
unknown-profile, credential-free validation, cancellation, and safe errors.
|
||
Do not introduce a required API-key placeholder.
|
||
3. Prepare registered prompts using representative existing tests. Fix real
|
||
content-context/schema/mount regressions without rewriting prompt text.
|
||
4. Update the integration reference's supported pin/API links and basic
|
||
constructor boundary. Full inspection/deployment docs follow later.
|
||
|
||
Acceptance: the full offline suite compiles/passes, registered assets prepare,
|
||
and the integration reference no longer describes removed APIs as current.
|
||
|
||
Validation: `GOWORK=off go test ./internal/cli ./internal/modules/...`, then
|
||
`GOWORK=off go test ./...`; `git diff --check`.
|
||
|
||
Prompt size: mechanical consumer fixture sweep, no new domain behavior.
|
||
|
||
## Stage 6 — Prove Native Operator Composition And Credential Policy
|
||
|
||
> Completion: complete. The source-composition tests exercise imported combined
|
||
> operator documents through runtime and preflight, layered and strict-group
|
||
> behavior, local-backend precedence, and optional/required synthetic
|
||
> credentials. `GOWORK=off go test ./internal/framework/llm ./internal/cli` and
|
||
> `git diff --check` pass.
|
||
|
||
Goal: verify accepted unified sources through the shared boundary.
|
||
|
||
Inspect source assembly, profile/preflight tests, and adapter source tests.
|
||
Use small real files and real PromptKit loading.
|
||
|
||
Work:
|
||
|
||
1. Add a combined operator document with profiles, a custom backend, a catalog
|
||
backend field override, and a partial embedded-prompt overlay. Assert
|
||
preflight/runtime construct the same effective definitions.
|
||
2. Cover an import, a derived profile, omission retaining a field, explicit
|
||
zero/false, and null clearing an inherited tier/provider map. Cover `{}`
|
||
versus null only where consequential for the workflow.
|
||
3. Cover directory-group conflicts versus valid later-layer overrides,
|
||
selected-missing/empty sources, an invalid lower field repaired by an
|
||
overlay, and an invalid unused final entry. No raw-layer audit is added.
|
||
4. Cover explicit Notarius local endpoint/limit precedence and custom backend
|
||
admission through existing injected-client tests, not an upstream matrix.
|
||
5. Check optional versus required credential policy with synthetic names.
|
||
Inspection succeeds unset; preparation/execution follows native requirements.
|
||
Explicit profile environment sources differ from inherited backend sources.
|
||
|
||
Acceptance: native presence/composition works without a second loader,
|
||
whitelist, implicit credential requirement, or generation during preflight.
|
||
|
||
Validation: `GOWORK=off go test ./internal/framework/llm ./internal/cli`;
|
||
`git diff --check`.
|
||
|
||
Prompt size: operator-source fixtures, not identity or content lifecycle.
|
||
|
||
## Stage 7 — Compute Immutable Captured Configuration Identity
|
||
|
||
> Completion: complete. Checkpoint identity is now a single digest captured
|
||
> from each constructed engine's effective configuration, profile targets,
|
||
> credential policy, and prompt hashes. Focused tests confirm source edits do
|
||
> not mutate an existing engine's identity, provider parameters and prompt
|
||
> bodies affect identity without being exposed, and local scheduling values
|
||
> remain excluded. `GOWORK=off go test ./internal/framework/llm`,
|
||
> `GOWORK=off go mod tidy -diff`, and `git diff --check` pass.
|
||
|
||
Goal: replace live-file fingerprinting with the settled snapshot algorithm.
|
||
|
||
Inspect `promptkit_profile_fingerprint.go`, client construction/fingerprint
|
||
methods, public inspection APIs, and module/schema identity contributions.
|
||
|
||
Work:
|
||
|
||
1. Implement the private captured projection and digest specified above using
|
||
the just-constructed engine, never a second load or current file scan.
|
||
2. Inspect all effective profile IDs, including valid partial definitions.
|
||
Hash presence-aware definitions plus resolved targets and captured prompt
|
||
hashes; redacted reports or zero-valued targets alone are insufficient.
|
||
3. Store only the resulting fingerprint slice. Delete old file/directory scans,
|
||
catalog-only marker, local-target fingerprint, and obsolete client fields.
|
||
4. Preserve local scheduling-only exclusion and module/schema/override identity.
|
||
Change no chunk-plan key or resume eligibility. Update focused fingerprint
|
||
tests for the new boundary.
|
||
|
||
Acceptance: immutable `promptkit_configuration` reflects the captured engine
|
||
and pinned graph. The fingerprint API performs no later read/reload. No raw
|
||
payload is persisted, returned, or logged.
|
||
|
||
Validation: `GOWORK=off go test ./internal/framework/llm`; private projection/
|
||
propagation review; `git diff --check`.
|
||
|
||
Prompt size: one private snapshot/digest mechanism; broader reuse tests follow.
|
||
|
||
## Stage 8 — Protect Captured Identity And Resume Behavior
|
||
|
||
> Completion: complete. Added edit-after-construction coverage for an imported
|
||
> inherited profile and its source-relative prompt message: the existing client
|
||
> retains its digest and execution behavior, while a new client captures both
|
||
> edits. Fingerprint tests cover model, endpoint, credential policy, redacted
|
||
> provider parameters, prompt content, null-versus-zero presence, and local
|
||
> scheduling exclusions without exposing their values. The recorded-checkpoint
|
||
> test confirms same-snapshot reuse and stale execution misses under the new
|
||
> `promptkit_configuration` identity; existing override and chunk-plan checks
|
||
> remain passing. `GOWORK=off go test ./internal/framework/llm ./internal/cli
|
||
> ./internal/modules/integration` and `git diff --check` pass.
|
||
|
||
Goal: prove identity correctness across edits and checkpoint reuse.
|
||
|
||
Inspect adapter identity tests, CLI fingerprint handoff, and existing recorded-
|
||
checkpoint/resume integration fixtures.
|
||
|
||
Work:
|
||
|
||
1. Construct A from operator definitions/imports/message files. Edit them;
|
||
A's captured content and fingerprint remain unchanged. B captures semantic
|
||
definition/message changes and receives a different identity.
|
||
2. Cover model, endpoint, credential-policy, inherited/imported changes, null
|
||
versus zero, and provider parameters whose keys are report-redacted. Cover
|
||
local concurrency-only identity separately from local endpoint changes.
|
||
3. Check detached fingerprint ownership and absence of raw source/settings/
|
||
bodies in values or ordinary metadata. Compare behavior, not fixed hashes.
|
||
4. Extend one recorded-checkpoint workflow: changed captured config prevents
|
||
stale execution reuse; same-snapshot reuse, override identity, and source-
|
||
only chunk-plan reuse remain intact.
|
||
|
||
Acceptance: reusable state follows the executing engine, not edited files;
|
||
local scheduling-only changes do not accidentally change the chosen identity.
|
||
|
||
Validation: `GOWORK=off go test ./internal/framework/llm ./internal/cli ./internal/modules/integration`;
|
||
`git diff --check`.
|
||
|
||
Prompt size: snapshot/resume regressions, no checkpoint redesign.
|
||
|
||
## Stage 9 — Verify Source-Relative Content And Capture Lifetime
|
||
|
||
> Completion: complete. Added integration coverage showing imported content
|
||
> resolves from its declaring directory, replacement content resolves from its
|
||
> replacement source while retaining the embedded prompt's other fields, and
|
||
> constructed engines keep captured messages across file edits while rebuilt
|
||
> engines observe changes. Static template and missing-file errors remain
|
||
> construction failures; missing request variables remain preparation errors.
|
||
> `GOWORK=off go test ./internal/framework/llm ./internal/modules/...` and
|
||
> `git diff --check` pass.
|
||
|
||
Goal: protect resource contexts and static template/content capture.
|
||
|
||
Inspect manifests, source construction, and existing prepared/prompt asset
|
||
tests; consult upstream origin/frozen-rendering contracts for timing.
|
||
|
||
Work:
|
||
|
||
1. Use directory-distinguishing files to prove an imported/inherited message
|
||
resolves from its declaring source. Add a later prompt replacement with
|
||
replacement-file-relative content and a partial overlay over embedded base.
|
||
2. Existing engines retain captured content/templates; a new engine sees edits.
|
||
Bad static syntax/missing message files fail construction, while missing
|
||
request variables remain preparation failures.
|
||
3. Keep schema preparation timing and inline request inputs unchanged. Preserve
|
||
representative rendered bytes/order/cache metadata. Fix only real context
|
||
or mounting defects; do not expand host-root policy.
|
||
|
||
Acceptance: no source is rebased to the caller/CWD; captured definitions remain
|
||
distinct from schemas and request artifacts.
|
||
|
||
Validation: `GOWORK=off go test ./internal/framework/llm ./internal/modules/...`;
|
||
`git diff --check`.
|
||
|
||
Prompt size: origin/lifecycle cases, not new execution policy.
|
||
|
||
## Stage 10 — Expose A Provider-Neutral Inspection Facade
|
||
|
||
Goal: support CLI reports without exposing upstream dependency types.
|
||
|
||
Inspect profile result/error types, shared source construction, and public
|
||
PromptKit inspection APIs.
|
||
|
||
Work:
|
||
|
||
1. Add adapter string/writer operations for effective/explanation reports.
|
||
Parse canonical pointers privately; preserve writer causes, invalid-versus-
|
||
unknown distinctions, and known historical paths. Keep upstream values private.
|
||
2. Extend the neutral profile result with copied fields for the settled JSON
|
||
DTO. Copy actual target, credential name, and `APIKeyRequired` metadata; do
|
||
not infer environment availability or provider defaults. Existing preflight
|
||
consumers retain their needed metadata.
|
||
3. Redact credential-shaped keys recursively in the reporting copy of provider
|
||
parameters. Reuse a suitable helper without reversing dependencies, or add
|
||
a small local projection using upstream's case-insensitive vocabulary:
|
||
`api_key`, `apikey`, `authorization`, `credential`, `credentials`,
|
||
`password`, `secret`, `token`, `access_token`. Do not reject those keys,
|
||
redact unrelated content, or mutate the private digest projection.
|
||
4. Test detached ownership, safe errors, parameter redaction, escaped pointers,
|
||
absent selectors, and unset synthetic credential names.
|
||
|
||
Acceptance: all three views are available without leaked types, credential
|
||
resolution, provider calls, automatic dumps, or mutation of identity.
|
||
|
||
Validation: `GOWORK=off go test ./internal/framework/llm`; `git diff --check`.
|
||
|
||
Prompt size: one facade/DTO, not CLI parsing.
|
||
|
||
## Stage 11 — Add Scoped Effective And Explanation Commands
|
||
|
||
Goal: expose captured configuration/history through explicit PromptKit scope.
|
||
|
||
Inspect `internal/cli/configkit_commands.go`, `runConfig`, usage text,
|
||
normalized composition options, and configuration command tests.
|
||
|
||
Work:
|
||
|
||
1. Add `--scope` with settled values/default; keep the ordinary report branch
|
||
unchanged. Only PromptKit scope constructs an inspector using selected
|
||
normalized config and registered assets.
|
||
2. Route pointer/history and writer failures through the facade and existing
|
||
exit classes. Allow empty root and escaped IDs. Do not invoke the LLM client
|
||
factory, resolve a pipeline, prepare requests, or materialize references.
|
||
3. Add focused CLI coverage for default report preservation, scopes, malformed/
|
||
unknown/historical selectors, import/overlay origins, redaction, and writer
|
||
failure. Use zero-call counters at external boundaries.
|
||
4. Update help and `docs/cli.md` syntax/output behavior. State that construction
|
||
validates/captures definitions and requested reports can contain ordinary
|
||
inline prompt/provider data.
|
||
|
||
Acceptance: current invocations retain meaning; explicit scope produces only
|
||
its requested report without transcript input, credentials, or provider work.
|
||
|
||
Validation: `GOWORK=off go test ./internal/cli`; `git diff --check`.
|
||
|
||
Prompt size: two scoped branches, not resolved-profile parsing.
|
||
|
||
## Stage 12 — Add Resolved Profile Inspection To The CLI
|
||
|
||
Goal: report backend-completed profile settings explicitly.
|
||
|
||
Inspect command dispatch, the profile facade, preflight, and JSON/error tests;
|
||
reuse Stage 11's inspector construction.
|
||
|
||
Work:
|
||
|
||
1. Add `config profile`, settled flags, and exactly one nonblank literal ID.
|
||
Emit the neutral DTO as one JSON object/newline, not the upstream Go struct.
|
||
2. Preserve syntax/runtime exit distinctions and safe missing-profile/config/
|
||
writer failures. Inspect no credential availability and apply no run override.
|
||
3. Cover inheritance/backend defaults, endpoint-only identity, null suppression,
|
||
required unset credentials, credential-shaped params, and unknown profiles.
|
||
Assert fields rather than whole JSON snapshots or inferred provider defaults.
|
||
4. Update help and `docs/cli.md`: JSON shape, request/default distinction,
|
||
zero/presence caveat, failures, and links to effective/explanation.
|
||
|
||
Acceptance: detached redacted profile inspection needs no transcript or model
|
||
call; preflight/run behavior remains unchanged.
|
||
|
||
Validation: `GOWORK=off go test ./internal/cli ./internal/framework/llm`;
|
||
`git diff --check`.
|
||
|
||
Prompt size: one focused subcommand and its contract.
|
||
|
||
## Stage 13 — Maintain Examples And Operator Guidance
|
||
|
||
Goal: make native features understandable and deployable.
|
||
|
||
Inspect the two maintained config examples, imported pipeline/development
|
||
variant, deployment profile file, example tests, config and operations docs.
|
||
|
||
Work:
|
||
|
||
1. Keep two top-level examples and normal default execution policy. Add a
|
||
small derived development profile to the existing deployment profile file,
|
||
demonstrating `extends` and a supported override without an unavailable
|
||
model. Do not select it for the normal complete pipeline or duplicate steps.
|
||
2. Explain unified sources, recursion/exact keys, imports, strict directories,
|
||
source roots, and Notarius overlays versus PromptKit layers. Use small
|
||
snippets for custom/catalog backends, local null clearing, prompt overlays,
|
||
and file-root imports. Add no third full pipeline/example framework.
|
||
3. Replace old whole-replacement, `base_profile`, embedded-ID, unsupported file
|
||
`api_key_required`, and stale format claims with v0.12 behavior. Include
|
||
external-file migration guidance, not aliases or a compatibility parser.
|
||
4. Add a concise operations workflow using three inspection views. Explain
|
||
engine reconstruction, credential names/availability, zero/presence, report
|
||
versus prepared/debug data, and operator-owned report handling.
|
||
5. Update existing example tests for the additional profile and actual native
|
||
behavior. Validate minimal, complete, and inherited development pipelines
|
||
offline with no required credentials or file-length assertions.
|
||
|
||
Acceptance: valid examples and accurate canonical config/operations guidance,
|
||
linked to CLI/upstream contracts rather than duplicating their manuals.
|
||
|
||
Validation: `GOWORK=off go test ./internal/core/config ./internal/cli`;
|
||
`git diff --check`; verify local/pinned links.
|
||
|
||
Prompt size: maintained examples and two operator documents.
|
||
|
||
## Stage 14 — Finish Developer Contracts And Record The Decision
|
||
|
||
Goal: document implemented ownership, capture, identity, and inspection.
|
||
|
||
Inspect `docs/internal/llm.md`, `docs/internal/cli.md`,
|
||
`docs/integrations/pkg-promptkit.md`, ADR-0010/0018, and final implementation.
|
||
|
||
Work:
|
||
|
||
1. Update LLM internals for shared groups, final-value validation, capture,
|
||
snapshot identity, schema lifecycle, and neutral reports. Remove live-file
|
||
fingerprint claims.
|
||
2. Update CLI internals for scoped/profile reports, source parity, and credential-
|
||
free construction. Keep exact syntax/defaults in public references.
|
||
3. Finish pinned v0.12 and `/v3` integration guidance. Preserve prepared
|
||
execution, observations, corrections, retry separation, omission, and resource
|
||
timing. Explain final-only validation, not superseded-source audits; verify
|
||
license links.
|
||
4. Add `docs/adr/0019-adopt-native-promptkit-configuration.md`: native overlays,
|
||
unified sources/ownership, capture, compatibility, and alternatives. Add a
|
||
concise status/supersession link to ADR-0010 for only its whole-profile
|
||
replacement clause; preserve historical rationale and workload ownership.
|
||
ADR-0018's Notarius loader decision remains in force.
|
||
5. Repair other genuinely affected links/claims. Do not rewrite policy
|
||
provenance, historical releases, or unrelated artifact contracts.
|
||
|
||
Acceptance: actual final behavior has canonical owners and a durable decision
|
||
record without erasing history or elevating incidental choices into policy.
|
||
|
||
Validation: documentation/link review, scoped stale-term search, and
|
||
`git diff --check`. Reuse code/example checks when unchanged; no new scanner
|
||
or paid inference is required for prose.
|
||
|
||
Prompt size: focused internal/integration references and one short ADR.
|
||
|
||
## Stage 15 — Verify Completeness And Remove Only Obsolete Machinery
|
||
|
||
Goal: prove the entire target state, not just a green dependency build.
|
||
|
||
Work:
|
||
|
||
1. Compare code/tests/examples/docs against every target section and settled
|
||
decision. Address small finishing defects. Report significant gaps rather
|
||
than silently narrowing source, identity, or reporting acceptance.
|
||
2. Find genuinely obsolete APIs, formats, option/fingerprint helpers, and
|
||
contradictory current documentation. Distinguish historical migration text
|
||
and normal application identifiers from obsolete PromptKit definitions.
|
||
Remove dead machinery, not retained behavioral tests. Review restrictions/
|
||
guarantees for accidental strengthening.
|
||
3. Run full offline tests, vet, tidy diff, and native temporary build. Verify
|
||
selected modules. With provider credential variables unset, smoke-test
|
||
help, all three new reports, and maintained complete/development pipeline
|
||
validation. No real run or inference is needed.
|
||
4. Check report source parity/redaction, captured identity/reuse and origin
|
||
coverage, attribution/source links, and local navigation. Optional live
|
||
evaluation is follow-up evidence, not a new release gate.
|
||
5. Report completion and actual validation. Leave release selection/publication
|
||
and completed-roadmap retirement to a separate request. No file change means
|
||
no empty commit.
|
||
|
||
Acceptance: all roadmap capabilities are implemented/demonstrated, deterministic
|
||
checks pass, no removed configuration-source API remains in implementation,
|
||
and no unrelated contract/restriction was introduced.
|
||
|
||
Validation:
|
||
|
||
```sh
|
||
GOWORK=off go test ./...
|
||
GOWORK=off go vet ./...
|
||
GOWORK=off go mod tidy -diff
|
||
GOWORK=off go list -m gitea.maximumdirect.net/eric/ConfigKit gitea.maximumdirect.net/eric/promptkit gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v3 gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/v3
|
||
git diff --check
|
||
```
|
||
|
||
Build beneath a `mktemp -d` directory and use that executable for smoke tests;
|
||
keep temporary products outside the repository. These are migration checks,
|
||
not additions to release policy or production-key requirements.
|
||
|
||
Prompt size: final evidence review/small cleanup, no new feature.
|
||
|
||
## Open Questions
|
||
|
||
None. Native overlays, unified sources/imports, and operator inspection are
|
||
accepted. Command syntax, report shapes, source precedence, captured identity,
|
||
and migration boundaries are specified above. Raise any new upstream API
|
||
incompatibility with evidence; do not silently drop a feature or add a shim.
|