# PromptKit Configuration Migration Review Remediation ## Scope And Overall Assessment Review baseline: Notarius `6700ed2`, compared with planning revision `f5b066b`, the accepted [feature roadmap](promptkit.md), and [implementation plan](implementation.md). PromptKit v0.12.0 is pinned at `4b8169da286917fde7875425bb32b799b12c6af3`; ConfigKit remains v0.5.0, with both PromptKit `/v3` catalogs at v3.20260930.0. The migration is substantially implemented. Native sources, combined operator documents and imports, recursive overlays, source-relative prompt content, captured engine lifetime, immutable checkpoint identity, and credential-free operator inspection are in place. The provider-neutral execution boundary, prepared execution, retries, scheduling, and durable artifact contracts remain intact. All 16 migrated prompt bodies match the previous definitions after removing their old embedded identity fields; prompt fragments and schemas are unchanged. Four bounded follow-ups remain below. No production code is changed by this review. No additional runtime security restriction, mandatory credential check, filesystem permission policy, compatibility shim, or release gate was found. Eager final-definition validation and the external keyed format are intentional upstream changes, not accidental strengthening. The previously recorded ConfigKit remediation was completed before this work set; it is not outstanding. ## Finding 1 — Compose Registered Application Assets In One Strict Group Priority: P2. Category: architecture/correctness gap. Status: open. Evidence: - `internal/framework/llm/asset_registry.go:106–139` creates one underlay per registered prompt or fallback-profile filesystem. - `internal/framework/llm/asset_registry_sources_test.go`, `TestPromptKitConfigurationSourcesKeepsNativeFilesystemGroups`, asserts that separate registrations yield separate underlays, encoding this divergence. - The plan's Sources And Engine Ownership section calls for one application underlay whose inputs each retain their own filesystem context; `docs/internal/llm.md:86–92` also describes that behavior. Separate underlays have recursive overlay semantics. Conflicting application definitions in different registrations therefore silently depend on module registration order instead of receiving native strict-composition diagnostics. An offline probe registered two fallback files with the same profile key and different models. The current inspector accepted them and selected the second model. The exact same filesystem inputs placed in one native underlay correctly failed composition. Distinct filesystem contexts do not require separate layers. Required changes: - Gather the native `FilesystemInputs` from every application prompt and fallback-profile registration into one application underlay. Preserve each input's existing label, filesystem, root, and disabled-import setting. Keep schemas on their separate runtime option and omit an empty application group. - Keep the operator and explicit local-backend contributions as later groups. Preserve their accepted recursive overrides and source precedence. Do not add an application-specific duplicate-key detector or flatten filesystem contexts into an artificial shared root. - Replace the incidental group-count test with behavioral coverage: conflicting application contributions fail through native strict composition; distinct declarations retain source-relative content; an operator overlay over the resulting application base still succeeds. Reuse existing origin coverage. Acceptance: application conflicts are not silently resolved by registration order; source-relative imports/content and valid later operator overlays remain unchanged. Runtime, preflight, and reports use the corrected shared construction. Validation: `GOWORK=off go test ./internal/framework/llm ./internal/cli ./internal/modules/...`. ## Finding 2 — Validate Explanation Syntax Before Loading Configuration Priority: P3. Category: CLI correctness. Status: open. Evidence: `internal/cli/configkit_commands.go:77–106`. PromptKit scope loads Notarius configuration and constructs the inspection engine before validating the pointer; the default Notarius scope parses the pointer first. Reproduction, using the native temporary build and a nonexistent config path: ```sh notarius config explain --scope promptkit --config /path/to/missing.yml '/profiles/~2' ``` The current PromptKit command returns exit 1 with a config-file error. The same malformed pointer returns exit 2 with a valid config, and the Notarius scope returns exit 2 even with the missing config. A bad PromptKit definition or missing content file can likewise mask malformed command syntax. This differs from the selected syntax/runtime distinction in the plan and CLI reference. Required changes: - Validate canonical JSON Pointer syntax once, before either scope loads config or constructs an engine. Reuse the existing parser; retain defensive parsing inside the adapter for direct callers. - Keep the empty root pointer, escaped literal keys, historical-only paths, unknown-path exit 1, and writer-error behavior unchanged. - Extend CLI coverage with a malformed pointer and unavailable configuration; verify exit 2, no stdout, and no configuration/provider work. No broader argument-parser refactor is needed. Acceptance: malformed pointers are syntax errors independent of configuration availability for both scopes; valid selectors still load and inspect normally. Validation: `GOWORK=off go test ./internal/cli ./internal/framework/llm`. ## Finding 3 — Correct Lifecycle And Fallback Documentation Priority: P3. Category: documentation accuracy. Status: open. Evidence: - `docs/config.md:321–323` says prompt **and schema** definitions are validated at engine construction. Schemas remain preparation-time resources supplied through `WithSchemaFS`, not captured configuration definitions. - `docs/internal/llm.md:71–77` and `docs/internal/cli.md:55–61` describe preflight as occurring without loading a prompt. Construction now loads and validates effective prompt definitions and captures message files/static templates; it does not prepare a request or generate output. - The comment on `RegisterFallbackProfileFS` in `internal/framework/llm/asset_registry.go:61–62` still says fallback assets are used only when the operator source lacks the ID. Under recursive overlays, their omitted fields remain active beneath a matching operator entry. Required changes: - Distinguish constructor-time definition/message capture from preparation-time schema loading and request-variable/artifact resolution. State accurately that preflight needs no transcript, credential value, or provider call; do not promise it performs no prompt-content file reads. - Describe fallback registration as a lower-precedence application contribution rather than an all-or-nothing lookup fallback. - While editing this text, clarify that an operator `backends.local` declaration can supply the backend as well as Notarius's optional `local_backend` object. The latter is a convenience contribution, not a required second declaration. Link canonical source rules rather than duplicating them. - Change documentation/comments only unless verification exposes a real runtime defect. Do not move schema loading earlier to make the stale prose true. Acceptance: public and internal guidance consistently explains actual lifecycle, native layering, and operator backend configuration without inventing a stronger preflight guarantee or an additional required config field. Validation: inspect current constructors/prepared execution and upstream v0.12 contracts; review affected local links; run `git diff --check`. Reuse unchanged behavioral checks rather than adding prose-snapshot tests. ## Finding 4 — Preserve ADR-0010's Historical Decision Body Priority: P3. Category: documentation-policy compliance. Status: open. Evidence: the migration diff rewrites ADR-0010's Decision and Consequences paragraphs at `docs/adr/0010-workload-oriented-llm-profile-defaults.md:26–33` and `:47–50`, as well as adding the appropriate status/supersession link. `docs/policy/documentation.md` requires accepted decision content to remain immutable; Stage 14 specified a concise status link rather than rewriting the historical body. The replacement correctly acknowledges the old behavior, but the historical record should retain its original wording. ADR-0019 already owns the new native overlay decision and accurately limits which part of ADR-0010 it supersedes. Required change: restore ADR-0010's Decision and Consequences body from `f5b066b`, keeping its new partial-supersession status and ADR-0019 link. Leave workload ownership and ADR-0019 unchanged. Current runtime behavior remains documented in the canonical current references, not retrofitted into the older decision. Acceptance: the historical body matches the accepted original, the changed clause's supersession remains discoverable, and no runtime behavior changes. Validation: compare the body with `git show f5b066b:docs/adr/0010-workload-oriented-llm-profile-defaults.md`, verify the ADR link, and run `git diff --check`. ## Review Evidence And Validation - `GOWORK=off go test ./...` passed. - `GOWORK=off go vet ./...` passed. - `GOWORK=off go test -race ./internal/framework/llm ./internal/cli` passed. - `GOWORK=off go mod tidy -diff` produced no changes. - Module inspection confirmed the selected ConfigKit, PromptKit, and catalog pins. - A native build outside the repository succeeded. Credential-free profile and explanation commands and complete/development pipeline validation succeeded. - A YAML-aware comparison confirmed all 16 migrated prompt definition bodies are unchanged after removing old embedded IDs/versions. Prompt fragments, schemas, and the embedded D&D execution settings are unchanged. - An offline Go overlay probe, stored entirely outside the repository, confirmed the application-group conflict behavior in Finding 1. CLI invocations reproduced Finding 2 without generation or provider calls. - The implementation diff passed whitespace checks. No production inference or production model-quality evaluation was performed. Recommended remediation is limited to these findings, not another configuration system, new security policy, or unrelated pipeline refactoring.