From 1c8494ec204ff75d35b311360680df26a1e0358a Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 4 Oct 2026 13:57:36 +0000 Subject: [PATCH] Record PromptKit adoption review findings --- docs/roadmap/remediation.md | 122 +++++++++++++++++++++++++++++++++++- 1 file changed, 120 insertions(+), 2 deletions(-) diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md index eb89013..e019b7c 100644 --- a/docs/roadmap/remediation.md +++ b/docs/roadmap/remediation.md @@ -1,16 +1,129 @@ # Implementation Review Remediation +## PromptKit v0.12.0 Review + +Reviewed on 2026-10-04 at +`58cb02f9732b09740ff7e22096d2a4072ddfddca`, covering the thirteen implementation +commits after `b1bb55a`. The runtime architecture matches the accepted +[feature roadmap](promptkit.md) and [implementation plan](implementation.md). +No runtime regression, missing framework capability, or newly imposed runtime +restriction was identified. Three low-priority documentation/help gaps need +correction. The first retains a loading guarantee that is incompatible with +the deliberately selected full-source engine construction. + +### R3: Correct The Definition-Inspection Loading Contract + +Priority: P3 (low). Status: Open. + +Locations: `docs/cli.md:208–224`; related orientation in +`docs/internal/adapters.md:49–56`. The CLI wording was retained in `b913ef0` +despite the engine-wiring change in `0df2aad`. + +The profile-inspection reference still says it never loads a prompt. Both +definition-inspection commands now construct the same complete selected engine +as execution. PromptKit resolves and validates all final definitions and +captures their message content before the selected inspection runs. Consequently, +an unused prompt with invalid syntax or an unreadable `content_file` can prevent +inspection of an otherwise valid profile. Prompt inspection's unqualified +claim that it does not resolve a profile also needs to distinguish engine +assembly from request-specific target resolution. + +Evidence: inspecting a valid profile in a selected combined file with an +unused prompt referring to a missing message file exits 1 before emitting a +profile report. The existing +`TestProfileInspectionUsesCombinedSourcesAndValidatesUnusedPrompts` also +protects this intentional eager-validation boundary. + +Required change: explain that both commands construct and validate the full +selected configuration, including unused definitions and message resources, +before inspecting the chosen definition. Preserve the distinction that they +do not prepare an execution request, load request-time schemas/artifacts, +resolve credential values, reserve capacity, or call a provider. Clarify that +no prompt collection is mandatory, but selected prompts are not ignored. +Align the internal summary with the CLI contract rather than restating a +stronger independence guarantee. Do not restore the empty-prompt-filesystem +workaround or weaken eager validation to satisfy the stale wording. + +Acceptance: the owning contract and internal summary agree with the roadmap, +shared engine construction, and existing tests. Review the documentation diff +and links; no production-code change or new runtime guarantee is needed. + +### R4: Update The Composed-Example Description + +Priority: P3 (low). Status: Open. + +Location: `docs/config.md:95–96`, after the example migration in `2027b80`. + +The reference says `examples/config.composed.yml` imports two application +fragments. It now imports only `config-fragments/sources.yml`; the former +backend fragment became `examples/framework/backends.yml`, selected through +`promptkit_sources` rather than imported into the application document. +The example and code are correct, but the explanation confuses the two source +boundaries introduced by this migration. + +Required change: describe the actual source-selection fragment and separately +selected framework backend file, with links where useful. Do not reintroduce +an application backend mapping or add an unnecessary fragment merely to match +the old prose. The old backend fragment had no application defaults to preserve. + +Acceptance: the description matches the maintained files and the existing +composed-overlay render command still succeeds offline. + +### R5: Include Framework Configuration Inspection In Built-In Usage + +Priority: P3 (low). Status: Open. + +Location: `internal/adapter/cli/run.go:742–749` (`printUsage`), following the +new command-family implementation in `2ba2ce1` and `411a67b`. + +The top-level usage lists `inspect prompt`, `inspect profile`, and application +`config`, but omits the implemented `inspect configuration` family entirely. +Invoking the binary without arguments reproduces this omission, so users +relying on its built-in command summary cannot discover the new framework +validation/provenance interface or distinguish it from application inspection. + +Required change: add a concise usage line identifying all four operations, +the supported selection flags, and the `--path` requirement for `explain`. +Preserve existing exit behavior and command parsing; this does not require a +new help subsystem or aliases. Protect discoverability with a small CLI-entry +assertion for the family/operations, not a snapshot of the entire usage text +or its incidental ordering. + +Acceptance: built-in usage advertises the actual family and remains consistent +with `docs/cli.md`; the focused CLI check and development baseline pass. + +### Review Verification + +- `go test -count=1 ./...`, `go test -race -count=1 ./...`, and + `go vet ./...` passed. A native binary built outside the repository. +- Changed Go files are formatted; diff whitespace checks passed. +- All four maintained application configurations passed both application and + framework validation with provider credentials unset. Both render scripts, + composed-overlay render, and inheritance render passed offline. +- All four captured framework inspection operations passed against the complete + example. Focused tests cover native layering/imports/resource capture, + literal IDs, presence/null inheritance, partial-profile completion, scoped + effective/history redaction, and unchanged existing inspection behavior. +- The selected graph is PromptKit v0.12.0, ConfigKit v0.5.0, and the two `/v3` + catalogs at v3.20260930.0, without a local replacement or workspace. Selected + PromptKit contract and notice links resolved. +- The broader redaction in the new framework-report family, removed legacy + source fields/flags, and eager definition capture are explicitly approved + changes, not accidental strengthening. Shared server capacity, request + independence, HTTP wire behavior, and lexical artifact containment remain + intact. No implementation fixes were made during this review. + ## ConfigKit v0.5.0 Review Reviewed on 2026-10-03 at `bcf2ddf`, covering the five implementation commits after `e1ce526`. The ConfigKit adoption and runtime ownership boundaries match the accepted roadmap. No runtime defect, missing application feature, or unrequested strengthening of contracts was identified. One -documentation handoff issue remains open below. +documentation handoff issue was identified; its current status is recorded below. ### R2: Correct The Completed Plan's Introductory Status -Priority: P3 (low). Status: Open. +Priority: P3 (low). Status: Superseded; no action against the current plan. Location: `docs/roadmap/implementation.md:5–9`. @@ -37,6 +150,11 @@ that implementation is complete, while release preparation remains separate. Review the documentation diff and run `git diff --check`; this correction does not require new tests. +The ConfigKit plan was replaced by the PromptKit plan in `b1bb55a`. Its +historical snapshot remains in Git, while the current introduction correctly +states that all thirteen PromptKit stages are complete. This old finding does +not direct changes to the new plan or repetition of either implementation. + ### Review Verification - `go test -race -count=1 ./...` passed, including the new configuration,