From 654d1b4a6dfe20ee2f16b3bccbeb374887bdac35 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 18 Oct 2026 13:48:31 +0000 Subject: [PATCH] Retire completed upgrade planning documents --- docs/development.md | 2 +- docs/roadmap/implementation.md | 915 --------------------------- docs/roadmap/promptkit-validation.md | 370 ----------- docs/roadmap/remediation.md | 213 ------- docs/roadmap/upgrades.md | 530 ---------------- 5 files changed, 1 insertion(+), 2029 deletions(-) delete mode 100644 docs/roadmap/implementation.md delete mode 100644 docs/roadmap/promptkit-validation.md delete mode 100644 docs/roadmap/remediation.md delete mode 100644 docs/roadmap/upgrades.md diff --git a/docs/development.md b/docs/development.md index 3cb586e..af5be1a 100644 --- a/docs/development.md +++ b/docs/development.md @@ -53,7 +53,7 @@ canonical documentation owners. | Public facade or adapter boundaries | [Internal overview](internal/overview.md), [ADR 0003](adr/0003-public-facade.md) as scoped by [ADR 0010](adr/0010-optional-integration-modules.md), and the architecture policy | | Module ownership and public contracts | [Consumer guide](consumers/guide.md), [internal overview](internal/overview.md), [core GoDoc](../doc.go), [ckconfig GoDoc](../integrations/ckconfig/doc.go), [ckprompt GoDoc](../integrations/ckprompt/doc.go), and applicable ADRs | | Deferred capabilities and proposals | [Deferred work](roadmap/deferred.md) | -| Completed planning and implementation history | [Completed ConfigKit and PromptKit adoption and remediation](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/8a82620dfcd34c0ae56bbecb0d014eb49786bc92/docs/roadmap), [earlier retired records](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/6bc37e7ba64ca0920b4d0cc77039a18d4006b24e/docs/roadmap), and [completed module implementation and remediation](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/bb2528532b78d30e52f5082e491a5e1fd9b0fb99/docs/roadmap) in Git history | +| Completed planning and implementation history | [Completed dependency upgrades, PromptKit validation, discovery, smoke testing, and remediation](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/a6278a6d9f940948ec7745955d745523aaff0734/docs/roadmap), [earlier ConfigKit and PromptKit adoption and remediation](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/8a82620dfcd34c0ae56bbecb0d014eb49786bc92/docs/roadmap), [earlier retired records](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/6bc37e7ba64ca0920b4d0cc77039a18d4006b24e/docs/roadmap), and [completed module implementation and remediation](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/bb2528532b78d30e52f5082e491a5e1fd9b0fb99/docs/roadmap) in Git history | | Release tooling | [Maintainer release procedure](release.md) | | Performance maintenance | [Internal overview](internal/overview.md), existing benchmarks, and [deferred efficiency work](roadmap/deferred.md#possible-further-efficiency-work) | | Licensing or incorporated material | [Project grant](../COPYRIGHT), [license texts](../LICENSE), and [attribution](oss/attribution.md) | diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index 13748b1..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,915 +0,0 @@ -# Dependency upgrades, discovery, and smoke testing implementation plan - -Status: Stages 1–18 complete. - -## Authority and baseline - -Implement the [accepted upgrade roadmap](upgrades.md) by following Stage 1 -through Stage 18 in order. Each stage is sized for one Luna coding prompt. -The roadmap owns the selected target and intent; this plan specifies the work -to reach it. The [architecture policy](../policy/architecture.md), accepted -[ADRs](../adr/), and applicable policies remain authoritative. - -The planning baseline is `38b49f29fb3a8f22ff02528d2a0ed413e5c6ab1e`. -It has ReleaseKit v0.4.0 at the root, ConfigKit v0.7.1 in both integrations, -and PromptKit v0.15.0. The target releases are ReleaseKit v0.4.2, ConfigKit -v0.8.0, and PromptKit v0.16.0, at the exact upstream revisions recorded in the -roadmap. The Go consumer minimum remains 1.26.0 and the selected maintainer -toolchain remains 1.26.8. - -This plan replaces the completed request-validation implementation plan, which -remains available in [Git history](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/38b49f29fb3a8f22ff02528d2a0ed413e5c6ab1e/docs/roadmap/implementation.md). -That feature is baseline behavior to migrate, not an unfinished assignment. - -## Working rules - -- Start with [development guidance](../development.md) and its policy reading - guide. Prefer the knowledge graph for code discovery; use current source when - graph evidence is incomplete. -- Deliver the complete roadmap in one coordinated sprint. Intermediate commits - need not build; stages identify useful units of work, not compatibility - layers or independently releasable intermediate APIs. Preserve established - contracts except where the accepted roadmap deliberately changes them. -- Keep core independent of optional libraries. ckconfig owns ConfigKit - adaptation; ckprompt may use ckconfig. Do not introduce the reverse - dependency or expose Cobra/pflag types. -- Use released upstream modules. Adjacent checkouts are review evidence, not - committed replacements or sources of internal imports. -- Implement straightforwardly in the owning packages. Private file names below - are suggested organization; do not create frameworks, exported abstractions, - or additional internal layers merely to match the plan. -- Retain application ownership of factories, callbacks, configuration policies, - credentials, defaults, and returned objects. No arbitrary deep copies, - pointer repair, new caching policy, or concurrency guarantee is selected. -- Apply the [upstream testing ownership boundary](../policy/testing.md#upstream-integration-testing-ownership). - Test CommandKit's forwarding, coordination, presentation, and outcomes. - Do not independently verify upstream composition, namespace resolution, - prompt preparation, provider diagnostics, or inference correctness. -- Use small test doubles at external boundaries and real upstream fixtures - where useful. This does not require production mocking interfaces. Default - tests remain offline and credential-independent. -- Format changed Go files. Run focused checks appropriate to each stage; - broad integration checks follow when the affected migration is coherent. - Record actual completion evidence and commit each completed nonempty unit - under the [agent policy](../policy/agents.md#git-commit-policy). -- This sprint does not publish releases, create tags, choose future module - release numbers, or add release gates. Release preparation remains governed - by the existing release procedure. - -## Implementation choices used by the stages - -These details complete the accepted target for coding. They define this -implementation's shape, not new cross-library domain policy. - -### Catalog requests and callbacks - -Introduce two small public request types in ckprompt: - -```go -type CatalogRequest struct { - Selection ckconfig.Selection - Filter promptkit.NamespaceFilter -} - -type NamespaceRequest struct { - Selection ckconfig.Selection - Kinds []promptkit.ResourceKind -} -``` - -Use `CatalogRequest` for `Suite.ListPrompts`, `Suite.ListProfiles`, and the new -`Suite.ListBackends`. Use `NamespaceRequest` for the new `Suite.ListNamespaces`. -`Suite.ListCollections` takes the existing `ckconfig.Selection` directly: -its native operation has no resource filter to wrap. - -All callbacks retain their existing context, typed input `I`, shared state `S`, -and `commandkit.Acquired[T]` plus error pattern. Result types are: - -| Callback | Acquired value | -| --- | --- | -| `ListPrompts` | `[]promptkit.PromptIdentity` | -| `ListProfiles`, `ListBackends`, `ListNamespaces` | `[]string` | -| `ListCollections` | `[]promptkit.CollectionInfo` | - -`FromEngine` supplies the original inspection operations and all five catalogs: -eight independently optional callbacks in total. Request validation and smoke -testing are added explicitly by their respective helpers, giving ten possible -suite capabilities. No universal operation registry is needed. - -For the filterable catalogs, forward the native `Filter` to the corresponding -listing method, including the zero-valued `NamespaceDefault`. For namespaces, -forward `Kinds...` without implementing union, membership, or sorting locally. - -### Report shapes - -Keep existing report organization except for the accepted changes and additions: - -- `config sources` retains its scope/source structure. Add `destination` as the - native canonical JSON Pointer, `placement` as `mapping` or `sequence_item`, - `destination_unresolved` as a boolean, and nullable `version_check`. - Its nested fields are `path`, `version` (`present`, `null`, `text`), - nullable `location` (`source_id`, `line`, `column`), and `redacted`. - Preserve false, empty, and null native values. Text exposes the same evidence. -- Prompt inspection, requirements inspection, and prompt catalog entries use - `version` as `*string` with YAML `omitempty`. Nil omits the field; a pointer - to `""` emits an empty string. Text inspection omits an absent version and - quotes a present empty version. -- Profiles, backends, and namespaces are bare YAML string sequences. - Prompt entries remain `{id, version?}`; collection entries are - `{namespace, version?}`, projecting native `ReleaseVersion` into `version`. - Preserve provider order and exact identities. Nil/empty inventories emit `[]`. -- A single smoke test emits the native report projected into a private YAML DTO. - Use snake_case native field names, with descriptive `PromptVersion` rendered - as optional `version` inside its target under the accepted version convention. - Include all report fields and their native nested evidence; do not serialize - the request or engine. -- Aggregate smoke output is one YAML mapping with `tests`, an ordered sequence - of entries containing `kind`, nullable `report`, and optional `error` text. - The outer `kind` identifies the selected concrete operation; a non-nil native - report retains its own returned `kind`. No extra readiness summary is added. - Include attempted operations only after cancellation stops the sequence. - -For smoke DTOs, preserve scalar zero values, nullable tri-state observations, -settings-presence booleans, and token counts' `value`/`known`/`complete` fields. -Render `time.Time` through the existing YAML encoder and durations as Go -duration strings, preserving nil duration pointers as null. Render an empty -attempt list as `[]`. Keep native effective settings and `extra_params` data; -do not introduce a second sensitivity classification. - -The aggregate `error` is presentation text, not a serialized Go error object or -a replacement for the returned error's identity. For a single test, deliver any -returned report and use existing diagnostics for the operation error. If a -callback returns neither a report nor an error, report an operational provider -contract failure rather than dereferencing nil or fabricating native evidence. - -### Smoke invocation and delivery - -Use the roadmap's `SmokeRequest` envelope, optional `Suite.SmokeTest` callback, -and `WithSmokeTesting(suite, load, build)` helper. The callback represents one -concrete operation, for both single and aggregate invocations. - -The helper's builder runs before its engine loader, as in request validation. -A nil builder forwards the seed; a supplied builder's complete native request -is authoritative. Return the native report and error together. Retain acquired -diagnostics. A nil loader attaches an installation issue even if the capability -is subsequently removed, following the existing suite convention. - -For `all`, the handler prepares input/checks/state once, then invokes the -captured concrete-test callback sequentially. The per-test seed varies `Kind`; -the optional builder sees each concrete seed. Keep the native request returned -by the builder authoritative, including its application-owned choices. -Engine reuse can occur through application shared state; do not add an -integration cache or make loader-call counts a public requirement. - -Collect reports and errors, then encode and deliver one report. Continue after -individual operation errors unless canceled. On cancellation, stop scheduling, -retain acquired results, and return operational cancellation with supported -identity. Never add synthetic results for operations that did not run. - -Join multiple operation causes using `errors.Join`, preserving `errors.Is` and -`errors.As` access. Keep diagnostic records in returned order and let the -existing invocation coordinator deliver them. Do not add duplicate error -diagnostics on top of the coordinator's existing presentation. - -Smoke report delivery must attempt available output even when the context has -become canceled; the success-only `writePromptReport` cannot be used unchanged. -Retain operation causes if encoding or delivery also fails. Report writer -errors/short writes through `DeliveryError` and `ReportIncomplete` under the -existing outcome contract. Do not change unrelated inspection delivery or add -rollback/blocking-writer interruption promises. - -A nil native error and successful delivery yield success for a single operation. -For `all`, every selected operation must return without error and delivery must -succeed. Cancellation and operational/delivery errors yield the established -operational classification. Native false, unknown, unsupported, or incomplete -observations do not independently change the outcome. - -## Stage 1: Adopt ReleaseKit v0.4.2 - -**Primary files:** root `go.mod`/`go.sum`, root attribution, -`docs/policy/release.md`, `docs/release.md`; existing launcher/tests for review. - -1. Update the root tool requirement to released ReleaseKit v0.4.2 and obtain real - checksums. Preserve the Go minimum and release toolchain. -2. Update the root selected-component attribution and pinned notice/source links - for the adopted revision. Compare applicable upstream notices; preserve - unchanged license texts. -3. Update factual tool-version and commit-pinned reference links in the release - policy and runbook. Leave their publication contracts unchanged. -4. Review `scripts/release` and its three profiles against the adopted public - contracts. Keep the existing launcher, bootstrap storage, shared files, and - source-only publication behavior unless a concrete compatibility defect - requires correction. Do not add Woodpecker workflows or hosted deliverables. - -**Validation:** Run existing launcher argument/delegation tests and the root -default suite as appropriate. Assert CommandKit's invocation contract, not -ReleaseKit's publication algorithms. Do not invoke publication. - -**Complete when:** the root selects v0.4.2 and its factual notices/references -agree, without new release machinery. - -**Completed:** 2026-10-17. The root selects ReleaseKit v0.4.2 with Go-generated -module checksums; attribution, release policy, and pinned runbook references -match upstream commit `7d033c5ca1c95aec7db9690293638a0a1ac887b8`. Upstream -`COPYRIGHT` and `LICENSE` were unchanged. Launcher argument/delegation tests and -`GOWORK=off GOTOOLCHAIN=local go test ./...` passed. Profiles and the existing -source-only launcher contract required no changes. - -## Stage 2: Adopt ConfigKit v0.8.0 in both integrations - -**Primary files:** integration manifests/sums, ConfigKit adapters and affected -literals, both integrations' attribution. - -1. Set both direct ConfigKit requirements to v0.8.0. Prepare actual sums under - each module's own graph; preserve released PromptKit v0.15.0 until Stage 4. -2. Review ConfigKit-facing code for unkeyed `SourceInfo` literals, equality, - or map-key use affected by its new noncomparable `Path` field. Adapt actual - uses without inventing an equality abstraction or changing source identity. -3. Preserve native `Result`/inspection records through existing providers. - Destination, placement, version checks, and sequence composition remain - available through application loader options, not new generic CLI policies. -4. Update ConfigKit attribution for both modules with the reviewed v0.8.0 - revision and applicable notices. - -**Validation:** Run ckconfig's existing suite and build affected ckprompt code -where its still-old PromptKit API permits. Review selected dependency versions. -Do not add native placement/composition/version-check feature matrices. - -**Complete when:** both integration manifests name v0.8.0 and the existing -ConfigKit boundary remains coherent. - -**Completed:** 2026-10-17. Both integration modules select ConfigKit v0.8.0 -with generated checksums, and ckprompt continues to select PromptKit v0.15.0. -The source record usage review found no unkeyed literals, equality, or map-key -uses requiring changes; the existing adapters preserve native records. The -ckconfig suite passed and ckprompt packages and tests compiled with no tests run. -Upstream `COPYRIGHT` and `LICENSE` were unchanged. - -## Stage 3: Extend configuration source reports - -**Primary files:** `integrations/ckconfig/sources.go` and focused source-report -tests, reusing the existing native source-reader boundary. - -1. Extend `sourceDTO` and text rendering with the fields and nested shape - specified above. Convert native paths with `Path.String()`, and map the two - native placement modes to their corresponding names. -2. Project `SourceVersionCheck` only from returned native records, preserving - native optional locations and sanitized version values. Do not reread files, - rerun checkers, or derive acceptance/validation judgments from the evidence. -3. Keep existing source fields, inventory order, scope behavior, source-reader - capability handling, and delivery paths. Keep `config explain` and compact - diagnostic legends focused on their existing responsibilities. -4. Add focused report tests using controlled returned `SourceInfo` values: - root/non-root destination, both placement modes, unresolved false/true, - absent/present version evidence, nil/non-nil location, and meaningful - present/null/text/redacted values. These test projection, not native - sanitization or source resolution. - -**Validation:** Decode YAML semantically and inspect text fields without key-order -or whole-output snapshots. Run focused source tests and the ckconfig suite. - -**Complete when:** both source formats expose the agreed native evidence with no -parallel ConfigKit behavior. - -**Completed:** 2026-10-17. YAML and text `config sources` now project native -destination, placement, unresolved-destination, and source-version evidence. -Focused semantic-YAML and text-field assertions cover the requested presence, -null, redaction, location, and placement cases. Focused source tests and the -complete ckconfig suite passed. - -## Stage 4: Migrate ckprompt production code to PromptKit v0.16.0 - -**Primary files:** ckprompt manifest/sums, `adapters.go`, `command.go`, -`validation.go`/`validation_command.go`, affected report DTOs, attribution. - -1. Select released PromptKit v0.16.0, retaining direct ConfigKit v0.8.0. - Update the PromptKit attribution to its reviewed release revision. -2. Remove `PromptRequest.Version` and the prompt-only `--version` bindings from - named prompt inspection, requirements show, and requirements validate. - Remove corresponding fields/assignments in native request seeds. -3. Call `InspectPrompt(ctx, request.ID)` and forward the revised native - requirements/run request types. Retain configuration selection, context, - selectors, builders, diagnostics, and native error identity. -4. Update named/requirements/catalog report DTOs to optional `version` fields - and adapt text presentation, using the report choices above. Do not use - label values for selection or convert nil to empty strings. -5. Review remaining production uses of removed catalogs, implicit profiles, - or old upstream fields and migrate those actual uses. - -**Validation:** Build the production ckprompt package against the selected -released dependencies. Test fixtures and consumer code are migrated in Stage 5; -do not create compatibility shims solely to make this intermediate stage green. - -**Complete when:** production adaptation uses the new ID-only API and optional -version representation. - -**Completed:** 2026-10-17. ckprompt selects PromptKit v0.16.0 with generated -checksums and attribution pinned to release revision -`9178f3eb91c2b2138caa78e22be93c048b4bc992`; upstream `COPYRIGHT` and `LICENSE` -are unchanged. Production adapters use ID-only prompt inspection, the three -prompt-version selectors and their seed assignments are removed, and inspection -and catalog reports retain optional version values. `GOTOOLCHAIN=local go build -./...` passed for the ckprompt module. Fixtures and consumer code remain for -Stage 5. - -## Stage 5: Migrate existing fixtures, tests, and composed loading - -**Primary files:** ckprompt tests, `examples/composed/main.go` and fixtures, -their tests. - -1. Convert native version-keyed prompt mappings to one direct definition per ID. - Retain useful versions as optional metadata. For tests that genuinely need - multiple resources, use distinct IDs rather than rebuilding version lookup. -2. Make needed backend/profile definitions explicit. Remove expectations about - installed catalogs and former standard/auto profiles. Use small synthetic - inputs; do not adopt external catalogs just to restore old fixtures. -3. Update the composed engine mapping to the direct prompt-definition format. - Keep its existing campaign selector, request builder, config scopes, state - cache, and session command. Its application `prompt_version` data may remain - descriptive input, used to populate native `version`, without selecting a - definition. Do not require a campaign-loader redesign or collections demo. -4. Update tests for ID-only requests and removed flags. Add focused projection - evidence for absent, present-empty, and nonempty `version` across the - affected reports; verify absent keys rather than YAML null. -5. Replace obsolete upstream-domain assertions encountered in these affected - tests with CommandKit boundary assertions. Preserve meaningful adapter - protection; do not expand this into a repository-wide test rewrite. - -**Validation:** Run ckprompt's existing suite, including the composed consumer. -Exercise a named inspection and requirements show/validate through the updated -fixture. Native errors may be fixtures; assertions protect forwarding, report -presentation, and error identity, not upstream detection algorithms. - -**Complete when:** the previous delivered feature set works through v0.16.0 -without compatibility scaffolding or implicit catalogs. - -**Completed:** 2026-10-17. PromptKit fixtures now use direct prompt definitions, -explicit profiles/backends, and distinct IDs for multiple prompts. ckprompt -tests use the ID-only inspection and validation requests and verify that the -removed version flags are unavailable. Semantic YAML tests cover absent, -present-empty, and nonempty versions in prompt inspection, requirements, and -catalog reports; PromptKit-specific requirement classification assertions were -removed from the affected adapter/consumer tests. The ckprompt suite passed, -and composed named inspection, requirements show, and request validation all -ran successfully against the migrated fixture. - -## Stage 6: Extend catalog contracts and engine adapters - -**Primary files:** ckprompt public request declarations, `Suite`, `FromEngine`, -and focused adapter tests; `catalog.go` is a suitable private organization. - -1. Add `CatalogRequest` and `NamespaceRequest` as specified above, revise - existing prompt/profile callback signatures, and add backend, namespace, - and collection callbacks. -2. Extend `FromEngine` to forward the correct request fields to each native - API. Continue using `loadPromptKitEngine` for selection and diagnostics. - Forward native filters/kinds and returned results/errors directly. -3. Preserve native engine views from loaders. Do not reconstruct views, - qualify IDs, filter lists, calculate namespace unions, or sort memberships. -4. Migrate existing custom catalog callbacks and call sites to the new request - signatures. Add GoDoc for request fields, native defaults, and independent - optionality. Keep nil-loader issue retention. -5. Use focused adapter evidence: direct native-call parity on a small fixture - can establish forwarding. Test selector/state/context propagation and loader - diagnostics/error identity through existing seams without creating a - parallel namespace semantics matrix. - -**Validation:** Build ckprompt and run focused adapter tests. Current command -call sites may need their mechanical signature adaptation here, with new -operator syntax delivered in Stage 7. - -**Complete when:** all five catalogs are available through narrow native adapters -and public callback shapes are coherent. - -**Completed:** 2026-10-17. Added `CatalogRequest` and `NamespaceRequest`, -updated the prompt/profile callbacks, and exposed backend, namespace, and -collection callbacks. `FromEngine` forwards selections and native filters or -kinds to all five PromptKit catalog methods, preserving returned values, -diagnostics, and errors. Existing catalog callers now send the default request. -Focused adapter tests compare each callback with its native method and cover -invocation values, loader diagnostics/error identity, and cancellation. The -focused adapter/catalog tests and `GOTOOLCHAIN=local go build ./...` passed. - -## Stage 7: Add catalog command bindings and YAML output - -**Primary files:** ckprompt catalog command construction, input preparation, -encoders, and basic assembled command tests. - -1. Install each supplied catalog independently under `promptkit list`. Extend - empty-suite detection and group assembly to include the new callbacks. - Keep eager metadata/callback capture, root-family ownership, and retained - assembly issues. -2. For prompts/profiles/backends, add singleton `--namespace` and boolean - `--all-namespaces`. Map omitted selection to `NamespaceDefault`, a supplied - nonempty name to `NamespaceNamed`, an explicit empty name to - `NamespaceUnqualified`, and true `--all-namespaces` to `NamespaceAll`. -3. An active all-namespaces choice plus supplied namespace is a usage error - before provider work. Explicit `--all-namespaces=false` does not select - an override; combine it with namespace or retain native default normally. - Preserve singleton last-occurrence and namespace presence behavior. -4. For namespaces, add repeatable `--kind`, preserving supplied native - `ResourceKind` values and forwarding the slice. Omission forwards the native - union/default. For collections, add no namespace/kind membership filter. -5. Retain common selection/selectors and YAML-only format preflight. Use normal - string-to-native-type conversion; leave native name/kind validity to - upstream APIs rather than duplicating domain validation. -6. Add collection projection and reuse string-list rendering for profiles, - backends, and namespaces. Follow the report shapes above, without local - membership reinterpretation. - -**Validation:** Add basic capturing-callback command tests for each new operation, -filter conversion, and collection optional versions; run focused catalog tests. - -**Complete when:** the accepted catalog CLI and reports invoke the intended -callbacks through the standard binding system. - -**Completed:** 2026-10-17. `promptkit list` now installs each supplied catalog -independently. Prompts, profiles, and backends bind namespace and all-namespace -filters; namespaces bind repeatable native resource kinds; collections use -their unfiltered operation and project optional release versions. Added -capturing-callback coverage for filter mapping, false and repeated values, -collection YAML presence, and conflicting filter preflight. The full ckprompt -suite and `GOTOOLCHAIN=local go build ./...` passed. - -## Stage 8: Verify discovery availability, lifetimes, and outcomes - -**Primary files:** focused ckprompt catalog command/adapter tests. - -1. Cover absent and independently supplied catalogs, help without acquisition, - retained wiring issues, and eager capture for new callbacks. Reuse existing - registry coverage rather than reproducing its full collision matrix. -2. Capture callback requests for native default/named/unqualified/all choices, - explicit false, conflicting active choices, singleton repetition, and - repeatable kinds. Assert original selection, selectors, context, and state. -3. Cover missing required selectors, extra operands, unsupported format, - unavailable flags, and input-check failures. Assert no provider work where - required, without insisting on factories running after an earlier failure. -4. Verify fresh parsed values across repeated invocations, nil-factory zero - values, native inventory order/strings, empty sequences, optional versions, - and representative error/cancellation/diagnostic/write outcomes. - -**Validation:** Run focused discovery tests and the ckprompt suite. Use synthetic -returned inventories for report/membership assertions; these tests do not -establish native namespace algorithms. - -**Complete when:** discovery exposes only declared capabilities and preserves -the selected CommandKit invocation contracts. - -**Completed:** 2026-10-17. Added independent installation and eager-capture -coverage for backend, namespace, and collection callbacks, including help -without provider acquisition and unavailable filters. Extended nil-loader issue -retention checks to all catalogs. Existing prompt/profile coverage plus the new -cases exercise default, named, unqualified, all, explicit-false, conflicting, -singleton, and repeatable-kind requests; selection, selector, state, inventory, -empty output, and delivery behavior remain protected. The full ckprompt suite -passed. - -## Stage 9: Add the optional smoke capability and native adapter - -**Primary files:** ckprompt `Suite`, new `smoke.go` or equivalent, adapter tests. - -1. Add `SmokeRequest` and `Suite.SmokeTest` with the roadmap's signatures. - Ensure the acquisition value can retain a report with an error. -2. Add `WithSmokeTesting` preserving the rest of the suite and retained issues. - Capture loader/builder references consistently with existing helpers. -3. Apply the builder-before-loader/nil-builder rules above. Forward the complete - resulting request to `Engine.SmokeTest`, preserving native report and error. - Check cancellation before further acquisition, but do not discard an - error-bearing report through a success-only post-call path. -4. Retain loader diagnostics and nil-engine/nil-loader handling from existing - conventions. Do not call requirements validation, resolve native targets, - implement retries, or derive success from report findings. -5. Add focused helper tests for nil builder, authoritative builder output, - typed input/state/context, builder failure before loading, retained wiring - issues, and report/error propagation. Use a small native engine with an - injected public client where the actual adapter needs exercising. - Observe the forwarded request/effective native call; do not test the private - diagnostic case or provider acceptance behavior. - -**Validation:** Run focused smoke adapter tests and existing adapter suites. -No live provider or real credentials are needed. - -**Complete when:** one concrete smoke request reaches the native API through an -independently optional, composable adapter. - -**Completed:** 2026-10-17. Added `SmokeRequest`, the independently optional -`Suite.SmokeTest` callback, and `WithSmokeTesting`. The helper preserves other -suite capabilities and issues, invokes an optional builder before loading, -forwards its complete request unchanged, retains loader diagnostics, checks -cancellation before acquisition, and returns the native report and error -without filtering either. Focused tests cover nil-builder forwarding, -authoritative builder output, context/input/state, builder failure before load, -nil-loader issue retention, cancellation, loader diagnostics, and native error -identity. PromptKit's injected-client fixture returned a report with nil error -when the client returned an error, so public fixture evidence for a -report-plus-error pair is unavailable; the adapter returns both native values -directly. Focused adapter tests and `GOTOOLCHAIN=local go build ./...` passed. - -## Stage 10: Implement faithful smoke YAML projection - -**Primary files:** private `smoke_report.go` and focused projection tests. - -1. Define small private DTOs for the report fields described above. Map every - exported v0.16.0 `SmokeTestReport` field and its nested public data explicitly; - avoid reflect-based conversion, a JSON round trip, and raw request dumping. -2. Include target IDs, optional target version, selection source, endpoint, - effective settings and settings-presence fields; native facts; timing; - attempts; presence-aware token usage; reason/failure values. -3. Preserve nullable booleans and durations, explicit false/zero, empty optional - version strings, unknown token metadata, and `extra_params` values. Use - existing YAML encoding and native types where their representation is - suitable; private DTOs provide naming/formatting only. -4. Add the aggregate envelope with ordered entries and optional error text. - Keep native report data unchanged if its kind differs from the selected - outer kind due to application request construction. -5. Add synthetic-report tests protecting omission, presence, nested values, - durations, empty attempts, all field categories, and report-plus-error - entries. Decode YAML semantically; do not freeze key order or timing values - produced by a live operation. - -**Validation:** Run focused projection tests. Verify no native request/API key is -serialized; do not introduce filtering of returned `ExtraParams`. - -**Complete when:** native smoke evidence and aggregate entries have a complete, -neutral YAML representation. - -**Completed:** 2026-10-17. Added private report and aggregate DTOs that explicitly -project every exported PromptKit smoke report field, including target settings, -presence, nullable facts, timing, attempts, usage, and failure details. Durations -render as Go duration strings, empty attempts as `[]`, and optional prompt -versions preserve empty values while omitting absent ones. Aggregate entries -retain order, nullable reports, native report kinds, and optional error text. -Semantic YAML tests cover all field categories, nested values, omission, -presence, direct-request/key exclusion, and report-plus-error entries. Focused -projection tests and `GOTOOLCHAIN=local go build ./...` passed. - -## Stage 11: Bind and execute single smoke tests - -**Primary files:** private `smoke_command.go`, `Command` assembly, basic command -tests. - -1. Add one runnable `test` command with one required `KIND` operand. Install it - only when `Suite.SmokeTest` is supplied. Include it in empty-suite detection, - captured suite metadata, and the promptkit family help. Do not add an - application-owned command beneath the family. -2. Bind every scalar/request field in the roadmap table using the existing - typed binding APIs. Use standard integer/floating-point conversion and - presence-aware setters; do not add a second parser or range validator. -3. Construct `Execution` only when at least one execution flag is supplied. - Omitted pointer fields stay nil; supplied zero or empty values remain - present. Bind `TransientRetries` independently. Use native enum strings for - individual kinds. Recognize `all` for Stage 12 expansion before constructing - a native request; never pass that token as a native kind. -4. Apply common selection, captured selectors, YAML-only preflight, input checks, - cancellation checks, and state initialization through existing patterns. - For a single operation, invoke the captured callback once. -5. Use the Stage 10 projection and a focused delivery helper to deliver any - acquired report, even when accompanied by an error/cancellation. Return - diagnostics and outcome fields under the delivery rules above. -6. A nil report with a non-nil error remains an ordinary operation error; emit - no fabricated single native report. Neither nil result with nil error nor - encoding/delivery failure may become success. - -**Validation:** Add basic assembled single-command tests with capturing callbacks, -including successful native findings, report-plus-error, and a short writer. -Run focused smoke command tests. - -**Complete when:** all single-kind bindings invoke the optional provider and -faithfully report its result and errors. - -**Completed:** 2026-10-17. Added the optional `promptkit test KIND` command with -all accepted scalar bindings, YAML preflight, prepared input/state, and native -report delivery. Explicit zero/empty overrides remain present, omitted -execution/retry overrides remain absent, and `all` is rejected before native -request construction pending aggregate coordination. Assembled tests cover -flag forwarding, findings, report-plus-error, nil-result contract failure, and -short-write delivery. `GOTOOLCHAIN=local go test ./... -run -'TestSmokeTestCommand' -count=1` passed in ckprompt. - -## Stage 12: Coordinate aggregate smoke execution - -**Primary files:** smoke command runner, aggregate projection/outcome helpers, -focused aggregate tests. - -1. Expand `all` to reachability, authentication, text inference, and structured - output in the accepted order. This is one invocation of the standard command, - not four dispatcher executions. -2. Prepare/check typed input once and create shared state once when reached. - Build a concrete seed for each selected kind, invoking `Suite.SmokeTest` and - therefore the optional builder separately. Keep `all` out of native requests. -3. Preserve the common CLI seed across operations. Construct each per-test seed - from the bound scalar values with its own execution override struct and - fresh pointers for supplied optional controls, including transient retries. - This keeps builder edits to one seed from changing later parsed defaults. - Do not deep-copy application input/state or arbitrary builder objects. -4. Append each acquired report, error, and diagnostics in operation order. - Continue after an individual error. If the caller context is canceled, - stop scheduling further operations, retain results, and preserve cancellation - identity alongside already returned causes. -5. Join errors with the standard library and encode one aggregate document. - Retain each associated error in its entry; preserve supported error identity - in the invocation cause. Do not promote false/unknown report facts to errors - or add an overall readiness score. -6. Deliver retained results despite cancellation, and preserve operation causes - separately from report/diagnostic delivery failures. With no operation - reached because preflight or initial cancellation failed, use the existing - early-failure handling; do not invent native observations. -7. Do not introduce loader memoization, parallel execution, custom retry loops, - synthetic skipped-test statuses, or a second public aggregate callback. - -**Validation:** Use controlled callback results to verify call order, distinct -concrete seeds, shared input/state, continuation, partial errors, cancellation -stopping, ordered output, and joined error identity. Run focused aggregate tests. - -**Complete when:** `promptkit test all` provides the agreed coordination through -the same concrete-operation callback as individual tests. - -**Completed:** 2026-10-17. `all` now runs reachability, authentication, text -inference, and structured output sequentially in one invocation, with one input -check and shared state. Each callback receives a concrete kind and a fresh -execution override with copied optional pointers. Reports and per-kind errors -remain ordered; ordinary errors do not stop later tests, while cancellation -stops scheduling and retains earlier results plus joined error identity. -Aggregate tests verify shared lifetimes, seed isolation, continuation, -projection order, and partial cancellation. Focused smoke/report tests and -`GOTOOLCHAIN=local go build ./...` passed in ckprompt. - -## Stage 13: Verify smoke bindings and invocation lifetimes - -**Primary files:** focused smoke command/adapter tests, reusing existing writer -and diagnostic doubles. - -1. Cover native seed fields from all bound flag categories, pointer omission - versus zero/empty, config/overlay presence, singleton last occurrence, - selectors, and per-invocation reset. Verify targets may be combined and - native semantic errors retain operational classification. -2. Cover syntax/conversion/operand/format failures and input-check handling. - Missing operands, malformed numeric syntax, unsupported format, and unknown - flags remain usage failures; factories need not run after failed preflight. -3. Cover a smoke-only suite, absent smoke with other capabilities, installation - issues after narrowing, eager callback/selector capture, and help/version - leaving factories/builders/loaders/providers uninvoked. -4. Verify one input preparation/state lifetime for `all`, nil-factory zero values, - and builders seeing concrete kinds. Check earlier reports remain available - when later operations fail or cancellation stops the sequence. - -**Validation:** Run focused smoke binding/lifecycle tests and the ckprompt suite. -Assert callback requests and invocation state, not native target resolution. - -**Complete when:** the new command obeys selected grammar, presence, capture, -availability, and lifetime contracts. - -**Completed:** 2026-10-17. Added smoke lifecycle coverage for repeated scalar -values, explicit zero and empty overrides, config/overlay presence, target -combinations, and reset across invocations. Usage and input-check failures -stop before state/provider work. Availability, captured callbacks/selectors, -retained installation issues, and help/version inactivity are verified, along -with nil-factory zero values and concrete kinds reaching builders. Focused smoke -tests and the full ckprompt suite passed with `GOTOOLCHAIN=local`. - -## Stage 14: Verify smoke outcomes and report delivery - -**Primary files:** focused smoke outcome tests and existing writer/diagnostic -fixtures. - -1. Cover report-only success, false/unknown native observations with nil errors, - report-plus-error, error-only, and nil-report contract failure. Confirm - supported native and cancellation error identity survives aggregation. -2. Cover cancellation before acquisition, after partial results, and between - tests. Verify available reports are delivered and no later callback starts - after cancellation is observed. Do not assert interruption of an already - running caller callback or blocking writer. -3. Cover encoding failures, report writer errors/short writes, diagnostic writer - failures, and operation plus delivery errors. Retain `Cause`, `DeliveryError`, - `ReportIncomplete`, and diagnostic delivery state as applicable. Reuse shared - coordinator coverage for unchanged mechanics. -4. Verify aggregate error entries correspond to the selected kinds and that - multiple causes remain discoverable through `errors.Is`/`errors.As`. - Diagnostic records keep their order and existing presentation responsibility. - -**Validation:** Run focused smoke outcome tests and the full ckprompt suite. -Each assertion must detect CommandKit presentation/coordination/delivery defects; -provider smoke-test correctness belongs upstream. - -**Complete when:** realistic smoke failures preserve their reports, native error -identity, and delivery outcomes without a parallel diagnostic test suite. - -**Completed:** 2026-10-17. Added outcome and delivery coverage for false/unknown -observations, error-only and report-plus-error results, cancellation before and -after acquisition, and retained aggregate results across cancellation. Tests -also cover error and short report writes, encoding failure, diagnostic writer -failure, operation plus delivery errors, ordered diagnostics, and joined -`errors.Is`/`errors.As` identity. The full ckprompt suite passed with -`GOTOOLCHAIN=local go test -count=1 ./...`. - -## Stage 15: Demonstrate the new interfaces in the maintained consumer - -**Primary files:** `integrations/ckprompt/examples/composed`, its README, -synthetic configuration, and tests. - -1. Extend the already migrated small consumer with the updated catalog callback - API and explicit `WithSmokeTesting` wiring. Keep configuration scopes, - existing requirements operations, campaign selection/cache, and session - restore intact. -2. Demonstrate backend/namespace/collection inventory and native namespace - selection with the smallest useful configuration. A small synthetic - collection may be used where useful; do not turn this into a collections - composition tutorial or catalog migration project. -3. Reuse shared state for engine/application acquisition where natural in this - example. This is consumer behavior, not a new library caching requirement. -4. Show nil-builder smoke wiring and explain optional typed request construction - with a minimal snippet or focused test. A builder starts from the native - seed and returns the chosen native request; no merge policy is added. -5. Keep executed consumer checks offline. Use a small injected public - `promptkit.LLMClient` double for inference when exercising the native helper; - HTTP diagnostics can retain native unsupported/not-applicable observations - when no probe is configured. Do not recreate native private expected - responses or assert provider diagnostic correctness to make a demo green. -6. Update documented commands for ID-only inspection, optional versions, discovery, - and `test KIND`/`test all`. State any example-specific fixture/client setup - plainly. Check metadata/requirements commands leave the injected client - unused; explicit smoke commands use their selected native path. - -**Validation:** Run composed-consumer tests and representative documented offline -commands. Assert wiring and adaptation; an intentionally returned native -diagnostic error is a documented example outcome, not a reason to redefine the -native API's success criteria. - -**Complete when:** the consumer demonstrates the new interfaces with little -application plumbing and no model-service prerequisite. - -**Completed:** 2026-10-17. The composed consumer now explicitly wires smoke -testing with a nil request builder and exposes backend, namespace, collection, -and namespace-filtered prompt discovery using one synthetic collection. Its -shared invocation state carries an injected in-memory PromptKit client; tests -confirm metadata and requirements leave it unused and smoke inference invokes -it. The README documents ID-only inspection, optional version metadata, all -catalogs, namespace filtering, typed builder customization, and single and -aggregate smoke commands. The composed test suite passed, as did the documented -offline discovery, inspection, requirements, and smoke commands. Native HTTP -checks reported their configured unsupported/not-applicable observations. - -## Stage 16: Reconcile dependency graphs and notices - -**Primary files:** all three module manifests/sums and attribution documents. - -1. Tidy/prepare dependencies with each module's own graph, using disposable - module files and version-specific replacements only for unpublished - CommandKit prerequisites where needed. Keep actual released upstream - dependencies selected; do not use sibling upstream replacements. Transfer - resulting dependency/sum edits back to the owning manifests without copying - temporary replacements or changing the declared CommandKit prerequisites. -2. Remove obsolete bundled backend-catalog requirements/sums and their notices - when absent from the actual selected ckprompt graph. Retain other indirect - requirements only as generated by the module tools. -3. Update notices for every affected selected graph, including ReleaseKit - selection through the local core. Do not blindly copy the root tool version - into each integration's notices; verify its effective graph. -4. Compare applicable copyright/license/notice material and pin upstream source - links to the selected release revisions. Reuse unchanged standard license - texts and preserve each module's existing local grant/material boundaries. -5. Keep checked-in CommandKit prerequisite versions unchanged until actual - release preparation selects and publishes successors. Do not fabricate - future versions/checksums or use temporary replacements as release metadata. - -**Validation:** Inspect independent `go list -m all` output and changed -manifests/sums/notices. Confirm ConfigKit v0.8.0 in both integrations, PromptKit -v0.16.0 in ckprompt, and no optional integration dependency in core. - -**Complete when:** selected graphs, real checksums, removed catalog dependencies, -and owning attribution entries agree. - -**Completed:** 2026-10-17. Each module's own graph was tidied in disposable -module files with only the unpublished CommandKit prerequisites replaced by -local paths; checked-in prerequisite versions remain unchanged. Removed the two -unused PromptKit backend-catalog requirements and sums, stale dependency sums, -and their ckprompt attribution entries. All three notices now identify -ReleaseKit v0.4.2 at its selected revision. Independent module lists confirm -ConfigKit v0.8.0 in both integrations and PromptKit v0.16.0 in ckprompt; root -contains no optional integration packages, and ckconfig contains no PromptKit -packages. `go mod verify` passed for all three isolated graphs. - -## Stage 17: Update canonical documentation and planning references - -**Primary files:** GoDoc, module READMEs, `docs/cli.md`, -`docs/consumers/guide.md`, `docs/development.md`, `docs/internal/overview.md`, -and affected roadmap references. - -1. Document the new callback requests, helper signatures, optional capabilities, - native defaults, builder authority, error-bearing reports, and aggregate - lifetimes in their owning GoDoc. Replace stale five/six-capability statements - with the actual eight `FromEngine` callbacks and two explicit additions. -2. Update the CLI reference for source evidence, ID-only selection, optional - `version` omission, all five catalogs and filters, and the complete smoke - vocabulary/bindings/report/outcome behavior. Include concrete aggregate - envelope and native projection examples from the implemented renderer. -3. Update consumer guidance, module summaries, internal inventory, and developer - example commands. Scope existing metadata-only/no-provider-call descriptions - to those operations; smoke is explicitly invoked native behavior. -4. Link to the adopted upstream contracts for native configuration, selection, - and diagnostic meanings. Use neutral field descriptions and avoid warnings, - extra promises, or independent restatements of native algorithms. -5. Reconcile the selected native smoke capability with `deferred.md`, preserving - unrelated future proposals. Record its implemented boundary rather than - treating all credential/runtime diagnostic ideas as delivered. -6. Repair references from completed `promptkit-validation.md` and `remediation.md` - to the previous implementation plan using its pinned historical link above. - Those records remain historical; do not update their old API examples to - describe the new feature. -7. Keep accepted ADRs and policy boundaries intact unless a concrete conflict - emerges. Do not create ADRs for report-key or private helper choices. Keep - historical release notes unchanged; document release impact under the - existing pre-v1 policy, with exact release versions chosen during preparation. - -**Validation:** Check Go examples and CLI commands against actual declarations, -local links/fragments, Markdown whitespace, stale selector/callback-count -claims, and accidental consumer mandates. No new documentation checker code -is required. - -**Complete when:** current contracts describe implemented behavior under their -canonical owners and historical references remain correctly scoped. - -**Completed:** 2026-10-17. GoDoc, module summaries, the README, CLI reference, -consumer guide, development commands, and internal inventory now describe the -adopted source evidence, eight `FromEngine` callbacks, five catalogs, optional -request validation, and native smoke behavior. Smoke examples and catalog -commands were run against the maintained composed consumer; `go doc` confirmed -the exported helper and request documentation. Local Markdown links/fragments, -stale upstream revision references, and whitespace checks passed. Deferred-work -scope and historical implementation-plan links were reconciled without changing -historical API examples or release notes. - -## Stage 18: Verify the complete target and finish - -1. Format all changed Go files and inspect the full diff for unrelated changes, - local replacements, generated caches, fabricated sums, or accidental public - exports. Keep ignored `go.work`/`go.work.sum` and `.releasekit/` local. -2. Run all affected module suites in the local workspace: - - ```sh - GOTOOLCHAIN=local go test -count=1 ./... - (cd integrations/ckconfig && GOTOOLCHAIN=local go test -count=1 ./...) - (cd integrations/ckprompt && GOTOOLCHAIN=local go test -count=1 ./...) - ``` - - Root `./...` does not traverse nested modules. Reuse unchanged successful - evidence where appropriate; none of these commands invokes publication. -3. Verify independent readonly graphs with `GOWORK=off` and - `GOTOOLCHAIN=local`. For unpublished CommandKit prerequisites, create - temporary absolute-path `.mod`/`.sum` pairs copied from the owning module. - Replace only the required core version with the local root for ckconfig, - and only the required core/ckconfig versions for ckprompt. -4. Prepare needed sums in those temporary files, then run - `go test -mod=readonly -count=1 ./...` from each actual module directory with - the applicable `-modfile=/absolute/path/module.mod`. Root uses its actual - manifest without optional-library replacements. Hash temporary module/sum - files before/after readonly checks and confirm they stay unchanged. - ConfigKit, PromptKit, and ReleaseKit remain real released modules. -5. Inspect `go list -m all` and `go list -deps -test ./...` using those same - isolated graphs. Prove core has no optional integration libraries, - ckconfig has no PromptKit/ckprompt dependency, and ckprompt selects the - intended adopted releases. A workspace pass alone is insufficient. -6. Review the target checklist below against actual source and test evidence. - Run prose/link checks and review maintained examples alongside their actual - command declarations. Do not introduce mandatory live probes, race/coverage - thresholds, benchmark gates, or platform matrices. -7. Mark stages complete only with actual evidence, update the roadmap's status, - and identify any genuine remaining limitation accurately. Retain planning - records until cleanup is requested. Commit the completed sprint's remaining - nonempty work, inspect status, and report the commits and validation. - Do not tag, push, or publish a release. - -**Complete when:** every target below is implemented and evidenced, documentation -and dependency metadata agree, and the workspace is clean. - -**Completed:** 2026-10-17. All three module suites passed in the workspace and -again with isolated readonly module graphs. The temporary ckconfig and ckprompt -module files and sums were unchanged by readonly checks. Module listings select -ReleaseKit v0.4.2, ConfigKit v0.8.0, and PromptKit v0.16.0; dependency listings -show no optional integration imports in core and no PromptKit imports in -ckconfig. All 40 maintained shell examples matched their expected outcomes, -including the documented validation failure. Go files were formatted, the full -change set contained only roadmap-scoped files and intended APIs, local links -and Markdown whitespace passed, and the worktree was clean after commit. No -release was tagged, pushed, or published. - -## Target verification checklist - -| Target | Primary stages | Completion evidence | -| --- | --- | --- | -| ReleaseKit v0.4.2, unchanged publication model and Go selections | 1, 16, 18 | Selected root tool graph, launcher boundary tests, factual docs/notices | -| ConfigKit v0.8.0 in both integrations | 2, 16, 18 | Independent module graphs and released checksums | -| New source evidence in text/YAML | 3 | Controlled-record projection tests; unchanged existing source fields | -| ID-only PromptKit v0.16.0 API and explicit configuration | 4–5 | Removed selectors, migrated native fixtures and consumer, boundary tests | -| Optional `version` omission/empty preservation | 4–5, 7, 10 | Semantic report assertions across affected reports | -| Independent backend/namespace/collection catalogs and native filters | 6–8 | Public callbacks, command capture, native forwarding, availability and YAML | -| Optional concrete smoke callback/helper and full agreed bindings | 9, 11, 13 | Seed/builder/native forwarding, pointer presence, lifecycle evidence | -| Faithful smoke reports, partial results, supported error identity | 9–14 | Synthetic projections and assembled outcomes/delivery tests | -| Ordered `test all`, continuation, cancellation, per-kind builders | 12–14 | Controlled aggregate call sequence, reports, and joined causes | -| Small maintained consumer and current canonical documentation | 15, 17 | Offline consumer evidence, reviewed commands and local links | -| No reverse optional dependencies, obsolete catalog graph, or local upstream replacements | 16, 18 | Independent manifests/module/test-import listings and notices | -| Upstream testing ownership respected | All testing stages | Assertions address CommandKit interfaces and coordination, not native domain correctness | - -No separate product or API decision was needed to complete the stages. Concrete -presentation and wiring choices stayed within the accepted roadmap; incidental -implementation details were not elevated into additional guarantees. diff --git a/docs/roadmap/promptkit-validation.md b/docs/roadmap/promptkit-validation.md deleted file mode 100644 index 4f1ab81..0000000 --- a/docs/roadmap/promptkit-validation.md +++ /dev/null @@ -1,370 +0,0 @@ -# PromptKit concrete request validation - -Status: accepted adoption and feature roadmap; implementation complete. -Command placement, adapter composition, request construction, artifact syntax, -and initial presentation are settled. This document defines the target and -required work; the [implementation plan](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/38b49f29fb3a8f22ff02528d2a0ed413e5c6ab1e/docs/roadmap/implementation.md) -sequences the historical implementation work. - -## Purpose - -Adopt PromptKit v0.15.0 and expose its concrete request validation capability as -`app promptkit requirements validate PROMPT_ID`, alongside the delivered -`app promptkit requirements show PROMPT_ID` command. - -The organizing principle is **CommandKit provides useful, consistent access to -capabilities owned elsewhere**. CommandKit owns vocabulary, registration, -selection, coordination, presentation, and invocation outcomes. PromptKit owns -what concrete request validation means. Applications supply their domain -mapping, defaults, artifacts, and runtime dependencies. - -The command answers whether the constructed request passes native preparation -at the time of the invocation. It complements discovery without making -discovery a prerequisite for validation or validation a prerequisite for -application execution. - -## Current baseline - -The baseline is CommandKit commit `700f10d78ed0e410b0cc1d754fdd98415cd5817a`. - -- The optional [ckprompt module](../../integrations/ckprompt/go.mod) requires - PromptKit v0.14.0, ConfigKit v0.7.1, and Go 1.26.0. -- Its [suite and engine adapters](../../integrations/ckprompt/adapters.go) - support independent prompt/profile catalogs, named definition inspection, - and requirements discovery. `FromEngine` supplies those callbacks using an - application-owned engine loader. There is no concrete validation callback. -- The [requirements command](../cli.md#promptkit-requirements) accepts a prompt - ID, configuration selection, typed suite selectors, version/profile/session - context, and repeatable variable assignments. It returns the native discovery - evidence as YAML. A partial inventory or missing variable is not itself an - invocation failure. -- `ConfigScope` separately exposes engine configuration inspection and - comparison through ckconfig. It does not install the PromptKit command family. -- The [composed consumer](../../integrations/ckprompt/examples/composed/main.go) - demonstrates discovery and application-owned configuration/engine loading. - It does not yet demonstrate construction or validation of a concrete request. - -PromptKit v0.15.0 adds -[`Engine.ValidateRequest`](https://gitea.maximumdirect.net/eric/promptkit/src/commit/635674c066d0b432187f96e72b9c03d085443ac5/engine.go) -for an ordinary public `RunRequest`. Its -[consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/commit/635674c066d0b432187f96e72b9c03d085443ac5/docs/consumers/pkg-promptkit.md#validate-a-concrete-request) -and [release note](https://gitea.maximumdirect.net/eric/promptkit/src/commit/635674c066d0b432187f96e72b9c03d085443ac5/docs/releases/v0.15.0.md) -describe the addition. The reviewed release leaves the upstream minimum Go -version, module dependencies, and licensing material unchanged from v0.14.0. - -## Targeted end state - -### Command placement and meaning - -The standard PromptKit family can expose both operations independently: - -| Command | Operator purpose | Owning native capability | -| --- | --- | --- | -| `app promptkit requirements show PROMPT_ID` | Discover input and variable requirements for a prompt context. | `Engine.InspectRequirements` | -| `app promptkit requirements validate PROMPT_ID` | Check the actual constructed request, including supplied values and artifact bindings. | `Engine.ValidateRequest` | - -The new handler delegates to concrete native validation. It does not infer -validity from the discovery report's `Complete`, inventory, or issues, and does -not independently interpret templates or decide which inputs are required. - -Native validation shares `Prepare`'s request conversion and preparation path. -It may resolve prompt/profile resources, load artifacts and schema resources, -render messages and session context, and perform the applicable configuration -and credential checks. All supplied inputs follow native preparation behavior, -including inputs unused by the selected template branch. Errors are returned -as encountered rather than as an exhaustive inventory. - -There is no model call or retained prepared snapshot. Success neither certifies -provider acceptance nor guarantees that a later execution will succeed. A later -run prepares independently. Applications needing execution tied to a checked -snapshot can use PromptKit's existing prepared-execution API in their own -workflow; that workflow is outside this feature. - -### Operator inputs - -Use the existing requirements command's common syntax for: - -- One required prompt ID operand. -- `--config` and `--overlay`, with existing explicit-presence handling. -- Typed suite selectors. -- `--version`, `--profile`, and `--session-id`, forwarding their native - empty-string semantics. -- Repeatable `--var KEY=VALUE`, preserving the existing first-separator split, - exact keys and values, and last assignment for a duplicate key. - -Keep singleton repetition and selector binding behavior consistent with the -current CLI contract. PromptKit owns semantic interpretation of the values. -The namespace identifies the library; configuration scope selection belongs -to the separate `config` family. - -Add repeatable `--input NAME=PATH` for file-backed bindings, translated to native -`promptkit.File(PATH)` references without eagerly reading files. Split at the -first `=`, preserve exact names and paths, and use the last binding for a -duplicate name. Every occurrence is parsed; a later valid binding does not -erase an earlier malformed one. - -| Invocation fragment | Result | -| --- | --- | -| `--input notes=notes.txt` | Bind `notes` to `promptkit.File("notes.txt")`. | -| `--input notes=dir/a=b.txt` | Preserve the path `dir/a=b.txt`. | -| `--input notes=` | Forward an empty file path to native validation. | -| `--input =notes.txt` | Forward the empty input name to native validation. | -| `--input notes=first.txt --input notes=second.txt` | Bind `notes` to `second.txt`. | -| `--input notes` | Usage error because the separator is missing. | - -PromptKit owns semantic artifact validation and loading. CommandKit adds no -input-name whitelist, path restrictions, normalization, shell expansion, or -configuration-relative rebasing. Paths are preserved after shell processing. - -Use `--format yaml` for the initial success report, with YAML as the default. -Application-specific settings continue to use typed application input and -callbacks. Applications can supply inline and domain-derived artifacts through -their builder; a standard inline-input flag is outside this delivery. The -command need not offer a built-in flag for every field in `RunRequest` to permit -applications to use those fields through their builder. - -### Public integration and request construction - -Add an independently optional concrete validation capability to `ckprompt.Suite`. -Installing validation alone must be useful; supplying discovery alone must -continue to install only discovery. Both operations belong to the same -`requirements` group when present. The existing installed-family ownership, -absent-suite behavior, and retained wiring errors apply. - -Use a ckprompt-owned envelope containing configuration selection and the native -seed request populated from the standard command inputs: - -```go -type ValidationRequest struct { - Selection ckconfig.Selection - Request promptkit.RunRequest -} -``` - -Add this optional callback field to `Suite[I, S]`: - -```go -ValidateRequest func( - context.Context, ValidationRequest, I, S, -) (commandkit.Acquired[struct{}], error) -``` - -The acquisition container carries diagnostics; validation has no native result -value to invent. Custom callbacks remain available for application-owned runtime -arrangements. - -Provide a suite-returning `WithRequestValidation` helper that takes a suite, the -existing engine-loader shape, and an optional typed builder: - -```go -func WithRequestValidation[I, S any]( - suite Suite[I, S], - load func(context.Context, ckconfig.Selection, I, S) (*promptkit.Engine, error), - build func(context.Context, ValidationRequest, I, S) (promptkit.RunRequest, error), -) Suite[I, S] -``` - -The helper attaches the engine-backed validation callback and retains construction -errors, including a nil loader, in the returned suite for installation to report. -Existing suite capabilities and attached issues are preserved. Narrowing the -capabilities afterward does not discard retained wiring errors. Returning a suite -gives these errors the same treatment as existing adapters. - -For application-defined `loadEngine` and `buildRequest` callbacks, composition is: - -```go -suite := ckprompt.FromEngine(loadEngine) -suite = ckprompt.WithRequestValidation(suite, loadEngine, buildRequest) -``` - -A validation-only consumer can start from an empty suite. Given its application -types `Input` and `State`, it can use the standard command-populated request -directly: - -```go -suite := ckprompt.WithRequestValidation( - ckprompt.Suite[Input, State]{}, loadEngine, nil, -) -``` - -A nil builder passes the seed request unchanged to native validation. Standard -flags already supply useful prompt context, variables, and file bindings; -application mapping is optional. `FromEngine` alone continues to supply its -existing inspection callbacks, so validation remains an explicit wiring choice. - -When present, the builder receives invocation context, the selection and seed -request, selected application input, and shared invocation state. It returns -the concrete native request. It can supply application artifacts, inline content, -execution/output-contract overrides, appended messages, credentials, or other -supported native settings. The adapter forwards that result without reapplying -or merging fields afterward. - -Builders should start from the seed, preserve the selected prompt identity and -operator values, and add application defaults deliberately. Map defaults respect -key presence, including a supplied empty value. For example, within a builder: - -```go -request := seed.Request -if request.Vars == nil { - request.Vars = make(map[string]string) -} -if _, supplied := request.Vars["audience"]; !supplied { - request.Vars["audience"] = "internal" -} -return request, nil -``` - -Here `seed` is the builder's `ValidationRequest`; `--var audience=` remains an -explicit empty value. Version, profile, and session fields retain their native -empty-string semantics, including equivalence between omitted and explicitly -empty scalar values. Configuration selection retains its existing presence -information. CommandKit documents these responsibilities without adding a second -native-field validation layer. Reusing a request builder between validation and -application execution is recommended consumer guidance. - -### Invocation lifecycle and outcomes - -Reuse the integration's current typed input preparation, selector application, -input checking, invocation state, configuration diagnostic adaptation, and -outcome/delivery machinery. Construction, installation, help, and version must -leave acquisition and request-building callbacks uninvoked, under the existing -[architecture policy](../policy/architecture.md). - -For the bundled adapter, use this order: - -1. Complete CLI preflight and application input checks. -2. Create shared invocation state. -3. Invoke the builder, if supplied; otherwise use the seed request unchanged. -4. Load the engine using the selected configuration, application input, and state. -5. Invoke native validation with the concrete request. -6. Deliver diagnostics and the success report through the established paths. - -Both builder and loader receive the same invocation state and context. -Applications can explicitly share configuration acquisition through that state. -A builder failure prevents engine loading. Check cancellation between stages, -preserve diagnostics already acquired, and leave later stages uninvoked after -failure. Preserve supported native error identity and configuration diagnostics, -including failures during engine loading. - -Use the existing classification model: malformed CLI syntax and missing -operands are usage failures. Application input-check failures retain their -existing handling. Builder, engine-loader, and native-validation failures are -operational failures with supported error identity preserved. Returned provider -errors do not carry arbitrary usage or domain outcome classifications; -application-owned handlers can return their own outcomes. Preserve -native cancellation error categories; do not add a promise that every native -failure matches `ctx.Err()`. - -Successful native validation produces the selected success presentation only -after it succeeds. Diagnostics and output failures follow the established -delivery contract. The presentation reports the validation outcome without -dumping the request, artifact content, credential values, or rendered messages. - -### Initial success presentation - -Emit one small YAML report, defaulting to YAML and accepting explicit -`--format yaml` as the only format for this delivery: - -```yaml -prompt_id: briefing -valid: true -``` - -The ID identifies the command's requested prompt. `valid` means that validation -of the concrete request returned nil at this invocation; with the bundled -adapter, this is the result of native preparation. Builder guidance preserves -the selected prompt identity so the confirmation describes the operator's -selection. CommandKit adds no independent identity-validation layer. - -Native validation supplies neither resolved version/profile metadata nor a -prepared snapshot. The report therefore contains no such claims or discovery -inventories. Failures use the existing diagnostic/error outcome path. Output -delivery is part of invocation success. This initial presentation provides an -explicit confirmation for operators and automation; additional formats can -follow concrete needs. - -### Package and dependency ownership - -All new provider contracts, request adaptation, standard handlers, and -presentation belong in `integrations/ckprompt`. Core continues to own generic -declarations, bindings, dispatch, and outcomes. ckconfig remains responsible -for its configuration adaptation; ckprompt can reuse it where appropriate. - -Adopt the released PromptKit v0.15.0 module. Preserve the current Go minimum -and dependency direction: ckprompt may depend on ckconfig; ckconfig and core -do not acquire a PromptKit dependency. Use public upstream APIs and released -module versions, without sibling-checkout requirements in committed manifests. - -## Work needed from the baseline - -### Dependency adoption - -Update ckprompt's PromptKit requirement and checksums to v0.15.0. Review the -resulting selected module graph, and update the PromptKit attribution entry's -selected version and source/notice links to release commit -`635674c066d0b432187f96e72b9c03d085443ac5`. Update other attribution entries only -where the selected graph or incorporated material actually changes. - -### Integration API and native adapter - -Add `ValidationRequest`, the optional suite callback, and the suite-returning -`WithRequestValidation` helper. Reuse existing engine-loading diagnostics and -supported error handling. -Support concrete request construction from typed application input and standard -CLI values, without copying PromptKit's preparation logic. Account for builder -failure and invalid loader wiring using the established integration conventions; -a nil builder uses the seed unchanged. - -Keep current discovery and configuration adapters available independently. -`FromEngine` keeps its current capabilities. Validation is explicitly enabled -through the helper or a custom suite callback, with an application builder only -when additional mapping or defaults are needed. - -### Command assembly, bindings, and presentation - -Extend `ckprompt.Command` to assemble the requirements group from its supplied -show and validate capabilities. Add the validate handler, file-binding syntax, -and YAML success report. Share common binding/report mechanics where -that simplifies the code, preserving the delivered discovery contract. - -### Maintained consumer and documentation - -Extend the composed consumer with a synthetic concrete request builder and -offline validation examples for a successful request and a missing required -binding. Demonstrate the existing typed domain selectors alongside the new -standard bindings. The example should make the different questions answered -by show and validate clear without requiring a model service or real credential. -Document the nil-builder path and a builder that preserves supplied map entries -while adding defaults. - -When implemented, update Go declarations/GoDoc, the CLI reference, consumer -guide, and affected orientation/internal descriptions under their canonical -owners. Describe native resource work and validation limits accurately, so -statements about metadata inspection do not accidentally claim that concrete -validation performs no rendering, artifact loading, or credential checks. - -This capability fits the accepted optional-family and owning-library boundaries. -No new ADR or revision to accepted ADRs is needed. The selected API and CLI -details belong in their canonical documentation when implemented. - -### Completion evidence for implementation - -Use proportionate public-boundary evidence for independent capability -availability, help without acquisition, standard/typed input mapping, fresh -invocation state, nil-builder forwarding, builder default precedence, retained -wiring errors, builder-before-loader behavior, native validation error identity, -and diagnostic/output delivery. Cover builder and loader failures and cancellation -between stages. Use a real PromptKit engine for representative -successful validation and a missing binding, with a model-call double that -detects accidental execution. Protect continued successful discovery of a -partial inventory; do not turn it into a validation failure. - -Check semantic output rather than YAML key ordering or native error wording. -Avoid duplicating PromptKit's complete preparation test suite. Validate the -owning integration and affected composed consumer; check isolated module -resolution and dependency boundaries for the upgrade, using the established -workspace and disposable-module practices in the -[developer guide](../development.md). Implementation evidence must not rely -solely on a workspace that silently substitutes a sibling PromptKit checkout. - -The decisions needed for an implementation plan are settled. diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md deleted file mode 100644 index d5296c8..0000000 --- a/docs/roadmap/remediation.md +++ /dev/null @@ -1,213 +0,0 @@ -# Dependency upgrade and PromptKit smoke review remediation - -Status: complete; R-1 through R-4 addressed. - -## Review scope and evidence - -Reviewed `29582dd..730f930b47c929fb2c59a25a75cbe2924dfc75bb` against the -[accepted upgrade roadmap](upgrades.md), [implementation plan](implementation.md), -[architecture policy](../policy/architecture.md), and -[testing ownership policy](../policy/testing.md#upstream-integration-testing-ownership). -The worktree was clean at review start. - -The intended architecture is substantially present: - -- Core owns generic declarations, binding, installation, dispatch, help/version, - diagnostics, and outcomes; its Cobra/pflag adapter remains internal. -- ckconfig owns ConfigKit adaptation and the extended source reports. -- ckprompt depends on public core/ckconfig/ConfigKit/PromptKit APIs. Its eight - bundled inspection/catalog callbacks and two optional validation/smoke - additions preserve the selected ownership boundary. -- Native IDs, optional versions, filters, source evidence, and smoke reports - pass through the owning integrations. Aggregate smoke testing coordinates - concrete callbacks in the selected order and retains partial reports/errors. -- ReleaseKit remains a root maintainer tool. The adoption leaves the launcher, - profiles, shared-file selection, and source-only publication model intact. - -This was a source, test, documentation, and dependency review. Readonly -workspace and independent module listings confirmed ReleaseKit v0.4.2, -ConfigKit v0.8.0, and PromptKit v0.16.0. Independent package dependency listings, -including test imports, confirmed no optional integration libraries in core and -no PromptKit/ckprompt dependency in ckconfig. Temporary manifests replaced only -unpublished CommandKit prerequisites; upstream dependencies remained released -modules. No test suite was run or added during this review. - -The completed implementation plan records earlier test runs; those completion -markers are historical evidence, not fresh test results from this review. -No production correction was applied during the review. Completion records -below describe the subsequent remediation. The preceding completed -request-validation review remains available in -[Git history](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/730f930b47c929fb2c59a25a75cbe2924dfc75bb/docs/roadmap/remediation.md). - -## R-1: Preserve aggregate error presence when its message is empty - -Status: complete. Priority: P2. Category: report fidelity. - -**Location:** [smoke_report.go](../../integrations/ckprompt/smoke_report.go), -`smokeResultDTO` and `encodeSmokeResults`, lines 101–125. - -**Finding:** The DTO uses `Error string` with `yaml:"error,omitempty"`. -`encodeSmokeResults` assigns `result.Err.Error()` only when the error is -non-nil. A non-nil error whose message is `""` therefore produces the same -empty DTO field as no error, and `omitempty` drops it. - -For example, a concrete callback returning a native report and `errors.New("")` -still causes an operational aggregate outcome, but its YAML entry has no -`error` key. Report consumers cannot distinguish that entry from one whose -callback returned no error. No nonempty-error-message requirement was selected -for application callbacks. - -**Required work:** Track error presence separately from its display text. A -private `*string` DTO field with `omitempty` is sufficient: nil for no error, -and a pointer to the returned message for a non-nil error, including an empty -message. Preserve the existing native report, aggregate ordering, joined cause -identity, and normal outcome classification. Do not replace empty messages, -reject the error, or add another error taxonomy. - -**Completion evidence:** Add a focused semantic YAML regression covering nil, -empty-message, and ordinary nonempty errors. Only nil omits `error`; an empty -message remains `error: ""`. Confirm the assembled aggregate still returns -operational failure and preserves the original error through `errors.Is`. -The appropriate focused tests and ckprompt suite pass. - -**Completed:** 2026-10-18. Changed the private aggregate error field to an -optional string pointer, preserving non-nil empty messages. Extended the YAML -projection and assembled aggregate regressions to distinguish nil, empty, and -nonempty errors while retaining operational classification and typed cause -identity. Both focused regressions failed before the fix and passed afterward. -The complete ckprompt suite, including the composed consumer, passed with -Go 1.26.8 and offline dependency resolution. - -## R-2: Remove assertions about PromptKit's private smoke-test execution - -Status: complete. Priority: P3. Category: upstream testing ownership. - -**Location:** [adapters_test.go](../../integrations/ckprompt/adapters_test.go), -`TestWithSmokeTestingForwardsAuthoritativeNativeRequestAndRetainsReport`, -line 167, and -`TestWithSmokeTestingNilBuilderForwardsSeedAndBuilderFailureSkipsLoader`, -line 182. - -**Finding:** The nil-builder test requires the native generated prompt to contain -exactly one message. Both tests require exactly one call to the injected native -LLM client. CommandKit forwards a `SmokeTestRequest` to `Engine.SmokeTest`; -PromptKit constructs its private diagnostic prompt and owns its internal -generation sequence. - -A legitimate upstream change to the private prompt shape or diagnostic -generation sequence could fail these tests without any incorrect CommandKit -adaptation. The selected testing boundary permits observing the client to -verify forwarding, but does not establish private native work as a CommandKit -contract. - -**Required work:** Remove the exact native message-count assertion and exact -native generation-count expectations. Retain evidence that the intended native -path is reached and that the seed or authoritative builder request controls -the observed model/credential values. Keep report/error preservation, -configuration selection, context/input/state forwarding, and builder-failure -ordering checks. Where counting is useful, checking that native client use -occurred is sufficient; do not fix its native internal call count. - -This does not remove zero-call guards for help, metadata operations, or failures -that must stop before acquisition. Those protect selected CommandKit behavior. - -**Completion evidence:** Focused helper tests and the ckprompt suite pass. -Assertions still detect request remerging, wrong forwarding, or dropped results, -without inspecting the native private diagnostic message inventory or fixing -the generation sequence. No production-code change is required. - -**Completed:** 2026-10-18. Removed the private diagnostic message-count -assertion and changed native generation-count checks to require only that the -configured client was used. Request/model/credential forwarding, authoritative -builder output, report/error handling, context/input/state checks, and the -builder-failure guard against loading remain covered. Focused -`TestWithSmokeTesting` tests and the complete ckprompt suite, including the -composed consumer, passed with Go 1.26.8 and offline dependency resolution. -Production code is unchanged. - -## R-3: Remove incidental loader-count and pointer-identity contracts - -Status: complete. Priority: P3. Category: unnecessary implementation constraints. - -**Locations:** - -- [smoke_lifecycle_test.go](../../integrations/ckprompt/smoke_lifecycle_test.go), - `TestSmokeTestBuilderReceivesConcreteAggregateKinds`, lines 201–212. -- [smoke_aggregate_test.go](../../integrations/ckprompt/smoke_aggregate_test.go), - `TestSmokeAggregateCoordinatesConcreteRequestsAndRetainsErrors`, - lines 89–95 and its pointer-collection bookkeeping. - -**Finding:** The builder test requires exactly four engine-loader calls. -The accepted aggregate behavior requires per-kind requests/builders and allows -engine/acquisition reuse where practical; it deliberately does not guarantee -a loader-call count. - -The aggregate test also requires pairwise distinct addresses for every -execution override and selected scalar pointer. It already checks the -meaningful behavior: mutating the first seed must not change the common CLI -values seen by subsequent operations. Pairwise allocation identity adds a -separate constraint on storage strategy that the operator and consumer -contracts do not require. - -The implementation plan describes a straightforward fresh-storage approach. -That instruction does not independently establish public pointer identity or -resource-management guarantees under the -[agent policy](../policy/agents.md#commandkit-contract-authority). - -**Required work:** Remove the exact aggregate loader count and pairwise pointer -inequality assertions, together with bookkeeping used only for those assertions. -Retain concrete builder/callback order, per-kind builder inputs, shared -invocation input/state, subsequent seed values after mutation, continuation, -cancellation, and native error identity. Keep justified guards preventing -loading after builder failure and during help/version. Do not add caching, -pooling, or a public lifetime abstraction to resolve a test-only issue. - -**Completion evidence:** Focused aggregate/lifecycle tests and the ckprompt suite -pass. The tests continue to detect cross-test value contamination and missing -operations, while accepting equivalent acquisition and storage implementations. -No production-code change is required. - -**Completed:** 2026-10-18. Removed the aggregate loader counter and pairwise -pointer-address assertions/bookkeeping. The tests retain ordered concrete -builder/callback inputs, shared input/state, mutation-isolated seed values, -continuation, cancellation, reports, and error identity. Focused aggregate, -builder, help/version, and builder-failure tests and the complete ckprompt suite, -including the composed consumer, passed with Go 1.26.8 and offline dependency -resolution. Production code is unchanged. - -## R-4: Reconcile the feature roadmap with completed implementation - -Status: complete. Priority: P3. Category: stale implementation status. - -**Location:** [upgrades.md](upgrades.md), status at lines 3–6, -current-implementation descriptions at lines 28–29 and 41–44, and the source -DTO statement at line 124. - -**Finding:** The roadmap still says implementation has not started and that the -current source DTO omits all four new evidence fields. The code now exposes -those fields, and the implementation plan marks Stages 1–18 complete. Other -present-tense descriptions still refer to the planning baseline. - -Stage 18 expressly called for updating the roadmap's implementation status. -Leaving these claims in place makes implemented features appear absent and -can cause a later agent to restart completed work. - -**Required work:** Record the delivered implementation and link to the completed -plan/current canonical documentation. Qualify old baseline descriptions and -version tables as historical rather than rewriting the planning baseline. -Remove or recast assertions that the current DTO/API lacks delivered behavior. -Preserve accepted decisions, dependency boundaries, and the remaining review -findings; feature delivery can be recorded as complete while these focused -remediations remain open. - -**Completion evidence:** The roadmap and implementation plan agree on delivery -status, current-code claims match the inspected implementation, and local -links/Markdown whitespace checks pass. No code or policy change is required. - -**Completed:** 2026-10-18. Recorded implementation completion and linked the -completed plan, review follow-up, and current CLI/consumer/GoDoc owners. Marked -the original dependency/capability descriptions as historical and corrected the -source-report and ID-only API descriptions to match delivered code. Preserved -the original baseline and accepted decisions unchanged. Local links/fragments, -inbound roadmap anchors, status/baseline consistency, and Markdown whitespace -checks passed. No code or policy change was needed for this issue. diff --git a/docs/roadmap/upgrades.md b/docs/roadmap/upgrades.md deleted file mode 100644 index 90063e4..0000000 --- a/docs/roadmap/upgrades.md +++ /dev/null @@ -1,530 +0,0 @@ -# ConfigKit, PromptKit, and ReleaseKit adoption - -Status: implementation complete, including all three dependency upgrades and -smoke testing. The [implementation plan](implementation.md) records completion -of Stages 1–18; [remediation.md](remediation.md) records focused review follow-up -and its completion evidence. The following sections preserve the accepted target -and original migration intent. - -## Purpose and authority - -Adopt ReleaseKit v0.4.2, ConfigKit v0.8.0, and PromptKit v0.16.0 while providing -useful operator access to their capabilities. The organizing principle remains: -**CommandKit provides useful, consistent access to capabilities owned elsewhere.** - -CommandKit owns vocabulary, registration, selection, coordination, presentation, -and invocation outcomes. Integrated libraries own configuration composition, -resource identity and resolution, validation, inspection evidence, and diagnostic -meaning. Applications choose their inputs, policies, defaults, credentials, -artifacts, and runtime construction. - -Keep adaptation proportionate to that ownership. Reports are direct projections -of upstream data, with small presentation DTOs where YAML naming and formatting -need them. Preserve native values and meanings rather than introducing a -parallel semantic model or normalization layer. - -The [architecture policy](../policy/architecture.md), accepted [ADRs](../adr/), -and [release policy](../policy/release.md) govern this work. Upstream contract -changes flow through the owning integration; they do not justify reimplementing -removed behavior in CommandKit. The [CLI reference](../cli.md), -[consumer guide](../consumers/guide.md), [ckconfig GoDoc](../../integrations/ckconfig/doc.go), -and [ckprompt GoDoc](../../integrations/ckprompt/doc.go) own implemented usage -and contracts. The completed implementation plan records delivery evidence. - -## Planning baseline and reviewed releases - -The original adoption review used CommandKit baseline -`5f633e3feea71f623e5e378a7df476ff7e811e5e`; the implementation plan began at -`38b49f29fb3a8f22ff02528d2a0ed413e5c6ab1e`. This section records the historical -starting point and reviewed releases, rather than current dependency metadata. - -| Dependency | Planning baseline | Adopted release | Owning CommandKit area | -| --- | --- | --- | --- | -| ReleaseKit | Root tool requirement v0.4.0; integration graphs selected v0.4.1 | v0.4.2 | Root tool pin, launcher integration, release documentation, and notices | -| ConfigKit | v0.7.1 in both integrations | v0.8.0 | ckconfig adaptation and reports; ckprompt's direct dependency | -| PromptKit | v0.15.0 | v0.16.0 | ckprompt adapters, standard commands, reports, and composed consumer | - -At that baseline, the root module owned generic commands, dispatch, bindings, -help/version, and outcomes. ckconfig owned the configuration family. ckprompt -supplied prompt/profile catalogs, named inspection, requirements discovery, and -concrete request validation; its engine configuration scope was composed through -ckconfig. Current capabilities are documented in the -[consumer guide](../consumers/guide.md#promptkit-configuration-definitions-and-requirements). - -The completed [requirements roadmap](promptkit-validation.md), -[original implementation plan](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/38b49f29fb3a8f22ff02528d2a0ed413e5c6ab1e/docs/roadmap/implementation.md), -and [preceding remediation record](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/730f930b47c929fb2c59a25a75cbe2924dfc75bb/docs/roadmap/remediation.md) -remain historical evidence for the pre-upgrade baseline. - -Reviewed upstream references: - -| Release | Source commit | Release record | -| --- | --- | --- | -| ReleaseKit v0.4.2 | `7d033c5ca1c95aec7db9690293638a0a1ac887b8` | [Release notes](https://gitea.maximumdirect.net/eric/ReleaseKit/src/commit/7d033c5ca1c95aec7db9690293638a0a1ac887b8/docs/releases/v0.4.2.md) | -| ConfigKit v0.8.0 | `137769f3f18a370c68444242fb5d88e620e307e5` | [Release notes](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/137769f3f18a370c68444242fb5d88e620e307e5/docs/releases/v0.8.0.md) | -| PromptKit v0.16.0 | `9178f3eb91c2b2138caa78e22be93c048b4bc992` | [Release notes](https://gitea.maximumdirect.net/eric/promptkit/src/commit/9178f3eb91c2b2138caa78e22be93c048b4bc992/docs/releases/v0.16.0.md) | - -All three releases retain Go 1.26.0 as their consumer minimum. The adoption -preserved CommandKit's Go minimum and selected release toolchain. - -## Dependency and delivery boundaries - -Delivery includes all three upgrades, richer configuration source inspection, -expanded PromptKit discovery, and explicitly invoked smoke testing. The completed -[implementation plan](implementation.md) records coding order and stage -boundaries for the coordinated sprint. - -ReleaseKit has no runtime ordering dependency on the other two. During adoption, -ConfigKit was the prerequisite for PromptKit v0.16.0: upgrading PromptKit first -would have selected ConfigKit v0.8.0 transitively while ckconfig still declared -v0.7.1. Both integrations now declare ConfigKit v0.8.0 directly, preserving -alignment across their independent module graphs. - -Publication remains source-only and follows the existing -[core, ckconfig, ckprompt dependency order](../release.md#scope-and-release-order) -for affected releases. CommandKit module versions remain independent of upstream -version numbers. Explain intentional public API/CLI changes under the existing -pre-v1 compatibility policy; select concrete release versions during release -preparation. - -## Target state: ReleaseKit v0.4.2 - -The root Go tool requirement selects ReleaseKit v0.4.2. The existing launcher, -three module profiles, shared-file selection, and source-publication procedure -continue using its supported public contracts. No application command family or -runtime ReleaseKit import is added. - -v0.4.1 changed upstream test synchronization. v0.4.2 corrected maintained -Woodpecker delivery workflows for annotated-tag-object events; their event and -tag IDs are resolved to commits before verification and delivery. CommandKit -has no copied Woodpecker delivery workflow and publishes no hosted deliverables, -so adopting this patch does not require adding either. - -The adoption updated the root requirement/checksums, selected-graph notices, -the version named in the release policy, and version/commit references in the -maintainer release procedure. The launcher and release integration retain the -supported CLI/profile contracts. The ignored `.releasekit/` directory remains -local bootstrap storage. - -If a delivery workflow is introduced later, start from the corrected maintained -example: changing a module pin does not update copied YAML. This adoption does -not add delivery automation or new publication gates. - -## Target state: ConfigKit v0.8.0 - -### Native behavior and ownership - -Both integration modules use ConfigKit v0.8.0. Its input destinations, relative -import destinations, sequence-item placement, shared source-version checking, -and registered identified-sequence composition are available to application -loaders through upstream public options and definitions. - -CommandKit does not need a parallel composition layer or new generic flags to -select those policies. Applications decide whether to enable source-version -checking, which declarations to accept, and which sequences use identity-based -composition. Existing ConfigKit defaults preserve prior loading behavior. - -### Source inspection - -`config sources` now exposes native destination, placement, -unresolved-destination, and source-version declaration evidence in text and YAML. -The [source DTO](../../integrations/ckconfig/sources.go) includes all four while -preserving the original source fields and native inventory order. OD-1 records -the accepted intent; the [CLI source-report contract](../cli.md#list-configuration-sources) -owns the implemented representation. - -Project the native fields and their values directly into the report, including -`version_check` corresponding to `SourceInfo.VersionCheck`. Use ConfigKit's -returned, sanitized evidence and describe its fields neutrally. ConfigKit owns -the meaning and disclosure of that evidence. - -`config explain` continues to expose the native explanation/history contract; -the new SourceInfo fields do not require adding a source inventory to every -explanation. Compact diagnostic source legends can continue identifying sources -by ID, label, and filesystem path. - -### Completed adaptation - -Both dependency graphs, owning notices, native fixtures, and affected consumer -guidance were updated. The compatibility review accounted for unkeyed literals -and equality/map-key assumptions: ConfigKit's expanded `SourceInfo` contains a -`Path` and is no longer comparable. -Its [release notes](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/137769f3f18a370c68444242fb5d88e620e307e5/docs/releases/v0.8.0.md) -describe this source-compatibility change. The existing provider model need not -be redesigned merely to adopt these additions. - -## Target state: PromptKit v0.16.0 - -### ID-based prompt selection and descriptive labels - -ckprompt uses native ID-only prompt selection. The prompt-specific `--version` -flag was removed from named prompt inspection, requirements show, and requirements -validate, together with `PromptRequest.Version` and obsolete selector plumbing. -The adapter uses `InspectPrompt(ctx, promptID)` and the revised native request -types. -Application version reporting remains governed by its existing identity contract. - -Version labels remain useful metadata. Inspection/catalog report DTOs use the -native optional representation: `version` for a prompt or collection's reported -version, omitted when absent and retained as `""` when explicitly empty, as -accepted in OD-2. Selection continues to use the native ID-only API. - -The standard requirements operations continue delegating discovery and concrete -preparation to PromptKit. Partial discovery remains a successful inspection; -native validation remains authoritative for a constructed request. The existing -typed input/state, nil-builder path, callback capture, error identity, and -delivery behavior apply through the revised API. - -### Explicit configuration and collections - -The native fixtures and composed consumer now use one direct definition per -prompt ID. Where an application formerly supplied multiple definitions under -one ID, it chooses which definition to install or gives distinct resources -distinct IDs. -PromptKit's optional resource labels do not make that deployment choice. - -Treat a root configuration `version` as the upstream source specification -header, distinct from a prompt's descriptive `version` or a collection's -`release_version`. Their meaning and any configured acceptance checks remain -with the owning library and application. - -Supply backend/profile resources explicitly where needed. Remove assumptions -about installed provider catalogs and the former built-in standard/auto profiles. -CommandKit must not silently reintroduce those catalogs. The owning application -chooses files, embedded inputs, programmatic mappings, and collections. - -Keep the maintained composed consumer migration small: update its configuration, -loader, request builder, and fixtures as needed for direct definitions and the -new interfaces, without requiring a model service. Use a small configuration to -illustrate the relevant wiring and identities; a comprehensive collections -example or migration framework is not required. Collections are available -through native `CollectionInput` and configuration sources; applications need -not use them to adopt v0.16.0. - -Configured references retain their native exact identities. Mounting a source -does not rewrite backend/profile references or extend profile inheritance beyond -its native scope. Migration guidance should link to the upstream -[configuration/API migration guide](https://gitea.maximumdirect.net/eric/promptkit/src/commit/9178f3eb91c2b2138caa78e22be93c048b4bc992/docs/consumers/configuration-migration.md). - -### Expanded operator discovery - -The optional family exposes these additional catalogs: - -| Operation | Native capability | -| --- | --- | -| `app promptkit list backends` | `Engine.ListBackends` | -| `app promptkit list namespaces` | `Engine.ListNamespaces` | -| `app promptkit list collections` | `Engine.ListCollections` | - -Keep prompt/profile listings and add native namespace filtering where applicable. -These capabilities remain independently optional; one missing inventory does -not disable other operations. Reuse the existing construction, registration, -typed input/state, configuration selection, and diagnostic machinery. - -Forward native namespace filters and resource-kind selections to their respective -APIs. PromptKit performs identity resolution and inventory membership selection; -the adapter translates the selected CLI syntax into those native inputs. - -The namespace behavior has two distinct owners: - -- An application loader may return a native `WithNamespaces` engine view. That - view controls short caller-ID resolution and default resource listing scopes. -- An explicit native listing filter overrides membership selection for one - listing call. It does not change the engine view or rewrite configuration. - -Omitted filters retain native defaults: the corresponding resource view scope, -or all identities when unscoped. Namespace and collection inventories remain -global under the upstream contract. Reports show native identities and versions. -OD-3 records the accepted request/API and CLI filter choices. - -### Explicit smoke testing - -Expose PromptKit's `Engine.SmokeTest` through an independently optional -`promptkit test` command in this roadmap. Keep request construction and report -presentation close to the native public API, reusing the suite's typed -input/state, configuration selection, and engine acquisition paths. - -#### Command vocabulary - -The selected syntax is: - -```text -app promptkit test KIND [options] -app promptkit test all [options] -``` - -One operand selects either a native kind or the aggregate `all` operation. -The native kind spellings are `reachability`, `authentication`, `text_inference`, -and `structured_output`. `all` is CommandKit-owned coordination syntax, not a -native `SmokeTestKind`. - -For example: - -```text -app promptkit test reachability --backend local -app promptkit test authentication --profile editorial -app promptkit test text_inference --prompt briefing -app promptkit test structured_output --profile editorial --model example-model -app promptkit test all --profile editorial -``` - -#### Request bindings - -Bind operator inputs directly to `promptkit.SmokeTestRequest` and its native -`ExecutionTargetOverride`: - -| CLI input | Native destination | Binding | -| --- | --- | --- | -| `KIND` | `Kind` | Native kind string; aggregate expansion is described below. | -| `--backend` | `BackendID` | String. | -| `--prompt` | `PromptID` | String. | -| `--profile` | `ProfileID` | String. | -| `--endpoint` | `Execution.Endpoint` | String. | -| `--model` | `Execution.Model` | String. | -| `--temperature` | `Execution.Temperature` | Optional floating-point value. | -| `--max-tokens` | `Execution.MaxTokens` | Optional integer. | -| `--top-p` | `Execution.TopP` | Optional floating-point value. | -| `--timeout-seconds` | `Execution.TimeoutSeconds` | Optional integer. | -| `--service-tier` | `Execution.ServiceTier` | String. | -| `--reasoning-effort` | `Execution.ReasoningEffort` | Optional string. | -| `--api-key-env` | `Execution.APIKeyEnv` | String. | -| `--transient-retries` | `TransientRetries` | Optional integer. | - -Also accept the existing `--config`, `--overlay`, and suite selectors. Reports -are YAML-only; `--format` defaults to `yaml` and accepts `yaml`. Built-in flags -are singleton bindings with the established last-occurrence behavior; suite -selectors retain their declared binding behavior. - -Preserve omission versus explicit zero or empty values for native pointer -fields. Leave `Execution` nil when no execution override was supplied. Backend, -prompt, and profile selectors may be supplied together: PromptKit owns their -precedence, resolution, defaults, ranges, and request validation. CommandKit -owns CLI syntax and primitive value conversion. - -Direct `APIKey` and structured `ExtraParams` remain available through the -optional application request builder. This scope uses that existing integration -pattern rather than adding a credential-source mechanism or generic override -language. - -#### Consumer API and request construction - -Use a narrow ckprompt-owned envelope: - -```go -type SmokeRequest struct { - Selection ckconfig.Selection - Request promptkit.SmokeTestRequest -} -``` - -An independently optional `Suite.SmokeTest` callback receives context, -`SmokeRequest`, typed application input, and shared state, and returns -`(commandkit.Acquired[*promptkit.SmokeTestReport], error)`. It represents one -concrete test and preserves both a returned native report and error. - -Provide `WithSmokeTesting(suite, load, build)` following the existing -request-validation adapter pattern. The loader uses the existing engine-loader -signature. The optional builder receives context, `SmokeRequest`, typed input, -and shared state, and returns a complete native `SmokeTestRequest` and error. -With no builder, forward the CLI-created native request. With a builder, its -returned request is authoritative; do not merge CLI values back over it. - -For aggregate execution, invoke the optional builder separately for each -concrete kind, seeding that kind into the otherwise common bound request. -Applications can supply per-kind credentials, structured parameters, or other -native request choices. The synthetic `all` value does not reach the native -request or the per-test builder. - -#### Aggregate execution - -`app promptkit test all` selects these four tests sequentially, in order: - -1. `reachability` -2. `authentication` -3. `text_inference` -4. `structured_output` - -Coordinate the concrete test callbacks within one CommandKit invocation, using -its configuration selection, prepared typed input, and shared state. Reuse -configuration acquisition and the engine where practical within the existing -loader model; this does not require a new caching framework or guarantee a -particular loader call count. - -Continue to the next kind after an individual test error so that remaining -results can be collected. When the invocation is canceled, stop scheduling -further tests and retain the results already available. PromptKit owns each -individual operation; CommandKit coordinates the selected sequence. - -#### Reports and invocation outcomes - -Present the native `SmokeTestReport`, including observations, timing, attempts, -and available usage. For `all`, deliver one YAML report containing the -individual results in execution order, with each selected kind's returned -report and associated error, when present. Retain useful partial reports -returned with errors. Preserve supported error identity through the invocation -outcome as well as presenting the errors. - -For a single test, a nil native error and successful report delivery mean -success. For `all`, success requires all selected calls to return without errors -and reporting to succeed. Native invocation errors use the normal operational -failure classification; cancellation and delivery failures retain their existing -outcome handling. Native negative or unknown findings remain reported facts; -CommandKit does not derive an additional readiness verdict or failure rule from -them. - -Use a focused report-delivery path that preserves a returned report alongside -its native error, including cancellation. The existing success-only inspection -writer cannot be reused unchanged for that case. Adapt delivery within the -current outcome model; this work does not require a general diagnostics -framework. Describe returned data neutrally, without adding warnings about -perceived upstream limitations. - -Smoke testing runs when its command is explicitly invoked. Configuration, -requirements, help, and release operations retain their own existing behavior. -OD-4 records its inclusion in the current delivery scope; a separate feature -roadmap is unnecessary. The broader deferred proposals in -[deferred work](deferred.md) do not defer this selected native capability. - -## Work across the integrations - -### Dependencies and attribution - -Update the owning module requirements and real checksums for the selected -releases. PromptKit v0.16.0 drops its bundled backend-catalog dependencies; -remove obsolete indirect requirements, sums, and notices when absent from the -resulting CommandKit graphs. Do not replace them with implicit catalog loading. - -Keep root and integration attribution entries matched to their actual selected -graphs. ConfigKit and PromptKit still declare ReleaseKit v0.4.1 upstream; a -CommandKit graph may select v0.4.2 through its own core dependency. Record the -effective selection rather than blindly assigning the root tool version to -every module. Verify pinned license/notice source links for each adopted release. - -Core remains independent of optional ConfigKit/PromptKit dependencies, including -through examples, tests, and tools. Committed builds use released public modules, -not adjacent checkout replacements. - -### Documentation and compatibility - -Update GoDoc, the CLI reference, consumer guidance, maintained examples, internal -inventory, and release-tool references as their owning implementations land. -Document changed prompt configuration, ID-only selection, optional label data, -source evidence, new discovery behavior, and smoke testing under their canonical -owners. - -The accepted architecture already supports independently optional operations and -native-library authority. Adding these optional capabilities and following -upstream selection semantics fit those accepted boundaries. Any concrete new -architectural decision is evaluated under the ADR policy. - -Retain historical release records. Explain incompatible ckprompt changes in an -appropriately versioned future module release. New source-report fields and -catalog and smoke capabilities also need accurate CLI/API release notes. This -roadmap does not require a CommandKit version to mirror any upstream version. - -### Implementation evidence and testing boundary - -The [completed implementation plan](implementation.md#stage-18-verify-the-complete-target-and-finish) -records the delivered target and its workspace/independent-graph validation. -The [remediation record](remediation.md) tracks the focused follow-up review. -Testing remains directed to CommandKit-owned interfaces and adaptation under the -[upstream testing ownership policy](../policy/testing.md#upstream-integration-testing-ownership). -Verify that command inputs and context reach the intended native APIs, including -ID-only prompt requests, default/explicit listing filters, and smoke requests. -Verify faithful representation of upstream-returned source evidence, optional -labels, inventories, diagnostics, and errors, together with CommandKit's optional -command availability, invocation lifetimes, outcome classification, and output -delivery. Include smoke request bindings, per-kind builder forwarding, aggregate -order and continuation, cancellation, report-plus-error delivery, and error -identity in this boundary coverage. - -These checks establish CommandKit's access to the capabilities. They should not -test whether ConfigKit correctly places or composes sources, checks versions, -or classifies sensitivity, nor whether PromptKit correctly resolves namespaces, -discovers requirements, or validates concrete requests. Those domain tests belong -to the upstream libraries. Real upstream fixtures and maintained offline -consumers can exercise CommandKit's boundary without duplicating native feature -test matrices. Existing release-launcher coverage likewise owns CommandKit's -tool invocation, not ReleaseKit's publication algorithms. - -Verify all affected modules both in the local workspace and with their own -readonly dependency graphs. Temporary replacements for unpublished CommandKit -prerequisites may support development checks; they are not evidence for a -sibling-replaced upstream dependency or committed build requirements. The -[developer guide](../development.md) owns the established module-check practices. - -New tests should protect selected contracts without requiring incidental -factory activity after failure, complete diagnostic text snapshots, YAML key -ordering, or arbitrary copies of caller-owned objects. The migration, discovery, -and smoke adapter checks can use offline fixtures and test doubles; smoke tests -verify request forwarding and report/error adaptation, leaving provider behavior -to PromptKit. - -## Accepted Decisions - -### OD-1: Representation of the added source evidence - -**Accepted:** Extend both existing source formats with `destination` using -ConfigKit's canonical JSON Pointer string (root is `""`), `placement` using native -mode names, `destination_unresolved`, and `version_check` corresponding to -`SourceInfo.VersionCheck`. - -Preserve the native nested evidence: path, version presence/null/text, optional -source location, and redaction flag. Use native sanitized values and maximize -API-to-report correspondence. Field descriptions explain the returned data -neutrally, without promising additional checker behavior or warning about -perceived upstream limitations. - -### OD-2: Optional versions in reports - -**Accepted:** Report a prompt or collection's version as `version`. Omit this -optional field when the native value is absent; retain `""` when it is present -and empty. For collections, this projects the native `ReleaseVersion` value. -Use the same naming and presence treatment across the affected reports. - -Version metadata retains its upstream meaning. ID-only selection and source -configuration headers remain governed by their respective native APIs. - -### OD-3: Catalog requests and namespace-filter syntax - -**Accepted:** Use narrow ckprompt-owned request envelopes carrying configuration -selection and native `promptkit.NamespaceFilter` for prompt, profile, and backend -catalogs. Give namespace enumeration its native resource-kind selection, and -leave collection enumeration aligned with its unfiltered native API. Revise -existing catalog callback shapes coherently rather than retaining parallel -filtered/unfiltered variants solely for the prototype API. - -For applicable listing commands, map operator input directly to native filters: - -| Operator input | Native filter mode | -| --- | --- | -| No namespace option | `NamespaceDefault` | -| `--namespace NAME` | `NamespaceNamed` with the supplied name | -| `--namespace ""` | `NamespaceUnqualified` | -| `--all-namespaces` | `NamespaceAll` | - -Existing flag-presence tracking distinguishes omission from an explicitly empty -namespace. These choices need no separate `--unqualified` flag or reserved -namespace name such as `all`. Conflicting namespace choices are usage errors: -one native listing accepts at most one namespace filter. - -Namespace enumeration uses repeatable `--kind prompt|profile|backend`; omission -retains the native union. Application-created engine views and explicit listing -filters retain their separate native meanings. - -### OD-4: Delivery boundary for smoke testing - -**Accepted:** Include an independently optional smoke command in this roadmap, -alongside dependency migration and expanded discovery. Expose the native request, -operation, and report directly through the existing integration model. - -Deliver native reports, including a report returned with an error. Successful -native invocation and report delivery yield success; native errors remain -operational failures with supported error identity preserved. Native observations -are reported neutrally. CommandKit does not add an aggregate readiness judgment -or warnings about perceived upstream limitations. - -The [explicit smoke testing target](#explicit-smoke-testing) records the accepted -`promptkit test KIND` and `promptkit test all` vocabulary, native request -bindings, per-kind builder, and aggregate execution/reporting semantics. A -separate smoke feature roadmap is not required.