From 603b6b375882e238136ba471015b9ac8aa6af627 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 12 Oct 2026 01:12:59 +0000 Subject: [PATCH] Plan PromptKit comparison and catalog adoption --- docs/roadmap/promptkit.md | 381 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 381 insertions(+) create mode 100644 docs/roadmap/promptkit.md diff --git a/docs/roadmap/promptkit.md b/docs/roadmap/promptkit.md new file mode 100644 index 0000000..1b94214 --- /dev/null +++ b/docs/roadmap/promptkit.md @@ -0,0 +1,381 @@ +# Adopt PromptKit v0.14.0 and expose comparison and catalogs + +Status: proposed feature end state; implementation has not started. The scope +and ownership direction are agreed. Catalog API and presentation choices remain +under [Open Decisions](#open-decisions). + +## Purpose and authority + +CommandKit provides useful, consistent access to capabilities owned elsewhere. +Adopt PromptKit v0.14.0 and expose its native configuration comparison and +definition catalogs without requiring applications to duplicate command +infrastructure. Defer a standard command for `Engine.InspectRequirements` to +the next pass. + +CommandKit owns command vocabulary, registration, selection, coordination, +presentation, and invocation outcomes. PromptKit owns the meaning of its +configuration inspection and catalogs, including which effective definitions +exist and the comparison capability it exposes. PromptKit delegates its native +comparison to ConfigKit. The adopted upstream public contracts remain +authoritative; this roadmap does not establish a second CommandKit definition +of their semantics or disclosure rules. + +The [architecture policy](../policy/architecture.md) controls dependency direction +and ownership. [ADR 0004](../adr/0004-standard-inspection-surface.md) already +selects the standard catalog commands; +[ADR 0008](../adr/0008-semantic-configuration-diff.md) governs native comparison, +[ADR 0009](../adr/0009-configuration-scopes.md) governs independent scope +capabilities, and [ADR 0010](../adr/0010-optional-integration-modules.md) governs +optional modules. [ADR 0011](../adr/0011-captured-declarations-and-installation.md) +and [ADR 0012](../adr/0012-typed-configuration-comparison-pairs.md) govern captured +declarations and input/state lifetimes. Existing deliberately selected contracts +control; prototype implementation shapes receive no compatibility preference. + +This is a feature roadmap, not a staged coding assignment. A later implementation +plan should sequence the work after the open decisions are resolved. Only this +roadmap is added in the present pass; it does not upgrade dependencies, revise +ADRs, change current behavior documentation, or run Go tests. + +## Current baseline + +Baseline: CommandKit commit `ab270e2`. + +| Owner | Current state | +| --- | --- | +| Core | Go 1.26.0; declarations, registration, typed bindings, dispatch, help/version, diagnostics, and outcomes. Cobra/pflag remain internal. No optional owning-library dependency. ReleaseKit v0.4.0 is the declared maintainer tool. | +| `integrations/ckconfig` | ConfigKit v0.7.1; independent validate/show/sources/explain/diff capabilities, paired selection/input preparation, and the shared YAML comparison report and delivery path. Report placement uses a report-local mapping index. | +| `integrations/ckprompt` | PromptKit v0.12.0 and ConfigKit v0.7.1. `ConfigScope` exposes validate/show/sources/explain, without comparison. `Suite` and `FromEngine` expose named prompt/profile inspection; `Command` generates only `promptkit show` children. | +| Maintained composed consumer | Demonstrates both configuration scopes, app configuration comparison, named prompt/profile inspection, and an application-owned session command. Application configuration caching distinguishes selections and supplied campaign inputs. | +| Deferred work | PromptKit comparison and catalogs are recorded as future capabilities in [deferred.md](deferred.md). They are targets of this adoption; requirement inspection is deferred separately. | + +The existing [ckprompt contracts and adapters](../../integrations/ckprompt/adapters.go) +and [command assembly](../../integrations/ckprompt/command.go) are the starting +points. ckprompt already depends on ckconfig's public scope contracts; no new +module or dependency reversal is needed. + +## Upstream adoption + +Target PromptKit v0.14.0, source commit +`143c6e9b3a675872117b0471e2ff8bcb25055a54`. Relevant pinned sources are its +[v0.13.0 release notes](https://gitea.maximumdirect.net/eric/promptkit/src/commit/143c6e9b3a675872117b0471e2ff8bcb25055a54/docs/releases/v0.13.0.md), +[v0.14.0 release notes](https://gitea.maximumdirect.net/eric/promptkit/src/commit/143c6e9b3a675872117b0471e2ff8bcb25055a54/docs/releases/v0.14.0.md), +[configuration inspection API](https://gitea.maximumdirect.net/eric/promptkit/src/commit/143c6e9b3a675872117b0471e2ff8bcb25055a54/configuration_inspection.go), +and [listing API](https://gitea.maximumdirect.net/eric/promptkit/src/commit/143c6e9b3a675872117b0471e2ff8bcb25055a54/listing.go). + +Relative to CommandKit's selected v0.12.0: + +- v0.13.0 adds `Engine.ListPrompts`, `Engine.ListProfiles`, and + `Engine.InspectRequirements`, plus allocation and inspection improvements. +- v0.14.0 adds `ConfigurationInspection.Diff`, returning ConfigKit's public + difference records from two captured configuration inspections. +- The Go consumer minimum becomes 1.26.0, already met by all CommandKit modules. + ConfigKit v0.7.1 and the YAML version also match CommandKit's current selection. +- PromptKit selects ReleaseKit v0.4.1 as its maintainer tool. That version is + already selected through ConfigKit in the integration graphs. Adoption does + not itself change core's separately declared release tool or launcher. + +The reviewed existing public inspection and construction signatures remain +available. Adoption must still check CommandKit's existing reports and error +handling against the upgraded implementation; a source review is not evidence +that the upgraded integration suites have passed. No upstream feature addition +is needed for the selected comparison and catalog goals. + +## Target ownership and package shape + +| Owner | Target responsibility | +| --- | --- | +| PromptKit | Supported configuration comparison and catalog APIs, effective membership, identity/version interpretation, native results, ownership, and disclosure contracts. | +| ConfigKit | Native effective-value comparison and paired safe snapshots used by PromptKit. | +| ckprompt | PromptKit adapters; engine acquisition and diagnostic adaptation; declaring comparison on its configuration scope; catalog callbacks, standard list handlers, and catalog presentation. | +| ckconfig | The existing `config` family, capability preflight, selection inheritance, paired typed inputs, shared-state coordination, comparison YAML rendering, and report delivery. | +| Core | Existing captured declarations, binding conversion/presence, installation, dispatch, help/version, and outcomes. No PromptKit-specific model. | +| Application | Source definitions, domain selector interpretation, loaders and caches, callback effects, runtime operations, stream selection, and process exit. | + +Dependencies continue to point from ckprompt to ckconfig and core. ckconfig does +not import or require ckprompt or PromptKit; core remains independent of both +integrations. Reuse ckconfig's public contracts and installed command paths. +No duplicate comparison renderer, core provider registry, comparison facet, +backend change, sibling-internal import, or general plugin framework is needed. +Private helper organization remains an implementation choice. + +## PromptKit configuration comparison + +### Operator end state + +`ckprompt.ConfigScope(loadEngine)` declares `ckconfig.Diff` alongside its existing +operations. Installing that scope through `ckconfig.Command` makes these +invocations available: + +```text +app config diff --scope promptkit --config before.yml --against-config after.yml +app config diff --scope promptkit --config base.yml --against-overlay candidate +``` + +These are target invocations, not currently implemented PromptKit comparisons. +They use the existing [comparison CLI contract](../cli.md#compare-configuration-selections): +one scope, left selectors and inherited right overrides, shared applicable domain +selector values, YAML-only additions/changes/deletions presentation, and success +for completed comparisons regardless of whether differences exist. There is no +new `promptkit diff` command, scope aggregation, report envelope, or exit status. +Help and `config scopes` reflect the actual declared capability. Other scopes +remain free to support their own operation subsets. + +### Adapter end state + +The scope's `Compare` callback consumes the existing +`ckconfig.ComparisonRequest[I]` and shared `S`. ckconfig prepares and checks each +reached side's typed input, then initializes shared state once. The bundled +PromptKit adapter invokes the supplied engine loader for left and then right, +using each side's selection/input and that same state, obtains each engine's +`InspectConfiguration` view, and calls `left.Diff(right)`. + +Return `commandkit.Acquired[[]configkit.Difference]` with accumulated diagnostics +and supported error identity preserved. The existing ckconfig report path +consumes those records. Ordinary operations continue through `Acquire` and the +existing inspection facets. A caller may narrow the scope's capabilities or +supply its own pair provider under the existing scope contract; original adapter +construction errors remain attached when capabilities are narrowed. + +For the bundled adapter, retain diagnostics from each reached load or inspection, +identify left/right context and native source legends, keep left diagnostics if +right acquisition fails, and keep both sides' diagnostics if comparison fails. +Use existing cancellation checks at operation boundaries and finish report +preparation before delivery. The upstream Diff method is synchronous and takes +no context; do not promise interruption inside it or transactional writer output. + +Equal selections alone do not skip the pair provider or the bundled adapter's +reached loader calls. Application caches determine whether calls construct new +engines or reuse captured ones. CommandKit does not supply, inspect, or repair +those caches or promise physical source reloads or distinct factory objects. + +### Meaning and presentation + +Delegate to PromptKit's adopted comparison contract. Its current API compares +captured effective configurations through ConfigKit, rather than source-history +text, referenced file contents, resolved credentials, or later request-specific +execution state. These are upstream semantics, not independently frozen +CommandKit guarantees. An empty configuration comparison does not establish +equivalent execution. + +Preserve native record evidence at the provider boundary and use the existing +safe right-side YAML presentation. Follow the adopted upstream classification; +do not add blanket masking of provider parameters or infer metadata from mask +text. Ordinary prompt/provider content can remain application data. Applications +retain their disclosure choices and callback effects. + +## Prompt and profile catalogs + +### Operator end state + +Extend the optional `promptkit` family with the command paths already selected +by ADR 0004: + +```text +app promptkit list prompts +app promptkit list profiles +``` + +Each command accepts configuration/overlay selection and the applicable existing +suite domain selectors, without requiring a definition ID or exact-version +operand. These commands use the namespace's engine selection, rather than a +new configuration-scope selector. Catalog report formats and YAML shape remain +the presentation decisions below. No filtering, pagination, version-priority +rule, requirement discovery, or per-definition readiness check is added in this +delivery. + +The bundled adapter obtains prompt ID/version pairs with `Engine.ListPrompts` +and profile IDs with `Engine.ListProfiles`. It preserves returned identities +and order without reconstructing catalogs from configuration YAML or invoking +named inspection for each entry. PromptKit's current lists describe effective +membership, include maintained definitions that remain effective, and order +identities lexically; prompt versions are not ranked semantically. Upstream +remains authoritative if its adopted contract evolves. + +Successful empty catalogs produce an empty collection under the selected format. +They do not fail because there are no definitions. Membership does not prove +execution readiness. Identifiers retain their exact values, including values +that require YAML quoting; CommandKit does not normalize identifiers or promise +that catalog data is safe for arbitrary public disclosure. + +### Consumer and invocation end state + +Add independently optional catalog providers to `ckprompt.Suite[I, S]`, using +`commandkit.Acquired[[]promptkit.PromptIdentity]` and +`commandkit.Acquired[[]string]` as their results. Reuse meaningful upstream public +types. The request shape recommendation is under OD-1. + +`FromEngine(loadEngine)` supplies both catalog callbacks alongside the existing +named-inspection callbacks. It acquires an engine only when the selected catalog +operation is reached, calls the corresponding native listing API, and returns +adapted load diagnostics with the result or error. Listing itself does not +execute prompts or contact model providers; application loaders and custom +callbacks retain their own effects. + +`Command(suite)` captures metadata, bindings, and callback references and emits +only supplied operations. Listing can be installed without either named-show +callback; named-show operations can remain installed without catalog callbacks. +An entirely absent suite contributes no namespace or commands, while attached +construction errors are still reported. A nonempty family remains root-only and +owns its namespace under the existing installation contract. Importing ckprompt +does not install a family; configuration-scope and definition-family installation +remain independent. + +For one reached catalog invocation, prepare one `I`, apply the captured converted +selectors, complete existing input checks, initialize `S` once, and invoke the +selected provider. Preserve supplied empty/false/zero values, original binding +identity, nil-factory zero values, and caller ownership. Installation, help, and +version invoke neither suite factories nor providers; operator-requested listing +does acquire its selected inventory. Use existing usage/operational outcomes, +diagnostic delivery, cancellation boundaries, and write-failure handling. + +## Work needed from the baseline + +### Dependencies and distribution + +- Upgrade ckprompt's PromptKit requirement from v0.12.0 to v0.14.0 and reconcile + real module-local checksums and legitimate indirect requirements. Retain + ConfigKit v0.7.1, Go 1.26.0, and the existing core/ckconfig versioning policy. +- Preserve ignored development workspace wiring. Distributed manifests must not + acquire sibling replacements or invented checksums for unpublished CommandKit + modules. Reuse the existing isolated-module validation approach where needed. +- Update PromptKit's module-local attribution and source-pinned license/notice + links under the [OSS policy](../policy/oss.md). Review actual selected graphs; + do not copy workspace-selected tool versions into core's declaration/notices. +- Reconcile prepared integration release notes with the delivered capabilities + under the [release policy](../policy/release.md). Publication, tag selection, + and independent module release ordering remain with the + [release procedure](../release.md); this adoption does not publish a release. + +### Adapters and command assembly + +- Extend `ConfigScope` with the native PromptKit comparison provider, retaining + ordinary acquisition and appropriate side diagnostics. Route results through + the existing ckconfig comparison implementation. +- Extend `Suite`, `FromEngine`, and family assembly for independent prompt/profile + catalog providers. Use core declarations and typed binding composition for + selection and input preparation; add no second parser. +- Implement catalog presentation after OD-2 and OD-3 are resolved. Define + CommandKit-owned report fields explicitly instead of treating serialization of + an upstream Go struct as a permanent CLI schema. Preserve upstream values. +- Keep implementation proportionate. Reuse existing acquisition, preparation, + diagnostic, YAML, and outcome helpers where they fit; introduce only the + concrete helper changes needed for the new operations. + +### Examples, documentation, and behavioral evidence + +- Extend the maintained composed consumer with PromptKit-scope comparison and + both catalog commands. Reuse its selection-aware application loading so two + configurations yield meaningfully different engine inspections. Its cache + choices remain example choices, not policy for consumers. +- When behavior lands, update owning GoDoc, the CLI reference, consumer guidance, + module README, and internal overview. Replace claims that PromptKit comparison + and catalogs are unavailable. Reconcile the completed ConfigKit roadmap's + PromptKit status and the deferred-work record; retain requirement inspection + as explicitly deferred without inventing its future syntax. +- Validate existing show/validate/sources/explain behavior, named prompt/profile + reports, app comparison, and the composed domain command against the upgraded + dependency. Add focused evidence for catalog-only and mixed capabilities, + callback-free help, input/state timing, exact identities and versions, empty + inventories, native comparison, diagnostics retained across side failures, + cancellation, and report delivery failures. +- Prefer assembled behavior and real fast native collaborators. Reuse ckconfig's + existing report/lifecycle coverage; do not duplicate ConfigKit's comparator or + PromptKit's full catalog suite, freeze private layout or polling counts, or + introduce performance, concurrency, or release gates through tests. +- Establish compatibility through relevant module suites, maintained consumers, + and workspace-off dependency/import checks. Local replacements for unpublished + CommandKit prerequisites prove local-source compatibility, not availability of + future tags. These checks belong to implementation, not this prose-only pass. + +## Scope limits and completion + +The target is PromptKit v0.14.0 adoption, usable PromptKit-scope configuration +comparison, and independently selectable prompt/profile catalog commands with +current API/CLI documentation and representative consumer evidence. Existing +ADRs cover this direction; exact catalog signatures and report shapes belong in +their canonical owners when implemented. + +`Engine.InspectRequirements` is available through the upgraded dependency, but a +standard CLI command for it is deferred to the next pass. It is not integrated +into configuration validation or ordinary prompt display here. Execution, +preparation, rendering, credential probing, mutation, independent per-side domain +selectors, assertion statuses, JSON output, new scope aggregation, and backend +changes remain outside this delivery. + +Completion requires all selected capabilities and relevant dependency metadata, +notices, docs, examples, and boundary validation. Resolve the open decisions +before producing a decision-complete implementation plan. Recommendations below +are proposals, not already accepted CLI or Go contracts. + +## Open Decisions + +### OD-1: Catalog callback request shape + +**Recommendation:** Add `ListPrompts` and `ListProfiles` fields with direct +`ckconfig.Selection` input, separate `I` and `S`, and the existing acquisition +convention. The proposed signatures are: + +```go +ListPrompts func(context.Context, ckconfig.Selection, I, S) (commandkit.Acquired[[]promptkit.PromptIdentity], error) +ListProfiles func(context.Context, ckconfig.Selection, I, S) (commandkit.Acquired[[]string], error) +``` + +Catalog commands have no selected definition ID or version. Their only standard +selection data is already represented by `ckconfig.Selection`; domain values +belong in `I`. This keeps the API small and matches the existing engine-loader +shape without adding a request wrapper solely for speculative future fields. + +**Alternative:** Introduce a ckprompt-owned `CatalogRequest` containing +`Selection`, and use it for both callbacks. This matches the request-object +style of existing named-inspection providers and creates a place for future +catalog-specific options. It is viable, but adds a public type with no additional +information today. Choose it if that consistent provider shape is judged more +valuable than the narrower current surface; neither choice requires a core API +change. + +### OD-2: Catalog report formats and default + +**Recommendation:** Deliver YAML only, default to YAML, and accept explicit +`--format yaml`. Catalog sequences are readable for operators and directly +usable by automation. This keeps one presentation path, preserves arbitrary +identity strings without designing a separate escaping convention, and follows +the recent comparison report's deliberately small format scope. This is a +recommendation for these new commands, not a policy that all families must use +the same formats. + +**Alternative:** Support text and YAML, defaulting to text as existing +`promptkit show` commands do. One profile per line and prompt ID/version columns +are convenient for terminal use and keep the namespace's format defaults more +uniform. The cost is a second renderer and a decision about unambiguous display +of identities containing whitespace or line breaks. YAML-only listings can still +coexist with the established show-command formats without changing them. + +### OD-3: YAML catalog document shape + +**Recommendation:** Emit one bare sequence per invocation. Prompt entries have +explicit `id` and `version` string fields; profile entries are strings. For +example: + +```yaml +- id: briefing + version: "1" +- id: briefing + version: "2" +``` + +```yaml +- autumn +- spring +``` + +An empty inventory renders as `[]`. The command path already identifies the +resource, so a bare collection is simple to consume and does not add an unused +metadata framework. Preserve provider order and exact strings; quote through +the YAML encoder rather than interpreting numeric-looking versions. + +**Alternative:** Wrap the collections under `prompts:` and `profiles:` keys. +This gives each report a self-describing resource label and leaves room for +later metadata without changing the root type. It is a reasonable choice if +reports will routinely be stored without their command context. It also adds +nesting and anticipates metadata this delivery does not need; an empty report +would be `prompts: []` or `profiles: []` instead of a bare empty sequence.