Retire completed upgrade planning documents
This commit is contained in:
@@ -53,7 +53,7 @@ canonical documentation owners.
|
||||
| Public facade or adapter boundaries | [Internal overview](internal/overview.md), [ADR 0003](adr/0003-public-facade.md) as scoped by [ADR 0010](adr/0010-optional-integration-modules.md), and the architecture policy |
|
||||
| Module ownership and public contracts | [Consumer guide](consumers/guide.md), [internal overview](internal/overview.md), [core GoDoc](../doc.go), [ckconfig GoDoc](../integrations/ckconfig/doc.go), [ckprompt GoDoc](../integrations/ckprompt/doc.go), and applicable ADRs |
|
||||
| Deferred capabilities and proposals | [Deferred work](roadmap/deferred.md) |
|
||||
| Completed planning and implementation history | [Completed ConfigKit and PromptKit adoption and remediation](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/8a82620dfcd34c0ae56bbecb0d014eb49786bc92/docs/roadmap), [earlier retired records](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/6bc37e7ba64ca0920b4d0cc77039a18d4006b24e/docs/roadmap), and [completed module implementation and remediation](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/bb2528532b78d30e52f5082e491a5e1fd9b0fb99/docs/roadmap) in Git history |
|
||||
| Completed planning and implementation history | [Completed dependency upgrades, PromptKit validation, discovery, smoke testing, and remediation](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/a6278a6d9f940948ec7745955d745523aaff0734/docs/roadmap), [earlier ConfigKit and PromptKit adoption and remediation](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/8a82620dfcd34c0ae56bbecb0d014eb49786bc92/docs/roadmap), [earlier retired records](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/6bc37e7ba64ca0920b4d0cc77039a18d4006b24e/docs/roadmap), and [completed module implementation and remediation](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/bb2528532b78d30e52f5082e491a5e1fd9b0fb99/docs/roadmap) in Git history |
|
||||
| Release tooling | [Maintainer release procedure](release.md) |
|
||||
| Performance maintenance | [Internal overview](internal/overview.md), existing benchmarks, and [deferred efficiency work](roadmap/deferred.md#possible-further-efficiency-work) |
|
||||
| Licensing or incorporated material | [Project grant](../COPYRIGHT), [license texts](../LICENSE), and [attribution](oss/attribution.md) |
|
||||
|
||||
@@ -1,915 +0,0 @@
|
||||
# Dependency upgrades, discovery, and smoke testing implementation plan
|
||||
|
||||
Status: Stages 1–18 complete.
|
||||
|
||||
## Authority and baseline
|
||||
|
||||
Implement the [accepted upgrade roadmap](upgrades.md) by following Stage 1
|
||||
through Stage 18 in order. Each stage is sized for one Luna coding prompt.
|
||||
The roadmap owns the selected target and intent; this plan specifies the work
|
||||
to reach it. The [architecture policy](../policy/architecture.md), accepted
|
||||
[ADRs](../adr/), and applicable policies remain authoritative.
|
||||
|
||||
The planning baseline is `38b49f29fb3a8f22ff02528d2a0ed413e5c6ab1e`.
|
||||
It has ReleaseKit v0.4.0 at the root, ConfigKit v0.7.1 in both integrations,
|
||||
and PromptKit v0.15.0. The target releases are ReleaseKit v0.4.2, ConfigKit
|
||||
v0.8.0, and PromptKit v0.16.0, at the exact upstream revisions recorded in the
|
||||
roadmap. The Go consumer minimum remains 1.26.0 and the selected maintainer
|
||||
toolchain remains 1.26.8.
|
||||
|
||||
This plan replaces the completed request-validation implementation plan, which
|
||||
remains available in [Git history](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/38b49f29fb3a8f22ff02528d2a0ed413e5c6ab1e/docs/roadmap/implementation.md).
|
||||
That feature is baseline behavior to migrate, not an unfinished assignment.
|
||||
|
||||
## Working rules
|
||||
|
||||
- Start with [development guidance](../development.md) and its policy reading
|
||||
guide. Prefer the knowledge graph for code discovery; use current source when
|
||||
graph evidence is incomplete.
|
||||
- Deliver the complete roadmap in one coordinated sprint. Intermediate commits
|
||||
need not build; stages identify useful units of work, not compatibility
|
||||
layers or independently releasable intermediate APIs. Preserve established
|
||||
contracts except where the accepted roadmap deliberately changes them.
|
||||
- Keep core independent of optional libraries. ckconfig owns ConfigKit
|
||||
adaptation; ckprompt may use ckconfig. Do not introduce the reverse
|
||||
dependency or expose Cobra/pflag types.
|
||||
- Use released upstream modules. Adjacent checkouts are review evidence, not
|
||||
committed replacements or sources of internal imports.
|
||||
- Implement straightforwardly in the owning packages. Private file names below
|
||||
are suggested organization; do not create frameworks, exported abstractions,
|
||||
or additional internal layers merely to match the plan.
|
||||
- Retain application ownership of factories, callbacks, configuration policies,
|
||||
credentials, defaults, and returned objects. No arbitrary deep copies,
|
||||
pointer repair, new caching policy, or concurrency guarantee is selected.
|
||||
- Apply the [upstream testing ownership boundary](../policy/testing.md#upstream-integration-testing-ownership).
|
||||
Test CommandKit's forwarding, coordination, presentation, and outcomes.
|
||||
Do not independently verify upstream composition, namespace resolution,
|
||||
prompt preparation, provider diagnostics, or inference correctness.
|
||||
- Use small test doubles at external boundaries and real upstream fixtures
|
||||
where useful. This does not require production mocking interfaces. Default
|
||||
tests remain offline and credential-independent.
|
||||
- Format changed Go files. Run focused checks appropriate to each stage;
|
||||
broad integration checks follow when the affected migration is coherent.
|
||||
Record actual completion evidence and commit each completed nonempty unit
|
||||
under the [agent policy](../policy/agents.md#git-commit-policy).
|
||||
- This sprint does not publish releases, create tags, choose future module
|
||||
release numbers, or add release gates. Release preparation remains governed
|
||||
by the existing release procedure.
|
||||
|
||||
## Implementation choices used by the stages
|
||||
|
||||
These details complete the accepted target for coding. They define this
|
||||
implementation's shape, not new cross-library domain policy.
|
||||
|
||||
### Catalog requests and callbacks
|
||||
|
||||
Introduce two small public request types in ckprompt:
|
||||
|
||||
```go
|
||||
type CatalogRequest struct {
|
||||
Selection ckconfig.Selection
|
||||
Filter promptkit.NamespaceFilter
|
||||
}
|
||||
|
||||
type NamespaceRequest struct {
|
||||
Selection ckconfig.Selection
|
||||
Kinds []promptkit.ResourceKind
|
||||
}
|
||||
```
|
||||
|
||||
Use `CatalogRequest` for `Suite.ListPrompts`, `Suite.ListProfiles`, and the new
|
||||
`Suite.ListBackends`. Use `NamespaceRequest` for the new `Suite.ListNamespaces`.
|
||||
`Suite.ListCollections` takes the existing `ckconfig.Selection` directly:
|
||||
its native operation has no resource filter to wrap.
|
||||
|
||||
All callbacks retain their existing context, typed input `I`, shared state `S`,
|
||||
and `commandkit.Acquired[T]` plus error pattern. Result types are:
|
||||
|
||||
| Callback | Acquired value |
|
||||
| --- | --- |
|
||||
| `ListPrompts` | `[]promptkit.PromptIdentity` |
|
||||
| `ListProfiles`, `ListBackends`, `ListNamespaces` | `[]string` |
|
||||
| `ListCollections` | `[]promptkit.CollectionInfo` |
|
||||
|
||||
`FromEngine` supplies the original inspection operations and all five catalogs:
|
||||
eight independently optional callbacks in total. Request validation and smoke
|
||||
testing are added explicitly by their respective helpers, giving ten possible
|
||||
suite capabilities. No universal operation registry is needed.
|
||||
|
||||
For the filterable catalogs, forward the native `Filter` to the corresponding
|
||||
listing method, including the zero-valued `NamespaceDefault`. For namespaces,
|
||||
forward `Kinds...` without implementing union, membership, or sorting locally.
|
||||
|
||||
### Report shapes
|
||||
|
||||
Keep existing report organization except for the accepted changes and additions:
|
||||
|
||||
- `config sources` retains its scope/source structure. Add `destination` as the
|
||||
native canonical JSON Pointer, `placement` as `mapping` or `sequence_item`,
|
||||
`destination_unresolved` as a boolean, and nullable `version_check`.
|
||||
Its nested fields are `path`, `version` (`present`, `null`, `text`),
|
||||
nullable `location` (`source_id`, `line`, `column`), and `redacted`.
|
||||
Preserve false, empty, and null native values. Text exposes the same evidence.
|
||||
- Prompt inspection, requirements inspection, and prompt catalog entries use
|
||||
`version` as `*string` with YAML `omitempty`. Nil omits the field; a pointer
|
||||
to `""` emits an empty string. Text inspection omits an absent version and
|
||||
quotes a present empty version.
|
||||
- Profiles, backends, and namespaces are bare YAML string sequences.
|
||||
Prompt entries remain `{id, version?}`; collection entries are
|
||||
`{namespace, version?}`, projecting native `ReleaseVersion` into `version`.
|
||||
Preserve provider order and exact identities. Nil/empty inventories emit `[]`.
|
||||
- A single smoke test emits the native report projected into a private YAML DTO.
|
||||
Use snake_case native field names, with descriptive `PromptVersion` rendered
|
||||
as optional `version` inside its target under the accepted version convention.
|
||||
Include all report fields and their native nested evidence; do not serialize
|
||||
the request or engine.
|
||||
- Aggregate smoke output is one YAML mapping with `tests`, an ordered sequence
|
||||
of entries containing `kind`, nullable `report`, and optional `error` text.
|
||||
The outer `kind` identifies the selected concrete operation; a non-nil native
|
||||
report retains its own returned `kind`. No extra readiness summary is added.
|
||||
Include attempted operations only after cancellation stops the sequence.
|
||||
|
||||
For smoke DTOs, preserve scalar zero values, nullable tri-state observations,
|
||||
settings-presence booleans, and token counts' `value`/`known`/`complete` fields.
|
||||
Render `time.Time` through the existing YAML encoder and durations as Go
|
||||
duration strings, preserving nil duration pointers as null. Render an empty
|
||||
attempt list as `[]`. Keep native effective settings and `extra_params` data;
|
||||
do not introduce a second sensitivity classification.
|
||||
|
||||
The aggregate `error` is presentation text, not a serialized Go error object or
|
||||
a replacement for the returned error's identity. For a single test, deliver any
|
||||
returned report and use existing diagnostics for the operation error. If a
|
||||
callback returns neither a report nor an error, report an operational provider
|
||||
contract failure rather than dereferencing nil or fabricating native evidence.
|
||||
|
||||
### Smoke invocation and delivery
|
||||
|
||||
Use the roadmap's `SmokeRequest` envelope, optional `Suite.SmokeTest` callback,
|
||||
and `WithSmokeTesting(suite, load, build)` helper. The callback represents one
|
||||
concrete operation, for both single and aggregate invocations.
|
||||
|
||||
The helper's builder runs before its engine loader, as in request validation.
|
||||
A nil builder forwards the seed; a supplied builder's complete native request
|
||||
is authoritative. Return the native report and error together. Retain acquired
|
||||
diagnostics. A nil loader attaches an installation issue even if the capability
|
||||
is subsequently removed, following the existing suite convention.
|
||||
|
||||
For `all`, the handler prepares input/checks/state once, then invokes the
|
||||
captured concrete-test callback sequentially. The per-test seed varies `Kind`;
|
||||
the optional builder sees each concrete seed. Keep the native request returned
|
||||
by the builder authoritative, including its application-owned choices.
|
||||
Engine reuse can occur through application shared state; do not add an
|
||||
integration cache or make loader-call counts a public requirement.
|
||||
|
||||
Collect reports and errors, then encode and deliver one report. Continue after
|
||||
individual operation errors unless canceled. On cancellation, stop scheduling,
|
||||
retain acquired results, and return operational cancellation with supported
|
||||
identity. Never add synthetic results for operations that did not run.
|
||||
|
||||
Join multiple operation causes using `errors.Join`, preserving `errors.Is` and
|
||||
`errors.As` access. Keep diagnostic records in returned order and let the
|
||||
existing invocation coordinator deliver them. Do not add duplicate error
|
||||
diagnostics on top of the coordinator's existing presentation.
|
||||
|
||||
Smoke report delivery must attempt available output even when the context has
|
||||
become canceled; the success-only `writePromptReport` cannot be used unchanged.
|
||||
Retain operation causes if encoding or delivery also fails. Report writer
|
||||
errors/short writes through `DeliveryError` and `ReportIncomplete` under the
|
||||
existing outcome contract. Do not change unrelated inspection delivery or add
|
||||
rollback/blocking-writer interruption promises.
|
||||
|
||||
A nil native error and successful delivery yield success for a single operation.
|
||||
For `all`, every selected operation must return without error and delivery must
|
||||
succeed. Cancellation and operational/delivery errors yield the established
|
||||
operational classification. Native false, unknown, unsupported, or incomplete
|
||||
observations do not independently change the outcome.
|
||||
|
||||
## Stage 1: Adopt ReleaseKit v0.4.2
|
||||
|
||||
**Primary files:** root `go.mod`/`go.sum`, root attribution,
|
||||
`docs/policy/release.md`, `docs/release.md`; existing launcher/tests for review.
|
||||
|
||||
1. Update the root tool requirement to released ReleaseKit v0.4.2 and obtain real
|
||||
checksums. Preserve the Go minimum and release toolchain.
|
||||
2. Update the root selected-component attribution and pinned notice/source links
|
||||
for the adopted revision. Compare applicable upstream notices; preserve
|
||||
unchanged license texts.
|
||||
3. Update factual tool-version and commit-pinned reference links in the release
|
||||
policy and runbook. Leave their publication contracts unchanged.
|
||||
4. Review `scripts/release` and its three profiles against the adopted public
|
||||
contracts. Keep the existing launcher, bootstrap storage, shared files, and
|
||||
source-only publication behavior unless a concrete compatibility defect
|
||||
requires correction. Do not add Woodpecker workflows or hosted deliverables.
|
||||
|
||||
**Validation:** Run existing launcher argument/delegation tests and the root
|
||||
default suite as appropriate. Assert CommandKit's invocation contract, not
|
||||
ReleaseKit's publication algorithms. Do not invoke publication.
|
||||
|
||||
**Complete when:** the root selects v0.4.2 and its factual notices/references
|
||||
agree, without new release machinery.
|
||||
|
||||
**Completed:** 2026-10-17. The root selects ReleaseKit v0.4.2 with Go-generated
|
||||
module checksums; attribution, release policy, and pinned runbook references
|
||||
match upstream commit `7d033c5ca1c95aec7db9690293638a0a1ac887b8`. Upstream
|
||||
`COPYRIGHT` and `LICENSE` were unchanged. Launcher argument/delegation tests and
|
||||
`GOWORK=off GOTOOLCHAIN=local go test ./...` passed. Profiles and the existing
|
||||
source-only launcher contract required no changes.
|
||||
|
||||
## Stage 2: Adopt ConfigKit v0.8.0 in both integrations
|
||||
|
||||
**Primary files:** integration manifests/sums, ConfigKit adapters and affected
|
||||
literals, both integrations' attribution.
|
||||
|
||||
1. Set both direct ConfigKit requirements to v0.8.0. Prepare actual sums under
|
||||
each module's own graph; preserve released PromptKit v0.15.0 until Stage 4.
|
||||
2. Review ConfigKit-facing code for unkeyed `SourceInfo` literals, equality,
|
||||
or map-key use affected by its new noncomparable `Path` field. Adapt actual
|
||||
uses without inventing an equality abstraction or changing source identity.
|
||||
3. Preserve native `Result`/inspection records through existing providers.
|
||||
Destination, placement, version checks, and sequence composition remain
|
||||
available through application loader options, not new generic CLI policies.
|
||||
4. Update ConfigKit attribution for both modules with the reviewed v0.8.0
|
||||
revision and applicable notices.
|
||||
|
||||
**Validation:** Run ckconfig's existing suite and build affected ckprompt code
|
||||
where its still-old PromptKit API permits. Review selected dependency versions.
|
||||
Do not add native placement/composition/version-check feature matrices.
|
||||
|
||||
**Complete when:** both integration manifests name v0.8.0 and the existing
|
||||
ConfigKit boundary remains coherent.
|
||||
|
||||
**Completed:** 2026-10-17. Both integration modules select ConfigKit v0.8.0
|
||||
with generated checksums, and ckprompt continues to select PromptKit v0.15.0.
|
||||
The source record usage review found no unkeyed literals, equality, or map-key
|
||||
uses requiring changes; the existing adapters preserve native records. The
|
||||
ckconfig suite passed and ckprompt packages and tests compiled with no tests run.
|
||||
Upstream `COPYRIGHT` and `LICENSE` were unchanged.
|
||||
|
||||
## Stage 3: Extend configuration source reports
|
||||
|
||||
**Primary files:** `integrations/ckconfig/sources.go` and focused source-report
|
||||
tests, reusing the existing native source-reader boundary.
|
||||
|
||||
1. Extend `sourceDTO` and text rendering with the fields and nested shape
|
||||
specified above. Convert native paths with `Path.String()`, and map the two
|
||||
native placement modes to their corresponding names.
|
||||
2. Project `SourceVersionCheck` only from returned native records, preserving
|
||||
native optional locations and sanitized version values. Do not reread files,
|
||||
rerun checkers, or derive acceptance/validation judgments from the evidence.
|
||||
3. Keep existing source fields, inventory order, scope behavior, source-reader
|
||||
capability handling, and delivery paths. Keep `config explain` and compact
|
||||
diagnostic legends focused on their existing responsibilities.
|
||||
4. Add focused report tests using controlled returned `SourceInfo` values:
|
||||
root/non-root destination, both placement modes, unresolved false/true,
|
||||
absent/present version evidence, nil/non-nil location, and meaningful
|
||||
present/null/text/redacted values. These test projection, not native
|
||||
sanitization or source resolution.
|
||||
|
||||
**Validation:** Decode YAML semantically and inspect text fields without key-order
|
||||
or whole-output snapshots. Run focused source tests and the ckconfig suite.
|
||||
|
||||
**Complete when:** both source formats expose the agreed native evidence with no
|
||||
parallel ConfigKit behavior.
|
||||
|
||||
**Completed:** 2026-10-17. YAML and text `config sources` now project native
|
||||
destination, placement, unresolved-destination, and source-version evidence.
|
||||
Focused semantic-YAML and text-field assertions cover the requested presence,
|
||||
null, redaction, location, and placement cases. Focused source tests and the
|
||||
complete ckconfig suite passed.
|
||||
|
||||
## Stage 4: Migrate ckprompt production code to PromptKit v0.16.0
|
||||
|
||||
**Primary files:** ckprompt manifest/sums, `adapters.go`, `command.go`,
|
||||
`validation.go`/`validation_command.go`, affected report DTOs, attribution.
|
||||
|
||||
1. Select released PromptKit v0.16.0, retaining direct ConfigKit v0.8.0.
|
||||
Update the PromptKit attribution to its reviewed release revision.
|
||||
2. Remove `PromptRequest.Version` and the prompt-only `--version` bindings from
|
||||
named prompt inspection, requirements show, and requirements validate.
|
||||
Remove corresponding fields/assignments in native request seeds.
|
||||
3. Call `InspectPrompt(ctx, request.ID)` and forward the revised native
|
||||
requirements/run request types. Retain configuration selection, context,
|
||||
selectors, builders, diagnostics, and native error identity.
|
||||
4. Update named/requirements/catalog report DTOs to optional `version` fields
|
||||
and adapt text presentation, using the report choices above. Do not use
|
||||
label values for selection or convert nil to empty strings.
|
||||
5. Review remaining production uses of removed catalogs, implicit profiles,
|
||||
or old upstream fields and migrate those actual uses.
|
||||
|
||||
**Validation:** Build the production ckprompt package against the selected
|
||||
released dependencies. Test fixtures and consumer code are migrated in Stage 5;
|
||||
do not create compatibility shims solely to make this intermediate stage green.
|
||||
|
||||
**Complete when:** production adaptation uses the new ID-only API and optional
|
||||
version representation.
|
||||
|
||||
**Completed:** 2026-10-17. ckprompt selects PromptKit v0.16.0 with generated
|
||||
checksums and attribution pinned to release revision
|
||||
`9178f3eb91c2b2138caa78e22be93c048b4bc992`; upstream `COPYRIGHT` and `LICENSE`
|
||||
are unchanged. Production adapters use ID-only prompt inspection, the three
|
||||
prompt-version selectors and their seed assignments are removed, and inspection
|
||||
and catalog reports retain optional version values. `GOTOOLCHAIN=local go build
|
||||
./...` passed for the ckprompt module. Fixtures and consumer code remain for
|
||||
Stage 5.
|
||||
|
||||
## Stage 5: Migrate existing fixtures, tests, and composed loading
|
||||
|
||||
**Primary files:** ckprompt tests, `examples/composed/main.go` and fixtures,
|
||||
their tests.
|
||||
|
||||
1. Convert native version-keyed prompt mappings to one direct definition per ID.
|
||||
Retain useful versions as optional metadata. For tests that genuinely need
|
||||
multiple resources, use distinct IDs rather than rebuilding version lookup.
|
||||
2. Make needed backend/profile definitions explicit. Remove expectations about
|
||||
installed catalogs and former standard/auto profiles. Use small synthetic
|
||||
inputs; do not adopt external catalogs just to restore old fixtures.
|
||||
3. Update the composed engine mapping to the direct prompt-definition format.
|
||||
Keep its existing campaign selector, request builder, config scopes, state
|
||||
cache, and session command. Its application `prompt_version` data may remain
|
||||
descriptive input, used to populate native `version`, without selecting a
|
||||
definition. Do not require a campaign-loader redesign or collections demo.
|
||||
4. Update tests for ID-only requests and removed flags. Add focused projection
|
||||
evidence for absent, present-empty, and nonempty `version` across the
|
||||
affected reports; verify absent keys rather than YAML null.
|
||||
5. Replace obsolete upstream-domain assertions encountered in these affected
|
||||
tests with CommandKit boundary assertions. Preserve meaningful adapter
|
||||
protection; do not expand this into a repository-wide test rewrite.
|
||||
|
||||
**Validation:** Run ckprompt's existing suite, including the composed consumer.
|
||||
Exercise a named inspection and requirements show/validate through the updated
|
||||
fixture. Native errors may be fixtures; assertions protect forwarding, report
|
||||
presentation, and error identity, not upstream detection algorithms.
|
||||
|
||||
**Complete when:** the previous delivered feature set works through v0.16.0
|
||||
without compatibility scaffolding or implicit catalogs.
|
||||
|
||||
**Completed:** 2026-10-17. PromptKit fixtures now use direct prompt definitions,
|
||||
explicit profiles/backends, and distinct IDs for multiple prompts. ckprompt
|
||||
tests use the ID-only inspection and validation requests and verify that the
|
||||
removed version flags are unavailable. Semantic YAML tests cover absent,
|
||||
present-empty, and nonempty versions in prompt inspection, requirements, and
|
||||
catalog reports; PromptKit-specific requirement classification assertions were
|
||||
removed from the affected adapter/consumer tests. The ckprompt suite passed,
|
||||
and composed named inspection, requirements show, and request validation all
|
||||
ran successfully against the migrated fixture.
|
||||
|
||||
## Stage 6: Extend catalog contracts and engine adapters
|
||||
|
||||
**Primary files:** ckprompt public request declarations, `Suite`, `FromEngine`,
|
||||
and focused adapter tests; `catalog.go` is a suitable private organization.
|
||||
|
||||
1. Add `CatalogRequest` and `NamespaceRequest` as specified above, revise
|
||||
existing prompt/profile callback signatures, and add backend, namespace,
|
||||
and collection callbacks.
|
||||
2. Extend `FromEngine` to forward the correct request fields to each native
|
||||
API. Continue using `loadPromptKitEngine` for selection and diagnostics.
|
||||
Forward native filters/kinds and returned results/errors directly.
|
||||
3. Preserve native engine views from loaders. Do not reconstruct views,
|
||||
qualify IDs, filter lists, calculate namespace unions, or sort memberships.
|
||||
4. Migrate existing custom catalog callbacks and call sites to the new request
|
||||
signatures. Add GoDoc for request fields, native defaults, and independent
|
||||
optionality. Keep nil-loader issue retention.
|
||||
5. Use focused adapter evidence: direct native-call parity on a small fixture
|
||||
can establish forwarding. Test selector/state/context propagation and loader
|
||||
diagnostics/error identity through existing seams without creating a
|
||||
parallel namespace semantics matrix.
|
||||
|
||||
**Validation:** Build ckprompt and run focused adapter tests. Current command
|
||||
call sites may need their mechanical signature adaptation here, with new
|
||||
operator syntax delivered in Stage 7.
|
||||
|
||||
**Complete when:** all five catalogs are available through narrow native adapters
|
||||
and public callback shapes are coherent.
|
||||
|
||||
**Completed:** 2026-10-17. Added `CatalogRequest` and `NamespaceRequest`,
|
||||
updated the prompt/profile callbacks, and exposed backend, namespace, and
|
||||
collection callbacks. `FromEngine` forwards selections and native filters or
|
||||
kinds to all five PromptKit catalog methods, preserving returned values,
|
||||
diagnostics, and errors. Existing catalog callers now send the default request.
|
||||
Focused adapter tests compare each callback with its native method and cover
|
||||
invocation values, loader diagnostics/error identity, and cancellation. The
|
||||
focused adapter/catalog tests and `GOTOOLCHAIN=local go build ./...` passed.
|
||||
|
||||
## Stage 7: Add catalog command bindings and YAML output
|
||||
|
||||
**Primary files:** ckprompt catalog command construction, input preparation,
|
||||
encoders, and basic assembled command tests.
|
||||
|
||||
1. Install each supplied catalog independently under `promptkit list`. Extend
|
||||
empty-suite detection and group assembly to include the new callbacks.
|
||||
Keep eager metadata/callback capture, root-family ownership, and retained
|
||||
assembly issues.
|
||||
2. For prompts/profiles/backends, add singleton `--namespace` and boolean
|
||||
`--all-namespaces`. Map omitted selection to `NamespaceDefault`, a supplied
|
||||
nonempty name to `NamespaceNamed`, an explicit empty name to
|
||||
`NamespaceUnqualified`, and true `--all-namespaces` to `NamespaceAll`.
|
||||
3. An active all-namespaces choice plus supplied namespace is a usage error
|
||||
before provider work. Explicit `--all-namespaces=false` does not select
|
||||
an override; combine it with namespace or retain native default normally.
|
||||
Preserve singleton last-occurrence and namespace presence behavior.
|
||||
4. For namespaces, add repeatable `--kind`, preserving supplied native
|
||||
`ResourceKind` values and forwarding the slice. Omission forwards the native
|
||||
union/default. For collections, add no namespace/kind membership filter.
|
||||
5. Retain common selection/selectors and YAML-only format preflight. Use normal
|
||||
string-to-native-type conversion; leave native name/kind validity to
|
||||
upstream APIs rather than duplicating domain validation.
|
||||
6. Add collection projection and reuse string-list rendering for profiles,
|
||||
backends, and namespaces. Follow the report shapes above, without local
|
||||
membership reinterpretation.
|
||||
|
||||
**Validation:** Add basic capturing-callback command tests for each new operation,
|
||||
filter conversion, and collection optional versions; run focused catalog tests.
|
||||
|
||||
**Complete when:** the accepted catalog CLI and reports invoke the intended
|
||||
callbacks through the standard binding system.
|
||||
|
||||
**Completed:** 2026-10-17. `promptkit list` now installs each supplied catalog
|
||||
independently. Prompts, profiles, and backends bind namespace and all-namespace
|
||||
filters; namespaces bind repeatable native resource kinds; collections use
|
||||
their unfiltered operation and project optional release versions. Added
|
||||
capturing-callback coverage for filter mapping, false and repeated values,
|
||||
collection YAML presence, and conflicting filter preflight. The full ckprompt
|
||||
suite and `GOTOOLCHAIN=local go build ./...` passed.
|
||||
|
||||
## Stage 8: Verify discovery availability, lifetimes, and outcomes
|
||||
|
||||
**Primary files:** focused ckprompt catalog command/adapter tests.
|
||||
|
||||
1. Cover absent and independently supplied catalogs, help without acquisition,
|
||||
retained wiring issues, and eager capture for new callbacks. Reuse existing
|
||||
registry coverage rather than reproducing its full collision matrix.
|
||||
2. Capture callback requests for native default/named/unqualified/all choices,
|
||||
explicit false, conflicting active choices, singleton repetition, and
|
||||
repeatable kinds. Assert original selection, selectors, context, and state.
|
||||
3. Cover missing required selectors, extra operands, unsupported format,
|
||||
unavailable flags, and input-check failures. Assert no provider work where
|
||||
required, without insisting on factories running after an earlier failure.
|
||||
4. Verify fresh parsed values across repeated invocations, nil-factory zero
|
||||
values, native inventory order/strings, empty sequences, optional versions,
|
||||
and representative error/cancellation/diagnostic/write outcomes.
|
||||
|
||||
**Validation:** Run focused discovery tests and the ckprompt suite. Use synthetic
|
||||
returned inventories for report/membership assertions; these tests do not
|
||||
establish native namespace algorithms.
|
||||
|
||||
**Complete when:** discovery exposes only declared capabilities and preserves
|
||||
the selected CommandKit invocation contracts.
|
||||
|
||||
**Completed:** 2026-10-17. Added independent installation and eager-capture
|
||||
coverage for backend, namespace, and collection callbacks, including help
|
||||
without provider acquisition and unavailable filters. Extended nil-loader issue
|
||||
retention checks to all catalogs. Existing prompt/profile coverage plus the new
|
||||
cases exercise default, named, unqualified, all, explicit-false, conflicting,
|
||||
singleton, and repeatable-kind requests; selection, selector, state, inventory,
|
||||
empty output, and delivery behavior remain protected. The full ckprompt suite
|
||||
passed.
|
||||
|
||||
## Stage 9: Add the optional smoke capability and native adapter
|
||||
|
||||
**Primary files:** ckprompt `Suite`, new `smoke.go` or equivalent, adapter tests.
|
||||
|
||||
1. Add `SmokeRequest` and `Suite.SmokeTest` with the roadmap's signatures.
|
||||
Ensure the acquisition value can retain a report with an error.
|
||||
2. Add `WithSmokeTesting` preserving the rest of the suite and retained issues.
|
||||
Capture loader/builder references consistently with existing helpers.
|
||||
3. Apply the builder-before-loader/nil-builder rules above. Forward the complete
|
||||
resulting request to `Engine.SmokeTest`, preserving native report and error.
|
||||
Check cancellation before further acquisition, but do not discard an
|
||||
error-bearing report through a success-only post-call path.
|
||||
4. Retain loader diagnostics and nil-engine/nil-loader handling from existing
|
||||
conventions. Do not call requirements validation, resolve native targets,
|
||||
implement retries, or derive success from report findings.
|
||||
5. Add focused helper tests for nil builder, authoritative builder output,
|
||||
typed input/state/context, builder failure before loading, retained wiring
|
||||
issues, and report/error propagation. Use a small native engine with an
|
||||
injected public client where the actual adapter needs exercising.
|
||||
Observe the forwarded request/effective native call; do not test the private
|
||||
diagnostic case or provider acceptance behavior.
|
||||
|
||||
**Validation:** Run focused smoke adapter tests and existing adapter suites.
|
||||
No live provider or real credentials are needed.
|
||||
|
||||
**Complete when:** one concrete smoke request reaches the native API through an
|
||||
independently optional, composable adapter.
|
||||
|
||||
**Completed:** 2026-10-17. Added `SmokeRequest`, the independently optional
|
||||
`Suite.SmokeTest` callback, and `WithSmokeTesting`. The helper preserves other
|
||||
suite capabilities and issues, invokes an optional builder before loading,
|
||||
forwards its complete request unchanged, retains loader diagnostics, checks
|
||||
cancellation before acquisition, and returns the native report and error
|
||||
without filtering either. Focused tests cover nil-builder forwarding,
|
||||
authoritative builder output, context/input/state, builder failure before load,
|
||||
nil-loader issue retention, cancellation, loader diagnostics, and native error
|
||||
identity. PromptKit's injected-client fixture returned a report with nil error
|
||||
when the client returned an error, so public fixture evidence for a
|
||||
report-plus-error pair is unavailable; the adapter returns both native values
|
||||
directly. Focused adapter tests and `GOTOOLCHAIN=local go build ./...` passed.
|
||||
|
||||
## Stage 10: Implement faithful smoke YAML projection
|
||||
|
||||
**Primary files:** private `smoke_report.go` and focused projection tests.
|
||||
|
||||
1. Define small private DTOs for the report fields described above. Map every
|
||||
exported v0.16.0 `SmokeTestReport` field and its nested public data explicitly;
|
||||
avoid reflect-based conversion, a JSON round trip, and raw request dumping.
|
||||
2. Include target IDs, optional target version, selection source, endpoint,
|
||||
effective settings and settings-presence fields; native facts; timing;
|
||||
attempts; presence-aware token usage; reason/failure values.
|
||||
3. Preserve nullable booleans and durations, explicit false/zero, empty optional
|
||||
version strings, unknown token metadata, and `extra_params` values. Use
|
||||
existing YAML encoding and native types where their representation is
|
||||
suitable; private DTOs provide naming/formatting only.
|
||||
4. Add the aggregate envelope with ordered entries and optional error text.
|
||||
Keep native report data unchanged if its kind differs from the selected
|
||||
outer kind due to application request construction.
|
||||
5. Add synthetic-report tests protecting omission, presence, nested values,
|
||||
durations, empty attempts, all field categories, and report-plus-error
|
||||
entries. Decode YAML semantically; do not freeze key order or timing values
|
||||
produced by a live operation.
|
||||
|
||||
**Validation:** Run focused projection tests. Verify no native request/API key is
|
||||
serialized; do not introduce filtering of returned `ExtraParams`.
|
||||
|
||||
**Complete when:** native smoke evidence and aggregate entries have a complete,
|
||||
neutral YAML representation.
|
||||
|
||||
**Completed:** 2026-10-17. Added private report and aggregate DTOs that explicitly
|
||||
project every exported PromptKit smoke report field, including target settings,
|
||||
presence, nullable facts, timing, attempts, usage, and failure details. Durations
|
||||
render as Go duration strings, empty attempts as `[]`, and optional prompt
|
||||
versions preserve empty values while omitting absent ones. Aggregate entries
|
||||
retain order, nullable reports, native report kinds, and optional error text.
|
||||
Semantic YAML tests cover all field categories, nested values, omission,
|
||||
presence, direct-request/key exclusion, and report-plus-error entries. Focused
|
||||
projection tests and `GOTOOLCHAIN=local go build ./...` passed.
|
||||
|
||||
## Stage 11: Bind and execute single smoke tests
|
||||
|
||||
**Primary files:** private `smoke_command.go`, `Command` assembly, basic command
|
||||
tests.
|
||||
|
||||
1. Add one runnable `test` command with one required `KIND` operand. Install it
|
||||
only when `Suite.SmokeTest` is supplied. Include it in empty-suite detection,
|
||||
captured suite metadata, and the promptkit family help. Do not add an
|
||||
application-owned command beneath the family.
|
||||
2. Bind every scalar/request field in the roadmap table using the existing
|
||||
typed binding APIs. Use standard integer/floating-point conversion and
|
||||
presence-aware setters; do not add a second parser or range validator.
|
||||
3. Construct `Execution` only when at least one execution flag is supplied.
|
||||
Omitted pointer fields stay nil; supplied zero or empty values remain
|
||||
present. Bind `TransientRetries` independently. Use native enum strings for
|
||||
individual kinds. Recognize `all` for Stage 12 expansion before constructing
|
||||
a native request; never pass that token as a native kind.
|
||||
4. Apply common selection, captured selectors, YAML-only preflight, input checks,
|
||||
cancellation checks, and state initialization through existing patterns.
|
||||
For a single operation, invoke the captured callback once.
|
||||
5. Use the Stage 10 projection and a focused delivery helper to deliver any
|
||||
acquired report, even when accompanied by an error/cancellation. Return
|
||||
diagnostics and outcome fields under the delivery rules above.
|
||||
6. A nil report with a non-nil error remains an ordinary operation error; emit
|
||||
no fabricated single native report. Neither nil result with nil error nor
|
||||
encoding/delivery failure may become success.
|
||||
|
||||
**Validation:** Add basic assembled single-command tests with capturing callbacks,
|
||||
including successful native findings, report-plus-error, and a short writer.
|
||||
Run focused smoke command tests.
|
||||
|
||||
**Complete when:** all single-kind bindings invoke the optional provider and
|
||||
faithfully report its result and errors.
|
||||
|
||||
**Completed:** 2026-10-17. Added the optional `promptkit test KIND` command with
|
||||
all accepted scalar bindings, YAML preflight, prepared input/state, and native
|
||||
report delivery. Explicit zero/empty overrides remain present, omitted
|
||||
execution/retry overrides remain absent, and `all` is rejected before native
|
||||
request construction pending aggregate coordination. Assembled tests cover
|
||||
flag forwarding, findings, report-plus-error, nil-result contract failure, and
|
||||
short-write delivery. `GOTOOLCHAIN=local go test ./... -run
|
||||
'TestSmokeTestCommand' -count=1` passed in ckprompt.
|
||||
|
||||
## Stage 12: Coordinate aggregate smoke execution
|
||||
|
||||
**Primary files:** smoke command runner, aggregate projection/outcome helpers,
|
||||
focused aggregate tests.
|
||||
|
||||
1. Expand `all` to reachability, authentication, text inference, and structured
|
||||
output in the accepted order. This is one invocation of the standard command,
|
||||
not four dispatcher executions.
|
||||
2. Prepare/check typed input once and create shared state once when reached.
|
||||
Build a concrete seed for each selected kind, invoking `Suite.SmokeTest` and
|
||||
therefore the optional builder separately. Keep `all` out of native requests.
|
||||
3. Preserve the common CLI seed across operations. Construct each per-test seed
|
||||
from the bound scalar values with its own execution override struct and
|
||||
fresh pointers for supplied optional controls, including transient retries.
|
||||
This keeps builder edits to one seed from changing later parsed defaults.
|
||||
Do not deep-copy application input/state or arbitrary builder objects.
|
||||
4. Append each acquired report, error, and diagnostics in operation order.
|
||||
Continue after an individual error. If the caller context is canceled,
|
||||
stop scheduling further operations, retain results, and preserve cancellation
|
||||
identity alongside already returned causes.
|
||||
5. Join errors with the standard library and encode one aggregate document.
|
||||
Retain each associated error in its entry; preserve supported error identity
|
||||
in the invocation cause. Do not promote false/unknown report facts to errors
|
||||
or add an overall readiness score.
|
||||
6. Deliver retained results despite cancellation, and preserve operation causes
|
||||
separately from report/diagnostic delivery failures. With no operation
|
||||
reached because preflight or initial cancellation failed, use the existing
|
||||
early-failure handling; do not invent native observations.
|
||||
7. Do not introduce loader memoization, parallel execution, custom retry loops,
|
||||
synthetic skipped-test statuses, or a second public aggregate callback.
|
||||
|
||||
**Validation:** Use controlled callback results to verify call order, distinct
|
||||
concrete seeds, shared input/state, continuation, partial errors, cancellation
|
||||
stopping, ordered output, and joined error identity. Run focused aggregate tests.
|
||||
|
||||
**Complete when:** `promptkit test all` provides the agreed coordination through
|
||||
the same concrete-operation callback as individual tests.
|
||||
|
||||
**Completed:** 2026-10-17. `all` now runs reachability, authentication, text
|
||||
inference, and structured output sequentially in one invocation, with one input
|
||||
check and shared state. Each callback receives a concrete kind and a fresh
|
||||
execution override with copied optional pointers. Reports and per-kind errors
|
||||
remain ordered; ordinary errors do not stop later tests, while cancellation
|
||||
stops scheduling and retains earlier results plus joined error identity.
|
||||
Aggregate tests verify shared lifetimes, seed isolation, continuation,
|
||||
projection order, and partial cancellation. Focused smoke/report tests and
|
||||
`GOTOOLCHAIN=local go build ./...` passed in ckprompt.
|
||||
|
||||
## Stage 13: Verify smoke bindings and invocation lifetimes
|
||||
|
||||
**Primary files:** focused smoke command/adapter tests, reusing existing writer
|
||||
and diagnostic doubles.
|
||||
|
||||
1. Cover native seed fields from all bound flag categories, pointer omission
|
||||
versus zero/empty, config/overlay presence, singleton last occurrence,
|
||||
selectors, and per-invocation reset. Verify targets may be combined and
|
||||
native semantic errors retain operational classification.
|
||||
2. Cover syntax/conversion/operand/format failures and input-check handling.
|
||||
Missing operands, malformed numeric syntax, unsupported format, and unknown
|
||||
flags remain usage failures; factories need not run after failed preflight.
|
||||
3. Cover a smoke-only suite, absent smoke with other capabilities, installation
|
||||
issues after narrowing, eager callback/selector capture, and help/version
|
||||
leaving factories/builders/loaders/providers uninvoked.
|
||||
4. Verify one input preparation/state lifetime for `all`, nil-factory zero values,
|
||||
and builders seeing concrete kinds. Check earlier reports remain available
|
||||
when later operations fail or cancellation stops the sequence.
|
||||
|
||||
**Validation:** Run focused smoke binding/lifecycle tests and the ckprompt suite.
|
||||
Assert callback requests and invocation state, not native target resolution.
|
||||
|
||||
**Complete when:** the new command obeys selected grammar, presence, capture,
|
||||
availability, and lifetime contracts.
|
||||
|
||||
**Completed:** 2026-10-17. Added smoke lifecycle coverage for repeated scalar
|
||||
values, explicit zero and empty overrides, config/overlay presence, target
|
||||
combinations, and reset across invocations. Usage and input-check failures
|
||||
stop before state/provider work. Availability, captured callbacks/selectors,
|
||||
retained installation issues, and help/version inactivity are verified, along
|
||||
with nil-factory zero values and concrete kinds reaching builders. Focused smoke
|
||||
tests and the full ckprompt suite passed with `GOTOOLCHAIN=local`.
|
||||
|
||||
## Stage 14: Verify smoke outcomes and report delivery
|
||||
|
||||
**Primary files:** focused smoke outcome tests and existing writer/diagnostic
|
||||
fixtures.
|
||||
|
||||
1. Cover report-only success, false/unknown native observations with nil errors,
|
||||
report-plus-error, error-only, and nil-report contract failure. Confirm
|
||||
supported native and cancellation error identity survives aggregation.
|
||||
2. Cover cancellation before acquisition, after partial results, and between
|
||||
tests. Verify available reports are delivered and no later callback starts
|
||||
after cancellation is observed. Do not assert interruption of an already
|
||||
running caller callback or blocking writer.
|
||||
3. Cover encoding failures, report writer errors/short writes, diagnostic writer
|
||||
failures, and operation plus delivery errors. Retain `Cause`, `DeliveryError`,
|
||||
`ReportIncomplete`, and diagnostic delivery state as applicable. Reuse shared
|
||||
coordinator coverage for unchanged mechanics.
|
||||
4. Verify aggregate error entries correspond to the selected kinds and that
|
||||
multiple causes remain discoverable through `errors.Is`/`errors.As`.
|
||||
Diagnostic records keep their order and existing presentation responsibility.
|
||||
|
||||
**Validation:** Run focused smoke outcome tests and the full ckprompt suite.
|
||||
Each assertion must detect CommandKit presentation/coordination/delivery defects;
|
||||
provider smoke-test correctness belongs upstream.
|
||||
|
||||
**Complete when:** realistic smoke failures preserve their reports, native error
|
||||
identity, and delivery outcomes without a parallel diagnostic test suite.
|
||||
|
||||
**Completed:** 2026-10-17. Added outcome and delivery coverage for false/unknown
|
||||
observations, error-only and report-plus-error results, cancellation before and
|
||||
after acquisition, and retained aggregate results across cancellation. Tests
|
||||
also cover error and short report writes, encoding failure, diagnostic writer
|
||||
failure, operation plus delivery errors, ordered diagnostics, and joined
|
||||
`errors.Is`/`errors.As` identity. The full ckprompt suite passed with
|
||||
`GOTOOLCHAIN=local go test -count=1 ./...`.
|
||||
|
||||
## Stage 15: Demonstrate the new interfaces in the maintained consumer
|
||||
|
||||
**Primary files:** `integrations/ckprompt/examples/composed`, its README,
|
||||
synthetic configuration, and tests.
|
||||
|
||||
1. Extend the already migrated small consumer with the updated catalog callback
|
||||
API and explicit `WithSmokeTesting` wiring. Keep configuration scopes,
|
||||
existing requirements operations, campaign selection/cache, and session
|
||||
restore intact.
|
||||
2. Demonstrate backend/namespace/collection inventory and native namespace
|
||||
selection with the smallest useful configuration. A small synthetic
|
||||
collection may be used where useful; do not turn this into a collections
|
||||
composition tutorial or catalog migration project.
|
||||
3. Reuse shared state for engine/application acquisition where natural in this
|
||||
example. This is consumer behavior, not a new library caching requirement.
|
||||
4. Show nil-builder smoke wiring and explain optional typed request construction
|
||||
with a minimal snippet or focused test. A builder starts from the native
|
||||
seed and returns the chosen native request; no merge policy is added.
|
||||
5. Keep executed consumer checks offline. Use a small injected public
|
||||
`promptkit.LLMClient` double for inference when exercising the native helper;
|
||||
HTTP diagnostics can retain native unsupported/not-applicable observations
|
||||
when no probe is configured. Do not recreate native private expected
|
||||
responses or assert provider diagnostic correctness to make a demo green.
|
||||
6. Update documented commands for ID-only inspection, optional versions, discovery,
|
||||
and `test KIND`/`test all`. State any example-specific fixture/client setup
|
||||
plainly. Check metadata/requirements commands leave the injected client
|
||||
unused; explicit smoke commands use their selected native path.
|
||||
|
||||
**Validation:** Run composed-consumer tests and representative documented offline
|
||||
commands. Assert wiring and adaptation; an intentionally returned native
|
||||
diagnostic error is a documented example outcome, not a reason to redefine the
|
||||
native API's success criteria.
|
||||
|
||||
**Complete when:** the consumer demonstrates the new interfaces with little
|
||||
application plumbing and no model-service prerequisite.
|
||||
|
||||
**Completed:** 2026-10-17. The composed consumer now explicitly wires smoke
|
||||
testing with a nil request builder and exposes backend, namespace, collection,
|
||||
and namespace-filtered prompt discovery using one synthetic collection. Its
|
||||
shared invocation state carries an injected in-memory PromptKit client; tests
|
||||
confirm metadata and requirements leave it unused and smoke inference invokes
|
||||
it. The README documents ID-only inspection, optional version metadata, all
|
||||
catalogs, namespace filtering, typed builder customization, and single and
|
||||
aggregate smoke commands. The composed test suite passed, as did the documented
|
||||
offline discovery, inspection, requirements, and smoke commands. Native HTTP
|
||||
checks reported their configured unsupported/not-applicable observations.
|
||||
|
||||
## Stage 16: Reconcile dependency graphs and notices
|
||||
|
||||
**Primary files:** all three module manifests/sums and attribution documents.
|
||||
|
||||
1. Tidy/prepare dependencies with each module's own graph, using disposable
|
||||
module files and version-specific replacements only for unpublished
|
||||
CommandKit prerequisites where needed. Keep actual released upstream
|
||||
dependencies selected; do not use sibling upstream replacements. Transfer
|
||||
resulting dependency/sum edits back to the owning manifests without copying
|
||||
temporary replacements or changing the declared CommandKit prerequisites.
|
||||
2. Remove obsolete bundled backend-catalog requirements/sums and their notices
|
||||
when absent from the actual selected ckprompt graph. Retain other indirect
|
||||
requirements only as generated by the module tools.
|
||||
3. Update notices for every affected selected graph, including ReleaseKit
|
||||
selection through the local core. Do not blindly copy the root tool version
|
||||
into each integration's notices; verify its effective graph.
|
||||
4. Compare applicable copyright/license/notice material and pin upstream source
|
||||
links to the selected release revisions. Reuse unchanged standard license
|
||||
texts and preserve each module's existing local grant/material boundaries.
|
||||
5. Keep checked-in CommandKit prerequisite versions unchanged until actual
|
||||
release preparation selects and publishes successors. Do not fabricate
|
||||
future versions/checksums or use temporary replacements as release metadata.
|
||||
|
||||
**Validation:** Inspect independent `go list -m all` output and changed
|
||||
manifests/sums/notices. Confirm ConfigKit v0.8.0 in both integrations, PromptKit
|
||||
v0.16.0 in ckprompt, and no optional integration dependency in core.
|
||||
|
||||
**Complete when:** selected graphs, real checksums, removed catalog dependencies,
|
||||
and owning attribution entries agree.
|
||||
|
||||
**Completed:** 2026-10-17. Each module's own graph was tidied in disposable
|
||||
module files with only the unpublished CommandKit prerequisites replaced by
|
||||
local paths; checked-in prerequisite versions remain unchanged. Removed the two
|
||||
unused PromptKit backend-catalog requirements and sums, stale dependency sums,
|
||||
and their ckprompt attribution entries. All three notices now identify
|
||||
ReleaseKit v0.4.2 at its selected revision. Independent module lists confirm
|
||||
ConfigKit v0.8.0 in both integrations and PromptKit v0.16.0 in ckprompt; root
|
||||
contains no optional integration packages, and ckconfig contains no PromptKit
|
||||
packages. `go mod verify` passed for all three isolated graphs.
|
||||
|
||||
## Stage 17: Update canonical documentation and planning references
|
||||
|
||||
**Primary files:** GoDoc, module READMEs, `docs/cli.md`,
|
||||
`docs/consumers/guide.md`, `docs/development.md`, `docs/internal/overview.md`,
|
||||
and affected roadmap references.
|
||||
|
||||
1. Document the new callback requests, helper signatures, optional capabilities,
|
||||
native defaults, builder authority, error-bearing reports, and aggregate
|
||||
lifetimes in their owning GoDoc. Replace stale five/six-capability statements
|
||||
with the actual eight `FromEngine` callbacks and two explicit additions.
|
||||
2. Update the CLI reference for source evidence, ID-only selection, optional
|
||||
`version` omission, all five catalogs and filters, and the complete smoke
|
||||
vocabulary/bindings/report/outcome behavior. Include concrete aggregate
|
||||
envelope and native projection examples from the implemented renderer.
|
||||
3. Update consumer guidance, module summaries, internal inventory, and developer
|
||||
example commands. Scope existing metadata-only/no-provider-call descriptions
|
||||
to those operations; smoke is explicitly invoked native behavior.
|
||||
4. Link to the adopted upstream contracts for native configuration, selection,
|
||||
and diagnostic meanings. Use neutral field descriptions and avoid warnings,
|
||||
extra promises, or independent restatements of native algorithms.
|
||||
5. Reconcile the selected native smoke capability with `deferred.md`, preserving
|
||||
unrelated future proposals. Record its implemented boundary rather than
|
||||
treating all credential/runtime diagnostic ideas as delivered.
|
||||
6. Repair references from completed `promptkit-validation.md` and `remediation.md`
|
||||
to the previous implementation plan using its pinned historical link above.
|
||||
Those records remain historical; do not update their old API examples to
|
||||
describe the new feature.
|
||||
7. Keep accepted ADRs and policy boundaries intact unless a concrete conflict
|
||||
emerges. Do not create ADRs for report-key or private helper choices. Keep
|
||||
historical release notes unchanged; document release impact under the
|
||||
existing pre-v1 policy, with exact release versions chosen during preparation.
|
||||
|
||||
**Validation:** Check Go examples and CLI commands against actual declarations,
|
||||
local links/fragments, Markdown whitespace, stale selector/callback-count
|
||||
claims, and accidental consumer mandates. No new documentation checker code
|
||||
is required.
|
||||
|
||||
**Complete when:** current contracts describe implemented behavior under their
|
||||
canonical owners and historical references remain correctly scoped.
|
||||
|
||||
**Completed:** 2026-10-17. GoDoc, module summaries, the README, CLI reference,
|
||||
consumer guide, development commands, and internal inventory now describe the
|
||||
adopted source evidence, eight `FromEngine` callbacks, five catalogs, optional
|
||||
request validation, and native smoke behavior. Smoke examples and catalog
|
||||
commands were run against the maintained composed consumer; `go doc` confirmed
|
||||
the exported helper and request documentation. Local Markdown links/fragments,
|
||||
stale upstream revision references, and whitespace checks passed. Deferred-work
|
||||
scope and historical implementation-plan links were reconciled without changing
|
||||
historical API examples or release notes.
|
||||
|
||||
## Stage 18: Verify the complete target and finish
|
||||
|
||||
1. Format all changed Go files and inspect the full diff for unrelated changes,
|
||||
local replacements, generated caches, fabricated sums, or accidental public
|
||||
exports. Keep ignored `go.work`/`go.work.sum` and `.releasekit/` local.
|
||||
2. Run all affected module suites in the local workspace:
|
||||
|
||||
```sh
|
||||
GOTOOLCHAIN=local go test -count=1 ./...
|
||||
(cd integrations/ckconfig && GOTOOLCHAIN=local go test -count=1 ./...)
|
||||
(cd integrations/ckprompt && GOTOOLCHAIN=local go test -count=1 ./...)
|
||||
```
|
||||
|
||||
Root `./...` does not traverse nested modules. Reuse unchanged successful
|
||||
evidence where appropriate; none of these commands invokes publication.
|
||||
3. Verify independent readonly graphs with `GOWORK=off` and
|
||||
`GOTOOLCHAIN=local`. For unpublished CommandKit prerequisites, create
|
||||
temporary absolute-path `.mod`/`.sum` pairs copied from the owning module.
|
||||
Replace only the required core version with the local root for ckconfig,
|
||||
and only the required core/ckconfig versions for ckprompt.
|
||||
4. Prepare needed sums in those temporary files, then run
|
||||
`go test -mod=readonly -count=1 ./...` from each actual module directory with
|
||||
the applicable `-modfile=/absolute/path/module.mod`. Root uses its actual
|
||||
manifest without optional-library replacements. Hash temporary module/sum
|
||||
files before/after readonly checks and confirm they stay unchanged.
|
||||
ConfigKit, PromptKit, and ReleaseKit remain real released modules.
|
||||
5. Inspect `go list -m all` and `go list -deps -test ./...` using those same
|
||||
isolated graphs. Prove core has no optional integration libraries,
|
||||
ckconfig has no PromptKit/ckprompt dependency, and ckprompt selects the
|
||||
intended adopted releases. A workspace pass alone is insufficient.
|
||||
6. Review the target checklist below against actual source and test evidence.
|
||||
Run prose/link checks and review maintained examples alongside their actual
|
||||
command declarations. Do not introduce mandatory live probes, race/coverage
|
||||
thresholds, benchmark gates, or platform matrices.
|
||||
7. Mark stages complete only with actual evidence, update the roadmap's status,
|
||||
and identify any genuine remaining limitation accurately. Retain planning
|
||||
records until cleanup is requested. Commit the completed sprint's remaining
|
||||
nonempty work, inspect status, and report the commits and validation.
|
||||
Do not tag, push, or publish a release.
|
||||
|
||||
**Complete when:** every target below is implemented and evidenced, documentation
|
||||
and dependency metadata agree, and the workspace is clean.
|
||||
|
||||
**Completed:** 2026-10-17. All three module suites passed in the workspace and
|
||||
again with isolated readonly module graphs. The temporary ckconfig and ckprompt
|
||||
module files and sums were unchanged by readonly checks. Module listings select
|
||||
ReleaseKit v0.4.2, ConfigKit v0.8.0, and PromptKit v0.16.0; dependency listings
|
||||
show no optional integration imports in core and no PromptKit imports in
|
||||
ckconfig. All 40 maintained shell examples matched their expected outcomes,
|
||||
including the documented validation failure. Go files were formatted, the full
|
||||
change set contained only roadmap-scoped files and intended APIs, local links
|
||||
and Markdown whitespace passed, and the worktree was clean after commit. No
|
||||
release was tagged, pushed, or published.
|
||||
|
||||
## Target verification checklist
|
||||
|
||||
| Target | Primary stages | Completion evidence |
|
||||
| --- | --- | --- |
|
||||
| ReleaseKit v0.4.2, unchanged publication model and Go selections | 1, 16, 18 | Selected root tool graph, launcher boundary tests, factual docs/notices |
|
||||
| ConfigKit v0.8.0 in both integrations | 2, 16, 18 | Independent module graphs and released checksums |
|
||||
| New source evidence in text/YAML | 3 | Controlled-record projection tests; unchanged existing source fields |
|
||||
| ID-only PromptKit v0.16.0 API and explicit configuration | 4–5 | Removed selectors, migrated native fixtures and consumer, boundary tests |
|
||||
| Optional `version` omission/empty preservation | 4–5, 7, 10 | Semantic report assertions across affected reports |
|
||||
| Independent backend/namespace/collection catalogs and native filters | 6–8 | Public callbacks, command capture, native forwarding, availability and YAML |
|
||||
| Optional concrete smoke callback/helper and full agreed bindings | 9, 11, 13 | Seed/builder/native forwarding, pointer presence, lifecycle evidence |
|
||||
| Faithful smoke reports, partial results, supported error identity | 9–14 | Synthetic projections and assembled outcomes/delivery tests |
|
||||
| Ordered `test all`, continuation, cancellation, per-kind builders | 12–14 | Controlled aggregate call sequence, reports, and joined causes |
|
||||
| Small maintained consumer and current canonical documentation | 15, 17 | Offline consumer evidence, reviewed commands and local links |
|
||||
| No reverse optional dependencies, obsolete catalog graph, or local upstream replacements | 16, 18 | Independent manifests/module/test-import listings and notices |
|
||||
| Upstream testing ownership respected | All testing stages | Assertions address CommandKit interfaces and coordination, not native domain correctness |
|
||||
|
||||
No separate product or API decision was needed to complete the stages. Concrete
|
||||
presentation and wiring choices stayed within the accepted roadmap; incidental
|
||||
implementation details were not elevated into additional guarantees.
|
||||
@@ -1,370 +0,0 @@
|
||||
# PromptKit concrete request validation
|
||||
|
||||
Status: accepted adoption and feature roadmap; implementation complete.
|
||||
Command placement, adapter composition, request construction, artifact syntax,
|
||||
and initial presentation are settled. This document defines the target and
|
||||
required work; the [implementation plan](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/38b49f29fb3a8f22ff02528d2a0ed413e5c6ab1e/docs/roadmap/implementation.md)
|
||||
sequences the historical implementation work.
|
||||
|
||||
## Purpose
|
||||
|
||||
Adopt PromptKit v0.15.0 and expose its concrete request validation capability as
|
||||
`app promptkit requirements validate PROMPT_ID`, alongside the delivered
|
||||
`app promptkit requirements show PROMPT_ID` command.
|
||||
|
||||
The organizing principle is **CommandKit provides useful, consistent access to
|
||||
capabilities owned elsewhere**. CommandKit owns vocabulary, registration,
|
||||
selection, coordination, presentation, and invocation outcomes. PromptKit owns
|
||||
what concrete request validation means. Applications supply their domain
|
||||
mapping, defaults, artifacts, and runtime dependencies.
|
||||
|
||||
The command answers whether the constructed request passes native preparation
|
||||
at the time of the invocation. It complements discovery without making
|
||||
discovery a prerequisite for validation or validation a prerequisite for
|
||||
application execution.
|
||||
|
||||
## Current baseline
|
||||
|
||||
The baseline is CommandKit commit `700f10d78ed0e410b0cc1d754fdd98415cd5817a`.
|
||||
|
||||
- The optional [ckprompt module](../../integrations/ckprompt/go.mod) requires
|
||||
PromptKit v0.14.0, ConfigKit v0.7.1, and Go 1.26.0.
|
||||
- Its [suite and engine adapters](../../integrations/ckprompt/adapters.go)
|
||||
support independent prompt/profile catalogs, named definition inspection,
|
||||
and requirements discovery. `FromEngine` supplies those callbacks using an
|
||||
application-owned engine loader. There is no concrete validation callback.
|
||||
- The [requirements command](../cli.md#promptkit-requirements) accepts a prompt
|
||||
ID, configuration selection, typed suite selectors, version/profile/session
|
||||
context, and repeatable variable assignments. It returns the native discovery
|
||||
evidence as YAML. A partial inventory or missing variable is not itself an
|
||||
invocation failure.
|
||||
- `ConfigScope` separately exposes engine configuration inspection and
|
||||
comparison through ckconfig. It does not install the PromptKit command family.
|
||||
- The [composed consumer](../../integrations/ckprompt/examples/composed/main.go)
|
||||
demonstrates discovery and application-owned configuration/engine loading.
|
||||
It does not yet demonstrate construction or validation of a concrete request.
|
||||
|
||||
PromptKit v0.15.0 adds
|
||||
[`Engine.ValidateRequest`](https://gitea.maximumdirect.net/eric/promptkit/src/commit/635674c066d0b432187f96e72b9c03d085443ac5/engine.go)
|
||||
for an ordinary public `RunRequest`. Its
|
||||
[consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/commit/635674c066d0b432187f96e72b9c03d085443ac5/docs/consumers/pkg-promptkit.md#validate-a-concrete-request)
|
||||
and [release note](https://gitea.maximumdirect.net/eric/promptkit/src/commit/635674c066d0b432187f96e72b9c03d085443ac5/docs/releases/v0.15.0.md)
|
||||
describe the addition. The reviewed release leaves the upstream minimum Go
|
||||
version, module dependencies, and licensing material unchanged from v0.14.0.
|
||||
|
||||
## Targeted end state
|
||||
|
||||
### Command placement and meaning
|
||||
|
||||
The standard PromptKit family can expose both operations independently:
|
||||
|
||||
| Command | Operator purpose | Owning native capability |
|
||||
| --- | --- | --- |
|
||||
| `app promptkit requirements show PROMPT_ID` | Discover input and variable requirements for a prompt context. | `Engine.InspectRequirements` |
|
||||
| `app promptkit requirements validate PROMPT_ID` | Check the actual constructed request, including supplied values and artifact bindings. | `Engine.ValidateRequest` |
|
||||
|
||||
The new handler delegates to concrete native validation. It does not infer
|
||||
validity from the discovery report's `Complete`, inventory, or issues, and does
|
||||
not independently interpret templates or decide which inputs are required.
|
||||
|
||||
Native validation shares `Prepare`'s request conversion and preparation path.
|
||||
It may resolve prompt/profile resources, load artifacts and schema resources,
|
||||
render messages and session context, and perform the applicable configuration
|
||||
and credential checks. All supplied inputs follow native preparation behavior,
|
||||
including inputs unused by the selected template branch. Errors are returned
|
||||
as encountered rather than as an exhaustive inventory.
|
||||
|
||||
There is no model call or retained prepared snapshot. Success neither certifies
|
||||
provider acceptance nor guarantees that a later execution will succeed. A later
|
||||
run prepares independently. Applications needing execution tied to a checked
|
||||
snapshot can use PromptKit's existing prepared-execution API in their own
|
||||
workflow; that workflow is outside this feature.
|
||||
|
||||
### Operator inputs
|
||||
|
||||
Use the existing requirements command's common syntax for:
|
||||
|
||||
- One required prompt ID operand.
|
||||
- `--config` and `--overlay`, with existing explicit-presence handling.
|
||||
- Typed suite selectors.
|
||||
- `--version`, `--profile`, and `--session-id`, forwarding their native
|
||||
empty-string semantics.
|
||||
- Repeatable `--var KEY=VALUE`, preserving the existing first-separator split,
|
||||
exact keys and values, and last assignment for a duplicate key.
|
||||
|
||||
Keep singleton repetition and selector binding behavior consistent with the
|
||||
current CLI contract. PromptKit owns semantic interpretation of the values.
|
||||
The namespace identifies the library; configuration scope selection belongs
|
||||
to the separate `config` family.
|
||||
|
||||
Add repeatable `--input NAME=PATH` for file-backed bindings, translated to native
|
||||
`promptkit.File(PATH)` references without eagerly reading files. Split at the
|
||||
first `=`, preserve exact names and paths, and use the last binding for a
|
||||
duplicate name. Every occurrence is parsed; a later valid binding does not
|
||||
erase an earlier malformed one.
|
||||
|
||||
| Invocation fragment | Result |
|
||||
| --- | --- |
|
||||
| `--input notes=notes.txt` | Bind `notes` to `promptkit.File("notes.txt")`. |
|
||||
| `--input notes=dir/a=b.txt` | Preserve the path `dir/a=b.txt`. |
|
||||
| `--input notes=` | Forward an empty file path to native validation. |
|
||||
| `--input =notes.txt` | Forward the empty input name to native validation. |
|
||||
| `--input notes=first.txt --input notes=second.txt` | Bind `notes` to `second.txt`. |
|
||||
| `--input notes` | Usage error because the separator is missing. |
|
||||
|
||||
PromptKit owns semantic artifact validation and loading. CommandKit adds no
|
||||
input-name whitelist, path restrictions, normalization, shell expansion, or
|
||||
configuration-relative rebasing. Paths are preserved after shell processing.
|
||||
|
||||
Use `--format yaml` for the initial success report, with YAML as the default.
|
||||
Application-specific settings continue to use typed application input and
|
||||
callbacks. Applications can supply inline and domain-derived artifacts through
|
||||
their builder; a standard inline-input flag is outside this delivery. The
|
||||
command need not offer a built-in flag for every field in `RunRequest` to permit
|
||||
applications to use those fields through their builder.
|
||||
|
||||
### Public integration and request construction
|
||||
|
||||
Add an independently optional concrete validation capability to `ckprompt.Suite`.
|
||||
Installing validation alone must be useful; supplying discovery alone must
|
||||
continue to install only discovery. Both operations belong to the same
|
||||
`requirements` group when present. The existing installed-family ownership,
|
||||
absent-suite behavior, and retained wiring errors apply.
|
||||
|
||||
Use a ckprompt-owned envelope containing configuration selection and the native
|
||||
seed request populated from the standard command inputs:
|
||||
|
||||
```go
|
||||
type ValidationRequest struct {
|
||||
Selection ckconfig.Selection
|
||||
Request promptkit.RunRequest
|
||||
}
|
||||
```
|
||||
|
||||
Add this optional callback field to `Suite[I, S]`:
|
||||
|
||||
```go
|
||||
ValidateRequest func(
|
||||
context.Context, ValidationRequest, I, S,
|
||||
) (commandkit.Acquired[struct{}], error)
|
||||
```
|
||||
|
||||
The acquisition container carries diagnostics; validation has no native result
|
||||
value to invent. Custom callbacks remain available for application-owned runtime
|
||||
arrangements.
|
||||
|
||||
Provide a suite-returning `WithRequestValidation` helper that takes a suite, the
|
||||
existing engine-loader shape, and an optional typed builder:
|
||||
|
||||
```go
|
||||
func WithRequestValidation[I, S any](
|
||||
suite Suite[I, S],
|
||||
load func(context.Context, ckconfig.Selection, I, S) (*promptkit.Engine, error),
|
||||
build func(context.Context, ValidationRequest, I, S) (promptkit.RunRequest, error),
|
||||
) Suite[I, S]
|
||||
```
|
||||
|
||||
The helper attaches the engine-backed validation callback and retains construction
|
||||
errors, including a nil loader, in the returned suite for installation to report.
|
||||
Existing suite capabilities and attached issues are preserved. Narrowing the
|
||||
capabilities afterward does not discard retained wiring errors. Returning a suite
|
||||
gives these errors the same treatment as existing adapters.
|
||||
|
||||
For application-defined `loadEngine` and `buildRequest` callbacks, composition is:
|
||||
|
||||
```go
|
||||
suite := ckprompt.FromEngine(loadEngine)
|
||||
suite = ckprompt.WithRequestValidation(suite, loadEngine, buildRequest)
|
||||
```
|
||||
|
||||
A validation-only consumer can start from an empty suite. Given its application
|
||||
types `Input` and `State`, it can use the standard command-populated request
|
||||
directly:
|
||||
|
||||
```go
|
||||
suite := ckprompt.WithRequestValidation(
|
||||
ckprompt.Suite[Input, State]{}, loadEngine, nil,
|
||||
)
|
||||
```
|
||||
|
||||
A nil builder passes the seed request unchanged to native validation. Standard
|
||||
flags already supply useful prompt context, variables, and file bindings;
|
||||
application mapping is optional. `FromEngine` alone continues to supply its
|
||||
existing inspection callbacks, so validation remains an explicit wiring choice.
|
||||
|
||||
When present, the builder receives invocation context, the selection and seed
|
||||
request, selected application input, and shared invocation state. It returns
|
||||
the concrete native request. It can supply application artifacts, inline content,
|
||||
execution/output-contract overrides, appended messages, credentials, or other
|
||||
supported native settings. The adapter forwards that result without reapplying
|
||||
or merging fields afterward.
|
||||
|
||||
Builders should start from the seed, preserve the selected prompt identity and
|
||||
operator values, and add application defaults deliberately. Map defaults respect
|
||||
key presence, including a supplied empty value. For example, within a builder:
|
||||
|
||||
```go
|
||||
request := seed.Request
|
||||
if request.Vars == nil {
|
||||
request.Vars = make(map[string]string)
|
||||
}
|
||||
if _, supplied := request.Vars["audience"]; !supplied {
|
||||
request.Vars["audience"] = "internal"
|
||||
}
|
||||
return request, nil
|
||||
```
|
||||
|
||||
Here `seed` is the builder's `ValidationRequest`; `--var audience=` remains an
|
||||
explicit empty value. Version, profile, and session fields retain their native
|
||||
empty-string semantics, including equivalence between omitted and explicitly
|
||||
empty scalar values. Configuration selection retains its existing presence
|
||||
information. CommandKit documents these responsibilities without adding a second
|
||||
native-field validation layer. Reusing a request builder between validation and
|
||||
application execution is recommended consumer guidance.
|
||||
|
||||
### Invocation lifecycle and outcomes
|
||||
|
||||
Reuse the integration's current typed input preparation, selector application,
|
||||
input checking, invocation state, configuration diagnostic adaptation, and
|
||||
outcome/delivery machinery. Construction, installation, help, and version must
|
||||
leave acquisition and request-building callbacks uninvoked, under the existing
|
||||
[architecture policy](../policy/architecture.md).
|
||||
|
||||
For the bundled adapter, use this order:
|
||||
|
||||
1. Complete CLI preflight and application input checks.
|
||||
2. Create shared invocation state.
|
||||
3. Invoke the builder, if supplied; otherwise use the seed request unchanged.
|
||||
4. Load the engine using the selected configuration, application input, and state.
|
||||
5. Invoke native validation with the concrete request.
|
||||
6. Deliver diagnostics and the success report through the established paths.
|
||||
|
||||
Both builder and loader receive the same invocation state and context.
|
||||
Applications can explicitly share configuration acquisition through that state.
|
||||
A builder failure prevents engine loading. Check cancellation between stages,
|
||||
preserve diagnostics already acquired, and leave later stages uninvoked after
|
||||
failure. Preserve supported native error identity and configuration diagnostics,
|
||||
including failures during engine loading.
|
||||
|
||||
Use the existing classification model: malformed CLI syntax and missing
|
||||
operands are usage failures. Application input-check failures retain their
|
||||
existing handling. Builder, engine-loader, and native-validation failures are
|
||||
operational failures with supported error identity preserved. Returned provider
|
||||
errors do not carry arbitrary usage or domain outcome classifications;
|
||||
application-owned handlers can return their own outcomes. Preserve
|
||||
native cancellation error categories; do not add a promise that every native
|
||||
failure matches `ctx.Err()`.
|
||||
|
||||
Successful native validation produces the selected success presentation only
|
||||
after it succeeds. Diagnostics and output failures follow the established
|
||||
delivery contract. The presentation reports the validation outcome without
|
||||
dumping the request, artifact content, credential values, or rendered messages.
|
||||
|
||||
### Initial success presentation
|
||||
|
||||
Emit one small YAML report, defaulting to YAML and accepting explicit
|
||||
`--format yaml` as the only format for this delivery:
|
||||
|
||||
```yaml
|
||||
prompt_id: briefing
|
||||
valid: true
|
||||
```
|
||||
|
||||
The ID identifies the command's requested prompt. `valid` means that validation
|
||||
of the concrete request returned nil at this invocation; with the bundled
|
||||
adapter, this is the result of native preparation. Builder guidance preserves
|
||||
the selected prompt identity so the confirmation describes the operator's
|
||||
selection. CommandKit adds no independent identity-validation layer.
|
||||
|
||||
Native validation supplies neither resolved version/profile metadata nor a
|
||||
prepared snapshot. The report therefore contains no such claims or discovery
|
||||
inventories. Failures use the existing diagnostic/error outcome path. Output
|
||||
delivery is part of invocation success. This initial presentation provides an
|
||||
explicit confirmation for operators and automation; additional formats can
|
||||
follow concrete needs.
|
||||
|
||||
### Package and dependency ownership
|
||||
|
||||
All new provider contracts, request adaptation, standard handlers, and
|
||||
presentation belong in `integrations/ckprompt`. Core continues to own generic
|
||||
declarations, bindings, dispatch, and outcomes. ckconfig remains responsible
|
||||
for its configuration adaptation; ckprompt can reuse it where appropriate.
|
||||
|
||||
Adopt the released PromptKit v0.15.0 module. Preserve the current Go minimum
|
||||
and dependency direction: ckprompt may depend on ckconfig; ckconfig and core
|
||||
do not acquire a PromptKit dependency. Use public upstream APIs and released
|
||||
module versions, without sibling-checkout requirements in committed manifests.
|
||||
|
||||
## Work needed from the baseline
|
||||
|
||||
### Dependency adoption
|
||||
|
||||
Update ckprompt's PromptKit requirement and checksums to v0.15.0. Review the
|
||||
resulting selected module graph, and update the PromptKit attribution entry's
|
||||
selected version and source/notice links to release commit
|
||||
`635674c066d0b432187f96e72b9c03d085443ac5`. Update other attribution entries only
|
||||
where the selected graph or incorporated material actually changes.
|
||||
|
||||
### Integration API and native adapter
|
||||
|
||||
Add `ValidationRequest`, the optional suite callback, and the suite-returning
|
||||
`WithRequestValidation` helper. Reuse existing engine-loading diagnostics and
|
||||
supported error handling.
|
||||
Support concrete request construction from typed application input and standard
|
||||
CLI values, without copying PromptKit's preparation logic. Account for builder
|
||||
failure and invalid loader wiring using the established integration conventions;
|
||||
a nil builder uses the seed unchanged.
|
||||
|
||||
Keep current discovery and configuration adapters available independently.
|
||||
`FromEngine` keeps its current capabilities. Validation is explicitly enabled
|
||||
through the helper or a custom suite callback, with an application builder only
|
||||
when additional mapping or defaults are needed.
|
||||
|
||||
### Command assembly, bindings, and presentation
|
||||
|
||||
Extend `ckprompt.Command` to assemble the requirements group from its supplied
|
||||
show and validate capabilities. Add the validate handler, file-binding syntax,
|
||||
and YAML success report. Share common binding/report mechanics where
|
||||
that simplifies the code, preserving the delivered discovery contract.
|
||||
|
||||
### Maintained consumer and documentation
|
||||
|
||||
Extend the composed consumer with a synthetic concrete request builder and
|
||||
offline validation examples for a successful request and a missing required
|
||||
binding. Demonstrate the existing typed domain selectors alongside the new
|
||||
standard bindings. The example should make the different questions answered
|
||||
by show and validate clear without requiring a model service or real credential.
|
||||
Document the nil-builder path and a builder that preserves supplied map entries
|
||||
while adding defaults.
|
||||
|
||||
When implemented, update Go declarations/GoDoc, the CLI reference, consumer
|
||||
guide, and affected orientation/internal descriptions under their canonical
|
||||
owners. Describe native resource work and validation limits accurately, so
|
||||
statements about metadata inspection do not accidentally claim that concrete
|
||||
validation performs no rendering, artifact loading, or credential checks.
|
||||
|
||||
This capability fits the accepted optional-family and owning-library boundaries.
|
||||
No new ADR or revision to accepted ADRs is needed. The selected API and CLI
|
||||
details belong in their canonical documentation when implemented.
|
||||
|
||||
### Completion evidence for implementation
|
||||
|
||||
Use proportionate public-boundary evidence for independent capability
|
||||
availability, help without acquisition, standard/typed input mapping, fresh
|
||||
invocation state, nil-builder forwarding, builder default precedence, retained
|
||||
wiring errors, builder-before-loader behavior, native validation error identity,
|
||||
and diagnostic/output delivery. Cover builder and loader failures and cancellation
|
||||
between stages. Use a real PromptKit engine for representative
|
||||
successful validation and a missing binding, with a model-call double that
|
||||
detects accidental execution. Protect continued successful discovery of a
|
||||
partial inventory; do not turn it into a validation failure.
|
||||
|
||||
Check semantic output rather than YAML key ordering or native error wording.
|
||||
Avoid duplicating PromptKit's complete preparation test suite. Validate the
|
||||
owning integration and affected composed consumer; check isolated module
|
||||
resolution and dependency boundaries for the upgrade, using the established
|
||||
workspace and disposable-module practices in the
|
||||
[developer guide](../development.md). Implementation evidence must not rely
|
||||
solely on a workspace that silently substitutes a sibling PromptKit checkout.
|
||||
|
||||
The decisions needed for an implementation plan are settled.
|
||||
@@ -1,213 +0,0 @@
|
||||
# Dependency upgrade and PromptKit smoke review remediation
|
||||
|
||||
Status: complete; R-1 through R-4 addressed.
|
||||
|
||||
## Review scope and evidence
|
||||
|
||||
Reviewed `29582dd..730f930b47c929fb2c59a25a75cbe2924dfc75bb` against the
|
||||
[accepted upgrade roadmap](upgrades.md), [implementation plan](implementation.md),
|
||||
[architecture policy](../policy/architecture.md), and
|
||||
[testing ownership policy](../policy/testing.md#upstream-integration-testing-ownership).
|
||||
The worktree was clean at review start.
|
||||
|
||||
The intended architecture is substantially present:
|
||||
|
||||
- Core owns generic declarations, binding, installation, dispatch, help/version,
|
||||
diagnostics, and outcomes; its Cobra/pflag adapter remains internal.
|
||||
- ckconfig owns ConfigKit adaptation and the extended source reports.
|
||||
- ckprompt depends on public core/ckconfig/ConfigKit/PromptKit APIs. Its eight
|
||||
bundled inspection/catalog callbacks and two optional validation/smoke
|
||||
additions preserve the selected ownership boundary.
|
||||
- Native IDs, optional versions, filters, source evidence, and smoke reports
|
||||
pass through the owning integrations. Aggregate smoke testing coordinates
|
||||
concrete callbacks in the selected order and retains partial reports/errors.
|
||||
- ReleaseKit remains a root maintainer tool. The adoption leaves the launcher,
|
||||
profiles, shared-file selection, and source-only publication model intact.
|
||||
|
||||
This was a source, test, documentation, and dependency review. Readonly
|
||||
workspace and independent module listings confirmed ReleaseKit v0.4.2,
|
||||
ConfigKit v0.8.0, and PromptKit v0.16.0. Independent package dependency listings,
|
||||
including test imports, confirmed no optional integration libraries in core and
|
||||
no PromptKit/ckprompt dependency in ckconfig. Temporary manifests replaced only
|
||||
unpublished CommandKit prerequisites; upstream dependencies remained released
|
||||
modules. No test suite was run or added during this review.
|
||||
|
||||
The completed implementation plan records earlier test runs; those completion
|
||||
markers are historical evidence, not fresh test results from this review.
|
||||
No production correction was applied during the review. Completion records
|
||||
below describe the subsequent remediation. The preceding completed
|
||||
request-validation review remains available in
|
||||
[Git history](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/730f930b47c929fb2c59a25a75cbe2924dfc75bb/docs/roadmap/remediation.md).
|
||||
|
||||
## R-1: Preserve aggregate error presence when its message is empty
|
||||
|
||||
Status: complete. Priority: P2. Category: report fidelity.
|
||||
|
||||
**Location:** [smoke_report.go](../../integrations/ckprompt/smoke_report.go),
|
||||
`smokeResultDTO` and `encodeSmokeResults`, lines 101–125.
|
||||
|
||||
**Finding:** The DTO uses `Error string` with `yaml:"error,omitempty"`.
|
||||
`encodeSmokeResults` assigns `result.Err.Error()` only when the error is
|
||||
non-nil. A non-nil error whose message is `""` therefore produces the same
|
||||
empty DTO field as no error, and `omitempty` drops it.
|
||||
|
||||
For example, a concrete callback returning a native report and `errors.New("")`
|
||||
still causes an operational aggregate outcome, but its YAML entry has no
|
||||
`error` key. Report consumers cannot distinguish that entry from one whose
|
||||
callback returned no error. No nonempty-error-message requirement was selected
|
||||
for application callbacks.
|
||||
|
||||
**Required work:** Track error presence separately from its display text. A
|
||||
private `*string` DTO field with `omitempty` is sufficient: nil for no error,
|
||||
and a pointer to the returned message for a non-nil error, including an empty
|
||||
message. Preserve the existing native report, aggregate ordering, joined cause
|
||||
identity, and normal outcome classification. Do not replace empty messages,
|
||||
reject the error, or add another error taxonomy.
|
||||
|
||||
**Completion evidence:** Add a focused semantic YAML regression covering nil,
|
||||
empty-message, and ordinary nonempty errors. Only nil omits `error`; an empty
|
||||
message remains `error: ""`. Confirm the assembled aggregate still returns
|
||||
operational failure and preserves the original error through `errors.Is`.
|
||||
The appropriate focused tests and ckprompt suite pass.
|
||||
|
||||
**Completed:** 2026-10-18. Changed the private aggregate error field to an
|
||||
optional string pointer, preserving non-nil empty messages. Extended the YAML
|
||||
projection and assembled aggregate regressions to distinguish nil, empty, and
|
||||
nonempty errors while retaining operational classification and typed cause
|
||||
identity. Both focused regressions failed before the fix and passed afterward.
|
||||
The complete ckprompt suite, including the composed consumer, passed with
|
||||
Go 1.26.8 and offline dependency resolution.
|
||||
|
||||
## R-2: Remove assertions about PromptKit's private smoke-test execution
|
||||
|
||||
Status: complete. Priority: P3. Category: upstream testing ownership.
|
||||
|
||||
**Location:** [adapters_test.go](../../integrations/ckprompt/adapters_test.go),
|
||||
`TestWithSmokeTestingForwardsAuthoritativeNativeRequestAndRetainsReport`,
|
||||
line 167, and
|
||||
`TestWithSmokeTestingNilBuilderForwardsSeedAndBuilderFailureSkipsLoader`,
|
||||
line 182.
|
||||
|
||||
**Finding:** The nil-builder test requires the native generated prompt to contain
|
||||
exactly one message. Both tests require exactly one call to the injected native
|
||||
LLM client. CommandKit forwards a `SmokeTestRequest` to `Engine.SmokeTest`;
|
||||
PromptKit constructs its private diagnostic prompt and owns its internal
|
||||
generation sequence.
|
||||
|
||||
A legitimate upstream change to the private prompt shape or diagnostic
|
||||
generation sequence could fail these tests without any incorrect CommandKit
|
||||
adaptation. The selected testing boundary permits observing the client to
|
||||
verify forwarding, but does not establish private native work as a CommandKit
|
||||
contract.
|
||||
|
||||
**Required work:** Remove the exact native message-count assertion and exact
|
||||
native generation-count expectations. Retain evidence that the intended native
|
||||
path is reached and that the seed or authoritative builder request controls
|
||||
the observed model/credential values. Keep report/error preservation,
|
||||
configuration selection, context/input/state forwarding, and builder-failure
|
||||
ordering checks. Where counting is useful, checking that native client use
|
||||
occurred is sufficient; do not fix its native internal call count.
|
||||
|
||||
This does not remove zero-call guards for help, metadata operations, or failures
|
||||
that must stop before acquisition. Those protect selected CommandKit behavior.
|
||||
|
||||
**Completion evidence:** Focused helper tests and the ckprompt suite pass.
|
||||
Assertions still detect request remerging, wrong forwarding, or dropped results,
|
||||
without inspecting the native private diagnostic message inventory or fixing
|
||||
the generation sequence. No production-code change is required.
|
||||
|
||||
**Completed:** 2026-10-18. Removed the private diagnostic message-count
|
||||
assertion and changed native generation-count checks to require only that the
|
||||
configured client was used. Request/model/credential forwarding, authoritative
|
||||
builder output, report/error handling, context/input/state checks, and the
|
||||
builder-failure guard against loading remain covered. Focused
|
||||
`TestWithSmokeTesting` tests and the complete ckprompt suite, including the
|
||||
composed consumer, passed with Go 1.26.8 and offline dependency resolution.
|
||||
Production code is unchanged.
|
||||
|
||||
## R-3: Remove incidental loader-count and pointer-identity contracts
|
||||
|
||||
Status: complete. Priority: P3. Category: unnecessary implementation constraints.
|
||||
|
||||
**Locations:**
|
||||
|
||||
- [smoke_lifecycle_test.go](../../integrations/ckprompt/smoke_lifecycle_test.go),
|
||||
`TestSmokeTestBuilderReceivesConcreteAggregateKinds`, lines 201–212.
|
||||
- [smoke_aggregate_test.go](../../integrations/ckprompt/smoke_aggregate_test.go),
|
||||
`TestSmokeAggregateCoordinatesConcreteRequestsAndRetainsErrors`,
|
||||
lines 89–95 and its pointer-collection bookkeeping.
|
||||
|
||||
**Finding:** The builder test requires exactly four engine-loader calls.
|
||||
The accepted aggregate behavior requires per-kind requests/builders and allows
|
||||
engine/acquisition reuse where practical; it deliberately does not guarantee
|
||||
a loader-call count.
|
||||
|
||||
The aggregate test also requires pairwise distinct addresses for every
|
||||
execution override and selected scalar pointer. It already checks the
|
||||
meaningful behavior: mutating the first seed must not change the common CLI
|
||||
values seen by subsequent operations. Pairwise allocation identity adds a
|
||||
separate constraint on storage strategy that the operator and consumer
|
||||
contracts do not require.
|
||||
|
||||
The implementation plan describes a straightforward fresh-storage approach.
|
||||
That instruction does not independently establish public pointer identity or
|
||||
resource-management guarantees under the
|
||||
[agent policy](../policy/agents.md#commandkit-contract-authority).
|
||||
|
||||
**Required work:** Remove the exact aggregate loader count and pairwise pointer
|
||||
inequality assertions, together with bookkeeping used only for those assertions.
|
||||
Retain concrete builder/callback order, per-kind builder inputs, shared
|
||||
invocation input/state, subsequent seed values after mutation, continuation,
|
||||
cancellation, and native error identity. Keep justified guards preventing
|
||||
loading after builder failure and during help/version. Do not add caching,
|
||||
pooling, or a public lifetime abstraction to resolve a test-only issue.
|
||||
|
||||
**Completion evidence:** Focused aggregate/lifecycle tests and the ckprompt suite
|
||||
pass. The tests continue to detect cross-test value contamination and missing
|
||||
operations, while accepting equivalent acquisition and storage implementations.
|
||||
No production-code change is required.
|
||||
|
||||
**Completed:** 2026-10-18. Removed the aggregate loader counter and pairwise
|
||||
pointer-address assertions/bookkeeping. The tests retain ordered concrete
|
||||
builder/callback inputs, shared input/state, mutation-isolated seed values,
|
||||
continuation, cancellation, reports, and error identity. Focused aggregate,
|
||||
builder, help/version, and builder-failure tests and the complete ckprompt suite,
|
||||
including the composed consumer, passed with Go 1.26.8 and offline dependency
|
||||
resolution. Production code is unchanged.
|
||||
|
||||
## R-4: Reconcile the feature roadmap with completed implementation
|
||||
|
||||
Status: complete. Priority: P3. Category: stale implementation status.
|
||||
|
||||
**Location:** [upgrades.md](upgrades.md), status at lines 3–6,
|
||||
current-implementation descriptions at lines 28–29 and 41–44, and the source
|
||||
DTO statement at line 124.
|
||||
|
||||
**Finding:** The roadmap still says implementation has not started and that the
|
||||
current source DTO omits all four new evidence fields. The code now exposes
|
||||
those fields, and the implementation plan marks Stages 1–18 complete. Other
|
||||
present-tense descriptions still refer to the planning baseline.
|
||||
|
||||
Stage 18 expressly called for updating the roadmap's implementation status.
|
||||
Leaving these claims in place makes implemented features appear absent and
|
||||
can cause a later agent to restart completed work.
|
||||
|
||||
**Required work:** Record the delivered implementation and link to the completed
|
||||
plan/current canonical documentation. Qualify old baseline descriptions and
|
||||
version tables as historical rather than rewriting the planning baseline.
|
||||
Remove or recast assertions that the current DTO/API lacks delivered behavior.
|
||||
Preserve accepted decisions, dependency boundaries, and the remaining review
|
||||
findings; feature delivery can be recorded as complete while these focused
|
||||
remediations remain open.
|
||||
|
||||
**Completion evidence:** The roadmap and implementation plan agree on delivery
|
||||
status, current-code claims match the inspected implementation, and local
|
||||
links/Markdown whitespace checks pass. No code or policy change is required.
|
||||
|
||||
**Completed:** 2026-10-18. Recorded implementation completion and linked the
|
||||
completed plan, review follow-up, and current CLI/consumer/GoDoc owners. Marked
|
||||
the original dependency/capability descriptions as historical and corrected the
|
||||
source-report and ID-only API descriptions to match delivered code. Preserved
|
||||
the original baseline and accepted decisions unchanged. Local links/fragments,
|
||||
inbound roadmap anchors, status/baseline consistency, and Markdown whitespace
|
||||
checks passed. No code or policy change was needed for this issue.
|
||||
@@ -1,530 +0,0 @@
|
||||
# ConfigKit, PromptKit, and ReleaseKit adoption
|
||||
|
||||
Status: implementation complete, including all three dependency upgrades and
|
||||
smoke testing. The [implementation plan](implementation.md) records completion
|
||||
of Stages 1–18; [remediation.md](remediation.md) records focused review follow-up
|
||||
and its completion evidence. The following sections preserve the accepted target
|
||||
and original migration intent.
|
||||
|
||||
## Purpose and authority
|
||||
|
||||
Adopt ReleaseKit v0.4.2, ConfigKit v0.8.0, and PromptKit v0.16.0 while providing
|
||||
useful operator access to their capabilities. The organizing principle remains:
|
||||
**CommandKit provides useful, consistent access to capabilities owned elsewhere.**
|
||||
|
||||
CommandKit owns vocabulary, registration, selection, coordination, presentation,
|
||||
and invocation outcomes. Integrated libraries own configuration composition,
|
||||
resource identity and resolution, validation, inspection evidence, and diagnostic
|
||||
meaning. Applications choose their inputs, policies, defaults, credentials,
|
||||
artifacts, and runtime construction.
|
||||
|
||||
Keep adaptation proportionate to that ownership. Reports are direct projections
|
||||
of upstream data, with small presentation DTOs where YAML naming and formatting
|
||||
need them. Preserve native values and meanings rather than introducing a
|
||||
parallel semantic model or normalization layer.
|
||||
|
||||
The [architecture policy](../policy/architecture.md), accepted [ADRs](../adr/),
|
||||
and [release policy](../policy/release.md) govern this work. Upstream contract
|
||||
changes flow through the owning integration; they do not justify reimplementing
|
||||
removed behavior in CommandKit. The [CLI reference](../cli.md),
|
||||
[consumer guide](../consumers/guide.md), [ckconfig GoDoc](../../integrations/ckconfig/doc.go),
|
||||
and [ckprompt GoDoc](../../integrations/ckprompt/doc.go) own implemented usage
|
||||
and contracts. The completed implementation plan records delivery evidence.
|
||||
|
||||
## Planning baseline and reviewed releases
|
||||
|
||||
The original adoption review used CommandKit baseline
|
||||
`5f633e3feea71f623e5e378a7df476ff7e811e5e`; the implementation plan began at
|
||||
`38b49f29fb3a8f22ff02528d2a0ed413e5c6ab1e`. This section records the historical
|
||||
starting point and reviewed releases, rather than current dependency metadata.
|
||||
|
||||
| Dependency | Planning baseline | Adopted release | Owning CommandKit area |
|
||||
| --- | --- | --- | --- |
|
||||
| ReleaseKit | Root tool requirement v0.4.0; integration graphs selected v0.4.1 | v0.4.2 | Root tool pin, launcher integration, release documentation, and notices |
|
||||
| ConfigKit | v0.7.1 in both integrations | v0.8.0 | ckconfig adaptation and reports; ckprompt's direct dependency |
|
||||
| PromptKit | v0.15.0 | v0.16.0 | ckprompt adapters, standard commands, reports, and composed consumer |
|
||||
|
||||
At that baseline, the root module owned generic commands, dispatch, bindings,
|
||||
help/version, and outcomes. ckconfig owned the configuration family. ckprompt
|
||||
supplied prompt/profile catalogs, named inspection, requirements discovery, and
|
||||
concrete request validation; its engine configuration scope was composed through
|
||||
ckconfig. Current capabilities are documented in the
|
||||
[consumer guide](../consumers/guide.md#promptkit-configuration-definitions-and-requirements).
|
||||
|
||||
The completed [requirements roadmap](promptkit-validation.md),
|
||||
[original implementation plan](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/38b49f29fb3a8f22ff02528d2a0ed413e5c6ab1e/docs/roadmap/implementation.md),
|
||||
and [preceding remediation record](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/730f930b47c929fb2c59a25a75cbe2924dfc75bb/docs/roadmap/remediation.md)
|
||||
remain historical evidence for the pre-upgrade baseline.
|
||||
|
||||
Reviewed upstream references:
|
||||
|
||||
| Release | Source commit | Release record |
|
||||
| --- | --- | --- |
|
||||
| ReleaseKit v0.4.2 | `7d033c5ca1c95aec7db9690293638a0a1ac887b8` | [Release notes](https://gitea.maximumdirect.net/eric/ReleaseKit/src/commit/7d033c5ca1c95aec7db9690293638a0a1ac887b8/docs/releases/v0.4.2.md) |
|
||||
| ConfigKit v0.8.0 | `137769f3f18a370c68444242fb5d88e620e307e5` | [Release notes](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/137769f3f18a370c68444242fb5d88e620e307e5/docs/releases/v0.8.0.md) |
|
||||
| PromptKit v0.16.0 | `9178f3eb91c2b2138caa78e22be93c048b4bc992` | [Release notes](https://gitea.maximumdirect.net/eric/promptkit/src/commit/9178f3eb91c2b2138caa78e22be93c048b4bc992/docs/releases/v0.16.0.md) |
|
||||
|
||||
All three releases retain Go 1.26.0 as their consumer minimum. The adoption
|
||||
preserved CommandKit's Go minimum and selected release toolchain.
|
||||
|
||||
## Dependency and delivery boundaries
|
||||
|
||||
Delivery includes all three upgrades, richer configuration source inspection,
|
||||
expanded PromptKit discovery, and explicitly invoked smoke testing. The completed
|
||||
[implementation plan](implementation.md) records coding order and stage
|
||||
boundaries for the coordinated sprint.
|
||||
|
||||
ReleaseKit has no runtime ordering dependency on the other two. During adoption,
|
||||
ConfigKit was the prerequisite for PromptKit v0.16.0: upgrading PromptKit first
|
||||
would have selected ConfigKit v0.8.0 transitively while ckconfig still declared
|
||||
v0.7.1. Both integrations now declare ConfigKit v0.8.0 directly, preserving
|
||||
alignment across their independent module graphs.
|
||||
|
||||
Publication remains source-only and follows the existing
|
||||
[core, ckconfig, ckprompt dependency order](../release.md#scope-and-release-order)
|
||||
for affected releases. CommandKit module versions remain independent of upstream
|
||||
version numbers. Explain intentional public API/CLI changes under the existing
|
||||
pre-v1 compatibility policy; select concrete release versions during release
|
||||
preparation.
|
||||
|
||||
## Target state: ReleaseKit v0.4.2
|
||||
|
||||
The root Go tool requirement selects ReleaseKit v0.4.2. The existing launcher,
|
||||
three module profiles, shared-file selection, and source-publication procedure
|
||||
continue using its supported public contracts. No application command family or
|
||||
runtime ReleaseKit import is added.
|
||||
|
||||
v0.4.1 changed upstream test synchronization. v0.4.2 corrected maintained
|
||||
Woodpecker delivery workflows for annotated-tag-object events; their event and
|
||||
tag IDs are resolved to commits before verification and delivery. CommandKit
|
||||
has no copied Woodpecker delivery workflow and publishes no hosted deliverables,
|
||||
so adopting this patch does not require adding either.
|
||||
|
||||
The adoption updated the root requirement/checksums, selected-graph notices,
|
||||
the version named in the release policy, and version/commit references in the
|
||||
maintainer release procedure. The launcher and release integration retain the
|
||||
supported CLI/profile contracts. The ignored `.releasekit/` directory remains
|
||||
local bootstrap storage.
|
||||
|
||||
If a delivery workflow is introduced later, start from the corrected maintained
|
||||
example: changing a module pin does not update copied YAML. This adoption does
|
||||
not add delivery automation or new publication gates.
|
||||
|
||||
## Target state: ConfigKit v0.8.0
|
||||
|
||||
### Native behavior and ownership
|
||||
|
||||
Both integration modules use ConfigKit v0.8.0. Its input destinations, relative
|
||||
import destinations, sequence-item placement, shared source-version checking,
|
||||
and registered identified-sequence composition are available to application
|
||||
loaders through upstream public options and definitions.
|
||||
|
||||
CommandKit does not need a parallel composition layer or new generic flags to
|
||||
select those policies. Applications decide whether to enable source-version
|
||||
checking, which declarations to accept, and which sequences use identity-based
|
||||
composition. Existing ConfigKit defaults preserve prior loading behavior.
|
||||
|
||||
### Source inspection
|
||||
|
||||
`config sources` now exposes native destination, placement,
|
||||
unresolved-destination, and source-version declaration evidence in text and YAML.
|
||||
The [source DTO](../../integrations/ckconfig/sources.go) includes all four while
|
||||
preserving the original source fields and native inventory order. OD-1 records
|
||||
the accepted intent; the [CLI source-report contract](../cli.md#list-configuration-sources)
|
||||
owns the implemented representation.
|
||||
|
||||
Project the native fields and their values directly into the report, including
|
||||
`version_check` corresponding to `SourceInfo.VersionCheck`. Use ConfigKit's
|
||||
returned, sanitized evidence and describe its fields neutrally. ConfigKit owns
|
||||
the meaning and disclosure of that evidence.
|
||||
|
||||
`config explain` continues to expose the native explanation/history contract;
|
||||
the new SourceInfo fields do not require adding a source inventory to every
|
||||
explanation. Compact diagnostic source legends can continue identifying sources
|
||||
by ID, label, and filesystem path.
|
||||
|
||||
### Completed adaptation
|
||||
|
||||
Both dependency graphs, owning notices, native fixtures, and affected consumer
|
||||
guidance were updated. The compatibility review accounted for unkeyed literals
|
||||
and equality/map-key assumptions: ConfigKit's expanded `SourceInfo` contains a
|
||||
`Path` and is no longer comparable.
|
||||
Its [release notes](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/137769f3f18a370c68444242fb5d88e620e307e5/docs/releases/v0.8.0.md)
|
||||
describe this source-compatibility change. The existing provider model need not
|
||||
be redesigned merely to adopt these additions.
|
||||
|
||||
## Target state: PromptKit v0.16.0
|
||||
|
||||
### ID-based prompt selection and descriptive labels
|
||||
|
||||
ckprompt uses native ID-only prompt selection. The prompt-specific `--version`
|
||||
flag was removed from named prompt inspection, requirements show, and requirements
|
||||
validate, together with `PromptRequest.Version` and obsolete selector plumbing.
|
||||
The adapter uses `InspectPrompt(ctx, promptID)` and the revised native request
|
||||
types.
|
||||
Application version reporting remains governed by its existing identity contract.
|
||||
|
||||
Version labels remain useful metadata. Inspection/catalog report DTOs use the
|
||||
native optional representation: `version` for a prompt or collection's reported
|
||||
version, omitted when absent and retained as `""` when explicitly empty, as
|
||||
accepted in OD-2. Selection continues to use the native ID-only API.
|
||||
|
||||
The standard requirements operations continue delegating discovery and concrete
|
||||
preparation to PromptKit. Partial discovery remains a successful inspection;
|
||||
native validation remains authoritative for a constructed request. The existing
|
||||
typed input/state, nil-builder path, callback capture, error identity, and
|
||||
delivery behavior apply through the revised API.
|
||||
|
||||
### Explicit configuration and collections
|
||||
|
||||
The native fixtures and composed consumer now use one direct definition per
|
||||
prompt ID. Where an application formerly supplied multiple definitions under
|
||||
one ID, it chooses which definition to install or gives distinct resources
|
||||
distinct IDs.
|
||||
PromptKit's optional resource labels do not make that deployment choice.
|
||||
|
||||
Treat a root configuration `version` as the upstream source specification
|
||||
header, distinct from a prompt's descriptive `version` or a collection's
|
||||
`release_version`. Their meaning and any configured acceptance checks remain
|
||||
with the owning library and application.
|
||||
|
||||
Supply backend/profile resources explicitly where needed. Remove assumptions
|
||||
about installed provider catalogs and the former built-in standard/auto profiles.
|
||||
CommandKit must not silently reintroduce those catalogs. The owning application
|
||||
chooses files, embedded inputs, programmatic mappings, and collections.
|
||||
|
||||
Keep the maintained composed consumer migration small: update its configuration,
|
||||
loader, request builder, and fixtures as needed for direct definitions and the
|
||||
new interfaces, without requiring a model service. Use a small configuration to
|
||||
illustrate the relevant wiring and identities; a comprehensive collections
|
||||
example or migration framework is not required. Collections are available
|
||||
through native `CollectionInput` and configuration sources; applications need
|
||||
not use them to adopt v0.16.0.
|
||||
|
||||
Configured references retain their native exact identities. Mounting a source
|
||||
does not rewrite backend/profile references or extend profile inheritance beyond
|
||||
its native scope. Migration guidance should link to the upstream
|
||||
[configuration/API migration guide](https://gitea.maximumdirect.net/eric/promptkit/src/commit/9178f3eb91c2b2138caa78e22be93c048b4bc992/docs/consumers/configuration-migration.md).
|
||||
|
||||
### Expanded operator discovery
|
||||
|
||||
The optional family exposes these additional catalogs:
|
||||
|
||||
| Operation | Native capability |
|
||||
| --- | --- |
|
||||
| `app promptkit list backends` | `Engine.ListBackends` |
|
||||
| `app promptkit list namespaces` | `Engine.ListNamespaces` |
|
||||
| `app promptkit list collections` | `Engine.ListCollections` |
|
||||
|
||||
Keep prompt/profile listings and add native namespace filtering where applicable.
|
||||
These capabilities remain independently optional; one missing inventory does
|
||||
not disable other operations. Reuse the existing construction, registration,
|
||||
typed input/state, configuration selection, and diagnostic machinery.
|
||||
|
||||
Forward native namespace filters and resource-kind selections to their respective
|
||||
APIs. PromptKit performs identity resolution and inventory membership selection;
|
||||
the adapter translates the selected CLI syntax into those native inputs.
|
||||
|
||||
The namespace behavior has two distinct owners:
|
||||
|
||||
- An application loader may return a native `WithNamespaces` engine view. That
|
||||
view controls short caller-ID resolution and default resource listing scopes.
|
||||
- An explicit native listing filter overrides membership selection for one
|
||||
listing call. It does not change the engine view or rewrite configuration.
|
||||
|
||||
Omitted filters retain native defaults: the corresponding resource view scope,
|
||||
or all identities when unscoped. Namespace and collection inventories remain
|
||||
global under the upstream contract. Reports show native identities and versions.
|
||||
OD-3 records the accepted request/API and CLI filter choices.
|
||||
|
||||
### Explicit smoke testing
|
||||
|
||||
Expose PromptKit's `Engine.SmokeTest` through an independently optional
|
||||
`promptkit test` command in this roadmap. Keep request construction and report
|
||||
presentation close to the native public API, reusing the suite's typed
|
||||
input/state, configuration selection, and engine acquisition paths.
|
||||
|
||||
#### Command vocabulary
|
||||
|
||||
The selected syntax is:
|
||||
|
||||
```text
|
||||
app promptkit test KIND [options]
|
||||
app promptkit test all [options]
|
||||
```
|
||||
|
||||
One operand selects either a native kind or the aggregate `all` operation.
|
||||
The native kind spellings are `reachability`, `authentication`, `text_inference`,
|
||||
and `structured_output`. `all` is CommandKit-owned coordination syntax, not a
|
||||
native `SmokeTestKind`.
|
||||
|
||||
For example:
|
||||
|
||||
```text
|
||||
app promptkit test reachability --backend local
|
||||
app promptkit test authentication --profile editorial
|
||||
app promptkit test text_inference --prompt briefing
|
||||
app promptkit test structured_output --profile editorial --model example-model
|
||||
app promptkit test all --profile editorial
|
||||
```
|
||||
|
||||
#### Request bindings
|
||||
|
||||
Bind operator inputs directly to `promptkit.SmokeTestRequest` and its native
|
||||
`ExecutionTargetOverride`:
|
||||
|
||||
| CLI input | Native destination | Binding |
|
||||
| --- | --- | --- |
|
||||
| `KIND` | `Kind` | Native kind string; aggregate expansion is described below. |
|
||||
| `--backend` | `BackendID` | String. |
|
||||
| `--prompt` | `PromptID` | String. |
|
||||
| `--profile` | `ProfileID` | String. |
|
||||
| `--endpoint` | `Execution.Endpoint` | String. |
|
||||
| `--model` | `Execution.Model` | String. |
|
||||
| `--temperature` | `Execution.Temperature` | Optional floating-point value. |
|
||||
| `--max-tokens` | `Execution.MaxTokens` | Optional integer. |
|
||||
| `--top-p` | `Execution.TopP` | Optional floating-point value. |
|
||||
| `--timeout-seconds` | `Execution.TimeoutSeconds` | Optional integer. |
|
||||
| `--service-tier` | `Execution.ServiceTier` | String. |
|
||||
| `--reasoning-effort` | `Execution.ReasoningEffort` | Optional string. |
|
||||
| `--api-key-env` | `Execution.APIKeyEnv` | String. |
|
||||
| `--transient-retries` | `TransientRetries` | Optional integer. |
|
||||
|
||||
Also accept the existing `--config`, `--overlay`, and suite selectors. Reports
|
||||
are YAML-only; `--format` defaults to `yaml` and accepts `yaml`. Built-in flags
|
||||
are singleton bindings with the established last-occurrence behavior; suite
|
||||
selectors retain their declared binding behavior.
|
||||
|
||||
Preserve omission versus explicit zero or empty values for native pointer
|
||||
fields. Leave `Execution` nil when no execution override was supplied. Backend,
|
||||
prompt, and profile selectors may be supplied together: PromptKit owns their
|
||||
precedence, resolution, defaults, ranges, and request validation. CommandKit
|
||||
owns CLI syntax and primitive value conversion.
|
||||
|
||||
Direct `APIKey` and structured `ExtraParams` remain available through the
|
||||
optional application request builder. This scope uses that existing integration
|
||||
pattern rather than adding a credential-source mechanism or generic override
|
||||
language.
|
||||
|
||||
#### Consumer API and request construction
|
||||
|
||||
Use a narrow ckprompt-owned envelope:
|
||||
|
||||
```go
|
||||
type SmokeRequest struct {
|
||||
Selection ckconfig.Selection
|
||||
Request promptkit.SmokeTestRequest
|
||||
}
|
||||
```
|
||||
|
||||
An independently optional `Suite.SmokeTest` callback receives context,
|
||||
`SmokeRequest`, typed application input, and shared state, and returns
|
||||
`(commandkit.Acquired[*promptkit.SmokeTestReport], error)`. It represents one
|
||||
concrete test and preserves both a returned native report and error.
|
||||
|
||||
Provide `WithSmokeTesting(suite, load, build)` following the existing
|
||||
request-validation adapter pattern. The loader uses the existing engine-loader
|
||||
signature. The optional builder receives context, `SmokeRequest`, typed input,
|
||||
and shared state, and returns a complete native `SmokeTestRequest` and error.
|
||||
With no builder, forward the CLI-created native request. With a builder, its
|
||||
returned request is authoritative; do not merge CLI values back over it.
|
||||
|
||||
For aggregate execution, invoke the optional builder separately for each
|
||||
concrete kind, seeding that kind into the otherwise common bound request.
|
||||
Applications can supply per-kind credentials, structured parameters, or other
|
||||
native request choices. The synthetic `all` value does not reach the native
|
||||
request or the per-test builder.
|
||||
|
||||
#### Aggregate execution
|
||||
|
||||
`app promptkit test all` selects these four tests sequentially, in order:
|
||||
|
||||
1. `reachability`
|
||||
2. `authentication`
|
||||
3. `text_inference`
|
||||
4. `structured_output`
|
||||
|
||||
Coordinate the concrete test callbacks within one CommandKit invocation, using
|
||||
its configuration selection, prepared typed input, and shared state. Reuse
|
||||
configuration acquisition and the engine where practical within the existing
|
||||
loader model; this does not require a new caching framework or guarantee a
|
||||
particular loader call count.
|
||||
|
||||
Continue to the next kind after an individual test error so that remaining
|
||||
results can be collected. When the invocation is canceled, stop scheduling
|
||||
further tests and retain the results already available. PromptKit owns each
|
||||
individual operation; CommandKit coordinates the selected sequence.
|
||||
|
||||
#### Reports and invocation outcomes
|
||||
|
||||
Present the native `SmokeTestReport`, including observations, timing, attempts,
|
||||
and available usage. For `all`, deliver one YAML report containing the
|
||||
individual results in execution order, with each selected kind's returned
|
||||
report and associated error, when present. Retain useful partial reports
|
||||
returned with errors. Preserve supported error identity through the invocation
|
||||
outcome as well as presenting the errors.
|
||||
|
||||
For a single test, a nil native error and successful report delivery mean
|
||||
success. For `all`, success requires all selected calls to return without errors
|
||||
and reporting to succeed. Native invocation errors use the normal operational
|
||||
failure classification; cancellation and delivery failures retain their existing
|
||||
outcome handling. Native negative or unknown findings remain reported facts;
|
||||
CommandKit does not derive an additional readiness verdict or failure rule from
|
||||
them.
|
||||
|
||||
Use a focused report-delivery path that preserves a returned report alongside
|
||||
its native error, including cancellation. The existing success-only inspection
|
||||
writer cannot be reused unchanged for that case. Adapt delivery within the
|
||||
current outcome model; this work does not require a general diagnostics
|
||||
framework. Describe returned data neutrally, without adding warnings about
|
||||
perceived upstream limitations.
|
||||
|
||||
Smoke testing runs when its command is explicitly invoked. Configuration,
|
||||
requirements, help, and release operations retain their own existing behavior.
|
||||
OD-4 records its inclusion in the current delivery scope; a separate feature
|
||||
roadmap is unnecessary. The broader deferred proposals in
|
||||
[deferred work](deferred.md) do not defer this selected native capability.
|
||||
|
||||
## Work across the integrations
|
||||
|
||||
### Dependencies and attribution
|
||||
|
||||
Update the owning module requirements and real checksums for the selected
|
||||
releases. PromptKit v0.16.0 drops its bundled backend-catalog dependencies;
|
||||
remove obsolete indirect requirements, sums, and notices when absent from the
|
||||
resulting CommandKit graphs. Do not replace them with implicit catalog loading.
|
||||
|
||||
Keep root and integration attribution entries matched to their actual selected
|
||||
graphs. ConfigKit and PromptKit still declare ReleaseKit v0.4.1 upstream; a
|
||||
CommandKit graph may select v0.4.2 through its own core dependency. Record the
|
||||
effective selection rather than blindly assigning the root tool version to
|
||||
every module. Verify pinned license/notice source links for each adopted release.
|
||||
|
||||
Core remains independent of optional ConfigKit/PromptKit dependencies, including
|
||||
through examples, tests, and tools. Committed builds use released public modules,
|
||||
not adjacent checkout replacements.
|
||||
|
||||
### Documentation and compatibility
|
||||
|
||||
Update GoDoc, the CLI reference, consumer guidance, maintained examples, internal
|
||||
inventory, and release-tool references as their owning implementations land.
|
||||
Document changed prompt configuration, ID-only selection, optional label data,
|
||||
source evidence, new discovery behavior, and smoke testing under their canonical
|
||||
owners.
|
||||
|
||||
The accepted architecture already supports independently optional operations and
|
||||
native-library authority. Adding these optional capabilities and following
|
||||
upstream selection semantics fit those accepted boundaries. Any concrete new
|
||||
architectural decision is evaluated under the ADR policy.
|
||||
|
||||
Retain historical release records. Explain incompatible ckprompt changes in an
|
||||
appropriately versioned future module release. New source-report fields and
|
||||
catalog and smoke capabilities also need accurate CLI/API release notes. This
|
||||
roadmap does not require a CommandKit version to mirror any upstream version.
|
||||
|
||||
### Implementation evidence and testing boundary
|
||||
|
||||
The [completed implementation plan](implementation.md#stage-18-verify-the-complete-target-and-finish)
|
||||
records the delivered target and its workspace/independent-graph validation.
|
||||
The [remediation record](remediation.md) tracks the focused follow-up review.
|
||||
Testing remains directed to CommandKit-owned interfaces and adaptation under the
|
||||
[upstream testing ownership policy](../policy/testing.md#upstream-integration-testing-ownership).
|
||||
Verify that command inputs and context reach the intended native APIs, including
|
||||
ID-only prompt requests, default/explicit listing filters, and smoke requests.
|
||||
Verify faithful representation of upstream-returned source evidence, optional
|
||||
labels, inventories, diagnostics, and errors, together with CommandKit's optional
|
||||
command availability, invocation lifetimes, outcome classification, and output
|
||||
delivery. Include smoke request bindings, per-kind builder forwarding, aggregate
|
||||
order and continuation, cancellation, report-plus-error delivery, and error
|
||||
identity in this boundary coverage.
|
||||
|
||||
These checks establish CommandKit's access to the capabilities. They should not
|
||||
test whether ConfigKit correctly places or composes sources, checks versions,
|
||||
or classifies sensitivity, nor whether PromptKit correctly resolves namespaces,
|
||||
discovers requirements, or validates concrete requests. Those domain tests belong
|
||||
to the upstream libraries. Real upstream fixtures and maintained offline
|
||||
consumers can exercise CommandKit's boundary without duplicating native feature
|
||||
test matrices. Existing release-launcher coverage likewise owns CommandKit's
|
||||
tool invocation, not ReleaseKit's publication algorithms.
|
||||
|
||||
Verify all affected modules both in the local workspace and with their own
|
||||
readonly dependency graphs. Temporary replacements for unpublished CommandKit
|
||||
prerequisites may support development checks; they are not evidence for a
|
||||
sibling-replaced upstream dependency or committed build requirements. The
|
||||
[developer guide](../development.md) owns the established module-check practices.
|
||||
|
||||
New tests should protect selected contracts without requiring incidental
|
||||
factory activity after failure, complete diagnostic text snapshots, YAML key
|
||||
ordering, or arbitrary copies of caller-owned objects. The migration, discovery,
|
||||
and smoke adapter checks can use offline fixtures and test doubles; smoke tests
|
||||
verify request forwarding and report/error adaptation, leaving provider behavior
|
||||
to PromptKit.
|
||||
|
||||
## Accepted Decisions
|
||||
|
||||
### OD-1: Representation of the added source evidence
|
||||
|
||||
**Accepted:** Extend both existing source formats with `destination` using
|
||||
ConfigKit's canonical JSON Pointer string (root is `""`), `placement` using native
|
||||
mode names, `destination_unresolved`, and `version_check` corresponding to
|
||||
`SourceInfo.VersionCheck`.
|
||||
|
||||
Preserve the native nested evidence: path, version presence/null/text, optional
|
||||
source location, and redaction flag. Use native sanitized values and maximize
|
||||
API-to-report correspondence. Field descriptions explain the returned data
|
||||
neutrally, without promising additional checker behavior or warning about
|
||||
perceived upstream limitations.
|
||||
|
||||
### OD-2: Optional versions in reports
|
||||
|
||||
**Accepted:** Report a prompt or collection's version as `version`. Omit this
|
||||
optional field when the native value is absent; retain `""` when it is present
|
||||
and empty. For collections, this projects the native `ReleaseVersion` value.
|
||||
Use the same naming and presence treatment across the affected reports.
|
||||
|
||||
Version metadata retains its upstream meaning. ID-only selection and source
|
||||
configuration headers remain governed by their respective native APIs.
|
||||
|
||||
### OD-3: Catalog requests and namespace-filter syntax
|
||||
|
||||
**Accepted:** Use narrow ckprompt-owned request envelopes carrying configuration
|
||||
selection and native `promptkit.NamespaceFilter` for prompt, profile, and backend
|
||||
catalogs. Give namespace enumeration its native resource-kind selection, and
|
||||
leave collection enumeration aligned with its unfiltered native API. Revise
|
||||
existing catalog callback shapes coherently rather than retaining parallel
|
||||
filtered/unfiltered variants solely for the prototype API.
|
||||
|
||||
For applicable listing commands, map operator input directly to native filters:
|
||||
|
||||
| Operator input | Native filter mode |
|
||||
| --- | --- |
|
||||
| No namespace option | `NamespaceDefault` |
|
||||
| `--namespace NAME` | `NamespaceNamed` with the supplied name |
|
||||
| `--namespace ""` | `NamespaceUnqualified` |
|
||||
| `--all-namespaces` | `NamespaceAll` |
|
||||
|
||||
Existing flag-presence tracking distinguishes omission from an explicitly empty
|
||||
namespace. These choices need no separate `--unqualified` flag or reserved
|
||||
namespace name such as `all`. Conflicting namespace choices are usage errors:
|
||||
one native listing accepts at most one namespace filter.
|
||||
|
||||
Namespace enumeration uses repeatable `--kind prompt|profile|backend`; omission
|
||||
retains the native union. Application-created engine views and explicit listing
|
||||
filters retain their separate native meanings.
|
||||
|
||||
### OD-4: Delivery boundary for smoke testing
|
||||
|
||||
**Accepted:** Include an independently optional smoke command in this roadmap,
|
||||
alongside dependency migration and expanded discovery. Expose the native request,
|
||||
operation, and report directly through the existing integration model.
|
||||
|
||||
Deliver native reports, including a report returned with an error. Successful
|
||||
native invocation and report delivery yield success; native errors remain
|
||||
operational failures with supported error identity preserved. Native observations
|
||||
are reported neutrally. CommandKit does not add an aggregate readiness judgment
|
||||
or warnings about perceived upstream limitations.
|
||||
|
||||
The [explicit smoke testing target](#explicit-smoke-testing) records the accepted
|
||||
`promptkit test KIND` and `promptkit test all` vocabulary, native request
|
||||
bindings, per-kind builder, and aggregate execution/reporting semantics. A
|
||||
separate smoke feature roadmap is not required.
|
||||
Reference in New Issue
Block a user