base-layer/commandkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Expand operator roadmap with ConfigKit CLI features

Browse Source
This commit is contained in:
Eric Rakestraw
2026-11-06 03:03:31 +00:00
parent 42af12cb0e
commit 6830b96821
2 changed files with 222 additions and 23 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
docs/roadmap/future.md
View File

@@ -36,7 +36,7 @@ The published ckprompt v0.2.0 retains its earlier dependencies; see the
### PromptKit v0.19.0 adoption
The [PromptKit v0.19.0 roadmap](promptkit-v0.19.0.md) owns the proposed adoption,
operator CLI additions, shared configuration inspection work, open decisions,
operator CLI additions for PromptKit and ConfigKit, open decisions,
and completion checklist. Scope is agreed in principle and remains subject to
roadmap review. Implementation has not started. Use released upstream contracts
only; do not inspect or monitor unreleased work in anticipation of adoption.
@@ -45,32 +45,35 @@ only; do not inspect or monitor unreleased work in anticipation of adoption.
### Comparison extensions
Independent per-side domain selectors and an optional assertion mode remain
unselected extensions to the delivered ConfigKit comparison. Consider them only
for a concrete consumer need and a deliberate API/behavior decision under the
relevant ADRs.
Independent per-side domain selectors and opt-in comparison assertions are now
included in the [working roadmap](promptkit-v0.19.0.md#additional-configkit-operator-workflows).
Their binding and outcome contracts remain open under roadmap OD-9 and OD-10;
retaining those proposals does not change current comparison behavior.
### Configuration inspection extensions
The [PromptKit v0.19.0 roadmap](promptkit-v0.19.0.md#shared-configuration-capabilities)
now includes the shared operator tasks previously retained here: metadata/path
discovery and explicitness, failed-load source inventory, detailed comparison,
and structured diagnostics. Their earlier ConfigKit adoption identifiers were
OD-3 through OD-6; the new roadmap has its own decision numbering. Implementation
and structured diagnostics. It also includes focused effective display,
source-ID selection, metadata/diagnostic filters, independent comparison
selectors, and assertion mode. The original four shared proposals had ConfigKit
adoption identifiers OD-3 through OD-6; the new roadmap has its own decision numbering. Implementation
belongs in `ckconfig`, with PromptKit exposing native evidence through its
configuration scope. CLI syntax and report contracts remain subject to review.
Additional unselected work:
- **Source-ID filtering.** Consider selecting a record already available in the
source inventory. Native `Source(id)` lookup is a convenience; this proposal
need not introduce another command or require a lookup method from every
custom source provider.
Transient acquisition overrides are an explicit open scope decision in
[roadmap OD-11](promptkit-v0.19.0.md#od-11-transient-configuration-overrides),
with a recommendation for opt-in CLI support and alternatives using
application-specific flags or a structured override document. Persistent
configuration editing remains deferred.
Discovery, import defaults, collection identities, and field-level resolution
recording remain available through application loaders. Standard inspection of
resource contexts or secrets-directory bootstrap has no immediate proposal;
those capabilities primarily serve application loading and startup.
recording remain available through application loaders. Discovery tracing,
resource-context inspection, and secrets-directory bootstrap commands remain
deferred. Acquired source records do not provide a complete trace of rejected
discovery candidates; bootstrap and resource contexts primarily serve
application loading and startup.
### PromptKit inspection and selection extensions
@@ -140,8 +143,8 @@ operator or consumer need and an appropriate ownership/behavior decision:
- JSON reports and output-file management or reloadable export.
- Shell completion extensions, including alias discovery described above and
dynamic discovery of configured or remote values.
- Configuration mutation and generic value assignment. Field metadata and
provenance inspection are scoped under
- Persistent configuration mutation. Transient CLI override scope is owned by
roadmap OD-11; field metadata and provenance inspection are scoped under
[configuration inspection extensions](#configuration-inspection-extensions).
- Credential diagnostics beyond PromptKit's native smoke checks and additional
live provider probes.

206
docs/roadmap/promptkit-v0.19.0.md
View File

@@ -1,4 +1,4 @@
# PromptKit v0.19.0 adoption and operator CLI
# PromptKit v0.19.0 adoption and ConfigKit operator CLI
Status: proposed roadmap for review. Scope is agreed in principle; implementation
has not started. Command spellings, report designs, and API changes below are
@@ -8,7 +8,7 @@ recommendations until the open decisions are resolved.
Adopt released PromptKit v0.19.0 and expose useful operator tasks through
standard CommandKit commands, flags, and reports. This roadmap covers CLI
exposure, including shared configuration inspection in `ckconfig`. It does not
exposure, including ConfigKit v1.1.0 operator features in `ckconfig`. It does not
turn `ckprompt` into a convenience library for application-internal PromptKit
operations. Applications continue to own engine construction, domain inputs,
credentials, execution, and their executable lifecycle.
@@ -45,6 +45,16 @@ and [smoke API](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/
Work against released versions only; do not inspect or monitor unreleased
upstream work in anticipation of another adoption.
ConfigKit v1.1.0 is already adopted at released commit
`191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9`. Its
[inspection API](https://forge.maximumdirect.net/base-layer/configkit/src/commit/191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9/inspection.go),
[metadata API](https://forge.maximumdirect.net/base-layer/configkit/src/commit/191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9/metadata.go),
and [configuration reference](https://forge.maximumdirect.net/base-layer/configkit/src/commit/191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9/docs/config.md)
support the additional CLI work below without an upstream dependency upgrade.
Its matched history for registered records moving within lists flows through
native inspection; it does not introduce identity-based CLI paths or
element-level effective diffs.
Current `ckprompt` supplies prompt/profile inspection, requirements discovery,
concrete request validation, prompt/profile/backend/namespace/collection
catalogs, and four smoke operations plus `test all`. Smoke commands already
@@ -62,6 +72,12 @@ are [ckconfig contracts](../../integrations/ckconfig/contracts.go),
[ConfigKit adapters](../../integrations/ckconfig/configkit_adapter.go), and
[PromptKit adapters](../../integrations/ckprompt/adapters.go).
Effective display currently writes the whole configuration. Sources are reported
as a complete inventory. Comparison accepts independent config/overlay choices,
but applies the same application selectors to both typed inputs and succeeds
when a comparison reports differences. The new tasks extend those boundaries;
they do not replace already-delivered explanation or smoke functionality.
## Targeted end state
The original adoption tasks remain included:
@@ -84,6 +100,17 @@ the decisions at the end of this document.
| Find configuration paths and inspect their explicitness | `config paths --scope promptkit`, with historical selection; metadata alongside path explanation | `ckconfig`, wired through `ckprompt.ConfigScope`: observed paths, effective presence, explicitness/null/origin evidence, and available history. |
| Understand both sides of configuration changes | `config diff --scope promptkit --details` | `ckconfig`: native paths, left/right snapshots, presence/redaction, and hidden-change evidence. |
| Investigate a failed configuration load | `config sources --scope promptkit` on failure; a structured diagnostics operation | `ckconfig`, with both native adapters: acquired source inventory and native codes, severities, paths, and locations while retaining the load failure. |
| Inspect one effective value or subtree | `config show --scope app --path /server` | `ckconfig`: the selected native effective snapshot, retaining redaction and absence/null distinctions. |
| Follow a source reference | `config sources --source-id ID` | `ckconfig`: the matching acquired source record, including evidence available after failure. |
| Compare different application selections | For example, `config diff --campaign spring --against-campaign autumn` | `ckconfig`: independently prepared typed inputs passed to the existing native comparison provider. |
| Detect configuration differences in scripts | `config diff --check` | `ckconfig`: a deliberate difference outcome based on native records, alongside the selected comparison report. |
| Narrow an inspection to relevant evidence | Subtree/explicitness filters on `config paths`; severity/code filters on `config diagnostics` | `ckconfig`: filtered native evidence with source references retained. |
Transient configuration overrides are an open scope decision under OD-11, with
a recommendation to include them as an optional loader capability. Illustrative
`config validate --set /workers=0` and `config show --set /server/port=8080`
would apply explicit overrides during acquisition without editing source files.
Their inclusion and exact CLI/API contract remain for review.
### Compatibility and native semantics
@@ -180,6 +207,50 @@ cancellation with partial evidence explicitly without promising interruption of
an arbitrary writer or rollback of already-delivered bytes. OD-6 selects the
structured diagnostics command and its interaction with ordinary diagnostics.
### Additional ConfigKit operator workflows
**Focused effective display.** Use the effective snapshot from native
`Explain(path)` for path-selected display. The ordinary whole-document path
continues to use `WriteEffective`. A historical-only path can be explainable
while lacking an effective value; distinguish that case from a present YAML
null. Select the missing-value behavior, provider availability, and aggregate
scope applicability under OD-8. Do not read mutable `Result.Config` or source
files to reconstruct inspection values.
**Source selection and report filters.** Select source IDs from the acquired
inventory, preserving existing record fields and upstream ordering. Native
`Source(id)` is a convenience, not an additional provider requirement. IDs are
local to a load and do not guarantee the same source in a later acquisition.
Apply subtree filtering to parsed configuration paths, not raw string prefixes
that would confuse `/server` with `/server2`. Specify explicitness filtering for
historical-only paths without treating unavailable effective metadata as false.
Diagnostic filters affect the presented records, not the underlying acquisition
outcome; retain the source references needed to interpret selected records.
**Independent comparison selections.** Extend the per-side typed-input design
in [ADR 0012](../adr/0012-typed-configuration-comparison-pairs.md). Capture
applicable selector bindings for each side, prepare and check both inputs before
comparison, and preserve the existing shared invocation state. Resolve omitted
right selectors, explicit empty values, repeated selectors, and flag-name
collisions under OD-9. Domain interpretation and loader caching remain with the
application; the comparison provider still receives two selections.
**Comparison assertions.** Base a difference outcome on native record presence,
including hidden changes whose redacted snapshots look equal. Equal effective
values can have different histories or explicitness. Comparison also does not
detect changed credential bytes behind an unchanged secret reference. Define
the opt-in outcome and report-delivery behavior under OD-10; retain the current
ordinary diff outcome. A load or comparison failure is not evidence of equality.
**Transient override candidate.** If selected in OD-11, parse explicitly supplied
CLI values into native `Override` records and pass them through a declared loader
capability. ConfigKit owns override path validity, supported values, precedence,
decoding, resolution, provenance, and validation. Its configuration overrides
recursively overlay mappings and replace sequences; this differs from the
PromptKit execution override's complete provider-map replacement. Do not reuse
one grammar implementation in a way that erases that distinction. Source-load
errors remain errors even if an override would replace the affected value.
## Work sequence
1. **Resolve the significant CLI choices.** Review the open decisions, record
@@ -196,7 +267,11 @@ structured diagnostics command and its interaction with ordinary diagnostics.
4. **Implement shared configuration inspection.** Add metadata discovery and
explicitness, detailed diff presentation, source evidence on failure, and
structured diagnostics in `ckconfig`; wire native PromptKit facets through
`ConfigScope`. Keep each change reviewable with its boundary coverage.
`ConfigScope`. Add focused effective display, source selection, and bounded
metadata/diagnostic filters using those evidence capabilities. Extend typed
comparison selection and add the selected assertion outcome. If OD-11 is
accepted, implement explicit override acquisition and comparison-side wiring
as a separate reviewable unit. Keep boundary coverage with each change.
5. **Update operator and consumer documentation.** The CLI reference owns
syntax, report schemas, availability, and outcomes; GoDoc owns callback and
value contracts. Extend maintained simple/composed examples to demonstrate
@@ -235,6 +310,21 @@ Default tests remain offline and independent of real credentials.
available history on absent lookup, and projects native field/change records.
- [ ] Detailed diff retains both snapshots and presence/redaction/hidden-change
evidence, including empty comparisons and sequence-level changes.
- [ ] Focused display preserves native snapshot content and distinguishes
missing/historical-only paths from explicit null. Whole-document display
retains its current contract.
- [ ] Source-ID selection works on successful and failed acquisitions. Path,
explicitness, severity, and code filters preserve the selected evidence and
source references without masking the acquisition outcome.
- [ ] Independent comparison selectors preserve the chosen inheritance and
repeatable-value rules, side-specific checks, and shared-state lifecycle.
- [ ] Assertion mode distinguishes equality, native differences (including
hidden changes), usage errors, and operational failures under the selected
outcome contract. Report-delivery failures remain observable.
- [ ] OD-11 has an explicit scope decision. If included, only supplied CLI
overrides reach opted-in loaders, with typed values, safe flag labels,
selected comparison-side behavior, native errors, and returned provenance
preserved. No tests independently re-establish ConfigKit override semantics.
- [ ] Source and diagnostic evidence can be reported with an acquisition error;
the original failure, unavailable-versus-empty evidence, cancellation, and
report-delivery failures remain distinguishable.
@@ -273,9 +363,12 @@ Focused backend inspection remains in [future work](future.md#promptkit-inspecti
until a released upstream API supports it. Do not reconstruct backend selection
from configuration to deliver it in this roadmap. General execution, prepared
handle lifecycle, sessions, and observer convenience APIs remain application
responsibilities. Configuration mutation, new live probes, dynamic completion,
responsibilities. Persistent configuration editing, new live probes, dynamic completion,
JSON report formats, and unrelated deferred proposals are not added by this
roadmap. A later ConfigKit review may propose further CLI tasks separately.
roadmap. Transient acquisition overrides remain an explicit choice in OD-11.
Discovery tracing and secrets-directory bootstrap commands remain deferred:
the source inventory is not a trace of rejected search candidates, and bootstrap
performs application startup effects.
## Open Decisions
@@ -396,3 +489,106 @@ available sooner, but creates an extra consumer upgrade and does not complete
this roadmap's targeted end state. A disposable local module replacement can
help test intermediate changes, but cannot substitute for final validation of
the declared released dependency.
### OD-8: Focused configuration reports and filtering
**Recommendation:** extend existing commands: `config show --path POINTER` and
`config sources --source-id ID`; add subtree/explicitness filters to
`config paths` and severity/code filters to `config diagnostics`. Keep full reports as
the default. Use the existing explanation capability for selected effective
YAML, and recommend an operational failure for an absent effective path or
unknown source ID, consistent with unknown-path explanation. Recommend starting
path-selected display with one scope; whole-document `--scope all` remains
available. Specify presence-aware explicitness filtering so historical-only
entries are not labeled implicitly defined. These choices reuse existing
operator vocabulary and native evidence with little additional acquisition work.
**Alternative:** introduce a `config get --path POINTER` command with a snapshot
envelope containing `present`, `redacted`, and `yaml`, and return absence as
report data. This is convenient for scripts and preserves unknown/null/absence
distinctions without relying on exit status, but adds another command/report
contract. Aggregate path-selected reports are also viable if scope-labeled
per-scope absence and delivery semantics are designed. Filters can be deferred
in favor of downstream YAML tools, reducing CLI surface but making routine
operator inspection more cumbersome.
### OD-9: Independent comparison selector bindings
**Recommendation:** let applications declare which domain selectors support
right-side comparison values, with explicit metadata for their `--against-*`
spellings. Omission inherits the left value and presence; an explicit right
value replaces it. For repeatable selectors, recommend replacing the inherited
collection with the supplied right collection. This follows config/overlay
selection conventions and supports comparisons such as two campaigns without
requiring different files. Review the selected refinement of ADR 0012 and flag
collision handling before implementation; exact helper signatures remain open.
**Alternative:** require symmetric explicit left/right selectors for opted-in
bindings. This makes each side independent and easier to reason about when
defaults are complex, but adds repetition and departs from the existing
right-inherits-left convention. Either design keeps application domain parsing
in its declared bindings and avoids inventing a generic selector language.
### OD-10: Comparison assertion outcomes
**Recommendation:** add opt-in `--check`, retaining normal report delivery and
ordinary diff behavior when omitted. Recommend status `0` for equality and a
domain difference status `3`; retain current usage `2` and operational `1`
classification. This lets scripts distinguish differences from failed
comparison. The status is a proposal, not an existing contract. Apply existing
reporting outcome rules, including preservation of a nonzero domain outcome
when presentation also fails; callers can inspect the separate presentation
evidence. Document that interaction explicitly.
**Alternative:** add a separate `config compare` assertion command, leaving
`diff` exclusively informational. This makes the intent visible in the command
path but duplicates selection/report options. Another viable choice is status
`1` for both differences and operational failures, which fits simple shell
checks but requires additional evidence to distinguish the two conditions.
### OD-11: Transient configuration overrides
**Recommendation:** include a declared, opt-in override capability for standard
configuration commands, with repeatable `--set POINTER=JSON` values passed to
application acquisition as native ConfigKit overrides. This enables validation,
inspection, and comparison of temporary settings through consistent CLI syntax.
Providers that do not accept overrides retain their existing operations. No
source file is edited, and no general application-internal convenience API is
required. The loader contract must state how it combines these records with
any existing application-owned overrides; the generic adapter cannot inject
them into an arbitrary loader after the load has completed.
If selected, settle these details before implementation:
- Split at the first `=`; JSON supplies typed null, boolean, string, numeric,
array, and object values. Specify numeric conversion to ConfigKit-supported
built-in forms rather than passing `json.Number` or silently promising
arbitrary numeric precision. An unambiguous alternate input form may be
needed for pointer keys containing `=`.
- Forward explicit records only, with safe display labels such as `--set`,
never raw value-bearing arguments as provenance labels. Preserve native
duplicate/overlap handling rather than silently collapsing assignments.
- Follow the released mapping-path constraints, including root/index and
schema-version restrictions, without extending them into a generic patch
language. Empty, false, zero, and null remain intentional supplied values.
- For comparison, recommend `--set` as the shared baseline and
`--against-set` as a right-side replacement list when supplied, following
OD-9's proposed repeatable-selector rule. Define an explicit way to select
an empty right list before finalizing this grammar; appending or replacing
individual paths would need a different selected rule.
- Reflect native CLI origins and subsequent resolution in existing reports.
Test CommandKit parsing, forwarding, and outcome/report adaptation, leaving
ConfigKit's override algorithm to its own tests.
**Alternative A:** retain application-specific typed flags and loader translation
to `Options.Overrides`. This gives applications tailored vocabulary and avoids
generic JSON numeric/path syntax, but repeats CLI infrastructure and prevents
uniform temporary-setting workflows across applications. This is a viable
deferral if the generic acquisition contract is too disruptive for this sprint.
**Alternative B:** accept a structured override document containing path/value
records instead of inline `--set` assignments. It handles complex values and
unusual pointer keys naturally and makes an empty comparison-side list explicit,
but adds file-input handling and is less convenient for one-value shell changes.
It still needs deliberate typed-value decoding and loader opt-in. Neither
alternative authorizes persistent configuration mutation.