Expand operator roadmap with ConfigKit CLI features
This commit is contained in:
@@ -1,4 +1,4 @@
|
||||
# PromptKit v0.19.0 adoption and operator CLI
|
||||
# PromptKit v0.19.0 adoption and ConfigKit operator CLI
|
||||
|
||||
Status: proposed roadmap for review. Scope is agreed in principle; implementation
|
||||
has not started. Command spellings, report designs, and API changes below are
|
||||
@@ -8,7 +8,7 @@ recommendations until the open decisions are resolved.
|
||||
|
||||
Adopt released PromptKit v0.19.0 and expose useful operator tasks through
|
||||
standard CommandKit commands, flags, and reports. This roadmap covers CLI
|
||||
exposure, including shared configuration inspection in `ckconfig`. It does not
|
||||
exposure, including ConfigKit v1.1.0 operator features in `ckconfig`. It does not
|
||||
turn `ckprompt` into a convenience library for application-internal PromptKit
|
||||
operations. Applications continue to own engine construction, domain inputs,
|
||||
credentials, execution, and their executable lifecycle.
|
||||
@@ -45,6 +45,16 @@ and [smoke API](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/
|
||||
Work against released versions only; do not inspect or monitor unreleased
|
||||
upstream work in anticipation of another adoption.
|
||||
|
||||
ConfigKit v1.1.0 is already adopted at released commit
|
||||
`191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9`. Its
|
||||
[inspection API](https://forge.maximumdirect.net/base-layer/configkit/src/commit/191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9/inspection.go),
|
||||
[metadata API](https://forge.maximumdirect.net/base-layer/configkit/src/commit/191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9/metadata.go),
|
||||
and [configuration reference](https://forge.maximumdirect.net/base-layer/configkit/src/commit/191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9/docs/config.md)
|
||||
support the additional CLI work below without an upstream dependency upgrade.
|
||||
Its matched history for registered records moving within lists flows through
|
||||
native inspection; it does not introduce identity-based CLI paths or
|
||||
element-level effective diffs.
|
||||
|
||||
Current `ckprompt` supplies prompt/profile inspection, requirements discovery,
|
||||
concrete request validation, prompt/profile/backend/namespace/collection
|
||||
catalogs, and four smoke operations plus `test all`. Smoke commands already
|
||||
@@ -62,6 +72,12 @@ are [ckconfig contracts](../../integrations/ckconfig/contracts.go),
|
||||
[ConfigKit adapters](../../integrations/ckconfig/configkit_adapter.go), and
|
||||
[PromptKit adapters](../../integrations/ckprompt/adapters.go).
|
||||
|
||||
Effective display currently writes the whole configuration. Sources are reported
|
||||
as a complete inventory. Comparison accepts independent config/overlay choices,
|
||||
but applies the same application selectors to both typed inputs and succeeds
|
||||
when a comparison reports differences. The new tasks extend those boundaries;
|
||||
they do not replace already-delivered explanation or smoke functionality.
|
||||
|
||||
## Targeted end state
|
||||
|
||||
The original adoption tasks remain included:
|
||||
@@ -84,6 +100,17 @@ the decisions at the end of this document.
|
||||
| Find configuration paths and inspect their explicitness | `config paths --scope promptkit`, with historical selection; metadata alongside path explanation | `ckconfig`, wired through `ckprompt.ConfigScope`: observed paths, effective presence, explicitness/null/origin evidence, and available history. |
|
||||
| Understand both sides of configuration changes | `config diff --scope promptkit --details` | `ckconfig`: native paths, left/right snapshots, presence/redaction, and hidden-change evidence. |
|
||||
| Investigate a failed configuration load | `config sources --scope promptkit` on failure; a structured diagnostics operation | `ckconfig`, with both native adapters: acquired source inventory and native codes, severities, paths, and locations while retaining the load failure. |
|
||||
| Inspect one effective value or subtree | `config show --scope app --path /server` | `ckconfig`: the selected native effective snapshot, retaining redaction and absence/null distinctions. |
|
||||
| Follow a source reference | `config sources --source-id ID` | `ckconfig`: the matching acquired source record, including evidence available after failure. |
|
||||
| Compare different application selections | For example, `config diff --campaign spring --against-campaign autumn` | `ckconfig`: independently prepared typed inputs passed to the existing native comparison provider. |
|
||||
| Detect configuration differences in scripts | `config diff --check` | `ckconfig`: a deliberate difference outcome based on native records, alongside the selected comparison report. |
|
||||
| Narrow an inspection to relevant evidence | Subtree/explicitness filters on `config paths`; severity/code filters on `config diagnostics` | `ckconfig`: filtered native evidence with source references retained. |
|
||||
|
||||
Transient configuration overrides are an open scope decision under OD-11, with
|
||||
a recommendation to include them as an optional loader capability. Illustrative
|
||||
`config validate --set /workers=0` and `config show --set /server/port=8080`
|
||||
would apply explicit overrides during acquisition without editing source files.
|
||||
Their inclusion and exact CLI/API contract remain for review.
|
||||
|
||||
### Compatibility and native semantics
|
||||
|
||||
@@ -180,6 +207,50 @@ cancellation with partial evidence explicitly without promising interruption of
|
||||
an arbitrary writer or rollback of already-delivered bytes. OD-6 selects the
|
||||
structured diagnostics command and its interaction with ordinary diagnostics.
|
||||
|
||||
### Additional ConfigKit operator workflows
|
||||
|
||||
**Focused effective display.** Use the effective snapshot from native
|
||||
`Explain(path)` for path-selected display. The ordinary whole-document path
|
||||
continues to use `WriteEffective`. A historical-only path can be explainable
|
||||
while lacking an effective value; distinguish that case from a present YAML
|
||||
null. Select the missing-value behavior, provider availability, and aggregate
|
||||
scope applicability under OD-8. Do not read mutable `Result.Config` or source
|
||||
files to reconstruct inspection values.
|
||||
|
||||
**Source selection and report filters.** Select source IDs from the acquired
|
||||
inventory, preserving existing record fields and upstream ordering. Native
|
||||
`Source(id)` is a convenience, not an additional provider requirement. IDs are
|
||||
local to a load and do not guarantee the same source in a later acquisition.
|
||||
Apply subtree filtering to parsed configuration paths, not raw string prefixes
|
||||
that would confuse `/server` with `/server2`. Specify explicitness filtering for
|
||||
historical-only paths without treating unavailable effective metadata as false.
|
||||
Diagnostic filters affect the presented records, not the underlying acquisition
|
||||
outcome; retain the source references needed to interpret selected records.
|
||||
|
||||
**Independent comparison selections.** Extend the per-side typed-input design
|
||||
in [ADR 0012](../adr/0012-typed-configuration-comparison-pairs.md). Capture
|
||||
applicable selector bindings for each side, prepare and check both inputs before
|
||||
comparison, and preserve the existing shared invocation state. Resolve omitted
|
||||
right selectors, explicit empty values, repeated selectors, and flag-name
|
||||
collisions under OD-9. Domain interpretation and loader caching remain with the
|
||||
application; the comparison provider still receives two selections.
|
||||
|
||||
**Comparison assertions.** Base a difference outcome on native record presence,
|
||||
including hidden changes whose redacted snapshots look equal. Equal effective
|
||||
values can have different histories or explicitness. Comparison also does not
|
||||
detect changed credential bytes behind an unchanged secret reference. Define
|
||||
the opt-in outcome and report-delivery behavior under OD-10; retain the current
|
||||
ordinary diff outcome. A load or comparison failure is not evidence of equality.
|
||||
|
||||
**Transient override candidate.** If selected in OD-11, parse explicitly supplied
|
||||
CLI values into native `Override` records and pass them through a declared loader
|
||||
capability. ConfigKit owns override path validity, supported values, precedence,
|
||||
decoding, resolution, provenance, and validation. Its configuration overrides
|
||||
recursively overlay mappings and replace sequences; this differs from the
|
||||
PromptKit execution override's complete provider-map replacement. Do not reuse
|
||||
one grammar implementation in a way that erases that distinction. Source-load
|
||||
errors remain errors even if an override would replace the affected value.
|
||||
|
||||
## Work sequence
|
||||
|
||||
1. **Resolve the significant CLI choices.** Review the open decisions, record
|
||||
@@ -196,7 +267,11 @@ structured diagnostics command and its interaction with ordinary diagnostics.
|
||||
4. **Implement shared configuration inspection.** Add metadata discovery and
|
||||
explicitness, detailed diff presentation, source evidence on failure, and
|
||||
structured diagnostics in `ckconfig`; wire native PromptKit facets through
|
||||
`ConfigScope`. Keep each change reviewable with its boundary coverage.
|
||||
`ConfigScope`. Add focused effective display, source selection, and bounded
|
||||
metadata/diagnostic filters using those evidence capabilities. Extend typed
|
||||
comparison selection and add the selected assertion outcome. If OD-11 is
|
||||
accepted, implement explicit override acquisition and comparison-side wiring
|
||||
as a separate reviewable unit. Keep boundary coverage with each change.
|
||||
5. **Update operator and consumer documentation.** The CLI reference owns
|
||||
syntax, report schemas, availability, and outcomes; GoDoc owns callback and
|
||||
value contracts. Extend maintained simple/composed examples to demonstrate
|
||||
@@ -235,6 +310,21 @@ Default tests remain offline and independent of real credentials.
|
||||
available history on absent lookup, and projects native field/change records.
|
||||
- [ ] Detailed diff retains both snapshots and presence/redaction/hidden-change
|
||||
evidence, including empty comparisons and sequence-level changes.
|
||||
- [ ] Focused display preserves native snapshot content and distinguishes
|
||||
missing/historical-only paths from explicit null. Whole-document display
|
||||
retains its current contract.
|
||||
- [ ] Source-ID selection works on successful and failed acquisitions. Path,
|
||||
explicitness, severity, and code filters preserve the selected evidence and
|
||||
source references without masking the acquisition outcome.
|
||||
- [ ] Independent comparison selectors preserve the chosen inheritance and
|
||||
repeatable-value rules, side-specific checks, and shared-state lifecycle.
|
||||
- [ ] Assertion mode distinguishes equality, native differences (including
|
||||
hidden changes), usage errors, and operational failures under the selected
|
||||
outcome contract. Report-delivery failures remain observable.
|
||||
- [ ] OD-11 has an explicit scope decision. If included, only supplied CLI
|
||||
overrides reach opted-in loaders, with typed values, safe flag labels,
|
||||
selected comparison-side behavior, native errors, and returned provenance
|
||||
preserved. No tests independently re-establish ConfigKit override semantics.
|
||||
- [ ] Source and diagnostic evidence can be reported with an acquisition error;
|
||||
the original failure, unavailable-versus-empty evidence, cancellation, and
|
||||
report-delivery failures remain distinguishable.
|
||||
@@ -273,9 +363,12 @@ Focused backend inspection remains in [future work](future.md#promptkit-inspecti
|
||||
until a released upstream API supports it. Do not reconstruct backend selection
|
||||
from configuration to deliver it in this roadmap. General execution, prepared
|
||||
handle lifecycle, sessions, and observer convenience APIs remain application
|
||||
responsibilities. Configuration mutation, new live probes, dynamic completion,
|
||||
responsibilities. Persistent configuration editing, new live probes, dynamic completion,
|
||||
JSON report formats, and unrelated deferred proposals are not added by this
|
||||
roadmap. A later ConfigKit review may propose further CLI tasks separately.
|
||||
roadmap. Transient acquisition overrides remain an explicit choice in OD-11.
|
||||
Discovery tracing and secrets-directory bootstrap commands remain deferred:
|
||||
the source inventory is not a trace of rejected search candidates, and bootstrap
|
||||
performs application startup effects.
|
||||
|
||||
## Open Decisions
|
||||
|
||||
@@ -396,3 +489,106 @@ available sooner, but creates an extra consumer upgrade and does not complete
|
||||
this roadmap's targeted end state. A disposable local module replacement can
|
||||
help test intermediate changes, but cannot substitute for final validation of
|
||||
the declared released dependency.
|
||||
|
||||
### OD-8: Focused configuration reports and filtering
|
||||
|
||||
**Recommendation:** extend existing commands: `config show --path POINTER` and
|
||||
`config sources --source-id ID`; add subtree/explicitness filters to
|
||||
`config paths` and severity/code filters to `config diagnostics`. Keep full reports as
|
||||
the default. Use the existing explanation capability for selected effective
|
||||
YAML, and recommend an operational failure for an absent effective path or
|
||||
unknown source ID, consistent with unknown-path explanation. Recommend starting
|
||||
path-selected display with one scope; whole-document `--scope all` remains
|
||||
available. Specify presence-aware explicitness filtering so historical-only
|
||||
entries are not labeled implicitly defined. These choices reuse existing
|
||||
operator vocabulary and native evidence with little additional acquisition work.
|
||||
|
||||
**Alternative:** introduce a `config get --path POINTER` command with a snapshot
|
||||
envelope containing `present`, `redacted`, and `yaml`, and return absence as
|
||||
report data. This is convenient for scripts and preserves unknown/null/absence
|
||||
distinctions without relying on exit status, but adds another command/report
|
||||
contract. Aggregate path-selected reports are also viable if scope-labeled
|
||||
per-scope absence and delivery semantics are designed. Filters can be deferred
|
||||
in favor of downstream YAML tools, reducing CLI surface but making routine
|
||||
operator inspection more cumbersome.
|
||||
|
||||
### OD-9: Independent comparison selector bindings
|
||||
|
||||
**Recommendation:** let applications declare which domain selectors support
|
||||
right-side comparison values, with explicit metadata for their `--against-*`
|
||||
spellings. Omission inherits the left value and presence; an explicit right
|
||||
value replaces it. For repeatable selectors, recommend replacing the inherited
|
||||
collection with the supplied right collection. This follows config/overlay
|
||||
selection conventions and supports comparisons such as two campaigns without
|
||||
requiring different files. Review the selected refinement of ADR 0012 and flag
|
||||
collision handling before implementation; exact helper signatures remain open.
|
||||
|
||||
**Alternative:** require symmetric explicit left/right selectors for opted-in
|
||||
bindings. This makes each side independent and easier to reason about when
|
||||
defaults are complex, but adds repetition and departs from the existing
|
||||
right-inherits-left convention. Either design keeps application domain parsing
|
||||
in its declared bindings and avoids inventing a generic selector language.
|
||||
|
||||
### OD-10: Comparison assertion outcomes
|
||||
|
||||
**Recommendation:** add opt-in `--check`, retaining normal report delivery and
|
||||
ordinary diff behavior when omitted. Recommend status `0` for equality and a
|
||||
domain difference status `3`; retain current usage `2` and operational `1`
|
||||
classification. This lets scripts distinguish differences from failed
|
||||
comparison. The status is a proposal, not an existing contract. Apply existing
|
||||
reporting outcome rules, including preservation of a nonzero domain outcome
|
||||
when presentation also fails; callers can inspect the separate presentation
|
||||
evidence. Document that interaction explicitly.
|
||||
|
||||
**Alternative:** add a separate `config compare` assertion command, leaving
|
||||
`diff` exclusively informational. This makes the intent visible in the command
|
||||
path but duplicates selection/report options. Another viable choice is status
|
||||
`1` for both differences and operational failures, which fits simple shell
|
||||
checks but requires additional evidence to distinguish the two conditions.
|
||||
|
||||
### OD-11: Transient configuration overrides
|
||||
|
||||
**Recommendation:** include a declared, opt-in override capability for standard
|
||||
configuration commands, with repeatable `--set POINTER=JSON` values passed to
|
||||
application acquisition as native ConfigKit overrides. This enables validation,
|
||||
inspection, and comparison of temporary settings through consistent CLI syntax.
|
||||
Providers that do not accept overrides retain their existing operations. No
|
||||
source file is edited, and no general application-internal convenience API is
|
||||
required. The loader contract must state how it combines these records with
|
||||
any existing application-owned overrides; the generic adapter cannot inject
|
||||
them into an arbitrary loader after the load has completed.
|
||||
|
||||
If selected, settle these details before implementation:
|
||||
|
||||
- Split at the first `=`; JSON supplies typed null, boolean, string, numeric,
|
||||
array, and object values. Specify numeric conversion to ConfigKit-supported
|
||||
built-in forms rather than passing `json.Number` or silently promising
|
||||
arbitrary numeric precision. An unambiguous alternate input form may be
|
||||
needed for pointer keys containing `=`.
|
||||
- Forward explicit records only, with safe display labels such as `--set`,
|
||||
never raw value-bearing arguments as provenance labels. Preserve native
|
||||
duplicate/overlap handling rather than silently collapsing assignments.
|
||||
- Follow the released mapping-path constraints, including root/index and
|
||||
schema-version restrictions, without extending them into a generic patch
|
||||
language. Empty, false, zero, and null remain intentional supplied values.
|
||||
- For comparison, recommend `--set` as the shared baseline and
|
||||
`--against-set` as a right-side replacement list when supplied, following
|
||||
OD-9's proposed repeatable-selector rule. Define an explicit way to select
|
||||
an empty right list before finalizing this grammar; appending or replacing
|
||||
individual paths would need a different selected rule.
|
||||
- Reflect native CLI origins and subsequent resolution in existing reports.
|
||||
Test CommandKit parsing, forwarding, and outcome/report adaptation, leaving
|
||||
ConfigKit's override algorithm to its own tests.
|
||||
|
||||
**Alternative A:** retain application-specific typed flags and loader translation
|
||||
to `Options.Overrides`. This gives applications tailored vocabulary and avoids
|
||||
generic JSON numeric/path syntax, but repeats CLI infrastructure and prevents
|
||||
uniform temporary-setting workflows across applications. This is a viable
|
||||
deferral if the generic acquisition contract is too disruptive for this sprint.
|
||||
|
||||
**Alternative B:** accept a structured override document containing path/value
|
||||
records instead of inline `--set` assignments. It handles complex values and
|
||||
unusual pointer keys naturally and makes an empty comparison-side list explicit,
|
||||
but adds file-input handling and is less convenient for one-value shell changes.
|
||||
It still needs deliberate typed-value decoding and loader opt-in. Neither
|
||||
alternative authorizes persistent configuration mutation.
|
||||
|
||||
Reference in New Issue
Block a user