base-layer/commandkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Record optional validation details and failure evidence

Browse Source
This commit is contained in:
Eric Rakestraw
2026-11-06 18:40:55 +00:00
parent 491f0b49f4
commit 7f2fdce8e7
2 changed files with 44 additions and 26 deletions
Download Patch File Download Diff File Expand all files Collapse all files

66
docs/roadmap/promptkit-v0.19.0.md
View File

@@ -100,7 +100,7 @@ in the decisions at the end of this document.
| Resolve unqualified resource selections in chosen namespaces | Independent prompt/profile/backend namespace flags | `ckprompt`: delegate to native namespace views, retaining the distinction from catalog filters. |
| Find configuration paths and inspect their explicitness | `config explain --list --scope promptkit`, with historical selection; metadata alongside path explanation | `ckconfig`, wired through `ckprompt.ConfigScope`: observed paths, effective presence, explicitness/null/origin evidence, and available history. |
| Understand both sides of configuration changes | `config diff --scope promptkit --details` | `ckconfig`: native paths, left/right snapshots, presence/redaction, and hidden-change evidence. |
| Investigate a failed configuration load | `config sources --scope promptkit` on failure; structured diagnostics within `config validate` | `ckconfig`, with both native adapters: acquired source inventory and native codes, severities, paths, and locations while retaining the load failure. |
| Investigate a failed configuration load | `config sources --scope promptkit` on failure; `config validate --details --scope promptkit` | `ckconfig`, with both native adapters: acquired source inventory and native codes, severities, paths, and locations while retaining the load failure. |
| Inspect one effective value or subtree | `config show --scope app --path /server` | `ckconfig`: the selected native effective snapshot, retaining redaction and absence/null distinctions. |
| Follow a source reference | `config sources --source-id ID` | `ckconfig`: the matching acquired source record, including evidence available after failure. |
| Compare different application selections | For example, `config diff --campaign spring --against-campaign autumn` | `ckconfig`: independently prepared typed inputs passed to the existing native comparison provider. |
@@ -111,8 +111,9 @@ The configuration suite retains six subcommands: `scopes`, `validate`, `show`,
`sources`, `explain`, and `diff`. Path discovery belongs within `config explain`;
structured diagnostics belong within `config validate`. This command placement
is agreed. OD-4 selects `--list` and `--path` as the explanation modes, with
help on a bare invocation. Diagnostic mode flags, report shapes, and filter
spellings remain proposals under OD-4, OD-6, and OD-8. No separate path inventory
help on a bare invocation. OD-6 selects `config validate --details` for detailed
diagnostics while retaining concise validation by default. Report shapes and
filter spellings remain proposals under OD-4, OD-6, and OD-8. No separate path inventory
or diagnostics subcommand is planned.
Transient configuration overrides are an open scope decision under OD-11, with
@@ -241,8 +242,9 @@ failure presentation. Keep the original load error and supported identity;
successful report delivery must not convert failure to success. Preserve
render/write failures separately through the current outcome machinery. Cover
cancellation with partial evidence explicitly without promising interruption of
an arbitrary writer or rollback of already-delivered bytes. OD-6 selects the
validation diagnostic mode and its interaction with ordinary diagnostics.
an arbitrary writer or rollback of already-delivered bytes. OD-6 selects
`config validate --details`; its interaction with ordinary diagnostic messages
remains a presentation design task.
Structured records describe that validation invocation, including a failed
load; they are not a retrieval interface for an earlier command. Preserve the
current aggregate validation stopping behavior unless deliberately revised;
@@ -311,7 +313,7 @@ errors remain errors even if an override would replace the affected value.
Reuse the standard suite's lifecycle and preserve existing application adapter
seams.
4. **Implement shared configuration inspection.** Add discovery and metadata
within `config explain`, structured diagnostics within `config validate`,
within `config explain`, structured diagnostics via `config validate --details`,
detailed diff presentation, and source evidence on failure in `ckconfig`.
Retain the six-command suite and wire native PromptKit facets through
`ConfigScope`. Add focused effective display, source selection, and bounded
@@ -405,7 +407,9 @@ Default tests remain offline and independent of real credentials.
overrides reach opted-in loaders, with typed values, safe flag labels,
selected comparison-side behavior, native errors, and returned provenance
preserved. No tests independently re-establish ConfigKit override semantics.
- [ ] `config sources` and the structured mode of `config validate` report
- [ ] Default validation remains concise; `--details` selects the additional
diagnostic report. Selecting YAML alone does not enable detailed mode.
- [ ] `config sources` and `config validate --details` report
available evidence with an acquisition error;
the original failure, unavailable-versus-empty evidence, cancellation, and
report-delivery failures remain distinguishable.
@@ -675,27 +679,39 @@ status remains a separate choice under OD-10.
`config validate`; there will be no separate diagnostics command. Source
inventory on failure remains part of `config sources`.
**Recommendation:** add an explicit validation diagnostic mode, for example
`config validate --diagnostics --format yaml`, reporting native records and a
source legend alongside the validation result. Keep the existing validation
report when that mode is omitted. Support severity/code filters in the mode
without changing the underlying outcome. This preserves existing consumers and
lets the same invocation both validate and explain its findings, including
available evidence on load failure.
**Agreed:** add `config validate --details`, consistent with the optional detail
mode on `config diff`. Retain concise validation by default, including its
ordinary warning and error reporting. Detailed mode reports available native
diagnostic records and source references alongside the validation result. The
purpose is to help a human operator locate and investigate configuration
problems; it does not introduce new validation rules. Selecting YAML alone does
not implicitly enable the additional detail.
Preserve available evidence when loading fails: `config validate --details`
reports diagnostic findings and relevant source references, and `config sources`
reports acquired source inventory. These describe the current invocation, not a
previous failed command. Keep the original failure and its supported cause
identity; successfully displaying evidence must not turn failure into success.
A failed load does not imply an effective configuration is available.
Build optional `ckconfig` evidence facets that can accompany an acquisition
error. Structured results use the result stream, ordinary messages use the
diagnostic stream, and terminal failure/outcome handling remains in place.
Resolve warning duplication, unavailable-versus-empty evidence, report shape,
and partial-result cancellation behavior during design. Successful delivery of
findings must not replace a failed load with a successful validation result.
error. Unlike detailed comparison, the existing adapters need to carry additional
structured evidence through to reporting. Use current core reporting and outcome
handling; no generic cross-family diagnostics framework is required.
**Alternative:** enrich `config validate --format yaml` directly with diagnostic
records and source evidence, without a new mode flag. This gives scripts richer
information by default but changes the existing YAML report and failure-output
contract, requiring deliberate compatibility treatment. Either choice stays
within validation and uses current core reporting; a generic cross-family core
diagnostics schema is not needed for this scope.
**Remaining presentation recommendations:** provide readable structured YAML,
for example `config validate --details --format yaml`, with native codes,
severities, paths, locations, and a source legend where available. Exact report
layout and format support remain to be designed. Support severity/code filters
under OD-8 without changing the underlying validation outcome. Distinguish
unavailable evidence from an available empty set of findings.
Structured results use the result stream and ordinary messages use the
diagnostic stream. Resolve warning duplication and partial-result cancellation
presentation before implementation. Preserve report-delivery failures separately
from the original operation failure through existing outcome handling. Retain
current aggregate stopping behavior; detailed output does not promise exhaustive
validation across all scopes.
### OD-7: Module release and interim workspace coordination