diff --git a/docs/roadmap/future.md b/docs/roadmap/future.md index ec66d25..c561df7 100644 --- a/docs/roadmap/future.md +++ b/docs/roadmap/future.md @@ -63,7 +63,9 @@ discovery and explicitness, failed-load source inventory, detailed comparison, and structured diagnostics. Path discovery is a mode of `config explain`, and structured diagnostics belong to `config validate`; the suite retains six subcommands. The roadmap records the agreed explanation modes and help behavior; -diagnostic mode spelling, report shapes, and filters remain open. +[OD-6](promptkit-v0.19.0.md#od-6-failure-evidence-and-structured-diagnostic-delivery) +selects optional validation details, a concise default, and available evidence +on failure. Report shapes and filters remain open. It also includes focused effective display, source-ID selection, metadata/diagnostic filters, independent comparison selectors, and assertion mode. The original four shared proposals had ConfigKit diff --git a/docs/roadmap/promptkit-v0.19.0.md b/docs/roadmap/promptkit-v0.19.0.md index 632f5a7..9f11a0d 100644 --- a/docs/roadmap/promptkit-v0.19.0.md +++ b/docs/roadmap/promptkit-v0.19.0.md @@ -100,7 +100,7 @@ in the decisions at the end of this document. | Resolve unqualified resource selections in chosen namespaces | Independent prompt/profile/backend namespace flags | `ckprompt`: delegate to native namespace views, retaining the distinction from catalog filters. | | Find configuration paths and inspect their explicitness | `config explain --list --scope promptkit`, with historical selection; metadata alongside path explanation | `ckconfig`, wired through `ckprompt.ConfigScope`: observed paths, effective presence, explicitness/null/origin evidence, and available history. | | Understand both sides of configuration changes | `config diff --scope promptkit --details` | `ckconfig`: native paths, left/right snapshots, presence/redaction, and hidden-change evidence. | -| Investigate a failed configuration load | `config sources --scope promptkit` on failure; structured diagnostics within `config validate` | `ckconfig`, with both native adapters: acquired source inventory and native codes, severities, paths, and locations while retaining the load failure. | +| Investigate a failed configuration load | `config sources --scope promptkit` on failure; `config validate --details --scope promptkit` | `ckconfig`, with both native adapters: acquired source inventory and native codes, severities, paths, and locations while retaining the load failure. | | Inspect one effective value or subtree | `config show --scope app --path /server` | `ckconfig`: the selected native effective snapshot, retaining redaction and absence/null distinctions. | | Follow a source reference | `config sources --source-id ID` | `ckconfig`: the matching acquired source record, including evidence available after failure. | | Compare different application selections | For example, `config diff --campaign spring --against-campaign autumn` | `ckconfig`: independently prepared typed inputs passed to the existing native comparison provider. | @@ -111,8 +111,9 @@ The configuration suite retains six subcommands: `scopes`, `validate`, `show`, `sources`, `explain`, and `diff`. Path discovery belongs within `config explain`; structured diagnostics belong within `config validate`. This command placement is agreed. OD-4 selects `--list` and `--path` as the explanation modes, with -help on a bare invocation. Diagnostic mode flags, report shapes, and filter -spellings remain proposals under OD-4, OD-6, and OD-8. No separate path inventory +help on a bare invocation. OD-6 selects `config validate --details` for detailed +diagnostics while retaining concise validation by default. Report shapes and +filter spellings remain proposals under OD-4, OD-6, and OD-8. No separate path inventory or diagnostics subcommand is planned. Transient configuration overrides are an open scope decision under OD-11, with @@ -241,8 +242,9 @@ failure presentation. Keep the original load error and supported identity; successful report delivery must not convert failure to success. Preserve render/write failures separately through the current outcome machinery. Cover cancellation with partial evidence explicitly without promising interruption of -an arbitrary writer or rollback of already-delivered bytes. OD-6 selects the -validation diagnostic mode and its interaction with ordinary diagnostics. +an arbitrary writer or rollback of already-delivered bytes. OD-6 selects +`config validate --details`; its interaction with ordinary diagnostic messages +remains a presentation design task. Structured records describe that validation invocation, including a failed load; they are not a retrieval interface for an earlier command. Preserve the current aggregate validation stopping behavior unless deliberately revised; @@ -311,7 +313,7 @@ errors remain errors even if an override would replace the affected value. Reuse the standard suite's lifecycle and preserve existing application adapter seams. 4. **Implement shared configuration inspection.** Add discovery and metadata - within `config explain`, structured diagnostics within `config validate`, + within `config explain`, structured diagnostics via `config validate --details`, detailed diff presentation, and source evidence on failure in `ckconfig`. Retain the six-command suite and wire native PromptKit facets through `ConfigScope`. Add focused effective display, source selection, and bounded @@ -405,7 +407,9 @@ Default tests remain offline and independent of real credentials. overrides reach opted-in loaders, with typed values, safe flag labels, selected comparison-side behavior, native errors, and returned provenance preserved. No tests independently re-establish ConfigKit override semantics. -- [ ] `config sources` and the structured mode of `config validate` report +- [ ] Default validation remains concise; `--details` selects the additional + diagnostic report. Selecting YAML alone does not enable detailed mode. +- [ ] `config sources` and `config validate --details` report available evidence with an acquisition error; the original failure, unavailable-versus-empty evidence, cancellation, and report-delivery failures remain distinguishable. @@ -675,27 +679,39 @@ status remains a separate choice under OD-10. `config validate`; there will be no separate diagnostics command. Source inventory on failure remains part of `config sources`. -**Recommendation:** add an explicit validation diagnostic mode, for example -`config validate --diagnostics --format yaml`, reporting native records and a -source legend alongside the validation result. Keep the existing validation -report when that mode is omitted. Support severity/code filters in the mode -without changing the underlying outcome. This preserves existing consumers and -lets the same invocation both validate and explain its findings, including -available evidence on load failure. +**Agreed:** add `config validate --details`, consistent with the optional detail +mode on `config diff`. Retain concise validation by default, including its +ordinary warning and error reporting. Detailed mode reports available native +diagnostic records and source references alongside the validation result. The +purpose is to help a human operator locate and investigate configuration +problems; it does not introduce new validation rules. Selecting YAML alone does +not implicitly enable the additional detail. + +Preserve available evidence when loading fails: `config validate --details` +reports diagnostic findings and relevant source references, and `config sources` +reports acquired source inventory. These describe the current invocation, not a +previous failed command. Keep the original failure and its supported cause +identity; successfully displaying evidence must not turn failure into success. +A failed load does not imply an effective configuration is available. Build optional `ckconfig` evidence facets that can accompany an acquisition -error. Structured results use the result stream, ordinary messages use the -diagnostic stream, and terminal failure/outcome handling remains in place. -Resolve warning duplication, unavailable-versus-empty evidence, report shape, -and partial-result cancellation behavior during design. Successful delivery of -findings must not replace a failed load with a successful validation result. +error. Unlike detailed comparison, the existing adapters need to carry additional +structured evidence through to reporting. Use current core reporting and outcome +handling; no generic cross-family diagnostics framework is required. -**Alternative:** enrich `config validate --format yaml` directly with diagnostic -records and source evidence, without a new mode flag. This gives scripts richer -information by default but changes the existing YAML report and failure-output -contract, requiring deliberate compatibility treatment. Either choice stays -within validation and uses current core reporting; a generic cross-family core -diagnostics schema is not needed for this scope. +**Remaining presentation recommendations:** provide readable structured YAML, +for example `config validate --details --format yaml`, with native codes, +severities, paths, locations, and a source legend where available. Exact report +layout and format support remain to be designed. Support severity/code filters +under OD-8 without changing the underlying validation outcome. Distinguish +unavailable evidence from an available empty set of findings. + +Structured results use the result stream and ordinary messages use the +diagnostic stream. Resolve warning duplication and partial-result cancellation +presentation before implementation. Preserve report-delivery failures separately +from the original operation failure through existing outcome handling. Retain +current aggregate stopping behavior; detailed output does not promise exhaustive +validation across all scopes. ### OD-7: Module release and interim workspace coordination