Expand ConfigKit roadmap with API coverage findings

docs/roadmap/configkit-v1.1.0.md

@@ -1,8 +1,9 @@
 # ConfigKit v1.1.0 adoption
 
 Status: proposed; dependency and report changes are not implemented. This
-roadmap covers the embedded `ckconfig` integration. Release coordination and
+roadmap covers the embedded `ckconfig` integration, the API coverage review,
-additional source evidence have recommendations under [Open Decisions](#open-decisions).
+and recommended extensions. Release coordination and capability scope have
+recommendations under [Open Decisions](#open-decisions).
 
 ## Purpose and authority
 
@@ -24,7 +25,9 @@ work; current contracts remain in GoDoc, the [CLI reference](../cli.md), and the
 
 ## Current baseline and review evidence
 
-Reviewed CommandKit revision: `60841227b7e2928d3fefd3326f0e3f4694f41e6d`.
+Initial review revision: `60841227b7e2928d3fefd3326f0e3f4694f41e6d`.
+API coverage follow-up revision: `7a9e3a1f7cdd0475f06a63ce0aababefe08651bc`;
+library code and module selections are unchanged between these revisions.
 The completed [dependency alignment roadmap](integration-dependency-alignment.md)
 records the preceding vanity migration and releases.
 
@@ -53,6 +56,45 @@ the existing adapter structure remains suitable. It did not exercise the
 proposed report-field renames, additional source evidence, or a migrated
 ckprompt; final implementation still needs validation of those selected changes.
 
+### API coverage findings
+
+The follow-up compared the public API of the pinned v1.1.0 module with
+ckconfig's provider facets, adapters, and report projections. These are gaps in
+current integration coverage, not necessarily APIs newly introduced in v1.1.0.
+
+| Capability | Current ckconfig exposure | Recommendation |
+| --- | --- | --- |
+| `SourceInfo.EmptyImportSelections` | Omitted from the source report. | Include in adoption; OD-2. |
+| `Result.Metadata()`, `Metadata.Lookup`, and `IsExplicit` | No metadata facet in `View`; explanation reports do not expose explicit input presence. | Prioritize optional field-metadata inspection as the next functional expansion; OD-3. |
+| `Metadata.HistoricalPaths` and `History` | Operators must already know a path to request an explanation; metadata history records are not exposed. | Pair path discovery and provenance history with metadata inspection; OD-3. |
+| `LoadError.Sources()` and `Source(id)` | Acquired sources support diagnostic context, but `config sources` returns before reporting an inventory on acquisition failure. | Consider failed-load source inspection separately; OD-4. |
+| Full `Difference` records | The Go comparison provider returns native records; the CLI reduces them to additions, right-side changes, and deletion paths. | Consider an additional detailed comparison report; OD-5. |
+| Structured `Diagnostic` records | Native writers present diagnostic content; there is no structured native diagnostic report. | Lower-priority candidate, coordinated with shared reporting; OD-6. |
+
+The upstream [metadata API](https://forge.maximumdirect.net/base-layer/configkit/src/commit/191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9/metadata.go),
+[source and diagnostic records](https://forge.maximumdirect.net/base-layer/configkit/src/commit/191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9/records.go),
+[load-error API](https://forge.maximumdirect.net/base-layer/configkit/src/commit/191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9/diagnostic.go),
+and [comparison API](https://forge.maximumdirect.net/base-layer/configkit/src/commit/191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9/diff.go)
+own the returned evidence and its semantics.
+
+The current explanation projection covers every field of v1.1.0's
+`Explanation`: effective and historical snapshots, origin and inheritance,
+reasons, dependencies, and secret-reference names. Improved list-history
+behavior can flow through that existing adapter. Metadata queries add distinct
+information: explicitness and null state from `FieldInfo`, observed paths
+including removed ones, and `Change` records with previous origins and operation
+paths. They need not replace the value-oriented explanation report.
+
+`Result.Source(id)` is a convenient lookup of records already available through
+the source inventory. A source-ID filter is a possible usability refinement;
+it does not inherently require another command or a required lookup method on
+every custom source provider. Discovery, import defaults, collection identities,
+and field-level resolution recording fit the application-supplied loader.
+Source resource contexts and explicit secrets-directory bootstrap primarily
+serve application loading and startup; there is no immediate recommendation to
+add standard inspection operations for them. A consumer need could motivate
+later presentation of their upstream results.
+
 ## Targeted end state
 
 ### Dependency and toolchain
@@ -100,6 +142,37 @@ ConfigKit's source inventory also supplies `EmptyImportSelections`, with
 declaration path, location, and selector. OD-2 addresses including this evidence
 in the existing source report; it is not yet a selected report-schema addition.
 
+### Recommended inspection expansion
+
+After the dependency and source-report adoption, prioritize an optional
+metadata capability for field inspection, path discovery, and provenance
+history. A narrow provider facet could expose ConfigKit's public `Metadata`
+value directly. `FromConfigKit` can supply it from the loaded result; custom
+scopes can declare the operations for which they provide the capability.
+This is a proposed extension, not a new requirement on existing providers.
+
+Use upstream `Lookup` and `IsExplicit` for effective field metadata,
+`HistoricalPaths` for observed-path enumeration, and `History` for definition,
+inheritance, and resolution records. `HistoricalPaths` includes effective and
+removed paths; it is not simply a list of current fields. Scope the report to
+the returned upstream evidence rather than inferring an independent provenance
+model. Command names, selectors, and report shapes need a focused design once
+this extension is selected; this roadmap does not establish their grammar.
+
+Field-origin inventories are currently among the
+[deferred proposals](deferred.md#other-deferred-proposals). OD-3 proposes
+reconsidering that scope based on the reviewed public API. If selected, update
+that record and the owning CLI/API documentation with implementation. Retaining
+the proposal here does not by itself mark the extension accepted or implemented.
+
+Failed-load source inspection, detailed comparison reports, and structured
+diagnostics remain separately considered additions under OD-4 through OD-6.
+They use existing upstream evidence, but each changes CommandKit's presentation
+or outcome coordination and needs its own scope choice. In particular, source
+inventory output on an acquisition error would retain the failed outcome;
+additional comparison detail would expose the native left/right snapshots,
+presence/redaction fields, and `HiddenChanged` already returned to the provider.
+
 ### Consumer migration and module coordination
 
 Public ckconfig signatures expose ConfigKit types, so switching import identity
@@ -125,33 +198,48 @@ not select an unreviewed PromptKit release or claim a new matched integration se
 ## Work from the baseline
 
 1. **Confirm scope choices and prepare module metadata.** Resolve OD-1 and
-   OD-2, select the Go 1.27 maintainer toolchain, update ckconfig's requirement,
+   OD-2 and record which proposed extensions, if any, join the adoption release.
+   Deferred extensions need not block the dependency migration. Select the
+   Go 1.27 maintainer toolchain, update ckconfig's requirement,
    imports, and minimum, and regenerate authentic sums with workspace discovery
    off. Inspect the resulting dependency graph; do not commit local replacements
    or environment files as an adoption mechanism.
 2. **Adapt APIs and source reports.** Update the native source field accesses,
    DTOs, text/YAML output, affected tests, and example code. Apply the selected
-   empty-import-evidence choice. Retain the existing coordination and outcome
+   empty-import-evidence choice. Preserve current coordination and outcomes
-   behavior, adapting upstream results rather than adding domain validation.
+   except where an explicitly selected extension changes presentation, such as
-3. **Update canonical documentation and notices.** Revise the ckconfig README
+   retaining a failure alongside returned source evidence. Adapt upstream
+   results rather than adding domain validation.
+3. **Design and implement selected extensions.** Treat metadata inspection and
+   path discovery as the next functional work item under OD-3, whether included
+   here or scheduled next. Define its optional provider capability, CLI usage,
+   and report projection together. For any OD-4 through OD-6 feature selected,
+   specify its report delivery and outcome behavior before implementation.
+   Reuse native records and rendering where suitable; avoid a second domain
+   implementation or a generic framework solely to mirror upstream APIs.
+4. **Update canonical documentation and notices.** Revise the ckconfig README
    and GoDoc, [CLI source-report reference](../cli.md#list-configuration-sources),
    consumer guide, and release toolchain instructions with the implemented
    changes. Review the selected graph and upstream notices, then update
    ckconfig attribution with the vanity identity, selected versions, and pinned
    source links. Update other module notices only when their actual selections
    change. Preserve published release history.
-4. **Validate the selected end state.** Run the ckconfig suite, including its
+5. **Validate the selected end state.** Run the ckconfig suite, including its
    maintained example, independently with `GOWORK=off` and read-only module
    metadata on the selected Go 1.27 toolchain. Update focused adapter/report
    assertions for the new vocabulary and faithful projection of returned
    evidence. Preserve coverage of request forwarding, diagnostics, cancellation,
    output failures, and supported error identity. Assertions establish
    CommandKit adaptation; ConfigKit owns tests of its domain correctness.
+   For selected metadata capabilities, verify query forwarding and returned
+   field/history projection. For failed-load inventory or detailed comparison,
+   verify the selected report and outcome coordination without re-establishing
+   upstream acquisition, provenance matching, or sensitivity behavior.
    Review changed Markdown links, fragments, and examples. If coordinated
    ckprompt work is selected, also validate that module and the composed consumer
    against the proposed dependency set; local-workspace success alone does not
    establish independent module compatibility.
-5. **Prepare the integration release.** Propose ckconfig v0.3.0, subject to the
+6. **Prepare the integration release.** Propose ckconfig v0.3.0, subject to the
    existing fresh-version guards, with tag `integrations/ckconfig/v0.3.0`.
    Explain changed ConfigKit type identity, Go minimum, report fields, and
    consumer migration in its checked-in note with the usual licensing footer.
@@ -166,9 +254,12 @@ Implementation is complete when ckconfig's actual module metadata and imports
 select the new dependency, the selected report changes faithfully expose its
 returned evidence, the module suite and maintained consumer pass independently,
 and current documentation, notices, and release preparation describe that state.
-Record the resolved scope choices and final validation evidence. A future source
+Record resolved scope choices, final validation evidence, and the status of
-release is reported separately from implementation completion; preparing this
+each proposed extension. Features selected for this implementation need their
-roadmap does not publish a version or migrate external applications.
+own provider/report evidence; recording a deferred proposal does not demonstrate
+its implementation. A future source release is reported separately from
+implementation completion; preparing this roadmap does not publish a version
+or migrate external applications.
 
 ## Open Decisions
 
@@ -203,3 +294,67 @@ reduces the immediate report change and test scope, and still makes the native
 metadata available to Go consumers of ConfigKit. CLI consumers would continue
 to lack this upstream evidence, and a later release would need another report
 and documentation update.
+
+### OD-3: When and how should metadata inspection and path discovery land?
+
+**Recommendation:** make optional metadata inspection and path discovery the
+next functional expansion after adoption. Expose the upstream metadata queries
+through an optional facet, and design field and history reports together. This
+adds explicitness, path discovery, and provenance information that current
+explanations do not fully expose, while keeping existing providers usable.
+It requires a focused CLI/API design and a later release if staged separately.
+
+**Viable alternative:** include that coherent expansion in the adoption release.
+This gives consumers the broader inspection surface with one migration and
+documentation update. It increases the current release's API, report, and
+validation scope and can delay the dependency upgrade. Either schedule can use
+the same upstream-owned query semantics; final command grammar remains a design
+choice rather than a contract selected by this roadmap.
+
+### OD-4: Expose acquired sources when a configuration load fails?
+
+**Recommendation:** evaluate this as a separate source-inspection enhancement,
+presenting `LoadError`'s returned inventory while retaining the load failure and
+diagnostics. It makes upstream acquisition evidence available for troubleshooting
+without treating the operation as successful. The design must account for
+returning the source facet on error and delivering a report before terminal
+failure presentation; that coordination is additional to mechanical adoption.
+
+**Viable alternative:** include this behavior in the adoption release alongside
+the source-report changes. It delivers successful and failed-load inventory
+inspection together and avoids another source-report release. It broadens the
+current adapter and outcome changes and requires focused coverage of report
+delivery failures as well as acquisition failure. Retaining current diagnostic
+context alone until that work is selected remains viable.
+
+### OD-5: Add a detailed native comparison report?
+
+**Recommendation:** consider an additional report form in a later comparison
+enhancement, retaining the current reduced form for its established operator
+use. The detailed form would project `Path`, both snapshots, and `HiddenChanged`
+from native records, making already-returned evidence available to CLI consumers.
+Supporting two forms adds selection, presentation, and documentation work;
+choose the interface when the feature is selected.
+
+**Viable alternative:** replace the reduced report with native-record projection
+in a pre-v1 minor release. One report form is simpler to maintain and exposes
+all current comparison fields directly. It requires deliberate migration of
+published report consumers and changes the concise additions/changes/deletions
+workflow. Keeping only the current form is also viable when that workflow meets
+consumer needs; richer reporting is a capability proposal, not a correctness fix.
+
+### OD-6: Where should structured diagnostic reporting be introduced?
+
+**Recommendation:** retain native diagnostic rendering for this adoption and
+consider structured records with CommandKit's shared reporting design. That
+keeps result and diagnostic delivery consistent across integrations and custom
+commands. It defers machine-readable native codes, paths, severities, and
+locations until their report channel and outcome behavior are designed.
+
+**Viable alternative:** add an optional ckconfig-specific diagnostic report that
+projects ConfigKit's records directly. This could deliver useful structured
+evidence sooner without redesigning the shared reporting API. It introduces a
+separate presentation path whose interaction with ordinary results, warnings,
+and terminal failures needs definition and may later need alignment with other
+families. It is a lower-priority extension rather than a dependency-adoption
+requirement.