Files
configkit/docs/roadmap/implementation.md

350 lines
20 KiB
Markdown
Raw Normal View History

# Implementation plan: typed result values and focused inspection
2026-11-21 04:12:15 +00:00
Status: implementation in progress; Stages 1–7 complete.
## Scope and working instructions
Implement the agreed end state in [result-value.md](result-value.md), including
settled OD-1 through OD-5 and accepted [ADR 0024](../adr/0024-compare-inspection-views-across-application-types.md).
The runtime baseline is v1.3.0 (`652ddb6`); subsequent documentation commits
settle the API and comparison choices. Inspect the current checkout before editing.
2026-11-21 04:00:27 +00:00
Independent source-version policies and isolation of sibling evidence are
outside this feature.
Audience: a Luna coding agent implementing stages sequentially in one end-to-end
sprint. Intermediate stages may leave compilation temporarily incomplete; do not
add temporary public bridges or duplicate implementations to keep them green.
Private helper names and file boundaries below are implementation suggestions;
public signatures and the roadmap's behavior are selected decisions.
Start with [development guidance](../development.md), [agent policy](../policy/agents.md),
[architecture policy](../policy/architecture.md), [testing policy](../policy/testing.md),
and [documentation policy](../policy/documentation.md). Preserve unrelated work.
Use focused behavioral checks as each part becomes runnable, then perform the
final checks once the complete feature is in place. Commit completed nonempty
units according to agent policy; do not publish a release or modify dependencies.
## Implementation boundaries
- Root `configkit` owns exported declarations and explicit conversions. Engine
helpers remain private to consumers. Keep root-to-engine dependency direction
and avoid exported aliases or anonymous internal embedding.
- `Value` reads current `Config`; inspection borrows the existing frozen load
state. Neither capability loads again, projects a new execution snapshot,
runs codecs or callbacks, reads resources, or resolves secrets.
- Preserve existing `Result` methods and generic `Diff[T]`. The new non-generic
view compares frozen snapshots structurally across root Go types, as ADR 0024
explicitly permits. No runtime root-type tracking is needed.
- Reuse descriptor field bindings, path parsing/index rules, history inventories,
snapshot redaction, and the existing comparison algorithm. No new global
caches, registry, synchronization, conversion framework, or load-time snapshot
is needed. Do not impose new limits or validation on returned terminal values.
- Ordinary Go assignment governs ownership. Maps, slices, and pointers may alias
`Config`; neither deep copying nor concurrent mutation safety is promised.
Typed value availability is independent of YAML presence and load evidence.
- Reporting scope is a canonical JSON Pointer location. Explicit typed lookup,
explanation, and metadata queries retain their existing semantic path intent.
## Stage 1 — Implement private current-value navigation
**Complete.**
Read `internal/engine/schema.go`, `schema_fields.go`, `value_operation.go`,
`decode.go`, and semantic paths before adding a small engine helper, for example
in `internal/engine/result_value.go`. Accept the current Go root, an engine Path,
and the existing root-supplied Secret binding; return `(any, bool)`.
1. Reject invalid paths. For a reached terminal, return its actual Go value before
dereferencing pointers or attempting structural validation. The root selects
the entire current configuration. Nil interfaces return `(nil, true)`;
typed nil pointers/maps/slices retain their typed nil value in `any` with
`true`, as ordinary Go assignment requires. Unsupported terminal values are
still returned without conversion or revalidation.
2. For continued navigation, follow non-nil pointers/interfaces and describe the
containing type using existing operation-local schema machinery. Reuse
`schemaBuilder.describe`, descriptor kinds and field bindings; share small
helpers where needed rather than introducing a second YAML-tag parser.
`buildSchema` enforces load-root requirements and `resolveSchemaPath` accepts
some merely schema-valid dynamic paths; neither alone proves runtime presence.
3. Struct fields use `fieldByName`, field index chains, and the inline-map fallback.
Reuse `fieldForDecode` with allocation disabled if suitable; ordinary named
fields take precedence over inline-map entries. Do not allocate nil embedded
pointers. Keep ignored/unexported fields inaccessible. Convert string map keys
to the declared key type when necessary, preserving named string-key support.
4. Check actual map membership and slice/array bounds. Use `schemaIndex` for
contextual numeric tokens, canonical index spelling, and explicit key/index
intent. Text-codec scalars, duration, Secret, and other scalar descriptors are
leaves, even if represented by structs. Inspect method sets only; do not call
codecs, projection, cloning, or secret-value getters during navigation.
5. For `any`, describe its current dynamic concrete value under existing supported
type/field rules, including supported structs supplied after loading or in a
manually constructed result. Reuse dynamic descriptor machinery where its
semantics fit; do not apply projection's secret-classification admission as a
new lookup validation pass. Existing Secret descriptors remain scalar leaves.
An unsupported containing shape returns false when further navigation needs it.
6. Keep descriptor/cache state local to the call. Traverse the requested path,
using direct field/map/index selection; do not scan or project all runtime
configuration values. Recursive types must work. If pointer/interface unwrapping
can repeat without consuming a segment, detect that loop locally and return
false instead of hanging; do not reject a whole object graph for a cycle on
an unvisited branch or introduce a general cycle-validation phase.
Acceptance: navigation can distinguish missing from present nil and can return
named values without changing the configuration. Descriptor work may inspect
reachable schema types; lookup should not depend on total runtime collection size.
## Stage 2 — Expose Value and verify its public contract
2026-11-21 04:05:27 +00:00
**Complete.**
Add `func (r *Result[T]) Value(path Path) (any, bool)` in a focused root file,
forwarding current `r.Config`, the converted path, and the existing Secret binding.
A nil receiver returns `(nil, false)`. Any non-nil result works independently of
its inspection field, including `&Result[T]{Config: ...}`.
Add focused public tests, for example `facade_result_value_test.go`, protecting:
- Root and nested lookup, YAML/default names, inline structs/pointers/maps,
ignored and unexported fields, and literal escaped/empty keys.
- Named scalars, structs, arrays, pointers at terminals, and non-nil intermediate
pointers. Present zero/false/empty values remain available without input;
metadata separately distinguishes defaulted from explicit values.
- Dynamic maps, sequences, and supported structs in `any`, including post-load
replacements and hand-built results. Unsupported navigation fails cleanly;
reached terminals are not revalidated.
- Numeric mapping keys versus sequence indices, parsed contextual tokens,
negative/invalid indices, bounds, missing entries, and nil intermediates.
A nil interface and typed nil values are tested using their Go semantics.
- Struct/array assignment copies and pointer/map/slice aliasing. Mutations to
current `Config` are visible to Value while existing inspection stays frozen.
- Trusted access to a resolved Secret with a disposable canary and unchanged
redaction through existing inspection. A codec that fails if invoked proves
lookup does not run application codecs after construction.
Use a few readable fixtures/tables rather than one test for every helper branch.
Reuse existing schema/path tests for rules unchanged by this work. Run focused
Value tests and the public boundary tests when this stage is runnable.
## Stage 3 — Add the inspection view and canonical scope selection
2026-11-21 04:06:45 +00:00
**Complete.**
Add a root-owned, non-generic `Inspection` with unexported state retaining the
existing engine Inspection and normalized scope. Put scope-aware implementation
in engine (a small private-to-consumers view/helper is sufficient); do not copy the
frozen result or construct a partially populated `Result`.
Implement these public signatures:
```go
func (r *Result[T]) InspectSubtree(path Path) (Inspection, bool)
func (i Inspection) Metadata() Metadata
func (i Inspection) HistoricalPaths() []Path
func (i Inspection) Sources() []SourceInfo
func (i Inspection) Source(id string) (SourceInfo, bool)
func (i Inspection) SourceContext(id string) (ResourceContext, bool)
func (i Inspection) Explain(path Path) (Explanation, bool)
func (i Inspection) WriteExplanation(output io.Writer, path Path) error
```
1. Constructor failure means nil result, missing completed-load inspection, or
invalid path. A valid absent path succeeds. Normalize only after checking
validity: parse the canonical pointer spelling into contextual tokens so
explicit numeric key/index spellings select the same reporting location.
Empty scope selects the whole result.
2. Centralize a small canonical-location containment operation for enumeration
and later diff filtering. Compare pointer tokens or canonical strings with
slash boundaries and an explicit root case. `/a` must not include `/ab`;
escaped slashes and empty keys retain their meaning. Existing `pathHasPrefix`
compares segment kinds and is not the reporting-location operation needed here.
Do not alter semantic path equality elsewhere or build a new path inventory.
3. HistoricalPaths filters the existing detached, pointer-sorted enumeration to
equal/descendant locations. Preserve its order and uniqueness. Effective
presence remains available via `Metadata().Lookup(path)`; no new effective-path
public API is needed. Metadata itself remains the original unfiltered view.
4. Forward explicit Explain and WriteExplanation paths unchanged, including paths
outside the focus. Forward source catalogs/IDs and contexts without ownership
filtering. Reuse existing root conversions, extracting small shared conversion
helpers only when useful. Ancestor snapshots and dependency evidence remain
intact even when they contain siblings outside the focus.
5. A zero view uses existing unavailable-state results: empty metadata/path
enumeration, nil source catalog, false query results, and explanation-rendering
errors as appropriate. Preserve the existing detached-record and borrowed
ResourceContext ownership rules.
Add constructor/forwarding tests for root, absent scopes, malformed paths, nil and
manual results, zero views, native paths outside scope, equivalent numeric scope
spellings, escaped keys, and sibling-prefix exclusion. Do not require a scope to
exist in either effective or historical data.
## Stage 4 — Render focused effective output with original sensitivity
**Complete.**
Add `func (i Inspection) WriteEffective(output io.Writer) error` and a scoped
engine rendering operation using the saved final tree.
1. Resolve the normalized contextual scope against the frozen document with
existing document traversal. No value means an error before writing bytes;
explicit null is a present node and renders normally. A zero view or nil
writer follows existing effective-rendering failure behavior.
2. Pass the selected node and its original application path to the existing
snapshot/redaction boundary (`safeSnapshotDocument`/`redactedDocument`), then
write with the existing YAML adapter. `pathIsSecret` already checks ancestors;
do not reset the selected value to root before sensitivity evaluation.
3. Output just the selected value, with existing formatting and redaction.
Writer errors propagate normally and may leave partial output. Do not use
trusted Value output or marshal/redecode typed settings for this operation.
Add public behavioral tests for mapping/scalar/sequence/root output, null versus
absent, historical-only scopes, ancestor and descendant sensitivity, numeric
mapping/sequence scopes, and unchanged output after Config mutation. Use a
controlled failing writer for failure propagation; preserve whole-result output
coverage rather than duplicating it comprehensively.
## Stage 5 — Compare focused views using the existing diff
2026-11-21 04:09:41 +00:00
**Complete.**
Implement `func (i Inspection) Diff(other Inspection) ([]Difference, error)`.
Validate both initialized views and equality of normalized reporting locations
before comparison. Errors return no records; do not add root-Go-type checks.
Delegate to `engine.Diff` on the original complete frozen inspections, then
retain each record whose native Path is equal to, beneath, or above the scope.
Use the Stage 3 location helper in both directions. Preserve record order,
original paths, snapshot text, Present/Redacted flags, and HiddenChanged.
Filter before root record conversion where convenient. Reuse the existing root
Difference conversion instead of copying comparison logic.
Do not extract and compare section roots: doing so would change sequence-atomic
regions, ancestor changes, or paired sensitivity. Whole-result comparison cost
is deliberately selected; no narrower algorithm or performance experiment is
part of this implementation.
Add focused tests covering:
- Equal views, ordinary in-scope changes, sibling-only differences filtered out,
same-scope comparisons across different application root Go types, and root scope.
- Scope addition/removal, absent on both sides, explicit null, and mismatched or
zero views. Equivalent numeric scope spellings compare successfully.
- A change to a containing sequence or cross-kind ancestor retains that ancestor
record, even when its cause is outside the selected descendant. This is the
documented consequence of focusing an atomic difference, not proof the selected
value changed. Include mapping-to-sequence shape changes at numeric locations.
- Sensitivity from either side, ancestor sensitivity and sensitive removal, with
disposable canaries absent from returned snapshots and unchanged HiddenChanged
semantics. Whole-result Diff retains its existing same-type API and behavior.
Run focused view-diff tests together with the existing diff tests.
## Stage 6 — Verify history, provenance, and view ownership together
**Complete.**
Use a small loaded fixture to exercise the interactions not covered by stages
3–5. Prefer extending their fixtures rather than creating another test framework.
- Select a component with effective and removed descendants. HistoricalPaths
includes only observed equal/descendant locations, including historical-only
paths; Metadata.History and Explain retain ordinary ancestor records and native
query paths. Check the distinction between constructor success and presence.
- Retain source catalogs, locations and SourceContext through placed inputs and
an existing structural transformation such as inheritance. Assert effective
field origins lead to the original resource base. A source that contributes
siblings remains discoverable; historical-only origins do not restore a lost
effective resource origin.
- Mutate current Config after selecting the view. Value sees that mutation while
view rendering, explanation and comparison still describe the completed load.
Mutating returned query records does not alter later reports. Reuse existing
detached metadata/source tests where forwarding already protects this risk.
- Confirm contextual scope enumeration across historical key/index shape changes
without rewriting explicit query paths or collapsing existing history semantics.
Run the focused history/metadata/render/source-context tests plus the new public
view tests. This stage verifies integration, not a new confidentiality or
concurrent-mutation contract.
## Stage 7 — Demonstrate one-library handoff
2026-11-21 04:12:15 +00:00
**Complete.**
Add a maintained example and focused test under `examples/consumers/`, for example
`result_component.go` and `result_component_test.go`, using representative small
library-owned settings rather than adding an actual PromptKit dependency.
Show an application-owned root and a library-owned consumer interface using
Value, Metadata, and SourceContext. The application passes the whole result;
the library selects its conventional path and asserts its public settings type.
Registration uses an ordinary typed field selector and existing Definition hooks
for defaults, normalization, validation, sensitivity and sequence identities.
Bundled defaults enter as ordinary underlays where needed. Runtime construction
and resource capture remain library/application responsibilities.
Show a library-owned inspection facade selecting InspectSubtree once and
forwarding effective output, explicit native-path queries and view comparison.
Test one normal load followed by handoff, useful typed values, presence/origin,
safe reporting, a relative resource resolved from its effective source, and a
component validation failure during the application load. Do not create a new
ConfigKit registration API or a second library-specific load.
## Stage 8 — Demonstrate two-library composition
Extend the example with a second small consumer using the same application
result and a smaller consumer-owned interface where sufficient. Compose existing
application and library callbacks with explicit ordering and authoritative
sequence identity registrations. Use fresh per-load state rather than shared
mutable registration captures.
Verify both consumers receive their own named settings from the same result,
application rules still run, explicit values survive absence-based defaults,
invalid component identities fail through the normal lifecycle, and source
contexts remain usable through placement/inheritance. Include repeated loads to
show callback/default state does not leak. Reuse Stage 7's fixtures and existing
source-operation tests; demonstrate representative integration without multiplying
every layering/import/override combination. Do not modify downstream repositories
or claim completion of real PromptKit integration.
## Stage 9 — Document the implemented boundary and validate the sprint
Update documentation only once the corresponding behavior exists:
- Declaration comments own signatures, nil/absence behavior, ordinary aliasing,
trusted values, current Config versus frozen evidence, and reporting-only scope.
Document canonical scope equivalence on InspectSubtree and errors on unavailable
output or invalid comparisons. Do not add public error codes/sentinels merely
for this feature; use existing error style.
- `docs/config.md` owns shared ownership/inspection and comparison behavior.
Distinguish same-type Diff[T] from cross-root-type view comparison, ancestor
differences and paired redaction, and value availability from input presence.
- `docs/consumers/README.md` links the maintained examples and explains one-load
registration, whole-result handoff, owned runtime settings, and native paths.
Link from README only with a short useful orientation entry.
- `docs/internal/overview.md` records actual helper ownership and reuse.
`docs/development.md` adds focused test guidance and ADR 0024 navigation.
Update the architecture policy's comparison wording to include accepted
ADR 0024 while preserving its existing provenance and other constraints.
- Update roadmap status accurately. Retain historical release records and the
deferred candidates in future.md. Release notes/publication are a separate task.
Run the focused tests introduced above, `go test -race ./...`, and the everyday
`./scripts/check.sh` using an installed compatible toolchain and pre-acquired
dependencies. The everyday check supplies formatting, whitespace, tests, vet and
build validation. Run the consumer examples and existing public API/dependency
boundary checks through their ordinary tests. Review changed Markdown links and
all newly documented behavior against the implementation; no new release gate,
performance threshold or benchmark requirement is introduced.
Completion review must establish that both consumers use one application result,
current typed access follows existing field/path conventions, frozen inspection
preserves native evidence and sensitivity, cross-type view comparison follows
ADR 0024, and whole-result loading/inspection remains compatible. Review the full
diff for accidental stronger contracts, unsafe reflect access and duplicated
machinery. Commit validated, nonempty work with plain-English messages and report
remaining limitations accurately.
## Open Questions
None. OD-1 through OD-5 and ADR 0024 settle the decisions required for this scope.
Private naming and file organization remain ordinary implementation choices.