Files
configkit/docs/roadmap/implementation.md

792 lines
49 KiB
Markdown
Raw Normal View History

2026-10-26 14:24:07 +00:00
# Interface Contract Remediation Implementation Plan
2026-10-25 20:52:11 +00:00
Status: in progress. Stages 1–17 are complete; Stage 18 remains pending.
2026-10-26 14:24:07 +00:00
Baseline: `f6d54a094a2b5ada5748686620659399dce7ebf0`.
Audience: a Luna coding agent executing the stages sequentially.
2026-10-25 20:52:11 +00:00
## Scope And Execution
2026-10-26 14:24:07 +00:00
Implement all four findings in the [interface audit](interfaces.md) and the
selected shared-history architecture in the
[interface remediation roadmap](interface-remediation.md). The roadmap owns
the target and policy choices; this plan selects private representations,
handoffs, migration steps, tests, and completion checks. The previous completed
C03/F07 plan is preserved at `f6d54a0:docs/roadmap/implementation.md`.
This is one end-to-end sprint. Intermediate stages and commits may fail to
compile. Do not create temporary adapters, parallel implementations, or extra
copies just to make intermediate stages green. Record deferred checks and run
them as soon as their surface is buildable. Commit focused nonempty units under
the [agent policy](../policy/agents.md); update each stage's status and actual
validation here. Do not claim completion based only on checked-off stages.
Read `AGENTS.md`, the [development guide](../development.md), and applicable
[architecture](../policy/architecture.md), [testing](../policy/testing.md), and
[documentation](../policy/documentation.md) policies. Use graph-first discovery
and inspect actual callers before removing private members. Preserve unrelated
work. Existing contract owners are [Go paths](../../path.go),
[metadata](../config.md#layered-metadata),
[resolution](../config.md#recorded-application-resolution),
[explanation](../config.md#historical-value-explanation), and
[composition](../config.md#strict-document-composition).
2026-10-25 20:52:11 +00:00
### Boundaries
2026-10-26 14:24:07 +00:00
- Keep root-owned public signatures, record fields, path syntax, source ordering,
composition/precedence, inheritance, decoding, CLI assignment, and callback
order. Correct the four reported defects without a general API redesign.
- Preserve the distinct metadata and explanation projections. In particular,
metadata retains composite definitions and specific resolution attribution;
explanation retains raw contributions, typed reconciliation, and net grouped
resolution events. They do not need identical record lists.
- Preserve explicit zero/false/empty/null, equal-valued writes, source/resource
attribution, historical sensitivity, and detached output records. Shape-aware
attribution must not narrow pointer-conservative redaction.
- Preserve schema-path diagnostics on `IsExplicit`, silent failed `InputInfo`
queries, helper validation, callback guards, and net-change resolution auditing.
Do not track transient callback mutations or infer identity across sequence
moves.
- Keep history-budget rejection before effective secret lookup and application
validation, using the existing quota and error code. Do not add limits,
dependencies, caches across loads, background work, public events, ordering
options, release gates, or performance promises.
- Do not publish, update consumers, or change accepted ADR decisions. This is a
private ownership refactor implementing existing public decisions.
## Selected Private Design
These choices settle the implementation design. Ordinary helper/file naming may
be adjusted for cohesion; the evidence ownership and selection rules must remain.
Do not defer choosing a different history architecture to an implementation stage.
### One Model, Owned Evidence, And Two Projections
Add `history_model.go`, `history_select.go`, and phase-specific files under
`internal/engine`. Use a concrete `historyBuilder` during loading and a concrete
`historyModel` after preparation; no interface hierarchy, generic event bus,
serialization format, or public API is needed.
The builder owns one model under construction. The model contains:
- The canonical baseline, pre-CLI typed checkpoint, post-CLI typed checkpoint,
and final projection, with the corresponding origin indexes where used.
- Ordered source `historyApplication` records and their sequence projections;
one owned inheritance-operation slice; ordered CLI applications; original
resolution snapshots; and replay groups referring to original snapshot IDs.
- Ordered phase groups and normalized semantic observations described below.
Groups refer to owned payloads by integer IDs rather than copying them.
- A pointer-to-concrete-shapes inventory and pointer-to-observation-ID index,
plus the existing prefix-index mechanism for enclosing/contained operations.
Pointer spelling is an index key, not a semantic identity.
- Eagerly built lookup/index state used by published queries. No query mutates
or lazily initializes the shared model.
Use these phase groups, in order: baseline; source contributions; inheritance;
pre-CLI reconciliation; CLI contributions; post-CLI reconciliation; resolution.
An inventory-only observation is not a public history event. Preserve phase and
producer order even for equal-valued contributions.
Keep `historyApplication`, sequence contribution descriptors, and
`inheritanceOperation` as useful private producer payloads. Their existing merge,
position-projection, and transfer algorithms remain authoritative. Move their
historical interpretation into this owner; do not rewrite configuration assembly
merely to produce a different inspection representation.
### Semantic Observations And Shared Selection
Use a `historyObservation` record with: concrete affected `Path`; phase and
payload/operation IDs; reported operation `Path`; observation role; before/after
presence-side flags; sensitivity facts where observed; and an attribution
reference. Attribution resolves to an owned origin, reasons, and dependencies
in the model. Roles distinguish exact definition, ancestor replacement/removal,
control consumption, typed checkpoint,
and inventory-only evidence. The affected path need not equal the reported path.
Do not put rendered YAML or a copied subtree in each observation.
Source evidence needs both individual contribution references and composite
definition references. Represent a composite definition as a model-owned summary
over its ordered producer origins/contributions, not an independently assembled
`Metadata.history` bucket. Metadata selects the composite summary; explanation
selects the individual effects. Both use the same observation matcher and
attribution resolver. Checkpoints are explanation transitions and inventory
evidence, not new public metadata `ChangeDecoding` records.
Implement `selectHistory(path)` inside the model. It returns ordered private
selections with matching shape/side, operation ID, reported path, and resolved
attribution. Both public projections and historical location fallback consume
these selections. Selection does not render YAML or replay the full configuration.
Selection rules:
1. Reject invalid paths and queries with no matching observed semantic shape.
2. Start with the pointer indexes and relevant prefix operations. Match the
complete concrete path using `matchesPathShape`; an ancestor key/index change
also changes every descendant's shape. Reuse existing path mechanics.
3. Exact definitions require the queried shape to exist in that definition.
An ancestor replacement is relevant only for a matching before or after shape;
it may remove one shape and introduce another at the same pointer.
4. Parsed tokens resolve against each observation's containers. A parsed query
can cover historical key and index states; emit a logical operation once when
both its sides match. Explicit `Key`/`Index` queries remain separate.
5. Derive `Previous` from preceding selected metadata definitions/operations,
not a discarded textual-pointer bucket. A newly introduced index must not
inherit the old mapping key's previous origin. A removed key still retains
its definition and the applicable ancestor-removal operation.
6. Retain existing projection-specific grouping, origin containment, and duplicate
suppression. Several strict contributions may supply one composite metadata
definition but several explanation events. Equal writes are not dropped merely
because their values compare equal.
Metadata query cost should be localized to indexed candidate observations and
returned records, without full-tree replay. Explanation may keep its existing
operation-local whole-tree replay; narrower replay is a separate deferred
experiment, not part of this sprint.
During normalization, use one builder-local working document where needed
to determine concrete sides of effects whose producer payload is relative, such
as the child state immediately before an inherited parent is installed. Initialize
it from the baseline and advance it through existing value-replay helpers in
phase order; reset it to each typed checkpoint at the corresponding boundary.
Capture affected path/side facts before mutation, not a full tree per effect.
Register original resolution facts from their original before/after subtrees;
advance working state using only the net replay groups, not a fictitious sequence
of original callback assignments. Release the scratch document when preparation
ends. It is neither published evidence nor a new historical-path inventory:
membership still comes from registered producer/checkpoint observations. Reuse
producer-side facts instead of replay when they already establish the shapes.
### Producer Evidence And Shape Preservation
Normalize producer facts once into the model, not independently in each public
projection. Use existing concrete displaced paths, raw application roots,
sequence position descriptors, origin indexes, inheritance parent/effective
trees, and resolution before/after trees.
For replacement records, extend private `overlayRecord` with an
`introduced []historyPathOrigin` field, where `historyPathOrigin` contains a
concrete path and detached origin from the incoming contribution at its assigned
coordinates. Capture that evidence at the actual replacement/sequence-combination
producer before later changes can erase its shape. Preserve existing displaced
evidence. These are path/origin facts, not a retained full-tree snapshot per
replacement. Apply the same representation to CLI replacement preparation.
Assign monotonically increasing operation IDs within phase order; keep a
substep ID for parent/child inheritance and similar multi-effect operations.
Index observations by those IDs and sort selected IDs, not map iteration order.
A removal observation and an introduction observation can refer to the same
operation/substep without becoming the same semantic affected path.
The source adapter normalizes displaced definitions, replacement facts, and
surviving source definitions from the pre-inheritance structural root/origins.
It groups composite definitions using existing origin-containment rules, then
ties individual raw replay contributions to their observations. Do not reconstruct
historical shapes by parsing pointers from final origins. Sequence coordinates
come from `walkHistoryApplication` and the existing projection descriptors.
Observe every tree currently contributing historical membership, including the
baseline, structural input, both typed checkpoints, final projection, raw source
and CLI applications, inheritance parent/effective trees, and consumed controls.
Observe original resolution subtrees even when replay coalesces their records.
Record path/sensitivity facts from an inventory-only tree and release it when
no other consumer needs its values; do not revive an unused published structural
tree solely to enumerate paths.
### Loading Handoffs And Frozen Ownership
Construct the builder's prefix in `preparePostDecodeOverrideContext`, after typed
CLI assignment but before application resolution. Supply baseline, structural
producer evidence, inheritance, pre-CLI checkpoint, CLI applications, and post-CLI
checkpoint in logical order. It must already support historical location fallback
for `referenceInput` at this point.
The callback's `resolutionInput` keeps only detached entry effective fields,
concrete entry paths, sources, and passive contexts, as today. It does not borrow
a growing history builder. Keep this view fixed while resolution mutates the
working Go configuration.
After successful resolution auditing, append original snapshots, derive replay
groups, register final shapes, and update effective metadata. Add resolution
evidence before rebuilding diagnostic locations for secret lookup/validation.
Prepare/freeze the model and admit its complete retained document graph at the
existing pre-secret boundary in `executeLoadRequest`.
`Inspection` owns final effective state plus one `*historyModel`.
`Metadata` holds effective fields/paths plus a private reference to that same
frozen model; remove its independent history/inheritance stores. Private frozen
state can be shared across these wrappers. Query outputs still clone paths,
origins, previous origins, reasons, and dependencies. Mutable builder state never
escapes and different loads do not share mutable evidence.
Own one inheritance payload, not one for metadata and another for explanation.
Original resolution records remain authoritative; replay groups contain record
IDs, merged presentation annotations, and references to existing before/after
roots. They do not replace or discard original records. Reuse already owned
immutable trees; clone once only where producers could otherwise mutate them.
Keep each required clone charged to the existing retained-node counter.
### Internal Readers And Retained Accounting
Provide model methods for metadata selections, historical path enumeration,
latest historical document location, sensitivity candidates/dependency edges,
and retained document-root enumeration. These are private methods, not public
extensions. Source records and source-version evidence retain their existing
owners; include the latter's mounted sensitivity candidates as today.
Secret classification consumes all relevant evidence, including original
resolution dependencies and inherited/control links, not only public projected
events. Preserve existing parsed/concrete predicate candidates and union
classification by pointer. An attribution correction must not expose a value
previously protected because another historical shape shared that pointer.
Retained-root enumeration includes every document node reachable from the model,
its payloads and checkpoints, and the final inspection projection. Derived
indexes hold IDs or references to those same roots. Reuse `retainedNodeCounter`
for pointer-identity deduplication; distinct copies still count. Keep query-local
replay copies outside the published-retention count, as today. This is not a
general heap or allocation limit.
### Diagnostic Ordering Rule
Preserve related-location order for `CodeComposeConflict` and retain generic
related-location sorting for other codes. This code already crosses the public
facade unchanged; no private marker or message matching is needed. Its ordinary
claimant producer has a semantically ordered list, and duplicate-identity
producers currently have one related claimant, whose result is unchanged.
Keep canonical ordering of diagnostic records and all primary-location choices.
Apply this rule at the shared canonicalization function, including rendering of
public records converted back into engine records.
## Producer And Consumer Migration Map
| Existing owner | Final owner/action |
| --- | --- |
| `metadataFromDecoded`, `collectFieldInfo` | Keep effective-field construction; baseline/source history goes to model normalization. |
| `applyOverlayMetadata`, `appendInputOriginEvent`, `appendInputChangeEvent` | Replace history-bucket reconstruction with source/CLI semantic observations and composite summaries inside the model. Retain useful origin-containment helpers. |
| `applyInheritanceMetadata`, `historyWithInheritance` | Remove independent source-view/history insertion; inheritance observations and common selection supply both projections. |
| `projectedInheritanceChanges`, `appendInheritanceExplanation` | Share the transfer/control attribution resolver in the model; replay helpers only apply values. |
| `appendPostDecodeCLIHistory`, `overrideMetadataRecords` | Replace history assembly; retain/adapt useful displacement preparation as CLI evidence production. |
| `applyResolutionMetadata`, `appendResolutionDescendantHistory` | Retain effective-field/origin updates; original model records and shared selection own history and previous attribution. |
| `finalizeMetadataPaths`, inheritance-history index | Keep effective indexing; model observations own historical shapes, membership, enumeration, and operation indexes. |
| `Inspection.Explain` phase/relevance/origin orchestration | Consume model selections and owned replay groups, preserving safe value replay/rendering. |
| `metadataFieldLocation` | Consume model latest-location selection, using the concrete query path. |
| `classifySecretPathsWithEvidence` historical reads | Consume model candidates, original dependency edges, and inheritance links; retain fixed-point sensitivity closure. |
| `prepareInspectionHistory`, `publishLoadResult`, `addInspectionHistory` | Prepare and freeze one owner; enumerate/count the new retained ownership graph before secrets/validation. |
| `cloneMetadata`, `Inspection.Metadata` | Clone effective mutable records as needed and share only frozen private history; query outputs remain detached. |
## Validation Conventions
Use the installed compatible Go toolchain and existing dependencies. From the
repository root, use `GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off` for Go checks.
Keep disposable artifacts outside the checkout. Do not change dependencies for
tooling or run publication commands.
Each stage names focused behavior owners. Run its focused tests when buildable;
otherwise record them as pending. New regression names must be included in the
selected expression or run explicitly. Reuse existing cases instead of duplicating
full matrices. Do not rewrite correct expectations to accommodate a refactor;
change only defect expectations and tests tied to retired representations.
## Stage 1 — Correct Callback Presence And Defaulting
Finding 1. Owners: `resolution.go`, `load_metadata.go`, resolution/provenance tests,
and one root regression in `facade_compatibility_test.go` or a focused new facade
test file. Status: complete. Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Resolution|Provenance|Default' -count=1`
and `GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test . -run 'TestFacadeResolutionPresenceUsesDynamicEntryShape' -count=1`.
2026-10-26 14:24:07 +00:00
1. Add a private entry-presence helper using `resolutionInput.lookup` and its
`FieldInfo.Explicit`. Use it in `Resolution.IsExplicit` after existing
schema-path validation and in `Default` after existing helper-target checks.
Remove the now-unused raw presence field from `resolutionState` if no other
production consumer remains. Do not change `InputInfo` error semantics.
2. Exercise explicit numeric keys and indexes beneath `any` in both directions.
Check parsed pointers, entry lookup/presence agreement, and an absent path.
3. To exercise `Default`, change the working dynamic container to the other shape,
record its parent normalization, default the new child through a correctly
typed local target, and write it back. The old shape remains explicit in the
entry view; the new shape must not suppress defaulting. Preserve final helper
matching and ordinary explicit zero/false/null cases.
Acceptance: the audit's first reproduction is corrected without changing
schema-invalid-path diagnostics, closed-context panics, or helper assignment rules.
Focused check: `go test ./internal/engine -run 'Resolution|Provenance|Default'`
and the selected root regression.
## Stage 2 — Correct Final Explicitness Across Shape Changes
Finding 2. Owners: `applyResolutionMetadata`, `refreshResolutionFields`,
`resolveLoadInput`, and resolution/metadata tests. Status: complete. Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Resolution|Metadata|Provenance' -count=1`
and `GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`.
2026-10-26 14:24:07 +00:00
1. Pass the frozen `resolutionInput` to effective metadata refresh rather than
attempting to infer entry shape from a final pointer bucket.
2. Walk the final projection with concrete paths. Obtain explicitness from the
matching entry field; a missing or shape-mismatched entry contributes false.
Retain the applicable resolution origin and final null state. Preserve the
existing delta-coverage origin rules and remove genuinely absent final fields.
3. Cover key-to-index and index-to-key changes, a change at an ancestor of a
deeper queried field, and same-shape ancestor normalization. Verify entry
queries stay fixed and final `Lookup`/`IsExplicit` agree. Add no sequence
record-identity inference or transient-mutation tracking.
Acceptance: the new semantic path is non-explicit while surviving same-shape
input remains explicit. Focused check:
`go test ./internal/engine -run 'Resolution|Metadata|Provenance'`.
## Stage 3 — Introduce The Shared Evidence And Selection Core
Finding 3 architecture. Owners: new `history_model.go` and `history_select.go`,
2026-10-26 14:47:18 +00:00
existing path/lookup helpers, focused history-selection tests. Status: complete.
Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'History.*Select|History.*Shape|Path' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`,
and `git diff --check`.
2026-10-26 14:24:07 +00:00
1. Implement the selected model, builder, phase/role enums, payload ID references,
and semantic observation types. Keep phase producers in later stages; do not
populate a second legacy history store from this core.
2. Implement concrete-shape inventory insertion with deduplication using actual
segment kinds, and observation indexing by pointer and relevant prefix.
Keep first-observed data order separately from sorted enumeration.
3. Implement shared selection, before/after-side matching, logical operation
deduplication for parsed queries, and attribution-reference resolution.
`Previous` is computed from the selected metadata sequence, not inherited
from a pointer-only record. Keep selectors independent of YAML serialization.
Add builder scratch-state support for the normalization procedure above;
do not retain per-event full-tree copies or query-time mutable caches.
4. Test a numeric key/index collision, an ancestor shape collision affecting a
deeper path, introduced/removed sides of one operation, parsed traversal,
and a never-observed descendant. Use small actual document fixtures.
Acceptance: both public projections have one internal semantic-selection protocol
to consume. Focused check once buildable:
`go test ./internal/engine -run 'History.*Select|History.*Shape|Path'`.
## Stage 4 — Normalize Baseline And Ordinary Source Evidence
Owners: new `history_source.go`, `load_metadata.go`, `overlay.go`, composition
evidence consumers. Status: complete. Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'History.*Source|Compose|Overlay|Provenance|Explain' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`,
and `git diff --check`.
2026-10-26 14:24:07 +00:00
1. Register baseline definitions by walking the canonical baseline with concrete
paths and default origins. Keep absent roots and explicit null distinct.
2. Build the source phase from structural applications, raw root/origins,
displacement records, and concrete introduced-path facts. Add introduced
evidence to `mergeOverlayMappings` and `recordOverlayReplacement` while the
relevant shape exists. Extend `cloneOverlayRecords` to detach the new path/
origin facts. Do not retain a new full root per replacement. With `NoSource`,
keep the source phase empty rather than fabricating an empty source mapping.
3. Move source-definition grouping/origin-containment behavior out of metadata
history building into this normalization. Preserve displaced and surviving
definitions, composite mapping locations in assembly order, empty mappings,
and equal definitions. Keep raw source application values for explanation.
4. Link fine contribution and composite summary observations to model-owned
attribution. Public projections must not separately recover locations by
traversing raw applications or by parsing final pointer strings.
Acceptance: baseline and ordinary source/overlay semantic observations preserve
the existing metadata-versus-explanation distinction. Reuse compose, overlay,
provenance, and explanation cases; focused check:
`go test ./internal/engine -run 'Compose|Overlay|Provenance|Explain'`.
## Stage 5 — Normalize Projected Collection Evidence
Owners: `sequence_history_projection.go`, sequence combination evidence,
`history_source.go`, and `history_sequence_test.go`. Status: complete. Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'SequenceHistory|Mapped.*History|Tier.*History|ProjectedSequenceHistory|Destination|HistorySourceSequence' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`,
and `git diff --check`.
2026-10-26 14:24:07 +00:00
1. Extend source normalization to assigned sequence coordinates using
`walkHistoryApplication`, `historyApplicationLocalPath`, and existing sequence
contribution indexes. Preserve append offsets, mapped upsert outputs,
replacement positions, and item-local/nested source placement.
2. Capture concrete introduced-path facts in `applyOverlaySequenceEvidence`
alongside existing displaced facts, for whole-sequence and matched-record
replacements. Distinguish the operation's container or
record path from the affected descendant's actual shape.
3. Preserve whole-record replacement/removal events when a higher record omits
an old descendant, and prevent a contribution that never addressed a record
from acquiring its attribution. Do not alter collection selection or merge
algorithms, source ordering, or identity equality.
Acceptance: raw and composite collection observations have common assigned paths
and correct displacement attribution. Focused check:
`go test ./internal/engine -run 'SequenceHistory|Mapped.*History|Tier.*History|ProjectedSequenceHistory|Destination'`.
## Stage 6 — Normalize Inheritance And Control Evidence
Owners: new `history_inheritance.go`, `inheritance_history.go`,
2026-10-26 15:25:02 +00:00
`inheritance_explain.go`, inheritance operation producers. Status: complete.
Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Inheritance.*History|History.*Inheritance|RecordInheritance|NestedInheritance' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`,
and `git diff --check`.
2026-10-26 14:24:07 +00:00
1. Register one ordered inheritance payload. Produce observations for control
removal/cancellation, parent transfer, effective child contribution, and
displaced paths using parent/effective trees and their origin indexes. Use
builder scratch state where the actual pre-transfer child/enclosing shape is
needed; neither the parent nor effective child alone proves that old shape.
2. Move transfer/control attribution and dependency selection into the common
resolver. Adapt useful parts of `transferredOperationOrigin` and
`projectedInheritanceChanges`; do not leave one attribution implementation for
metadata and another for explanation.
3. Preserve destination-local paths, contributor traces, directive locations,
equal child writes, nested unresolved templates, and child identity fields.
Apply concrete before/after shape matching at every ancestor.
4. Keep value replay helpers that install the parent, restore child identity,
and install the effective child, but have them receive selected attribution
rather than compute it independently.
Acceptance: controls and transfers are shared evidence, including paths omitted
by a particular public projection. Focused check:
`go test ./internal/engine -run 'Inheritance.*History|History.*Inheritance|RecordInheritance|NestedInheritance'`.
## Stage 7 — Normalize Typed Checkpoints And CLI Evidence
Owners: new `history_cli.go`, `override_context.go`, override metadata/replay
2026-10-26 15:36:42 +00:00
helpers. Status: complete. Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'CLI.*History|Explain|Decoding|IntegerVersionInspection|Override' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`,
and `git diff --check`.
2026-10-26 14:24:07 +00:00
1. Register pre-CLI and post-CLI canonical checkpoints with their origin indexes
on the correct sides of the CLI contribution phase. Preserve raw-versus-typed
number/string representations and retained defaults.
2. Normalize prepared CLI applications and displacement/introduced facts into
the same source-effect representation, with CLI attribution. Adapt
`overrideMetadataRecords`/`collectAffectedOverrideOrigins` as producers, not
a second public-history assembly path.
3. Register checkpoint observations for inventory and conditional reconciliation.
Only explanation emits decoding transitions when replay differs; metadata
continues to expose definitions and operations rather than these transitions.
4. Cover the audit's CLI mapping-to-sequence shape change and its reverse,
equal writes, whole-sequence removal, partial mappings, and default
reconciliation. Reuse existing post-inheritance CLI chronology tests.
Acceptance: the model distinguishes baseline, structural source state, typed
pre-CLI input, raw CLI application, and typed post-CLI entry state. Focused check:
`go test ./internal/engine -run 'CLI.*History|Explain|Decoding|IntegerVersionInspection|Override'`.
## Stage 8 — Normalize Original Resolution And Net Replay Groups
Owners: new `history_resolution.go`, `history.go`, original resolution snapshots
and coverage indexes. Status: complete. Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Resolution|Coalesced|Overlapping|Explain' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./...`,
and `git diff --check`.
2026-10-26 14:24:07 +00:00
1. Store the ordered original snapshots after successful audit. Normalize their
before/after subtrees into concrete observations and retain every original
reason/dependency, even if explanation coalesces its public event.
2. Adapt `coalescedResolutionSnapshots` to return groups of original record IDs,
a covering record/root reference, and merged presentation annotations. Preserve
ordering, first-occurrence reason/dependency deduplication, normalization
dominance, and the existing overlapping-record coverage rules.
3. Keep metadata's specific original record attribution and explanation's net
subtree replay as explicit projections of those references. Compute shape-
specific previous origins through common selection, not copied `Previous`
from an earlier pointer bucket.
4. Preserve original subtree inventory and sensitivity dependencies even where
a covering replay group omits a separate public record.
Acceptance: shape-changing resolution history is selectable and original
attribution is not lost to replay grouping. Focused check:
`go test ./internal/engine -run 'Resolution|Coalesced|Overlapping|Explain'`.
## Stage 9 — Complete Historical Inventory And Internal Query Services
Owners: model finalization, `metadata.go`, history selection indexes.
Status: complete. Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'HistoricalPaths|Metadata|History.*Shape|SecretClassification' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./...`, and `git diff --check`.
2026-10-26 14:24:07 +00:00
1. Derive historical membership and enumeration from all model observations and
registered payload roots, including inventory-only checkpoints and controls.
Do not maintain the old parallel tree-walk inventory in `finalizeMetadataPaths`.
2. Preserve all concrete shapes for one pointer, but return one sorted parsed
`Path` per pointer from `HistoricalPaths`. Root/default-only paths and removed
descendants remain included; never-observed descendants remain absent.
3. Provide a latest historical location query over selected definitions and
previous origins, a complete dependency/candidate iterator, and an eager
immutable lookup over model document roots. Preserve producer chronology;
indexes are accelerators, not alternate evidence owners.
4. Retain `effectivePathIndex` for entry/final shape lookup. Register concrete
sensitivity facts while observing any tree whose values can then be released.
Acceptance: membership and all internal historical queries have one evidence
source. Focused check:
`go test ./internal/engine -run 'HistoricalPaths|Metadata|History.*Shape|SecretClassification'`.
## Stage 10 — Cut Metadata History Over To The Model
Owners: `Metadata`, `Metadata.History`, `cloneMetadata`, model metadata projection.
Status: complete. Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Metadata|Provenance|History' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test . -run 'TestPublicHistoricalPathsEnumerateContextualShapes|TestPublicHistoryUsesResolutionShapeAttribution' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./...`, and `git diff --check`.
2026-10-26 14:24:07 +00:00
1. Replace metadata's independent history map, inheritance slice, and inheritance
index with a private frozen-model reference. Keep effective fields/paths and
their shape-aware lookup separate from historical evidence.
2. Implement `History` as shared selection followed by metadata projection:
default definitions, composite source summaries, inheritance operations,
CLI operations, and original resolution attribution. Derive previous origins
after shape filtering and projection-specific grouping.
3. Clone every returned nested record. Share only private frozen model state
when copying `Metadata`; preserve zero-value behavior.
4. Add a focused public regression comparing explicit key/index histories after
CLI and resolution shape changes. Check no false base definition or previous
origin appears for the introduced shape, while removed paths retain applicable
ancestor replacement history. Include a parsed-query/enumeration case.
Acceptance: finding 3's metadata defect is fixed without calling `Explain`,
rendering YAML, or replaying full trees per metadata query. Focused check:
`go test ./internal/engine -run 'Metadata|Provenance|History'` and the selected
root history regressions.
## Stage 11 — Cut Explanation Over To Common Selections
Owners: `explain.go`, `inheritance_explain.go`, `snapshot.go`, model replay helpers.
Status: complete. Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Explain|Explanation|History|Render|Diff' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./...`, and `git diff --check`.
2026-10-26 14:24:07 +00:00
1. Drive phase chronology and event relevance through model selections. Remove
independent relevance/origin interpretation from `Inspection.Explain` and
inheritance explanation orchestration. Keep safe snapshot serialization and
ordinary value-application helpers.
2. Replay source, inheritance, checkpoint, CLI, and grouped resolution payloads
through references from the model. Use each selection's resolved attribution;
do not call metadata's public `History` or translate its records into values.
3. Preserve baseline presence, conditional decoding reconciliation on both sides
of CLI, raw superseded values, equal input/child/CLI operations, resolution
group reasons/dependencies, secret-reference annotations, and unknown-path
behavior. Keep caller mutation and repeated queries detached.
4. Compare both interfaces on semantic attribution while asserting their selected
differences for composite mappings, decoding, and overlapping records.
Acceptance: both views use common semantic selection and attribution, not merely
the same payload storage with old independent interpretation. Focused check:
`go test ./internal/engine -run 'Explain|Explanation|History|Render|Diff'`.
## Stage 12 — Integrate Load Prefixes And Diagnostic Location Readers
Owners: `preparePostDecodeOverrideContext`, `resolveLoadInput`,
`documentWithMetadataLocations`, `metadataFieldLocation`, and load handoffs.
Status: complete. Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Location|Provenance|ResolutionInput|CLISecret|Validation' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./...`, and `git diff --check`.
2026-10-26 14:24:07 +00:00
1. Thread one builder through `postDecodeOverrideContext` and `resolvedLoad` using
the lifecycle specified above. Create the pre-resolution prefix once; append
resolution evidence once. Remove transport fields that duplicate model-owned
applications, checkpoints, origins, and snapshots after their callers migrate.
2. Keep the callback's `resolutionInput` as its independent fixed effective view.
Do not put a mutable history reference in that context or let publication
change its captured fields/sources.
3. Change historical location fallback to take a concrete `Path` and query the
current model prefix. Preserve the existing priority: effective source
location first, then latest applicable historical location, then the nearest
structural input fallback. Shape filtering must not fabricate an exact
location from a different key/index state.
4. Ensure prefix queries work before resolution and after resolution, before
secrets and validation need their `referenceInput`. Do not postpone model
construction until `publishLoadResult`.
Acceptance: diagnostic attribution has no dependency on the retired history map.
Reuse metadata-location, secret-location, CLI, and provenance tests. Focused check:
`go test ./internal/engine -run 'Location|Provenance|ResolutionInput|CLISecret|Validation'`.
## Stage 13 — Migrate Historical Sensitivity Readers
Owners: `secret.go`, `secret_propagation.go`, model candidates/dependencies.
Status: complete. Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Secret|Sensitive|Redact|Diff' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test . -run 'Secret|Sensitive|Redact|Diff' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./...`, and `git diff --check`.
2026-10-26 14:24:07 +00:00
1. Replace reads of metadata history and private inheritance copies with the
complete model candidate and dependency services. Preserve original resolution
dependencies, inherited contributor/control links, historical tags, and
source-version mounted evidence.
2. Preserve parsed and concrete candidate shapes used by current predicate/tag
classification. Fold sensitivity by pointer conservatively after collecting
relevant shapes; neither history projection's omissions may erase candidates.
3. Keep fixed-point dependency closure and cross-result diff masking unchanged.
Do not narrow classification because an exact metadata query now excludes a
different historical shape at the same pointer.
Acceptance: effective output, historical snapshots, version evidence, and paired
diff snapshots retain disclosure behavior. Focused check:
`go test ./internal/engine -run 'Secret|Sensitive|Redact|Diff'` and existing root
secret/diff integration tests.
## Stage 14 — Publish One Frozen History Owner
Owners: `Inspection`, `prepareInspectionHistory`, `publishLoadResult`,
`Inspection.Metadata`, cloning and lookup setup. Status: complete. Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Detached|Independent|ConcurrentReadOnly|History|Explain' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test -race ./internal/engine -run 'Detached|Independent|ConcurrentReadOnly|History|Explain' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./...`, and `git diff --check`.
2026-10-26 14:24:07 +00:00
1. Finish the model before publication, with eager indexes and final observations.
Attach the same frozen owner to inspection and metadata. Remove independent
inspection copies of source/CLI applications, checkpoint origins, original
and grouped resolution payloads, and inheritance evidence once moved.
2. Retain only document roots needed for active replay, effective rendering/diff,
and classification. Convert inventory-only projections to path/sensitivity
facts rather than publishing an otherwise unused structural tree.
3. Detach producer-owned mutable slices, origins, and trees where necessary once
at the handoff; reuse already immutable engine-owned roots. Preserve final
configuration detachment, validated secret state, and no further lookup or
callback invocation during publication.
4. Reuse existing detached-query, independent-load, and concurrent read-only
inspection tests. Update representation-only fixtures expecting two private
inheritance copies; replace that assertion with protection against mutation
leaking through either public projection.
Acceptance: one frozen history model owns the published evidence and no query
mutates it. Focused check:
`go test ./internal/engine -run 'Detached|Independent|ConcurrentReadOnly|History|Explain'`.
## Stage 15 — Reconcile Complete Retained-History Accounting
Owners: `executeLoadRequest`, `retainedNodeCounter`, model root enumeration,
2026-10-26 17:46:31 +00:00
`history_budget_test.go`, inheritance budget tests. Status: complete. Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'History.*Budget|HistoryNode|Retained|ProjectionBudget' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./...`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test -race ./internal/engine -run 'History.*Budget|HistoryNode|Retained|ProjectionBudget|CLIHistoryEvidenceKeepsCheckpoints|Detached|ConcurrentReadOnly' -count=1`,
and `git diff --check`.
2026-10-26 14:24:07 +00:00
1. Replace `addInspectionHistory`'s old field/copy enumeration with the actual
model and final-inspection root graph. Include all checkpoint, application,
inheritance, original resolution, and any distinct replay-retained roots.
Count shared nodes once by pointer; check required clones before allocating
them where practical.
2. Perform final admission at the existing pre-secret/pre-validation boundary.
After that boundary publication must not introduce an uncounted retained tree;
keep index finalization there if it retains additional roots. Leave the
existing structural-expansion limit checks intact.
3. Update the independent test graph enumeration for the new owner. Derive an
exact small fixture requirement, test success at that count and rejection
below it, and assert no secret lookup or validator call occurs on rejection.
4. Cover ordinary loading and a layered fixture combining inheritance, CLI,
original/overlapping resolution, and checkpoints. Drop obsolete copy-count
assertions, not ownership, complete-accounting, or failure-timing protection.
Acceptance: the existing quota governs the complete new retained graph; no new
quota or representation-specific node total is promised. Focused check:
`go test ./internal/engine -run 'History.*Budget|HistoryNode|Retained|ProjectionBudget'`.
## Stage 16 — Remove Superseded Attribution And Repair Test Seams
Owners: all entries in the migration map and tests constructing private inspection
or metadata values. Status: complete. Validation: `GOTOOLCHAIN=local
GOPROXY=off GOTELEMETRY=off go test ./internal/engine` and
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./...` passed.
2026-10-26 14:24:07 +00:00
1. Trace remaining production reads/writes of old history/inheritance fields and
helpers. Remove history-building portions of `metadataFromDecoded`,
`applyOverlayMetadata`, `appendPostDecodeCLIHistory`,
`appendResolutionDescendantHistory`, and `historyWithInheritance` once their
replacements are active. Preserve effective-field construction and reusable
domain mechanisms with real consumers.
2. Remove the old parallel historical path walker, duplicate published payload
ownership, and independent explanation relevance/attribution paths. Do not
leave them for sensitivity, location fallback, or tests alone.
3. Update private test constructors to build shared evidence through actual
adapters. Retain behavioral assertions. Rewrite or remove isolated tests of
a deleted representation only when their meaningful protection is preserved
at a surviving owner; do not add production compatibility shims for fixtures.
4. Check each migration-map row against actual references. All historical
membership and attribution readers must now trace to the model. A shared
container wrapping both old reconstruction paths is not completion.
Acceptance: there is one active historical attribution system. Run the full
ordinary suite when buildable: `go test ./...`.
## Stage 17 — Preserve Composition Conflict Ordering End To End
Finding 4. Owners: `canonicalDiagnostics`, composition diagnostic producers,
root diagnostic conversion/rendering tests. Status: complete. Validation:
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test ./internal/engine -run 'Diagnostic|Compose|Conflict' -count=1`,
`GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test . -run 'TestFacadeCompositionConflictKeepsClaimantOrder' -count=1`,
and `git diff --check` passed.
2026-10-26 14:24:07 +00:00
1. Implement the code-based ordering rule specified above. Verify every
`CodeComposeConflict` producer; keep claimant order and leave single-related
duplicate-identity diagnostics unchanged. Retain generic sorting otherwise.
2. Add the audit fixture using `fstest.MapFS`: main imports a/b/c; a imports z;
b/c/z each claim x. Assert primary b and related c,z despite acquisition IDs
4,5,3, without assuming source IDs are assembly keys.
3. Assert the same order from public `LoadError.Diagnostics` and after passing
those records to root `WriteDiagnostics`. Preserve detached records, safe
formatting, short writes, canonical ordering of diagnostic records, and
generic related-location sorting for a non-composition code.
Acceptance: accumulation and facade round-tripping no longer reorder composition
claimants. Focused check:
`go test ./internal/engine -run 'Diagnostic|Compose|Conflict'` and the selected
root diagnostic regression.
## Stage 18 — Integrated Verification And Documentation
Owners: integrated engine/facade, documentation, audit and plan status.
Status: pending.
1. Run all deferred focused checks. Review representative ordinary and layered
public loads for agreement among final configuration, entry/final presence,
origins, history, explanation, source contexts, and safe effective output.
Use existing examples/integration fixtures; do not invent another matrix.
2. Ensure regressions include a deeper ancestor shape change, inventory omitted
by public grouping, equal contributions, raw superseded values, decoding on
both sides of CLI, inherited controls/identity, mapped sequence displacement,
and overlapping resolution intent. Extend an existing owner only for a
remaining meaningful gap. Verify intentional projection differences as well
as shared semantic attribution.
3. Run with the same installed toolchain:
```sh
env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off ./scripts/check.sh
env GOTOOLCHAIN=local GOPROXY=off GOTELEMETRY=off go test -race ./...
```
The race run protects the changed frozen-ownership handoff using existing
concurrency coverage; it does not add a new public concurrency promise or
release gate. No benchmark campaign or external consumer update is required.
4. Update `docs/internal/overview.md` for the implemented shared-history owner,
concrete responsibilities, and lifecycle handoffs. Update Go comments and
`docs/config.md` only where needed to clarify corrected existing behavior or
remove obsolete mechanics. Preserve canonical ownership and accepted ADRs;
describe no new public guarantee. Adjust contributor test links only if their
owners move. Do not retire these roadmap records during implementation.
5. Mark the four audit findings completed with concise implementation/test
evidence; mark this plan and the feature roadmap implemented only after the
full integrated result satisfies acceptance. Review the final diff for scope
and unrelated work, record actual validation, and commit completed units.
Completion: all four reproductions are corrected; both projections and internal
historical readers derive attribution and membership from one model; old parallel
machinery is removed; ownership and quota timing hold; applicable checks pass.
The result must compile and be coherent even though intermediate stages need not.
## Open Questions
None. Roadmap-level decisions are settled, and this plan selects the private
evidence representation, semantic-selection protocol, lifecycle handoffs,
diagnostic rule, migration responsibilities, and validation boundaries. If an
unforeseen issue requires changing public semantics or the selected ownership
architecture, report that concrete conflict rather than silently narrowing the
target or adding a new restriction.