Define identity-aware resolution history in the sequence roadmap

docs/roadmap/resolution-sequences.md

@@ -1,8 +1,8 @@
-# Field-level resolution auditing for sequences
+# Field-level resolution and identity-aware sequence history
 
-Status: proposed; not implemented. This roadmap records the proposed contract
+Status: agreed direction; not implemented. This roadmap describes field-level
-for reviewing changes made during application resolution. A staged implementation
+resolution auditing and identity-aware inspection of registered collections.
-plan can follow once the remaining decisions are settled.
+A staged implementation plan will follow review of the integrated roadmap.
 
 ## Purpose
 
@@ -11,9 +11,11 @@ record those fields directly. Preserve the requirement that every net change
 made during resolution has an exact or ancestor record, together with coherent
 final values, provenance, history, and sensitivity metadata.
 
-Use explicit collection identity where available to recognize records across
+For a collection with an applicable application registration, the declared key
-positions. For other lists, compare positions as addresses without inferring
+defines a record across positions. Use that correspondence in auditing and
-which application object an element represents.
+inspection. For ordinary lists, position is the only available correspondence;
+compare positions as addresses without inferring which application object an
+element represents.
 
 ## Current baseline
 
@@ -31,6 +33,12 @@ The comparator also serves other inspection and metadata operations. Making
 resolution auditing more precise must account for those callers rather than
 silently changing the public diff contract.
 
+Resolution currently captures before and after values at the same numeric path.
+Its metadata refresh also consults entry metadata at that path. Following a
+registered record through a move therefore requires more than changing which
+deltas pass the coverage audit: the retained evidence must identify the matching
+entry and final record.
+
 Application registrations already define identities for collections and constrain
 their final values. YAML-only declarations have a shorter lifetime. The
 [identity contract](../config.md#application-registrations-and-identity-details)
@@ -54,6 +62,8 @@ identity registrations:
 Apply these rules recursively to nested values. Preserve existing treatment of
 additions, removals, null, scalar differences, and changes of value kind except
 where finer sequence comparison deliberately changes the reported path.
+Ordinary-list inspection remains positional; similar field values do not justify
+moving provenance or history between positions.
 
 ### Lists with registered identity
 
@@ -73,9 +83,32 @@ Compare matched records by identity even when their numeric positions differ.
 Structural changes still require whole-list coverage. A field record by itself
 does not cover an accompanying reorder or membership change.
 
-This identity-aware comparison applies to resolution auditing; it does not
+Identity matching serves both audit coverage and inspection. Although a
-introduce a new collection merge operation or change whole-record `upsert`.
+whole-list record covers every edit in a moved collection, matching still
-YAML-only identity declarations do not acquire a new final-resolution guarantee.
+identifies which record's earlier values and source evidence belong to each
+final record. Stopping at a whole-list delta is insufficient for that purpose.
+
+Match only the resolution-entry and final snapshots. Identity changes are
+removal and addition; temporary identities, intermediate moves, and the order of
+callback mutations are not reconstructed. Matching does not introduce a new
+collection merge operation or change whole-record `upsert`.
+
+Use the existing registration selectors and scope. They traverse mapping keys,
+not sequence indexes; recursive comparison of nested lists does not expand the
+registration API. YAML-only declarations do not acquire a new final-resolution
+guarantee or become identity selectors for this work.
+
+### Matching eligibility
+
+Use identity matching when both snapshots provide valid, unique identities under
+the applicable registration. Otherwise use conservative whole-list auditing for
+a changed collection, without inventing correspondence. An unchanged collection
+does not require a resolution record merely because matching is unavailable.
+
+Preserve existing final identity validation. Resolution may repair invalid entry
+values; this feature does not reject them earlier or require valid identities
+throughout the callback. When matching is unavailable, the recorded container
+operation describes the change without claiming that individual records survived.
 
 ### Example
 
@@ -92,26 +125,64 @@ resources:
 Changing the first path to `/srv/reports` can be covered by a normalization record
 at `/resources/0/path`. The unchanged `invoices` record requires no record.
 
-Swapping the two records also changes their order, so recording only one path
+If resolution also swaps the records, the final collection is:
-does not suffice. Recording `/resources` covers that structural operation and
+
-any field edits within the list. Matching uses the declared identity rather than
+```yaml
-trying to deduce correspondence from similar field values.
+resources:
+  - name: invoices
+    path: ./invoices
+  - name: reports
+    path: /srv/reports
+```
+
+Recording only `/resources/1/path` does not cover the reorder. Recording
+`/resources` covers that structural operation and any field edits within the
+list. A field record can additionally explain the path normalization.
+
+Inspection at the final `reports` position includes that record's earlier
+evidence from position 0, with its original source coordinates, and the recorded
+collection operation explaining the move. Its previous path is `./reports`,
+not the `./invoices` value that previously occupied position 1. The `invoices`
+record moved without a field change. An ordinary unregistered list instead
+describes changes at positions 0 and 1, without identifying records across them.
 
 ### Recording, inspection, and ownership
 
-The recommended addressing convention is for resolution records and helper
+Resolution records and helper targets for surviving values describe final
-targets to describe final positions. `Resolution.EntryInfo` retains its existing
+positions. Preserve the existing ability to record removed values.
-fixed entry-snapshot meaning. Numeric paths are not automatically rebound to an
+`Resolution.EntryInfo` and `Resolution.IsExplicit` retain their existing fixed
-identity when the application later moves an element.
+entry-snapshot meaning. Numeric paths and helper targets are not automatically
+rebound to an identity when the application later moves an element. A helper
+assignment must still agree with the final value at its declared target.
+
+Identity establishes correspondence internally; this feature adds no public
+identity-addressed path or mutation-tracking API. An application that restructures
+a list can record its container rather than maintain individual helper targets
+through moves.
 
 Preserve helper-value consistency checks, reason and dependency recording,
 presence distinctions, detached query results, and independent-load ownership.
 Fine-grained accepted changes must appear consistently in the returned values,
-metadata, history, explanation, and sensitivity propagation. Do not attach an
+metadata, history, explanation, and sensitivity propagation. For eligible
-old element's attribution to a different element merely because it occupies the
+registered collections, use the same entry-to-final correspondence for before
-same index afterward. Explicit structural records remain the explanation of
+values, source attribution, input explicitness, and sensitivity. Do not attach
-application-owned list restructuring; the proposal does not reconstruct the
+the previous occupant's evidence to a different record merely because it now
-callback's intermediate mutations.
+occupies that index. Identity matching alone does not make a newly added record
+explicit input or give it an invented source.
+
+A history query at a final record's position includes the matched record's
+earlier evidence from its entry position. Preserve original source coordinates
+and include the recorded collection operation explaining the move. Recorded
+normalization and derivation retain their resolution origins; following prior
+evidence does not erase the application operation. Keep removed evidence
+available under the historical inspection contract.
+
+Explicit structural records remain the explanation of application-owned list
+restructuring. Dependency paths retain their existing recording semantics;
+identity matching does not turn supplied paths into selectors that follow later
+mutations. Check sensitivity propagation through both the matched evidence and
+recorded dependencies, including moves and removed values. The proposal does not
+reconstruct the callback's intermediate mutations.
 
 Public effective-configuration diff output remains atomic for lists. Resolution
 comparison may need a dedicated internal operation or comparison mode so that
@@ -127,13 +198,26 @@ its new behavior does not alter unrelated callers.
   structural coverage for the cases above.
 - Reuse collection identity machinery and validate matching eligibility without
   strengthening the existing final-only registration guarantee.
+- Carry matched entry and final positions into resolution snapshots, metadata,
+  history, and sensitivity handling through the existing shared history model.
+  A coverage-only implementation does not complete this feature. Preserve
+  source coordinates and distinguish a record's lineage from the prior occupant
+  of its final position.
 - Verify field edits, nested lists, zero/false/null values, multiple simultaneous
   changes, length changes, reorders, identity changes, helper targets, and
   metadata behavior across the public loading boundary. Include cases in which
   a field record must not conceal another unrecorded change.
+- Exercise moved records with distinct source evidence, explicitness, and
+  sensitivity; combined reorders and field edits; removed values; repaired entry
+  identities; and dynamic registration selectors. Check agreement between final
+  metadata and explanation while retaining entry-query behavior and atomic public
+  diff output. Review historical queries at removed and reused positions against
+  the existing historical-path contract.
 - Update the resolution contract and relevant API comments when implemented.
-  Review whether an ADR is warranted for the distinction between resolution
+  Review ADR and policy follow-up for registered identity as the correspondence
-  comparison and public diff presentation; preserve accepted ADR decision bodies.
+  used during resolution inspection, as well as the distinction between
+  resolution comparison and public diff presentation. Preserve accepted ADR
+  decision bodies.
 
 Use identity lookup tables for expected linear matching work in collection size,
 plus recursive value-comparison work. Preserve deterministic output and existing
@@ -142,31 +226,5 @@ design objective, not a new public performance guarantee.
 
 No configuration syntax, dependency, release gate, or new mutation-history system
 is proposed. Existing callbacks that record a whole list should continue to work.
-
+The matching-eligibility and final-position-addressing decisions are settled and
-## Open Decisions
+integrated above.
-
-### Eligibility when the entry snapshot has invalid registered identities
-
-**Recommendation:** use identity matching when both snapshots provide valid,
-unique identities under the applicable registration. Otherwise use conservative
-whole-list auditing for that collection and preserve existing final validation.
-This allows resolution to repair invalid entry values, as the current lifecycle
-permits, without inventing item correspondence or rejecting them earlier.
-
-**Alternative:** use positional comparison when identities cannot be matched.
-This permits finer recording of repairs but makes a registered collection switch
-between identity and positional semantics depending on its contents. Consider
-whether that flexibility justifies the additional explanation and metadata cases.
-
-### Final-position addressing for records and helper targets
-
-**Recommendation:** adopt the final-position convention described above, retaining
-entry positions for `EntryInfo`. This fits the existing verification of helper
-values against final targets and avoids introducing identity selectors into the
-public path API. Applications that restructure a list can record its container.
-
-**Alternative:** add explicit identity-based resolution targets that follow a
-record across moves. This could simplify callbacks that edit and then reorder
-records, but adds a public addressing mechanism and requires decisions about
-removed records, dependencies, nested collections, and inspection. It is a larger
-extension than the field-level auditing improvement motivating this proposal.