Record the proposal for field-level list resolution

docs/roadmap/resolution-sequences.md

@@ -0,0 +1,172 @@
+# Field-level resolution auditing for sequences
+
+Status: proposed; not implemented. This roadmap records the proposed contract
+for reviewing changes made during application resolution. A staged implementation
+plan can follow once the remaining decisions are settled.
+
+## Purpose
+
+Allow applications to normalize or derive individual fields inside lists and
+record those fields directly. Preserve the requirement that every net change
+made during resolution has an exact or ancestor record, together with coherent
+final values, provenance, history, and sensitivity metadata.
+
+Use explicit collection identity where available to recognize records across
+positions. For other lists, compare positions as addresses without inferring
+which application object an element represents.
+
+## Current baseline
+
+The [resolution contract](../config.md#recorded-application-resolution) audits
+the typed configuration before and after `Definition.Resolve`. Mappings are
+compared recursively, while the private structural comparator reports any list
+change at the whole-list path. A field-level record inside a changed list
+therefore fails to cover that list-level delta. Recording the whole list works.
+
+For example, changing `/resources/0/path` and recording only that path currently
+produces an unrecorded-change diagnostic at `/resources`. This is established
+behavior rather than a new regression associated with v1.0.0.
+
+The comparator also serves other inspection and metadata operations. Making
+resolution auditing more precise must account for those callers rather than
+silently changing the public diff contract.
+
+Application registrations already define identities for collections and constrain
+their final values. YAML-only declarations have a shorter lifetime. The
+[identity contract](../config.md#application-registrations-and-identity-details)
+and [ADR 0017](../adr/0017-collection-sequence-policy.md) own those distinctions.
+
+## Target behavior
+
+### Mappings and ordinary lists
+
+Mapping comparison remains recursive. For lists without applicable application
+identity registrations:
+
+- If the lists have equal lengths, compare corresponding positions recursively.
+  A position is an address, not an assertion of element identity.
+- If the length changes, report a structural change at the list path, requiring
+  a record for that list or an ancestor.
+- For equal-length lists, a reorder manifests as changes at the affected
+  positions. Every detected change needs coverage; a whole-list record remains
+  a convenient way to cover the operation.
+
+Apply these rules recursively to nested values. Preserve existing treatment of
+additions, removals, null, scalar differences, and changes of value kind except
+where finer sequence comparison deliberately changes the reported path.
+
+### Lists with registered identity
+
+Use applicable `Definition.SequenceIdentities` registrations to match records
+between the entry and final snapshots. Reuse existing identity-field selection,
+compound-identity, scalar-equality, and duplicate-detection semantics. Do not infer
+an identity from a field merely named `name` or `id`.
+
+| Observed change | Proposed auditing behavior |
+| --- | --- |
+| An ordinary field changes on a matched record | Compare recursively and allow a record covering that field. |
+| Identity order changes | Report a structural change at the list path; require a list or ancestor record. |
+| An identity is added or removed | Report a structural change at the list path; require a list or ancestor record. |
+| An identity changes | Treat it as removal and addition, without inferring a rename. |
+
+Compare matched records by identity even when their numeric positions differ.
+Structural changes still require whole-list coverage. A field record by itself
+does not cover an accompanying reorder or membership change.
+
+This identity-aware comparison applies to resolution auditing; it does not
+introduce a new collection merge operation or change whole-record `upsert`.
+YAML-only identity declarations do not acquire a new final-resolution guarantee.
+
+### Example
+
+Assume the application registers `name` as the identity for `resources`:
+
+```yaml
+resources:
+  - name: reports
+    path: ./reports
+  - name: invoices
+    path: ./invoices
+```
+
+Changing the first path to `/srv/reports` can be covered by a normalization record
+at `/resources/0/path`. The unchanged `invoices` record requires no record.
+
+Swapping the two records also changes their order, so recording only one path
+does not suffice. Recording `/resources` covers that structural operation and
+any field edits within the list. Matching uses the declared identity rather than
+trying to deduce correspondence from similar field values.
+
+### Recording, inspection, and ownership
+
+The recommended addressing convention is for resolution records and helper
+targets to describe final positions. `Resolution.EntryInfo` retains its existing
+fixed entry-snapshot meaning. Numeric paths are not automatically rebound to an
+identity when the application later moves an element.
+
+Preserve helper-value consistency checks, reason and dependency recording,
+presence distinctions, detached query results, and independent-load ownership.
+Fine-grained accepted changes must appear consistently in the returned values,
+metadata, history, explanation, and sensitivity propagation. Do not attach an
+old element's attribution to a different element merely because it occupies the
+same index afterward. Explicit structural records remain the explanation of
+application-owned list restructuring; the proposal does not reconstruct the
+callback's intermediate mutations.
+
+Public effective-configuration diff output remains atomic for lists. Resolution
+comparison may need a dedicated internal operation or comparison mode so that
+its new behavior does not alter unrelated callers.
+
+## Work needed from the baseline
+
+- Trace resolution comparison, record coverage, helper verification, metadata
+  updates, history replay, and sensitivity handling to identify where each needs
+  finer sequence changes or explicit before/after correspondence.
+- Add positional recursive comparison for ordinary equal-length lists and
+  identity-aware comparison for eligible registered lists, while retaining
+  structural coverage for the cases above.
+- Reuse collection identity machinery and validate matching eligibility without
+  strengthening the existing final-only registration guarantee.
+- Verify field edits, nested lists, zero/false/null values, multiple simultaneous
+  changes, length changes, reorders, identity changes, helper targets, and
+  metadata behavior across the public loading boundary. Include cases in which
+  a field record must not conceal another unrecorded change.
+- Update the resolution contract and relevant API comments when implemented.
+  Review whether an ADR is warranted for the distinction between resolution
+  comparison and public diff presentation; preserve accepted ADR decision bodies.
+
+Use identity lookup tables for expected linear matching work in collection size,
+plus recursive value-comparison work. Preserve deterministic output and existing
+equality semantics; avoid pairwise searches for matching records. This is a
+design objective, not a new public performance guarantee.
+
+No configuration syntax, dependency, release gate, or new mutation-history system
+is proposed. Existing callbacks that record a whole list should continue to work.
+
+## Open Decisions
+
+### Eligibility when the entry snapshot has invalid registered identities
+
+**Recommendation:** use identity matching when both snapshots provide valid,
+unique identities under the applicable registration. Otherwise use conservative
+whole-list auditing for that collection and preserve existing final validation.
+This allows resolution to repair invalid entry values, as the current lifecycle
+permits, without inventing item correspondence or rejecting them earlier.
+
+**Alternative:** use positional comparison when identities cannot be matched.
+This permits finer recording of repairs but makes a registered collection switch
+between identity and positional semantics depending on its contents. Consider
+whether that flexibility justifies the additional explanation and metadata cases.
+
+### Final-position addressing for records and helper targets
+
+**Recommendation:** adopt the final-position convention described above, retaining
+entry positions for `EntryInfo`. This fits the existing verification of helper
+values against final targets and avoids introducing identity selectors into the
+public path API. Applications that restructure a list can record its container.
+
+**Alternative:** add explicit identity-based resolution targets that follow a
+record across moves. This could simplify callbacks that edit and then reorder
+records, but adds a public addressing mechanism and requires decisions about
+removed records, dependencies, nested collections, and inspection. It is a larger
+extension than the field-level auditing improvement motivating this proposal.