Record correction repair scaling findings

This commit is contained in:
2026-11-11 12:18:54 +00:00
parent d3fbddce11
commit d3a9edb29d

View File

@@ -2,10 +2,9 @@
**Updated:** 2026-11-11
**Status:** Four open findings from the stage-by-stage review of the
[implementation plan](implementation.md), against implementation revision
`aa8073c`. R1 and R2 were confirmed in the follow-up audit, which added R3 and
R4.
**Status:** Nine open findings against implementation revision `aa8073c`. The
algorithmic audit expanded R4 and added R5–R9. See the
[implementation plan](implementation.md).
## Review Result
@@ -18,9 +17,11 @@ checks and stage barrier. Ordinary reports summarize branches; debug artifacts
retain detailed evidence.
The review found gaps in outcome attribution, retained diagnostic evidence,
and replay of configured cache settings. The default validation path also
repeats replacement previews across validators. The reproduced cases preserve
correction acceptance and application.
replay of configured cache settings, and protected-alias feedback. The
algorithmic audit found repeated section scans, quadratic proposal bookkeeping,
and debug-only history retained during ordinary runs. These are local
implementation costs. The plan deliberately calls for full conversation
replay and fresh validation of each revision.
No additional strengthening of invariants, contracts, prohibitions, or guarantees
was identified in the reviewed implementation. In particular, the new semantic
@@ -144,6 +145,15 @@ again for ambiguity, and [validator request construction](../../internal/framewo
rebuilds the preview for the LLM payload. The default positive repair budget
adds this work even when all proposals pass the initial chain.
**Complexity:** For a section with `S` segments and `M` candidate versions
(initial proposals and revisions), the feedback selector alone can perform
`O(V × M × S)` target lookup work across `V` validators. Each singleton
revalidation also rebuilds section-wide indexes inside presence, nonempty-result,
and semantic validators. With a shared section index and one preview per
version that needs it, lookup and candidate traversal take expected
`O(S + V × M)` time, plus the actual text and payload work. That removes the
quadratic lookup term when proposals grow with section size.
**Measured behavior:** An offline benchmark with 100 segments and 100 valid
proposals exercised the same five deterministic validators at budgets zero
and one, with identical approvals and no repairs. On the review machine,
@@ -157,25 +167,157 @@ ambiguity, and validation payload construction for a given candidate version
where the rules use the same before/after text. Reuse a section target index and
compute previews only for checks that need them. Recompute the preview for a
revision, and preserve the separate application check against the current text.
Pass the same read-only section lookup into singleton revalidation so validators
can use the original target and its neighbors without rebuilding their indexes.
**Acceptance:** Keep the existing independent-feedback and zero-budget coverage
passing, including ambiguous spans, blank results, invalid confidence, and
protection exceptions. Re-run a focused offline benchmark to verify reduced
scanning and allocation in the feedback path.
### R5 — Index Initial Findings and Compact Filtering Once (P3)
**Affected plan:** Stage 6, items 1 and 5.
**Evidence:** [repairCandidates](../../internal/framework/runner/repair.go#L59)
calls `actionable` with the full initial rejection list for every proposal.
Each eligible proposal then calls `repairFindings` over the same full list.
Every successful revision also walks and compacts the original filtered-defect
slice. [Findings extraction](../../internal/framework/runner/feedback.go#L72)
already appends in validator order.
**Complexity:** With `P` original proposals and `D` initial rejection
records, branch selection and initial feedback can take `Θ(P × D)` time.
If a validation outage leaves one unavailable decision per proposal, this
already takes quadratic time before the runner determines no repairs can start.
With `Q` approved repairs and `F` original filtering records, repeated
compaction can add `Θ(Q × F)` work in the worst case. Both products can be
quadratic in proposal count even with a fixed validator chain.
**Refactoring:** Group initial rejections by proposal index in a single pass.
Walk the proposals in their existing order and read only each proposal's
findings. Record the approved replacements in a set and filter the original
defect list once when returning the section result, including partial results
on error. The combined bookkeeping then takes expected `O(P + D + F)` time
apart from the individual repair calls and revalidations.
**Acceptance:** Preserve validator order within findings, proposal order
across branches, no-effect and unavailable-only handling, and original
filtering evidence for candidates that remain excluded.
### R6 — Index Application Results for Repair Outcomes (P3)
**Affected plan:** Stage 6, item 5, and Stage 7, item 3.
**Evidence:** After applying the stage, [Runner.Run](../../internal/framework/runner/runner.go#L244)
scans the stage's entire applied-change slice for each approved repair. A
repair that was skipped scans the whole slice.
**Complexity:** With `B` repair branches, `Q` approved repairs, and `A`
applied changes in the stage, outcome classification costs
`O(B + Q × A)` beyond walking the work units. It can be quadratic even when
every approved repair is applied and the matches occur at successive positions.
**Refactoring:** Index the application results once by the existing stage-local
proposal index, then look up each approved repair. This takes expected
`O(A + B)` time and `O(A)` auxiliary space for an applied-index set. An
index carrying skip details could also support the evidence join in R2.
**Acceptance:** Keep disposition based on actual application. Cover a stage
with an ordinary approval, an applied revision, and an approved revision
skipped by the existing conflict checks.
### R7 — Keep Detailed Repair History in Debug Runs (P3)
**Affected plan:** Stage 7, item 3, and Stage 8, items 2–3.
**Evidence:** [repairCandidates](../../internal/framework/runner/repair.go#L178)
stores the revision, decisions, rejections, and operation slice on every
attempt, even when debug capture is disabled. The run retains those attempts in
its [work units](../../internal/framework/runner/runner.go#L597). The
[ordinary report](../../internal/framework/processreport/report.go#L126)
needs a branch's proposal index, attempt count, and disposition. It counts
executions from the work unit's separate operation list. The debug ledger reads
the revision, decisions, and artifact references. Rejections are read during
the interaction write, and the saved attempt-level operation slice has no
consumer.
**Complexity:** For `K` attempts, the runner retains `O(K + H)` additional
history across the run, where `H` is the size of the retained revision text and
validator records. With debug off, that history can be reduced to
`O(B)` branch summaries for `B` branches after ordinary operation facts
have been emitted. Those operation facts still scale with the number of calls.
The conversation and findings needed for the next attempt remain branch-local
working state.
**Refactoring:** Keep compact outcome fields separate from optional ledger
history. Retain detailed records for the ledger when debug capture is active.
Use rejections locally for feedback and artifact writes and keep one copy of
operation facts for ordinary accounting. Pass the final revision through the
existing approved-candidate path for application.
**Acceptance:** Preserve counts and compact outcomes with debug off, including
when ordinary reporting is disabled. Debug runs retain per-attempt decisions
and available artifact paths on success and interruption.
### R8 — Select a Protected-Spelling Finding in One Pass (P3)
**Affected plan:** Stage 5, item 4.
**Evidence:** The [protection checks](../../internal/framework/validators/protected_terms.go#L98)
collect every violating spelling into a slice, sort it, and read only the first
value. This code runs during validation and through the application-time
protection guard.
**Complexity:** For `K` reported violations, selection uses up to
`O(K log K)` string comparisons and `O(K)` extra storage. A running
lexical minimum needs `O(K)` comparisons and constant auxiliary space.
The matching and identity checks still do their existing work.
**Refactoring:** Select the same deterministic representative as violations
are detected, using the evidence corrected in R9. Preserve the precedence
of identity loss over capitalization and the existing rejection rules.
### R9 — Name the Affected Protected Alias (P3)
**Affected plan:** Stage 5, item 4.
**Evidence:** When an identity loses a mention, the
[identity-preservation check](../../internal/framework/validators/protected_terms.go#L98)
uses the first prior occurrence's spelling for its finding. An identity can
have multiple protected aliases. The first spelling can still be present
after the edit.
**Reproduced behavior:** Protect both `Mira` and `Captain Mira` under one
identity. In a glossary stage, replace `Captain Mira` with `the captain` in
`Mira spoke to Captain Mira`. The validator correctly rejects the edit but
says it may alter `Mira`, the spelling that stayed. The same misleading
message becomes repair feedback.
**Refactoring:** Derive a specific spelling from before/after occurrence
counts for the identity that lost a mention, or describe the identity loss
when a spelling cannot be identified reliably. Counting spelling deficits
can be done in expected linear time in matched occurrences. Preserve the
existing identity-level protection decision and spoken-word repetition
exception.
**Acceptance:** Exercise two aliases of one identity. The finding identifies
the removed alias or the affected identity without attributing the edit to
the unchanged spelling.
## Stage Review Coverage
| Stage | Evidence reviewed | Result |
| --- | --- | --- |
| 1 — Response and native composition | Repair schema, correspondence decoder, native multi-turn wire/schema test with a separate repair profile | No additional finding |
| 2 — Module prompts | Four registered assets, shared contract, rendering tests, operator replacement and message-file loading | No additional finding |
| 3 — Exchange and adapter | Capture through proposal contracts, frozen variables, prefix comparison, independent suffixes, cache control, failure classification, native structural-repair accounting | R3 |
| 4 — Configuration and readiness | Root/stage resolution, null fallback, zero and positive budgets, recorded resolution, disabled-profile readiness, CLI role wiring | No additional finding |
| 5 — Findings | Applicability selection, preview reuse, independent deterministic/semantic findings, unavailable assessments, no-effect handling, zero-budget shortlist, cardinality and protection checks | R4 |
| 6 — Branch execution and application | Per-proposal budget, retargeting, deletion, withdrawal, exhaustion, cancellation, full revalidation, deterministic application and stage commit | R1; application evidence feeds R2 |
| 7 — Ordinary outcomes | Operation ordering, logical/native counts, compact branch rows, report-disabled summaries, error/recovery mapping | R1 |
| 8 — Debug and ledger | Artifact coordinates, redaction, optional-write failures, original/revised identities, interrupted evidence, application-result join | R2 |
| 9 — Assembled command and docs | Controlled-provider subprocess flows, prompt override, cancellation, output selections, canonical documentation and preview example | Existing checks pass |
| 1 — Response and native composition | Repair schema, typed decode, correspondence, native multi-turn test | Response processing scales with response size. |
| 2 — Module prompts | Four registered definitions, shared contract, rendering and operator replacements | Per-request feedback serialization scales with supplied content. |
| 3 — Exchange and adapter | Shared section exchange, prepared prefix, branch history, native failure handling | R3. Cost grows with the sum of the full requests sent, as specified in the plan. |
| 4 — Configuration and readiness | Root/stage resolution, zero and positive budgets, profile readiness, role creation | A fixed number of role selections per stage. Readiness deduplicates profile inspections. |
| 5 — Findings | Applicability, previews, cardinality, protection, semantic input and batching | R4, R8, R9. Decision cardinality uses indexed checks. |
| 6 — Branch execution and application | Eligibility, versioned revalidation, defect attribution, application outcomes and cancellation | R1, R4–R6. Application evidence feeds R2. |
| 7 — Ordinary outcomes | Operation aggregation and sorting, report-disabled summaries, compact branch rows | R1. New presentation work scans and sorts the recorded facts. |
| 8 — Debug and ledger | Attempt retention, optional artifacts, redaction, indexed lineage and interrupted joins | R2, R7. The ledger joins decisions and branches by identity. |
| 9 — Assembled command and docs | Controlled-provider subprocess flows, cancellation, output selection, documentation and preview | Existing tests cover the principal execution paths. The recorded findings identify the gaps. |
## Validation Performed
@@ -190,6 +332,11 @@ scanning and allocation in the feedback path.
checked the links in this roadmap and ran `git diff --check`.
- Temporary Go overlays reproduced the defects in R1–R3 and measured the
validation overhead in R4 against the committed implementation.
- The algorithmic audit derived scaling from the current source and its
callers. A temporary Go overlay reproduced the wrong-alias finding in R9.
- Fresh tests passed for runner, validators, process reporting, LLM adapter,
preflight, response schemas, proposal generation, configuration, and prompts
during the algorithmic audit.
Resolve the findings with focused checks at their owning boundaries. Retain the
current architecture and accepted operator choices.