diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md index 469a278..eeee227 100644 --- a/docs/roadmap/remediation.md +++ b/docs/roadmap/remediation.md @@ -2,10 +2,9 @@ **Updated:** 2026-11-11 -**Status:** Four open findings from the stage-by-stage review of the -[implementation plan](implementation.md), against implementation revision -`aa8073c`. R1 and R2 were confirmed in the follow-up audit, which added R3 and -R4. +**Status:** Nine open findings against implementation revision `aa8073c`. The +algorithmic audit expanded R4 and added R5–R9. See the +[implementation plan](implementation.md). ## Review Result @@ -18,9 +17,11 @@ checks and stage barrier. Ordinary reports summarize branches; debug artifacts retain detailed evidence. The review found gaps in outcome attribution, retained diagnostic evidence, -and replay of configured cache settings. The default validation path also -repeats replacement previews across validators. The reproduced cases preserve -correction acceptance and application. +replay of configured cache settings, and protected-alias feedback. The +algorithmic audit found repeated section scans, quadratic proposal bookkeeping, +and debug-only history retained during ordinary runs. These are local +implementation costs. The plan deliberately calls for full conversation +replay and fresh validation of each revision. No additional strengthening of invariants, contracts, prohibitions, or guarantees was identified in the reviewed implementation. In particular, the new semantic @@ -144,6 +145,15 @@ again for ambiguity, and [validator request construction](../../internal/framewo rebuilds the preview for the LLM payload. The default positive repair budget adds this work even when all proposals pass the initial chain. +**Complexity:** For a section with `S` segments and `M` candidate versions +(initial proposals and revisions), the feedback selector alone can perform +`O(V × M × S)` target lookup work across `V` validators. Each singleton +revalidation also rebuilds section-wide indexes inside presence, nonempty-result, +and semantic validators. With a shared section index and one preview per +version that needs it, lookup and candidate traversal take expected +`O(S + V × M)` time, plus the actual text and payload work. That removes the +quadratic lookup term when proposals grow with section size. + **Measured behavior:** An offline benchmark with 100 segments and 100 valid proposals exercised the same five deterministic validators at budgets zero and one, with identical approvals and no repairs. On the review machine, @@ -157,25 +167,157 @@ ambiguity, and validation payload construction for a given candidate version where the rules use the same before/after text. Reuse a section target index and compute previews only for checks that need them. Recompute the preview for a revision, and preserve the separate application check against the current text. +Pass the same read-only section lookup into singleton revalidation so validators +can use the original target and its neighbors without rebuilding their indexes. **Acceptance:** Keep the existing independent-feedback and zero-budget coverage passing, including ambiguous spans, blank results, invalid confidence, and protection exceptions. Re-run a focused offline benchmark to verify reduced scanning and allocation in the feedback path. +### R5 — Index Initial Findings and Compact Filtering Once (P3) + +**Affected plan:** Stage 6, items 1 and 5. + +**Evidence:** [repairCandidates](../../internal/framework/runner/repair.go#L59) +calls `actionable` with the full initial rejection list for every proposal. +Each eligible proposal then calls `repairFindings` over the same full list. +Every successful revision also walks and compacts the original filtered-defect +slice. [Findings extraction](../../internal/framework/runner/feedback.go#L72) +already appends in validator order. + +**Complexity:** With `P` original proposals and `D` initial rejection +records, branch selection and initial feedback can take `Θ(P × D)` time. +If a validation outage leaves one unavailable decision per proposal, this +already takes quadratic time before the runner determines no repairs can start. +With `Q` approved repairs and `F` original filtering records, repeated +compaction can add `Θ(Q × F)` work in the worst case. Both products can be +quadratic in proposal count even with a fixed validator chain. + +**Refactoring:** Group initial rejections by proposal index in a single pass. +Walk the proposals in their existing order and read only each proposal's +findings. Record the approved replacements in a set and filter the original +defect list once when returning the section result, including partial results +on error. The combined bookkeeping then takes expected `O(P + D + F)` time +apart from the individual repair calls and revalidations. + +**Acceptance:** Preserve validator order within findings, proposal order +across branches, no-effect and unavailable-only handling, and original +filtering evidence for candidates that remain excluded. + +### R6 — Index Application Results for Repair Outcomes (P3) + +**Affected plan:** Stage 6, item 5, and Stage 7, item 3. + +**Evidence:** After applying the stage, [Runner.Run](../../internal/framework/runner/runner.go#L244) +scans the stage's entire applied-change slice for each approved repair. A +repair that was skipped scans the whole slice. + +**Complexity:** With `B` repair branches, `Q` approved repairs, and `A` +applied changes in the stage, outcome classification costs +`O(B + Q × A)` beyond walking the work units. It can be quadratic even when +every approved repair is applied and the matches occur at successive positions. + +**Refactoring:** Index the application results once by the existing stage-local +proposal index, then look up each approved repair. This takes expected +`O(A + B)` time and `O(A)` auxiliary space for an applied-index set. An +index carrying skip details could also support the evidence join in R2. + +**Acceptance:** Keep disposition based on actual application. Cover a stage +with an ordinary approval, an applied revision, and an approved revision +skipped by the existing conflict checks. + +### R7 — Keep Detailed Repair History in Debug Runs (P3) + +**Affected plan:** Stage 7, item 3, and Stage 8, items 2–3. + +**Evidence:** [repairCandidates](../../internal/framework/runner/repair.go#L178) +stores the revision, decisions, rejections, and operation slice on every +attempt, even when debug capture is disabled. The run retains those attempts in +its [work units](../../internal/framework/runner/runner.go#L597). The +[ordinary report](../../internal/framework/processreport/report.go#L126) +needs a branch's proposal index, attempt count, and disposition. It counts +executions from the work unit's separate operation list. The debug ledger reads +the revision, decisions, and artifact references. Rejections are read during +the interaction write, and the saved attempt-level operation slice has no +consumer. + +**Complexity:** For `K` attempts, the runner retains `O(K + H)` additional +history across the run, where `H` is the size of the retained revision text and +validator records. With debug off, that history can be reduced to +`O(B)` branch summaries for `B` branches after ordinary operation facts +have been emitted. Those operation facts still scale with the number of calls. +The conversation and findings needed for the next attempt remain branch-local +working state. + +**Refactoring:** Keep compact outcome fields separate from optional ledger +history. Retain detailed records for the ledger when debug capture is active. +Use rejections locally for feedback and artifact writes and keep one copy of +operation facts for ordinary accounting. Pass the final revision through the +existing approved-candidate path for application. + +**Acceptance:** Preserve counts and compact outcomes with debug off, including +when ordinary reporting is disabled. Debug runs retain per-attempt decisions +and available artifact paths on success and interruption. + +### R8 — Select a Protected-Spelling Finding in One Pass (P3) + +**Affected plan:** Stage 5, item 4. + +**Evidence:** The [protection checks](../../internal/framework/validators/protected_terms.go#L98) +collect every violating spelling into a slice, sort it, and read only the first +value. This code runs during validation and through the application-time +protection guard. + +**Complexity:** For `K` reported violations, selection uses up to +`O(K log K)` string comparisons and `O(K)` extra storage. A running +lexical minimum needs `O(K)` comparisons and constant auxiliary space. +The matching and identity checks still do their existing work. + +**Refactoring:** Select the same deterministic representative as violations +are detected, using the evidence corrected in R9. Preserve the precedence +of identity loss over capitalization and the existing rejection rules. + +### R9 — Name the Affected Protected Alias (P3) + +**Affected plan:** Stage 5, item 4. + +**Evidence:** When an identity loses a mention, the +[identity-preservation check](../../internal/framework/validators/protected_terms.go#L98) +uses the first prior occurrence's spelling for its finding. An identity can +have multiple protected aliases. The first spelling can still be present +after the edit. + +**Reproduced behavior:** Protect both `Mira` and `Captain Mira` under one +identity. In a glossary stage, replace `Captain Mira` with `the captain` in +`Mira spoke to Captain Mira`. The validator correctly rejects the edit but +says it may alter `Mira`, the spelling that stayed. The same misleading +message becomes repair feedback. + +**Refactoring:** Derive a specific spelling from before/after occurrence +counts for the identity that lost a mention, or describe the identity loss +when a spelling cannot be identified reliably. Counting spelling deficits +can be done in expected linear time in matched occurrences. Preserve the +existing identity-level protection decision and spoken-word repetition +exception. + +**Acceptance:** Exercise two aliases of one identity. The finding identifies +the removed alias or the affected identity without attributing the edit to +the unchanged spelling. + ## Stage Review Coverage | Stage | Evidence reviewed | Result | | --- | --- | --- | -| 1 — Response and native composition | Repair schema, correspondence decoder, native multi-turn wire/schema test with a separate repair profile | No additional finding | -| 2 — Module prompts | Four registered assets, shared contract, rendering tests, operator replacement and message-file loading | No additional finding | -| 3 — Exchange and adapter | Capture through proposal contracts, frozen variables, prefix comparison, independent suffixes, cache control, failure classification, native structural-repair accounting | R3 | -| 4 — Configuration and readiness | Root/stage resolution, null fallback, zero and positive budgets, recorded resolution, disabled-profile readiness, CLI role wiring | No additional finding | -| 5 — Findings | Applicability selection, preview reuse, independent deterministic/semantic findings, unavailable assessments, no-effect handling, zero-budget shortlist, cardinality and protection checks | R4 | -| 6 — Branch execution and application | Per-proposal budget, retargeting, deletion, withdrawal, exhaustion, cancellation, full revalidation, deterministic application and stage commit | R1; application evidence feeds R2 | -| 7 — Ordinary outcomes | Operation ordering, logical/native counts, compact branch rows, report-disabled summaries, error/recovery mapping | R1 | -| 8 — Debug and ledger | Artifact coordinates, redaction, optional-write failures, original/revised identities, interrupted evidence, application-result join | R2 | -| 9 — Assembled command and docs | Controlled-provider subprocess flows, prompt override, cancellation, output selections, canonical documentation and preview example | Existing checks pass | +| 1 — Response and native composition | Repair schema, typed decode, correspondence, native multi-turn test | Response processing scales with response size. | +| 2 — Module prompts | Four registered definitions, shared contract, rendering and operator replacements | Per-request feedback serialization scales with supplied content. | +| 3 — Exchange and adapter | Shared section exchange, prepared prefix, branch history, native failure handling | R3. Cost grows with the sum of the full requests sent, as specified in the plan. | +| 4 — Configuration and readiness | Root/stage resolution, zero and positive budgets, profile readiness, role creation | A fixed number of role selections per stage. Readiness deduplicates profile inspections. | +| 5 — Findings | Applicability, previews, cardinality, protection, semantic input and batching | R4, R8, R9. Decision cardinality uses indexed checks. | +| 6 — Branch execution and application | Eligibility, versioned revalidation, defect attribution, application outcomes and cancellation | R1, R4–R6. Application evidence feeds R2. | +| 7 — Ordinary outcomes | Operation aggregation and sorting, report-disabled summaries, compact branch rows | R1. New presentation work scans and sorts the recorded facts. | +| 8 — Debug and ledger | Attempt retention, optional artifacts, redaction, indexed lineage and interrupted joins | R2, R7. The ledger joins decisions and branches by identity. | +| 9 — Assembled command and docs | Controlled-provider subprocess flows, cancellation, output selection, documentation and preview | Existing tests cover the principal execution paths. The recorded findings identify the gaps. | ## Validation Performed @@ -190,6 +332,11 @@ scanning and allocation in the feedback path. checked the links in this roadmap and ran `git diff --check`. - Temporary Go overlays reproduced the defects in R1–R3 and measured the validation overhead in R4 against the committed implementation. +- The algorithmic audit derived scaling from the current source and its + callers. A temporary Go overlay reproduced the wrong-alias finding in R9. +- Fresh tests passed for runner, validators, process reporting, LLM adapter, + preflight, response schemas, proposal generation, configuration, and prompts + during the algorithmic audit. Resolve the findings with focused checks at their owning boundaries. Retain the current architecture and accepted operator choices.