Record the algorithmic audit findings

docs/roadmap/remediation.md

@@ -11,6 +11,12 @@ provider. A normal grammar correction produced the same transcript and
 completed both validators on each revision. Differences at the output,
 credential, and glossary boundaries are recorded in R10–R12.
 
+The algorithmic audit traced the implementation and its pinned library calls
+without performance measurements. R13–R20 identify avoidable work within the
+adopted design. Fixed transcript chunks, ordered stages, native provider
+admission, glossary selection and matching rules, and report schemas remain
+the basis of the analysis.
+
 ## Architecture Assessment
 
 ConfigKit selects and composes the v3 application configuration, including
@@ -30,12 +36,12 @@ validation, source publication, and asynchronous Linux delivery.
 | Stages | Reviewed behavior |
 | --- | --- |
 | 1 and 15 | Dependency and toolchain pins, release profile, launcher, tag verification, and candidate evidence. A disposable ReleaseKit check passed with Go 1.27.3, including race tests, vet, Markdown and module checks, both Linux builds, and version output. The plan records the annotated-tag delivery rehearsal. |
-| 2–4 | Native application load, main-relative engine inputs, prompt resources and source accounting, prepared role clients, retries and repairs, and redaction. See R2, R4, and R10. |
-| 5–6 | Native app and PromptKit scopes, typed routing, preview, validation, and explicit smoke, including controlled transport tests. See R5. |
-| 7–8 | Destination checks, deferred device/pipe delivery, transcript replacement, ordinary reports, optional debug, and reporting modes. See R1, R7, R9, and R11. |
-| 9–10 | Native identified glossary composition, typed extensions and selections, recorded defaults, and explicit clearing after CLI assignments. |
-| 11–12 | Stage-specific glossary views, late-stage readiness, fixed chunks, ordered mutation, and partial success. See R3, R9, and R12. |
-| 13–14 | Stage attribution, report schemas, canonical docs, migration, examples, ADR, and attribution. See R1, R6, R7, and R8. |
+| 2–4 | Native application load, main-relative engine inputs, prompt resources and source accounting, prepared role clients, retries and repairs, and redaction. See R2, R4, R10, R13, and R14. |
+| 5–6 | Native app and PromptKit scopes, typed routing, preview, validation, and explicit smoke, including controlled transport tests. See R5 and R13. |
+| 7–8 | Destination checks, deferred device/pipe delivery, transcript replacement, ordinary reports, optional debug, and reporting modes. The output-path loop compares a growing source list against at most four destinations. See R1, R7, R9, R11, and R18. |
+| 9–10 | Native identified glossary composition, typed extensions and selections, recorded defaults, and explicit clearing after CLI assignments. See R15. |
+| 11–12 | Stage-specific glossary views, late-stage readiness, fixed chunks, ordered mutation, and partial success. See R3, R9, R12, R15–R17, and R20. |
+| 13–14 | Stage attribution, report schemas, canonical docs, migration, examples, ADR, and attribution. The complete stage-by-chunk matrices are part of enabled reports. See R1, R6–R8, R18, and R19. |
 
 ## R1 — P2: Keep Failure Records and Counts Consistent
 
@@ -417,3 +423,275 @@ An exempt full name can be corrected when a shorter protected term occurs
 inside it. A standalone occurrence of the shorter term and a distinct longer
 protected name retain protection. Other module stages keep their independently
 selected vocabulary.
+
+## R13 — P2: Collect Prompt Message Paths From Effective Data
+
+**Category**
+
+Repeated configuration history replay in Stages 3, 5, and 6.
+
+**Evidence and complexity**
+
+Audita enumerates historical fields and calls `Explain` for every effective
+host-backed `content_file`, then reads only the effective YAML scalar.
+ConfigKit v1.2.0's implementation of `Explain` replays source applications
+and clones configuration checkpoints on each call. With `M` such fields and
+`H` source history records, scanning history alone costs at least
+`Ω(M × H)`. One file-backed message per source gives quadratic growth in
+the number of sources, before accounting for replay and cloning costs.
+The native resource commands also do the work because they build a runtime
+and then retain only its engine.
+[Source-path capture](../../internal/framework/llm/source_paths.go#L40) ·
+[Engine wiring](../../internal/cli/app_wiring.go#L62)
+
+**Refactoring opportunity**
+
+Collect collision paths only for processing. Use one walk of the native
+effective projection with per-field origin lookups, or use an upstream
+effective-only accessor if it can expose the required path evidence. Keep
+the actual load and source-relative resolution with the owning libraries.
+
+**Acceptance**
+
+Source accounting for processing uses a single effective pass and origin
+lookups without replaying history per message. Inherited and relocated
+message files still receive collision protection, including the host context
+covered by R2. Inspection retains its native history commands.
+
+## R14 — P3: Reuse Credential Source Discovery Across Diagnostics
+
+**Category**
+
+Repeated catalog scans in Stages 4 and 8.
+
+**Evidence and complexity**
+
+`ConfiguredSecrets` lists and inspects every backend and profile each time.
+Debug processing calls it per proposal interaction and per validator in each
+active section, including deterministic validators that never write a model
+interaction. Finalization also recollects it for each warning. With `D`
+redaction preparations and `C` catalog resources, this is at least
+`Θ(D × C)` resource visits, plus the work of copying each inspection result.
+[Credential collection](../../internal/framework/llm/secrets.go#L11) ·
+[Validator writer setup](../../internal/framework/runner/runner.go#L620) ·
+[Warning presentation](../../internal/cli/process_output.go#L14)
+
+**Refactoring opportunity**
+
+Discover deduplicated credential source names once per captured engine.
+Read values at the existing masking boundary, including additional names
+supplied by prepared requests and normalized values from R10. Reuse writers
+or redactors where they already share a value set, and construct interaction
+writers only for consumers that can use them. This reduces catalog discovery
+to one pass while retaining the cost of reading the referenced values.
+
+**Acceptance**
+
+Increasing the number of sections or warnings does not cause repeated
+resource inventory scans. All existing masking behavior remains available,
+and PromptKit still controls credential selection and availability.
+
+## R15 — P3: Avoid Repeated Glossary Selection and Membership Scans
+
+**Category**
+
+Redundant list work and quadratic deduplication in Stages 10–12.
+
+**Evidence and complexity**
+
+- `resolveSettings` clones root lists before ConfigKit checks whether a stage
+  already supplied a list. Across `S` stages and `R` root list elements,
+  this copies `Θ(S × R)` elements even if every stage overrides both lists.
+- `Select` walks a collection on every occurrence of its name. Selecting a
+  collection of `G` entries `K` times visits `Θ(K × G)` entries and
+  returns only `G`. Readiness also materializes a full rich selection just
+  to test whether it is empty.
+- `Vocabulary` scans a spelling's existing contributor slice for every
+  insertion. When `K` distinct entries share an alias, the membership tests
+  for that alias alone take `Θ(K²)` comparisons.
+
+[Recorded defaults](../../internal/core/config/resolve.go#L21) ·
+[Glossary selection](../../internal/core/glossary/selection.go#L19) ·
+[Readiness](../../internal/framework/preflight/preflight.go#L95) ·
+[Contributor grouping](../../internal/core/glossary/vocabulary.go#L19)
+
+**Refactoring opportunity**
+
+Clone only lists that inherit. Skip repeated collection names before walking
+their entries, and stop the readiness test at the first populated selection.
+Use a membership set keyed by spelling and entry identity while keeping the
+ordered output slices. That gives expected constant-time membership per
+contribution, excluding the cost of normalizing and hashing the text.
+
+**Acceptance**
+
+First occurrence, contributor order, exact collection-scoped IDs, detached
+inherited lists, and the distinction between omission and explicit clearing
+are preserved. Readiness keeps the same late-stage diagnostics.
+
+## R16 — P2: Reuse Prepared Glossary Views and Index Validation Context
+
+**Category**
+
+Per-section rebuilds and per-candidate glossary scans in Stages 11 and 12.
+
+**Evidence and complexity**
+
+The selected stage vocabulary stays fixed, but protected-term validation
+compiles its matcher for each active section. Each LLM validator rebuilds the
+context matcher again for its section. With `Q` such invocations and
+`B` the cost of building a matcher, that is `Θ(Q × B)` preparation for
+the same data. Proposal requests also re-encode identical glossary JSON in
+each section. Within LLM validation, after matching text, every candidate
+scans all `G` selected entries to recover their order. That adds
+`Θ(P × G)` work for `P` candidates, even when nothing matches.
+[Protected-term validation](../../internal/framework/validators/deterministic.go#L139) ·
+[LLM validator request](../../internal/framework/validators/llm_validators.go#L252) ·
+[Context selection](../../internal/framework/validators/llm_context.go#L12) ·
+[Proposal variables](../../internal/framework/proposal_generation/variables.go#L14)
+
+**Refactoring opportunity**
+
+Prepare immutable stage-local matchers and, where used, serialized glossary
+variables once. Build an identity-to-entry-position map alongside the context
+view. Gather just the matched positions and order those positions before
+materializing entries. After indexing, selecting `k` distinct matches per
+candidate costs expected `O(k + k log k)` apart from text matching, rather
+than a full glossary scan. Sharing the encoding saves local serialization
+while leaving the selected glossary payload in each request.
+
+**Acceptance**
+
+Rich entries retain selection order and all contributing identities. Scope
+and overlap rules remain as specified, including R3 and R12. Matchers and
+immutable render inputs are shared safely across sections of the same stage.
+
+## R17 — P3: Reuse Validation Payload Costs While Fitting and Batching
+
+**Category**
+
+Quadratic prefix serialization in the existing budgeting path now used with
+stage-specific budgets and shared glossary context.
+
+**Evidence and complexity**
+
+`fitValidationContext` removes one optional entry at a time and serializes
+the remaining item again. Removing most of `K` similarly sized entries
+serializes `Θ(K²)` entry data. `ChunkLLMValidationItems` copies, renumbers,
+and marshals the entire tentative batch on each append. A batch that grows
+to `N` similarly sized candidates costs `Θ(N²)` in those operations.
+Small budgets bound the batch size, but the configured budget may grow and
+many glossary entries can match one candidate.
+[Context fitting](../../internal/framework/validators/llm_context.go#L43) ·
+[Batch construction](../../internal/framework/validators/llm_batching.go#L16) ·
+[Stage budget use](../../internal/framework/validators/llm_validators.go#L74)
+
+**Refactoring opportunity**
+
+Keep the existing drop order, greedy batch boundaries, and production
+estimator. For that estimator, compose counts of whitespace-delimited runs
+and punctuation from encoded components, accounting for join boundaries
+and rounding once on the complete payload. Use those costs for prefix
+selection and incremental batch construction, then render the selected
+payload. Keep exact serialization for estimators whose costs cannot be
+composed. This can reduce production work to one pass over the encoded
+components and inexpensive checks over the candidate prefixes.
+
+**Acceptance**
+
+Representative payloads produce the same retained context, estimates,
+oversized-candidate decisions, and batch boundaries. Include local reference
+numbering and punctuation or escaped text at encoding boundaries.
+
+## R18 — P3: Skip Evidence Work for Disabled Outputs
+
+**Category**
+
+Unused report rendering and event collection in Stages 8 and 13.
+
+**Evidence and complexity**
+
+Preparation renders both initial report bundles and finalization renders
+the final bundle even with reporting disabled. For `S` stages and `Q`
+chunks, this still builds two `Θ(S × Q)` Markdown matrices at finalization,
+in addition to serializing error and recovery records. Processing also
+retains all detailed LLM observations, copies them at completion, and sorts
+them before testing whether debug artifacts are active. For `E` calls and
+events, that is `Θ(E)` storage and up to `O(E log E)` sort comparisons.
+Ordinary outcome counts use the role client's terminal event, independently
+of that detailed collector.
+[Report preparation](../../internal/cli/process_inputs.go#L90) ·
+[Report delivery](../../internal/cli/process_output.go#L32) ·
+[Observation wiring](../../internal/cli/process_pipeline.go#L18) ·
+[Snapshot sorting](../../internal/framework/llm/observations.go#L148)
+
+**Refactoring opportunity**
+
+Build file payloads only when ordinary reporting is enabled. Install detailed
+observation hooks only for an active debug writer, while retaining terminal
+counts and compact completion facts for ordinary reports and CLI summaries.
+Coordinate removal of unused utilization bookkeeping with R7.
+
+**Acceptance**
+
+Disabled outputs skip their formatting and optional observation costs.
+Enabled reports and debug artifacts retain their data, and verbose completion
+counts remain available with ordinary reporting disabled.
+
+## R19 — P3: Index Interrupted Correction Ledger Entries
+
+**Category**
+
+Repeated joins in the Stage 13 debug ledger.
+
+**Evidence and complexity**
+
+The first pass indexes decisions by proposal for each stage and discards the
+index after emitting completed entries. Every interrupted candidate then
+searches stage results and scans the matching stage's entire decision list
+again. With `I` interrupted candidates, `S` stages, and `D` decisions in
+one failed stage, lookup can cost `Θ(I × (S + D))`. Interruption late in a
+large stage can make both `I` and `D` grow with candidate count.
+[Decision indexing and interrupted fallback](../../internal/framework/processreport/correction_ledger.go#L64)
+
+**Refactoring opportunity**
+
+Reuse the first-pass decision index through the interrupted-entry join and
+index stage metadata by generated instance. Expected lookup and grouping
+become linear in stages, candidates, and decisions visited, plus the
+existing final sort. Keep operator IDs as labels.
+
+**Acceptance**
+
+Interrupted entries retain the same stage identity, partial validator
+decisions, and output order without rescanning decisions for each candidate.
+
+## R20 — P3: Skip Apply-Time Copies for Sections With No Approved Edits
+
+**Category**
+
+Avoidable transcript copying during Stage 12 application.
+
+**Evidence and complexity**
+
+The runner invokes application for every section even when its approved
+proposal slice is empty. Application clones the section's segments and
+categories and builds the segment-ID map before inspecting proposals. For a
+stage with `N` segments and no approved edits, that is `Θ(N)` segment
+copies and map entries, plus category copies, to preserve identical text.
+Across repeated no-op stages this grows with the number of stages times the
+transcript size.
+[Section application](../../internal/framework/runner/runner.go#L289) ·
+[Per-section clone and index](../../internal/framework/proposals/apply.go#L42)
+
+**Refactoring opportunity**
+
+Retain the existing section when it has no approved proposals. Invoke
+application and its protection checks only for sections with candidates.
+Keep the runner's work-unit bookkeeping for every section.
+
+**Acceptance**
+
+No-edit and all-recoverable-failure runs produce the same normalized
+transcript and reports. Sections with approved proposals still use ordered
+application and recheck the current text before each edit.