Plan correction repair after validation

docs/roadmap/correction-repair.md

@@ -0,0 +1,371 @@
+# Correction Repair After Validation
+
+**Updated:** 2026-11-10
+
+**Status:** Feature direction agreed; implementation pending. Open decisions
+below require resolution before the dependent implementation work. This document
+owns the target and remaining work, not a release date or implementation status
+for the completed library adoption.
+
+## Objective
+
+Give a rejected correction an opportunity to improve using specific validator
+feedback, while preserving proposals that have already passed validation. Each
+correction module supplies a repair prompt. Each validator supplies findings
+within its own remit. Audita coordinates bounded revision conversations and
+acceptance through the existing PromptKit runtime.
+
+The unit of repair is an individual correction proposal. Multiple proposals
+against one transcript segment remain separately identifiable. A model may revise
+the selected proposal or explicitly withdraw it. Every revised proposal must pass
+the full applicable validation chain before it becomes eligible for application.
+
+This adds domain revision after rejection. It is distinct from PromptKit's
+structural output repair and from obtaining an unavailable validator decision.
+The latter remains separate [deferred work](future.md#domain-aware-response-recovery).
+
+## Current Baseline
+
+Reviewed application revision: `8946a82`. Recheck the owning components when
+implementation begins, including any subsequent correctness-review fixes.
+
+- The [pipeline](../internal/pipeline.md) partitions the transcript once and runs
+  ordered stages. Proposals can be generated concurrently; indexing and
+  application remain deterministic. Approved corrections are applied in proposal
+  order, with existing replacement and conflict checks.
+- [Proposal generation](../internal/modules.md#proposal-generation-ownership)
+  returns a correction set, potentially with several proposals per segment.
+  Model-facing segment references are request-local and are mapped to transcript
+  identities. The four modules are glossary, homophones, spoken-word cleanup,
+  and grammar.
+- The [validator runtime](../internal/validators.md) runs deterministic checks
+  before LLM-backed checks and filters rejected candidates before later
+  validators. Decisions already carry proposal identity, disposition, reason code,
+  and message. Unavailable decisions are distinguished from editorial rejection.
+  Validator chains are currently selected by module, not individually configured
+  by operators.
+- The [LLM adapter](../internal/llm-runtime.md) uses PromptKit v0.21.0 prepared
+  execution with ConfigKit v1.2.0. It captures prepared-request data and final
+  response text. Proposal-generation results carry candidates, operation outcomes,
+  and artifact references, but do not retain a conversation for domain revision.
+- [Embedded prompts](../internal/prompts.md) live in the `audita` collection and
+  support operator replacement through native underlays. Audita supplies response
+  schemas separately. Full interaction artifacts are optional debug output.
+- Existing `llm.repair_attempts` controls structural repairs. It is not a budget
+  for the proposed domain revision loop. No such loop is implemented.
+
+## Targeted End State
+
+### Proposal Lifecycle
+
+1. Generate the initial correction set using the stage's instruction, transcript
+   section, relevant glossary/protection data, and response schema.
+2. Validate candidates, collecting findings from checks that can meaningfully
+   evaluate them. Checks that cannot run do not become invented rejection reasons.
+3. Retain approved proposals for application. Give each rejected proposal a repair
+   branch containing its conversation and findings. Budget and eligibility rules
+   determine whether another request can run.
+4. Request revision or withdrawal of that one proposal. An accepted withdrawal
+   ends the branch without an edit. A revised proposal returns to the complete
+   applicable validator chain; approvals of a previous version do not transfer.
+5. Continue until approval, withdrawal, budget exhaustion, or a classified failure.
+   Preserve independently accepted work under the existing partial-success rules.
+6. Apply accepted corrections in deterministic proposal order, retaining the
+   existing apply-time checks for stale spans, protection, and conflicts.
+
+Withdrawal or exhaustion contributes no edit from that proposal. Other accepted
+proposals against the same segment may still change its text. Retaining an
+approval does not guarantee application when another edit makes it inapplicable.
+This feature does not add a second repair loop for application-time conflicts.
+
+Each revision remains associated with the original proposal and its processing
+stage and section. Repair responses must identify the requested proposal; unrelated
+edits do not become additional candidates. OD-6 addresses whether revisions can
+change the proposed span or target reference within the original section context;
+the ordinary mapping and validators still determine applicability.
+Exact internal types and model-facing reference spelling are implementation
+choices, not new globally unique identity requirements.
+
+### Full Conversation and Independent Branches
+
+The first repair request contains the complete initial rendered request messages,
+with their original roles and order, followed by the complete assistant response
+that supplied the decoded proposal set, then the targeted repair instruction.
+Retain the original section and request inputs for this purpose; do not rebuild
+the conversation from a transcript already changed by other corrections.
+
+Every proposal branches independently from that same initial exchange. Later
+attempts append that proposal's previous repair response and new feedback to its
+own branch. Repairs of other proposals never become turns in that conversation.
+Keep approved proposals outside the set being reconsidered. Branch scheduling
+must not change proposal attribution or final application order.
+
+Here, conversation history means the application-level message exchange. Preserve
+the full response used for candidate decoding rather than serializing only the
+failed proposal as if it were the model's original answer. Provider transport
+attempts and PromptKit's internal structural-repair operations remain accounted
+for separately. Do not manufacture hidden provider messages or reasoning.
+
+Keep required conversation state in memory when repair is enabled, independently
+of debug artifact saving. Its lifetime need only cover the associated work; this
+does not introduce durable conversation storage or restart/resume support. Count
+the complete replay, feedback, and applicable output contract when assessing
+request size. Context-limit handling is OD-2 below.
+
+### Module Repair Prompts and Response Contract
+
+Each built-in correction module owns a corresponding repair definition in the
+`audita` namespace, alongside its proposal definition. Use the existing embedded
+filesystem and ConfigKit underlay mechanism so operators can replace repair
+prompts without rebuilding Audita. Shared instruction fragments may be reused;
+module-specific instructions stay with the module's prompt.
+
+The repair instruction identifies the proposal, repeats its original and proposed
+replacement text, presents the findings, and asks for a revision or withdrawal.
+It explains that only this proposal is under review and states the response
+contract for this turn. This must remain clear even though the initial prompt
+requested a complete correction set. Later turns refer explicitly to the current
+proposal version rather than relying on an ambiguous "last message" reference.
+
+Use a structured response that distinguishes revision from withdrawal and carries
+the requested proposal reference. A revision contains the correction data needed
+by the existing validators. Empty replacement text remains a possible span
+deletion; it must not double as a withdrawal signal. The response schema owns
+shape validation; Audita owns correspondence and domain acceptance. Exact field
+names can be selected with the schema implementation.
+
+### Validator-Owned Feedback
+
+Validators contribute their own actionable findings: validator identity, stable
+reason code, an explanation, and relevant evidence or constraints where available.
+Build on existing decision data rather than creating a parallel reason registry.
+For example, a protection finding can identify the spelling affected, while an
+editorial finding can explain the meaning change it detected.
+
+The module repair template owns presentation of the collected findings.
+Validators do not construct chat histories or own the correction response schema.
+Order feedback by the applicable validator chain for understandable reports and
+prompts. Model-backed validators should contribute their assessment through the
+existing structured decision path.
+
+Collect failures from all applicable checks that can meaningfully evaluate the
+candidate. A malformed target may prevent contextual checks; a provider failure
+means the assessment is unavailable, not that the edit violated a domain rule.
+OD-4 settles the eligibility and continuation policy. Making validator chains
+operator-configurable is not required by this feature.
+
+### Ownership and Library Integration
+
+Audita owns proposal lineage, semantic feedback, branch state, revision budgets,
+revalidation, and transcript application. PromptKit continues to own rendering,
+profile/backend selection, transport retry, structural repair, and per-backend
+admission. Configuration continues through ConfigKit's native resolution and
+the existing root/stage scope conventions. Standard CommandKit inspection and
+request commands expose the configured prompt assets through their normal APIs.
+
+PromptKit already supports
+[appending rendered assistant and corrective messages](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/171fb11f4e967a1da3af57843c6467954b14ba09/docs/consumers/pkg-promptkit.md#append-already-rendered-messages).
+Use its public preparation and execution APIs to render the module repair
+definition and assemble the intended conversation. Appended messages are already
+rendered; appending template text alone will not expand its variables. Confirm
+the full-history path and per-call response-schema selection with a focused
+integration example before settling adapter details. If that exposes a genuine
+upstream gap, document a small reproducer and request focused support rather than
+introducing another renderer or provider client.
+
+Maintain the existing distinctions between recoverable model-operation failures,
+fatal application/invariant failures, and cancellation. Domain revision attempts
+are additional logical calls, each potentially containing PromptKit retries and
+structural repairs; account for these separately. Share existing backend admission
+and cancellation instead of adding a repair-specific global concurrency limiter.
+Reporting choices remain in OD-5.
+
+## Work From the Baseline
+
+1. **Resolve the open policies and prove the request path.** Settle the decisions
+   below. Exercise native message composition with an original multi-proposal
+   response, a rendered module repair instruction, a revision response schema,
+   and a subsequent branch turn. Confirm operator prompt replacements and the
+   same behavior with debug disabled.
+2. **Carry conversation state through the existing boundaries.** Extend
+   `framework/structuredoutput`, the LLM adapter, proposal-generation results,
+   and module/framework contracts as needed to retain and replay the original
+   request and full response. Preserve source-reference mapping and metadata.
+   Use compact lineage data and shared immutable initial context where practical;
+   no particular storage layout or new package hierarchy is prescribed.
+3. **Add module repair assets and the response schema.** Extend the prompt
+   registry, embedded definitions, shared fragments where useful, and
+   `framework/responseschema`. Provide revision/withdrawal decoding and
+   correspondence checks without changing ordinary span-deletion semantics.
+4. **Adapt feedback collection.** Update validator findings where their current
+   messages are insufficient for repair. Change runner filtering to collect the
+   agreed applicable findings, distinguishing failures, unavailable decisions,
+   and checks that could not run. Keep complete validation mandatory for approval.
+5. **Implement the bounded branch loop.** Integrate revision calls and full
+   revalidation into the current stage/section flow. Preserve accepted proposals,
+   branch isolation, deterministic application, cancellation, and existing
+   apply-time checks. Unsuccessful branches must not discard unrelated accepted
+   work. This does not require redesigning stage barriers or chunk scheduling.
+6. **Wire configuration and readiness.** Add the selected budget/profile settings
+   with native presence and stage inheritance. Keep semantic attempts distinct
+   from structural `llm.repair_attempts`. Preflight checks any additional required
+   role/prompt readiness when enabled, without making disabled repair require
+   an unused profile. Let existing inspection expose resolved settings.
+7. **Extend outcomes and documentation.** Implement OD-5 through ordinary
+   reporting, correction ledgers, and optional redacted debug artifacts. Update
+   the canonical [configuration](../config.md), [pipeline](../internal/pipeline.md),
+   [module](../internal/modules.md), [validator](../internal/validators.md),
+   [prompt](../internal/prompts.md), [runtime](../internal/llm-runtime.md), and
+   [output](../integrations/output-files.md) documentation when behavior lands.
+   Record significant ownership decisions through the documentation policy's
+   ADR process if warranted; routine API and template choices need no ADR.
+
+These are work dependencies for later implementation planning. They do not
+mandate separate commits, a general workflow engine, or a new release process.
+
+## Acceptance and Completion
+
+Use controlled model responses and real loading/rendering at integration
+boundaries. Cover the following behavior with proportionate tests:
+
+- One approved and one rejected proposal from the same response: repair only
+  the rejected proposal, then apply the accepted results in deterministic order.
+  Include several proposals against one segment and an overlapping-edit case.
+- Full initial messages and full response retained, correct roles and ordering,
+  explicit proposal selection, independent branches, and a second repair turn
+  containing only its own branch's history. Exercise debug enabled and disabled.
+- Repair success, explicit withdrawal, budget exhaustion, and configured zero
+  attempts. Distinguish withdrawal from an empty-string deletion proposal.
+- Validator-owned findings collected under the chosen continuation policy;
+  inapplicable or unavailable checks do not masquerade as editorial rejection.
+  Revised candidates must pass all applicable checks, including checks that
+  approved their earlier versions.
+- Wrong proposal references and unusable repair responses cannot introduce
+  unrelated edits or approval. Preserve ordinary identity reconciliation and
+  structural validation; no generic salvage of malformed JSON is implied.
+- Context-size handling, provider failure, structural-repair exhaustion, and
+  cancellation follow the selected policies while preserving existing
+  partial-success and fatal-error boundaries. Account for semantic attempts,
+  structural repairs, and transport retries without conflating them.
+- Embedded module repair prompts and operator replacements work through the
+  shared engine. Root and stage settings follow the selected scope and fallback
+  rules. Reports identify attempts and final outcomes without presenting every
+  attempt as a separate applied correction or exposing raw history by default.
+
+Run focused checks during development and assembled application validation at
+completion under the [testing policy](../policy/testing.md). Representative
+transcript comparisons can help tune prompts and defaults, but this roadmap
+sets no numerical quality threshold or requirement for paid provider calls.
+Completion requires the agreed loop, resolved open policies, acceptance evidence,
+and updated canonical documentation. Successful schema validation alone is not
+evidence that a revised correction is acceptable.
+
+## Open Decisions
+
+### OD-1: Semantic Repair Budget and Configuration Scope
+
+**Recommendation:** start with one additional semantic repair attempt per
+rejected proposal, with a root default and per-stage override; zero disables the
+loop. A revision that fails validation can continue when the operator configures
+more attempts. Use a distinct setting from structural `llm.repair_attempts`.
+Retain existing per-call timeouts and parent cancellation, and document that
+each semantic attempt may include transport retries and structural repairs.
+This provides useful feedback-driven revision with a simple finite bound.
+The proposed default and setting names are not yet accepted configuration.
+
+**Alternative:** default to zero and require explicit enabling per run or stage.
+This avoids additional model work until operators choose it and supports gradual
+evaluation, but leaves the new behavior inactive in ordinary use. A shared
+per-stage call budget is another viable model for bounding aggregate work; it
+offers tighter run-cost control but introduces competition between proposals
+and requires a deterministic allocation policy.
+
+### OD-2: Full History That Does Not Fit
+
+**Recommendation:** account for the complete conversation and feedback using
+the supported request-sizing facilities; if a repair cannot fit the selected
+limit, end that branch with a reported size failure and keep unrelated accepted
+work. Preserve existing context-estimation limits rather than promising exact
+provider token accounting. This honors the agreed full-history design without
+silently truncating context or changing the execution profile.
+
+**Alternative:** support an explicitly configured fallback repair profile with
+a larger context allowance. It can preserve full history and rescue otherwise
+blocked corrections, but adds profile selection, readiness, and failure-handling
+work and may change cost and model behavior. Smaller initial chunks are also
+available through existing configuration, at the expense of reduced proposal
+context; automatic rechunking is not necessary for the initial feature.
+
+### OD-3: Repair Execution Profile
+
+**Recommendation:** inherit the effective proposal profile by default, with an
+optional root/per-stage repair-profile override. Resolve fallback after stage
+settings are combined, following the existing validation-profile pattern. This
+keeps ordinary configuration small while allowing a different model for a
+difficult revision. Backend capacity remains shared through PromptKit.
+
+**Alternative:** always use the proposal profile, with no separate selector.
+This is simpler and keeps model behavior consistent across the conversation,
+but prevents targeted use of a more capable or larger-context repair model.
+Neither choice requires a separate backend or credential registry.
+
+### OD-4: Feedback Eligibility and Unavailable Validation
+
+**Recommendation:** collect all meaningful deterministic failures, then run the
+applicable semantic validators when the candidate has the usable target and
+before/after data they need, even if another independent check rejected it.
+An actionable rejection may enter repair; no-effect proposals can finish without
+repair because they contribute no edit. Build a small applicability map from
+the actual checks rather than running validators on unusable inputs or encoding
+universal dependencies between them.
+
+Keep unavailable validation as a separate operation outcome. If a proposal also
+has an actionable rejection, its repair may proceed using the known findings,
+with unavailable assessments identified as such; any revision still needs a full
+successful validation chain. An unavailable assessment by itself should retain
+the current unavailable outcome, leaving decision-recovery calls to the deferred
+feature. This produces useful feedback without treating infrastructure failure
+as evidence against the proposed wording.
+
+**Alternative:** collect the same applicable findings but require all applicable
+validator operations to finish successfully before starting a repair. If any
+assessment is unavailable, keep the candidate unavailable for this stage. This
+gives each repair a complete assessment and avoids spending calls on a candidate
+whose review could not finish, but sacrifices opportunities to address a known
+rejection while another validator is temporarily unavailable. Both approaches
+preserve full validation as a condition of eventual approval.
+
+### OD-5: Ordinary Reports and Attempt History
+
+**Recommendation:** retain compact proposal lineage, attempt counts, and final
+dispositions in ordinary results, with meaningful errors and recoveries in the
+existing report framework. Distinguish approval, actual application, withdrawal,
+and exhausted or unavailable repair. Keep full conversations and detailed
+per-attempt evidence in optional redacted debug artifacts. A rejected proposal
+or voluntary withdrawal is not automatically an operational error. This makes
+outcomes explainable without turning ordinary reports into conversation dumps.
+Choose concrete fields and any output-schema version change during implementation
+planning, preserving the existing failure and exit-status policy unless an
+explicitly identified requirement calls for a change.
+
+**Alternative:** include every revision and validator finding in the ordinary
+correction ledger. This improves analysis without enabling debug capture, but
+increases report size, retained transcript content, and public schema surface.
+It requires clear grouping so attempted revisions are not counted as independent
+corrections or successful recoveries.
+
+### OD-6: Revising the Span or Target Reference
+
+**Recommendation:** allow the revised proposal to correct its original-text span
+and segment reference within the original request's section, while retaining its
+proposal lineage. Apply the ordinary target mapping, replacement policy, and full
+validation to the new candidate. This permits repair of an incorrect source span
+or model-facing target reference, both of which can cause validation failures.
+It adds no new transcript-wide targeting mechanism.
+
+**Alternative:** keep a resolvable target segment fixed, allowing changes to the
+span and replacement only within that segment; terminate invalid-target proposals
+without domain repair. This makes continuity easier to interpret and reduces
+the chance of revising a different edit, but cannot repair misidentified targets.
+The restriction would be a deliberate feature choice, not an invariant inferred
+from the current proposal representation.

docs/roadmap/future.md

@@ -1,6 +1,6 @@
 # Deferred Feature Work
 
-**Last updated:** 2026-10-11
+**Last updated:** 2026-11-10
 
 **Status:** Deferred or exploratory work; not implemented or scheduled. These
 entries preserve work beyond completed feature and library upgrades.
@@ -10,10 +10,11 @@ implementing every capability.
 ## Domain-Aware Response Recovery
 
 The implemented [partial-success decision reconciliation](../internal/validators.md#llm-validator-batching)
-preserves usable decisions without another model call. Guided recovery remains
-deferred pending evidence from failures that remain. General per-entry
-salvage from structurally invalid responses is also a separate design question;
-the accepted reconciliation feature retains whole-response structural validation.
+preserves usable decisions without another model call. Revision of rejected
+correction proposals is now an agreed feature direction in the
+[correction-repair roadmap](correction-repair.md), with its own open decisions.
+That work includes substantive validator feedback and explicit proposal withdrawal.
+It is distinct from recovering an unavailable validator decision.
 
 Consider bounded feedback and regeneration for model responses that satisfy the
 structural schema but leave requested decisions unavailable after reconciliation.
@@ -30,11 +31,15 @@ secrets, and how regeneration fits bounded budgets and operation deadlines. Reus
 shared library capabilities where appropriate; do not introduce an uncoordinated
 outer retry loop that multiplies existing retry/repair budgets.
 
-Ordinary editorial rejection is a valid decision, not a reason to regenerate until
-an edit is approved. Recovery must preserve required validation, candidate identity,
-partial-success safety, and truthful error/recovery reporting. Validate with
-controlled responses that distinguish recoverable protocol defects from valid
-negative decisions. Exact APIs, budgets, and sequencing remain undecided.
+Editorial rejection remains a valid decision about the evaluated proposal. The
+correction-repair roadmap owns any subsequent revision of that proposal and full
+revalidation; this deferred entry concerns unavailable decisions. Decision recovery
+must preserve required validation, candidate identity, partial-success safety,
+and truthful error/recovery reporting. Validate with controlled responses that
+distinguish protocol defects from valid negative decisions. Exact APIs, budgets,
+and sequencing remain undecided. General per-entry salvage from structurally
+invalid responses also remains a separate design question; current reconciliation
+retains whole-response structural validation.
 
 ## Independent Whole-Pipeline Traversal by Chunk