Clarify correction repair limits and reporting

This commit is contained in:
2026-11-11 03:48:30 +00:00
parent 4f525e07cc
commit b036336e2a

View File

@@ -109,10 +109,9 @@ for separately. Do not manufacture hidden provider messages or reasoning.
Keep required conversation state in memory when repair is enabled, independently Keep required conversation state in memory when repair is enabled, independently
of debug artifact saving. Its lifetime need only cover the associated work; this of debug artifact saving. Its lifetime need only cover the associated work; this
does not introduce durable conversation storage or restart/resume support. Count does not introduce durable conversation storage or restart/resume support. Send
the complete replay, feedback, and applicable output contract when assessing the full replay and feedback under the revision output contract, subject to the
request size. Apply the accepted selected provider's [request limits](#provider-request-limits).
[context-limit behavior](#full-history-that-does-not-fit).
### Module Repair Prompts and Response Contract ### Module Repair Prompts and Response Contract
@@ -200,9 +199,10 @@ Use the accepted [reporting scope](#ordinary-reports-and-attempt-history).
`framework/responseschema`. Provide revision/withdrawal decoding and `framework/responseschema`. Provide revision/withdrawal decoding and
correspondence checks without changing ordinary span-deletion semantics. correspondence checks without changing ordinary span-deletion semantics.
4. **Adapt feedback collection.** Update validator findings where their current 4. **Adapt feedback collection.** Update validator findings where their current
messages are insufficient for repair. Change runner filtering to collect the messages are insufficient for repair. When repair is enabled, collect the
agreed applicable findings, distinguishing failures, unavailable decisions, agreed applicable findings and distinguish failures, unavailable decisions,
and checks that could not run. Keep complete validation mandatory for approval. and checks that could not run. With an effective budget of zero, use the
existing validator filtering. Keep complete validation mandatory for approval.
5. **Implement the bounded branch loop.** Integrate revision calls and full 5. **Implement the bounded branch loop.** Integrate revision calls and full
revalidation into the current stage/section flow. Preserve accepted proposals, revalidation into the current stage/section flow. Preserve accepted proposals,
branch isolation, deterministic application, cancellation, and existing branch isolation, deterministic application, cancellation, and existing
@@ -241,8 +241,8 @@ boundaries. Cover the following behavior with proportionate tests:
containing only its own branch's history. Exercise debug enabled and disabled. containing only its own branch's history. Exercise debug enabled and disabled.
- Repair success, explicit withdrawal, budget exhaustion, the default of one - Repair success, explicit withdrawal, budget exhaustion, the default of one
semantic attempt, additional configured attempts, and global or stage-level semantic attempt, additional configured attempts, and global or stage-level
zero disabling repair. Distinguish withdrawal from an empty-string deletion zero disabling both repair and extra feedback collection. Distinguish
proposal. withdrawal from an empty-string deletion proposal.
- Validator-owned findings collected under the accepted continuation policy; - Validator-owned findings collected under the accepted continuation policy;
inapplicable or unavailable checks do not masquerade as editorial rejection. inapplicable or unavailable checks do not masquerade as editorial rejection.
Cover known rejection alongside an unavailable assessment, unavailable Cover known rejection alongside an unavailable assessment, unavailable
@@ -255,18 +255,17 @@ boundaries. Cover the following behavior with proportionate tests:
- A revision can correct its source span or segment reference within the original - A revision can correct its source span or segment reference within the original
request's section, preserving proposal lineage and passing ordinary target request's section, preserving proposal lineage and passing ordinary target
mapping and full validation. mapping and full validation.
- Context-size handling, provider failure, structural-repair exhaustion, and - A provider error on a repair request, including a context-size rejection,
cancellation follow the accepted policies while preserving existing fails only that proposal after any applicable PromptKit retries. Other approved
partial-success and fatal-error boundaries. Account for semantic attempts, work remains eligible for application. Cover structural-repair exhaustion and
structural repairs, and transport retries without conflating them. cancellation under the existing failure boundaries. Account for semantic
An oversized conversation ends only the affected branch with a reported size attempts, structural repairs, and transport retries separately.
failure, preserving unrelated accepted work.
- Embedded module repair prompts and operator replacements work through the - Embedded module repair prompts and operator replacements work through the
shared engine. Root and stage settings follow the accepted scope and fallback shared engine. Root and stage settings follow the accepted scope and fallback
rules, including inheritance of the effective proposal profile after stage rules, including inheritance of the effective proposal profile after stage
settings are combined and explicit repair-profile overrides. Reports identify settings are combined and explicit repair-profile overrides. Verify that
attempts and final outcomes without presenting every attempt as a separate ordinary reporting records one compact outcome per repair branch and that
applied correction or exposing raw history by default. optional redacted debug output carries the detailed attempt history.
Run focused checks during development and assembled application validation at Run focused checks during development and assembled application validation at
completion under the [testing policy](../policy/testing.md). Representative completion under the [testing policy](../policy/testing.md). Representative
@@ -281,23 +280,24 @@ evidence that a revised correction is acceptable.
### Semantic Repair Budget and Configuration Scope ### Semantic Repair Budget and Configuration Scope
Allow one additional semantic repair attempt per rejected proposal by default, Allow one additional semantic repair attempt per rejected proposal by default,
configurable at the root and per stage; zero disables the loop. A revision that configurable at the root and per stage. Zero disables both the repair loop and
fails validation can continue when the operator configures extra feedback collection. A revision that fails validation can continue when
more attempts. Use a distinct setting from structural `llm.repair_attempts`. the operator configures more attempts. Use a distinct setting from structural
`llm.repair_attempts`.
Retain existing per-call timeouts and parent cancellation, and document that Retain existing per-call timeouts and parent cancellation, and document that
each semantic attempt may include transport retries and structural repairs. each semantic attempt may include transport retries and structural repairs.
This provides useful feedback-driven revision with a simple finite bound. This provides useful feedback-driven revision with a simple finite bound.
The default and scope are accepted; exact configuration names remain an The default and scope are accepted; exact configuration names remain an
implementation-planning choice. implementation-planning choice.
### Full History That Does Not Fit ### Provider Request Limits
Account for the complete conversation and feedback using the supported Submit the full repair conversation and let the selected provider enforce its
request-sizing facilities; if a repair cannot fit the selected request limits. If a provider error prevents revision after any applicable
limit, end that branch with a reported size failure and keep unrelated accepted PromptKit retries, including when the conversation exceeds the provider's context
work. Preserve existing context-estimation limits rather than promising exact window, record the repair as unavailable and end that proposal's branch.
provider token accounting. This honors the agreed full-history design without Continue processing other proposals. Audita's existing fatal application error
silently truncating context or changing the execution profile. and cancellation behavior still applies.
### Repair Execution Profile ### Repair Execution Profile
@@ -309,9 +309,11 @@ difficult revision. Backend capacity remains shared through PromptKit.
### Feedback Eligibility and Unavailable Validation ### Feedback Eligibility and Unavailable Validation
Collect all meaningful deterministic failures, then run the applicable semantic When repair is enabled, collect all meaningful deterministic failures, then run
validators when the candidate has the usable target and the applicable semantic validators when the candidate has the usable target and
before/after data they need, even if another independent check rejected it. before/after data they need, even if another independent check rejected it. With
an effective repair budget of zero, later validators receive only the candidates
that advance under the current validation flow.
An actionable rejection may enter repair; no-effect proposals can finish without An actionable rejection may enter repair; no-effect proposals can finish without
repair because they contribute no edit. Build a small applicability map from repair because they contribute no edit. Build a small applicability map from
the actual checks rather than running validators on unusable inputs or encoding the actual checks rather than running validators on unusable inputs or encoding
@@ -327,16 +329,16 @@ as evidence against the proposed wording.
### Ordinary Reports and Attempt History ### Ordinary Reports and Attempt History
Retain compact proposal lineage, attempt counts, and final dispositions in Use one compact repair-outcomes section in ordinary reporting. Give each repair
ordinary results, with meaningful errors and recoveries in the branch one entry with the original proposal reference, its attempt count, and
existing report framework. Distinguish approval, actual application, withdrawal, final disposition. Use the existing work-unit context for stage and chunk
and exhausted or unavailable repair. Keep full conversations and detailed identity. Make it clear whether an approved revision was applied. Record
per-attempt evidence in optional redacted debug artifacts. A rejected proposal operational errors and recoveries through the existing report framework and keep
or voluntary withdrawal is not automatically an operational error. This makes the current exit-status policy.
outcomes explainable without turning ordinary reports into conversation dumps. Editorial rejection and voluntary withdrawal are correction outcomes. Keep
Choose concrete fields and any output-schema version change during implementation conversation text and per-attempt details in optional redacted debug artifacts.
planning, preserving the existing failure and exit-status policy unless an Choose the report location, fields, and any schema version change during
explicitly identified requirement calls for a change. implementation planning.
### Revising the Span or Target Reference ### Revising the Span or Target Reference