Clarify correction repair limits and reporting

This commit is contained in:
2026-11-11 03:48:30 +00:00
parent 4f525e07cc
commit b036336e2a

View File

@@ -109,10 +109,9 @@ for separately. Do not manufacture hidden provider messages or reasoning.
Keep required conversation state in memory when repair is enabled, independently
of debug artifact saving. Its lifetime need only cover the associated work; this
does not introduce durable conversation storage or restart/resume support. Count
the complete replay, feedback, and applicable output contract when assessing
request size. Apply the accepted
[context-limit behavior](#full-history-that-does-not-fit).
does not introduce durable conversation storage or restart/resume support. Send
the full replay and feedback under the revision output contract, subject to the
selected provider's [request limits](#provider-request-limits).
### Module Repair Prompts and Response Contract
@@ -200,9 +199,10 @@ Use the accepted [reporting scope](#ordinary-reports-and-attempt-history).
`framework/responseschema`. Provide revision/withdrawal decoding and
correspondence checks without changing ordinary span-deletion semantics.
4. **Adapt feedback collection.** Update validator findings where their current
messages are insufficient for repair. Change runner filtering to collect the
agreed applicable findings, distinguishing failures, unavailable decisions,
and checks that could not run. Keep complete validation mandatory for approval.
messages are insufficient for repair. When repair is enabled, collect the
agreed applicable findings and distinguish failures, unavailable decisions,
and checks that could not run. With an effective budget of zero, use the
existing validator filtering. Keep complete validation mandatory for approval.
5. **Implement the bounded branch loop.** Integrate revision calls and full
revalidation into the current stage/section flow. Preserve accepted proposals,
branch isolation, deterministic application, cancellation, and existing
@@ -241,8 +241,8 @@ boundaries. Cover the following behavior with proportionate tests:
containing only its own branch's history. Exercise debug enabled and disabled.
- Repair success, explicit withdrawal, budget exhaustion, the default of one
semantic attempt, additional configured attempts, and global or stage-level
zero disabling repair. Distinguish withdrawal from an empty-string deletion
proposal.
zero disabling both repair and extra feedback collection. Distinguish
withdrawal from an empty-string deletion proposal.
- Validator-owned findings collected under the accepted continuation policy;
inapplicable or unavailable checks do not masquerade as editorial rejection.
Cover known rejection alongside an unavailable assessment, unavailable
@@ -255,18 +255,17 @@ boundaries. Cover the following behavior with proportionate tests:
- A revision can correct its source span or segment reference within the original
request's section, preserving proposal lineage and passing ordinary target
mapping and full validation.
- Context-size handling, provider failure, structural-repair exhaustion, and
cancellation follow the accepted policies while preserving existing
partial-success and fatal-error boundaries. Account for semantic attempts,
structural repairs, and transport retries without conflating them.
An oversized conversation ends only the affected branch with a reported size
failure, preserving unrelated accepted work.
- A provider error on a repair request, including a context-size rejection,
fails only that proposal after any applicable PromptKit retries. Other approved
work remains eligible for application. Cover structural-repair exhaustion and
cancellation under the existing failure boundaries. Account for semantic
attempts, structural repairs, and transport retries separately.
- Embedded module repair prompts and operator replacements work through the
shared engine. Root and stage settings follow the accepted scope and fallback
rules, including inheritance of the effective proposal profile after stage
settings are combined and explicit repair-profile overrides. Reports identify
attempts and final outcomes without presenting every attempt as a separate
applied correction or exposing raw history by default.
settings are combined and explicit repair-profile overrides. Verify that
ordinary reporting records one compact outcome per repair branch and that
optional redacted debug output carries the detailed attempt history.
Run focused checks during development and assembled application validation at
completion under the [testing policy](../policy/testing.md). Representative
@@ -281,23 +280,24 @@ evidence that a revised correction is acceptable.
### Semantic Repair Budget and Configuration Scope
Allow one additional semantic repair attempt per rejected proposal by default,
configurable at the root and per stage; zero disables the loop. A revision that
fails validation can continue when the operator configures
more attempts. Use a distinct setting from structural `llm.repair_attempts`.
configurable at the root and per stage. Zero disables both the repair loop and
extra feedback collection. A revision that fails validation can continue when
the operator configures more attempts. Use a distinct setting from structural
`llm.repair_attempts`.
Retain existing per-call timeouts and parent cancellation, and document that
each semantic attempt may include transport retries and structural repairs.
This provides useful feedback-driven revision with a simple finite bound.
The default and scope are accepted; exact configuration names remain an
implementation-planning choice.
### Full History That Does Not Fit
### Provider Request Limits
Account for the complete conversation and feedback using the supported
request-sizing facilities; if a repair cannot fit the selected
limit, end that branch with a reported size failure and keep unrelated accepted
work. Preserve existing context-estimation limits rather than promising exact
provider token accounting. This honors the agreed full-history design without
silently truncating context or changing the execution profile.
Submit the full repair conversation and let the selected provider enforce its
request limits. If a provider error prevents revision after any applicable
PromptKit retries, including when the conversation exceeds the provider's context
window, record the repair as unavailable and end that proposal's branch.
Continue processing other proposals. Audita's existing fatal application error
and cancellation behavior still applies.
### Repair Execution Profile
@@ -309,9 +309,11 @@ difficult revision. Backend capacity remains shared through PromptKit.
### Feedback Eligibility and Unavailable Validation
Collect all meaningful deterministic failures, then run the applicable semantic
validators when the candidate has the usable target and
before/after data they need, even if another independent check rejected it.
When repair is enabled, collect all meaningful deterministic failures, then run
the applicable semantic validators when the candidate has the usable target and
before/after data they need, even if another independent check rejected it. With
an effective repair budget of zero, later validators receive only the candidates
that advance under the current validation flow.
An actionable rejection may enter repair; no-effect proposals can finish without
repair because they contribute no edit. Build a small applicability map from
the actual checks rather than running validators on unusable inputs or encoding
@@ -327,16 +329,16 @@ as evidence against the proposed wording.
### Ordinary Reports and Attempt History
Retain compact proposal lineage, attempt counts, and final dispositions in
ordinary results, with meaningful errors and recoveries in the
existing report framework. Distinguish approval, actual application, withdrawal,
and exhausted or unavailable repair. Keep full conversations and detailed
per-attempt evidence in optional redacted debug artifacts. A rejected proposal
or voluntary withdrawal is not automatically an operational error. This makes
outcomes explainable without turning ordinary reports into conversation dumps.
Choose concrete fields and any output-schema version change during implementation
planning, preserving the existing failure and exit-status policy unless an
explicitly identified requirement calls for a change.
Use one compact repair-outcomes section in ordinary reporting. Give each repair
branch one entry with the original proposal reference, its attempt count, and
final disposition. Use the existing work-unit context for stage and chunk
identity. Make it clear whether an approved revision was applied. Record
operational errors and recoveries through the existing report framework and keep
the current exit-status policy.
Editorial rejection and voluntary withdrawal are correction outcomes. Keep
conversation text and per-attempt details in optional redacted debug artifacts.
Choose the report location, fields, and any schema version change during
implementation planning.
### Revising the Span or Target Reference