Clarify correction repair limits and reporting
This commit is contained in:
@@ -109,10 +109,9 @@ for separately. Do not manufacture hidden provider messages or reasoning.
|
||||
|
||||
Keep required conversation state in memory when repair is enabled, independently
|
||||
of debug artifact saving. Its lifetime need only cover the associated work; this
|
||||
does not introduce durable conversation storage or restart/resume support. Count
|
||||
the complete replay, feedback, and applicable output contract when assessing
|
||||
request size. Apply the accepted
|
||||
[context-limit behavior](#full-history-that-does-not-fit).
|
||||
does not introduce durable conversation storage or restart/resume support. Send
|
||||
the full replay and feedback under the revision output contract, subject to the
|
||||
selected provider's [request limits](#provider-request-limits).
|
||||
|
||||
### Module Repair Prompts and Response Contract
|
||||
|
||||
@@ -200,9 +199,10 @@ Use the accepted [reporting scope](#ordinary-reports-and-attempt-history).
|
||||
`framework/responseschema`. Provide revision/withdrawal decoding and
|
||||
correspondence checks without changing ordinary span-deletion semantics.
|
||||
4. **Adapt feedback collection.** Update validator findings where their current
|
||||
messages are insufficient for repair. Change runner filtering to collect the
|
||||
agreed applicable findings, distinguishing failures, unavailable decisions,
|
||||
and checks that could not run. Keep complete validation mandatory for approval.
|
||||
messages are insufficient for repair. When repair is enabled, collect the
|
||||
agreed applicable findings and distinguish failures, unavailable decisions,
|
||||
and checks that could not run. With an effective budget of zero, use the
|
||||
existing validator filtering. Keep complete validation mandatory for approval.
|
||||
5. **Implement the bounded branch loop.** Integrate revision calls and full
|
||||
revalidation into the current stage/section flow. Preserve accepted proposals,
|
||||
branch isolation, deterministic application, cancellation, and existing
|
||||
@@ -241,8 +241,8 @@ boundaries. Cover the following behavior with proportionate tests:
|
||||
containing only its own branch's history. Exercise debug enabled and disabled.
|
||||
- Repair success, explicit withdrawal, budget exhaustion, the default of one
|
||||
semantic attempt, additional configured attempts, and global or stage-level
|
||||
zero disabling repair. Distinguish withdrawal from an empty-string deletion
|
||||
proposal.
|
||||
zero disabling both repair and extra feedback collection. Distinguish
|
||||
withdrawal from an empty-string deletion proposal.
|
||||
- Validator-owned findings collected under the accepted continuation policy;
|
||||
inapplicable or unavailable checks do not masquerade as editorial rejection.
|
||||
Cover known rejection alongside an unavailable assessment, unavailable
|
||||
@@ -255,18 +255,17 @@ boundaries. Cover the following behavior with proportionate tests:
|
||||
- A revision can correct its source span or segment reference within the original
|
||||
request's section, preserving proposal lineage and passing ordinary target
|
||||
mapping and full validation.
|
||||
- Context-size handling, provider failure, structural-repair exhaustion, and
|
||||
cancellation follow the accepted policies while preserving existing
|
||||
partial-success and fatal-error boundaries. Account for semantic attempts,
|
||||
structural repairs, and transport retries without conflating them.
|
||||
An oversized conversation ends only the affected branch with a reported size
|
||||
failure, preserving unrelated accepted work.
|
||||
- A provider error on a repair request, including a context-size rejection,
|
||||
fails only that proposal after any applicable PromptKit retries. Other approved
|
||||
work remains eligible for application. Cover structural-repair exhaustion and
|
||||
cancellation under the existing failure boundaries. Account for semantic
|
||||
attempts, structural repairs, and transport retries separately.
|
||||
- Embedded module repair prompts and operator replacements work through the
|
||||
shared engine. Root and stage settings follow the accepted scope and fallback
|
||||
rules, including inheritance of the effective proposal profile after stage
|
||||
settings are combined and explicit repair-profile overrides. Reports identify
|
||||
attempts and final outcomes without presenting every attempt as a separate
|
||||
applied correction or exposing raw history by default.
|
||||
settings are combined and explicit repair-profile overrides. Verify that
|
||||
ordinary reporting records one compact outcome per repair branch and that
|
||||
optional redacted debug output carries the detailed attempt history.
|
||||
|
||||
Run focused checks during development and assembled application validation at
|
||||
completion under the [testing policy](../policy/testing.md). Representative
|
||||
@@ -281,23 +280,24 @@ evidence that a revised correction is acceptable.
|
||||
### Semantic Repair Budget and Configuration Scope
|
||||
|
||||
Allow one additional semantic repair attempt per rejected proposal by default,
|
||||
configurable at the root and per stage; zero disables the loop. A revision that
|
||||
fails validation can continue when the operator configures
|
||||
more attempts. Use a distinct setting from structural `llm.repair_attempts`.
|
||||
configurable at the root and per stage. Zero disables both the repair loop and
|
||||
extra feedback collection. A revision that fails validation can continue when
|
||||
the operator configures more attempts. Use a distinct setting from structural
|
||||
`llm.repair_attempts`.
|
||||
Retain existing per-call timeouts and parent cancellation, and document that
|
||||
each semantic attempt may include transport retries and structural repairs.
|
||||
This provides useful feedback-driven revision with a simple finite bound.
|
||||
The default and scope are accepted; exact configuration names remain an
|
||||
implementation-planning choice.
|
||||
|
||||
### Full History That Does Not Fit
|
||||
### Provider Request Limits
|
||||
|
||||
Account for the complete conversation and feedback using the supported
|
||||
request-sizing facilities; if a repair cannot fit the selected
|
||||
limit, end that branch with a reported size failure and keep unrelated accepted
|
||||
work. Preserve existing context-estimation limits rather than promising exact
|
||||
provider token accounting. This honors the agreed full-history design without
|
||||
silently truncating context or changing the execution profile.
|
||||
Submit the full repair conversation and let the selected provider enforce its
|
||||
request limits. If a provider error prevents revision after any applicable
|
||||
PromptKit retries, including when the conversation exceeds the provider's context
|
||||
window, record the repair as unavailable and end that proposal's branch.
|
||||
Continue processing other proposals. Audita's existing fatal application error
|
||||
and cancellation behavior still applies.
|
||||
|
||||
### Repair Execution Profile
|
||||
|
||||
@@ -309,9 +309,11 @@ difficult revision. Backend capacity remains shared through PromptKit.
|
||||
|
||||
### Feedback Eligibility and Unavailable Validation
|
||||
|
||||
Collect all meaningful deterministic failures, then run the applicable semantic
|
||||
validators when the candidate has the usable target and
|
||||
before/after data they need, even if another independent check rejected it.
|
||||
When repair is enabled, collect all meaningful deterministic failures, then run
|
||||
the applicable semantic validators when the candidate has the usable target and
|
||||
before/after data they need, even if another independent check rejected it. With
|
||||
an effective repair budget of zero, later validators receive only the candidates
|
||||
that advance under the current validation flow.
|
||||
An actionable rejection may enter repair; no-effect proposals can finish without
|
||||
repair because they contribute no edit. Build a small applicability map from
|
||||
the actual checks rather than running validators on unusable inputs or encoding
|
||||
@@ -327,16 +329,16 @@ as evidence against the proposed wording.
|
||||
|
||||
### Ordinary Reports and Attempt History
|
||||
|
||||
Retain compact proposal lineage, attempt counts, and final dispositions in
|
||||
ordinary results, with meaningful errors and recoveries in the
|
||||
existing report framework. Distinguish approval, actual application, withdrawal,
|
||||
and exhausted or unavailable repair. Keep full conversations and detailed
|
||||
per-attempt evidence in optional redacted debug artifacts. A rejected proposal
|
||||
or voluntary withdrawal is not automatically an operational error. This makes
|
||||
outcomes explainable without turning ordinary reports into conversation dumps.
|
||||
Choose concrete fields and any output-schema version change during implementation
|
||||
planning, preserving the existing failure and exit-status policy unless an
|
||||
explicitly identified requirement calls for a change.
|
||||
Use one compact repair-outcomes section in ordinary reporting. Give each repair
|
||||
branch one entry with the original proposal reference, its attempt count, and
|
||||
final disposition. Use the existing work-unit context for stage and chunk
|
||||
identity. Make it clear whether an approved revision was applied. Record
|
||||
operational errors and recoveries through the existing report framework and keep
|
||||
the current exit-status policy.
|
||||
Editorial rejection and voluntary withdrawal are correction outcomes. Keep
|
||||
conversation text and per-attempt details in optional redacted debug artifacts.
|
||||
Choose the report location, fields, and any schema version change during
|
||||
implementation planning.
|
||||
|
||||
### Revising the Span or Target Reference
|
||||
|
||||
|
||||
Reference in New Issue
Block a user