Record decision to retain repair context and limits
This commit is contained in:
@@ -10,6 +10,8 @@ await implementation planning. D4 was also resolved on 2026-11-15: improve
|
||||
input memory use without introducing an application-level input-size limit.
|
||||
D5 was resolved on the same date: retain best-effort debug capture and operator
|
||||
ownership of retention and cleanup; efficiency improvements remain pending.
|
||||
D6 was also resolved on 2026-11-15: retain full repair conversation history and
|
||||
per-correction limits, without aggregate budgets or context reduction at this time.
|
||||
This document owns classification, recommended
|
||||
scope, and pending decisions for the [catalog](catalog.md). The source audits
|
||||
retain their evidence; their suggested remedies are not approved requirements.
|
||||
@@ -43,8 +45,8 @@ when substantial restructuring is appropriate.
|
||||
|
||||
**Ready** means suitable for implementation planning within the stated scope,
|
||||
not implemented or validated by this document. **Investigate** denotes technical
|
||||
work the agent can undertake without a product decision. **D6–D7** identify
|
||||
pending substantive choices; D1–D5 are recorded below as resolved. **Closed**
|
||||
work the agent can undertake without a product decision. **D7** identifies
|
||||
the remaining substantive choice; D1–D6 are recorded below as resolved. **Closed**
|
||||
denotes no remediation for the specified scope. Mixed entries deliberately
|
||||
separate ready work from decisions and exclusions.
|
||||
|
||||
@@ -107,7 +109,7 @@ behavior from proposed extensions.
|
||||
| [31. Reduce peak memory for large transcript input and output](catalog.md#31-reduce-peak-memory-for-large-transcript-input-and-output) | E, A | Ready under resolved D4 | Remove generic JSON round trips and release whole-input representations after their last use. The D1 audit additionally identifies source-debug reconstruction: write the acquired bytes directly and retain parsed/normalized views separately, without reopening the input. Share typed output projection; preserve encoding-failure and stream-delivery semantics. Do not introduce an application-level input-size limit; calling integrations own their upload/process budgets. Preserve accepted input formats and stream support. |
|
||||
| [32. Retain only the run evidence required by enabled outputs](catalog.md#32-retain-only-the-run-evidence-required-by-enabled-outputs) | E, A | Ready | The production consumers establish that much completed evidence is unused without its corresponding output. Retain only required live repair state, compact counts, and selected report/debug detail. Derive not-started counts without a full eager grid where possible; keep complete matrices when enabled and preserve fatal-origin evidence. |
|
||||
| [33. Limit debug retention costs within a run](catalog.md#33-limit-debug-retention-costs-within-a-run) | E, A | Ready under resolved D5 | Reduce duplicate event representations and serialization; coalesce equivalent warnings while retaining failure counts and affected-artifact information. Preserve requested evidence, self-contained interactions, nonfatal debug failures, and continued best-effort capture. Add no quotas, automatic deletion, or automatic stopping after write failures; operators own retention and cleanup. |
|
||||
| [34. Account for cumulative correction-repair history and work](catalog.md#34-account-for-cumulative-correction-repair-history-and-work) | E; P | Investigate costs; D6 | Repeated full context follows the deliberately selected repair architecture, so growth alone is not a defect. Measure cumulative requests, bytes, revalidation, and debug costs using existing observations or development instrumentation. Avoid duplicate local representations. New aggregate stopping budgets or reduced model context require D6; do not add ordinary-report fields merely to perform this analysis. |
|
||||
| [34. Account for cumulative correction-repair history and work](catalog.md#34-account-for-cumulative-correction-repair-history-and-work) | E | Investigate costs under resolved D6 | Retain full conversation history and existing per-correction limits. Measure cumulative requests, tokens/bytes, revalidation, and debug costs using existing observations or development instrumentation; remove unnecessary local duplication. Do not add aggregate stopping budgets or reduce model context at this time. Revisit only if production evidence demonstrates a need; no new ordinary-report fields are required for this analysis. |
|
||||
| [35. Avoid rebuilding growing normalization prefixes](catalog.md#35-avoid-rebuilding-growing-normalization-prefixes) | E | Ready | Growing merge prefixes are repeatedly scanned and copied. Use incremental text/category accumulation and exact estimator components while retaining joiner behavior, punctuation rounding, limits, and category order. Benchmarks should verify the improvement; large refactoring is not a reason to retain quadratic work. |
|
||||
| [36. Skip validation context work beyond the usable budget](catalog.md#36-skip-validation-context-work-beyond-the-usable-budget) | E, A, T | Ready | Unusable optional context is serialized needlessly. Stop measurement once required-only or ordered-prefix costs settle the fit. Share trimming/batching policy without losing the independent serialized estimator oracle or changing context priority and oversized-item handling. |
|
||||
| [37. Reuse checked processing selections and matching views](catalog.md#37-reuse-checked-processing-selections-and-matching-views) | E, A | Ready | Readiness and execution derive the same selections separately. Carry one checked processing plan and reuse immutable effective views and inspections per captured runtime. Preserve provenance and first-failure attribution; cache keys must include ordered selections and exemptions. Do not add cross-invocation caches or reload configuration. |
|
||||
@@ -289,30 +291,30 @@ repeated failed work, but deliberately omit evidence and add configuration and
|
||||
reporting obligations. They are not selected. Internal efficiency improvements
|
||||
should preserve the evidence operators requested.
|
||||
|
||||
## Resolved D6: retain repair context and per-correction limits
|
||||
|
||||
**Accepted:** 2026-11-15. **Applies to:** #34.
|
||||
**Status:** decision recorded; cost analysis and efficiency work remain pending.
|
||||
|
||||
Retain full conversation history and the existing per-correction limits. Do not
|
||||
add aggregate section/run repair budgets or reduce conversation context at this
|
||||
time. Measure cumulative requests, tokens/bytes, revalidation, and debug costs,
|
||||
and remove unnecessary local copies without changing the model's evidence or
|
||||
which corrections receive repair attempts. Use existing observations or focused
|
||||
development instrumentation; no new ordinary-report fields are needed merely
|
||||
to perform the analysis.
|
||||
|
||||
Resource growth partly reflects the deliberately selected repair behavior.
|
||||
Aggregate budgets could leave later corrections unrepaired because earlier
|
||||
ones consumed the allowance, while context reduction could reduce repair
|
||||
quality. Neither change follows from the audit finding alone. Revisit these
|
||||
options only if production evidence demonstrates a need, through a separate
|
||||
policy decision.
|
||||
|
||||
## Open Decisions
|
||||
|
||||
The two decisions D6–D7 below remain pending. Accepting or rejecting an optional
|
||||
extension must not delay the independent ready portions of its catalog entries.
|
||||
|
||||
### D6. Should correction repair gain aggregate budgets or reduced conversation context?
|
||||
|
||||
**Applies to:** the optional behavior changes in #34.
|
||||
|
||||
**Recommendation:** retain the selected full conversation history and
|
||||
per-correction limits. Measure cumulative work and remove local duplication
|
||||
before proposing any new stopping or context policy. The resource growth partly
|
||||
reflects deliberate useful work rather than an implementation defect.
|
||||
|
||||
**Alternative:** add an opt-in section/run budget covering repair and
|
||||
revalidation, with explicit termination outcomes; or separately evaluate reduced
|
||||
context. Budgets provide predictable resource envelopes but may leave later
|
||||
corrections unrepaired. Context reduction can lower costs but can also reduce
|
||||
editorial quality. Neither follows automatically from the audit's asymptotic
|
||||
description.
|
||||
|
||||
**Why approval:** these changes affect which corrections get another chance or
|
||||
what evidence the model sees. Measurements using existing observations do not
|
||||
require approval or new ordinary-report schema fields.
|
||||
D7 is the only remaining open decision. Its resolution must not delay the
|
||||
independent ready portions of the catalog.
|
||||
|
||||
### D7. Should this remediation add processing progress events?
|
||||
|
||||
|
||||
@@ -347,8 +347,11 @@ failures; operators own retention and cleanup.
|
||||
### 34. Account for cumulative correction-repair history and work
|
||||
|
||||
Each rejected candidate can start its own bounded conversation and repeat the
|
||||
original prompt history, with additional revalidation. Measure total requests and
|
||||
bytes across branches before deciding on changes to context or run-level budgets.
|
||||
original prompt history, with additional revalidation. **Resolved D6:** retain
|
||||
full conversation history and existing per-correction limits; add no aggregate
|
||||
budgets or context reduction at this time. Measure cumulative work and remove
|
||||
unnecessary local duplication. Revisit behavior changes only if production
|
||||
evidence demonstrates a need.
|
||||
[Resources §7](05-resources.md#7-correction-repair-replays-growing-history-under-per-correction-budgets)
|
||||
|
||||
### 35. Avoid rebuilding growing normalization prefixes
|
||||
|
||||
Reference in New Issue
Block a user