Record decision to preserve best-effort debug capture
This commit is contained in:
@@ -8,6 +8,8 @@ the operator-documentation update remains pending. D3 was resolved on the same
|
||||
date: native admission semantics remain in place; local scheduling improvements
|
||||
await implementation planning. D4 was also resolved on 2026-11-15: improve
|
||||
input memory use without introducing an application-level input-size limit.
|
||||
D5 was resolved on the same date: retain best-effort debug capture and operator
|
||||
ownership of retention and cleanup; efficiency improvements remain pending.
|
||||
This document owns classification, recommended
|
||||
scope, and pending decisions for the [catalog](catalog.md). The source audits
|
||||
retain their evidence; their suggested remedies are not approved requirements.
|
||||
@@ -41,8 +43,8 @@ when substantial restructuring is appropriate.
|
||||
|
||||
**Ready** means suitable for implementation planning within the stated scope,
|
||||
not implemented or validated by this document. **Investigate** denotes technical
|
||||
work the agent can undertake without a product decision. **D5–D7** identify
|
||||
pending substantive choices; D1–D4 are recorded below as resolved. **Closed**
|
||||
work the agent can undertake without a product decision. **D6–D7** identify
|
||||
pending substantive choices; D1–D5 are recorded below as resolved. **Closed**
|
||||
denotes no remediation for the specified scope. Mixed entries deliberately
|
||||
separate ready work from decisions and exclusions.
|
||||
|
||||
@@ -104,7 +106,7 @@ behavior from proposed extensions.
|
||||
| [30. Bound pending proposal work before backend admission](catalog.md#30-bound-pending-proposal-work-before-backend-admission) | E, A, T | Investigate implementation under resolved D3 | Bound local preparation/dispatch and drain results while preserving PromptKit admission ownership, configured waiting/rejection, stable identities, module barriers, and cancellation joins. Do not add an outer capacity-retry loop, duplicate backend limiter, or eventual-admission promise. The current scheduling-dependent selection of rejected chunks is not a compatibility requirement. Measure representative memory/contention to guide the implementation; any future admission-policy gap belongs upstream. |
|
||||
| [31. Reduce peak memory for large transcript input and output](catalog.md#31-reduce-peak-memory-for-large-transcript-input-and-output) | E, A | Ready under resolved D4 | Remove generic JSON round trips and release whole-input representations after their last use. The D1 audit additionally identifies source-debug reconstruction: write the acquired bytes directly and retain parsed/normalized views separately, without reopening the input. Share typed output projection; preserve encoding-failure and stream-delivery semantics. Do not introduce an application-level input-size limit; calling integrations own their upload/process budgets. Preserve accepted input formats and stream support. |
|
||||
| [32. Retain only the run evidence required by enabled outputs](catalog.md#32-retain-only-the-run-evidence-required-by-enabled-outputs) | E, A | Ready | The production consumers establish that much completed evidence is unused without its corresponding output. Retain only required live repair state, compact counts, and selected report/debug detail. Derive not-started counts without a full eager grid where possible; keep complete matrices when enabled and preserve fatal-origin evidence. |
|
||||
| [33. Limit debug retention costs within a run](catalog.md#33-limit-debug-retention-costs-within-a-run) | E, A; P | Ready in part; D5 | Reduce duplicate event representations and serialization; coalesce equivalent warnings while retaining failure counts and affected evidence. Those are efficiency improvements. Quotas, automatic deletion, stopping future capture, or replacing self-contained interactions with reference-based artifacts change retained evidence and need D5. |
|
||||
| [33. Limit debug retention costs within a run](catalog.md#33-limit-debug-retention-costs-within-a-run) | E, A | Ready under resolved D5 | Reduce duplicate event representations and serialization; coalesce equivalent warnings while retaining failure counts and affected-artifact information. Preserve requested evidence, self-contained interactions, nonfatal debug failures, and continued best-effort capture. Add no quotas, automatic deletion, or automatic stopping after write failures; operators own retention and cleanup. |
|
||||
| [34. Account for cumulative correction-repair history and work](catalog.md#34-account-for-cumulative-correction-repair-history-and-work) | E; P | Investigate costs; D6 | Repeated full context follows the deliberately selected repair architecture, so growth alone is not a defect. Measure cumulative requests, bytes, revalidation, and debug costs using existing observations or development instrumentation. Avoid duplicate local representations. New aggregate stopping budgets or reduced model context require D6; do not add ordinary-report fields merely to perform this analysis. |
|
||||
| [35. Avoid rebuilding growing normalization prefixes](catalog.md#35-avoid-rebuilding-growing-normalization-prefixes) | E | Ready | Growing merge prefixes are repeatedly scanned and copied. Use incremental text/category accumulation and exact estimator components while retaining joiner behavior, punctuation rounding, limits, and category order. Benchmarks should verify the improvement; large refactoring is not a reason to retain quadratic work. |
|
||||
| [36. Skip validation context work beyond the usable budget](catalog.md#36-skip-validation-context-work-beyond-the-usable-budget) | E, A, T | Ready | Unusable optional context is serialized needlessly. Stop measurement once required-only or ordered-prefix costs settle the fit. Share trimming/batching policy without losing the independent serialized estimator oracle or changing context priority and oversized-item handling. |
|
||||
@@ -114,7 +116,7 @@ behavior from proposed extensions.
|
||||
| [40. Use one validator contract and one owner for decision semantics](catalog.md#40-use-one-validator-contract-and-one-owner-for-decision-semantics) | A, C, M | Ready | Duplicate ports and runner-specific verdict rewriting obscure one existing validation contract. Consolidate types, applicability, and decision meaning with their owner; remove unused envelope fields. Preserve zero-budget shortlisting, independent repair feedback, cardinality, and unavailable outcomes. No broad plugin framework or new validator policy is needed. |
|
||||
| [41. Carry structured failure facts through model operations and reporting](catalog.md#41-carry-structured-failure-facts-through-model-operations-and-reporting) | A, C | Ready | Failure class and reason already exist but are discarded and reconstructed. Carry typed facts through role-specific handling and report counts. Use a consistent internal-failure reason fallback for missing reasons while retaining the original failure class and cause. Avoid reclassifying an operation from its display reason. |
|
||||
| [42. Keep required output publication separate from optional debug retention](catalog.md#42-keep-required-output-publication-separate-from-optional-debug-retention) | A, C | Ready | Required publication and optional retained evidence have distinct lifecycles. Separate their APIs and share regular-file staging where behavior is common. Preserve transcript-before-report ordering, nontransactional finalization, private staging, and nonfatal debug errors. Coordinate with #1, #7, #16, and #22. |
|
||||
| [43. Inject one interaction diagnostics contract](catalog.md#43-inject-one-interaction-diagnostics-contract) | A, M | Ready under resolved D1 | The focused audit identifies removable secret-source assertions, lazy discovery closures, combined-role lists, and preflight/writer credential arguments. Complete common interaction writing without those dependencies. Preserve role-owned payloads, artifact association, and nonfatal write callbacks and synchronization; coordinate with #42/#48. D5 still governs retention changes. |
|
||||
| [43. Inject one interaction diagnostics contract](catalog.md#43-inject-one-interaction-diagnostics-contract) | A, M | Ready under resolved D1 | The focused audit identifies removable secret-source assertions, lazy discovery closures, combined-role lists, and preflight/writer credential arguments. Complete common interaction writing without those dependencies. Preserve role-owned payloads, artifact association, and nonfatal write callbacks and synchronization; coordinate with #42/#48. Resolved D5 preserves best-effort capture and operator-owned retention. |
|
||||
| [44. Simplify structured-call completion and remove obsolete record fields](catalog.md#44-simplify-structured-call-completion-and-remove-obsolete-record-fields) | A, M, T | Ready | Ordinary calls and coordinator turns share completion responsibilities; obsolete payload copies have no production reader. Use explicit once-only completion and remove superseded state after moving meaningful assertions. Preserve native terminal counts, typed causes, and undelivered diagnostic evidence. |
|
||||
| [45. Retire the test-only direct continuation branch](catalog.md#45-retire-the-test-only-direct-continuation-branch) | M, T | Ready | Direct continuation is test-only; production uses coordination. Move sibling-isolation and structural-repair coverage onto the coordinator before removing the branch. Removing the redundant adapter surface does not remove PromptKit capability or production repair. |
|
||||
| [46. Use a single proposal handoff and test real runner indexing](catalog.md#46-use-a-single-proposal-handoff-and-test-real-runner-indexing) | A, E, M, T | Ready | Generator enrichment is discarded and real indices are assigned by the runner. Keep one proposal handoff with original local references, and assert actual deterministic runner indices under observed out-of-order completion. Review emitted diagnostic start_index separately before dropping it; move useful tests before removing their harness. |
|
||||
@@ -144,8 +146,8 @@ Group related work by owner rather than implementing catalog numbers as 65
|
||||
independent tasks. Fixes and their supporting ownership changes belong together:
|
||||
|
||||
- Output publication and diagnostic integrity: #1, #3–9, #16, #22, #26,
|
||||
#42–43, #63, #65, incorporating resolved D1 cleanup and D2 caller ownership while
|
||||
keeping D5 policy portions separate.
|
||||
#42–43, #63, #65, incorporating resolved D1 cleanup, D2 caller ownership, and D5
|
||||
best-effort capture with operator-owned retention.
|
||||
- Transcript and glossary correctness: #11–13, #17–21, #35–39, #51–54.
|
||||
- Runner and model-operation lifecycle: #10, #14–15, #23–25, #27, #30,
|
||||
#32, #40–41, #44–49, #55–56, #64.
|
||||
@@ -267,31 +269,31 @@ that justifies rejecting otherwise valid CLI workloads. An optional input-byte
|
||||
budget could help integrations without their own enforcement; it is not selected
|
||||
because it adds configuration and rejection behavior without a demonstrated need.
|
||||
|
||||
## Resolved D5: preserve best-effort debug capture
|
||||
|
||||
**Accepted:** 2026-11-15. **Applies to:** #33.
|
||||
**Status:** decision recorded; efficiency improvements await implementation.
|
||||
|
||||
Preserve requested diagnostic evidence and continue best-effort capture. Debug
|
||||
write failures remain nonfatal. Remove redundant in-memory representations and
|
||||
serialization, and consolidate equivalent warnings while retaining counts and
|
||||
affected-artifact information. Keep interaction artifacts self-contained.
|
||||
|
||||
Do not add capture quotas, automatic deletion, or automatic shutdown of later
|
||||
capture after write failures in this remediation. An individual failure need
|
||||
not prevent later writes from succeeding; continued capture can retain useful
|
||||
evidence for an unsuccessful run. Operators own retention and cleanup.
|
||||
|
||||
Optional budgets or persistent-failure stopping rules could reduce disk use and
|
||||
repeated failed work, but deliberately omit evidence and add configuration and
|
||||
reporting obligations. They are not selected. Internal efficiency improvements
|
||||
should preserve the evidence operators requested.
|
||||
|
||||
## Open Decisions
|
||||
|
||||
The three decisions D5–D7 below remain pending. Accepting or rejecting an optional
|
||||
The two decisions D6–D7 below remain pending. Accepting or rejecting an optional
|
||||
extension must not delay the independent ready portions of its catalog entries.
|
||||
|
||||
### D5. Should debug capture acquire quotas, automatic retention rules, or an early-stop policy?
|
||||
|
||||
**Applies to:** the policy portion of #33.
|
||||
|
||||
**Recommendation:** preserve requested evidence and best-effort capture; optimize
|
||||
its representation internally and coalesce equivalent write warnings with counts
|
||||
and affected-artifact information. Do not add quotas, automatic deletion, or
|
||||
disable later writes after the first failure in this remediation.
|
||||
|
||||
**Alternative:** adopt explicit opt-in limits or a documented stop-capture policy
|
||||
after persistent failures. This can protect disk and reduce repeated failed
|
||||
work, but intentionally loses requested evidence and must expose what was
|
||||
omitted. Automatic cleanup across runs requires separate retention ownership.
|
||||
Replacing self-contained interaction artifacts with shared-history references
|
||||
also needs a deliberate consumer contract.
|
||||
|
||||
**Why approval:** these policies change what operators receive from debug mode.
|
||||
Elimination of duplicate in-memory representations remains independent.
|
||||
The permissions/default-root proposal in #4 is being reassessed under D1.
|
||||
|
||||
### D6. Should correction repair gain aggregate budgets or reduced conversation context?
|
||||
|
||||
**Applies to:** the optional behavior changes in #34.
|
||||
|
||||
@@ -337,8 +337,11 @@ collect compact counts when detailed evidence is disabled.
|
||||
|
||||
Execution histories and repeated serialization increase peak memory, and retained
|
||||
interaction files and warnings can accumulate after write failures. Reduce
|
||||
duplicate representations and define retention and failure behavior appropriate to
|
||||
opt-in capture.
|
||||
duplicate representations and serialization, and consolidate equivalent warnings
|
||||
with counts and affected-artifact information. **Resolved D5:** preserve requested
|
||||
evidence, self-contained interactions, nonfatal failures, and continued best-effort
|
||||
capture. Add no quotas, automatic deletion, or automatic stopping after write
|
||||
failures; operators own retention and cleanup.
|
||||
[Resources §6](05-resources.md#6-debug-capture-adds-memory-and-disk-pressure)
|
||||
|
||||
### 34. Account for cumulative correction-repair history and work
|
||||
|
||||
Reference in New Issue
Block a user