diff --git a/docs/roadmap/audit/assessment.md b/docs/roadmap/audit/assessment.md index f25aca7..aeb6df1 100644 --- a/docs/roadmap/audit/assessment.md +++ b/docs/roadmap/audit/assessment.md @@ -6,7 +6,8 @@ policy updated; the focused cleanup audit is complete and implementation remains pending. D2 was resolved on 2026-11-15: callers own report destination isolation; the operator-documentation update remains pending. D3 was resolved on the same date: native admission semantics remain in place; local scheduling improvements -await implementation planning. +await implementation planning. D4 was also resolved on 2026-11-15: improve +input memory use without introducing an application-level input-size limit. This document owns classification, recommended scope, and pending decisions for the [catalog](catalog.md). The source audits retain their evidence; their suggested remedies are not approved requirements. @@ -40,8 +41,8 @@ when substantial restructuring is appropriate. **Ready** means suitable for implementation planning within the stated scope, not implemented or validated by this document. **Investigate** denotes technical -work the agent can undertake without a product decision. **D4–D7** identify -pending substantive choices; D1–D3 are recorded below as resolved. **Closed** +work the agent can undertake without a product decision. **D5–D7** identify +pending substantive choices; D1–D4 are recorded below as resolved. **Closed** denotes no remediation for the specified scope. Mixed entries deliberately separate ready work from decisions and exclusions. @@ -101,7 +102,7 @@ behavior from proposed extensions. | [28. Verify signal and exit handling in the production executable](catalog.md#28-verify-signal-and-exit-handling-in-the-production-executable) | T | Ready | The startup helper duplicates the production boundary, so coverage does not prove actual executable signal wiring. Exercise representative startup and cancellation through the real executable or shared production entry point; keep narrower tests for other CLI behavior. | | [29. Fail the release bootstrap when cleanliness cannot be established](catalog.md#29-fail-the-release-bootstrap-when-cleanliness-cannot-be-established) | C, T | Ready | The shell probe establishes that a failed status command can appear clean. Check its exit result explicitly and test that boundary. ReleaseKit already owns the independent delivery validation; do not add another local release validator or publication gate. | | [30. Bound pending proposal work before backend admission](catalog.md#30-bound-pending-proposal-work-before-backend-admission) | E, A, T | Investigate implementation under resolved D3 | Bound local preparation/dispatch and drain results while preserving PromptKit admission ownership, configured waiting/rejection, stable identities, module barriers, and cancellation joins. Do not add an outer capacity-retry loop, duplicate backend limiter, or eventual-admission promise. The current scheduling-dependent selection of rejected chunks is not a compatibility requirement. Measure representative memory/contention to guide the implementation; any future admission-policy gap belongs upstream. | -| [31. Reduce peak memory for large transcript input and output](catalog.md#31-reduce-peak-memory-for-large-transcript-input-and-output) | E, A; P | Ready in part; D4 | Remove generic JSON round trips and release whole-input representations after their last use. The D1 audit additionally identifies source-debug reconstruction: write the acquired bytes directly and retain parsed/normalized views separately, without reopening the input. Share typed output projection; preserve encoding-failure and stream-delivery semantics. An input-size limit remains D4, and FIFO exclusion is not selected. | +| [31. Reduce peak memory for large transcript input and output](catalog.md#31-reduce-peak-memory-for-large-transcript-input-and-output) | E, A | Ready under resolved D4 | Remove generic JSON round trips and release whole-input representations after their last use. The D1 audit additionally identifies source-debug reconstruction: write the acquired bytes directly and retain parsed/normalized views separately, without reopening the input. Share typed output projection; preserve encoding-failure and stream-delivery semantics. Do not introduce an application-level input-size limit; calling integrations own their upload/process budgets. Preserve accepted input formats and stream support. | | [32. Retain only the run evidence required by enabled outputs](catalog.md#32-retain-only-the-run-evidence-required-by-enabled-outputs) | E, A | Ready | The production consumers establish that much completed evidence is unused without its corresponding output. Retain only required live repair state, compact counts, and selected report/debug detail. Derive not-started counts without a full eager grid where possible; keep complete matrices when enabled and preserve fatal-origin evidence. | | [33. Limit debug retention costs within a run](catalog.md#33-limit-debug-retention-costs-within-a-run) | E, A; P | Ready in part; D5 | Reduce duplicate event representations and serialization; coalesce equivalent warnings while retaining failure counts and affected evidence. Those are efficiency improvements. Quotas, automatic deletion, stopping future capture, or replacing self-contained interactions with reference-based artifacts change retained evidence and need D5. | | [34. Account for cumulative correction-repair history and work](catalog.md#34-account-for-cumulative-correction-repair-history-and-work) | E; P | Investigate costs; D6 | Repeated full context follows the deliberately selected repair architecture, so growth alone is not a defect. Measure cumulative requests, bytes, revalidation, and debug costs using existing observations or development instrumentation. Avoid duplicate local representations. New aggregate stopping budgets or reduced model context require D6; do not add ordinary-report fields merely to perform this analysis. | @@ -250,29 +251,27 @@ and seek upstream support for any gap. Native rejection is not a PromptKit defec Representative memory, contention, and cancellation checks should guide the local implementation without creating a second admission system. +## Resolved D4: improve memory use without an input-size limit + +**Accepted:** 2026-11-15. **Applies to:** #31. +**Status:** decision recorded; efficiency improvements await implementation. + +Do not add an application-level transcript input-size limit in this remediation. +Remove unnecessary copies, generic serialization round trips, and retained +representations after their last use while preserving accepted input formats, +stream support, and output/error behavior. Calling applications remain +responsible for their upload and process budgets. + +The audit establishes memory amplification, but provides no measured threshold +that justifies rejecting otherwise valid CLI workloads. An optional input-byte +budget could help integrations without their own enforcement; it is not selected +because it adds configuration and rejection behavior without a demonstrated need. + ## Open Decisions -The four decisions D4–D7 below remain pending. Accepting or rejecting an optional +The three decisions D5–D7 below remain pending. Accepting or rejecting an optional extension must not delay the independent ready portions of its catalog entries. -### D4. Should Audita impose an application-level transcript input budget? - -**Applies to:** the optional restriction in #31. - -**Recommendation:** do not add a new input limit in this remediation. Remove -avoidable copies and let upload-serving integrations impose their own upload -and process budgets. The source audit identifies an exposure condition, not a -measured size threshold suitable for all CLI workloads. - -**Alternative:** add an explicit opt-in input-byte budget checked during -acquisition. This helps integrations without their own enforcement, but adds -configuration, a new rejection condition, and diagnostic/documentation work. -A default cap would additionally reject legitimate large jobs and needs -independent justification. - -**Why approval:** accepted input would become conditional on a new constraint. -No input-format narrowing or regular-file-only policy is selected. - ### D5. Should debug capture acquire quotas, automatic retention rules, or an early-stop policy? **Applies to:** the policy portion of #33. diff --git a/docs/roadmap/audit/catalog.md b/docs/roadmap/audit/catalog.md index 4f594a3..849169d 100644 --- a/docs/roadmap/audit/catalog.md +++ b/docs/roadmap/audit/catalog.md @@ -316,7 +316,9 @@ Whole-file reads, a generic JSON tree, retained source bytes, and full output buffers amplify memory before and after model work. Reduce live copies and write the acquired source bytes directly to its debug snapshot instead of reconstructing them for redaction. Keep parsed/normalized views separate. -Input-size limits remain a separate D4 decision. +**Resolved D4:** improve memory use without adding an application-level +transcript-size limit. Calling integrations own their upload/process budgets; +preserve accepted input formats and stream support. [Redaction removal §3](14-redaction.md#3-save-the-acquired-source-snapshot-without-reconstructing-it) · [Resources §4](05-resources.md#4-whole-transcript-copies-amplify-memory-at-input-and-publication) · [Security §7](07-security.md#7-untrusted-input-size-and-blocking-io-can-exhaust-subprocess-workers)