From 571ad70a5495926e530a914e5252ae6ed6ebe71c Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 15 Nov 2026 23:38:22 +0000 Subject: [PATCH] Record native backend admission decision --- docs/roadmap/audit/assessment.md | 60 ++++++++++++++++++-------------- docs/roadmap/audit/catalog.md | 6 +++- 2 files changed, 38 insertions(+), 28 deletions(-) diff --git a/docs/roadmap/audit/assessment.md b/docs/roadmap/audit/assessment.md index ab85bf2..f25aca7 100644 --- a/docs/roadmap/audit/assessment.md +++ b/docs/roadmap/audit/assessment.md @@ -4,7 +4,9 @@ remediation not implemented. D1 was resolved on 2026-11-15 and the architecture policy updated; the focused cleanup audit is complete and implementation remains pending. D2 was resolved on 2026-11-15: callers own report destination isolation; -the operator-documentation update remains pending. +the operator-documentation update remains pending. D3 was resolved on the same +date: native admission semantics remain in place; local scheduling improvements +await implementation planning. This document owns classification, recommended scope, and pending decisions for the [catalog](catalog.md). The source audits retain their evidence; their suggested remedies are not approved requirements. @@ -38,8 +40,8 @@ when substantial restructuring is appropriate. **Ready** means suitable for implementation planning within the stated scope, not implemented or validated by this document. **Investigate** denotes technical -work the agent can undertake without a product decision. **D3–D7** identify -pending substantive choices; D1 and D2 are recorded below as resolved. **Closed** +work the agent can undertake without a product decision. **D4–D7** identify +pending substantive choices; D1–D3 are recorded below as resolved. **Closed** denotes no remediation for the specified scope. Mixed entries deliberately separate ready work from decisions and exclusions. @@ -98,7 +100,7 @@ behavior from proposed extensions. | [27. Keep the validator execution class used by the runner](catalog.md#27-keep-the-validator-execution-class-used-by-the-runner) | C, A | Ready | Runtime and ledger can disagree for supported injected validator classes. Record the effective execution class with decisions and project it into the ledger. Preserve existing output fields and fallback behavior; no new validator registration policy. | | [28. Verify signal and exit handling in the production executable](catalog.md#28-verify-signal-and-exit-handling-in-the-production-executable) | T | Ready | The startup helper duplicates the production boundary, so coverage does not prove actual executable signal wiring. Exercise representative startup and cancellation through the real executable or shared production entry point; keep narrower tests for other CLI behavior. | | [29. Fail the release bootstrap when cleanliness cannot be established](catalog.md#29-fail-the-release-bootstrap-when-cleanliness-cannot-be-established) | C, T | Ready | The shell probe establishes that a failed status command can appear clean. Check its exit result explicitly and test that boundary. ReleaseKit already owns the independent delivery validation; do not add another local release validator or publication gate. | -| [30. Bound pending proposal work before backend admission](catalog.md#30-bound-pending-proposal-work-before-backend-admission) | E, A, T; P | Investigate implementation; D3 for policy | Fan-out and retained results scale with chunks; finite native admission rejection is itself working as configured. Design bounded local preparation/dispatch and concurrent result draining without changing backend ownership, stable identities, cancellation joins, or configured rejection semantics. Measure representative memory/contention to size the implementation. Guaranteeing a turn despite capacity rejection or adding a new scheduling control requires D3. | +| [30. Bound pending proposal work before backend admission](catalog.md#30-bound-pending-proposal-work-before-backend-admission) | E, A, T | Investigate implementation under resolved D3 | Bound local preparation/dispatch and drain results while preserving PromptKit admission ownership, configured waiting/rejection, stable identities, module barriers, and cancellation joins. Do not add an outer capacity-retry loop, duplicate backend limiter, or eventual-admission promise. The current scheduling-dependent selection of rejected chunks is not a compatibility requirement. Measure representative memory/contention to guide the implementation; any future admission-policy gap belongs upstream. | | [31. Reduce peak memory for large transcript input and output](catalog.md#31-reduce-peak-memory-for-large-transcript-input-and-output) | E, A; P | Ready in part; D4 | Remove generic JSON round trips and release whole-input representations after their last use. The D1 audit additionally identifies source-debug reconstruction: write the acquired bytes directly and retain parsed/normalized views separately, without reopening the input. Share typed output projection; preserve encoding-failure and stream-delivery semantics. An input-size limit remains D4, and FIFO exclusion is not selected. | | [32. Retain only the run evidence required by enabled outputs](catalog.md#32-retain-only-the-run-evidence-required-by-enabled-outputs) | E, A | Ready | The production consumers establish that much completed evidence is unused without its corresponding output. Retain only required live repair state, compact counts, and selected report/debug detail. Derive not-started counts without a full eager grid where possible; keep complete matrices when enabled and preserve fatal-origin evidence. | | [33. Limit debug retention costs within a run](catalog.md#33-limit-debug-retention-costs-within-a-run) | E, A; P | Ready in part; D5 | Reduce duplicate event representations and serialization; coalesce equivalent warnings while retaining failure counts and affected evidence. Those are efficiency improvements. Quotas, automatic deletion, stopping future capture, or replacing self-contained interactions with reference-based artifacts change retained evidence and need D5. | @@ -222,33 +224,37 @@ a workflow the caller already controls. The audit demonstrates overwriting when callers share destinations; it does not establish an application promise of per-run isolation. Correct writes to checked destinations in #1 remain independent. +## Resolved D3: retain native backend admission semantics + +**Accepted:** 2026-11-15. **Applies to:** #30, with scheduling interactions in #15. +**Status:** decision recorded; local scheduling/resource improvements await +implementation planning. + +PromptKit retains ownership of backend admission, including configured waiting +and recoverable capacity rejection. Use native settings where operators want +more waiting capacity. Audita will not add an outer admission-retry loop, a +separate backend limiter, or a guarantee that every chunk eventually receives +admission. + +Improve Audita's local scheduling and memory use as engineering work: bound +pending preparation/dispatch, drain completed results, and avoid unnecessary +whole-transcript allocation. Preserve module barriers, stable identities, +ordered application, cancellation/join behavior, and native admission semantics. +The scheduling-dependent choice of chunks rejected by the current fan-out is +not a compatibility requirement; a different admission outcome caused by better +local scheduling does not itself establish a new admission policy. + +Choosing to wait until every chunk can receive a turn would be a separate +processing policy. If reconsidered, first assess native PromptKit configuration +and seek upstream support for any gap. Native rejection is not a PromptKit defect. +Representative memory, contention, and cancellation checks should guide the +local implementation without creating a second admission system. + ## Open Decisions -The five decisions D3–D7 below remain pending. Accepting or rejecting an optional +The four decisions D4–D7 below remain pending. Accepting or rejecting an optional extension must not delay the independent ready portions of its catalog entries. -### D3. Should local dispatch change what backend saturation means? - -**Applies to:** #30, with scheduling interactions in #15. - -**Recommendation:** retain PromptKit's configured waiting and recoverable -capacity-rejection behavior. Improve local resource use and dispatch as -engineering work, but do not add an outer capacity-retry loop, a separate -backend limiter, or a promise that every chunk eventually receives admission. -Use native settings where operators want more waiting capacity. - -**Alternative:** explicitly select an application policy that holds pending -chunks until they can receive a turn, subject to cancellation and existing -operation budgets. That could avoid losing useful work under finite queues, but -requires defining admission timing, fairness across proposal/validation/repair, -and whether deadlines cover the new wait. Seek suitable PromptKit support before -implementing local admission policy. - -**Why approval:** replacing configured rejection with guaranteed waiting changes -run outcomes and timing. A bounded internal work window alone is not inherently -a policy change; its implementation must demonstrate that it stays on the chosen -side of this boundary. Native rejection is not a PromptKit defect. - ### D4. Should Audita impose an application-level transcript input budget? **Applies to:** the optional restriction in #31. diff --git a/docs/roadmap/audit/catalog.md b/docs/roadmap/audit/catalog.md index b08dc7a..4f594a3 100644 --- a/docs/roadmap/audit/catalog.md +++ b/docs/roadmap/audit/catalog.md @@ -301,7 +301,11 @@ comparison helper. The delivery tool has a separate prepared-checkout check. One goroutine per chunk makes finite-capacity rejection depend on scheduling and can retain requests and results across the whole run. Bound dispatch and result retention without changing the module barrier or backend admission ownership. -Verify actual contention and cancellation in tests. +**Resolved D3:** preserve PromptKit's configured waiting and recoverable rejection. +Do not add an outer capacity-retry loop, duplicate backend limiter, or eventual- +admission promise. Current scheduling-dependent rejection order is not a +compatibility requirement. Verify memory use, actual contention, and cancellation +while improving local scheduling; any future admission-policy gap belongs upstream. [Concurrency §5](04-concurrency.md#5-proposal-fan-out-makes-admission-under-finite-capacity-depend-on-scheduling) · [Resources §3](05-resources.md#3-backend-admission-leaves-chunk-fan-out-and-result-retention-unbounded) · [Testing §6](08-testing.md#6-concurrency-fixtures-leave-some-schedules-unproved)