Files
audita/docs/roadmap/audit/05-resources.md

277 lines
18 KiB
Markdown
Raw Normal View History

# Resource management audit
Reviewed 2026-11-15 at `e4e5dd7`. This audit covers the Audita CLI, file and stream
I/O, configuration and prompt capture, transcript processing, the pinned
PromptKit execution path, and required and optional artifact generation. The
review also checked local release entry points for resource ownership. The
findings below identify cleanup defects and avoidable resource growth. The
remediation options are proposals for follow-up work.
## Architecture and lifecycle
Audita is a single-process Go CLI. The executable owns the signal context and
passes caller-owned streams through CommandKit. The CLI loads configuration and
captured prompt sources, reads the transcript, and preflights destinations. An
output session owns the temporary file and directory used to publish a regular
transcript by rename. Report replacements use their own short-lived staging
directories. Named pipes and stdout are delivered directly. Debug files are
written to a run-specific directory for later inspection.
[Signal setup](../../../cmd/audita/main.go#L16) · [Package map](../../internal/overview.md)
· [Preflight](../../../internal/framework/preflight/preflight.go#L41)
· [Output session](../../../internal/core/diagnostics/output_session.go#L45)
· [Artifact lifecycle](../../operations.md#outputs-and-lifecycle)
The CLI normalizes the transcript before passing it to the runner. The runner
holds the working transcript and outcome records in memory, creates fixed
chunks, and joins proposal and validator workers at each module boundary.
Role-bound clients share an invocation-local PromptKit engine for HTTP
requests, backend admission, retries, structural repair, and correction-repair
exchanges. The CLI assembles report and debug artifacts from the completed or
interrupted run.
[Normalization and invocation](../../../internal/cli/process.go#L55)
· [Runner ownership](../../../internal/framework/runner/runner.go#L121)
· [Worker joins](../../../internal/framework/runner/runner.go#L435)
· [Validation drain](../../../internal/framework/runner/runner.go#L592)
· [LLM runtime](../../internal/llm-runtime.md)
· [Report assembly](../../../internal/cli/process_pipeline.go#L50)
Resource release on the reviewed cooperative paths includes the executable's
signal subscription, proposal and validator worker joins, and PromptKit's
operation and request contexts, retry timers, admission reservations, and active
permits. PromptKit removes canceled waiters or transfers ownership of a permit
to the caller that won the race with cancellation. Its built-in HTTP path closes
response bodies and reuses the standard Go transport in the CLI. The pinned
source bounds successful response reads at 16 MiB and provider error reads at
64 KiB, applying reader limits even without a known content length. These are
per-response byte bounds. In the pinned module, the relevant ownership is in
`engine.go`, `internal/usecase/runner.go`, `internal/generation/coordinator.go`,
`internal/capacity/manager.go`, and the HTTP code in `internal/llm/`.
[Pinned version](../../../go.mod#L11) · [CLI transport selection](../../../internal/cli/process_inputs.go#L42)
· [Engine construction](../../../internal/framework/llm/runtime.go#L103)
· [Capacity cancellation test](../../../internal/framework/llm/adapter_capacity_test.go#L42)
· [Peer cancellation test](../../../internal/framework/runner/cancellation_test.go#L48)
## Findings
### 1. Failed output cleanup loses ownership of temporary paths
`OutputSession.Close` closes the transcript descriptor, then tries to remove the
staged payload and its private directory. It clears both path fields even when
removal returns an error. A temporary permission or filesystem failure can leave
a directory or an unpublished payload behind. Finalization surfaces the first
cleanup failure, but the CLI's deferred `Close` has lost the paths it needs to
retry. The preparation and early preflight cancellation branches also discard
cleanup errors. The separate `atomicReplace` report helper ignores the results
of its deferred removes, so it can silently leave staging files or directories
on a cleanup failure. Recurring failures can accumulate disk blocks and inodes.
[Transcript preparation and cleanup](../../../internal/core/diagnostics/output_session.go#L163)
· [Finalization](../../../internal/core/diagnostics/output_session.go#L273)
· [Deferred close](../../../internal/cli/process.go#L41)
· [Preflight cancellation](../../../internal/framework/preflight/preflight.go#L61)
· [Report replacement](../../../internal/core/diagnostics/output_session.go#L410)
A disposable probe against the unmodified `OutputSession` forced a nonempty
staging directory. The first `Close` removed the payload and returned an error.
After the probe removed the extra entry, the second `Close` returned success
while the empty directory remained. The probe cleaned up its own files.
Keep each path until removal succeeds or the filesystem says it is absent. Join
cleanup errors with preparation or publication failures and surface report
staging cleanup failures through the output owner. Distinguish a published file
with leftover staging from a failure to publish it.
### 2. Blocked stream I/O extends resource lifetime past cancellation
The input path uses `os.ReadFile` without a context. A FIFO can block before open
completes or hold an open descriptor while waiting for more input. Final
delivery similarly opens or writes a FIFO, or writes to the supplied stdout,
after the last cancellation check. With no peer or a peer that stops consuming,
the invocation retains its memory and any acquired stream descriptor. During
output it also retains the encoded transcript and rendered report buffers, and
the session cannot finish report finalization. A supervisor can therefore end up
waiting on canceled processes indefinitely. Prior subprocess probes reproduced
the blocked input, FIFO output, and stdout paths with SIGTERM and released them
by supplying or draining the streams.
[Transcript read](../../../internal/core/io/files.go#L9)
· [Final output check](../../../internal/cli/process_output.go#L54)
· [Output write and close](../../../internal/core/diagnostics/output_session.go#L318)
· [Output probe](04-concurrency.md#1-blocked-output-delivery-can-ignore-cancellation-and-report-success)
· [Input probe](04-concurrency.md#2-waiting-for-transcript-input-can-prevent-shutdown-before-preflight)
Make open and readiness waits cancellable for streams Audita owns on supported
platforms. For caller-owned writers, define how the caller supplies cancellation
or drains the stream. Ensure canceled delivery reaches session finalization,
and account for any bytes already delivered. A detached write goroutine alone
would still retain the writer and buffer if the peer stays blocked.
### 3. Backend admission leaves chunk fan-out and result retention unbounded
Each module starts one proposal goroutine per chunk. A worker builds a section
transcript and the domain and adapter prepare request inputs before PromptKit
admission. Backend capacity limits admitted runs and active provider attempts
when configured. The pinned `internal/capacity/manager.go` imposes no limit for
backend IDs without a configured pool. With many chunks, local goroutines and
preparation grow with chunk count even when a backend has a small configured
queue. Without a pool, concurrent HTTP requests can also grow with chunk count,
bringing sockets, request timers, encoded prompts, and response buffers with
them. Timeouts and byte limits apply individually. The `examples/service` profile
illustrates explicit backend limits.
[Proposal workers](../../../internal/framework/runner/runner.go#L385)
· [Adapter request preparation](../../../internal/framework/llm/adapter.go#L113)
· [Backend capacity example](../../../examples/service/prompts.yml#L1)
The proposal and validation channels and their companion maps are sized to the
chunk count. A slow leading chunk can keep many later proposal results, including
captured exchanges when correction repair is active, in the ordered pending map.
Validation can also start a goroutine for each ready chunk and holds completed
results in its channel until the join. PromptKit has already released those
calls' permits while Audita still holds their data. A long, finely chunked
transcript can raise client memory and descriptor demand even when each response
fits the per-call limit. Earlier probes substantiate the ordered backlog and the
interaction with a finite queue.
[Ordered buffering](../../../internal/framework/runner/runner.go#L460)
· [Validation workers](../../../internal/framework/runner/runner.go#L547)
· [Result collection](../../../internal/framework/runner/runner.go#L592)
· [Ordered backlog probe](04-concurrency.md#4-a-slow-earlier-proposal-holds-back-later-validation-and-retention)
· [Admission probe](04-concurrency.md#5-proposal-fan-out-makes-admission-under-finite-capacity-depend-on-scheduling)
Consider a bounded dispatch window tied to explicit scheduling or operator
settings. Drain validation results while workers run before reducing channel
capacity. A bounded result window or earlier use of section-local identities
could reduce retention while preserving deterministic correction identifiers.
Preserve the module barrier and PromptKit's permit ownership. Decide explicitly
when backend saturation should wait or remain a recoverable capacity rejection.
### 4. Whole-transcript copies amplify memory at input and publication
`loadProcessInputs` reads the entire file into a byte slice. The parser then
validates the bytes, decodes the whole JSON document into generic maps and
slices, re-encodes the selected segment array, and decodes that array into typed
segments. Extra fields can occupy memory in the generic tree even if the typed
transcript discards them. The CLI keeps the raw input and typed source reachable
through its `processInputs` struct, which it passes into finalization. The raw
input's last consumer is the optional source debug artifact. At the other end,
both output encoders build one map per segment and a complete indented JSON
payload before delivery. A large transcript can run out of memory before the
first provider call or after all correction work has completed.
[Input load and ownership](../../../internal/cli/process_inputs.go#L25)
· [Parser](../../../internal/core/schema/transcript.go#L35)
· [Segment re-encoding](../../../internal/core/schema/transcript.go#L115)
· [Last source use and finalization](../../../internal/cli/process.go#L51)
· [Bare-array encoder](../../../internal/core/schema/transcript.go#L224)
· [Envelope encoder](../../../internal/core/outputschema/registry.go#L62)
· [Related parse fidelity finding](01-correctness.md#6-transcript-shape-detection-loses-integer-and-unknown-field-fidelity)
Decode the supported input shapes into typed segments without the generic
round trip and release source representations after their last consumers. Typed
output projections would remove the per-segment maps. For regular files, an
incremental array encoder can write into the prepared temporary file and publish
only after encoding succeeds. Preserve both schemas and decide how a writer-based
encoder would handle partial output on a stream.
### 5. Summary-only runs retain a full grid of work and operation records
`Runner.Run` eagerly allocates and initializes one `WorkUnitOutcome` per
configured module and chunk before the first module starts. The work-unit grid
retains nested operations and repair summaries until `processreport.Build`.
Completed modules also flatten the same operations into `ModuleResult.Operations`.
The CLI selects `SummaryOnly` only after the runner returns when reports are
disabled. The builder scans the operation and work-unit records to count them,
then skips detailed rows. On many-module, many-chunk runs, this incurs memory
proportional to the module-by-chunk product and all retained operation records,
including entries for work never started after early failure. This is in
addition to the debug decision history identified in the efficiency audit.
[Grid allocation](../../../internal/framework/runner/runner.go#L127)
· [Operation copies](../../../internal/framework/runner/runner.go#L264)
· [CLI report selection](../../../internal/cli/process_pipeline.go#L78)
· [Summary aggregation](../../../internal/framework/processreport/report.go#L41)
· [Decision retention](02-efficiency.md#5-ordinary-runs-retain-debug-only-decision-history)
When neither detailed reports nor the debug ledger is requested, fold finalized
work into invocation counters and derive not-started counts from the total
inventory. Retain the failed module identity for run-level errors. With details
enabled, consider one ownership path for operation records and deriving
not-started rows on demand, preserving the complete reporting grid and ordered
evidence.
### 6. Debug capture adds memory and disk pressure
Debug runs retain all logical calls and execution events in memory while also
writing several files per proposal, validation, and repair interaction. At the
end, the observation collector copies its records into a sorted snapshot.
`BuildLLMExecution` groups events by call and builds a second representation
for serialization. The JSON redactor creates encoded bytes, a decoded generic
tree, and final formatted bytes. Large event histories can produce a memory
peak while the transcript and the evidence needed for the correction ledger
are still live. Each debug invocation adds a run directory. Large or repeated
debug runs can consume space or inodes needed by the transcript and reports
when they share the filesystem. Once an artifact write fails, the CLI can
continue to collect and retain a warning for each later failure until
finalization, even if the destination stays full.
[Observer retention and snapshot](../../../internal/framework/llm/observations.go#L81)
· [Execution artifact assembly](../../../internal/framework/processreport/llm.go#L61)
· [JSON redaction](../../../internal/core/redaction/redaction.go#L53)
· [Run directories](../../../internal/core/diagnostics/run_dir.go#L34)
· [Interaction files](../../../internal/framework/llm/diagnostics.go#L35)
· [Warning accumulation](../../../internal/cli/process.go#L42)
· [Warning delivery](../../../internal/cli/process_output.go#L17)
Reuse a single ordered set of observations for final aggregation, or stream
redacted records through a disk-backed artifact while preserving the final
summary. Coalesce repeated debug write failures and report which evidence was
lost. Give operators visibility into debug footprint and an explicit retention
or quota decision for runs that need it. Keep optional evidence failure distinct
from required output delivery.
### 7. Correction repair replays growing history under per-correction budgets
Each actionable rejection starts a branch from the original section exchange.
PromptKit's pinned `internal/domain/exchange.go` shares the immutable history
through parent references. Its `internal/usecase/runner.go` flattens that history
with the next turn during preparation, and the HTTP client encodes it into the
request body. Audita revalidates returned revisions and restarts the configured
turn allowance for the next rejected correction. A large section prompt is sent
again on every branch. If a branch uses `A` turns, starts with `S` bytes of
history, and each turn adds roughly `H` bytes, the cumulative logical repair
request payload is on the order of `A*S + A²*H` for that branch. Retries and
revalidation add work. With debug enabled, Audita writes the prepared history
again for each logical repair turn. Across many actionable rejections, this can
increase provider context usage, outbound traffic, and disk use even when every
call fits the configured attempt and time budgets.
[Per-correction configuration](../../config.md#settings)
· [Branch start](../../../internal/framework/runner/repair.go#L85)
· [Coordination and per-turn artifacts](../../../internal/framework/runner/repair.go#L146)
· [Revalidation](../../../internal/framework/runner/repair.go#L227)
· [Prepared snapshot](../../../internal/framework/llm/adapter.go#L152)
· [Pinned version](../../../go.mod#L11)
Expose aggregate repair work and request size so operators can size runs. If a
work envelope is needed, consider an opt-in section or invocation budget that
accounts for revalidation as well as repair turns and records why any branch
stopped. For debug storage, consider recording a shared history once with
references sufficient to reconstruct the redacted request. Preserve the
conversation context used for editorial decisions.
## Verification
- The repository's complete `go test -count=1 ./...` suite passed on Go 1.27.3.
It includes output write and rename failure cleanup and release of backend
capacity after cancellation. The disposable probe in finding 1 exercises a
removal failure beyond those existing tests.
[Rename failure test](../../../internal/core/diagnostics/output_session_test.go#L98)
· [Transcript failure test](../../../internal/core/diagnostics/output_session_test.go#L394)
· [Capacity release test](../../../internal/framework/llm/adapter_capacity_test.go#L42)
- The pinned PromptKit `internal/llm`, `internal/capacity`, and
`internal/generation` packages passed their existing tests. These exercise
response byte limits and body closure, capacity contention and cancellation,
and cancellation during retry delays.
[Pinned version](../../../go.mod#L11)
- Local release entry points were reviewed without executing a release. The
wrapper replaces its shell with ReleaseKit, and the CI bootstrap registers an
exit trap to remove its temporary directory.
[Release wrapper](../../../scripts/release.sh#L10)
· [CI bootstrap](../../../.woodpecker/release.yml#L29)
- The scaling findings describe allocation and ownership from source. Peak RSS,
throughput, provider cost, and filesystem-specific failure frequency were
outside the review's measurement scope.