Record concurrency audit findings

This commit is contained in:
2026-11-15 03:35:07 +00:00
parent c0193df387
commit e4e5dd74e3

View File

@@ -0,0 +1,227 @@
# Concurrency audit
Reviewed source at `c0193df`. The full offline suite passed under
`go test -race -count=1 ./...` with Go 1.27.3. Selected capacity,
cancellation, coordination, and observation tests also passed eight runs
under the race detector. The timing findings below use Linux subprocesses,
synthetic loopback responses, and a temporary runner probe. The audit used
no live provider calls.
## Architecture and synchronization reviewed
Audita is a single-process Go CLI. The executable translates SIGINT and
SIGTERM into context cancellation. CommandKit handles commands and streams,
ConfigKit resolves settings, and the deterministic core normalizes and chunks
the transcript. The runner owns one stage barrier across fixed chunks.
Proposal and validation goroutines operate within a stage, and the runner
collects their results in a stable order before applying approved corrections.
The invocation shares a PromptKit engine across roles, with backend-local
admission, synchronous execution observations, and coordinated repair
branches. The CLI joins processing before assembling reports and delivering
output. [Signal setup](../../../cmd/audita/main.go#L16)
· [Package map](../../internal/overview.md)
· [Pipeline](../../internal/pipeline.md)
· [LLM runtime](../../internal/llm-runtime.md)
- Proposal workers send one result per chunk. Their owner waits for all
senders before closing the channel. The validation channel is drained after
the worker join, which works because it has one slot per chunk and at most
one send per validation worker. Reducing that buffer would require changing
the drain order. A mutex protects the first fatal error, and capture of
warnings and execution evidence completes before the CLI reads it.
[Proposal worker lifecycle](../../../internal/framework/runner/runner.go#L385)
· [Error and result ownership](../../../internal/framework/runner/runner.go#L463)
· [Validation join](../../../internal/framework/runner/runner.go#L592)
· [Warning capture](../../../internal/cli/process.go#L42)
· [Evidence snapshot](../../../internal/cli/process_pipeline.go#L51)
- Existing tests cover cancellation while a peer call is active and another
waits for backend capacity, release after cancellation, independent backends,
repair response withholding, and nested validation on the repair backend.
Observations use synchronized collections, and a snapshot is taken after
workers join. Config and prompt capture happen before worker startup.
[Runner cancellation test](../../../internal/framework/runner/cancellation_test.go#L48)
· [Backend capacity tests](../../../internal/framework/llm/adapter_capacity_test.go#L42)
· [Repair coordination tests](../../../internal/framework/llm/coordinate_test.go#L77)
· [Concurrent observation test](../../../internal/framework/llm/observations_test.go#L59)
· [Snapshot contract](../../../internal/framework/llm/observations.go#L81)
## Findings
### 1. Blocked output delivery can ignore cancellation and report success
The CLI checks the context immediately before `OutputSession.Finish`.
Delivery then opens a named pipe or writes stdout without observing the
context. With no FIFO reader, SIGTERM left the child blocked with
`in_progress` reports. When a reader attached, the child delivered the
transcript, finalized the reports as `completed`, and exited 0. The same
outcome occurred with a full stdout pipe and a 240 KB transcript. Draining
the pipe after SIGTERM allowed the write to finish and the child to report
success. The FIFO case was first recorded in the correctness audit.
[Last context check](../../../internal/cli/process_output.go#L54)
· [Stream open and write](../../../internal/core/diagnostics/output_session.go#L318)
· [Earlier FIFO finding](01-correctness.md#5-cancellation-during-fifo-delivery-can-hang-and-later-report-success)
Pass cancellation through delivery. For cancellable file descriptors,
make both reader availability and write readiness interruptible. Keep the
guidance for subprocess clients to drain both streams, and define how cancellation
after a partial stream write is reflected in status and reports. Check
cancellation before committing a regular-file rename and use the existing
failed-report path when delivery is interrupted.
[Output lifecycle](../../operations.md#outputs-and-lifecycle)
· [Subprocess cancellation](../../integrations/subprocess.md#result-handling)
### 2. Waiting for transcript input can prevent shutdown before preflight
`loadProcessInputs` calls a context-free `os.ReadFile` for the transcript.
When the path names a FIFO, it blocks on a writer before the application
can check for cancellation again. In a subprocess probe with no writer,
SIGTERM left the child blocked without starting report initialization.
Supplying the input afterward allowed the child to exit 1. The process
therefore relies on a producer or forced termination to unblock that
read. The input parser accepts the JSON once it arrives.
[Input acquisition](../../../internal/cli/process_inputs.go#L46)
· [File read](../../../internal/core/io/files.go#L9)
If pipe input remains supported, use a context-aware open and read on
supported platforms and stop reading after cancellation. If support is
intentionally limited to regular input files, make that contract explicit
and validate the path before a blocking open.
[Transcript input contract](../../integrations/transcript-glossary-files.md#transcript-json-contract)
### 3. Cancellation can cross the runner's module commit boundary
`Runner.Run` checks the context after the module's workers have joined,
then applies corrections, classifies work units, and commits the new sections
without another cancellation check. A temporary runner probe canceled the
parent immediately after that check sampled a live context. The final
module returned success, an applied correction, and a completed work unit.
An already-canceled invocation with no modules also returned success, as
the check is inside the module loop. The CLI has later context checks that
prevent publication in these cases, but can render canceled-run evidence
counting edits committed after the cancellation. Normalization and encoding
also run synchronously between CLI cancellation checks, which can delay
shutdown for large workloads.
[Before application](../../../internal/framework/runner/runner.go#L190)
· [Commit and return](../../../internal/framework/runner/runner.go#L264)
· [Local processing boundaries](../../../internal/cli/process.go#L55)
· [Earlier empty-pipeline finding](03-complexity.md#12-the-runner-accepts-an-already-canceled-empty-pipeline)
Check cancellation at runner entry, before starting another module,
and just before committing computed updates. Keep the previous working
transcript until the commit check passes. If cancellation during large local
passes needs a shorter response time, add cooperative checks at natural
iteration boundaries.
[Cancellation and partial-result contract](../../internal/pipeline.md)
### 4. A slow earlier proposal holds back later validation and retention
The runner buffers completed proposals until the next chunk in order is
available. It assigns stage-wide indices, starts validators, and captures
ledger candidates only as that prefix advances. With two chunks and the
first response deliberately held open, the second response reached its
debug artifact while no validator call started. Releasing the first
response let both validators check the later correction. In a separate
run, canceling at the same point retained the later raw response and
counted one usable proposal response, but the correction ledger contained
only the module failure row. A slow earlier retry or timeout can therefore
stall independent validator work, and interruption can omit a completed
candidate from the ledger.
[Ordered proposal buffer](../../../internal/framework/runner/runner.go#L494)
· [Candidate capture and dispatch](../../../internal/framework/runner/runner.go#L520)
· [Earlier ledger finding](01-correctness.md#10-a-completed-proposal-can-disappear-from-the-interruption-ledger)
Retain completed responses independently of scheduling and assign
deterministic indices to the retained evidence on a fatal return. For
validation latency, consider section-local identities during checking and
repair, with stable stage-wide identities assigned during collection.
Account for diagnostic filenames and report references in that change.
This can preserve the existing stage barrier.
[Stage barrier decision](../future.md#independent-whole-pipeline-traversal-by-chunk)
### 5. Proposal fan-out makes admission under finite capacity depend on scheduling
Each chunk immediately attempts a proposal in its own goroutine. PromptKit
limits admitted calls, while Audita has no ordering at the admission boundary.
With 12 chunks, one backend slot, and zero queue capacity, six otherwise
identical runs each reached the synthetic provider once and recorded 11
recoverable capacity errors. Chunk 12 entered in five runs and chunk 2
entered in the other. Which chunks can produce corrections can therefore
vary before the provider sees any prompt. The backend enforced its configured
limit and the error reports reflected the rejections. The weakness comes
from submitting a complete transcript's worth of proposals at once to a
finite queue.
[Proposal fan-out](../../../internal/framework/runner/runner.go#L385)
· [Capacity classification](../../../internal/framework/llm/adapter.go#L254)
· [Engine-local admission](../../policy/architecture.md#correction-and-concurrency-invariants)
Make the interaction between chunk fan-out and finite queue capacity
explicit in operator guidance. If the desired processing policy is to wait
for more chunks to receive a turn, introduce bounded or ordered dispatch for
that policy while preserving PromptKit's ownership of backend permits and
the existing recoverable-rejection behavior for configurations that select
it.
### 6. Report paths can change referent between initialization and finalization
Preflight checks report destinations once. Each subsequent `atomicReplace`
resolves the original path again, including during the final report pass.
In a loopback probe, a report-directory symlink pointed to one directory
during initialization and was switched to another while the proposal
request was held. The CLI exited 0. The original directory still held
`in_progress` reports, and the replacement directory received
`completed` reports for the same run ID. Destination collision checks only
covered the referents inspected during preflight.
[Destination selection](../../../internal/core/diagnostics/output_session.go#L61)
· [Report initialization](../../../internal/core/diagnostics/output_session.go#L222)
· [Per-write path resolution](../../../internal/core/diagnostics/output_session.go#L410)
Bind the checked report destinations for the session, or detect a changed
referent and fail before writing to it. Use the same target identity for
initialization, finalization, and the failed-report retry. This complements
the earlier path-resolution finding, which covers a path whose meaning is
wrong even without a concurrent change.
[Earlier path resolution finding](01-correctness.md#1-output-path-resolution-can-replace-a-different-file)
### 7. Concurrent invocations sharing a report directory replace each other's evidence
Different transcript output paths can select the same report directory,
including by default when the outputs have the same parent. Each invocation
owns temporary files and has its own run ID, but all replace the same three
report basenames. In a probe, run A initialized reports and then waited for
its proposal response. Run B completed with a different transcript output in
the shared directory. Releasing A let it complete and replace B's completed
reports. Both commands exited 0. Because report writes are per file, a
consumer reading during overlap can also observe files from different runs.
[Report-directory selection](../../../internal/cli/process_inputs.go#L72)
· [Fixed basenames](../../../internal/core/diagnostics/output_session.go#L12)
· [Final report writes](../../../internal/core/diagnostics/output_session.go#L286)
Document per-invocation report destinations for concurrent subprocess
consumers. If supporting shared destinations directly, consider
invocation-specific publication or explicit serialization for that
destination. Per-file atomic replacement should remain in place.
[Report placement and lifecycle](../../operations.md#outputs-and-lifecycle)
### 8. Shared module and validator ownership is implicit at their interfaces
The runner shares one module instance across proposal workers and one
validator instance per chain entry across validation workers. Requests
also carry shared stage-level glossary data and transcript segment copies
whose category slices still alias the stage's data. The interfaces accept a
context but do not say that methods can run concurrently, borrowed data must
be read-only, or a canceled call must return for the stage to join. A
stateful implementation that mutates cached state or retains and mutates
borrowed data can race with peers. An implementation that blocks through
cancellation can prevent the worker join. Bundled modules and validators
use read-only shared configuration and per-call working state in the paths
reviewed. This is an internal ownership gap.
[Module and validator contracts](../../../internal/framework/contracts/contracts.go#L21)
· [Shared proposal instance](../../../internal/framework/runner/runner.go#L403)
· [Shared validation chain](../../../internal/framework/runner/runner.go#L548)
· [Shallow section copy](../../../internal/framework/runner/runner.go#L966)
Document concurrent invocation, read-only inputs, and cooperative cancellation
at the contract boundary. Keep state local to each call or synchronize shared
mutable state explicitly when a future implementation needs it. If an
implementation needs to mutate section data, give it an owned copy of that
data.