From e4e5dd74e33bd646481cf6b3893885c7781ef37f Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 15 Nov 2026 03:35:07 +0000 Subject: [PATCH] Record concurrency audit findings --- docs/roadmap/audit/04-concurrency.md | 227 +++++++++++++++++++++++++++ 1 file changed, 227 insertions(+) create mode 100644 docs/roadmap/audit/04-concurrency.md diff --git a/docs/roadmap/audit/04-concurrency.md b/docs/roadmap/audit/04-concurrency.md new file mode 100644 index 0000000..d93cf34 --- /dev/null +++ b/docs/roadmap/audit/04-concurrency.md @@ -0,0 +1,227 @@ +# Concurrency audit + +Reviewed source at `c0193df`. The full offline suite passed under +`go test -race -count=1 ./...` with Go 1.27.3. Selected capacity, +cancellation, coordination, and observation tests also passed eight runs +under the race detector. The timing findings below use Linux subprocesses, +synthetic loopback responses, and a temporary runner probe. The audit used +no live provider calls. + +## Architecture and synchronization reviewed + +Audita is a single-process Go CLI. The executable translates SIGINT and +SIGTERM into context cancellation. CommandKit handles commands and streams, +ConfigKit resolves settings, and the deterministic core normalizes and chunks +the transcript. The runner owns one stage barrier across fixed chunks. +Proposal and validation goroutines operate within a stage, and the runner +collects their results in a stable order before applying approved corrections. +The invocation shares a PromptKit engine across roles, with backend-local +admission, synchronous execution observations, and coordinated repair +branches. The CLI joins processing before assembling reports and delivering +output. [Signal setup](../../../cmd/audita/main.go#L16) +· [Package map](../../internal/overview.md) +· [Pipeline](../../internal/pipeline.md) +· [LLM runtime](../../internal/llm-runtime.md) + +- Proposal workers send one result per chunk. Their owner waits for all + senders before closing the channel. The validation channel is drained after + the worker join, which works because it has one slot per chunk and at most + one send per validation worker. Reducing that buffer would require changing + the drain order. A mutex protects the first fatal error, and capture of + warnings and execution evidence completes before the CLI reads it. + [Proposal worker lifecycle](../../../internal/framework/runner/runner.go#L385) + · [Error and result ownership](../../../internal/framework/runner/runner.go#L463) + · [Validation join](../../../internal/framework/runner/runner.go#L592) + · [Warning capture](../../../internal/cli/process.go#L42) + · [Evidence snapshot](../../../internal/cli/process_pipeline.go#L51) +- Existing tests cover cancellation while a peer call is active and another + waits for backend capacity, release after cancellation, independent backends, + repair response withholding, and nested validation on the repair backend. + Observations use synchronized collections, and a snapshot is taken after + workers join. Config and prompt capture happen before worker startup. + [Runner cancellation test](../../../internal/framework/runner/cancellation_test.go#L48) + · [Backend capacity tests](../../../internal/framework/llm/adapter_capacity_test.go#L42) + · [Repair coordination tests](../../../internal/framework/llm/coordinate_test.go#L77) + · [Concurrent observation test](../../../internal/framework/llm/observations_test.go#L59) + · [Snapshot contract](../../../internal/framework/llm/observations.go#L81) + +## Findings + +### 1. Blocked output delivery can ignore cancellation and report success + +The CLI checks the context immediately before `OutputSession.Finish`. +Delivery then opens a named pipe or writes stdout without observing the +context. With no FIFO reader, SIGTERM left the child blocked with +`in_progress` reports. When a reader attached, the child delivered the +transcript, finalized the reports as `completed`, and exited 0. The same +outcome occurred with a full stdout pipe and a 240 KB transcript. Draining +the pipe after SIGTERM allowed the write to finish and the child to report +success. The FIFO case was first recorded in the correctness audit. +[Last context check](../../../internal/cli/process_output.go#L54) +· [Stream open and write](../../../internal/core/diagnostics/output_session.go#L318) +· [Earlier FIFO finding](01-correctness.md#5-cancellation-during-fifo-delivery-can-hang-and-later-report-success) + +Pass cancellation through delivery. For cancellable file descriptors, +make both reader availability and write readiness interruptible. Keep the +guidance for subprocess clients to drain both streams, and define how cancellation +after a partial stream write is reflected in status and reports. Check +cancellation before committing a regular-file rename and use the existing +failed-report path when delivery is interrupted. +[Output lifecycle](../../operations.md#outputs-and-lifecycle) +· [Subprocess cancellation](../../integrations/subprocess.md#result-handling) + +### 2. Waiting for transcript input can prevent shutdown before preflight + +`loadProcessInputs` calls a context-free `os.ReadFile` for the transcript. +When the path names a FIFO, it blocks on a writer before the application +can check for cancellation again. In a subprocess probe with no writer, +SIGTERM left the child blocked without starting report initialization. +Supplying the input afterward allowed the child to exit 1. The process +therefore relies on a producer or forced termination to unblock that +read. The input parser accepts the JSON once it arrives. +[Input acquisition](../../../internal/cli/process_inputs.go#L46) +· [File read](../../../internal/core/io/files.go#L9) + +If pipe input remains supported, use a context-aware open and read on +supported platforms and stop reading after cancellation. If support is +intentionally limited to regular input files, make that contract explicit +and validate the path before a blocking open. +[Transcript input contract](../../integrations/transcript-glossary-files.md#transcript-json-contract) + +### 3. Cancellation can cross the runner's module commit boundary + +`Runner.Run` checks the context after the module's workers have joined, +then applies corrections, classifies work units, and commits the new sections +without another cancellation check. A temporary runner probe canceled the +parent immediately after that check sampled a live context. The final +module returned success, an applied correction, and a completed work unit. +An already-canceled invocation with no modules also returned success, as +the check is inside the module loop. The CLI has later context checks that +prevent publication in these cases, but can render canceled-run evidence +counting edits committed after the cancellation. Normalization and encoding +also run synchronously between CLI cancellation checks, which can delay +shutdown for large workloads. +[Before application](../../../internal/framework/runner/runner.go#L190) +· [Commit and return](../../../internal/framework/runner/runner.go#L264) +· [Local processing boundaries](../../../internal/cli/process.go#L55) +· [Earlier empty-pipeline finding](03-complexity.md#12-the-runner-accepts-an-already-canceled-empty-pipeline) + +Check cancellation at runner entry, before starting another module, +and just before committing computed updates. Keep the previous working +transcript until the commit check passes. If cancellation during large local +passes needs a shorter response time, add cooperative checks at natural +iteration boundaries. +[Cancellation and partial-result contract](../../internal/pipeline.md) + +### 4. A slow earlier proposal holds back later validation and retention + +The runner buffers completed proposals until the next chunk in order is +available. It assigns stage-wide indices, starts validators, and captures +ledger candidates only as that prefix advances. With two chunks and the +first response deliberately held open, the second response reached its +debug artifact while no validator call started. Releasing the first +response let both validators check the later correction. In a separate +run, canceling at the same point retained the later raw response and +counted one usable proposal response, but the correction ledger contained +only the module failure row. A slow earlier retry or timeout can therefore +stall independent validator work, and interruption can omit a completed +candidate from the ledger. +[Ordered proposal buffer](../../../internal/framework/runner/runner.go#L494) +· [Candidate capture and dispatch](../../../internal/framework/runner/runner.go#L520) +· [Earlier ledger finding](01-correctness.md#10-a-completed-proposal-can-disappear-from-the-interruption-ledger) + +Retain completed responses independently of scheduling and assign +deterministic indices to the retained evidence on a fatal return. For +validation latency, consider section-local identities during checking and +repair, with stable stage-wide identities assigned during collection. +Account for diagnostic filenames and report references in that change. +This can preserve the existing stage barrier. +[Stage barrier decision](../future.md#independent-whole-pipeline-traversal-by-chunk) + +### 5. Proposal fan-out makes admission under finite capacity depend on scheduling + +Each chunk immediately attempts a proposal in its own goroutine. PromptKit +limits admitted calls, while Audita has no ordering at the admission boundary. +With 12 chunks, one backend slot, and zero queue capacity, six otherwise +identical runs each reached the synthetic provider once and recorded 11 +recoverable capacity errors. Chunk 12 entered in five runs and chunk 2 +entered in the other. Which chunks can produce corrections can therefore +vary before the provider sees any prompt. The backend enforced its configured +limit and the error reports reflected the rejections. The weakness comes +from submitting a complete transcript's worth of proposals at once to a +finite queue. +[Proposal fan-out](../../../internal/framework/runner/runner.go#L385) +· [Capacity classification](../../../internal/framework/llm/adapter.go#L254) +· [Engine-local admission](../../policy/architecture.md#correction-and-concurrency-invariants) + +Make the interaction between chunk fan-out and finite queue capacity +explicit in operator guidance. If the desired processing policy is to wait +for more chunks to receive a turn, introduce bounded or ordered dispatch for +that policy while preserving PromptKit's ownership of backend permits and +the existing recoverable-rejection behavior for configurations that select +it. + +### 6. Report paths can change referent between initialization and finalization + +Preflight checks report destinations once. Each subsequent `atomicReplace` +resolves the original path again, including during the final report pass. +In a loopback probe, a report-directory symlink pointed to one directory +during initialization and was switched to another while the proposal +request was held. The CLI exited 0. The original directory still held +`in_progress` reports, and the replacement directory received +`completed` reports for the same run ID. Destination collision checks only +covered the referents inspected during preflight. +[Destination selection](../../../internal/core/diagnostics/output_session.go#L61) +· [Report initialization](../../../internal/core/diagnostics/output_session.go#L222) +· [Per-write path resolution](../../../internal/core/diagnostics/output_session.go#L410) + +Bind the checked report destinations for the session, or detect a changed +referent and fail before writing to it. Use the same target identity for +initialization, finalization, and the failed-report retry. This complements +the earlier path-resolution finding, which covers a path whose meaning is +wrong even without a concurrent change. +[Earlier path resolution finding](01-correctness.md#1-output-path-resolution-can-replace-a-different-file) + +### 7. Concurrent invocations sharing a report directory replace each other's evidence + +Different transcript output paths can select the same report directory, +including by default when the outputs have the same parent. Each invocation +owns temporary files and has its own run ID, but all replace the same three +report basenames. In a probe, run A initialized reports and then waited for +its proposal response. Run B completed with a different transcript output in +the shared directory. Releasing A let it complete and replace B's completed +reports. Both commands exited 0. Because report writes are per file, a +consumer reading during overlap can also observe files from different runs. +[Report-directory selection](../../../internal/cli/process_inputs.go#L72) +· [Fixed basenames](../../../internal/core/diagnostics/output_session.go#L12) +· [Final report writes](../../../internal/core/diagnostics/output_session.go#L286) + +Document per-invocation report destinations for concurrent subprocess +consumers. If supporting shared destinations directly, consider +invocation-specific publication or explicit serialization for that +destination. Per-file atomic replacement should remain in place. +[Report placement and lifecycle](../../operations.md#outputs-and-lifecycle) + +### 8. Shared module and validator ownership is implicit at their interfaces + +The runner shares one module instance across proposal workers and one +validator instance per chain entry across validation workers. Requests +also carry shared stage-level glossary data and transcript segment copies +whose category slices still alias the stage's data. The interfaces accept a +context but do not say that methods can run concurrently, borrowed data must +be read-only, or a canceled call must return for the stage to join. A +stateful implementation that mutates cached state or retains and mutates +borrowed data can race with peers. An implementation that blocks through +cancellation can prevent the worker join. Bundled modules and validators +use read-only shared configuration and per-call working state in the paths +reviewed. This is an internal ownership gap. +[Module and validator contracts](../../../internal/framework/contracts/contracts.go#L21) +· [Shared proposal instance](../../../internal/framework/runner/runner.go#L403) +· [Shared validation chain](../../../internal/framework/runner/runner.go#L548) +· [Shallow section copy](../../../internal/framework/runner/runner.go#L966) + +Document concurrent invocation, read-only inputs, and cooperative cancellation +at the contract boundary. Keep state local to each call or synchronize shared +mutable state explicitly when a future implementation needs it. If an +implementation needs to mutate section data, give it an owned copy of that +data.