The intra-module pipelining gap identified in this audit has now been addressed:
- section proposal jobs are launched promptly for each module;
- as section proposals become available in deterministic section order, section-local validator work starts without waiting for all section proposals to finish;
- deterministic validators run before LLM-backed validators for each section;
- proposal and validation LLM calls can overlap through the existing composed scheduler path;
- module application remains a single deterministic apply barrier.
This audit checks whether Audita currently maximizes available LLM concurrency within each module by overlapping proposal and validation work.
Current state:
- Modules are serial and deterministic.
- Section proposal generation is concurrent and scheduler-limited.
- Validation (deterministic + LLM-backed) starts only after all section proposals complete.
- Proposal and validation LLM work therefore does **not** overlap in time within a module today.
Conclusion:
- Current behavior is correct and deterministic, but it does **not** fully match the target pipelined intra-module behavior.
- A narrow runner refactor is needed to pipeline section-level validation after section proposal completion while preserving apply-once-per-module semantics.
- Not as currently orchestrated by runner; validators receive module-wide `CandidateProposal` slices.
6.**Does LLM validator batching depend on all module proposals being present?**
- Current invocation pattern does. `LLMBackedValidator.Validate` batches over the full provided candidate set; runner currently supplies full-module candidates.
- Yes, if indexes are preallocated/stable by section-order offsets (or equivalent deterministic mapping) before emitting validator work.
10.**Are diagnostics paths stable/deterministic enough if validator calls interleave with proposal calls?**
- Mostly yes: stage names are deterministic (`module:proposal:section-*`, `module:validator:batch-*`).
- But batch indices and artifact emission timing could change unless explicitly stabilized by per-section deterministic indexing.
11.**Do schedulers enforce total/proposal/validation limits correctly when proposal and validation overlap?**
- Scheduler composition supports this (`global` + proposal/validation subcaps), but runner currently does not create overlap, so end-to-end overlap behavior is not exercised by runtime path.
12.**Do existing tests prove proposal and validation jobs can overlap while respecting total concurrency?**
- No direct end-to-end runner test proving overlap of proposal and validation jobs in one module.
13.**Are there tests proving FIFO queue contains both proposal and validation jobs in submission order?**
- No cross-type FIFO test. FIFO is tested at scheduler unit level (`internal/framework/llm/scheduler_test.go`), and composed-cap tests exist in CLI tests, but not mixed proposal+validation submission order in runner.
14.**Smallest safe implementation change to reach target behavior?**
- Refactor runner module execution into a two-lane pipeline:
- keep current concurrent section proposal launch,
- on each section completion, run deterministic validators for that section’s proposals,
- immediately enqueue section LLM-validator work for survivors,
- collect all approved proposals in deterministic global proposal-index order,
- apply once per module exactly as today.
## Minimal Change Plan (No Redesign)
Primary change area:
-`internal/framework/runner/runner.go`
Narrow implementation approach:
1. Split current `collectSectionProposals` into pipeline stages that emit per-section proposal results plus deterministic section ordering metadata.
2. Introduce per-section validator execution helper that preserves existing validator semantics but operates on section-local candidate subsets where safe.
3. Preserve current final aggregation shape (`ModuleResult`, applied/skipped/rejected records) and apply-once-per-module behavior.
4. Preserve deterministic proposal index assignment by precomputing stable per-section index ranges or equivalent deterministic indexing strategy.
5. Keep scheduler interfaces unchanged; continue using composed proposal/validation schedulers from CLI wiring.
Tests to add/update (minimum):
- Runner test proving proposal and validation LLM calls overlap in time while total cap is respected.
- Runner test proving deterministic final transcript/proposal indexes/report ordering under out-of-order section completion.
- Runner/CLI test proving mixed proposal+validation jobs still honor global + subcap limits.
- Optional scheduler integration test proving mixed proposal/validation FIFO submission behavior at runtime boundary.
## Notes
- No runtime behavior was changed in this audit.
- Existing docs contain prior-audit historical notes (`docs/llm-concurrency-audit.md`) that may now be stale relative to current repository state; this file focuses only on current intra-module pipeline behavior.