From c0193df3873f2055942d09b391501789c2932512 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 15 Nov 2026 03:09:51 +0000 Subject: [PATCH] Record maintainability and complexity audit findings --- docs/roadmap/audit/03-complexity.md | 322 ++++++++++++++++++++++++++++ 1 file changed, 322 insertions(+) create mode 100644 docs/roadmap/audit/03-complexity.md diff --git a/docs/roadmap/audit/03-complexity.md b/docs/roadmap/audit/03-complexity.md new file mode 100644 index 0000000..9d45456 --- /dev/null +++ b/docs/roadmap/audit/03-complexity.md @@ -0,0 +1,322 @@ +# Maintainability and complexity audit + +Reviewed source at `717d59c`. The codebase graph identified complex control +flow and cross-package call paths. A Go AST scan counted branches and +declaration spans for 353 functions and methods across 43 tracked Go +directories, excluding test files. The score below starts at one and adds +one per conditional, loop, non-default switch or select case, and short-circuit +operator, including those inside callbacks. The ranking guided source review. +Findings are based on static analysis. + +## Architecture + +Audita is a single-process Go CLI. CommandKit routes commands and streams. +ConfigKit resolves application settings, and the core packages parse, +normalize, and chunk transcripts and derive glossary selections. The +runner executes modules in order over fixed chunks. Deterministic and +LLM-backed validators gate proposals and revalidate repairs. PromptKit owns +model execution, backend admission, and bounded conversation history. Audita +applies approved edits and derives reports and debug artifacts from processing +outcomes. [Package map](../../internal/overview.md) +· [Pipeline](../../internal/pipeline.md) +· [Ownership rules](../../policy/architecture.md#ownership-and-dependency-direction) + +The runner and validator runtime had the largest non-test Go source totals, +at 1,373 and 1,351 physical lines respectively. The highest-scoring functions +concentrated in orchestration, model integration, and report projection. + +| Function | Branch score | Declaration lines | +| --- | ---: | ---: | +| `runner.repairCandidates` | 46 | 225 | +| `runner.runModulePipeline` | 36 | 212 | +| `processreport.Build` | 36 | 140 | +| `(*LLMBackedValidator).Validate` | 31 | 170 | +| `(*Runner).Run` | 30 | 168 | +| `runner.validateSectionCandidates` | 30 | 154 | +| `(*RoleClient).prepareCall` | 28 | 153 | +| `processreport.BuildCorrectionLedger` | 27 | 176 | +| `proposal_generation.GenerateCandidates` | 25 | 148 | +| `diagnostics.SelectOutput` | 21 | 66 | + +## Refactoring opportunities + +### 1. Give module execution one place to commit its result + +`Run` owns the stage barrier, working transcript, inventory, module setup, +application, and partial-return assembly. The pipeline-error, application-error, +and success paths each construct a module result and copy work units into the +invocation inventory. Repair approvals are marked interrupted in both `Run` +and `runModulePipeline`. The latter also coordinates two worker lifetimes, +ordered proposal indexing, cancellation, and evidence aggregation. Changing +what an interrupted unit retains requires following both functions and their +callbacks. The earlier missing-ledger-candidate finding follows from indexing +and retention occurring only while scheduling validation. [Module commit and failure paths](../../../internal/framework/runner/runner.go#L187) +· [Section orchestration](../../../internal/framework/runner/runner.go#L441) +· [Interrupted evidence finding](01-correctness.md#10-a-completed-proposal-can-disappear-from-the-interruption-ledger) + +Extract the section validation-and-repair worker from the collector. Let the +collector own ordered indexing and the evidence returned by workers, with +one path to register a fatal cause and cancel peers. Let one module-level +completion path resolve applied, skipped, and interrupted repair outcomes +and commit updated sections after successful application. Preserve validation +starting as ordered proposals become available, deterministic proposal IDs, +joined workers on fatal paths, and the stage barrier. + +### 2. Separate one repair branch from the section's repair loop + +`repairCandidates` selects candidates, builds each request, interprets +delivered turns, revalidates revisions, attributes outcomes, and writes debug +artifacts. Nested closures share mutable `current`, `localRef`, `findings`, +attempt counts, and section-level slices. For example, a successful revision +updates the branch, original proposal filtering, revalidation operations, and +approval list inside the coordinator callback. Error, withdrawal, exhaustion, +and accepted final-turn paths finish that work at different points. +[Branch setup](../../../internal/framework/runner/repair.go#L56) +· [Turn handling](../../../internal/framework/runner/repair.go#L146) +· [Repair behavior](../../internal/modules.md#failure-and-outcome-behavior) + +Keep the outer loop responsible for selecting and collecting branches. Have +one branch-local operation return the terminal outcome, optional approved +revision, and operation evidence. Separate construction of the next request, +assessment of a delivered turn, and optional attempt recording. Define the +branch's internal states once so application and reports can project their +own stable labels. Keep PromptKit responsible for history and turn limits. +Retain final-turn revalidation, target changes within the section, and +independence of sibling approvals. + +### 3. Put validator applicability and decision meaning at their owning boundary + +`feedbackCandidates` switches on validator names and module keys to decide +whether each validator has usable evidence. The validator implementations +also check previews and module policy. The runner then rewrites an +`original_text_presence` approval if its preview is ambiguous, so the same +validator's returned verdict can mean something different outside the runner. +`validateSectionCandidates` interleaves those checks with two eligibility +modes, cardinality checks, filtering, and evidence recording. The framework +also declares identical validator interfaces under `contracts` and +`validators`. [Feedback selection](../../../internal/framework/runner/feedback.go#L13) +· [Runner verdict handling](../../../internal/framework/runner/runner.go#L680) +· [Original-text validator](../../../internal/framework/validators/deterministic.go#L37) +· [Validator contract](../../../internal/framework/contracts/contracts.go#L29) +· [Duplicate interface](../../../internal/framework/validators/models.go#L95) + +Put each check's applicability and verdict together in its validator or its +built-in registration. Use one validator interface and one internal decision +disposition, deriving serialized approval from it. Keep the runner responsible +for submitting the shortlist or collecting independent findings, checking +returned identities, and applying only fully approved candidates. Preserve +ordered chains, zero-budget shortlisting, unavailable decisions, and separate +revalidation of revisions. The earlier audit records the span-only +protection mismatch. [Protection correctness](01-correctness.md#8-span-only-protection-can-reject-a-valid-edit-or-miss-a-repair-opportunity) + +### 4. Separate validation-batch assessment from the diagnostic writer + +`LLMBackedValidator.Validate` interleaves request preparation, prompt selection, +batch dispatch, reconciliation, partial-result assembly, and debug payload +construction. Its nested writer consumes the original request, local +reference mapping, raw response, reconciliation result, and failure. Proposal +generation and validation declare writer interfaces with matching signatures +and duplicate four-field artifact types. Their adapter wrappers copy each +path from `llm.InteractionArtifacts`. Repair writes through the same concrete +writer directly. A change to the common artifact envelope or safe error +serialization therefore touches several packages. +[Batch loop](../../../internal/framework/validators/llm_validators.go#L112) +· [Proposal writer contract](../../../internal/framework/proposal_generation/generate.go#L23) +· [Validation writer contract](../../../internal/framework/validators/models.go#L84) +· [Writer adapter](../../../internal/framework/runner/runner.go#L994) +· [Repair artifact capture](../../../internal/framework/runner/repair.go#L165) + +Give one batch operation responsibility for dispatch and assessment. Return +reconciled decisions, its compact outcome, and only the evidence needed by the +diagnostic path. Reuse one internal artifact value type and writer seam for +common metadata, redaction, and safe write failures. Keep the per-role +payload construction with its domain owner. Preserve local-reference +reconciliation, missing and duplicate decision handling, partial results on +fatal return, and nonfatal debug-write warnings. + +### 5. Make the structured client's call lifecycle explicit + +`prepareCall` allocates a logical call, validates the output target, builds +the native request, installs two observer paths, captures prepared evidence, +and returns a closure for decoding and completion. That closure assigns +captured `err` from a named result and executes a reverse-order stack of +two finalizers before returning the mutated response. `Coordinate` also +finalizes a pending call on its separate undelivered-result path. Following an +error across both entry points requires keeping the prepared snapshot, native +terminal event, raw output, typed result, and logical call status in mind. +`Failure.Kind` and reason-based logical-call classification both describe +malformed-output failures. The ordinary request also retains a continuation +field whose current callers are adapter tests. [Call preparation and finalization](../../../internal/framework/llm/adapter.go#L54) +· [Coordinated call](../../../internal/framework/llm/coordinate.go#L15) +· [Request contract](../../../internal/framework/structuredoutput/call.go#L28) +· [Failure contract](../../../internal/framework/structuredoutput/failure.go#L17) + +Keep the two native entry points and make their shared finish operation +explicit. Move native request construction and result decoding out of the +bookkeeping closure. Have a concrete per-call record finalize observation, +execution counts, and response evidence exactly once on both success and +failure. Derive the existing diagnostic failure label from one classification +and assess whether the test-only continuation path still needs to be exposed. +Preserve terminal counts, preparation failures without a snapshot, and the +rule that an undelivered response supplies diagnostic evidence only. +[LLM runtime](../../internal/llm-runtime.md) + +### 6. Use a single proposal handoff and wire repair capability explicitly + +`GenerateCandidates` returns a correction slice, parallel segment references, +and a second enriched proposal slice with its own indexing inputs. +`ExecuteModuleProposal` forwards only the first two slices and the +exchange. The runner later builds its own enriched candidates, which is where +stage-wide index assignment takes place. Repair recovers the request-local +segment reference by the proposal's slice position or falls back to looking +up the resolved target. Separately, an enabled repair budget, proposal-exchange +capture, the module's repair prompt method, and the repair client's coordinator +capability are established at different boundaries. +[Proposal result and mapping](../../../internal/framework/proposal_generation/generate.go#L72) +· [Module handoff](../../../internal/framework/proposal_generation/module_proposal.go#L75) +· [Ordered index assignment](../../../internal/framework/runner/runner.go#L521) +· [Repair readiness at use](../../../internal/framework/runner/repair.go#L98) + +Carry a single proposal record with the resolved target and original local +reference, then assign the stage-wide index at the ordered runner boundary. +Assess retirement of the generator's separate enriched return once its +callers are migrated. Resolve repair availability with the concrete +module and role client at module setup. Use that capability to determine +exchange capture, and keep the disabled path simple. Preserve original +response order, invalid reference evidence, required profile readiness, +and zero-budget behavior. + +### 7. Give reporting one source for operation classification and counts + +`processreport.Build` has eleven work-unit and operation counters, nested +aggregation and projection, and a separate pass for run failures. +`SummaryOnly` bypasses selected detail paths inside that aggregation. +The LLM adapter decides recoverability through `structuredoutput.Failure`, +while `Build` uses a separate allowlist of recoverable reason codes after +the class has been discarded from the operation. Adding a recoverable +reason can therefore require coordinated edits to the adapter, domain +operation, and report counting. Safe report messages are a presentation +concern and can stay at that boundary. +[Snapshot aggregation](../../../internal/framework/processreport/report.go#L36) +· [Report-side classification](../../../internal/framework/processreport/report.go#L212) +· [Processing operation](../../../internal/framework/outcomes/outcomes.go#L84) +· [Adapter classification](../../../internal/framework/llm/adapter.go#L228) + +Give operation outcomes a single source for success and recoverability. +Accumulate work-unit facts and summary counts in one pass, then project +rows and detail records when requested. Keep presentation ordering and safe +messages explicit. Preserve unknown versus zero counts, a single recovery +per successful operation, and agreement among YAML, Markdown, and CLI counts. +[Ordinary report contract](../../integrations/output-files.md#summary-counts) + +### 8. Join correction evidence before building ledger rows + +`BuildCorrectionLedger` indexes decisions, repairs, and skips, then emits +applied, skipped, rejected, repaired, interrupted, and module-failure rows +from separate loops. Each candidate path repeats identity, replacement, +and decision projections. Precedence is implicit in iteration order, +the `seen` map, and checks for repair evidence. It is easy to add a field to +one row shape and miss the other cases. The ledger also looks up validator +class by name after execution, while the runner can use metadata supplied +by the validator itself. [Ledger assembly](../../../internal/framework/processreport/correction_ledger.go#L70) +· [Repair and interruption rows](../../../internal/framework/processreport/correction_ledger.go#L197) +· [Decision grouping](../../../internal/framework/processreport/correction_ledger.go#L247) +· [Runtime class selection](../../../internal/validators/metadata/metadata.go#L44) + +Gather the existing evidence into one per-proposal record keyed by module +instance and proposal index. Determine final disposition in one place, then +project common fields once and add the application and repair fields that +belong to that disposition. Consider recording the execution class used +when the decision is made, preserving built-in metadata. Keep module-only +failures distinct and retain the existing ordering, empty replacement fields, +partial decisions, and repaired skip reasons. +[Debug ledger contract](../../integrations/output-files.md#debug-only-files) + +### 9. Clarify output preparation, publication, and failure ownership + +The CLI builds initial and final report bundles and two failed-report +renderers. Preflight wraps the initial renderer to cover setup cancellation. +`OutputSession.Initialize` and `Finalize` each iterate required reports and +run a second write pass for failed status. File failure purposes are strings +interpreted again in the CLI. Destination selection also infers its role by +comparing the path string to the transcript destination, and preparation and +delivery each re-derive aspects of the path and file type. +[CLI preparation](../../../internal/cli/process_inputs.go#L70) +· [Preflight handoff](../../../internal/framework/preflight/preflight.go#L41) +· [Output selection](../../../internal/core/diagnostics/output_session.go#L61) +· [Publication and retry](../../../internal/core/diagnostics/output_session.go#L222) +· [Failure projection](../../../internal/cli/process_output.go#L92) + +Keep a checked destination with its explicit role and publication mode. +Reuse a report-write helper for both passes and make write or cleanup phases +typed facts of the output session. Keep report rendering and status in one CLI +handoff that can distinguish a valid failed snapshot from an unavailable one. +Preserve readiness before writes, intentional in-place transcript replacement, +required-output ordering, cleanup, and the bounded best-effort failed-report +pass. Related destination, cancellation, and phase defects are in the +correctness audit. [Output lifecycle](../../operations.md#outputs-and-lifecycle) +· [Correctness audit](01-correctness.md) + +### 10. Share root and per-module configuration rules + +`resolveSettings` repeats the same presence, null, inheritance, and fallback +logic for the validation and repair profile fields. Each root field also +has a separate fallback path after the stage loop. `validate` repeats the +LLM value checks at the root and at every stage, with a separately built +ConfigKit path for each. Repair and validation currently agree, but changing +one profile rule requires reading multiple versions of it. +[Profile resolution](../../../internal/core/config/resolve.go#L24) +· [Configuration validation](../../../internal/core/config/config.go#L143) + +Use a small typed helper for profile fallback that receives the correct +path, value, and dependency. Capture explicitly supplied root values before +recording root fallbacks, so a stage that selects a different proposal +profile still resolves correctly. Validate the common LLM settings with one +function taking a path prefix. Keep root-only constraints and readiness +separate, and preserve recorded origins, explicit null, and zero values. +[Configuration contract](../../config.md#settings) + +## Additional correctness findings + +### 11. A contradictory validator decision can advance an unavailable candidate + +The internal `Decision` accepts `Approved: true` together with +`DispositionUnavailable`. Cardinality checks validate only indices. With +one such decision and no other rejection for that candidate, the runner +treats it as approved in the shortlist path and leaves it unblocked in the +independent-feedback path. This violates the invariant that an unavailable +assessment cannot advance as approval. Bundled producers set matching +values. The defect is at the internal validator result boundary. Make the +disposition authoritative or reject contradictions before processing the +decision. [Decision and cardinality](../../../internal/framework/validators/models.go#L56) +· [Approval and blocking](../../../internal/framework/runner/runner.go#L769) +· [Validator guarantees](../../internal/validators.md#llm-validator-batching) + +### 12. The runner accepts an already canceled empty pipeline + +With valid inputs and no module specifications, `Run` constructs the chunk +inventory and returns success even when the parent context was canceled +before entry. Its cancellation check sits inside the module loop. The +processing CLI checks cancellation around the runner, so the gap is in the +runner contract. Check the context at entry and at the run's successful +completion boundary, including normalization-only invocations. +[Runner entry](../../../internal/framework/runner/runner.go#L121) +· [Runner success return](../../../internal/framework/runner/runner.go#L286) +· [CLI cancellation check](../../../internal/cli/process.go#L65) +· [Pipeline cancellation contract](../../internal/pipeline.md) + +### 13. Missing failed-report rendering can erase initialized evidence + +`preflight.Prepare` converts a nil `FailedReports` callback into a callable +function returning a zero-value `ReportBundle`. If reporting is enabled and +cancellation arrives after initialization, preflight passes that empty +bundle to `Finish`. `Finalize` writes its three nil payloads as empty files, +replacing the existing `in_progress` reports. The initialization retry can +similarly replace reports that were written successfully in its first pass. +The CLI supplies a failed renderer for its ordinary reporting path. At the +internal preflight seam, carry the absence of a failed bundle through to +publication or require a valid failed bundle before acquiring resources, +so the existing evidence survives when a failed snapshot cannot be rendered. +[Callback wrapping](../../../internal/framework/preflight/preflight.go#L68) +· [Initialization write retry](../../../internal/core/diagnostics/output_session.go#L222) +· [Final report write](../../../internal/core/diagnostics/output_session.go#L273)