Files
audita/docs/roadmap/audit/03-complexity.md

323 lines
20 KiB
Markdown
Raw Normal View History

# Maintainability and complexity audit
Reviewed source at `717d59c`. The codebase graph identified complex control
flow and cross-package call paths. A Go AST scan counted branches and
declaration spans for 353 functions and methods across 43 tracked Go
directories, excluding test files. The score below starts at one and adds
one per conditional, loop, non-default switch or select case, and short-circuit
operator, including those inside callbacks. The ranking guided source review.
Findings are based on static analysis.
## Architecture
Audita is a single-process Go CLI. CommandKit routes commands and streams.
ConfigKit resolves application settings, and the core packages parse,
normalize, and chunk transcripts and derive glossary selections. The
runner executes modules in order over fixed chunks. Deterministic and
LLM-backed validators gate proposals and revalidate repairs. PromptKit owns
model execution, backend admission, and bounded conversation history. Audita
applies approved edits and derives reports and debug artifacts from processing
outcomes. [Package map](../../internal/overview.md)
· [Pipeline](../../internal/pipeline.md)
· [Ownership rules](../../policy/architecture.md#ownership-and-dependency-direction)
The runner and validator runtime had the largest non-test Go source totals,
at 1,373 and 1,351 physical lines respectively. The highest-scoring functions
concentrated in orchestration, model integration, and report projection.
| Function | Branch score | Declaration lines |
| --- | ---: | ---: |
| `runner.repairCandidates` | 46 | 225 |
| `runner.runModulePipeline` | 36 | 212 |
| `processreport.Build` | 36 | 140 |
| `(*LLMBackedValidator).Validate` | 31 | 170 |
| `(*Runner).Run` | 30 | 168 |
| `runner.validateSectionCandidates` | 30 | 154 |
| `(*RoleClient).prepareCall` | 28 | 153 |
| `processreport.BuildCorrectionLedger` | 27 | 176 |
| `proposal_generation.GenerateCandidates` | 25 | 148 |
| `diagnostics.SelectOutput` | 21 | 66 |
## Refactoring opportunities
### 1. Give module execution one place to commit its result
`Run` owns the stage barrier, working transcript, inventory, module setup,
application, and partial-return assembly. The pipeline-error, application-error,
and success paths each construct a module result and copy work units into the
invocation inventory. Repair approvals are marked interrupted in both `Run`
and `runModulePipeline`. The latter also coordinates two worker lifetimes,
ordered proposal indexing, cancellation, and evidence aggregation. Changing
what an interrupted unit retains requires following both functions and their
callbacks. The earlier missing-ledger-candidate finding follows from indexing
and retention occurring only while scheduling validation. [Module commit and failure paths](../../../internal/framework/runner/runner.go#L187)
· [Section orchestration](../../../internal/framework/runner/runner.go#L441)
· [Interrupted evidence finding](01-correctness.md#10-a-completed-proposal-can-disappear-from-the-interruption-ledger)
Extract the section validation-and-repair worker from the collector. Let the
collector own ordered indexing and the evidence returned by workers, with
one path to register a fatal cause and cancel peers. Let one module-level
completion path resolve applied, skipped, and interrupted repair outcomes
and commit updated sections after successful application. Preserve validation
starting as ordered proposals become available, deterministic proposal IDs,
joined workers on fatal paths, and the stage barrier.
### 2. Separate one repair branch from the section's repair loop
`repairCandidates` selects candidates, builds each request, interprets
delivered turns, revalidates revisions, attributes outcomes, and writes debug
artifacts. Nested closures share mutable `current`, `localRef`, `findings`,
attempt counts, and section-level slices. For example, a successful revision
updates the branch, original proposal filtering, revalidation operations, and
approval list inside the coordinator callback. Error, withdrawal, exhaustion,
and accepted final-turn paths finish that work at different points.
[Branch setup](../../../internal/framework/runner/repair.go#L56)
· [Turn handling](../../../internal/framework/runner/repair.go#L146)
· [Repair behavior](../../internal/modules.md#failure-and-outcome-behavior)
Keep the outer loop responsible for selecting and collecting branches. Have
one branch-local operation return the terminal outcome, optional approved
revision, and operation evidence. Separate construction of the next request,
assessment of a delivered turn, and optional attempt recording. Define the
branch's internal states once so application and reports can project their
own stable labels. Keep PromptKit responsible for history and turn limits.
Retain final-turn revalidation, target changes within the section, and
independence of sibling approvals.
### 3. Put validator applicability and decision meaning at their owning boundary
`feedbackCandidates` switches on validator names and module keys to decide
whether each validator has usable evidence. The validator implementations
also check previews and module policy. The runner then rewrites an
`original_text_presence` approval if its preview is ambiguous, so the same
validator's returned verdict can mean something different outside the runner.
`validateSectionCandidates` interleaves those checks with two eligibility
modes, cardinality checks, filtering, and evidence recording. The framework
also declares identical validator interfaces under `contracts` and
`validators`. [Feedback selection](../../../internal/framework/runner/feedback.go#L13)
· [Runner verdict handling](../../../internal/framework/runner/runner.go#L680)
· [Original-text validator](../../../internal/framework/validators/deterministic.go#L37)
· [Validator contract](../../../internal/framework/contracts/contracts.go#L29)
· [Duplicate interface](../../../internal/framework/validators/models.go#L95)
Put each check's applicability and verdict together in its validator or its
built-in registration. Use one validator interface and one internal decision
disposition, deriving serialized approval from it. Keep the runner responsible
for submitting the shortlist or collecting independent findings, checking
returned identities, and applying only fully approved candidates. Preserve
ordered chains, zero-budget shortlisting, unavailable decisions, and separate
revalidation of revisions. The earlier audit records the span-only
protection mismatch. [Protection correctness](01-correctness.md#8-span-only-protection-can-reject-a-valid-edit-or-miss-a-repair-opportunity)
### 4. Separate validation-batch assessment from the diagnostic writer
`LLMBackedValidator.Validate` interleaves request preparation, prompt selection,
batch dispatch, reconciliation, partial-result assembly, and debug payload
construction. Its nested writer consumes the original request, local
reference mapping, raw response, reconciliation result, and failure. Proposal
generation and validation declare writer interfaces with matching signatures
and duplicate four-field artifact types. Their adapter wrappers copy each
path from `llm.InteractionArtifacts`. Repair writes through the same concrete
writer directly. A change to the common artifact envelope or safe error
serialization therefore touches several packages.
[Batch loop](../../../internal/framework/validators/llm_validators.go#L112)
· [Proposal writer contract](../../../internal/framework/proposal_generation/generate.go#L23)
· [Validation writer contract](../../../internal/framework/validators/models.go#L84)
· [Writer adapter](../../../internal/framework/runner/runner.go#L994)
· [Repair artifact capture](../../../internal/framework/runner/repair.go#L165)
Give one batch operation responsibility for dispatch and assessment. Return
reconciled decisions, its compact outcome, and only the evidence needed by the
diagnostic path. Reuse one internal artifact value type and writer seam for
common metadata, redaction, and safe write failures. Keep the per-role
payload construction with its domain owner. Preserve local-reference
reconciliation, missing and duplicate decision handling, partial results on
fatal return, and nonfatal debug-write warnings.
### 5. Make the structured client's call lifecycle explicit
`prepareCall` allocates a logical call, validates the output target, builds
the native request, installs two observer paths, captures prepared evidence,
and returns a closure for decoding and completion. That closure assigns
captured `err` from a named result and executes a reverse-order stack of
two finalizers before returning the mutated response. `Coordinate` also
finalizes a pending call on its separate undelivered-result path. Following an
error across both entry points requires keeping the prepared snapshot, native
terminal event, raw output, typed result, and logical call status in mind.
`Failure.Kind` and reason-based logical-call classification both describe
malformed-output failures. The ordinary request also retains a continuation
field whose current callers are adapter tests. [Call preparation and finalization](../../../internal/framework/llm/adapter.go#L54)
· [Coordinated call](../../../internal/framework/llm/coordinate.go#L15)
· [Request contract](../../../internal/framework/structuredoutput/call.go#L28)
· [Failure contract](../../../internal/framework/structuredoutput/failure.go#L17)
Keep the two native entry points and make their shared finish operation
explicit. Move native request construction and result decoding out of the
bookkeeping closure. Have a concrete per-call record finalize observation,
execution counts, and response evidence exactly once on both success and
failure. Derive the existing diagnostic failure label from one classification
and assess whether the test-only continuation path still needs to be exposed.
Preserve terminal counts, preparation failures without a snapshot, and the
rule that an undelivered response supplies diagnostic evidence only.
[LLM runtime](../../internal/llm-runtime.md)
### 6. Use a single proposal handoff and wire repair capability explicitly
`GenerateCandidates` returns a correction slice, parallel segment references,
and a second enriched proposal slice with its own indexing inputs.
`ExecuteModuleProposal` forwards only the first two slices and the
exchange. The runner later builds its own enriched candidates, which is where
stage-wide index assignment takes place. Repair recovers the request-local
segment reference by the proposal's slice position or falls back to looking
up the resolved target. Separately, an enabled repair budget, proposal-exchange
capture, the module's repair prompt method, and the repair client's coordinator
capability are established at different boundaries.
[Proposal result and mapping](../../../internal/framework/proposal_generation/generate.go#L72)
· [Module handoff](../../../internal/framework/proposal_generation/module_proposal.go#L75)
· [Ordered index assignment](../../../internal/framework/runner/runner.go#L521)
· [Repair readiness at use](../../../internal/framework/runner/repair.go#L98)
Carry a single proposal record with the resolved target and original local
reference, then assign the stage-wide index at the ordered runner boundary.
Assess retirement of the generator's separate enriched return once its
callers are migrated. Resolve repair availability with the concrete
module and role client at module setup. Use that capability to determine
exchange capture, and keep the disabled path simple. Preserve original
response order, invalid reference evidence, required profile readiness,
and zero-budget behavior.
### 7. Give reporting one source for operation classification and counts
`processreport.Build` has eleven work-unit and operation counters, nested
aggregation and projection, and a separate pass for run failures.
`SummaryOnly` bypasses selected detail paths inside that aggregation.
The LLM adapter decides recoverability through `structuredoutput.Failure`,
while `Build` uses a separate allowlist of recoverable reason codes after
the class has been discarded from the operation. Adding a recoverable
reason can therefore require coordinated edits to the adapter, domain
operation, and report counting. Safe report messages are a presentation
concern and can stay at that boundary.
[Snapshot aggregation](../../../internal/framework/processreport/report.go#L36)
· [Report-side classification](../../../internal/framework/processreport/report.go#L212)
· [Processing operation](../../../internal/framework/outcomes/outcomes.go#L84)
· [Adapter classification](../../../internal/framework/llm/adapter.go#L228)
Give operation outcomes a single source for success and recoverability.
Accumulate work-unit facts and summary counts in one pass, then project
rows and detail records when requested. Keep presentation ordering and safe
messages explicit. Preserve unknown versus zero counts, a single recovery
per successful operation, and agreement among YAML, Markdown, and CLI counts.
[Ordinary report contract](../../integrations/output-files.md#summary-counts)
### 8. Join correction evidence before building ledger rows
`BuildCorrectionLedger` indexes decisions, repairs, and skips, then emits
applied, skipped, rejected, repaired, interrupted, and module-failure rows
from separate loops. Each candidate path repeats identity, replacement,
and decision projections. Precedence is implicit in iteration order,
the `seen` map, and checks for repair evidence. It is easy to add a field to
one row shape and miss the other cases. The ledger also looks up validator
class by name after execution, while the runner can use metadata supplied
by the validator itself. [Ledger assembly](../../../internal/framework/processreport/correction_ledger.go#L70)
· [Repair and interruption rows](../../../internal/framework/processreport/correction_ledger.go#L197)
· [Decision grouping](../../../internal/framework/processreport/correction_ledger.go#L247)
· [Runtime class selection](../../../internal/validators/metadata/metadata.go#L44)
Gather the existing evidence into one per-proposal record keyed by module
instance and proposal index. Determine final disposition in one place, then
project common fields once and add the application and repair fields that
belong to that disposition. Consider recording the execution class used
when the decision is made, preserving built-in metadata. Keep module-only
failures distinct and retain the existing ordering, empty replacement fields,
partial decisions, and repaired skip reasons.
[Debug ledger contract](../../integrations/output-files.md#debug-only-files)
### 9. Clarify output preparation, publication, and failure ownership
The CLI builds initial and final report bundles and two failed-report
renderers. Preflight wraps the initial renderer to cover setup cancellation.
`OutputSession.Initialize` and `Finalize` each iterate required reports and
run a second write pass for failed status. File failure purposes are strings
interpreted again in the CLI. Destination selection also infers its role by
comparing the path string to the transcript destination, and preparation and
delivery each re-derive aspects of the path and file type.
[CLI preparation](../../../internal/cli/process_inputs.go#L70)
· [Preflight handoff](../../../internal/framework/preflight/preflight.go#L41)
· [Output selection](../../../internal/core/diagnostics/output_session.go#L61)
· [Publication and retry](../../../internal/core/diagnostics/output_session.go#L222)
· [Failure projection](../../../internal/cli/process_output.go#L92)
Keep a checked destination with its explicit role and publication mode.
Reuse a report-write helper for both passes and make write or cleanup phases
typed facts of the output session. Keep report rendering and status in one CLI
handoff that can distinguish a valid failed snapshot from an unavailable one.
Preserve readiness before writes, intentional in-place transcript replacement,
required-output ordering, cleanup, and the bounded best-effort failed-report
pass. Related destination, cancellation, and phase defects are in the
correctness audit. [Output lifecycle](../../operations.md#outputs-and-lifecycle)
· [Correctness audit](01-correctness.md)
### 10. Share root and per-module configuration rules
`resolveSettings` repeats the same presence, null, inheritance, and fallback
logic for the validation and repair profile fields. Each root field also
has a separate fallback path after the stage loop. `validate` repeats the
LLM value checks at the root and at every stage, with a separately built
ConfigKit path for each. Repair and validation currently agree, but changing
one profile rule requires reading multiple versions of it.
[Profile resolution](../../../internal/core/config/resolve.go#L24)
· [Configuration validation](../../../internal/core/config/config.go#L143)
Use a small typed helper for profile fallback that receives the correct
path, value, and dependency. Capture explicitly supplied root values before
recording root fallbacks, so a stage that selects a different proposal
profile still resolves correctly. Validate the common LLM settings with one
function taking a path prefix. Keep root-only constraints and readiness
separate, and preserve recorded origins, explicit null, and zero values.
[Configuration contract](../../config.md#settings)
## Additional correctness findings
### 11. A contradictory validator decision can advance an unavailable candidate
The internal `Decision` accepts `Approved: true` together with
`DispositionUnavailable`. Cardinality checks validate only indices. With
one such decision and no other rejection for that candidate, the runner
treats it as approved in the shortlist path and leaves it unblocked in the
independent-feedback path. This violates the invariant that an unavailable
assessment cannot advance as approval. Bundled producers set matching
values. The defect is at the internal validator result boundary. Make the
disposition authoritative or reject contradictions before processing the
decision. [Decision and cardinality](../../../internal/framework/validators/models.go#L56)
· [Approval and blocking](../../../internal/framework/runner/runner.go#L769)
· [Validator guarantees](../../internal/validators.md#llm-validator-batching)
### 12. The runner accepts an already canceled empty pipeline
With valid inputs and no module specifications, `Run` constructs the chunk
inventory and returns success even when the parent context was canceled
before entry. Its cancellation check sits inside the module loop. The
processing CLI checks cancellation around the runner, so the gap is in the
runner contract. Check the context at entry and at the run's successful
completion boundary, including normalization-only invocations.
[Runner entry](../../../internal/framework/runner/runner.go#L121)
· [Runner success return](../../../internal/framework/runner/runner.go#L286)
· [CLI cancellation check](../../../internal/cli/process.go#L65)
· [Pipeline cancellation contract](../../internal/pipeline.md)
### 13. Missing failed-report rendering can erase initialized evidence
`preflight.Prepare` converts a nil `FailedReports` callback into a callable
function returning a zero-value `ReportBundle`. If reporting is enabled and
cancellation arrives after initialization, preflight passes that empty
bundle to `Finish`. `Finalize` writes its three nil payloads as empty files,
replacing the existing `in_progress` reports. The initialization retry can
similarly replace reports that were written successfully in its first pass.
The CLI supplies a failed renderer for its ordinary reporting path. At the
internal preflight seam, carry the absence of a failed bundle through to
publication or require a valid failed bundle before acquiring resources,
so the existing evidence survives when a failed snapshot cannot be rendered.
[Callback wrapping](../../../internal/framework/preflight/preflight.go#L68)
· [Initialization write retry](../../../internal/core/diagnostics/output_session.go#L222)
· [Final report write](../../../internal/core/diagnostics/output_session.go#L273)