323 lines
20 KiB
Markdown
323 lines
20 KiB
Markdown
# Maintainability and complexity audit
|
|
|
|
Reviewed source at `717d59c`. The codebase graph identified complex control
|
|
flow and cross-package call paths. A Go AST scan counted branches and
|
|
declaration spans for 353 functions and methods across 43 tracked Go
|
|
directories, excluding test files. The score below starts at one and adds
|
|
one per conditional, loop, non-default switch or select case, and short-circuit
|
|
operator, including those inside callbacks. The ranking guided source review.
|
|
Findings are based on static analysis.
|
|
|
|
## Architecture
|
|
|
|
Audita is a single-process Go CLI. CommandKit routes commands and streams.
|
|
ConfigKit resolves application settings, and the core packages parse,
|
|
normalize, and chunk transcripts and derive glossary selections. The
|
|
runner executes modules in order over fixed chunks. Deterministic and
|
|
LLM-backed validators gate proposals and revalidate repairs. PromptKit owns
|
|
model execution, backend admission, and bounded conversation history. Audita
|
|
applies approved edits and derives reports and debug artifacts from processing
|
|
outcomes. [Package map](../../internal/overview.md)
|
|
· [Pipeline](../../internal/pipeline.md)
|
|
· [Ownership rules](../../policy/architecture.md#ownership-and-dependency-direction)
|
|
|
|
The runner and validator runtime had the largest non-test Go source totals,
|
|
at 1,373 and 1,351 physical lines respectively. The highest-scoring functions
|
|
concentrated in orchestration, model integration, and report projection.
|
|
|
|
| Function | Branch score | Declaration lines |
|
|
| --- | ---: | ---: |
|
|
| `runner.repairCandidates` | 46 | 225 |
|
|
| `runner.runModulePipeline` | 36 | 212 |
|
|
| `processreport.Build` | 36 | 140 |
|
|
| `(*LLMBackedValidator).Validate` | 31 | 170 |
|
|
| `(*Runner).Run` | 30 | 168 |
|
|
| `runner.validateSectionCandidates` | 30 | 154 |
|
|
| `(*RoleClient).prepareCall` | 28 | 153 |
|
|
| `processreport.BuildCorrectionLedger` | 27 | 176 |
|
|
| `proposal_generation.GenerateCandidates` | 25 | 148 |
|
|
| `diagnostics.SelectOutput` | 21 | 66 |
|
|
|
|
## Refactoring opportunities
|
|
|
|
### 1. Give module execution one place to commit its result
|
|
|
|
`Run` owns the stage barrier, working transcript, inventory, module setup,
|
|
application, and partial-return assembly. The pipeline-error, application-error,
|
|
and success paths each construct a module result and copy work units into the
|
|
invocation inventory. Repair approvals are marked interrupted in both `Run`
|
|
and `runModulePipeline`. The latter also coordinates two worker lifetimes,
|
|
ordered proposal indexing, cancellation, and evidence aggregation. Changing
|
|
what an interrupted unit retains requires following both functions and their
|
|
callbacks. The earlier missing-ledger-candidate finding follows from indexing
|
|
and retention occurring only while scheduling validation. [Module commit and failure paths](../../../internal/framework/runner/runner.go#L187)
|
|
· [Section orchestration](../../../internal/framework/runner/runner.go#L441)
|
|
· [Interrupted evidence finding](01-correctness.md#10-a-completed-proposal-can-disappear-from-the-interruption-ledger)
|
|
|
|
Extract the section validation-and-repair worker from the collector. Let the
|
|
collector own ordered indexing and the evidence returned by workers, with
|
|
one path to register a fatal cause and cancel peers. Let one module-level
|
|
completion path resolve applied, skipped, and interrupted repair outcomes
|
|
and commit updated sections after successful application. Preserve validation
|
|
starting as ordered proposals become available, deterministic proposal IDs,
|
|
joined workers on fatal paths, and the stage barrier.
|
|
|
|
### 2. Separate one repair branch from the section's repair loop
|
|
|
|
`repairCandidates` selects candidates, builds each request, interprets
|
|
delivered turns, revalidates revisions, attributes outcomes, and writes debug
|
|
artifacts. Nested closures share mutable `current`, `localRef`, `findings`,
|
|
attempt counts, and section-level slices. For example, a successful revision
|
|
updates the branch, original proposal filtering, revalidation operations, and
|
|
approval list inside the coordinator callback. Error, withdrawal, exhaustion,
|
|
and accepted final-turn paths finish that work at different points.
|
|
[Branch setup](../../../internal/framework/runner/repair.go#L56)
|
|
· [Turn handling](../../../internal/framework/runner/repair.go#L146)
|
|
· [Repair behavior](../../internal/modules.md#failure-and-outcome-behavior)
|
|
|
|
Keep the outer loop responsible for selecting and collecting branches. Have
|
|
one branch-local operation return the terminal outcome, optional approved
|
|
revision, and operation evidence. Separate construction of the next request,
|
|
assessment of a delivered turn, and optional attempt recording. Define the
|
|
branch's internal states once so application and reports can project their
|
|
own stable labels. Keep PromptKit responsible for history and turn limits.
|
|
Retain final-turn revalidation, target changes within the section, and
|
|
independence of sibling approvals.
|
|
|
|
### 3. Put validator applicability and decision meaning at their owning boundary
|
|
|
|
`feedbackCandidates` switches on validator names and module keys to decide
|
|
whether each validator has usable evidence. The validator implementations
|
|
also check previews and module policy. The runner then rewrites an
|
|
`original_text_presence` approval if its preview is ambiguous, so the same
|
|
validator's returned verdict can mean something different outside the runner.
|
|
`validateSectionCandidates` interleaves those checks with two eligibility
|
|
modes, cardinality checks, filtering, and evidence recording. The framework
|
|
also declares identical validator interfaces under `contracts` and
|
|
`validators`. [Feedback selection](../../../internal/framework/runner/feedback.go#L13)
|
|
· [Runner verdict handling](../../../internal/framework/runner/runner.go#L680)
|
|
· [Original-text validator](../../../internal/framework/validators/deterministic.go#L37)
|
|
· [Validator contract](../../../internal/framework/contracts/contracts.go#L29)
|
|
· [Duplicate interface](../../../internal/framework/validators/models.go#L95)
|
|
|
|
Put each check's applicability and verdict together in its validator or its
|
|
built-in registration. Use one validator interface and one internal decision
|
|
disposition, deriving serialized approval from it. Keep the runner responsible
|
|
for submitting the shortlist or collecting independent findings, checking
|
|
returned identities, and applying only fully approved candidates. Preserve
|
|
ordered chains, zero-budget shortlisting, unavailable decisions, and separate
|
|
revalidation of revisions. The earlier audit records the span-only
|
|
protection mismatch. [Protection correctness](01-correctness.md#8-span-only-protection-can-reject-a-valid-edit-or-miss-a-repair-opportunity)
|
|
|
|
### 4. Separate validation-batch assessment from the diagnostic writer
|
|
|
|
`LLMBackedValidator.Validate` interleaves request preparation, prompt selection,
|
|
batch dispatch, reconciliation, partial-result assembly, and debug payload
|
|
construction. Its nested writer consumes the original request, local
|
|
reference mapping, raw response, reconciliation result, and failure. Proposal
|
|
generation and validation declare writer interfaces with matching signatures
|
|
and duplicate four-field artifact types. Their adapter wrappers copy each
|
|
path from `llm.InteractionArtifacts`. Repair writes through the same concrete
|
|
writer directly. A change to the common artifact envelope or safe error
|
|
serialization therefore touches several packages.
|
|
[Batch loop](../../../internal/framework/validators/llm_validators.go#L112)
|
|
· [Proposal writer contract](../../../internal/framework/proposal_generation/generate.go#L23)
|
|
· [Validation writer contract](../../../internal/framework/validators/models.go#L84)
|
|
· [Writer adapter](../../../internal/framework/runner/runner.go#L994)
|
|
· [Repair artifact capture](../../../internal/framework/runner/repair.go#L165)
|
|
|
|
Give one batch operation responsibility for dispatch and assessment. Return
|
|
reconciled decisions, its compact outcome, and only the evidence needed by the
|
|
diagnostic path. Reuse one internal artifact value type and writer seam for
|
|
common metadata, redaction, and safe write failures. Keep the per-role
|
|
payload construction with its domain owner. Preserve local-reference
|
|
reconciliation, missing and duplicate decision handling, partial results on
|
|
fatal return, and nonfatal debug-write warnings.
|
|
|
|
### 5. Make the structured client's call lifecycle explicit
|
|
|
|
`prepareCall` allocates a logical call, validates the output target, builds
|
|
the native request, installs two observer paths, captures prepared evidence,
|
|
and returns a closure for decoding and completion. That closure assigns
|
|
captured `err` from a named result and executes a reverse-order stack of
|
|
two finalizers before returning the mutated response. `Coordinate` also
|
|
finalizes a pending call on its separate undelivered-result path. Following an
|
|
error across both entry points requires keeping the prepared snapshot, native
|
|
terminal event, raw output, typed result, and logical call status in mind.
|
|
`Failure.Kind` and reason-based logical-call classification both describe
|
|
malformed-output failures. The ordinary request also retains a continuation
|
|
field whose current callers are adapter tests. [Call preparation and finalization](../../../internal/framework/llm/adapter.go#L54)
|
|
· [Coordinated call](../../../internal/framework/llm/coordinate.go#L15)
|
|
· [Request contract](../../../internal/framework/structuredoutput/call.go#L28)
|
|
· [Failure contract](../../../internal/framework/structuredoutput/failure.go#L17)
|
|
|
|
Keep the two native entry points and make their shared finish operation
|
|
explicit. Move native request construction and result decoding out of the
|
|
bookkeeping closure. Have a concrete per-call record finalize observation,
|
|
execution counts, and response evidence exactly once on both success and
|
|
failure. Derive the existing diagnostic failure label from one classification
|
|
and assess whether the test-only continuation path still needs to be exposed.
|
|
Preserve terminal counts, preparation failures without a snapshot, and the
|
|
rule that an undelivered response supplies diagnostic evidence only.
|
|
[LLM runtime](../../internal/llm-runtime.md)
|
|
|
|
### 6. Use a single proposal handoff and wire repair capability explicitly
|
|
|
|
`GenerateCandidates` returns a correction slice, parallel segment references,
|
|
and a second enriched proposal slice with its own indexing inputs.
|
|
`ExecuteModuleProposal` forwards only the first two slices and the
|
|
exchange. The runner later builds its own enriched candidates, which is where
|
|
stage-wide index assignment takes place. Repair recovers the request-local
|
|
segment reference by the proposal's slice position or falls back to looking
|
|
up the resolved target. Separately, an enabled repair budget, proposal-exchange
|
|
capture, the module's repair prompt method, and the repair client's coordinator
|
|
capability are established at different boundaries.
|
|
[Proposal result and mapping](../../../internal/framework/proposal_generation/generate.go#L72)
|
|
· [Module handoff](../../../internal/framework/proposal_generation/module_proposal.go#L75)
|
|
· [Ordered index assignment](../../../internal/framework/runner/runner.go#L521)
|
|
· [Repair readiness at use](../../../internal/framework/runner/repair.go#L98)
|
|
|
|
Carry a single proposal record with the resolved target and original local
|
|
reference, then assign the stage-wide index at the ordered runner boundary.
|
|
Assess retirement of the generator's separate enriched return once its
|
|
callers are migrated. Resolve repair availability with the concrete
|
|
module and role client at module setup. Use that capability to determine
|
|
exchange capture, and keep the disabled path simple. Preserve original
|
|
response order, invalid reference evidence, required profile readiness,
|
|
and zero-budget behavior.
|
|
|
|
### 7. Give reporting one source for operation classification and counts
|
|
|
|
`processreport.Build` has eleven work-unit and operation counters, nested
|
|
aggregation and projection, and a separate pass for run failures.
|
|
`SummaryOnly` bypasses selected detail paths inside that aggregation.
|
|
The LLM adapter decides recoverability through `structuredoutput.Failure`,
|
|
while `Build` uses a separate allowlist of recoverable reason codes after
|
|
the class has been discarded from the operation. Adding a recoverable
|
|
reason can therefore require coordinated edits to the adapter, domain
|
|
operation, and report counting. Safe report messages are a presentation
|
|
concern and can stay at that boundary.
|
|
[Snapshot aggregation](../../../internal/framework/processreport/report.go#L36)
|
|
· [Report-side classification](../../../internal/framework/processreport/report.go#L212)
|
|
· [Processing operation](../../../internal/framework/outcomes/outcomes.go#L84)
|
|
· [Adapter classification](../../../internal/framework/llm/adapter.go#L228)
|
|
|
|
Give operation outcomes a single source for success and recoverability.
|
|
Accumulate work-unit facts and summary counts in one pass, then project
|
|
rows and detail records when requested. Keep presentation ordering and safe
|
|
messages explicit. Preserve unknown versus zero counts, a single recovery
|
|
per successful operation, and agreement among YAML, Markdown, and CLI counts.
|
|
[Ordinary report contract](../../integrations/output-files.md#summary-counts)
|
|
|
|
### 8. Join correction evidence before building ledger rows
|
|
|
|
`BuildCorrectionLedger` indexes decisions, repairs, and skips, then emits
|
|
applied, skipped, rejected, repaired, interrupted, and module-failure rows
|
|
from separate loops. Each candidate path repeats identity, replacement,
|
|
and decision projections. Precedence is implicit in iteration order,
|
|
the `seen` map, and checks for repair evidence. It is easy to add a field to
|
|
one row shape and miss the other cases. The ledger also looks up validator
|
|
class by name after execution, while the runner can use metadata supplied
|
|
by the validator itself. [Ledger assembly](../../../internal/framework/processreport/correction_ledger.go#L70)
|
|
· [Repair and interruption rows](../../../internal/framework/processreport/correction_ledger.go#L197)
|
|
· [Decision grouping](../../../internal/framework/processreport/correction_ledger.go#L247)
|
|
· [Runtime class selection](../../../internal/validators/metadata/metadata.go#L44)
|
|
|
|
Gather the existing evidence into one per-proposal record keyed by module
|
|
instance and proposal index. Determine final disposition in one place, then
|
|
project common fields once and add the application and repair fields that
|
|
belong to that disposition. Consider recording the execution class used
|
|
when the decision is made, preserving built-in metadata. Keep module-only
|
|
failures distinct and retain the existing ordering, empty replacement fields,
|
|
partial decisions, and repaired skip reasons.
|
|
[Debug ledger contract](../../integrations/output-files.md#debug-only-files)
|
|
|
|
### 9. Clarify output preparation, publication, and failure ownership
|
|
|
|
The CLI builds initial and final report bundles and two failed-report
|
|
renderers. Preflight wraps the initial renderer to cover setup cancellation.
|
|
`OutputSession.Initialize` and `Finalize` each iterate required reports and
|
|
run a second write pass for failed status. File failure purposes are strings
|
|
interpreted again in the CLI. Destination selection also infers its role by
|
|
comparing the path string to the transcript destination, and preparation and
|
|
delivery each re-derive aspects of the path and file type.
|
|
[CLI preparation](../../../internal/cli/process_inputs.go#L70)
|
|
· [Preflight handoff](../../../internal/framework/preflight/preflight.go#L41)
|
|
· [Output selection](../../../internal/core/diagnostics/output_session.go#L61)
|
|
· [Publication and retry](../../../internal/core/diagnostics/output_session.go#L222)
|
|
· [Failure projection](../../../internal/cli/process_output.go#L92)
|
|
|
|
Keep a checked destination with its explicit role and publication mode.
|
|
Reuse a report-write helper for both passes and make write or cleanup phases
|
|
typed facts of the output session. Keep report rendering and status in one CLI
|
|
handoff that can distinguish a valid failed snapshot from an unavailable one.
|
|
Preserve readiness before writes, intentional in-place transcript replacement,
|
|
required-output ordering, cleanup, and the bounded best-effort failed-report
|
|
pass. Related destination, cancellation, and phase defects are in the
|
|
correctness audit. [Output lifecycle](../../operations.md#outputs-and-lifecycle)
|
|
· [Correctness audit](01-correctness.md)
|
|
|
|
### 10. Share root and per-module configuration rules
|
|
|
|
`resolveSettings` repeats the same presence, null, inheritance, and fallback
|
|
logic for the validation and repair profile fields. Each root field also
|
|
has a separate fallback path after the stage loop. `validate` repeats the
|
|
LLM value checks at the root and at every stage, with a separately built
|
|
ConfigKit path for each. Repair and validation currently agree, but changing
|
|
one profile rule requires reading multiple versions of it.
|
|
[Profile resolution](../../../internal/core/config/resolve.go#L24)
|
|
· [Configuration validation](../../../internal/core/config/config.go#L143)
|
|
|
|
Use a small typed helper for profile fallback that receives the correct
|
|
path, value, and dependency. Capture explicitly supplied root values before
|
|
recording root fallbacks, so a stage that selects a different proposal
|
|
profile still resolves correctly. Validate the common LLM settings with one
|
|
function taking a path prefix. Keep root-only constraints and readiness
|
|
separate, and preserve recorded origins, explicit null, and zero values.
|
|
[Configuration contract](../../config.md#settings)
|
|
|
|
## Additional correctness findings
|
|
|
|
### 11. A contradictory validator decision can advance an unavailable candidate
|
|
|
|
The internal `Decision` accepts `Approved: true` together with
|
|
`DispositionUnavailable`. Cardinality checks validate only indices. With
|
|
one such decision and no other rejection for that candidate, the runner
|
|
treats it as approved in the shortlist path and leaves it unblocked in the
|
|
independent-feedback path. This violates the invariant that an unavailable
|
|
assessment cannot advance as approval. Bundled producers set matching
|
|
values. The defect is at the internal validator result boundary. Make the
|
|
disposition authoritative or reject contradictions before processing the
|
|
decision. [Decision and cardinality](../../../internal/framework/validators/models.go#L56)
|
|
· [Approval and blocking](../../../internal/framework/runner/runner.go#L769)
|
|
· [Validator guarantees](../../internal/validators.md#llm-validator-batching)
|
|
|
|
### 12. The runner accepts an already canceled empty pipeline
|
|
|
|
With valid inputs and no module specifications, `Run` constructs the chunk
|
|
inventory and returns success even when the parent context was canceled
|
|
before entry. Its cancellation check sits inside the module loop. The
|
|
processing CLI checks cancellation around the runner, so the gap is in the
|
|
runner contract. Check the context at entry and at the run's successful
|
|
completion boundary, including normalization-only invocations.
|
|
[Runner entry](../../../internal/framework/runner/runner.go#L121)
|
|
· [Runner success return](../../../internal/framework/runner/runner.go#L286)
|
|
· [CLI cancellation check](../../../internal/cli/process.go#L65)
|
|
· [Pipeline cancellation contract](../../internal/pipeline.md)
|
|
|
|
### 13. Missing failed-report rendering can erase initialized evidence
|
|
|
|
`preflight.Prepare` converts a nil `FailedReports` callback into a callable
|
|
function returning a zero-value `ReportBundle`. If reporting is enabled and
|
|
cancellation arrives after initialization, preflight passes that empty
|
|
bundle to `Finish`. `Finalize` writes its three nil payloads as empty files,
|
|
replacing the existing `in_progress` reports. The initialization retry can
|
|
similarly replace reports that were written successfully in its first pass.
|
|
The CLI supplies a failed renderer for its ordinary reporting path. At the
|
|
internal preflight seam, carry the absence of a failed bundle through to
|
|
publication or require a valid failed bundle before acquiring resources,
|
|
so the existing evidence survives when a failed snapshot cannot be rendered.
|
|
[Callback wrapping](../../../internal/framework/preflight/preflight.go#L68)
|
|
· [Initialization write retry](../../../internal/core/diagnostics/output_session.go#L222)
|
|
· [Final report write](../../../internal/core/diagnostics/output_session.go#L273)
|