264 lines
18 KiB
Markdown
264 lines
18 KiB
Markdown
# Package cohesion and architectural boundaries audit
|
|
|
|
Reviewed at `0f34231`. Audita is a single-process Go CLI. `internal/cli`
|
|
assembles configuration, inputs, model clients, and output destinations. Core
|
|
packages provide deterministic transcript, glossary, and configuration
|
|
operations. The framework runner coordinates module proposals, validators,
|
|
repair, and application. PromptKit integration lives in `framework/llm`.
|
|
Processing facts feed ordinary reports and opt-in debug artifacts.
|
|
[Package map](../../internal/overview.md) · [Pipeline](../../internal/pipeline.md)
|
|
|
|
The audit used the code graph for discovery and call tracing, the `go list`
|
|
import graph to check dependency direction, and source and tests to check what
|
|
crosses each boundary. The import inventory covered 42 `internal` packages,
|
|
including test support. No core package directly imports a repository package
|
|
outside `internal/core`. The framework imports concrete correction modules
|
|
through its production factory and built-in validator metadata through its
|
|
runner and ledger. The findings focus on the responsibilities behind those
|
|
dependencies and on implicit coupling that the import graph cannot show.
|
|
Third-party implementations were outside the review. The existing suite
|
|
passed with `go test -count=1 ./...` on Go 1.27.3.
|
|
[Ownership policy](../../policy/architecture.md#ownership-and-dependency-direction)
|
|
|
|
## 1. Give the validator runtime interface one owner
|
|
|
|
`framework/contracts` declares the interface modules expose to the runner, but
|
|
aliases its request to `framework/validators.Request` and returns that package's
|
|
result. `framework/validators` declares a second interface with the same method
|
|
set. Its request ties together candidate evidence, resolved application
|
|
settings, the LLM client, the glossary matching cache, and diagnostic writer.
|
|
The implementation package also contains the built-in deterministic rule bodies
|
|
and a switch selecting built-in validator prompts. The named validator packages
|
|
mostly construct or wrap those implementations. As a result, a module that
|
|
only needs the validator port depends on the package that implements budgeting,
|
|
model reconciliation, and concrete rule policy.
|
|
[Module and validator ports](../../../internal/framework/contracts/contracts.go#L21) ·
|
|
[Validator request and result](../../../internal/framework/validators/models.go#L30) ·
|
|
[Duplicate interface](../../../internal/framework/validators/models.go#L95) ·
|
|
[Built-in check implementation](../../../internal/framework/validators/deterministic.go#L15) ·
|
|
[Built-in prompt selection](../../../internal/framework/validators/llm_validators.go#L222) ·
|
|
[Editorial constructor](../../../internal/validators/editorial_review/validator.go#L11)
|
|
|
|
Choose one shared owner for the port, decision types, and common cardinality
|
|
invariant. Give validators the evidence and collaborators they need through
|
|
that boundary. Keep reusable batch construction and reconciliation in the
|
|
framework, with concrete prompt selection and editorial rules owned by the
|
|
built-in validators. A small first change would establish one interface and
|
|
remove redundant aliases before narrowing the request. Preserve deterministic
|
|
ordering and the separate full and partial cardinality rules exercised by the
|
|
runner and validator tests. [Interface test](../../../internal/framework/contracts/contracts_test.go#L64) ·
|
|
[Cardinality checks](../../../internal/framework/validators/models.go#L101) ·
|
|
[Validator ownership](../../internal/validators.md#ownership)
|
|
|
|
## 2. Preserve the execution class assigned to each validator decision
|
|
|
|
The metadata package provides both a registry keyed by validator name and an
|
|
instance-level execution class. The runner uses `ClassOf`, which prefers a
|
|
valid explicit class, to order calls and decide which calls receive LLM
|
|
interaction diagnostics. The decision record carries the name but omits the
|
|
class. The correction ledger calls `ClassForKey` and buckets the record into
|
|
deterministic or LLM results. An injected validator with a valid class that
|
|
differs from its name's default can therefore execute under one classification
|
|
and appear under another. The built-in registry test checks that the two agree
|
|
for built-ins. The instance-override and ledger tests exercise their respective
|
|
inputs independently. [Class resolution](../../../internal/validators/metadata/metadata.go#L39) ·
|
|
[Runtime ordering](../../../internal/framework/runner/runner.go#L913) ·
|
|
[Decision recording](../../../internal/framework/runner/runner.go#L775) ·
|
|
[Ledger bucketing](../../../internal/framework/processreport/correction_ledger.go#L247) ·
|
|
[Built-in agreement](../../../internal/validators/registry_test.go#L97) ·
|
|
[Instance override](../../../internal/validators/metadata/metadata_test.go#L35) ·
|
|
[Ledger coverage](../../../internal/framework/processreport/correction_ledger_test.go#L12)
|
|
|
|
Treat execution class as a runtime fact. Resolve it at the runner boundary and
|
|
carry it with initial and repair-revalidation decision records. The ledger can
|
|
then project the class used for processing. Let the built-in registry attach
|
|
default class metadata during construction. The shared validation port can own
|
|
the class type and expose instance overrides. Preserve the current ledger
|
|
field names and default behavior for unclassified validators.
|
|
|
|
## 3. Put production module construction at the composition boundary
|
|
|
|
`framework/runner` already takes a `ModuleFactory` interface. The concrete
|
|
`framework/modules` package imports every built-in module and installs the four
|
|
constructors in `NewFactory`, which the CLI selects. All four production
|
|
constructor adapters ignore their context and run spec. Configuration validates
|
|
selectable identities through the dependency-light `core/modulecatalog`.
|
|
Keeping the registry populated with all built-in implementations in the
|
|
framework gives a composition concern the appearance and import direction of
|
|
a reusable execution package. [Runner factory port](../../../internal/framework/runner/runner.go#L30) ·
|
|
[Production registration](../../../internal/framework/modules/registry.go#L46) ·
|
|
[Constructor adapters](../../../internal/framework/modules/registry.go#L74) ·
|
|
[CLI composition](../../../internal/cli/process_pipeline.go#L50) ·
|
|
[Module key catalog](../../../internal/core/modulecatalog/catalog.go#L5)
|
|
|
|
Place the production factory alongside the built-in modules or in the CLI
|
|
composition code, and keep the injected factory interface in runner. A small
|
|
registration table can still support the existing constructor override seam.
|
|
Keep the config-facing catalog free of implementation imports, and use the
|
|
factory's tests to check that the catalog keys remain constructible. Module
|
|
packages can continue choosing their own proposal prompts and resolving their
|
|
explicit validator chains. [Factory and override tests](../../../internal/framework/modules/registry_test.go#L81) ·
|
|
[Module contract](../../internal/modules.md#module-contract)
|
|
|
|
## 4. Carry one checked processing plan from preflight into execution
|
|
|
|
`preflight.CheckReadiness` reads configured glossary selections and lists the
|
|
required proposal and validation profiles, adding a repair profile when the
|
|
budget is enabled. The CLI supplies an inspection callback that reaches into
|
|
the concrete runtime engine to inspect the profile and its backend. Later,
|
|
`executePipeline` traverses the settings again and binds the same roles,
|
|
directly dereferencing profile pointers based on the earlier check. In between,
|
|
`contracts.ResolveModuleRunSpecs` derives instance names and the effective
|
|
glossary protection view, including the glossary module's exemptions. The
|
|
readiness and execution steps therefore agree by convention on which roles
|
|
and selections will run. Adding a role or changing when it is active requires
|
|
coordinated edits in both places.
|
|
[Readiness checks](../../../internal/framework/preflight/preflight.go#L96) ·
|
|
[CLI inspection](../../../internal/cli/process_inputs.go#L125) ·
|
|
[Run-spec construction](../../../internal/framework/contracts/contracts.go#L106) ·
|
|
[Role binding](../../../internal/cli/process_pipeline.go#L20)
|
|
|
|
Have processing preparation derive the effective inputs and active role
|
|
selections once. Preflight can check that plan, and composition can bind the
|
|
role clients from the checked values. Keep model-facing profile inspection in
|
|
the runtime adapter behind the existing narrow shape, and let preflight own
|
|
the whole-pipeline readiness decision. The plan can preserve the typed resolved
|
|
configuration as the provenance source. This keeps inspection independent of
|
|
run readiness and preserves the requirement to reject a later unusable
|
|
selection before acquiring outputs or dispatching any model work.
|
|
[Later-stage and repair-budget tests](../../../internal/framework/preflight/readiness_test.go#L16) ·
|
|
[Glossary and readiness ADR](../../adr/0002-glossary-configuration-and-preflight.md#decision)
|
|
|
|
## 5. Give glossary matching and protection a domain owner
|
|
|
|
`core/glossary` derives identity-aware terms and exemptions. The actual
|
|
occurrence matcher and comparisons for identity and capitalization live in
|
|
`framework/validators.ProtectedVocabulary`. The LLM validation context builder
|
|
uses the same type to find mentioned glossary entries. The runner also builds
|
|
the combined validator `GlossaryView` and extracts its protection field for
|
|
apply-time checks. Matching, change protection, and prompt-context selection
|
|
have distinct callers, but the generic matcher and the apply-time policy are
|
|
contained in the validator implementation package. [Term derivation](../../../internal/core/glossary/vocabulary.go#L20) ·
|
|
[Matcher and change comparison](../../../internal/framework/validators/protected_terms.go#L23) ·
|
|
[Context matching](../../../internal/framework/validators/llm_context.go#L11) ·
|
|
[Runner preparation](../../../internal/framework/runner/runner.go#L168) ·
|
|
[Application call site](../../../internal/framework/runner/runner.go#L221)
|
|
|
|
Make glossary occurrence matching and text-to-text protection assessment
|
|
reusable by placing them next to glossary identity and vocabulary. Keep
|
|
validator-specific evidence selection, decision reasons, prompt neighbors,
|
|
and budgeting with validation. The application path should receive the
|
|
protection predicate it needs and reassess against the current text. Preserve
|
|
longest-match exemptions and the explicit spoken-word repetition allowance.
|
|
Aligning ordinary validation and application on full-segment evidence would
|
|
be a separate behavior decision. [Application and exemption tests](../../../internal/framework/validators/vocabulary_test.go#L47) ·
|
|
[Repetition contract](../../internal/validators.md#protected-name-repetition) ·
|
|
[Evidence-scope finding](10-equivalence.md#9-reconcile-the-protection-rule-at-validation-and-application)
|
|
|
|
## 6. Inject interaction diagnostics at the adapter boundary
|
|
|
|
Proposal generation advertises an injectable interaction writer, yet its
|
|
production path passes a directory and constructs `llm.DiagnosticsWriter`
|
|
itself. It discovers redaction values by type-asserting an extra method on
|
|
its structured client. Runner validation uses a second adapter around that
|
|
writer and resolves secrets lazily. Repair constructs the concrete writer and
|
|
collects values from both repair and validation clients because its retained
|
|
payload includes validation evidence. These paths make proposal generation
|
|
and the runner know about artifact destinations and credential discovery even
|
|
though structured completion already has an explicit port.
|
|
[Proposal fallback](../../../internal/framework/proposal_generation/generate.go#L129) ·
|
|
[Production proposal wiring](../../../internal/framework/proposal_generation/module_proposal.go#L55) ·
|
|
[Validation adapter](../../../internal/framework/runner/runner.go#L994) ·
|
|
[Repair write](../../../internal/framework/runner/repair.go#L173) ·
|
|
[Repair credential sources](../../../internal/framework/runner/repair.go#L282)
|
|
|
|
Give interaction capture one writer contract and let composition supply the
|
|
sink when debug is enabled. Put file naming, writes, and redaction in the
|
|
diagnostics adapter, with lazy credential sources supplied by the runtime.
|
|
Give proposal, validation, and repair callers responsibility for their own
|
|
semantic coordinates and evidence. Preserve repair's combined redaction scope
|
|
and the nonfatal result of optional write errors. The same boundary could
|
|
host debug execution projection, which currently makes `processreport` import
|
|
the concrete LLM observation model for a debug artifact.
|
|
[Artifact writer](../../../internal/framework/llm/diagnostics.go#L24) ·
|
|
[Execution debug projection](../../../internal/framework/processreport/llm.go#L61) ·
|
|
[Proposal write-failure test](../../../internal/framework/proposal_generation/generate_test.go#L391) ·
|
|
[Repair artifact tests](../../../internal/framework/runner/repair_test.go#L291)
|
|
|
|
## 7. Separate required publication from retained debug artifacts
|
|
|
|
`core/diagnostics.OutputSession` owns required transcript publication and
|
|
enabled report initialization and delivery. The same type also exposes
|
|
`PrepareDebug`, which creates a retained run directory and writes invocation
|
|
and effective-config artifacts. That method uses none of the output session
|
|
state. Preflight invokes it through the session and returns the two resources
|
|
together. This couples APIs with different lifetimes and failure semantics.
|
|
The CLI integration test exercises independently selected transcript, report,
|
|
and debug destinations. [Output session](../../../internal/core/diagnostics/output_session.go#L45) ·
|
|
[Debug method](../../../internal/core/diagnostics/output_session.go#L252) ·
|
|
[Preflight handoff](../../../internal/framework/preflight/preflight.go#L76) ·
|
|
[Independent destinations](../../../cmd/audita/main_integration_test.go#L844)
|
|
|
|
First expose debug setup directly through the run-directory API. Where package
|
|
separation pays for itself, put the required output session in an output
|
|
adapter and keep retained evidence in diagnostics. Preflight or CLI composition
|
|
can still coordinate the sequence. Preserve the existing early required-output
|
|
checks, independent debug selection and warnings, transcript-before-report
|
|
publication, and cleanup of unpublished output. This split addresses which
|
|
package owns delivery. It is compatible with sharing the regular-file staging
|
|
mechanics inside that owner. [Operational lifecycle](../../operations.md#outputs-and-lifecycle) ·
|
|
[Regular-file staging finding](10-equivalence.md#2-share-regular-file-staging-between-transcript-and-report-publication)
|
|
|
|
## 8. Give run-level report facts one assembly path
|
|
|
|
`processreport.Build` maps typed processing failures to report records and
|
|
computes counts. The CLI independently assembles the initial snapshot, derives
|
|
a failure reason and origin from runner state, then mutates status, phase,
|
|
error records, and counts during finalization. The filesystem session supplies
|
|
a formatted `Kind` string, so the CLI recognizes report initialization by
|
|
matching that string. For other required-output failures the report record
|
|
currently has an empty phase. Report meaning consequently depends on how the
|
|
failure reached the renderer, even though the YAML and Markdown share the
|
|
same presentation snapshot. [Report assembly](../../../internal/framework/processreport/report.go#L13) ·
|
|
[Processing classification in CLI](../../../internal/cli/process_pipeline.go#L60) ·
|
|
[Initial snapshot in CLI](../../../internal/cli/process_inputs.go#L94) ·
|
|
[Final status and error mapping](../../../internal/cli/process_output.go#L75) ·
|
|
[File failure type](../../../internal/core/diagnostics/output_session.go#L36)
|
|
|
|
Have the runner and publication adapter supply structured failure facts with
|
|
the phase and source coordinates they know, and have `processreport` own the
|
|
initial and final mappings to the report model. The CLI can still choose when
|
|
to finalize the run and how to present its outcome. Keep the artifact renderer
|
|
and file delivery separate so failed-output handling can render an updated
|
|
snapshot with the same record and count rules. Preserve the distinction between
|
|
unknown pre-run counts and known counts, and keep work-unit and run-level
|
|
failures independent. [Delivery failure test](../../../internal/cli/process_output_test.go#L79) ·
|
|
[Existing phase finding](01-correctness.md#11-required-delivery-failures-omit-the-phase-in-run-error-records) ·
|
|
[Report count contract](../../integrations/output-files.md#summary-counts)
|
|
|
|
## 9. Put transcript conversion and output shape at their owning boundaries
|
|
|
|
`schema.ParseSourceTranscriptJSON` owns source decoding and preserves optional
|
|
IDs. The processing CLI converts that object to the canonical segment model,
|
|
using one-based positions for missing IDs. `schema.ParseTranscriptJSONLenient`
|
|
has its own copy of the conversion. On output, `outputschema` chooses the
|
|
format but implements only the envelope. `schema.TranscriptToJSON` implements
|
|
the bare form and projects the same segment fields independently. This splits
|
|
the input model transition between CLI and schema, and the public segment
|
|
encoding between schema and the output encoder registry.
|
|
[CLI conversion](../../../internal/cli/run.go#L12) ·
|
|
[Lenient conversion](../../../internal/core/schema/transcript.go#L90) ·
|
|
[Bare encoder](../../../internal/core/schema/transcript.go#L224) ·
|
|
[Envelope encoder](../../../internal/core/outputschema/registry.go#L62)
|
|
|
|
Give the source-to-canonical transition a schema-level operation usable after
|
|
source acquisition, preserving the separate original source held for debug.
|
|
Let `outputschema` own both public envelopes and the common segment projection.
|
|
Keep parsing and normalization behavior independent of output-format choices.
|
|
Maintain separate fixtures for optional source IDs and the two public output
|
|
shapes. The current envelope test covers schema, version, and segment count,
|
|
so a compatibility refactor should verify payload fields independently as well.
|
|
[Lenient ID test](../../../internal/core/schema/transcript_test.go#L180) ·
|
|
[Encoder shape tests](../../../internal/core/outputschema/registry_test.go#L18) ·
|
|
[Encoding ownership](../../internal/output-schemas.md#scope-and-ownership)
|