Files
audita/docs/internal/pipeline.md

67 lines
4.2 KiB
Markdown
Raw Permalink Normal View History

# Audita Internal Pipeline
The [input contract](../integrations/transcript-glossary-files.md) owns input
formats; [configuration](../config.md) owns settings; the
[output contract](../integrations/output-files.md) owns serialized results.
1. CLI loads resolved configuration and transcript input. Preflight checks all
selected glossary views and roles before the output session prepares resources.
Transcript segments then enter canonical normalization.
2. Normalization sorts segments by start time and preserves input order for
ties. It evaluates merge limits using the later end time and the same token
estimate as chunking, then reassigns IDs where needed.
3. The runner partitions the normalized transcript exactly once before its
module loop. Initial section identity, segment membership, and token
estimates stay fixed while text evolves.
4. Module instances execute in configured order. Each section hands its decoded
proposals and original local segment references to the runner along with
operation facts and any captured exchange. A bounded window admits sections
in order and lets validation and repair proceed as proposals become ready.
The runner assigns final proposal indices in section and response order
when it records completed work. It then writes diagnostics that use those
indices. PromptKit controls provider capacity across all three roles.
5. Deterministic validators precede LLM-backed validators. Every submitted
candidate receives one decision per validator on a normal return. Fatal returns
may preserve a checked subset of completed decisions as diagnostic evidence.
Recoverable failed batches mark their candidates unavailable; successful
survivors continue through all required validators.
6. Actionable rejections can enter independent section-local repair branches.
PromptKit coordinates each branch from the captured proposal exchange, owning
its history and additional-turn limit. Each revision, including the final
permitted response, is mapped to the original section and revalidated from the
start. Withdrawal, exhaustion, and recoverable failures finish the branch
without affecting independent approvals.
7. Only fully approved proposals reach application. Corrections apply in
proposal-index order within their owning section. Existing replacement
policies and apply-time safety checks resolve stale conflicts. Skip
attribution uses available validation previews of the original text to
distinguish stale edits from defects in a proposal or repair response.
8. CLI encodes the final transcript and publishes active files through the
output session. `framework/processreport` builds one snapshot for YAML,
Markdown, and CLI totals. Debug evidence is captured separately when enabled.
`runner.Run` returns the final working transcript, initial section inventory,
explicit `not_started`/`interrupted`/`completed` work units, compact model
operation outcomes, and per-module results. Candidate payloads are retained
in the returned output only when debug capture is requested. A work unit can
complete with model errors and applied corrections. Recoverable model failures
never trigger shared cancellation. Fatal failures and parent cancellation stop
further dispatch and join active workers. The initiating error carries its
processing phase and available origin. With ordinary reports enabled, work-unit
evidence includes available peer interruption details. The runner drains active
outcomes and assigns indices to usable proposals from later sections even when
it could not validate them. The working transcript advances at the module's
successful commit boundary.
Earlier committed text remains available to an interrupted run, and transcript
delivery requires successful processing.
Required transcript/report write failures are fatal. Optional debug writes
report safely to stderr without changing decisions or exit status. See
[operations](../operations.md) for file placement and failure handling, and
[architecture policy](../policy/architecture.md#correction-and-concurrency-invariants)
for invariants.
Key tests: `internal/framework/runner/*_test.go`,
`internal/framework/validators/*_test.go`, and
`cmd/audita/main_integration_test.go`.