Files
audita/docs/internal/validators.md

188 lines
9.1 KiB
Markdown

# Audita Validators
## Scope
This document describes internal validator composition, execution order, and
decision handling. These registry keys identify implementation components; they
are not independently selectable through [configuration](../config.md#settings).
## Ownership
Built-in validator keys and chains:
- `internal/validators`
Shared validator runtime mechanics:
- `internal/framework/validators`
Execution-class metadata:
- `internal/validators/metadata`
## Built-In Validator Keys
Deterministic:
- `proposal_shape`
- `original_text_presence`
- `non_empty_resulting_segment` — rejects edits that would leave the segment
blank, regardless of replacement-span length.
- `no_effect`
- `protected_terms`
LLM-backed:
- `spoken_form_plausibility`
- `meaning_reversal_review`
- `editorial_review`
## Built-In Chains
Module chains are defined in `internal/validators/chains.go`.
Glossary, homophones, spoken_word, and grammar each resolve a fixed ordered chain.
## Runtime Execution
For each module section:
1. run deterministic validators;
2. run LLM-backed validators;
3. record decisions and compact operation outcomes;
4. carry only approved proposals forward.
When correction repair is enabled, the runner also gathers independent findings
for rejected candidates when a validator has usable evidence. Unavailable
assessments remain unavailable, and only candidates that pass every applicable
check can advance. A known rejection can enter repair even when another
assessment is unavailable; an unavailable assessment alone does not. Revised
candidates run the applicable checks again. A zero repair budget retains the
original shortlist flow.
The shared validator port lives in `internal/framework/validators`. Instances
may run concurrently across sections and can keep synchronized state. Request
data is borrowed for the duration of the call. Copy data before modifying or
retaining it.
The runner checks decision identity and disposition before allowing a candidate
to advance. A normal return supplies one decision per submitted candidate. A
fatal return may provide a checked subset as evidence. Contradictory approval
and disposition are a fatal contract failure. The recorded disposition governs
approval, and each record carries the effective execution class, including
instance overrides, for initial validation and revalidation.
## LLM Validator Batching
LLM validators:
- build canonical validation request payloads;
- batch using `validation.max_prompt_tokens` and the existing domain token estimator;
- select an embedded prompt ID and pass payload/description variables to the
role-bound structured client with the validator-decision schema.
Oversized single inputs are rejected with a consolidated
`validation_input_too_large` operation, without a provider call. Recoverable
model failures make only their batch candidates unavailable while other
batches continue. PromptKit validates each complete response against the
validator-decision JSON Schema and performs configured structural repair before
Audita receives decoded decisions. Each actual batch presents candidates
numbered from 1, independent of their internal proposal indices. Audita then
reconciles those local references: one usable entry keeps its
approval or editorial rejection; a missing or duplicate requested entry makes
only that candidate unavailable; unexpected indices are ignored. An empty list
makes all requested candidates unavailable. Decisions and unavailable indices
map back to internal proposal identities; results are ordered by request identity
and satisfy strict internal cardinality before the runner advances
approved candidates. An unavailable candidate never advances as approved.
A partially unavailable batch records one failed
operation with only those requested indices, while a successful batch with
ignored extras records a decision-filtering recovery. Fatal errors stop the
chain without returning unchecked candidates as approved. A local result builder
retains already-collected decisions and operations on early returns; full
cardinality checks and final sorting remain explicit on completed paths.
Reconciliation makes no semantic retry or extra model call; guided recovery remains
[deferred](../roadmap/future.md#domain-aware-response-recovery).
Debug request metadata records `proposal_indices` in local-reference order
(reference 1 is the first entry). Decoding preserves the supplied numeric
tokens, including integral decimal or exponent spellings. Reconciliation
checks the batch range before converting to a local integer, so an extra
reference beyond the native integer range can be ignored. Raw decoded responses
and ignored indices use local references. Reconciled decisions use internal
proposal indices. Batch token estimates use the same locally numbered payload
as prompt encoding.
## Decision and Rejection Reporting
Runner collects decisions, rejected candidates, and operation outcomes. Ledger
assembly groups decisions by deterministic versus LLM-backed execution metadata.
The serialized names and fields belong to the
[report and ledger contract](../integrations/output-files.md); implementation mapping
is covered by [report assembly](diagnostics-reporting.md).
## Changing Validators
1. Implement validator behavior under `internal/validators/<validator_key>`;
update registry and chain wiring in `internal/validators`.
2. Put shared runtime mechanics in `internal/framework/validators` only when
needed. Keep deterministic versus LLM-backed execution metadata explicit.
3. Preserve exposed identities and decision semantics under the
[architecture policy](../policy/architecture.md#correction-and-concurrency-invariants).
4. Review validator, registry, chain, batching, malformed-output, diagnostics,
and decision-cardinality tests; update meaningful coverage under the
[testing policy](../policy/testing.md).
5. Update implementation guidance here and any affected external contract in its
[canonical owner](../policy/documentation.md#canonical-ownership).
## Key Tests
- `internal/validators/*_test.go`
- `internal/framework/validators/*_test.go`
- `internal/framework/processreport/correction_ledger_test.go`
- `internal/cli/process_ordered_integration_test.go`
## Supplemental Validation Context
Each stage prepares a context matcher and indexes its selected glossary entries
by identity. Request construction uses a segment-position lookup and returns
matched entries in selection order for each candidate, including matches in the
replacement text. The deterministic guard and application check share the
stage's separate protection matcher.
Each candidate carries its target segment's supplied `speaker`, including an
empty value when no speaker is provided. It also includes the immediately
preceding and following source segments available within its fixed chunk,
labeled `preceding` or `following` with speaker,
text, and categories, plus glossary entries whose names, aliases, or
recognized plurals appear in its original/resulting text or those neighbors.
Glossary matching uses the same recognition rules; entries retain selection order.
Model-facing candidate and neighboring context omit internal segment IDs.
This supplies evidence such as a troupe definition for a proposed `Jesters`
correction or an adjacent speaker turn explaining deliberate repetition. A correction
already clear from its own segment does not require such evidence. No audio,
out-of-chunk context, or complete transcript is implied.
Context is serialized inside the validation item and included in the existing
payload token estimate, including the target speaker. Fitting measures the
required data first, then considers glossary entries followed by neighboring
segments in order. It stops at the first entry that exceeds the budget. Required
before/after text is never truncated. A candidate whose required data exceeds
the budget receives an unavailable decision. The built-in estimator reuses
encoded fragment counts for fitting and batching.
## Protected Names
Protection compares the source segment with the full replacement preview.
Missing and ambiguous spans have no protection verdict. An empty preview can
produce a protection finding alongside the blank-result rule. Application
rechecks the preview against the current segment after earlier edits.
The glossary matcher selects the longest spelling at a match position.
Punctuation-edged spellings use literal edge checks with Go's ASCII regexp
boundary semantics. Active glossary exemptions take part in match selection.
Validation context uses the same scan to select identities in glossary order.
### Protected-Name Repetition
Spoken-word cleanup may reduce repeated protected-name occurrences when every
original protected identity still appears in the resulting segment and surviving
spellings satisfy existing capitalization checks. Full-segment preview supports
both a replacement of the repeated phrase and deletion of a redundant span. This
exception only relaxes the deterministic count guard; editorial and meaning-reversal
validation still decide whether the repetition was accidental and the edit preserves
meaning. Intentional emphasis and substantive references remain protected by those
reviews.