16 KiB
Functional correctness audit
Reviewed source at c4ccd78. All findings below were reproduced through the
built CLI with temporary files or a synthetic loopback provider. The existing
offline suite also passed with Go 1.27.3 using
go test -count=1 -race -coverprofile=/tmp/audita-correctness-coverage.out ./....
The provider fixtures exercised Audita's handling of scripted decisions.
Live-model editorial quality was outside the review.
Architecture and review scope
Audita is a single-process Go CLI. CommandKit handles commands and streams, ConfigKit composes configuration, and processing preflight checks role profiles, glossary selections, and output destinations. Core packages parse and normalize the transcript, derive glossary views, and partition the normalized segments into fixed chunks. The framework runs configured correction modules in order while sections can generate proposals concurrently. Deterministic checks and LLM validators gate application. PromptKit handles the provider and continuation history for bounded repair branches, and Audita revalidates revisions. The output session publishes the transcript and reports, with opt-in diagnostics capturing request, decision, and execution evidence. Package map · Processing sequence
The review traced configuration and readiness into file handling, transcript decoding and normalization, chunk feasibility, glossary matching, proposal preview and application, validator batching and response reconciliation, repair outcomes, cancellation, and ordinary and debug reporting. It checked representative success paths alongside malformed inputs, boundaries, recoverable failures, and interrupted work.
1. Output path resolution can replace a different file
Expected. A regular transcript should be published at the selected output destination. Collision checks and publication should agree on which file that path names. Output lifecycle
Observed. With logical/link pointing to physical/inside, Linux resolves
logical/link/../out.json to physical/out.json. With two distinct existing
files and reporting disabled, Audita exited 0, left physical/out.json untouched,
and overwrote logical/out.json. Prepare stats the supplied path, but
canonicalDestination uses filepath.Abs and cleans the path before resolving
symlinks. The cleaned path is then used for publication. The same helper feeds
collision checks and report replacement. Output preparation
· Destination resolution
Remediation. Resolve directory components with filesystem traversal
semantics before collapsing ... Use the resulting destination for collision
checks, regular-file preparation, and publication. Derive the default report
directory from the same destination when file output is selected.
2. Glossary matching misses names with accented letters or punctuation at their edges
Expected. Selected protected names and aliases should participate in both deterministic protection checks. Matching names should also be available to validators when building context. The glossary accepts ordinary strings for names and aliases. Glossary selections · Validator context
Observed. With Élodie or C++ in the selected context and protection
collection, scripted grammar proposals replaced the names in Élodie arrived.
and Use C++ for this.. Both edits reached model validation and were applied
after scripted approvals. The validator requests also omitted the corresponding
glossary entries. The shared matcher surrounds every alternative with Go
regular-expression \b, whose word character class is ASCII. At the start of
Élodie and after the final + in C++, the required boundary fails.
Protected vocabulary
· Context matching
· Apply-time protection
Remediation. Match configured spellings with boundary checks against surrounding runes, including spellings that begin or end with punctuation. Preserve longest-match precedence and exemption behavior, and use consistent case matching for both the scan and identity lookup.
3. Merging nested segments truncates the earlier timestamp range
Expected. The merged interval should cover both source intervals. The same interval should determine whether the merge fits the configured duration limit and the gap to the next segment. Overlapping source intervals pass input validation. Transcript input contract · Processing sequence
Observed. With the default normalization settings and an empty pipeline,
same-speaker intervals [0,60], [5,6], and [20,21] produced [0,6]
containing the first two utterances and a separate [20,21] segment.
mergeSegments takes next.End even when it is earlier than the current end.
The next gap is then measured from 6, and duration checks likewise use
next.End - current.Start, which can let an over-limit merged interval pass.
Merge loop
· Duration and merged segment
Remediation. Use the later of the two end times for both the prospective duration and the resulting segment. Carry that end forward when calculating subsequent gaps.
4. Balanced chunking rejects feasible explicit targets
Expected. A configured section count should succeed when the input can be partitioned into that many contiguous sections within the token limit. Balancing is a preference. Chunking contract
Observed. Three segments with token estimates [2,5,1], alternating speakers
to prevent normalization merges, fit in two chunks with a maximum of 6.
The feasible partition is [2] and [5,1]. The feasibility check accepts
the target, but the builder stops the second chunk at 5 because it exceeds the
average target of 4. It leaves the last segment unassigned and the CLI exits 1
with “target_sections (2) is impossible under current constraints (got 2 sections).”
Feasibility check
· Explicit-target builder
Remediation. Choose balanced cuts only when the remaining suffix can still fit in the remaining count. A simpler fallback is to build the minimum number of max-bounded chunks and split them deterministically to reach a feasible target.
5. Cancellation during FIFO delivery can hang and later report success
Expected. A FIFO can wait for a reader. A catchable cancellation while it waits should end processing and attempt failed-status report finalization. Output lifecycle · Subprocess cancellation
Observed. With an empty pipeline and a named pipe as output, the child
stayed blocked after SIGTERM and left reports at in_progress. When a reader
attached after the signal, it received the transcript and the child exited 0
with completed reports. The main signal handler cancels a context, and the
CLI checks it before calling Finish. The direct-output branch then blocks
in os.OpenFile without observing that context. Delivery and final report
generation have no check for cancellation during that wait.
Signal handling
· Output finalization
· Direct stream delivery
Remediation. Pass cancellation into delivery. For supported FIFO platforms, use a context-aware wait for reader availability and for writable capacity, handling cancellation through the same failed-report path. Define how a cancellation that arrives after stream bytes have already been written is reported.
6. Transcript shape detection loses integer and unknown-field fidelity
Expected. Distinct supplied IDs in the supported integer range should remain distinct through validation. Unknown fields should be ignored. Transcript input contract
Observed. On Linux amd64, valid IDs 9007199254740992 and
9007199254740993 collapsed to the former value and the CLI rejected them as
duplicates. An envelope with a valid segments array and an unknown field
"ignored":1e400 also failed. The first decode targets any, so it converts
numbers throughout the payload to float64 before the segment array is
marshaled and decoded again. Initial decode
· Array extraction
Remediation. Select the top-level shape while retaining the raw segment array, then decode it directly into the typed source segment fields. Let the typed decode handle range and type validation.
7. An extra numeric validator reference can discard valid sibling decisions
Expected. If a validator returns one usable decision per requested reference, unexpected reference numbers should be ignored and the successful batch should record a filtering recovery. Reconciliation contract
Observed. A fixture returned two valid approvals and an extra decision
with index 34. Both corrections applied. Keeping the approvals and changing
the extra index to 9223372036854775808 or 34.0 made both decisions
unavailable with invalid_output, so neither correction applied. The schema
accepts those nonnegative integer values. The adapter decodes all indices into
Go int before reconciliation, so one schema-valid but incompatible number
discards the batch before the out-of-range index can be ignored.
Response schema
· Adapter decode
· Decision index type
· Reconciliation
Remediation. Preserve validator references as JSON numbers until matching them against the known local range. Normalize integral numeric spellings and classify out-of-range values as extras before converting requested references to Go indices.
8. Span-only protection can reject a valid edit or miss a repair opportunity
Expected. The protection validator should check whether a correction changes protected occurrences in the target segment. A known protection rejection can feed repair, and application should still recheck against the text produced by earlier corrections. Validator execution · Processing sequence
Observed. With AIN protected, a grammar edit from AIN to ain in
The certAIN result follows. was rejected because the isolated span looks
like a protected occurrence. The full segment has no occurrence of AIN as a
name. In the other direction, a grammar change from Silver to Golden in
The Silver Shield is equipped. passed deterministic protection with
Silver Shield protected. It received two scripted model approvals and was
skipped only at application. With a positive repair budget, the provider
received no repair request. The validator normally compares the spans alone,
while application compares the whole segment.
Deterministic protection
· Repair eligibility
· Apply-time protection
Remediation. Use the shared segment preview for applicable candidates in all modules and preserve the spoken-word repetition exception. Keep the application check against the current working text for conflicts introduced by earlier edits.
9. Credential redaction can change the ordinary YAML schema
Expected. The ordinary reports should retain their fixed schema, including
the required summary key and documented status values, while sanitizing
credential content. Report schema
Observed. With the synthetic credential value summary, a successful
processing run produced YAML with "[REDACTED]" in place of its required
summary key. After serializing the typed report, redaction applies a
substitution to every string scalar, including field names and fixed status
values. The CLI exited 0 despite the schema change.
Report serialization
· YAML redaction
Remediation. Sanitize input-derived report values before serialization. Keep Audita-owned field names, status enums, and reason codes under the control of the typed report serializer.
10. A completed proposal can disappear from the interruption ledger
Expected. Debug capture should retain proposed candidates that reached a usable response and were subsequently interrupted. Their ledger rows should record that they never reached application. Debug contract · Candidate capture
Observed. In a two-chunk run, the first chunk's proposal request was held open while the second returned a correction. After the second chunk's debug response artifact appeared, SIGTERM stopped the first request. The run published no transcript and correctly reported one usable proposal response and two interrupted work units. The correction ledger contained only the module failure row. The runner appends candidates only as it drains the ordered validation queue. A fatal return or cancellation can leave a complete later response in that queue or skip queuing it after recording its operation. Ordered collection · Ledger assembly
Remediation. Retain successful section results independently of validation scheduling. After joining workers on a fatal path, assign indices to unprocessed successful responses in section and response order and record their candidates as interrupted.
11. Required delivery failures omit the phase in run error records
Expected. A run-scoped error should identify its phase, including failed required output delivery. Error record contract
Observed. Delivering a normalized transcript to Linux /dev/full exited 1
and wrote a failed errors.yml record with reason: output_failure and the
destination path. The record omitted phase. The output session distinguishes
transcript writes, report writes, and cleanup through FileFailure.Kind.
The CLI mapper fills Phase only for report initialization.
Failure classification
· Run error mapping
Remediation. Map each required output failure kind to the corresponding run error phase, preserving the specific write or cleanup operation through finalization and any failed-report retry.