Record algorithmic correctness and efficiency audit

This commit is contained in:
2026-11-15 03:00:28 +00:00
parent 0eca7b85d8
commit 717d59c3b0

View File

@@ -0,0 +1,223 @@
# Algorithmic correctness and efficiency audit
Reviewed the implementation at `0eca7b8`. Complexity estimates come from
source and call paths. Small synthetic probes checked match expansion,
ambiguous spans, timestamp ties, and strict parsing. The audit used no
production workloads or provider measurements.
The full offline suite passed with `go test -count=1 ./...` on Go 1.27.3.
## Architecture and review scope
Audita is a single-process Go CLI. CommandKit routes commands and streams,
ConfigKit resolves configuration, and the core packages parse, normalize, and
chunk transcripts and project glossary selections. The runner executes
correction modules in order over fixed chunks. Deterministic and LLM-backed
validators check proposals and each repair revision. PromptKit owns provider
admission, transport, and conversation coordination. The output session
publishes corrected transcripts, ordinary reports, and optional debug evidence.
[Package map](../../internal/overview.md) · [Pipeline](../../internal/pipeline.md)
The review followed work from configuration and normalization through proposal
application and evidence assembly. It preserves the selected forward chunker,
stage barriers, replacement policies, PromptKit admission and replay, and
validator budgeting rules. Costs required by emitted payloads or full report
matrices are treated as output costs. The earlier correctness findings are
recorded separately. [Pipeline](../../internal/pipeline.md)
· [Validator budgeting](../../internal/validators.md#supplemental-validation-context)
· [Deferred traversal design](../future.md#independent-whole-pipeline-traversal-by-chunk)
· [Correctness audit](01-correctness.md)
## Efficiency and scalability findings
### 1. Normalization repeatedly copies and scans growing merge runs
`shouldMergeWithReason` constructs the merged text and estimates its tokens
from scratch. On success, `mergeSegments` constructs the text again and
allocates a new category slice that copies the entire prefix. For a run of
`k` merged segments with fixed-size nonempty text and category lists, the
merge pass reads and copies `Θ(k²)` text bytes and category elements.
The exact bound is proportional to the sum of the merge-prefix sizes.
The token cap usually limits the run, but
nonempty whitespace-only fragments with short gaps can merge at zero estimated
tokens. The cost is therefore possible under the default token cap.
[Merge loop](../../../internal/core/normalization/normalize.go#L46)
· [Merge check and construction](../../../internal/core/normalization/normalize.go#L153)
· [Token estimator](../../../internal/core/normalization/tokens.go#L14)
· [Accepted segment text](../../integrations/transcript-glossary-files.md#transcript-json-contract)
Keep the forward merge pass. Accumulate text in an amortized builder and append
categories to an owned buffer. Maintain unrounded word and punctuation counts
per segment and combine them with the joiner before testing the token cap.
The ellipsis joiner can start a word when the previous text ends in whitespace,
and punctuation must be rounded only after adding all components. Check time
limits before processing text. This makes the merge pass linear in text bytes
and category elements, in addition to the existing chronological sort.
### 2. Application reporting searches the source chunk for each skipped edit
Application builds a segment-ID index inside `ApplyProposals`. The subsequent
`attributeApplyFiltering` pass indexes the approved proposals and sections,
but searches the original section's segment slice again for each skipped
proposal. With `s` segments and `k` skips in a section, this is
`O(s × k)` segment comparisons. Two identical, approved corrections per
segment make the second skip after the first changes its span. With one skip
per segment, the scans visit exactly `s(s+1)/2` entries. These are ordinary
stale conflicts, so the work can produce zero filtering records.
[Indexed application](../../../internal/framework/proposals/apply.go#L42)
· [Skip attribution](../../../internal/framework/runner/runner.go#L877)
Return immediately when there are no skips. For affected sections, index the
original segments once or carry original applicability forward from validation.
This reduces target lookup to expected `O(s + k)` work per affected section.
If defensive apply-time filtering adds actual defects, merge them once per
originating operation. The current singleton merge can rebuild the growing
deduplication map and sort the list after each addition. Preserve the
distinction between a response defect and a stale conflict.
[Filtering aggregation](../../../internal/framework/runner/runner.go#L837)
· [Recovery contract](../../integrations/output-files.md#error-and-recovery-records)
### 3. Validator glossary context expands every occurrence into every contributor
Context uses `ProtectedVocabulary.occurrences`, which emits one record per
matched occurrence per glossary identity. The context builder immediately
deduplicates those records into a set. For a spelling that occurs `m` times
and belongs to `g` selected entries, this costs `Θ(m × g)` records and
identity insertions after matching. A synthetic probe with 128 occurrences
of an alias shared by 96 entries produced 12,288 records to select 96
definitions. The deterministic protection check needs occurrence multiplicity.
The context builder only needs membership. [Occurrence expansion](../../../internal/framework/validators/protected_terms.go#L76)
· [Context selection](../../../internal/framework/validators/llm_context.go#L40)
Share the same scanner and match precedence. Let context deduplicate matched
spellings across the source, result, and neighbors before expanding their
identity lists. This case then takes expected `O(m + g)` membership work
before ordering the selected entries. Keep occurrence-level evidence for the
protection check and preserve glossary selection order.
### 4. Validation measures optional context that cannot fit
The production estimator correctly composes encoded fragment counts. Its
fitter nevertheless serializes all optional glossary entries and neighbors,
builds both prefix tables, and then removes entries from the end. It does
that even when the required fields alone exceed the budget. For `p` items
sharing `b` bytes of optional context across `g` entries, this can repeat
`Θ(p × b)` serialization and allocate `Θ(g)` prefix counters per fit even
when no optional entry is retained.
[Prefix measurement and fitting](../../../internal/framework/validators/llm_budget.go#L42)
· [Production batching caller](../../../internal/framework/validators/llm_batching.go#L17)
Return the required-only result as soon as its cost exceeds the budget.
Otherwise measure glossary entries in order and stop at the first one that
makes the prefix exceed the budget. Measure neighbors only if the entire
glossary prefix fits. Costs are monotone, so this preserves the existing policy
of dropping neighbors before definitions and keeping the longest affordable
ordered prefixes. An incremental fit uses constant auxiliary space for
counters and meters only retained entries plus the first exclusion in each
eligible field. Reusable glossary entry costs can also be stored with the
immutable glossary view if repeated encodings remain material. The cost of
matching candidates to glossary entries is separate.
[Context priority](../../internal/validators.md#supplemental-validation-context)
### 5. Ordinary runs retain debug-only decision history
The runner only adds candidates to the returned candidate list when debug
capture is active. It still builds a per-module candidate slice on every run
and aggregates, sorts, and stores initial validator decisions and rejections
in every `ModuleResult`. Rejected records carry original and proposed text
references. With `d_i` decision and rejection records in module `i`, finished
modules retain `Θ(Σ d_i)` records through invocation finalization, and the
runner pays for ordering them even when their production consumer, the
correction ledger, is disabled. Ordinary reports read the work-unit states and
compact operation outcomes. The current section's decisions and rejections
still serve candidate validation and repair eligibility.
[Candidate collection](../../../internal/framework/runner/runner.go#L537)
· [History sorting](../../../internal/framework/runner/runner.go#L617)
· [Module retention](../../../internal/framework/runner/runner.go#L271)
· [Ordinary reporting](../../../internal/framework/processreport/report.go#L51)
· [Debug ledger wiring](../../../internal/cli/process_pipeline.go#L54)
Separate detailed evidence retention from section-local decision state. In
the compact CLI path, skip copying candidates into aggregate slices and release
decision and rejection detail once the section finishes. Keep the operation
outcomes, counts, and failure origin used by ordinary reporting. Debug runs can
continue to retain and order the full evidence, including partially completed
work after a fatal return.
### 6. Repeated pipeline selections duplicate setup work
Module specifications select and copy the rich entries and project the
protected vocabulary independently for each pipeline entry, before the runner
starts. For `m` entries with the same selections of size `g`, setup time and
retained projections grow as `Θ(m × g)` with bounded-size entry fields. The
same data is used to prepare stage prompt variables and compile matchers again
on every repeated stage. Separately, readiness inspects a profile and its
backend for every role, even when they share IDs. PromptKit returns detached
inspection data, including copies of extra parameters. Selecting the same
resources `m` times with `b` extra parameter values can add `Θ(m × b)`
inspection work before processing.
[Module projection](../../../internal/framework/contracts/contracts.go#L107)
· [Entry copies](../../../internal/core/glossary/selection.go#L20)
· [Glossary preparation](../../../internal/framework/runner/runner.go#L169)
· [Role readiness](../../../internal/framework/preflight/preflight.go#L129)
· [Profile and backend inspection](../../../internal/cli/process_inputs.go#L125)
Reuse read-only projections and prepared data for identical effective ordered
selections, including the glossary correction exemption in the key.
Alternatively, build the projection at the module boundary and release it
before the next stage to bound retained memory when selections differ. Cache
successful profile and backend inspections for the captured engine while
keeping role-by-role checks and first-failure attribution. Shared data keeps
identical views and inspections proportional to the number of distinct inputs
plus the number of stage entries. Request messages still carry the selected
context for each call.
## Additional correctness findings
### 7. The unique-span check accepts overlapping matches
The grammar prompt requires an original span to appear exactly once.
`PreviewProposalForSegment` uses `strings.Count`, which counts nonoverlapping
matches. A probe proposed changing `ha ha` to `heh` in `ha ha ha`. The span
starts at byte offsets 0 and 3, but the preview returned applicable with
`heh ha` as the result. The same preview governs validation and application.
[Unique-span instruction](../../../internal/prompts/assets/prompts/modules.grammar.proposal.yaml#L19)
· [Preview](../../../internal/framework/proposals/preview.go#L35)
· [Application](../../../internal/framework/proposals/apply.go#L59)
For `require_unique`, locate the first occurrence and check for another
starting after its first byte, including an overlapping start. Stop as soon
as ambiguity is established. A successful check can use the recorded offset
to assemble the replacement. This retains a linear scan in the worst case for
a fixed literal and can exit early on ambiguity. Preserve the existing
nonoverlapping replacement behavior for `replace_all`.
### 8. Normalization can reorder utterances with exactly tied timestamps
`Normalize` compares only start times in an unstable sort. With 14 segments
whose starts alternate between 1 and 0 and whose ends are start plus one, a
normalization-only CLI run emitted the start-zero group in source-index order
`9,1,13,3,7,5,11`. Those segments have identical start and end times. With
distinct speakers, the ordering change occurs without any merge. The earlier
Python implementation at `0b1b670` ordered exact ties by source position, and
the Go rewrite recorded preservation of existing behavior as its baseline.
[Current sort](../../../internal/core/normalization/normalize.go#L58)
Preserve the input order for exact time ties. A stable sort or an explicit
source-position tiebreaker retains `O(n log n)` comparisons. Specify the
timestamp tiebreaker so later refactoring preserves utterance order.
### 9. The strict transcript parser panics on a missing ID
`ParseTranscriptJSON` calls the source parser, which allows omitted and null
IDs, then dereferences every ID before checking that IDs are sequential. A
package probe with an otherwise valid segment and no ID reproduced the nil
pointer panic. This affects the strict package helper, which currently has
test callers. The processing command uses the tolerant source parser and
assigns canonical IDs separately.
[Strict parser](../../../internal/core/schema/transcript.go#L65)
· [Sequential ID validation](../../../internal/core/schema/transcript.go#L204)
· [CLI input acquisition](../../../internal/cli/process_inputs.go#L46)
Check ID presence and return the normal field-validation error before
dereferencing. This adds a constant-time check per segment and preserves the
linear typed conversion pass.