Files
audita/docs/roadmap/audit/02-efficiency.md

13 KiB
Raw Blame History

Algorithmic correctness and efficiency audit

Reviewed the implementation at 0eca7b8. Complexity estimates come from source and call paths. Small synthetic probes checked match expansion, ambiguous spans, timestamp ties, and strict parsing. The audit used no production workloads or provider measurements. The full offline suite passed with go test -count=1 ./... on Go 1.27.3.

Architecture and review scope

Audita is a single-process Go CLI. CommandKit routes commands and streams, ConfigKit resolves configuration, and the core packages parse, normalize, and chunk transcripts and project glossary selections. The runner executes correction modules in order over fixed chunks. Deterministic and LLM-backed validators check proposals and each repair revision. PromptKit owns provider admission, transport, and conversation coordination. The output session publishes corrected transcripts, ordinary reports, and optional debug evidence. Package map · Pipeline

The review followed work from configuration and normalization through proposal application and evidence assembly. It preserves the selected forward chunker, stage barriers, replacement policies, PromptKit admission and replay, and validator budgeting rules. Costs required by emitted payloads or full report matrices are treated as output costs. The earlier correctness findings are recorded separately. Pipeline · Validator budgeting · Deferred traversal design · Correctness audit

Efficiency and scalability findings

1. Normalization repeatedly copies and scans growing merge runs

shouldMergeWithReason constructs the merged text and estimates its tokens from scratch. On success, mergeSegments constructs the text again and allocates a new category slice that copies the entire prefix. For a run of k merged segments with fixed-size nonempty text and category lists, the merge pass reads and copies Θ(k²) text bytes and category elements. The exact bound is proportional to the sum of the merge-prefix sizes. The token cap usually limits the run, but nonempty whitespace-only fragments with short gaps can merge at zero estimated tokens. The cost is therefore possible under the default token cap. Merge loop · Merge check and construction · Token estimator · Accepted segment text

Keep the forward merge pass. Accumulate text in an amortized builder and append categories to an owned buffer. Maintain unrounded word and punctuation counts per segment and combine them with the joiner before testing the token cap. The ellipsis joiner can start a word when the previous text ends in whitespace, and punctuation must be rounded only after adding all components. Check time limits before processing text. This makes the merge pass linear in text bytes and category elements, in addition to the existing chronological sort.

2. Application reporting searches the source chunk for each skipped edit

Application builds a segment-ID index inside ApplyProposals. The subsequent attributeApplyFiltering pass indexes the approved proposals and sections, but searches the original section's segment slice again for each skipped proposal. With s segments and k skips in a section, this is O(s × k) segment comparisons. Two identical, approved corrections per segment make the second skip after the first changes its span. With one skip per segment, the scans visit exactly s(s+1)/2 entries. These are ordinary stale conflicts, so the work can produce zero filtering records. Indexed application · Skip attribution

Return immediately when there are no skips. For affected sections, index the original segments once or carry original applicability forward from validation. This reduces target lookup to expected O(s + k) work per affected section. If defensive apply-time filtering adds actual defects, merge them once per originating operation. The current singleton merge can rebuild the growing deduplication map and sort the list after each addition. Preserve the distinction between a response defect and a stale conflict. Filtering aggregation · Recovery contract

3. Validator glossary context expands every occurrence into every contributor

Context uses ProtectedVocabulary.occurrences, which emits one record per matched occurrence per glossary identity. The context builder immediately deduplicates those records into a set. For a spelling that occurs m times and belongs to g selected entries, this costs Θ(m × g) records and identity insertions after matching. A synthetic probe with 128 occurrences of an alias shared by 96 entries produced 12,288 records to select 96 definitions. The deterministic protection check needs occurrence multiplicity. The context builder only needs membership. Occurrence expansion · Context selection

Share the same scanner and match precedence. Let context deduplicate matched spellings across the source, result, and neighbors before expanding their identity lists. This case then takes expected O(m + g) membership work before ordering the selected entries. Keep occurrence-level evidence for the protection check and preserve glossary selection order.

4. Validation measures optional context that cannot fit

The production estimator correctly composes encoded fragment counts. Its fitter nevertheless serializes all optional glossary entries and neighbors, builds both prefix tables, and then removes entries from the end. It does that even when the required fields alone exceed the budget. For p items sharing b bytes of optional context across g entries, this can repeat Θ(p × b) serialization and allocate Θ(g) prefix counters per fit even when no optional entry is retained. Prefix measurement and fitting · Production batching caller

Return the required-only result as soon as its cost exceeds the budget. Otherwise measure glossary entries in order and stop at the first one that makes the prefix exceed the budget. Measure neighbors only if the entire glossary prefix fits. Costs are monotone, so this preserves the existing policy of dropping neighbors before definitions and keeping the longest affordable ordered prefixes. An incremental fit uses constant auxiliary space for counters and meters only retained entries plus the first exclusion in each eligible field. Reusable glossary entry costs can also be stored with the immutable glossary view if repeated encodings remain material. The cost of matching candidates to glossary entries is separate. Context priority

5. Ordinary runs retain debug-only decision history

The runner only adds candidates to the returned candidate list when debug capture is active. It still builds a per-module candidate slice on every run and aggregates, sorts, and stores initial validator decisions and rejections in every ModuleResult. Rejected records carry original and proposed text references. With d_i decision and rejection records in module i, finished modules retain Θ(Σ d_i) records through invocation finalization, and the runner pays for ordering them even when their production consumer, the correction ledger, is disabled. Ordinary reports read the work-unit states and compact operation outcomes. The current section's decisions and rejections still serve candidate validation and repair eligibility. Candidate collection · History sorting · Module retention · Ordinary reporting · Debug ledger wiring

Separate detailed evidence retention from section-local decision state. In the compact CLI path, skip copying candidates into aggregate slices and release decision and rejection detail once the section finishes. Keep the operation outcomes, counts, and failure origin used by ordinary reporting. Debug runs can continue to retain and order the full evidence, including partially completed work after a fatal return.

6. Repeated pipeline selections duplicate setup work

Module specifications select and copy the rich entries and project the protected vocabulary independently for each pipeline entry, before the runner starts. For m entries with the same selections of size g, setup time and retained projections grow as Θ(m × g) with bounded-size entry fields. The same data is used to prepare stage prompt variables and compile matchers again on every repeated stage. Separately, readiness inspects a profile and its backend for every role, even when they share IDs. PromptKit returns detached inspection data, including copies of extra parameters. Selecting the same resources m times with b extra parameter values can add Θ(m × b) inspection work before processing. Module projection · Entry copies · Glossary preparation · Role readiness · Profile and backend inspection

Reuse read-only projections and prepared data for identical effective ordered selections, including the glossary correction exemption in the key. Alternatively, build the projection at the module boundary and release it before the next stage to bound retained memory when selections differ. Cache successful profile and backend inspections for the captured engine while keeping role-by-role checks and first-failure attribution. Shared data keeps identical views and inspections proportional to the number of distinct inputs plus the number of stage entries. Request messages still carry the selected context for each call.

Additional correctness findings

7. The unique-span check accepts overlapping matches

The grammar prompt requires an original span to appear exactly once. PreviewProposalForSegment uses strings.Count, which counts nonoverlapping matches. A probe proposed changing ha ha to heh in ha ha ha. The span starts at byte offsets 0 and 3, but the preview returned applicable with heh ha as the result. The same preview governs validation and application. Unique-span instruction · Preview · Application

For require_unique, locate the first occurrence and check for another starting after its first byte, including an overlapping start. Stop as soon as ambiguity is established. A successful check can use the recorded offset to assemble the replacement. This retains a linear scan in the worst case for a fixed literal and can exit early on ambiguity. Preserve the existing nonoverlapping replacement behavior for replace_all.

8. Normalization can reorder utterances with exactly tied timestamps

Normalize compares only start times in an unstable sort. With 14 segments whose starts alternate between 1 and 0 and whose ends are start plus one, a normalization-only CLI run emitted the start-zero group in source-index order 9,1,13,3,7,5,11. Those segments have identical start and end times. With distinct speakers, the ordering change occurs without any merge. The earlier Python implementation at 0b1b670 ordered exact ties by source position, and the Go rewrite recorded preservation of existing behavior as its baseline. Current sort

Preserve the input order for exact time ties. A stable sort or an explicit source-position tiebreaker retains O(n log n) comparisons. Specify the timestamp tiebreaker so later refactoring preserves utterance order.

9. The strict transcript parser panics on a missing ID

ParseTranscriptJSON calls the source parser, which allows omitted and null IDs, then dereferences every ID before checking that IDs are sequential. A package probe with an otherwise valid segment and no ID reproduced the nil pointer panic. This affects the strict package helper, which currently has test callers. The processing command uses the tolerant source parser and assigns canonical IDs separately. Strict parser · Sequential ID validation · CLI input acquisition

Check ID presence and return the normal field-validation error before dereferencing. This adds a constant-time check per segment and preserves the linear typed conversion pass.