The initial use case is merging independently transcribed speaker audio tracks from the same recorded session, such as a weekly tabletop RPG session. The architecture should also support meetings, podcasts, interviews, and other multi-speaker events.
The merge algorithm assumes that all input transcript timestamps are measured against the same session clock.
This is expected when each speaker has a separate recording that preserves silence and starts at the same session recording time. If input files have independent local timelines, `seriatim` cannot safely merge them without a separate alignment step.
## Pipeline Overview
The internal pipeline is:
```text
configuration check
-> input
-> preprocessing
-> merge
-> postprocessing
-> output
```
Each stage has an explicit data contract. Input and output stages perform I/O. Processing stages should be deterministic transformations over in-memory models and should record report events for validation findings, corrections, and transformations.
The configuration stage validates all CLI flags, environment variables, module names, input paths, output paths, and module-specific options before transcript data is processed.
Configuration validation should fail fast for:
- Missing required input.
- Unknown module names.
- Unknown input or output formats.
- Ambiguous speaker mappings.
- Invalid correction policies.
- Invalid timing thresholds.
- Invalid output paths.
The configuration stage produces an application config value that is passed through the pipeline.
### 2. Input Stage
The input stage converts external inputs into raw transcript documents with source metadata.
Input readers should be selected from an explicit registry. A reader is responsible for loading external data and returning raw transcript documents, not for canonical normalization.
### 3. Preprocessing Stage
The preprocessing stage applies zero or more modules before global merge.
Preprocessing starts with raw transcript documents from input readers and must end with canonical per-speaker transcripts. Some preprocessing modules operate on raw transcripts, some perform raw-to-canonical normalization, and some operate only on canonical transcripts.
Preprocessing modules are selected at runtime with a comma-separated list of canonical module names:
Modules run in the exact order provided. Unknown module names are configuration errors.
Potential preprocessing modules include:
- Structural raw transcript validation.
- Semantic transcript validation.
- Raw-to-canonical transcript normalization.
- Speaker name normalization based on input filename.
- Timing validation and deterministic correction.
- Text trimming.
Preprocessing should not depend on global chronological ordering across speakers. Modules that need the globally merged transcript belong in postprocessing.
Each preprocessing module must declare the model state it requires and the model state it produces. For example, `validate-raw` requires raw transcripts and produces raw transcripts, while `normalize-speakers` requires raw transcripts and produces canonical transcripts. Configuration validation should reject module orders that cannot type-check.
### 4. Merge Stage
The merge stage extracts all canonical segments from the preprocessed per-speaker transcripts and sorts them into a single deterministic chronological sequence.
The exact tie-breaker must be documented and stable across runs.
The merge stage should assign temporary internal references if needed, but it should not assign final output IDs until after all order-affecting postprocessing is complete.
### 5. Postprocessing Stage
The postprocessing stage applies zero or more modules to the merged transcript.
Postprocessing modules are selected at runtime with a comma-separated list of canonical module names:
The current runtime default selection is `seriatim-intermediate`, but default selection may change over time. Consumers that depend on a specific schema should request it explicitly.
Output writers should be selected from an explicit registry and should consume the final transcript model read-only. Multiple output writers may run for a single invocation.
`trim` is an artifact-level command that reads an existing seriatim output artifact and emits a projected artifact containing a segment-ID subset.
Design constraints:
-`trim` runs after `merge`, not as a merge postprocessor.
-`trim` validates the input artifact against supported seriatim output schemas.
-`trim` performs deterministic keep/remove selection by segment ID.
-`trim` renumbers retained IDs to `1..N` in transcript order.
-`trim` validates the final output against the selected output schema before writing.
-`trim` records audit metadata in report output.
`trim` is intentionally separate from merge postprocessing because it consumes already-emitted public artifacts. This separation keeps merge semantics stable and avoids rerunning merge-only transforms on projected artifacts.
`trim` must not rerun merge postprocessors such as `resolve-overlaps`, `coalesce`, or `autocorrect`.
-`normalize` sorts segments deterministically by chronological keys and stable input-index tie-breakers.
-`normalize` assigns fresh sequential output IDs (`1..N`) after sorting.
-`normalize` validates final output against the selected schema before writing.
-`normalize` writes optional deterministic report diagnostics when `--report-file` is requested.
`normalize` is intended for canonicalizing external transcript outputs (including Audita-style bare arrays) into seriatim contracts, not for running merge-time language or overlap transformations.
`normalize` must not run merge postprocessors such as overlap detection, overlap resolution, coalescing, or autocorrect.
Module names are canonical strings registered at startup. CLI flags refer to those names. The configuration stage resolves names into module instances before the pipeline runs.
- Module order is exactly the order specified by the user.
- An empty module list is valid when the stage supports zero modules.
- Unknown module names are fatal configuration errors.
- Module-specific options are read from the validated application config.
- A module must declare which pipeline stage and model type it supports.
- Modules should be deterministic for the same inputs, config, and application version.
- Modules should not perform I/O unless their class explicitly allows it.
Some modules may be recommended defaults. Defaults should be explicit in documentation and should be equivalent to passing the corresponding module list.
## Go Interface Sketch
The exact implementation may evolve, but the core interfaces should resemble:
`PreprocessState` should carry either raw transcripts, canonical transcripts, or both during migration between representations. The pipeline should validate that the ordered preprocessing list transitions from raw input state to canonical output state exactly once before merge.
The interfaces should favor value returns over hidden mutation. If pointer-based implementations are chosen for performance, mutation boundaries must still be clear and tested.
## Canonical Internal Model
The canonical model should be richer than the final output schema.
Canonical segment fields should include:
- Temporary internal reference.
- Source identifier.
- Source segment index.
- Canonical speaker.
- Start time.
- End time.
- Text.
- Word-level timing data, if available.
- Raw diarization labels, if useful for reporting.
- Validation and correction metadata, if needed internally.
The final output model can omit internal-only fields, but the report should retain enough provenance to diagnose corrections and transformations.
## Validation Strategy
Validation occurs at multiple boundaries:
- Configuration validation before processing.
- Raw input structural validation after input loading.
- Raw input semantic validation before normalization or correction.
- Canonical model validation after normalization and preprocessing.
- Merged model validation after merge and postprocessing.
- Final output schema validation before writing artifacts.
Structural validation answers whether data has the required shape and types.
Semantic validation answers whether the data is plausible and internally consistent.
Correctable issues should be deterministic and reportable. Fatal issues should stop the run with a non-zero exit code.
The `resolve-overlaps` module uses preserved word-level timing to replace detected overlap-group segments with smaller word-run segments when usable timing is available. Resolution expands each overlap window by the configured coalesce gap so nearby same-speaker context can be absorbed into the replacement runs. Once a segment is selected for replacement, all timed words from that segment participate in word-run construction so text is not clipped at the window boundary. Groups without usable word timing remain unresolved for later passes or human review.
For full-schema trim output, overlap groups are recomputed from retained segments so overlap annotations and group references remain internally consistent after projection.
Package boundaries should follow data ownership. Shared models belong in `internal/model`; stage-specific behavior belongs in the relevant stage package.
