`narratio` is a Go orchestrator for D&D session processing. It coordinates a stage-based pipeline from recorded audio through transcript generation, transcript polishing, and downstream artifact generation, while preserving durable run state for skip/rerun/resume behavior.
The repository currently implements the orchestration scaffold, local state model, and stage framework, including real WhisperX/Seriatim/Audita adapters plus real `prepare`/`transcribe`/`merge`/`polish` stages. It intentionally does **not** yet implement real Scriptorium-backed artifact generation, remote archive, or notification integrations.
The future `analyze` stage will use `scriptorium` as a subprocess to generate analysis artifacts such as a session recap. The initial Scriptorium-backed artifact will be a **session recap** generated from:
1. the processed/polished transcript, and
2. the previous session recap, when available.
The design should remain modular and composable so later artifact-generation workflows can produce intermediate artifacts, such as structured event logs or state maps, and then use those artifacts as inputs to later outputs such as session recaps, player summaries, or table-read analyses.
Do not implement a general DAG engine initially. The design should, however, avoid hardcoding assumptions that would make later artifact dependencies difficult.
llm_api_key_env stores only the environment variable name.
API key values are read from the process environment at runtime.
API key values are not stored in pipeline.yml, manifest metadata, generated configs, or logs.
If llm_api_key_env is configured and the named env var is missing/empty, the Audita adapter fails before invocation with a redacted error.
If llm_api_key_env is empty/omitted, the Audita adapter omits AUDITA_LLM_API_KEY from subprocess env overrides and continues.
Planned Scriptorium Config
The future Scriptorium integration should add a top-level pipeline.scriptorium config section.
Illustrative shape:
scriptorium:
binary: "scriptorium"
config_path: "/etc/scriptorium/config.yml"
timeout: "10m"
render_debug: false
artifacts:
session_recap:
enabled: true
prompt_id: "dnd.session_recap"
profile_id: "local-quality"
output_path: "artifacts/session_recap.md"
timeout: "10m"
inputs:
transcript:
source: "processed_transcript"
required: true
previous_recap:
source: "previous_session_artifact"
artifact: "session_recap"
required: false
vars:
session_id: true
session_date: true
campaign_name: true
previous_session_id: true
output_kind: "session_recap"
This shape is illustrative, not yet implemented. The important design points are:
scriptorium.binary should accept either scriptorium on PATH or a full path.
scriptorium.config_path should be optional.
When omitted, Scriptorium may use its own default config behavior.
When provided, Narratio should pass --config <path>.
Prompt IDs are configuration values, not hardcoded in the analyze stage.
Profile IDs are optional configuration values.
When omitted, Scriptorium prompt defaults may apply.
Artifact definitions should map:
artifact name
prompt ID
optional profile ID
named inputs
small metadata variables
output destination
timeout
optional render-debug behavior
Input names must match the Scriptorium prompt definition.
Large content should be passed as input files, not as --var values.
API keys should never be passed directly on the command line.
Runtime overrides such as --llm-base-url, --model, --api-key-env, --temperature, --max-tokens, --top-p, and --timeout should be used only when explicitly configured.
Validation Scope
Validation currently enforces the implemented config sections listed above.
Future Scriptorium validation should enforce:
scriptorium.binary is required for real analyze.
configured artifact names are unique.
enabled artifacts have a non-empty prompt_id.
output paths are session-workdir-relative or otherwise safe.
input names are non-empty.
required configured inputs can be resolved before execution.
optional inputs may be omitted without failing the stage.
explicit config_path, when provided, exists and is a regular file.
timeout values parse as Go durations.
unknown YAML fields continue to fail strict decoding.
Do not validate Scriptorium prompt library internals inside Narratio. Scriptorium owns prompt/profile/schema loading and validation.
Current manifest model includes optional pipeline_version, but does not include an explicit manifest schema/version field.
Before manifests become long-term compatibility-sensitive across releases/storage backends, add explicit manifest schema/versioning and migration policy.
Future Artifact Records for Scriptorium
The Scriptorium-backed analyze stage should record each generated artifact with enough metadata to support rerun, inspection, and future artifact dependency resolution.
Stage interface with Name, Declares, and Run(ctx, env, manifest).
IODecl for declared input/output artifact intent.
StageResult for outputs, logs, generated configs, and metadata.
Current Declares role:
Stage.Declares is currently contract metadata.
The runner does not yet enforce declared inputs/outputs at execution time.
Real-stage implementation work should strengthen enforcement by validating declared inputs before execution and declared outputs after execution.
Current Stage Behaviors
prepare real
Ensures layout.
Resolves session-relative input paths.
Materializes required input files into inputs/.
Writes pipeline.resolved.yml.
Resolves/copies .flac audio inputs from audio_dir or audio_files.
Computes checksums and records deterministic manifest.inputs.
Uses checksum-aware write/copy reuse for idempotency.
transcribe real
Discovers prepared .flac audio inputs from manifest.inputs with kind=audio or work/.../audio fallback.
Derives per-speaker output files at transcripts/raw/{audio_basename}.json.
Calls whisperx.Client with bounded parallelism from pipeline.whisperx.concurrency.
Validates each output file exists and is valid JSON before stage success.
Records output refs plus stage metadata.
merge real
Discovers raw transcript inputs from manifest.stages.transcribe.outputs with kind=transcript_raw or work/.../transcripts/raw fallback.
Validates input transcript files as existing regular JSON files.
Uses prepared inputs/speakers.yml and inputs/autocorrect.yml directly.
Does not translate speaker-map format.
Invokes seriatim.Runner using canonical merged/report/log/generated-config paths under the session workdir.
Validates merged transcript output JSON and report JSON before stage success.
Records merged/report output refs plus stage metadata.
polish real
Discovers merged transcript input from manifest.stages.merge.outputs with kind=transcript_merged or fallback work/.../transcripts/merged.json.
Validates merged transcript as JSON.
Requires prepared inputs/glossary.yml.
Invokes audita.Runner using canonical paths:
transcripts/processed.json
artifacts/audita.report.json
artifacts/audita-work
logs/audita.stdout.log
logs/audita.stderr.log
config/audita.generated.yml
Validates processed transcript output JSON with required top-level segments array.
Validates report JSON when enabled.
Records processed/report outputs plus non-secret provenance metadata.
normalize placeholder
normalize remains a placeholder. Audita and Seriatim both perform some normalization-like behavior internally. A separate real normalize stage should be implemented only if Narratio develops a clear stage-specific transformation that belongs outside Seriatim and Audita.
analyze planned
analyze will become the Scriptorium-backed artifact generation stage.
Initial behavior should be narrowly scoped:
Generate a session_recap artifact.
Consume transcripts/processed.json.
Include the previous session recap as previous_recap input when configured and available.
Invoke scriptorium run.
Write output to artifacts/session_recap.md.
Capture stdout/stderr logs.
Generate a redacted invocation/config file for provenance.
Validate that the output artifact file exists and is non-empty.
Record artifact refs and non-secret provenance in manifest.
Future behavior should generalize artifact generation:
multiple configured artifact definitions
named inputs
optional inputs
artifact-to-artifact dependencies
render diagnostics for testing/debugging
structured artifacts such as JSON event logs
sequential artifact generation where one artifact feeds another
Do not implement a generic DAG engine in the first pass. The first Scriptorium integration should be an intentionally small, synchronous subprocess implementation.
archive placeholder
Future remote artifact persistence. Storage backends may include local filesystem, S3-compatible object storage, SFTP, or similar.
notify placeholder
Future notification backend, such as email or chat notification, once pipeline execution completes.
11. Application Environment and Adapters
internal/app uses stage.Env as the shared dependency container to avoid duplicate environment definitions.
render does not call the LLM and should not be treated as artifact generation.
Inputs
Inputs must be passed as repeated:
--input name=path
Rules:
name must match the Scriptorium prompt definition input name.
Prefer absolute paths, or paths controlled by Narratio's session workdir.
The processed transcript is the primary transcript input.
Optional inputs may include previous recap, glossary, campaign notes, event logs, final state maps, or other generated artifacts.
Scriptorium reads files directly; Narratio should not inline large input content.
Variables
Small metadata should be passed as repeated:
--var name=value
Examples:
session_id
session_date
campaign_name
previous_session_id
output_kind
Large content belongs in input files, not --var.
Prompt IDs and Profiles
Prompt IDs are configuration values.
Narratio should not hardcode prompt IDs such as dnd.session_recap in stage logic. Prompt IDs should come from pipeline.yml artifact definitions.
Profiles:
May be omitted to use the prompt's default_profile.
May be provided to force a profile such as local-fast, local-quality, frontier, batch, or test.
Should be configuration values.
Runtime Overrides
Scriptorium supports runtime override flags such as:
--llm-base-url
--model
--api-key-env
--temperature
--max-tokens
--top-p
--timeout
Guidance:
Keep normal model/runtime settings in Scriptorium execution profiles.
Use runtime overrides only for explicit operator-directed exceptions or tests.
Never pass raw API keys on the command line.
--api-key-env names an environment variable; Narratio should ensure that variable is set in the subprocess environment when configured.
Output Handling
For scriptorium run:
Narratio should always pass --out.
Treat the output path as the generated artifact.
Capture stdout/stderr separately.
Non-empty stderr alone does not imply failure.
For scriptorium render:
Use --out for diagnostics.
Use --format json when tests need structured render output.
Exit Status
Scriptorium exit status behavior:
0: success.
1: runtime, parse, config, load, render, generation, or I/O error.
2: run completed but output validation failed.
Narratio should treat non-zero exit codes as failed stage execution.
However, when exit code is 2, Scriptorium may have already written an output artifact. Narratio may record that output path in failure metadata, but should not mark the artifact as successfully generated.
13. Run Control, Locking, Logging, and Long-Running Stages
Run Control
internal/app runner is sequential and manifest-driven:
Loads/creates manifest.
Computes per-stage action: run vs skip.
Skip rule today: succeeded + not forced => skip.
--force reruns previously succeeded stages.
resume starts at first non-succeeded stage.
Stale detection is intentionally TODO pending checksum-based invalidation logic.
Locking
run/resume/run-stage execution paths acquire a session lock via artifact store and release it with defer.
Logging
Structured logger is initialized via internal/logging using slog.
Runner emits concise stage lifecycle logs.
Real WhisperX/Seriatim/Audita stages already use explicit output/log/config paths.
Future Scriptorium stage should use the same pattern:
logs/scriptorium.<artifact>.stdout.log
logs/scriptorium.<artifact>.stderr.log
config/scriptorium.<artifact>.generated.yml
Long-Running Stage Expectations
The architecture expects long-running stages.
WhisperX, Seriatim, and Audita already use context timeout/cancellation in their adapters. The future Scriptorium adapter should also use context timeout/cancellation through the shared subprocess helper.
14. Previous Session Context
The first Scriptorium-backed artifact, session_recap, may optionally consume the previous session recap.
Narratio should support this as an optional configured input.
Initial implementation may use a simple local path or session config reference. Future implementations may resolve previous-session artifacts from:
current workspace
prior session workdir
manifest records
archive/storage backend
campaign/session index
The important rule: missing optional previous recap should not fail the initial session_recap artifact generation.
If previous recap is configured as required and cannot be resolved, analyze should fail clearly before invoking Scriptorium.
Do not implement a full campaign history index in the initial Scriptorium pass.
Manifest is the source of truth for stage status and resume/skip decisions.
Stage outputs/provenance are recorded durably and deterministically.
Local workdir is the primary execution area; remote storage is an adapter concern.
Main entrypoint stays thin; cmd/narratio delegates to internal/app.
Pipeline behavior remains explicit and maintainable; no generic DAG engine introduction.
Every stage must consume declared artifacts, produce declared artifacts, validate them, record provenance, and remain safely skippable/rerunnable.
Artifact generation must remain configuration-driven; prompt IDs, profile IDs, and artifact input mappings should not be hardcoded in the orchestration core.
Scriptorium integration must use the public CLI subprocess contract until a deliberate future architecture change is made.
Secret values must never be stored in configs, generated invocation files, logs, or manifest metadata.
