15 KiB
Session workspace organization
Purpose
Give operators predictable locations for session inputs, completed outputs, logs, and debug evidence. Apply the taxonomy in ADR 0001 and the ownership boundary in ADR 0002.
This roadmap defines the target behavior and required changes. Implementation sequencing belongs in a subsequent implementation plan. The ADRs remain Proposed pending their acceptance.
Confirmed Requirements
- Completed artifacts occupy a flat
artifacts/directory. Intermediate products such assession_bounds.jsonbelong here. - Combined transcripts occupy
transcripts/directly. Initial per-speaker transcripts occupy a subdirectory beneath it. - Prepared external audio occupies
audio/; other prepared external inputs occupyinputs/. Outputs consumed by later stages retain their output paths. - Logs and Debug use matching latest-attempt views:
logs/<stage>anddebug/<stage>are symlinks to the producing run's directories. Both views advance when a stage starts an actual attempt, including attempts that fail. A skipped stage leaves its previous views intact. - Execution reports and invocation records are Logs. Historical Logs and Debug remain under their producing runs, subject to cleanup.
- Narratio can selectively request supported subprocess debug capture through configuration. Omission preserves application configuration and defaults; Narratio needs no explicit downstream debug-disable override.
- Prepared session audio is the single retained Narratio-managed audio copy. Remove Narratio's separate persistent audio cache and spool duplicates.
- Configurations that produce colliding artifact names or destinations fail validation before execution. Validation covers the effective pipeline, including expanded character families and built-in artifact destinations.
- Narratio selects input, output, log, and debug paths. Subprocess applications own their working State and Cache through their supported configuration.
- A dedicated cache root such as
/var/cache/notariusis a deployment recommendation. The application owns its cache defaults, internal layout, identity, and reuse semantics. Narratio session cleanup respects that boundary. - Narratio retains its own manifest, recovery records, and managed staging State.
- This is a clean cutover for newly prepared sessions. Existing-session migration and backward compatibility shims are outside the feature.
- Treatment of older completed files after invalidation or failed replacement remains unchanged, including their availability through manifest-backed resolution. Directory reorganization does not authorize new stale-file deletion, automatic fallback, or rollback behavior.
Current Baseline
The operator guide owns the current physical layout. Relevant differences from the target are:
- Extraction moves the complete Notarius bundle to
artifacts/notarius/<run_id>/. Resume evidence and artifact resolution depend on that bundle and its index. - Merge and normalize promote Seriatim execution reports into
artifacts/. - Analysis output paths and character-family patterns can introduce nested artifact directories. Output-path configuration also permits locations that need reconciliation with the selected directory categories.
- Standard combined transcript paths already match the target. Layout creation
also creates an unused
transcripts/trimmed/directory. - Most subprocess diagnostics live under individual run directories. Root-level log paths lack a consistent latest-attempt view. Audita reports sit beside its staged transcript, and restore writes a separate top-level report.
- Prepared previous-session inputs occupy
previous/. Narratio-owned reference snapshots supplied to Notarius occupy sessioncache/notarius/references/. These snapshots are distinct from Notarius's own execution caches. - Audita debug placement follows downstream configuration. Narratio does not expose Notarius debug capture, and Scriptorium render-debug placement varies.
- Narratio maintains separate S3 audio cache and spool paths in addition to the prepared session audio copy.
Target Layout And Ownership
Retain the configurable workspace root and its current system-wide default,
/var/lib/narratio. Sessions remain under
/var/lib/narratio/work/<campaign>/<session> by default. The following paths
express the selected categories:
| Path | Contents and role |
|---|---|
manifest.json |
Narratio's durable progress and output references. |
artifacts/<filename> |
Completed artifact payloads, including session bounds. |
transcripts/<filename> |
Completed combined transcripts and rendered transcripts. |
transcripts/raw/<speaker>.json |
Completed initial per-speaker transcripts. |
audio/ |
Prepared external audio. |
inputs/ |
Prepared configuration, references, and previous-session inputs. |
logs/<stage> |
Symlink to the latest attempt's run-local logs and execution reports. |
debug/<stage> |
Symlink to the latest attempt's run-local debug directory, empty when no capture is produced. |
runs/<run_id>/<stage>/logs/ |
Actual logs and execution reports for that attempt. |
runs/<run_id>/<stage>/debug/ |
Actual debug captures for that attempt, including retained state snapshots. |
runs/<run_id>/ |
Narratio staging, original tool bundles, and retained execution evidence. |
For example, the latest views use relative links within the session:
logs/polish -> ../runs/<run_id>/polish/logs/
debug/polish -> ../runs/<run_id>/polish/debug/
Each stage's latest attempt can belong to a different run. Producers write to the actual run paths; the symlinks provide operator navigation without duplicate diagnostic storage.
Narratio's lock and restore marker remain its own State. Their current placement can remain; introducing another root-level State directory is unnecessary for this feature. Generated records that describe an invocation belong with Logs; configuration actually supplied as an input follows its input or staging role.
Subprocesses may create internal working directories wherever their configuration directs. Narratio-selected staging destinations for result bundles remain valid under ADR 0002. Cache locations are documented as operator recommendations using each application's existing configuration surface.
Required Changes
Output paths and configuration
Centralize the revised paths in the existing path owner. Apply the flat artifact
rule to explicit artifacts, expanded families, extraction outputs, and session
bounds. Preserve logical source identities used by dependencies and publication.
Keep explicit analysis filenames and require flat destinations. Derive extraction
filenames from configured extraction keys with a .json extension, with an
explicit extraction filename override for naming preferences or collisions.
Retain session_bounds.json as the default bounds filename. Do not silently
rename conflicting files. Report conflicting producers and the shared destination
in configuration errors. Validate the effective configuration after composition
and family expansion.
Keep combined transcript destinations directly beneath transcripts/. Remove
obsolete directory scaffolding and reconcile configurable output paths with
their category. Local path changes do not require flattening explicitly
configured remote publish destinations.
Notarius result handling and reuse
Receive and validate the complete Notarius bundle in Narratio staging. Promote selected accepted payloads to their flat artifact destinations. Classify the receipt, warnings, rejection summaries, and diagnostics as Logs. Keep bundle indexes and other management metadata with the bundle or Narratio State according to their use.
Revise extraction evidence, catalog hydration, and resume checks to resolve
promoted products independently of the old artifacts/notarius/ layout. Retain
the identity, provenance, contract, and checksum evidence needed for current
reuse behavior. Historical bundles may remain available for inspection under
their existing retention semantics; relocating them beneath runs/ must not
silently shorten their lifetime through run cleanup.
Preserve ordinary resume and forced-extraction behavior. Subprocess caches remain under Notarius's control.
Inputs
Place prepared previous-session material beneath inputs/previous/. Place the
stable reference snapshots Narratio supplies to Notarius beneath
inputs/references/, retaining their content identities where needed for reuse.
Update preparation, input resolution, manifest references, and restore mappings.
Keep source configuration ownership and prepared-file validation unchanged.
Audio acquisition
Simplify acquisition to one prepared session copy under audio/. Remove the
separate Narratio S3 audio cache and routine retained spool copies, together with
their obsolete configuration, manifest bookkeeping, and cleanup controls.
Preparation may use temporary download files before installing validated audio.
Preserve reuse of valid prepared audio and update restore to use the same layout.
Recreating a session after its prepared audio is removed requires fetching the source again. Original recordings remain outside this cleanup change. Removing spool/cache handling does not authorize new deletion of prepared session audio. Subprocess caches remain independently configured and managed.
Logs and execution reports
Provide one consistent latest-attempt view per stage. Include failed attempts;
a skipped stage leaves its previous view intact. Distinguish multiple applications
or artifact invocations within a stage through their filenames. Route Seriatim
reports, Audita reports, Notarius diagnostic records, and Narratio invocation
records through this model. Treat restore execution as its own diagnostic group
under logs/restore/, using run-local evidence and a latest-attempt view.
Prevent a new attempt from presenting old diagnostic files as newly generated ones. At the start of each actual stage attempt, create its run-local log and debug directories and refresh both latest symlinks. An attempt without debug output exposes an empty debug directory rather than an older capture. Preserve subprocess errors and available evidence on failure. Retain existing captured-stream redaction and limits.
Keep historical files at their run paths; rotating the links does not move files. Run cleanup must preserve directories targeted by current latest links. Do not introduce an automatic history age or count limit. Publication and manifest diagnostic references use actual run paths. Adjust path handling for these Narratio-managed navigation links without relying on arbitrary symlink traversal.
Debug controls and placement
Add selective Narratio configuration for Audita and Notarius debug capture using their supported enabling CLI flags. An enabled Narratio selector requests capture; an omitted or false selector passes no debug override and preserves the application's configured/default behavior. Do not add a downstream disable flag.
Supply the actual run-local debug directories, exposed through debug/polish
and debug/extract. Control the destination when capture can be enabled through
downstream configuration as well, including Audita overlays, using supported
destination controls without changing capture enablement. Audita's debug
work-directory option selects diagnostic storage and belongs to this placement
responsibility.
Route Scriptorium's existing analysis and bounds render-debug products to the
run-local directories exposed through debug/analyze and debug/trim. Preserve
its existing per-artifact selection capability. Seriatim requires no invented
debug mode; its ordinary reports are
Logs. Subprocess-defined bundle contents and nested run directories are allowed
beneath the selected run-local debug root. Logs and Debug share the same
latest-attempt semantics and cleanup protection.
Keep new full Audita and Notarius debug bundles local in this feature. Preserve publication of currently declared diagnostics, including existing render-debug products. Enabling capture does not implicitly add full debug bundles to remote archives.
Debug controls govern capture when a stage executes. They do not independently require regeneration of valid substantive outputs. Document the existing force workflow for obtaining fresh diagnostic evidence when a stage would be reused. Preserve downstream failure semantics for diagnostic writes and the existing boundary of Narratio's captured-stream redaction.
Audita destination support
Request upstream support for independently selecting the transcript output,
ordinary report/log directory, and debug/retained working directory, then adopt
a supporting release. Audita currently ignores --report-dir when transcript
output is a file; honoring that destination is the principal missing capability.
Narratio continues to capture subprocess stdout and stderr itself.
Direct ordinary reports, including failure reports, to the actual polish log directory, the transcript to its staged product destination, and retained debug material to the actual polish debug directory. This feature uses the supported upstream controls in a clean cutover without version-based compatibility branches.
Manifest, publication, restore, and cleanup
Update all consumers of changed paths and file categories together. Preserve per-speaker and per-artifact completed-work reuse. Keep publication driven by declared products and diagnostics, with new full debug bundles remaining local. Restore new-format products and inputs to their new locations.
Define an explicit way to identify supported session state for the clean cutover. Document regeneration with the new version and manual configuration updates. Avoid accepting old state through path fallbacks or silently rewriting old local sessions or remote snapshots.
Preserve existing invalidation and failed-replacement behavior. Review retention by content role before changing cleanup targets, especially when durable products or historical bundles move. Run cleanup preserves the current log and debug link targets; explicit removal of an entire session removes its local views with it. Subprocess working State and Cache remain outside Narratio's cleanup responsibility.
Documentation and verification
Update configuration examples, the operator guide, adapter contracts, workspace
internals, and the architecture policy when implementation lands. Replace the
current architectural assumption that extraction requires a complete immutable
bundle beneath artifacts/ with the selected product-promotion model. Retire
policy and documentation for Narratio's removed audio cache/spool lifecycle and
document the managed latest-attempt symlinks. Keep per-application cache locations
as deployment recommendations owned by each application's configuration.
Use focused behavioral coverage for flat output placement, collision rejection, latest-attempt diagnostics on success and failure, debug selection and routing, unchanged completed-work reuse, failed replacement, and new-format publish/restore. Cover empty debug views after attempts without capture, preserved views for skipped stages, cleanup protection of latest targets, and audio acquisition, reuse, and restore without persistent spool/cache duplicates. Exercise cleanup against products and separately owned caches. Use temporary filesystems and subprocess doubles; default tests remain offline and independent of sibling checkouts or paid providers.