281 lines
15 KiB
Markdown
281 lines
15 KiB
Markdown
# Session workspace organization
|
|
|
|
## Purpose
|
|
|
|
Give operators predictable locations for session inputs, completed outputs,
|
|
logs, and debug evidence. Apply the taxonomy in
|
|
[ADR 0001](../adr/0001-classify-session-content.md) and the ownership boundary in
|
|
[ADR 0002](../adr/0002-assign-subprocess-content-ownership.md).
|
|
|
|
This roadmap defines the target behavior and required changes. Implementation
|
|
sequencing belongs in a subsequent implementation plan. The ADRs remain
|
|
Proposed pending their acceptance.
|
|
|
|
## Confirmed Requirements
|
|
|
|
- Completed artifacts occupy a flat `artifacts/` directory. Intermediate
|
|
products such as `session_bounds.json` belong here.
|
|
- Combined transcripts occupy `transcripts/` directly. Initial per-speaker
|
|
transcripts occupy a subdirectory beneath it.
|
|
- Prepared external audio occupies `audio/`; other prepared external inputs
|
|
occupy `inputs/`. Outputs consumed by later stages retain their output paths.
|
|
- Logs and Debug use matching latest-attempt views: `logs/<stage>` and
|
|
`debug/<stage>` are symlinks to the producing run's directories. Both views
|
|
advance when a stage starts an actual attempt, including attempts that fail.
|
|
A skipped stage leaves its previous views intact.
|
|
- Execution reports and invocation records are Logs. Historical Logs and Debug
|
|
remain under their producing runs, subject to cleanup.
|
|
- Narratio can selectively request supported subprocess debug capture through
|
|
configuration. Omission preserves application configuration and defaults;
|
|
Narratio needs no explicit downstream debug-disable override.
|
|
- Prepared session audio is the single retained Narratio-managed audio copy.
|
|
Remove Narratio's separate persistent audio cache and spool duplicates.
|
|
- Configurations that produce colliding artifact names or destinations fail
|
|
validation before execution. Validation covers the effective pipeline,
|
|
including expanded character families and built-in artifact destinations.
|
|
- Narratio selects input, output, log, and debug paths. Subprocess applications
|
|
own their working State and Cache through their supported configuration.
|
|
- A dedicated cache root such as `/var/cache/notarius` is a deployment
|
|
recommendation. The application owns its cache defaults, internal layout,
|
|
identity, and reuse semantics. Narratio session cleanup respects that boundary.
|
|
- Narratio retains its own manifest, recovery records, and managed staging State.
|
|
- This is a clean cutover for newly prepared sessions. Existing-session migration
|
|
and backward compatibility shims are outside the feature.
|
|
- Treatment of older completed files after invalidation or failed replacement
|
|
remains unchanged, including their availability through manifest-backed
|
|
resolution. Directory reorganization does not authorize new stale-file
|
|
deletion, automatic fallback, or rollback behavior.
|
|
|
|
## Current Baseline
|
|
|
|
The [operator guide](../operations.md#local-state-layout) owns the current
|
|
physical layout. Relevant differences from the target are:
|
|
|
|
- Extraction moves the complete Notarius bundle to
|
|
`artifacts/notarius/<run_id>/`. Resume evidence and artifact resolution depend
|
|
on that bundle and its index.
|
|
- Merge and normalize promote Seriatim execution reports into `artifacts/`.
|
|
- Analysis output paths and character-family patterns can introduce nested
|
|
artifact directories. Output-path configuration also permits locations that
|
|
need reconciliation with the selected directory categories.
|
|
- Standard combined transcript paths already match the target. Layout creation
|
|
also creates an unused `transcripts/trimmed/` directory.
|
|
- Most subprocess diagnostics live under individual run directories. Root-level
|
|
log paths lack a consistent latest-attempt view. Audita reports sit beside
|
|
its staged transcript, and restore writes a separate top-level report.
|
|
- Prepared previous-session inputs occupy `previous/`. Narratio-owned reference
|
|
snapshots supplied to Notarius occupy session `cache/notarius/references/`.
|
|
These snapshots are distinct from Notarius's own execution caches.
|
|
- Audita debug placement follows downstream configuration. Narratio does not
|
|
expose Notarius debug capture, and Scriptorium render-debug placement varies.
|
|
- Narratio maintains separate S3 audio cache and spool paths in addition to the
|
|
prepared session audio copy.
|
|
|
|
## Target Layout And Ownership
|
|
|
|
Retain the configurable workspace root and its current system-wide default,
|
|
`/var/lib/narratio`. Sessions remain under
|
|
`/var/lib/narratio/work/<campaign>/<session>` by default. The following paths
|
|
express the selected categories:
|
|
|
|
| Path | Contents and role |
|
|
| --- | --- |
|
|
| `manifest.json` | Narratio's durable progress and output references. |
|
|
| `artifacts/<filename>` | Completed artifact payloads, including session bounds. |
|
|
| `transcripts/<filename>` | Completed combined transcripts and rendered transcripts. |
|
|
| `transcripts/raw/<speaker>.json` | Completed initial per-speaker transcripts. |
|
|
| `audio/` | Prepared external audio. |
|
|
| `inputs/` | Prepared configuration, references, and previous-session inputs. |
|
|
| `logs/<stage>` | Symlink to the latest attempt's run-local logs and execution reports. |
|
|
| `debug/<stage>` | Symlink to the latest attempt's run-local debug directory, empty when no capture is produced. |
|
|
| `runs/<run_id>/<stage>/logs/` | Actual logs and execution reports for that attempt. |
|
|
| `runs/<run_id>/<stage>/debug/` | Actual debug captures for that attempt, including retained state snapshots. |
|
|
| `runs/<run_id>/` | Narratio staging, original tool bundles, and retained execution evidence. |
|
|
|
|
For example, the latest views use relative links within the session:
|
|
|
|
```text
|
|
logs/polish -> ../runs/<run_id>/polish/logs/
|
|
debug/polish -> ../runs/<run_id>/polish/debug/
|
|
```
|
|
|
|
Each stage's latest attempt can belong to a different run. Producers write to
|
|
the actual run paths; the symlinks provide operator navigation without duplicate
|
|
diagnostic storage.
|
|
|
|
Narratio's lock and restore marker remain its own State. Their current placement
|
|
can remain; introducing another root-level State directory is unnecessary for
|
|
this feature. Generated records that describe an invocation belong with Logs;
|
|
configuration actually supplied as an input follows its input or staging role.
|
|
|
|
Subprocesses may create internal working directories wherever their configuration
|
|
directs. Narratio-selected staging destinations for result bundles remain valid
|
|
under ADR 0002. Cache locations are documented as operator recommendations using
|
|
each application's existing configuration surface.
|
|
|
|
## Required Changes
|
|
|
|
### Output paths and configuration
|
|
|
|
Centralize the revised paths in the existing path owner. Apply the flat artifact
|
|
rule to explicit artifacts, expanded families, extraction outputs, and session
|
|
bounds. Preserve logical source identities used by dependencies and publication.
|
|
Keep explicit analysis filenames and require flat destinations. Derive extraction
|
|
filenames from configured extraction keys with a `.json` extension, with an
|
|
explicit extraction filename override for naming preferences or collisions.
|
|
Retain `session_bounds.json` as the default bounds filename. Do not silently
|
|
rename conflicting files. Report conflicting producers and the shared destination
|
|
in configuration errors. Validate the effective configuration after composition
|
|
and family expansion.
|
|
|
|
Keep combined transcript destinations directly beneath `transcripts/`. Remove
|
|
obsolete directory scaffolding and reconcile configurable output paths with
|
|
their category. Local path changes do not require flattening explicitly
|
|
configured remote publish destinations.
|
|
|
|
### Notarius result handling and reuse
|
|
|
|
Receive and validate the complete Notarius bundle in Narratio staging. Promote
|
|
selected accepted payloads to their flat artifact destinations. Classify the
|
|
receipt, warnings, rejection summaries, and diagnostics as Logs. Keep bundle
|
|
indexes and other management metadata with the bundle or Narratio State according
|
|
to their use.
|
|
|
|
Revise extraction evidence, catalog hydration, and resume checks to resolve
|
|
promoted products independently of the old `artifacts/notarius/` layout. Retain
|
|
the identity, provenance, contract, and checksum evidence needed for current
|
|
reuse behavior. Historical bundles may remain available for inspection under
|
|
their existing retention semantics; relocating them beneath `runs/` must not
|
|
silently shorten their lifetime through run cleanup.
|
|
|
|
Preserve ordinary resume and forced-extraction behavior. Subprocess caches remain
|
|
under Notarius's control.
|
|
|
|
### Inputs
|
|
|
|
Place prepared previous-session material beneath `inputs/previous/`. Place the
|
|
stable reference snapshots Narratio supplies to Notarius beneath
|
|
`inputs/references/`, retaining their content identities where needed for reuse.
|
|
Update preparation, input resolution, manifest references, and restore mappings.
|
|
Keep source configuration ownership and prepared-file validation unchanged.
|
|
|
|
### Audio acquisition
|
|
|
|
Simplify acquisition to one prepared session copy under `audio/`. Remove the
|
|
separate Narratio S3 audio cache and routine retained spool copies, together with
|
|
their obsolete configuration, manifest bookkeeping, and cleanup controls.
|
|
Preparation may use temporary download files before installing validated audio.
|
|
Preserve reuse of valid prepared audio and update restore to use the same layout.
|
|
|
|
Recreating a session after its prepared audio is removed requires fetching the
|
|
source again. Original recordings remain outside this cleanup change. Removing
|
|
spool/cache handling does not authorize new deletion of prepared session audio.
|
|
Subprocess caches remain independently configured and managed.
|
|
|
|
### Logs and execution reports
|
|
|
|
Provide one consistent latest-attempt view per stage. Include failed attempts;
|
|
a skipped stage leaves its previous view intact. Distinguish multiple applications
|
|
or artifact invocations within a stage through their filenames. Route Seriatim
|
|
reports, Audita reports, Notarius diagnostic records, and Narratio invocation
|
|
records through this model. Treat restore execution as its own diagnostic group
|
|
under `logs/restore/`, using run-local evidence and a latest-attempt view.
|
|
|
|
Prevent a new attempt from presenting old diagnostic files as newly generated
|
|
ones. At the start of each actual stage attempt, create its run-local log and
|
|
debug directories and refresh both latest symlinks. An attempt without debug
|
|
output exposes an empty debug directory rather than an older capture. Preserve
|
|
subprocess errors and available evidence on failure. Retain existing
|
|
captured-stream redaction and limits.
|
|
|
|
Keep historical files at their run paths; rotating the links does not move files.
|
|
Run cleanup must preserve directories targeted by current latest links. Do not
|
|
introduce an automatic history age or count limit. Publication and manifest
|
|
diagnostic references use actual run paths. Adjust path handling for these
|
|
Narratio-managed navigation links without relying on arbitrary symlink traversal.
|
|
|
|
### Debug controls and placement
|
|
|
|
Add selective Narratio configuration for Audita and Notarius debug capture using
|
|
their supported enabling CLI flags. An enabled Narratio selector requests
|
|
capture; an omitted or false selector passes no debug override and preserves the
|
|
application's configured/default behavior. Do not add a downstream disable flag.
|
|
|
|
Supply the actual run-local debug directories, exposed through `debug/polish`
|
|
and `debug/extract`. Control the destination when capture can be enabled through
|
|
downstream configuration as well, including Audita overlays, using supported
|
|
destination controls without changing capture enablement. Audita's debug
|
|
work-directory option selects diagnostic storage and belongs to this placement
|
|
responsibility.
|
|
|
|
Route Scriptorium's existing analysis and bounds render-debug products to the
|
|
run-local directories exposed through `debug/analyze` and `debug/trim`. Preserve
|
|
its existing per-artifact selection capability. Seriatim requires no invented
|
|
debug mode; its ordinary reports are
|
|
Logs. Subprocess-defined bundle contents and nested run directories are allowed
|
|
beneath the selected run-local debug root. Logs and Debug share the same
|
|
latest-attempt semantics and cleanup protection.
|
|
|
|
Keep new full Audita and Notarius debug bundles local in this feature. Preserve
|
|
publication of currently declared diagnostics, including existing render-debug
|
|
products. Enabling capture does not implicitly add full debug bundles to remote
|
|
archives.
|
|
|
|
Debug controls govern capture when a stage executes. They do not independently
|
|
require regeneration of valid substantive outputs. Document the existing force
|
|
workflow for obtaining fresh diagnostic evidence when a stage would be reused.
|
|
Preserve downstream failure semantics for diagnostic writes and the existing
|
|
boundary of Narratio's captured-stream redaction.
|
|
|
|
### Audita destination support
|
|
|
|
Request upstream support for independently selecting the transcript output,
|
|
ordinary report/log directory, and debug/retained working directory, then adopt
|
|
a supporting release. Audita currently ignores `--report-dir` when transcript
|
|
output is a file; honoring that destination is the principal missing capability.
|
|
Narratio continues to capture subprocess stdout and stderr itself.
|
|
|
|
Direct ordinary reports, including failure reports, to the actual polish log
|
|
directory, the transcript to its staged product destination, and retained debug
|
|
material to the actual polish debug directory. This feature uses the supported
|
|
upstream controls in a clean cutover without version-based compatibility branches.
|
|
|
|
### Manifest, publication, restore, and cleanup
|
|
|
|
Update all consumers of changed paths and file categories together. Preserve
|
|
per-speaker and per-artifact completed-work reuse. Keep publication driven by
|
|
declared products and diagnostics, with new full debug bundles remaining local.
|
|
Restore new-format products and inputs to their new locations.
|
|
|
|
Define an explicit way to identify supported session state for the clean
|
|
cutover. Document regeneration with the new version and manual configuration
|
|
updates. Avoid accepting old state through path fallbacks or silently rewriting
|
|
old local sessions or remote snapshots.
|
|
|
|
Preserve existing invalidation and failed-replacement behavior. Review retention
|
|
by content role before changing cleanup targets, especially when durable products
|
|
or historical bundles move. Run cleanup preserves the current log and debug
|
|
link targets; explicit removal of an entire session removes its local views
|
|
with it. Subprocess working State and Cache remain outside Narratio's cleanup
|
|
responsibility.
|
|
|
|
### Documentation and verification
|
|
|
|
Update configuration examples, the operator guide, adapter contracts, workspace
|
|
internals, and the architecture policy when implementation lands. Replace the
|
|
current architectural assumption that extraction requires a complete immutable
|
|
bundle beneath `artifacts/` with the selected product-promotion model. Retire
|
|
policy and documentation for Narratio's removed audio cache/spool lifecycle and
|
|
document the managed latest-attempt symlinks. Keep per-application cache locations
|
|
as deployment recommendations owned by each application's configuration.
|
|
|
|
Use focused behavioral coverage for flat output placement, collision rejection,
|
|
latest-attempt diagnostics on success and failure, debug selection and routing,
|
|
unchanged completed-work reuse, failed replacement, and new-format publish/restore.
|
|
Cover empty debug views after attempts without capture, preserved views for
|
|
skipped stages, cleanup protection of latest targets, and audio acquisition,
|
|
reuse, and restore without persistent spool/cache duplicates. Exercise cleanup
|
|
against products and separately owned caches. Use temporary filesystems and
|
|
subprocess doubles; default tests remain offline and independent
|
|
of sibling checkouts or paid providers.
|