From fe9817f354d521f7a761cd4860d0d2bb4fbb93a0 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 4 Oct 2026 12:28:09 +0000 Subject: [PATCH] Plan the ConfigKit upgrade and workspace development --- docs/adr/0001-classify-session-content.md | 84 +++++ ...002-assign-subprocess-content-ownership.md | 76 +++++ docs/roadmap/audita.md | 175 ++++++++++ docs/roadmap/config.md | 202 +++++++++++ docs/roadmap/implementation.md | 319 ++++++++++++++++++ docs/roadmap/workspace.md | 280 +++++++++++++++ 6 files changed, 1136 insertions(+) create mode 100644 docs/adr/0001-classify-session-content.md create mode 100644 docs/adr/0002-assign-subprocess-content-ownership.md create mode 100644 docs/roadmap/audita.md create mode 100644 docs/roadmap/config.md create mode 100644 docs/roadmap/implementation.md create mode 100644 docs/roadmap/workspace.md diff --git a/docs/adr/0001-classify-session-content.md b/docs/adr/0001-classify-session-content.md new file mode 100644 index 0000000..f868676 --- /dev/null +++ b/docs/adr/0001-classify-session-content.md @@ -0,0 +1,84 @@ +# 0001. Classify session content by purpose and lifecycle + +## Status + +Proposed + +## Date + +2026-10-01 + +## Context + +Narratio coordinates applications that produce transcripts, artifacts, +diagnostics, and working files. The current session layout mixes some of these +roles, including complete Notarius bundles under `artifacts/`. Operators use +transcripts and artifacts to evaluate results during development and need +predictable locations for diagnostics. + +A shared classification model will guide the workspace design and subsequent +adapter changes. The [operator guide](../operations.md#local-state-layout) +describes the implemented layout. + +## Decision + +Classify content by its primary purpose and lifecycle: + +| Category | Purpose | +| --- | --- | +| Inputs | Material consumed by an application, including audio, configuration, and reference documents. | +| Outputs | Completed substantive products, including intermediate artifacts exposed for further use. | +| Logs | Records of execution, including process logs, execution reports, warnings, and diagnostic summaries. | +| Cache | Reusable information whose absence permits ordinary recomputation or retrieval. | +| State | Application condition, progress, recovery records, and unfinished work. | + +Classification is relative to the consuming application. An output of one stage +can serve as an input to another while retaining its output location. Completed +outputs reused to skip work remain Outputs. Narratio's manifest is State and +records whether those outputs are reusable. + +Classify checkpoints according to their recovery behavior. A checkpoint needed +to continue a particular interrupted operation is State. A saved work result +that can be reused or recomputed is Cache. Recomputation cost informs retention +policy. File-specific classification remains necessary for stores serving both +purposes. + +Treat Debug as a capture and retention mode that can include detailed Logs, +snapshots of State, and other diagnostic evidence. Retained debug snapshots have +a diagnostic role; live recovery records retain their State role. + +Apply these directory principles to Narratio: + +- Store completed artifacts, including intermediate artifacts like + `session_bounds.json`, directly in a flat `artifacts/` directory. +- Store combined transcripts directly in `transcripts/`, with a subdirectory + for the initial per-speaker transcripts. +- Store prepared external audio in `audio/` and other prepared external inputs + in `inputs/`. +- Organize Logs under `logs/` by stage. Execution reports belong to this category. +- Organize Cache under a cache root by the application that owns its format and + reuse behavior. +- Treat scratch files and unfinished outputs as State until completion permits + promotion to Outputs. + +## Alternatives Considered + +- Group outputs by producer. This exposes subprocess packaging to operators; + flat artifact and transcript locations better support comparison and review. +- Group Logs by application. Stage names match the operator's workflow and + collect diagnostics from stages that invoke multiple applications. +- Classify all checkpoints as State. Recovery behavior provides a more useful + distinction for eviction and reuse decisions. +- Treat all Debug material as Logs. A separate debug location can accommodate + retained state snapshots and larger diagnostic bundles when needed. + +## Consequences + +Each managed file family needs an identified role, owner, scope, and retention +rule. Subprocess bundles require classification before their contents are +placed in Narratio's output and diagnostic directories. + +Physical roots, retention rules, the mechanism for presenting latest logs, +debug placement, and ambiguous checkpoint classifications remain open. The file +inventory and subsequent roadmap will resolve these choices and define the +migration. Current versus historical views are separate from content categories. diff --git a/docs/adr/0002-assign-subprocess-content-ownership.md b/docs/adr/0002-assign-subprocess-content-ownership.md new file mode 100644 index 0000000..a8db506 --- /dev/null +++ b/docs/adr/0002-assign-subprocess-content-ownership.md @@ -0,0 +1,76 @@ +# 0002. Assign subprocess content ownership + +## Status + +Proposed + +## Date + +2026-10-03 + +## Context + +[ADR 0001](0001-classify-session-content.md) defines content categories and +operator-facing directory principles. Narratio also needs an ownership boundary +for the applications it invokes. That boundary determines who selects locations, +manages retained files, and controls cleanup. + +The current subprocess workflows can execute with fresh working state. Notarius +can reuse persistent chunk-plan and checkpoint caches. Narratio maintains its own +session progress and recovery records. + +## Decision + +Assign responsibility by content category: + +| Category | Responsibility | +| --- | --- | +| Inputs | Narratio supplies paths to prepared inputs, transcripts, and artifacts. | +| Outputs | Narratio selects destinations and promotes completed products into the output locations defined by ADR 0001. | +| Logs | Narratio selects stage-based destinations and captures subprocess output streams. | +| Debug | Narratio selects the destination when capture is enabled; the subprocess defines the diagnostic contents. | +| Subprocess State | Each application manages its working directory, temporary files, and execution lifecycle through its own configuration and defaults. | +| Subprocess Cache | Each application manages its configured persistent cache, including its format, identity, and reuse behavior. | + +Narratio retains ownership of its own State, including the session manifest, +recovery bookkeeping, and staging files it manages. + +Recommend a dedicated cache root for each cache-aware application, configurable +through that application's supported controls. For system-wide deployments, +`/var/cache/` is a suggested location; `/var/cache/notarius` is an +example for Notarius. Each application defines its cache configuration, defaults, +and internal layout. Operators select locations and retention settings through +those controls. Narratio's session cleanup leaves subprocess caches under their +separate lifecycle policies. + +Subprocesses retain ownership of their file formats and bundle structures. +Narratio can receive a complete result bundle at a selected staging destination +and promote the selected products into its canonical output directories. +Diagnostic snapshots of subprocess State follow the Debug destination selected +by Narratio. + +Narratio's `runs/` directory can hold staging outputs and retained execution +evidence. Subprocess working directories follow the application's configuration +and may reside elsewhere. Narratio cleanup manages Narratio-owned material; +subprocess State and Cache follow their separate lifecycle policies. + +## Alternatives Considered + +- Place all subprocess working files and caches inside each Narratio session. + This makes sessions easier to collect as a unit, but couples application + internals and cache retention to Narratio's session lifecycle. +- Allow subprocesses to choose output, log, and debug destinations independently. + This simplifies invocation but scatters the material operators inspect. +- Give Narratio control of downstream bundle formats. This simplifies some + destination mappings but increases integration coupling. Selecting and + promoting products preserves application ownership of those formats. + +## Consequences + +Adapters need supported controls for Narratio-selected output, log, and debug +locations. Upstream changes may be needed where those destinations are coupled +to application working directories or to one another. + +Workspace layout and file classification can proceed independently of subprocess +working-directory conventions. Exact debug placement, retained diagnostic views, +and migration steps remain for the workspace design and roadmap. diff --git a/docs/roadmap/audita.md b/docs/roadmap/audita.md new file mode 100644 index 0000000..d4d0ab4 --- /dev/null +++ b/docs/roadmap/audita.md @@ -0,0 +1,175 @@ +# Adopt Audita v3 and independent diagnostic destinations + +## Purpose + +Adopt Audita v3.0.0 for transcript polishing and use its independent output +destinations to implement the Audita portion of the +[session workspace roadmap](workspace.md). Operators should find the completed +transcript, ordinary reports, and optional debug evidence in their respective +Narratio locations. + +This document defines the Audita adoption scope and target. The workspace +roadmap owns shared layout, latest-attempt views, and retention behavior. Its +referenced ADRs remain Proposed. Implementation sequencing belongs in a +subsequent implementation plan. + +## Reviewed Upstream Release + +The reviewed Audita v3.0.0 tag identifies commit +`3b4933256eff84327b812e7ae12babf743bc04a6`. Its +[release notes](https://gitea.maximumdirect.net/eric/audita/src/tag/v3.0.0/docs/releases/v3.0.0.md), +[CLI reference](https://gitea.maximumdirect.net/eric/audita/src/tag/v3.0.0/docs/cli.md), +[configuration reference](https://gitea.maximumdirect.net/eric/audita/src/tag/v3.0.0/docs/config.md), +and [operations guide](https://gitea.maximumdirect.net/eric/audita/src/tag/v3.0.0/docs/operations.md) +own the downstream contracts. + +Audita now honors an explicit `--report-dir` alongside file transcript output. +Its existing `--work-dir` selects the parent of retained debug captures, with +capture enabled separately through configuration or `--debug`. These controls +satisfy Narratio's requested destination separation. Further upstream changes +are unnecessary for this scope. + +## Current Baseline + +Narratio's [Audita integration](../integrations/audita.md) and +[polish stage](../internal/stage-polish.md) describe the implemented behavior: + +- The adapter invokes `process` with a transcript, glossary, and staged transcript + destination. It supports independent downstream configuration, overlay, + profile, module, description, and output-schema selectors. +- The adapter supplies no reporting, debug, or work-directory flags. Audita + controls debug capture and placement through its own configuration. +- Ordinary reports are discovered beside the staged transcript. The polish + stage validates returned report paths against those adjacent filenames. +- Only the validated polished transcript becomes a canonical stage output. + Reports, captured streams, and the generated invocation record are diagnostics. +- Selected credential references use Narratio's shared exact-name forwarding and + captured-stream redaction boundary. + +The existing invocation remains usable with Audita v3 after deployment-side +profile migration. Narratio changes are needed to request and consume the +new destination separation. + +## Targeted End State + +### Products and diagnostics + +Narratio supplies separate destinations for each actual polish attempt: + +| Content | Destination and control | +| --- | --- | +| Corrected transcript | Run-local staging via `--output`; validated and promoted to `transcripts/polished.json` on success. | +| Ordinary reports | `runs//polish/logs/` via `--report-dir`. | +| Captured stdout, stderr, and Narratio invocation record | The same run-local polish log directory. | +| Optional debug evidence | `runs//polish/debug/` via `--work-dir`; Audita may create its own `run-*` subdirectory beneath it. | + +The workspace roadmap's `logs/polish` and `debug/polish` links expose the latest +attempt. Narratio supplies actual run destinations to Audita and records those +paths in manifest diagnostics. Shared link updates and cleanup protection are +implemented by the workspace work rather than separately inside the adapter. + +### Capture selection + +Add an optional `pipeline.audita.debug` boolean, defaulting to false. A true +value passes `--debug`. False or omission supplies no capture override and +preserves Audita configuration and overlay behavior; it does not disable debug +enabled downstream. + +Supply `--work-dir` for each executing attempt so downstream-enabled capture +also lands in the Narratio-selected debug directory. Selecting that destination +alone does not enable capture. Ordinary production operation retains Audita's +default reporting and debug settings. Narratio adds no reporting enable/disable +flag or debug-disable flag. + +Debug selection affects diagnostics for an executing stage. Changing it alone +does not invalidate a completed transcript. Operators use the existing force +workflow when fresh polish or debug evidence is needed. + +### Ownership and compatibility + +Use a clean adoption of Audita v3's CLI contract, without version-dependent +adapter branches or legacy profile conversion. Audita continues to own its +configuration, profiles, prompts, runtime State, and diagnostic contents. +Narratio owns supplied destinations and transcript promotion. + +Keep existing selectors and shared credentials such as `OPENROUTER_API_KEY` and +`RAKESTRAWHOME_API_KEY` under their configured exact names. Audita's examples +do not require Narratio to adopt application-specific credential aliases. +Operators continue to list the credential references required by all configured +Audita backends. + +Audita is a separately installed executable. Its adoption of ConfigKit v0.5.0 +and PromptKit v0.12.0 does not require changing Narratio's own library +dependencies as part of this feature. + +## Required Changes + +### Deployment configuration migration + +Document the manual migration of Audita-owned PromptKit files: + +- Wrap profile definitions under keyed `profiles:` mappings, remove inner `id` + fields, and replace `base_profile` with `extends`. Preserve selected profile + IDs when appropriate. +- Review explicit zeros, omitted values, and nulls against the new generation + control semantics. Validate the final configuration root, including unused + definitions and backend references. +- Rename any backend literally named `extends` and update its references. +- Validate production configuration and each used testing overlay with Audita's + offline `config validate` command. Audita application configuration continues + to use `version: 2`. + +Keep the detailed downstream format owned by the pinned Audita documentation. +Narratio does not inspect, rewrite, or automatically migrate these files. +Deployment can migrate Audita before the workspace changes land. + +### Adapter and polish stage + +Carry explicit report and debug destinations through the Audita request and +construct the supported flags inside the adapter. Wire the debug selector +through configuration and runner composition. + +Discover the existing three report filenames in the requested report directory. +Update polish's expected-path validation to the same location. Update invocation +records and safe result metadata to describe the selected destinations and +capture request. Preserve available failure evidence and the original subprocess +error. + +Keep existing transcript validation, report read bounds, timeouts, credential +forwarding, and captured-stream redaction. Reports remain optional diagnostics; +early failures or downstream-disabled reporting may produce none. Exit-zero +runs with recoverable model errors remain successful. A nonzero exit, including +a late report-write failure, never authorizes transcript promotion. Narratio +does not add report-schema parsing or report-status success gates. + +### Workspace integration and lifecycle + +Use the shared workspace path owner for run-local log/debug destinations and +the agreed latest-attempt behavior. Preserve previous views on skipped stages; +actual attempts refresh both views, including on failure. An attempt without +debug capture exposes an empty current debug directory. + +Keep full Audita debug bundles local in this feature. Preserve existing declared +diagnostic publication and apply the workspace roadmap's retention protection +to latest link targets. This feature inherits that roadmap's clean session +cutover and unchanged treatment of older completed outputs after invalidation +or failed replacement. + +### Documentation and verification + +When implemented, update the configuration reference and examples, Audita +integration contract, polish internals, and operator guide. Reference Audita v3's +profile migration and explain the shared Audita/Notarius overlay development +workflow. Downstream configuration edits behind unchanged selectors still +require explicitly forcing the affected stage to replace reused outputs. + +Revise the workspace roadmap's upstream-request description when this adoption +is incorporated; Audita v3.0.0 supplies the previously missing capability. + +Use focused offline tests for separate transcript/report/debug destinations, +enable-only debug selection, capture enabled by downstream configuration, +available failure reports, and preservation of nonzero-exit behavior. Verify +that reports remain diagnostics and only a successful validated transcript is +promoted. Shared latest-view and cleanup tests belong to the workspace change. +Default Narratio tests use subprocess doubles and temporary filesystems and +remain independent of sibling checkouts, real credentials, and paid providers. diff --git a/docs/roadmap/config.md b/docs/roadmap/config.md new file mode 100644 index 0000000..935bc5c --- /dev/null +++ b/docs/roadmap/config.md @@ -0,0 +1,202 @@ +# Adopt ConfigKit v0.5.0 + +## Purpose + +Upgrade Narratio from ConfigKit v0.4.2 to v0.5.0 and integrate its native +sibling inheritance with Narratio's configuration validation and inspection. +Keep the upgrade focused on the existing loading model and the adjustments +needed to consume the new release correctly. + +This adoption precedes [Audita adoption](audita.md) and the +[workspace reorganization](workspace.md). This document defines the target +and required changes; [the implementation plan](implementation.md) defines +the steps for reaching that target. + +## Reviewed Upstream Release + +ConfigKit v0.5.0 identifies commit +`dc09959cca4e105f49ec316c6d4ba585bd2340bd`. The +[release notes](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/releases/v0.5.0.md), +[configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/config.md), +and [consumer guide](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/consumers/README.md) +own the upstream behavior. + +The release introduces native `extends` relationships between sibling mappings, +layered loading with ordered underlays, caller-owned filesystem and mapping +sources, and passive source-resource contexts. Inheritance preserves explicit +presence, provenance, and secret classification. + +The existing loading API remains available. ConfigKit's minimum Go line, YAML +dependency, and LGPL-3.0-or-later license are unchanged. Narratio can retain its +current Go toolchain and ordinary loading calls. + +## Current Baseline + +The [configuration reference](../config.md) owns Narratio's current public +semantics; [configuration internals](../internal/configuration.md) describe +loading and inspection. + +- Pipeline loading uses ConfigKit with explicit additive imports and one + selected overlay, typed defaults, recorded resolution, and domain validation. +- Campaign, session, and canonical-party inputs use separate single-source + loads. Secret-directory bootstrap remains an explicit invocation operation. +- Fields with root-relative pipeline path semantics retain that base when their + declarations move into imports. For example, enabled Notarius configuration + and working-directory paths resolve from the root pipeline directory. +- Narratio adds a declaration-ownership check: each artifact or artifact-family + key may be declared by only one base file. Overlay modifications are allowed. +- Source inspection and artifact catalogs consume retained ConfigKit metadata. + Effective show, diff, and digest projections describe the resolved domain + configuration rather than source layout. + +The review selected v0.5.0 through a temporary module file and ran Narratio's +existing test suite successfully. An additional probe exposed an integration +gap: a child artifact inheriting from a parent in another imported file is +rejected as declared by multiple base files. The same definitions in one file +load successfully. The check currently counts inherited history contributions +as if they were direct declarations. + +## Targeted End State + +### Native inheritance + +Use ConfigKit's built-in sibling inheritance wherever its input contract and +Narratio's domain schema permit it. ConfigKit resolves `extends` after source +composition and overlay precedence, before typed decoding and application +resolution. Narratio receives the resulting typed configuration. + +Inheritance can reduce repetition between artifacts and artifact families, +including definitions in separate imported files. Distinct child keys retain +their identities. Each resulting entry undergoes ordinary Narratio validation, +family expansion, dependency resolution, and execution selection. + +Parent entries remain ordinary domain entries. They must satisfy applicable +validation and participate in the effective configuration. This adoption does +not introduce abstract artifacts or a separate template registry. Operators +must override inherited fields where needed to keep the resulting configuration +valid, such as output destinations. + +Inherited false, zero, empty, and null input retains ConfigKit's explicit +presence semantics. Narratio's absence-based defaults must not refill such +values. Normal domain validation still determines whether a resulting value +is acceptable. + +### Declaration ownership and provenance + +Distinguish direct base declarations from inherited contributions. A child +declared in one file may inherit from a separately declared parent in another +file without becoming a duplicate declaration. + +Retain the existing rejection of one artifact or family key directly declared +by multiple base files, including disjoint contributions and declarations +later removed by an overlay. Inheritance is not permission to merge multiple +direct declarations of the same key. + +Keep defining-file ownership distinct from value-contributor provenance. +Existing source and artifact inspection must describe inherited contributors +accurately without presenting them as conflicting declarations. ConfigKit's +retained metadata remains the source of that evidence. + +For generated family members, the family's defining source identifies the +child family declaration; inherited parent files remain contributor evidence. +Keep existing logical artifact IDs and inspection interfaces. + +Effective configuration, diffs, and digests continue to reflect resolved +domain values. Equivalent inherited and explicitly authored configurations +should have equivalent effective projections, regardless of their provenance. +Stage reuse continues to use existing semantic evidence. + +### Loading and ownership boundaries + +Retain ordinary ConfigKit loading, centralized typed defaults, imports, selected +overlays, domain resolution, credential bootstrap, and exact-name credential +forwarding. Preserve current path bases and read-only inspection workflows. + +Layered loading, embedded configuration sources, programmatic mapping sources, +and source-relative resource rebasing are available upstream but are outside +this focused adoption. Narratio continues to own its domain rules; ConfigKit +owns structural inheritance, presence, provenance, and secret resolution. + +## Required Changes + +### Dependency and attribution + +Update `go.mod` and `go.sum` to ConfigKit v0.5.0. Update its entry in +[attribution](../oss/attribution.md) to the selected version and the reviewed +commit's source and notice links. Existing applicable license texts can remain. + +### Configuration integration + +Adjust the existing duplicate-definition check to identify direct declarations +using public ConfigKit history and origin metadata. Preserve genuine duplicate +detection while excluding structural transfers from declaration counting. +Use the completed load's retained evidence rather than reparsing files or +performing a second load that could resolve superseded credential references. + +Review source ownership and artifact-definition projections against the new +inheritance metadata. Update only where needed to report contributors correctly. +Keep effective rendering and digest construction grounded in the existing safe +typed projection, with credential identities rather than resolved values. + +Use ConfigKit's native operation directly. No application-owned inheritance +engine, collection registration, alternative keyword, or compatibility loader +is needed. + +### Compatibility and operator guidance + +Document that `extends` is now reserved structural input throughout ConfigKit +loads, including dynamic mappings such as artifact IDs and prompt variables. +Operators using that key as ordinary data must rename it. Inheritance is active +without an opt-in switch; imports and overlay-file selection remain separately +controlled as before. + +Explain native sibling lookup, inherited explicit presence, and the requirement +that parent and child entries satisfy Narratio's domain rules. Link to the +pinned upstream contract for detailed inheritance semantics. Clarify that a +selected overlay may contribute sibling inheritance directives; this does not +add inheritance or stacking between overlay files. + +Update the configuration reference, internal configuration guide, and a small +maintained example when implementation lands. Preserve each field's current +path-resolution rule when its value is inherited. Record the reserved-key +compatibility impact in the eventual release note; automatic configuration +conversion is outside this feature. + +### Verification + +Add focused Narratio coverage for the confirmed cross-file artifact inheritance +gap and the corresponding artifact-family behavior. Retain coverage for real +duplicate declarations, including those hidden by overlay replacement or +removal. Verify inherited explicit values against Narratio's defaulting rules, +credential safety, contributor reporting, and equivalent effective projections. + +Concentrate tests on Narratio's integration behavior. ConfigKit's own tests own +its generic inheritance algorithm and source lifecycle. Narratio's default +tests remain offline, use temporary configuration bundles and controlled secret +values, and do not depend on the sibling ConfigKit checkout. Run the existing +suite against the selected released module before completing the upgrade. + +## Completion Criteria + +Narratio selects the released ConfigKit v0.5.0 module without a local checkout +replacement. Cross-file artifact and family inheritance works through the +ordinary loader, while genuinely duplicated base declarations remain invalid. +Inspection distinguishes declaration ownership from inherited contributors; +effective projections and credential handling retain their existing semantics. + +Maintained guidance and examples explain native inheritance and the reserved +`extends` key. Attribution identifies the selected upstream revision. Focused +regression coverage and the existing suite pass without implementing Audita or +workspace changes as part of this upgrade. + +## Relationship To Subsequent Work + +ConfigKit adoption provides a verified configuration baseline for the later +Audita and workspace features. It makes no changes to subprocess destinations, +session layout, or cleanup behavior. + +Audita adoption then migrates downstream profiles and adds adapter support for +explicit report/debug destinations. Audita owns its separate ConfigKit and +PromptKit dependencies. Shared latest-attempt links, final workspace paths, and +retention protection remain part of the workspace reorganization, so those +facilities are implemented once at their shared owner. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..efba12d --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,319 @@ +# ConfigKit v0.5.0 implementation plan + +## Scope and authority + +Implement [the ConfigKit roadmap](config.md). It owns the target, compatibility +choices, and rationale. This plan covers Narratio's upgrade from v0.4.2 to +v0.5.0 and integration with native sibling inheritance. Audita adoption and +workspace reorganization remain subsequent, separately planned work. + +The reviewed upstream revision is +`dc09959cca4e105f49ec316c6d4ba585bd2340bd`, tagged `v0.5.0`. +Use the released module, not a local replacement. The pinned +[configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/config.md) +and [consumer guide](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/consumers/README.md) +own upstream semantics. Read their sibling-inheritance and metadata sections +when working on the relevant stages; source may also be inspected in the +selected Go module cache. The sibling checkout is optional reference material, +not a build or test dependency. + +## Working instructions + +The audience is a Luna coding agent. Complete the numbered stages in order. +Each stage is a bounded unit with its own verification and completion criteria. +Inspect current code before editing; the paths below identify owners rather +than requiring changes to every listed file. + +- Start with `docs/development.md`, `docs/policy/agents.md`, and `config.md` in + this directory. Follow the development policy-reading guide. Architecture, + testing, documentation, and OSS policies apply to this work; release policy + applies if discussing compatibility or eventual release preparation. +- Use Go 1.26. Prefer the repository's graph tools for code discovery and + inspect focused tests at each package boundary. +- Keep ordinary `Load`, typed defaults, explicit imports, one selected overlay, + domain validation, and current path-resolution rules. Native inheritance is + available wherever ConfigKit and Narratio's schema permit it, with no new + collection allowlist, opt-in flag, or Narratio-owned inheritance engine. +- Distinguish original declarations from inherited contributors. Preserve + genuine duplicate-base rejection and truthful provenance. Parent entries + remain ordinary entries subject to existing domain rules. +- Keep invocation-owned secrets bootstrap and shared exact-name credential + forwarding unchanged. Do not introduce a second base load, file reparse, or + historical-secret lookup to recover declaration information. +- Tests use temporary files, controlled synthetic secrets, and real internal + collaborators. No downstream executable, sibling checkout, cloud service, + or paid provider is needed. Protect integration behavior rather than + duplicating ConfigKit's generic algorithm tests or freezing incidental output. +- Commit each completed nonempty unit with a short plain-English message after + suitable validation. Report blockers and incomplete work accurately. If a + review-only task needs no source fix, retain useful behavioral coverage and + report that result rather than making a cosmetic rewrite. + +Do not adopt layered loading, new source types, source-relative rebasing, +abstract artifacts, automatic YAML conversion, new execution selection rules, +new output collision policy, or changes to subprocess credentials and workspace +paths in this feature. Existing domain validation and reuse rules still apply; +do not add prohibitions, hardening, or guarantees merely because they can be +tested. This plan does not authorize a release, push, tag, ADR acceptance, or +retirement of pending roadmaps. + +## Stage 1: Select the released dependency and update attribution + +Purpose: establish the actual dependency baseline before integration work. + +Read `go.mod`, `go.sum`, `docs/oss/attribution.md`, the OSS policy, and the +selected release's notes, `COPYRIGHT`, and `LICENSE`. + +1. Select `gitea.maximumdirect.net/eric/ConfigKit@v0.5.0` with the normal Go + module workflow. Keep Go 1.26 and avoid unrelated dependency upgrades. +2. Update ConfigKit's attribution entry to v0.5.0 and use the full reviewed + commit for its Source, COPYRIGHT, and LICENSE links. Preserve upstream + copyright wording, SPDX expression, and applicable local license links. + Its license and YAML dependency are unchanged; no new license copy is needed. +3. Inspect module changes for unintended churn or a local `replace`. Existing + configuration loaders should compile without adopting optional APIs. + +Verify with `go test ./internal/config ./internal/app ./internal/doccheck` +and `go mod tidy -diff`. Confirm +`go list -m gitea.maximumdirect.net/eric/ConfigKit` selects v0.5.0 without a +replacement. + +Complete when the released dependency and notices agree and current focused +tests pass. The cross-file inheritance defect is addressed in the next stage; +this stage does not claim that new inheritance behavior is fully integrated. + +## Stage 2: Correct direct declaration accounting + +Purpose: allow distinct artifact and family keys to inherit across imported +files while preserving existing duplicate-declaration rejection. + +Read `internal/config/pipeline_configkit.go`, particularly +`pipelineMetadataFromConfigKit` and `rejectDuplicateBaseDefinitions`, and +`internal/config/pipeline_configkit_test.go`. Inspect the public ConfigKit +`Metadata.History`, `HistoricalPaths`, `Origin`, and `ChangeKind` contracts. + +1. Add package-boundary regression fixtures with a valid parent in one imported + file and a child using `extends` in another. Cover ordinary artifacts and + artifact families. Give parent and child distinct valid output paths or + family patterns, and provide canonical-party context for expanded families. + Both keys remain ordinary domain entries; child IDs must stay their own. +2. Adjust declaration accounting using retained public metadata. In v0.5.0, + original base-input events have `LayerBase` and no origin operation; + projected transfers carry `ChangeInheritance`. Count original base + declarations, not these transfers or subsequent decoding/resolution events. + Consumed structural `extends` paths are not artifact declarations. +3. Keep the historical-path check, including original declarations erased by + a selected overlay. Do not switch to surviving effective keys only or count + all displaced origins as fresh declarations. Reuse a small package-local + helper only if it makes the next stage clearer. +4. Retain `TestPipelineConfigKitRejectsDuplicateBaseDeclarations`, including + disjoint fields supplied to the same key, container removal, and removal of + Scriptorium by an overlay. An overlay may still modify an existing entry. + Add only any missing regression case needed to exercise the new filter. + +Verify with `go test ./internal/config`. The positive cross-file regression +should expose the baseline defect before the fix. Assert loaded values and +identities, or the duplicate category and relevant sources, without matching an +entire diagnostic or ConfigKit's private event ordering. + +Complete when cross-file inheritance loads and expands correctly, and actual +duplicates still fail even when the overlay hides them. Do not use the direct +declaration filter to remove contributor evidence from inspection. + +## Stage 3: Preserve contributor provenance and family ownership + +Purpose: keep inspection useful when one entry contains values from several +files, without confusing its defining file with those contributors. + +Read `internal/config/artifact_catalog.go`, +`internal/config/artifact_families.go`, `internal/config/effective_pipeline.go`, +and metadata-to-ownership projection in `pipeline_configkit.go`. Inspect the +focused catalog, family, and effective-source tests. + +1. Exercise the inherited configurations from Stage 2 through + `EffectiveArtifactCatalog` and `EffectivePipelineSources`. Check inherited + values against their actual parent contributors and child overrides against + the child/overlay contributors. Keep historical definition evidence distinct + from effective field ownership. +2. Preserve `DefinitionSources` aggregation, including inherited contributors. + Do not relabel all inherited values as authored in the child's file. +3. Correct `artifactFamilySource` where its first inherited field currently + selects the parent's file. For an explicitly declared child family, identify + its original declaration using retained metadata: use its base defining file + when present, or its overlay defining file for an overlay-only family. + Overlay edits to a base family do not relocate its base definition. +4. Retain the existing fallback for entries without a direct declaration or + programmatically constructed configuration. Do not restrict valid enclosing + inheritance merely to force every entry into a single-file ownership model. +5. Keep logical `SourceID`, artifact keys, family/character/party provenance, + existing source roles, and inspection interfaces. Update projections only + where inherited evidence is otherwise lost or incorrectly attributed. + +Verify with `go test ./internal/config`. Cover an explicitly declared child +family inheriting from another file and a selected-overlay contribution. Check +meaningful records and source identities, not a complete formatted snapshot. + +Complete when contributor reporting remains truthful and generated members +identify their child family definition rather than an arbitrary inherited field. + +## Stage 4: Verify inherited presence, domain resolution, and path bases + +Purpose: prevent Narratio's application defaults or normalization from undoing +the library's inherited explicit input. + +Read `pipelineConfigDefaults`, `resolvePipelineConfig`, and +`internal/config/load.go`, including `finalizeLoadedPipeline` and +`resolveNotariusPaths`. Inspect the current presence/defaulting tests and +the relevant domain validators. + +1. Add representative inheritance cases for explicit false, zero, empty, + and null input versus genuinely absent input. Include a pointer boolean + and an entry-level absence-based default where Narratio consumes metadata. + For example, inherited `render_debug: false` can exercise artifact presence; + inherited empty Scriptorium timeout can exercise an existing invalid value + that must not silently become its default. +2. Preserve the existing domain outcome for each chosen field. Valid explicit + values retain their value; explicit values already invalid must still fail + ordinary validation rather than be refilled. An empty mapping merges under + ConfigKit's rules and must not be described as universal container removal. +3. Exercise overlay composition before inheritance: modify a parent's field + in the selected overlay and confirm the child's inherited result uses that + effective value. A child override still wins over its parent. +4. Verify a known root-relative field, such as enabled Notarius `config_path` + or `working_directory`, when inherited from an imported declaration. + Resolve it exactly as the existing field contract does. Do not use passive + SourceContext to rebase it to the parent's or child's source directory, or + newly normalize other path fields merely to make their behavior uniform. +5. Fix any demonstrated Narratio integration defect at its existing defaulting + or domain-resolution owner. Use ConfigKit presence and recorded operations; + do not recreate structural inheritance or weaken ordinary validation. + +Verify with `go test ./internal/config`. Keep coverage representative; generic +missing-parent, cycle, resource-context, and expansion-limit matrices belong to +ConfigKit. Parents must remain valid Narratio entries, not special templates. + +Complete when inherited explicit input and absent defaults have distinct, +correct domain outcomes and existing path bases are unchanged. + +## Stage 5: Verify safe effective projections and inherited credentials + +Purpose: ensure inheritance changes neither semantic comparison nor the +existing secret boundary. + +Read `internal/config/effective_pipeline.go`, `effective_digest.go`, +`effective_projection.go`, `scriptorium_credentials_test.go`, and the effective +projection tests in `load_validate_test.go`. For diff composition, read +`internal/app/config_diff.go`. Reuse +`TestPipelineOverlayDoesNotRequireUnusedBaseCredentials` and +`TestCredentialValuesNeverEnterEffectiveProjectionOrDigest` coverage. + +1. Load two configurations that produce the same effective domain values: + one uses inheritance, the other writes the expanded values explicitly. + Include ordinary artifacts and representative resolved family members. + Verify equal safe effective mappings and digests and no effective diff. + Do not compare private provenance as part of semantic equality. +2. Add a representative typed credential inheritance case using shared fields + of valid Narratio sibling mappings and controlled synthetic environment + variables. Verify resolved credentials retain their exact configured names; + serialization and inspection expose those identities without values. +3. Verify overlay removal/replacement of a credential before inheritance does + not resolve the superseded reference. Reuse the existing selected-overlay + regression rather than creating a second loader or bootstrap algorithm. +4. Check that changing only synthetic secret values leaves semantic projections + and digest unchanged. Fix only demonstrated projection defects; preserve + the existing safe typed rendering boundary and semantic reuse evidence. + +Verify with `go test ./internal/config ./internal/app`. Existing shared adapter +credential-forwarding tests remain the owner of subprocess forwarding; do not +rewrite adapters or duplicate those tests simply because input can be inherited. + +Complete when resolved values, not source layout or secret bytes, determine +effective comparison, and inherited credentials obey the existing safe boundary. + +## Stage 6: Document inheritance and maintain a small example + +Purpose: make the adopted behavior understandable and usable without expanding +Narratio's configuration surface. + +Read `docs/config.md`, `docs/internal/configuration.md`, `examples/README.md`, +the production/testing bundle, and `TestExamplesLoadAndValidate` in +`internal/config/load_validate_test.go`. Follow the documentation policy. + +1. Add native sibling inheritance guidance to the configuration reference. + Explain exact sibling lookup, overlay-before-inheritance resolution, + explicit presence, ordinary parent validation, distinct child identity, + and cross-file declaration/contributor distinctions. Link to the pinned + upstream contract for detailed syntax and structural rules. +2. Explain the reserved `extends` key throughout ConfigKit inputs, including + dynamic artifact IDs and prompt-variable mappings. Operators must rename + existing ordinary-data keys with that name; there is no opt-in or automatic + conversion. Campaign, session, and party inputs also use ConfigKit. +3. Clarify the existing statement that overlays cannot inherit: overlay files + cannot inherit or stack, but the selected overlay may supply native sibling + directives within its values. Keep imports and overlay selection unchanged. +4. Update configuration internals for inheritance before typed decoding, + original-declaration counting, contributor projection, and the safe + effective boundary. Describe implemented mechanisms, linking to the public + reference for their operator-facing meaning. +5. In `examples/production-testing/overlays/testing.yml`, add + `extends: session_recap` to the existing `testing_notes` artifact. Keep its + disabled status and distinct output override. It inherits the imported + parent's prompt/input and the parent's testing profile after overlay + application. The parent remains an ordinary artifact. +6. Update the example guide and extend existing example validation to check + those inherited fields. Preserve the bundle's current identities and + artifact count; no new example root or template convention is necessary. + +Verify with `go test ./internal/config -run '^TestExamplesLoadAndValidate$'` +and `go test ./internal/doccheck`. Review examples for synthetic-only content +and validate local links. Keep unimplemented Audita/workspace behavior in its +roadmaps. Do not create a release note under an invented Narratio version; +reserved-key compatibility impact remains an explicit release-preparation item. + +Complete when canonical documentation and a tested copyable example describe +the implemented adoption, without promoting new restrictions to policy. + +## Stage 7: Verify the assembled inspection workflow and close the feature + +Purpose: establish that real command composition consumes the integrated +configuration correctly and finish the upgrade as a reviewable baseline. + +Read `internal/app/config_commands_test.go`, `config_artifacts_test.go`, +and existing read-only inspection workflow coverage. Reuse the preceding +package coverage rather than repeating its entire matrix at the CLI boundary. + +1. Add or extend a representative command-level temporary imported bundle with + inheritance. Exercise validation, effective show, source reporting, artifact + catalog, and diff through existing command entry points. Use a campaign/party + fixture for family inspection, and paired overlays with equivalent inherited + and explicitly expanded values when testing command-level diff equality. +2. Check semantic output: resolved child values, expected contributor/defining + sources, logical artifact identities, and no secret values. Keep inspection + read-only with respect to session/runtime state, as existing tests establish. + Do not invoke downstream applications or add CLI modes or columns. +3. Review the combined diff against `config.md` and this plan. Confirm that + ConfigKit v0.5.0 is selected, attribution links use the reviewed revision, + no optional loading feature was adopted, and no Audita/workspace change or + compatibility shim slipped into scope. +4. Run `go test ./...`, `go test -race ./...`, `go vet ./...`, + `go build ./...`, `go mod tidy -diff`, and `git diff --check` against the + final source. These reuse existing repository checks, not new release gates. + The full suite includes example and documentation-link validation; focused + checks need not be repeated solely for this stage if unchanged. +5. Commit the completed unit and report the dependency revision, checks, + compatibility impact, and any unfinished work. Leave the pending Audita and + workspace plans and Proposed ADRs intact. Retire completed planning files + only during separately requested cleanup/release preparation, repairing links. + +Complete when the feature roadmap's completion criteria are evidenced by the +released dependency, integrated behavior, maintained documentation/examples, +and passing final checks. The next implementation scope is Audita; its concrete +plan is prepared separately. Release version selection and publication are +outside this implementation pass. + +## Open Questions + +None required for this adoption. Existing decisions select native inheritance, +current loading/path/credential semantics, and ConfigKit before Audita before +workspace work. Eventual release preparation must account for the reserved-key +compatibility impact under the release policy; no version is selected here. diff --git a/docs/roadmap/workspace.md b/docs/roadmap/workspace.md new file mode 100644 index 0000000..4d59146 --- /dev/null +++ b/docs/roadmap/workspace.md @@ -0,0 +1,280 @@ +# Session workspace organization + +## Purpose + +Give operators predictable locations for session inputs, completed outputs, +logs, and debug evidence. Apply the taxonomy in +[ADR 0001](../adr/0001-classify-session-content.md) and the ownership boundary in +[ADR 0002](../adr/0002-assign-subprocess-content-ownership.md). + +This roadmap defines the target behavior and required changes. Implementation +sequencing belongs in a subsequent implementation plan. The ADRs remain +Proposed pending their acceptance. + +## Confirmed Requirements + +- Completed artifacts occupy a flat `artifacts/` directory. Intermediate + products such as `session_bounds.json` belong here. +- Combined transcripts occupy `transcripts/` directly. Initial per-speaker + transcripts occupy a subdirectory beneath it. +- Prepared external audio occupies `audio/`; other prepared external inputs + occupy `inputs/`. Outputs consumed by later stages retain their output paths. +- Logs and Debug use matching latest-attempt views: `logs/` and + `debug/` are symlinks to the producing run's directories. Both views + advance when a stage starts an actual attempt, including attempts that fail. + A skipped stage leaves its previous views intact. +- Execution reports and invocation records are Logs. Historical Logs and Debug + remain under their producing runs, subject to cleanup. +- Narratio can selectively request supported subprocess debug capture through + configuration. Omission preserves application configuration and defaults; + Narratio needs no explicit downstream debug-disable override. +- Prepared session audio is the single retained Narratio-managed audio copy. + Remove Narratio's separate persistent audio cache and spool duplicates. +- Configurations that produce colliding artifact names or destinations fail + validation before execution. Validation covers the effective pipeline, + including expanded character families and built-in artifact destinations. +- Narratio selects input, output, log, and debug paths. Subprocess applications + own their working State and Cache through their supported configuration. +- A dedicated cache root such as `/var/cache/notarius` is a deployment + recommendation. The application owns its cache defaults, internal layout, + identity, and reuse semantics. Narratio session cleanup respects that boundary. +- Narratio retains its own manifest, recovery records, and managed staging State. +- This is a clean cutover for newly prepared sessions. Existing-session migration + and backward compatibility shims are outside the feature. +- Treatment of older completed files after invalidation or failed replacement + remains unchanged, including their availability through manifest-backed + resolution. Directory reorganization does not authorize new stale-file + deletion, automatic fallback, or rollback behavior. + +## Current Baseline + +The [operator guide](../operations.md#local-state-layout) owns the current +physical layout. Relevant differences from the target are: + +- Extraction moves the complete Notarius bundle to + `artifacts/notarius//`. Resume evidence and artifact resolution depend + on that bundle and its index. +- Merge and normalize promote Seriatim execution reports into `artifacts/`. +- Analysis output paths and character-family patterns can introduce nested + artifact directories. Output-path configuration also permits locations that + need reconciliation with the selected directory categories. +- Standard combined transcript paths already match the target. Layout creation + also creates an unused `transcripts/trimmed/` directory. +- Most subprocess diagnostics live under individual run directories. Root-level + log paths lack a consistent latest-attempt view. Audita reports sit beside + its staged transcript, and restore writes a separate top-level report. +- Prepared previous-session inputs occupy `previous/`. Narratio-owned reference + snapshots supplied to Notarius occupy session `cache/notarius/references/`. + These snapshots are distinct from Notarius's own execution caches. +- Audita debug placement follows downstream configuration. Narratio does not + expose Notarius debug capture, and Scriptorium render-debug placement varies. +- Narratio maintains separate S3 audio cache and spool paths in addition to the + prepared session audio copy. + +## Target Layout And Ownership + +Retain the configurable workspace root and its current system-wide default, +`/var/lib/narratio`. Sessions remain under +`/var/lib/narratio/work//` by default. The following paths +express the selected categories: + +| Path | Contents and role | +| --- | --- | +| `manifest.json` | Narratio's durable progress and output references. | +| `artifacts/` | Completed artifact payloads, including session bounds. | +| `transcripts/` | Completed combined transcripts and rendered transcripts. | +| `transcripts/raw/.json` | Completed initial per-speaker transcripts. | +| `audio/` | Prepared external audio. | +| `inputs/` | Prepared configuration, references, and previous-session inputs. | +| `logs/` | Symlink to the latest attempt's run-local logs and execution reports. | +| `debug/` | Symlink to the latest attempt's run-local debug directory, empty when no capture is produced. | +| `runs///logs/` | Actual logs and execution reports for that attempt. | +| `runs///debug/` | Actual debug captures for that attempt, including retained state snapshots. | +| `runs//` | Narratio staging, original tool bundles, and retained execution evidence. | + +For example, the latest views use relative links within the session: + +```text +logs/polish -> ../runs//polish/logs/ +debug/polish -> ../runs//polish/debug/ +``` + +Each stage's latest attempt can belong to a different run. Producers write to +the actual run paths; the symlinks provide operator navigation without duplicate +diagnostic storage. + +Narratio's lock and restore marker remain its own State. Their current placement +can remain; introducing another root-level State directory is unnecessary for +this feature. Generated records that describe an invocation belong with Logs; +configuration actually supplied as an input follows its input or staging role. + +Subprocesses may create internal working directories wherever their configuration +directs. Narratio-selected staging destinations for result bundles remain valid +under ADR 0002. Cache locations are documented as operator recommendations using +each application's existing configuration surface. + +## Required Changes + +### Output paths and configuration + +Centralize the revised paths in the existing path owner. Apply the flat artifact +rule to explicit artifacts, expanded families, extraction outputs, and session +bounds. Preserve logical source identities used by dependencies and publication. +Keep explicit analysis filenames and require flat destinations. Derive extraction +filenames from configured extraction keys with a `.json` extension, with an +explicit extraction filename override for naming preferences or collisions. +Retain `session_bounds.json` as the default bounds filename. Do not silently +rename conflicting files. Report conflicting producers and the shared destination +in configuration errors. Validate the effective configuration after composition +and family expansion. + +Keep combined transcript destinations directly beneath `transcripts/`. Remove +obsolete directory scaffolding and reconcile configurable output paths with +their category. Local path changes do not require flattening explicitly +configured remote publish destinations. + +### Notarius result handling and reuse + +Receive and validate the complete Notarius bundle in Narratio staging. Promote +selected accepted payloads to their flat artifact destinations. Classify the +receipt, warnings, rejection summaries, and diagnostics as Logs. Keep bundle +indexes and other management metadata with the bundle or Narratio State according +to their use. + +Revise extraction evidence, catalog hydration, and resume checks to resolve +promoted products independently of the old `artifacts/notarius/` layout. Retain +the identity, provenance, contract, and checksum evidence needed for current +reuse behavior. Historical bundles may remain available for inspection under +their existing retention semantics; relocating them beneath `runs/` must not +silently shorten their lifetime through run cleanup. + +Preserve ordinary resume and forced-extraction behavior. Subprocess caches remain +under Notarius's control. + +### Inputs + +Place prepared previous-session material beneath `inputs/previous/`. Place the +stable reference snapshots Narratio supplies to Notarius beneath +`inputs/references/`, retaining their content identities where needed for reuse. +Update preparation, input resolution, manifest references, and restore mappings. +Keep source configuration ownership and prepared-file validation unchanged. + +### Audio acquisition + +Simplify acquisition to one prepared session copy under `audio/`. Remove the +separate Narratio S3 audio cache and routine retained spool copies, together with +their obsolete configuration, manifest bookkeeping, and cleanup controls. +Preparation may use temporary download files before installing validated audio. +Preserve reuse of valid prepared audio and update restore to use the same layout. + +Recreating a session after its prepared audio is removed requires fetching the +source again. Original recordings remain outside this cleanup change. Removing +spool/cache handling does not authorize new deletion of prepared session audio. +Subprocess caches remain independently configured and managed. + +### Logs and execution reports + +Provide one consistent latest-attempt view per stage. Include failed attempts; +a skipped stage leaves its previous view intact. Distinguish multiple applications +or artifact invocations within a stage through their filenames. Route Seriatim +reports, Audita reports, Notarius diagnostic records, and Narratio invocation +records through this model. Treat restore execution as its own diagnostic group +under `logs/restore/`, using run-local evidence and a latest-attempt view. + +Prevent a new attempt from presenting old diagnostic files as newly generated +ones. At the start of each actual stage attempt, create its run-local log and +debug directories and refresh both latest symlinks. An attempt without debug +output exposes an empty debug directory rather than an older capture. Preserve +subprocess errors and available evidence on failure. Retain existing +captured-stream redaction and limits. + +Keep historical files at their run paths; rotating the links does not move files. +Run cleanup must preserve directories targeted by current latest links. Do not +introduce an automatic history age or count limit. Publication and manifest +diagnostic references use actual run paths. Adjust path handling for these +Narratio-managed navigation links without relying on arbitrary symlink traversal. + +### Debug controls and placement + +Add selective Narratio configuration for Audita and Notarius debug capture using +their supported enabling CLI flags. An enabled Narratio selector requests +capture; an omitted or false selector passes no debug override and preserves the +application's configured/default behavior. Do not add a downstream disable flag. + +Supply the actual run-local debug directories, exposed through `debug/polish` +and `debug/extract`. Control the destination when capture can be enabled through +downstream configuration as well, including Audita overlays, using supported +destination controls without changing capture enablement. Audita's debug +work-directory option selects diagnostic storage and belongs to this placement +responsibility. + +Route Scriptorium's existing analysis and bounds render-debug products to the +run-local directories exposed through `debug/analyze` and `debug/trim`. Preserve +its existing per-artifact selection capability. Seriatim requires no invented +debug mode; its ordinary reports are +Logs. Subprocess-defined bundle contents and nested run directories are allowed +beneath the selected run-local debug root. Logs and Debug share the same +latest-attempt semantics and cleanup protection. + +Keep new full Audita and Notarius debug bundles local in this feature. Preserve +publication of currently declared diagnostics, including existing render-debug +products. Enabling capture does not implicitly add full debug bundles to remote +archives. + +Debug controls govern capture when a stage executes. They do not independently +require regeneration of valid substantive outputs. Document the existing force +workflow for obtaining fresh diagnostic evidence when a stage would be reused. +Preserve downstream failure semantics for diagnostic writes and the existing +boundary of Narratio's captured-stream redaction. + +### Audita destination support + +Request upstream support for independently selecting the transcript output, +ordinary report/log directory, and debug/retained working directory, then adopt +a supporting release. Audita currently ignores `--report-dir` when transcript +output is a file; honoring that destination is the principal missing capability. +Narratio continues to capture subprocess stdout and stderr itself. + +Direct ordinary reports, including failure reports, to the actual polish log +directory, the transcript to its staged product destination, and retained debug +material to the actual polish debug directory. This feature uses the supported +upstream controls in a clean cutover without version-based compatibility branches. + +### Manifest, publication, restore, and cleanup + +Update all consumers of changed paths and file categories together. Preserve +per-speaker and per-artifact completed-work reuse. Keep publication driven by +declared products and diagnostics, with new full debug bundles remaining local. +Restore new-format products and inputs to their new locations. + +Define an explicit way to identify supported session state for the clean +cutover. Document regeneration with the new version and manual configuration +updates. Avoid accepting old state through path fallbacks or silently rewriting +old local sessions or remote snapshots. + +Preserve existing invalidation and failed-replacement behavior. Review retention +by content role before changing cleanup targets, especially when durable products +or historical bundles move. Run cleanup preserves the current log and debug +link targets; explicit removal of an entire session removes its local views +with it. Subprocess working State and Cache remain outside Narratio's cleanup +responsibility. + +### Documentation and verification + +Update configuration examples, the operator guide, adapter contracts, workspace +internals, and the architecture policy when implementation lands. Replace the +current architectural assumption that extraction requires a complete immutable +bundle beneath `artifacts/` with the selected product-promotion model. Retire +policy and documentation for Narratio's removed audio cache/spool lifecycle and +document the managed latest-attempt symlinks. Keep per-application cache locations +as deployment recommendations owned by each application's configuration. + +Use focused behavioral coverage for flat output placement, collision rejection, +latest-attempt diagnostics on success and failure, debug selection and routing, +unchanged completed-work reuse, failed replacement, and new-format publish/restore. +Cover empty debug views after attempts without capture, preserved views for +skipped stages, cleanup protection of latest targets, and audio acquisition, +reuse, and restore without persistent spool/cache duplicates. Exercise cleanup +against products and separately owned caches. Use temporary filesystems and +subprocess doubles; default tests remain offline and independent +of sibling checkouts or paid providers.