Plan the ConfigKit upgrade and workspace development
This commit is contained in:
175
docs/roadmap/audita.md
Normal file
175
docs/roadmap/audita.md
Normal file
@@ -0,0 +1,175 @@
|
||||
# Adopt Audita v3 and independent diagnostic destinations
|
||||
|
||||
## Purpose
|
||||
|
||||
Adopt Audita v3.0.0 for transcript polishing and use its independent output
|
||||
destinations to implement the Audita portion of the
|
||||
[session workspace roadmap](workspace.md). Operators should find the completed
|
||||
transcript, ordinary reports, and optional debug evidence in their respective
|
||||
Narratio locations.
|
||||
|
||||
This document defines the Audita adoption scope and target. The workspace
|
||||
roadmap owns shared layout, latest-attempt views, and retention behavior. Its
|
||||
referenced ADRs remain Proposed. Implementation sequencing belongs in a
|
||||
subsequent implementation plan.
|
||||
|
||||
## Reviewed Upstream Release
|
||||
|
||||
The reviewed Audita v3.0.0 tag identifies commit
|
||||
`3b4933256eff84327b812e7ae12babf743bc04a6`. Its
|
||||
[release notes](https://gitea.maximumdirect.net/eric/audita/src/tag/v3.0.0/docs/releases/v3.0.0.md),
|
||||
[CLI reference](https://gitea.maximumdirect.net/eric/audita/src/tag/v3.0.0/docs/cli.md),
|
||||
[configuration reference](https://gitea.maximumdirect.net/eric/audita/src/tag/v3.0.0/docs/config.md),
|
||||
and [operations guide](https://gitea.maximumdirect.net/eric/audita/src/tag/v3.0.0/docs/operations.md)
|
||||
own the downstream contracts.
|
||||
|
||||
Audita now honors an explicit `--report-dir` alongside file transcript output.
|
||||
Its existing `--work-dir` selects the parent of retained debug captures, with
|
||||
capture enabled separately through configuration or `--debug`. These controls
|
||||
satisfy Narratio's requested destination separation. Further upstream changes
|
||||
are unnecessary for this scope.
|
||||
|
||||
## Current Baseline
|
||||
|
||||
Narratio's [Audita integration](../integrations/audita.md) and
|
||||
[polish stage](../internal/stage-polish.md) describe the implemented behavior:
|
||||
|
||||
- The adapter invokes `process` with a transcript, glossary, and staged transcript
|
||||
destination. It supports independent downstream configuration, overlay,
|
||||
profile, module, description, and output-schema selectors.
|
||||
- The adapter supplies no reporting, debug, or work-directory flags. Audita
|
||||
controls debug capture and placement through its own configuration.
|
||||
- Ordinary reports are discovered beside the staged transcript. The polish
|
||||
stage validates returned report paths against those adjacent filenames.
|
||||
- Only the validated polished transcript becomes a canonical stage output.
|
||||
Reports, captured streams, and the generated invocation record are diagnostics.
|
||||
- Selected credential references use Narratio's shared exact-name forwarding and
|
||||
captured-stream redaction boundary.
|
||||
|
||||
The existing invocation remains usable with Audita v3 after deployment-side
|
||||
profile migration. Narratio changes are needed to request and consume the
|
||||
new destination separation.
|
||||
|
||||
## Targeted End State
|
||||
|
||||
### Products and diagnostics
|
||||
|
||||
Narratio supplies separate destinations for each actual polish attempt:
|
||||
|
||||
| Content | Destination and control |
|
||||
| --- | --- |
|
||||
| Corrected transcript | Run-local staging via `--output`; validated and promoted to `transcripts/polished.json` on success. |
|
||||
| Ordinary reports | `runs/<run_id>/polish/logs/` via `--report-dir`. |
|
||||
| Captured stdout, stderr, and Narratio invocation record | The same run-local polish log directory. |
|
||||
| Optional debug evidence | `runs/<run_id>/polish/debug/` via `--work-dir`; Audita may create its own `run-*` subdirectory beneath it. |
|
||||
|
||||
The workspace roadmap's `logs/polish` and `debug/polish` links expose the latest
|
||||
attempt. Narratio supplies actual run destinations to Audita and records those
|
||||
paths in manifest diagnostics. Shared link updates and cleanup protection are
|
||||
implemented by the workspace work rather than separately inside the adapter.
|
||||
|
||||
### Capture selection
|
||||
|
||||
Add an optional `pipeline.audita.debug` boolean, defaulting to false. A true
|
||||
value passes `--debug`. False or omission supplies no capture override and
|
||||
preserves Audita configuration and overlay behavior; it does not disable debug
|
||||
enabled downstream.
|
||||
|
||||
Supply `--work-dir` for each executing attempt so downstream-enabled capture
|
||||
also lands in the Narratio-selected debug directory. Selecting that destination
|
||||
alone does not enable capture. Ordinary production operation retains Audita's
|
||||
default reporting and debug settings. Narratio adds no reporting enable/disable
|
||||
flag or debug-disable flag.
|
||||
|
||||
Debug selection affects diagnostics for an executing stage. Changing it alone
|
||||
does not invalidate a completed transcript. Operators use the existing force
|
||||
workflow when fresh polish or debug evidence is needed.
|
||||
|
||||
### Ownership and compatibility
|
||||
|
||||
Use a clean adoption of Audita v3's CLI contract, without version-dependent
|
||||
adapter branches or legacy profile conversion. Audita continues to own its
|
||||
configuration, profiles, prompts, runtime State, and diagnostic contents.
|
||||
Narratio owns supplied destinations and transcript promotion.
|
||||
|
||||
Keep existing selectors and shared credentials such as `OPENROUTER_API_KEY` and
|
||||
`RAKESTRAWHOME_API_KEY` under their configured exact names. Audita's examples
|
||||
do not require Narratio to adopt application-specific credential aliases.
|
||||
Operators continue to list the credential references required by all configured
|
||||
Audita backends.
|
||||
|
||||
Audita is a separately installed executable. Its adoption of ConfigKit v0.5.0
|
||||
and PromptKit v0.12.0 does not require changing Narratio's own library
|
||||
dependencies as part of this feature.
|
||||
|
||||
## Required Changes
|
||||
|
||||
### Deployment configuration migration
|
||||
|
||||
Document the manual migration of Audita-owned PromptKit files:
|
||||
|
||||
- Wrap profile definitions under keyed `profiles:` mappings, remove inner `id`
|
||||
fields, and replace `base_profile` with `extends`. Preserve selected profile
|
||||
IDs when appropriate.
|
||||
- Review explicit zeros, omitted values, and nulls against the new generation
|
||||
control semantics. Validate the final configuration root, including unused
|
||||
definitions and backend references.
|
||||
- Rename any backend literally named `extends` and update its references.
|
||||
- Validate production configuration and each used testing overlay with Audita's
|
||||
offline `config validate` command. Audita application configuration continues
|
||||
to use `version: 2`.
|
||||
|
||||
Keep the detailed downstream format owned by the pinned Audita documentation.
|
||||
Narratio does not inspect, rewrite, or automatically migrate these files.
|
||||
Deployment can migrate Audita before the workspace changes land.
|
||||
|
||||
### Adapter and polish stage
|
||||
|
||||
Carry explicit report and debug destinations through the Audita request and
|
||||
construct the supported flags inside the adapter. Wire the debug selector
|
||||
through configuration and runner composition.
|
||||
|
||||
Discover the existing three report filenames in the requested report directory.
|
||||
Update polish's expected-path validation to the same location. Update invocation
|
||||
records and safe result metadata to describe the selected destinations and
|
||||
capture request. Preserve available failure evidence and the original subprocess
|
||||
error.
|
||||
|
||||
Keep existing transcript validation, report read bounds, timeouts, credential
|
||||
forwarding, and captured-stream redaction. Reports remain optional diagnostics;
|
||||
early failures or downstream-disabled reporting may produce none. Exit-zero
|
||||
runs with recoverable model errors remain successful. A nonzero exit, including
|
||||
a late report-write failure, never authorizes transcript promotion. Narratio
|
||||
does not add report-schema parsing or report-status success gates.
|
||||
|
||||
### Workspace integration and lifecycle
|
||||
|
||||
Use the shared workspace path owner for run-local log/debug destinations and
|
||||
the agreed latest-attempt behavior. Preserve previous views on skipped stages;
|
||||
actual attempts refresh both views, including on failure. An attempt without
|
||||
debug capture exposes an empty current debug directory.
|
||||
|
||||
Keep full Audita debug bundles local in this feature. Preserve existing declared
|
||||
diagnostic publication and apply the workspace roadmap's retention protection
|
||||
to latest link targets. This feature inherits that roadmap's clean session
|
||||
cutover and unchanged treatment of older completed outputs after invalidation
|
||||
or failed replacement.
|
||||
|
||||
### Documentation and verification
|
||||
|
||||
When implemented, update the configuration reference and examples, Audita
|
||||
integration contract, polish internals, and operator guide. Reference Audita v3's
|
||||
profile migration and explain the shared Audita/Notarius overlay development
|
||||
workflow. Downstream configuration edits behind unchanged selectors still
|
||||
require explicitly forcing the affected stage to replace reused outputs.
|
||||
|
||||
Revise the workspace roadmap's upstream-request description when this adoption
|
||||
is incorporated; Audita v3.0.0 supplies the previously missing capability.
|
||||
|
||||
Use focused offline tests for separate transcript/report/debug destinations,
|
||||
enable-only debug selection, capture enabled by downstream configuration,
|
||||
available failure reports, and preservation of nonzero-exit behavior. Verify
|
||||
that reports remain diagnostics and only a successful validated transcript is
|
||||
promoted. Shared latest-view and cleanup tests belong to the workspace change.
|
||||
Default Narratio tests use subprocess doubles and temporary filesystems and
|
||||
remain independent of sibling checkouts, real credentials, and paid providers.
|
||||
Reference in New Issue
Block a user