eric/audita
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 6 Wiki Activity

Resolve output failures and preflight resource preparation

Browse Source
This commit is contained in:
Eric Rakestraw
2026-11-09 15:25:57 +00:00
parent 547be2d9ab
commit 866e4b9b23
3 changed files with 150 additions and 69 deletions
Download Patch File Download Diff File Expand all files Collapse all files

137
docs/roadmap/commandkit.md
View File

@@ -2,9 +2,10 @@
**Updated:** 2026-11-09
**Status:** Adoption target selected; the upstream release gate is satisfied.
Implementation remains pending. The [master roadmap](library-adoption.md) owns
shared direction, exact dependency revisions, and sequencing; ConfigKit and
**Status:** Adoption target and design decisions resolved; the upstream release
gate is satisfied. Implementation remains pending. The
[master roadmap](library-adoption.md) owns shared direction, exact dependency
revisions, and sequencing; ConfigKit and
PromptKit contracts are developed in [their](configkit.md) [outlines](promptkit.md).
## Baseline and Target
@@ -149,6 +150,84 @@ Check effective mode after native ordering, boolean values, and aliases:
explicit destination. `--debug --quiet` also requires one; debug capture remains
independent once the invocation passes this check.
## Output-Failure Behavior — Resolved
Require successful delivery of the transcript and enabled ordinary reports.
Debug artifacts remain optional even when capture was explicitly requested.
Adopt CommandKit's native handling of CLI output and diagnostic write failures.
| Output | Failure behavior |
| --- | --- |
| Transcript file or stdout | Fail the invocation. |
| Enabled ordinary reports | Fail the invocation if any required report cannot be written. |
| Debug directory or individual debug artifacts | Continue processing and submit a redacted warning through CommandKit. |
| CommandKit output and diagnostic streams | Preserve the native outcome and delivery-error behavior. |
Ordinary reports describe incomplete work, recoverable model failures, and
repairs. When enabled, delivering that account is part of completing the run.
Debug artifacts provide supplemental investigative detail; their loss alone does
not invalidate a delivered transcript and reports. Do not add a debug strictness
option.
Detect required-destination problems before model work wherever practical through
the [output preparation below](#output-resource-preparation-during-preflight--resolved).
If a required output fails later, retain successfully written files, fail the
invocation, and attempt to finalize the remaining reports with the failure
recorded. A report failure after transcript publication must not erase that
transcript or repeat model calls. Preserve the original processing failure when
secondary reporting or cleanup also fails. Emit the completion summary only
after processing and all enabled required outputs have succeeded, including
required final writes, closes, and file replacements.
Use the existing redaction and resolved visibility rules for debug warnings.
In particular, `--debug --quiet` still attempts capture but suppresses warnings
about missing optional artifacts; required-output failures retain the normal
fatal-error explanation. A separate failure while writing a warning can still
make the invocation fail under CommandKit's native
[output-failure contract](https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.6.0/docs/cli.md#output-failures).
Keep that outcome under upstream ownership rather than overriding it to make
diagnostic stream failures nonfatal.
## Output Resource Preparation During Preflight — Resolved
Prepare output resources as part of Audita's expressly named
`internal/framework/preflight` phase, after ConfigKit handoff and before model
calls. Follow the [preflight package boundary](glossary-stages.md#empty-collections-and-audita-preflight--resolved).
Complete configuration, invocation, and domain-readiness checks for the entire
pipeline first, including the quiet-destination rule and active destination
collision checks. Only then prepare output resources, and begin execution after
required preparation succeeds. Disabled report and debug destinations remain
inactive.
- For a regular transcript destination, create and open a temporary file in its
destination directory before processing. Preserve the existing destination
until processing and writing succeed, then replace it. This also preserves
input when the transcript intentionally replaces its own source; preparation
must not truncate the final path.
- Initialize enabled ordinary reports before processing, using the existing
`in_progress` report lifecycle and per-file replacement. An initialization
failure stops the invocation before model work.
- When debug capture is enabled, create its directory and initial artifacts
early. Create later artifacts as needed instead of opening every possible
artifact in advance. Debug preparation failures follow the nonfatal artifact
rule above.
- Use CommandKit's supplied writer for transcript stdout. Defer opening pipe and
device destinations until output delivery; in particular, opening a named pipe
during preflight could block waiting for its reader. Do not replace these
destinations with regular temporary files.
The preflight package coordinates preparation through the output-session
component. Reuse and adapt that component to own prepared handles, cleanup, and
final publication; keep filesystem machinery out of individual readiness checks.
Close owned handles and remove unpublished temporary files on failure or
cancellation, while retaining published outputs and retained diagnostic evidence.
The caller retains ownership of supplied standard streams.
Opening resources early establishes readiness at that point; later writes,
closes, and replacements can still fail. Apply the output-failure rules throughout
execution and finalization. Preserve per-file publication without adding a
transaction across the transcript and reports.
## Request Preview, Validation, and Smoke Commands — Resolved
Include standard ckprompt request preview, request validation, and the complete
@@ -198,8 +277,11 @@ released dependency set when implementing this outline.
framework around CommandKit.
4. Adapt domain process handlers to reporting/outcome boundaries, preserving
transcript/artifact integrity, cancellation, and partial success. Apply the
resolved visibility and completion-summary rules above; resolve
[output-failure behavior](#output-failure-behavior) before implementation.
resolved visibility, completion-summary, and
[output-failure rules](#output-failure-behavior--resolved). Establish the
preflight package and its [output preparation](#output-resource-preparation-during-preflight--resolved)
through the output-session component. The later glossary feature extends the
same package with its collection-readiness checks before resource preparation.
Standard command statuses, including comparison status
3, should not be forced into the old Audita-only status table without a domain
reason. Validate quiet mode against the transcript destination before creating
@@ -233,6 +315,23 @@ runs. Fatal error explanations remain visible in every mode, and a required-outp
failure must not produce a successful completion summary. Cover ordinary reporting disabled as
well as enabled, and document the loss of the former default summary.
Cover required-output preparation failure before model calls, late write/close/
replacement failures, and report failure after a transcript has been published.
Verify retention of published output, preservation of the original processing
failure when reporting also fails, and attempts to finalize available reports.
Exercise optional debug-directory and artifact failures, including suppressed
warnings in quiet mode, and the native CommandKit outcome when a diagnostic
write itself fails. Keep these checks focused on Audita's integration and output
responsibilities rather than duplicating upstream reporter tests.
Verify that readiness failures occur before output resources are created.
Check that preparing regular transcript output preserves existing content,
including an in-place transcript, and that failure or cancellation closes owned
resources and removes unpublished temporary files. Cover enabled report
initialization, inactive disabled destinations, and debug artifacts created as
needed. Confirm that preflight does not open pipe/device destinations and that
supplied standard streams remain caller-owned.
Cover omitted and empty output selections in effective quiet mode, including
with reports or debug capture enabled: reject before model calls and artifact
creation. Accept valid explicit file and discard destinations. Exercise native
@@ -299,31 +398,3 @@ capture/mode combinations, aliases, and quiet-destination policy. The resolved
graph selected the intended ConfigKit v1.1.0 and PromptKit v0.20.0 dependencies.
These checks establish the upstream gate; they do not replace Audita's assembled
integration acceptance. No additional upstream request was identified.
## Open Decisions
### Output-Failure Behavior
This is the remaining open decision in the
[master roadmap](library-adoption.md#open-decisions).
Specify how failed transcript, report, debug-artifact, and standard reporter writes
influence the process outcome. The existing visibility rules and prohibition on
a successful completion summary after required-output failure remain selected.
**Recommendation:** required transcript and enabled ordinary-report failures fail
the invocation. Optional debug-artifact failures remain nonfatal and use the
standard reporting path with existing redaction and visibility rules. Standard
CommandKit reporter failures follow native behavior rather than an Audita-owned
replacement policy. This preserves the distinction between delivering the
requested product and retaining supplemental diagnostic evidence, while avoiding
a second reporting implementation. Validate the exact adopted reporter behavior
and cover early and late output failures, including a report failure after the
transcript has already been written.
**Alternative:** also make debug-artifact failures fatal when debug capture was
explicitly requested. This treats capture as a required deliverable and makes
incomplete evidence visible in the process outcome, but can fail an otherwise
successful transcript run solely because supplemental evidence could not be
saved. Standard reporter behavior would still remain upstream-owned. Choose the
artifact requirement deliberately rather than infer it from the debug reporting
mode, which remains independent of capture.

41
docs/roadmap/glossary-stages.md
View File

@@ -394,7 +394,7 @@ assets, whether imported for reuse, inspected, or selected alongside populated
collections. A nonempty list of selected collection names does not establish
that a stage has entries to process.
Add an explicit Audita preflight phase after ConfigKit hands off the fully
Use an explicit Audita preflight phase after ConfigKit hands off the fully
resolved configuration and before pipeline execution begins. Preflight evaluates
execution-readiness business rules against that resolved data. It does not load
sources, resolve configuration, restore defaults, or change stage inheritance.
@@ -402,7 +402,8 @@ Keep the nonempty glossary-stage check in Audita preflight rather than ConfigKit
configuration-validation callbacks.
Implement this phase in an expressly named `internal/framework/preflight`
package. Accept fully resolved configuration and the domain inputs needed for
package, established during CommandKit adoption and extended by this feature.
Accept fully resolved configuration and the domain inputs needed for
readiness checks through ordinary Go functions and concrete inputs. Return
errors identifying the affected stage and reason; CLI presentation remains with
the caller. Keep the package small and independent of CLI and provider transport.
@@ -411,6 +412,15 @@ retains configuration resolution, and correction validators retain evaluation of
proposed transcript changes. Future execution-readiness checks belong here when
they fit this responsibility; no generic check-registration framework is needed.
Complete invocation and domain-readiness checks before preparing output
resources, then begin execution after required preparation succeeds. Preflight
coordinates preparation through the output-session component, which owns file
handles, cleanup, and final publication. The CommandKit outline owns the
[output preparation contract](commandkit.md#output-resource-preparation-during-preflight--resolved)
and [output-failure rules](commandkit.md#output-failure-behavior--resolved).
This extends readiness preparation after handoff without adding configuration
resolution or duplicating filesystem machinery in the glossary checks.
Preflight checks every configured glossary-correction stage. Its combined selected
collections must supply at least one entry. Empty selections, null selections,
and nonempty selections whose collections collectively contain no entries fail
@@ -420,8 +430,9 @@ no protected vocabulary.
Complete preflight for the entire pipeline before executing any stage or making
processing model calls, so an empty later glossary stage cannot fail only after
earlier stages have incurred work. Report a clear readiness error identifying
the stage by position, module, and optional ID, with its selected collections.
earlier stages have incurred work. Reject it before output resource preparation.
Report a clear readiness error identifying the stage by position, module, and
optional ID, with its selected collections.
Do not silently skip the stage. This is a processing preflight phase, not another
operator-configured pipeline module; ordinary configuration/resource inspection
can still inspect valid empty collections.
@@ -782,7 +793,8 @@ do not compensate for a mismatch with a private configuration system.
Completion requires durable documentation before retiring this roadmap. Record
the glossary-configuration ownership boundary, typed handoff, and Audita preflight
responsibility in an ADR under the [documentation policy](../policy/documentation.md#architecture-decision-records).
responsibility, including its coordination with the output-session lifecycle, in
an ADR under the [documentation policy](../policy/documentation.md#architecture-decision-records).
This is an architectural ownership decision; routine scope and clearing choices
do not each require an ADR. The roadmap owns implementation status and sequencing.
@@ -811,7 +823,9 @@ Implement after ConfigKit, PromptKit, CommandKit, and ReleaseKit adoption, and
before final release validation and preparation. This lets the feature use the
adopted configuration and CLI conventions and avoids adapting it twice. Keep the
requirement visible during library design without expanding those migrations
into feature implementation.
into feature implementation. CommandKit adoption establishes the common preflight
package and output preparation; this feature adds glossary-specific readiness
checks to that phase.
1. Apply named collection selection, default file/directory underlay imports,
recursive placement, and the extensible entry content contract. Apply native
@@ -822,10 +836,12 @@ into feature implementation.
collection identities, validate configuration contracts, and deliver final typed
definitions and selections before transcript processing. Remove the redundant
standalone YAML parser and local composition path.
3. Add `internal/framework/preflight` and invoke it after configuration handoff.
Reject any glossary stage whose selected collections supply no entries before
any stage executes. Return readiness errors for CLI presentation, permit empty
collection definitions, and preserve inspection access to them.
3. Extend `internal/framework/preflight`, established during CommandKit adoption,
with the glossary-stage readiness check after configuration handoff. Reject any
glossary stage whose selected collections supply no entries before output
resource preparation or stage execution. Return readiness errors for CLI
presentation, permit empty collection definitions, and preserve inspection
access to them.
4. Derive rich LLM inputs, deduplicated protection vocabulary, and glossary-stage
exemptions from resolved configuration. Route the appropriate views through
proposal generation and validators without mutating configuration or other
@@ -936,8 +952,9 @@ Verify ConfigKit can hand off empty collections and empty effective selections
and that ordinary inspection remains available. Cover a selected empty collection,
several collectively empty collections, and mixed empty/populated collections.
Include an empty glossary stage late in the pipeline and verify preflight fails
before any stage executes or processing model calls occur. Check useful stage
and collection attribution in the readiness error, and no silent stage skipping.
before output resources are created, any stage executes, or processing model
calls occur. Check useful stage and collection attribution in the readiness
error, and no silent stage skipping.
Exercise the preflight package directly with resolved inputs and through the
processing entry point. Verify it leaves configuration unchanged and returns
readiness errors without provider calls or CLI presentation responsibilities.

41
docs/roadmap/library-adoption.md
View File

@@ -2,10 +2,11 @@
**Updated:** 2026-11-09
**Status:** Agreed direction with proposed sequencing. Initial outlines only;
no implementation or dependency changes have been made for this plan. Resolve
the remaining [output-failure decision](#open-decisions) before implementation
planning. ReleaseKit workflow and validator selection are resolved below.
**Status:** Design decisions are resolved across the master and all four child
roadmaps; ready for implementation planning. Implementation and dependency
changes remain pending. The sequence below includes common preflight and output
preparation during CommandKit adoption, followed by glossary-specific readiness
checks with the glossary feature.
## Objective and Document Ownership
@@ -127,11 +128,11 @@ implemented and accepted in Audita.
## Proposed Order of Operations
1. **Resolve the target contracts across all four libraries.** Establish the
1. **Carry the agreed contracts into implementation planning.** Use the selected
standard command inventory, discovery behavior, source ownership, and
reporting and release-workflow expectations before implementation. Identify
concrete upstream requests early; do not postpone CommandKit design until the
runtime is done.
reporting and release-workflow expectations to define implementation units.
Identify concrete upstream requests early; do not postpone CommandKit design
until the runtime is done.
Reconcile planned changes with the current architecture policy, especially
preference for standard-library CLI parsing. Backend, credential, and
concurrency ownership are resolved in the [PromptKit outline](promptkit.md):
@@ -156,15 +157,19 @@ implemented and accepted in Audita.
4. **Integrate the selected CommandKit core and both standard integrations.**
The upstream release gate is satisfied. Complete the designed operator
interface using the selected v0.6.0 set, adapting Audita domain handlers and
verifying the assembled application. Resolve concrete new integration gaps
upstream. This remains required scope for the next major release.
verifying the assembled application. Establish the named preflight package
and its [output preparation](commandkit.md#output-resource-preparation-during-preflight--resolved),
applying the [resolved output-failure rules](commandkit.md#output-failure-behavior--resolved).
Resolve concrete new integration gaps upstream. This remains required scope
for the next major release.
5. **Add stage-specific glossary inputs.** After all four library adoptions,
complete the [glossary-stage roadmap](glossary-stages.md). Use the adopted
configuration and CLI conventions to compose reusable glossary assets through
ConfigKit, then derive Audita's runtime views for ordered stages such as a
party pass followed by a spell pass. Remove the standalone glossary parsing
path. This sequencing avoids designing the feature around interfaces that
the upgrades replace.
path and extend the common preflight package with glossary-stage readiness
checks before output preparation. This sequencing avoids designing the
feature around interfaces that the upgrades replace.
6. **Validate and document the assembled application.** Complete the offline
behavioral checks from the outlines and the full Audita suite. Update canonical
configuration, CLI, operations, integration, and internal documentation with
@@ -306,15 +311,3 @@ Refresh [attribution](../oss/attribution.md) for the exact selected dependencies
and toolchain under [OSS policy](../policy/oss.md). The reviewed library grants
remain LGPL-3.0-or-later; remove old catalog entries when no longer incorporated
and retain shared license texts. Do not copy upstream policy inventories.
## Open Decisions
CommandKit release selection and ReleaseKit workflow/validator selection are
resolved above. One decision remains before implementation planning.
### Output-Failure Behavior
Resolve the [output-failure decision](commandkit.md#output-failure-behavior) in
the CommandKit roadmap. That document owns the recommendation, alternative, and
behavior contract. This decision completes the
remaining write-failure detail before implementing the output/reporting boundary.