Resolve output failures and preflight resource preparation
This commit is contained in:
@@ -2,9 +2,10 @@
|
||||
|
||||
**Updated:** 2026-11-09
|
||||
|
||||
**Status:** Adoption target selected; the upstream release gate is satisfied.
|
||||
Implementation remains pending. The [master roadmap](library-adoption.md) owns
|
||||
shared direction, exact dependency revisions, and sequencing; ConfigKit and
|
||||
**Status:** Adoption target and design decisions resolved; the upstream release
|
||||
gate is satisfied. Implementation remains pending. The
|
||||
[master roadmap](library-adoption.md) owns shared direction, exact dependency
|
||||
revisions, and sequencing; ConfigKit and
|
||||
PromptKit contracts are developed in [their](configkit.md) [outlines](promptkit.md).
|
||||
|
||||
## Baseline and Target
|
||||
@@ -149,6 +150,84 @@ Check effective mode after native ordering, boolean values, and aliases:
|
||||
explicit destination. `--debug --quiet` also requires one; debug capture remains
|
||||
independent once the invocation passes this check.
|
||||
|
||||
## Output-Failure Behavior — Resolved
|
||||
|
||||
Require successful delivery of the transcript and enabled ordinary reports.
|
||||
Debug artifacts remain optional even when capture was explicitly requested.
|
||||
Adopt CommandKit's native handling of CLI output and diagnostic write failures.
|
||||
|
||||
| Output | Failure behavior |
|
||||
| --- | --- |
|
||||
| Transcript file or stdout | Fail the invocation. |
|
||||
| Enabled ordinary reports | Fail the invocation if any required report cannot be written. |
|
||||
| Debug directory or individual debug artifacts | Continue processing and submit a redacted warning through CommandKit. |
|
||||
| CommandKit output and diagnostic streams | Preserve the native outcome and delivery-error behavior. |
|
||||
|
||||
Ordinary reports describe incomplete work, recoverable model failures, and
|
||||
repairs. When enabled, delivering that account is part of completing the run.
|
||||
Debug artifacts provide supplemental investigative detail; their loss alone does
|
||||
not invalidate a delivered transcript and reports. Do not add a debug strictness
|
||||
option.
|
||||
|
||||
Detect required-destination problems before model work wherever practical through
|
||||
the [output preparation below](#output-resource-preparation-during-preflight--resolved).
|
||||
If a required output fails later, retain successfully written files, fail the
|
||||
invocation, and attempt to finalize the remaining reports with the failure
|
||||
recorded. A report failure after transcript publication must not erase that
|
||||
transcript or repeat model calls. Preserve the original processing failure when
|
||||
secondary reporting or cleanup also fails. Emit the completion summary only
|
||||
after processing and all enabled required outputs have succeeded, including
|
||||
required final writes, closes, and file replacements.
|
||||
|
||||
Use the existing redaction and resolved visibility rules for debug warnings.
|
||||
In particular, `--debug --quiet` still attempts capture but suppresses warnings
|
||||
about missing optional artifacts; required-output failures retain the normal
|
||||
fatal-error explanation. A separate failure while writing a warning can still
|
||||
make the invocation fail under CommandKit's native
|
||||
[output-failure contract](https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.6.0/docs/cli.md#output-failures).
|
||||
Keep that outcome under upstream ownership rather than overriding it to make
|
||||
diagnostic stream failures nonfatal.
|
||||
|
||||
## Output Resource Preparation During Preflight — Resolved
|
||||
|
||||
Prepare output resources as part of Audita's expressly named
|
||||
`internal/framework/preflight` phase, after ConfigKit handoff and before model
|
||||
calls. Follow the [preflight package boundary](glossary-stages.md#empty-collections-and-audita-preflight--resolved).
|
||||
Complete configuration, invocation, and domain-readiness checks for the entire
|
||||
pipeline first, including the quiet-destination rule and active destination
|
||||
collision checks. Only then prepare output resources, and begin execution after
|
||||
required preparation succeeds. Disabled report and debug destinations remain
|
||||
inactive.
|
||||
|
||||
- For a regular transcript destination, create and open a temporary file in its
|
||||
destination directory before processing. Preserve the existing destination
|
||||
until processing and writing succeed, then replace it. This also preserves
|
||||
input when the transcript intentionally replaces its own source; preparation
|
||||
must not truncate the final path.
|
||||
- Initialize enabled ordinary reports before processing, using the existing
|
||||
`in_progress` report lifecycle and per-file replacement. An initialization
|
||||
failure stops the invocation before model work.
|
||||
- When debug capture is enabled, create its directory and initial artifacts
|
||||
early. Create later artifacts as needed instead of opening every possible
|
||||
artifact in advance. Debug preparation failures follow the nonfatal artifact
|
||||
rule above.
|
||||
- Use CommandKit's supplied writer for transcript stdout. Defer opening pipe and
|
||||
device destinations until output delivery; in particular, opening a named pipe
|
||||
during preflight could block waiting for its reader. Do not replace these
|
||||
destinations with regular temporary files.
|
||||
|
||||
The preflight package coordinates preparation through the output-session
|
||||
component. Reuse and adapt that component to own prepared handles, cleanup, and
|
||||
final publication; keep filesystem machinery out of individual readiness checks.
|
||||
Close owned handles and remove unpublished temporary files on failure or
|
||||
cancellation, while retaining published outputs and retained diagnostic evidence.
|
||||
The caller retains ownership of supplied standard streams.
|
||||
|
||||
Opening resources early establishes readiness at that point; later writes,
|
||||
closes, and replacements can still fail. Apply the output-failure rules throughout
|
||||
execution and finalization. Preserve per-file publication without adding a
|
||||
transaction across the transcript and reports.
|
||||
|
||||
## Request Preview, Validation, and Smoke Commands — Resolved
|
||||
|
||||
Include standard ckprompt request preview, request validation, and the complete
|
||||
@@ -198,8 +277,11 @@ released dependency set when implementing this outline.
|
||||
framework around CommandKit.
|
||||
4. Adapt domain process handlers to reporting/outcome boundaries, preserving
|
||||
transcript/artifact integrity, cancellation, and partial success. Apply the
|
||||
resolved visibility and completion-summary rules above; resolve
|
||||
[output-failure behavior](#output-failure-behavior) before implementation.
|
||||
resolved visibility, completion-summary, and
|
||||
[output-failure rules](#output-failure-behavior--resolved). Establish the
|
||||
preflight package and its [output preparation](#output-resource-preparation-during-preflight--resolved)
|
||||
through the output-session component. The later glossary feature extends the
|
||||
same package with its collection-readiness checks before resource preparation.
|
||||
Standard command statuses, including comparison status
|
||||
3, should not be forced into the old Audita-only status table without a domain
|
||||
reason. Validate quiet mode against the transcript destination before creating
|
||||
@@ -233,6 +315,23 @@ runs. Fatal error explanations remain visible in every mode, and a required-outp
|
||||
failure must not produce a successful completion summary. Cover ordinary reporting disabled as
|
||||
well as enabled, and document the loss of the former default summary.
|
||||
|
||||
Cover required-output preparation failure before model calls, late write/close/
|
||||
replacement failures, and report failure after a transcript has been published.
|
||||
Verify retention of published output, preservation of the original processing
|
||||
failure when reporting also fails, and attempts to finalize available reports.
|
||||
Exercise optional debug-directory and artifact failures, including suppressed
|
||||
warnings in quiet mode, and the native CommandKit outcome when a diagnostic
|
||||
write itself fails. Keep these checks focused on Audita's integration and output
|
||||
responsibilities rather than duplicating upstream reporter tests.
|
||||
|
||||
Verify that readiness failures occur before output resources are created.
|
||||
Check that preparing regular transcript output preserves existing content,
|
||||
including an in-place transcript, and that failure or cancellation closes owned
|
||||
resources and removes unpublished temporary files. Cover enabled report
|
||||
initialization, inactive disabled destinations, and debug artifacts created as
|
||||
needed. Confirm that preflight does not open pipe/device destinations and that
|
||||
supplied standard streams remain caller-owned.
|
||||
|
||||
Cover omitted and empty output selections in effective quiet mode, including
|
||||
with reports or debug capture enabled: reject before model calls and artifact
|
||||
creation. Accept valid explicit file and discard destinations. Exercise native
|
||||
@@ -299,31 +398,3 @@ capture/mode combinations, aliases, and quiet-destination policy. The resolved
|
||||
graph selected the intended ConfigKit v1.1.0 and PromptKit v0.20.0 dependencies.
|
||||
These checks establish the upstream gate; they do not replace Audita's assembled
|
||||
integration acceptance. No additional upstream request was identified.
|
||||
|
||||
## Open Decisions
|
||||
|
||||
### Output-Failure Behavior
|
||||
|
||||
This is the remaining open decision in the
|
||||
[master roadmap](library-adoption.md#open-decisions).
|
||||
Specify how failed transcript, report, debug-artifact, and standard reporter writes
|
||||
influence the process outcome. The existing visibility rules and prohibition on
|
||||
a successful completion summary after required-output failure remain selected.
|
||||
|
||||
**Recommendation:** required transcript and enabled ordinary-report failures fail
|
||||
the invocation. Optional debug-artifact failures remain nonfatal and use the
|
||||
standard reporting path with existing redaction and visibility rules. Standard
|
||||
CommandKit reporter failures follow native behavior rather than an Audita-owned
|
||||
replacement policy. This preserves the distinction between delivering the
|
||||
requested product and retaining supplemental diagnostic evidence, while avoiding
|
||||
a second reporting implementation. Validate the exact adopted reporter behavior
|
||||
and cover early and late output failures, including a report failure after the
|
||||
transcript has already been written.
|
||||
|
||||
**Alternative:** also make debug-artifact failures fatal when debug capture was
|
||||
explicitly requested. This treats capture as a required deliverable and makes
|
||||
incomplete evidence visible in the process outcome, but can fail an otherwise
|
||||
successful transcript run solely because supplemental evidence could not be
|
||||
saved. Standard reporter behavior would still remain upstream-owned. Choose the
|
||||
artifact requirement deliberately rather than infer it from the debug reporting
|
||||
mode, which remains independent of capture.
|
||||
|
||||
Reference in New Issue
Block a user