From 866e4b9b23e1c3dfbff6ce151d3913044a1ad9e6 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 9 Nov 2026 15:25:57 +0000 Subject: [PATCH] Resolve output failures and preflight resource preparation --- docs/roadmap/commandkit.md | 137 +++++++++++++++++++++++-------- docs/roadmap/glossary-stages.md | 41 ++++++--- docs/roadmap/library-adoption.md | 41 ++++----- 3 files changed, 150 insertions(+), 69 deletions(-) diff --git a/docs/roadmap/commandkit.md b/docs/roadmap/commandkit.md index a318454..1a463b0 100644 --- a/docs/roadmap/commandkit.md +++ b/docs/roadmap/commandkit.md @@ -2,9 +2,10 @@ **Updated:** 2026-11-09 -**Status:** Adoption target selected; the upstream release gate is satisfied. -Implementation remains pending. The [master roadmap](library-adoption.md) owns -shared direction, exact dependency revisions, and sequencing; ConfigKit and +**Status:** Adoption target and design decisions resolved; the upstream release +gate is satisfied. Implementation remains pending. The +[master roadmap](library-adoption.md) owns shared direction, exact dependency +revisions, and sequencing; ConfigKit and PromptKit contracts are developed in [their](configkit.md) [outlines](promptkit.md). ## Baseline and Target @@ -149,6 +150,84 @@ Check effective mode after native ordering, boolean values, and aliases: explicit destination. `--debug --quiet` also requires one; debug capture remains independent once the invocation passes this check. +## Output-Failure Behavior — Resolved + +Require successful delivery of the transcript and enabled ordinary reports. +Debug artifacts remain optional even when capture was explicitly requested. +Adopt CommandKit's native handling of CLI output and diagnostic write failures. + +| Output | Failure behavior | +| --- | --- | +| Transcript file or stdout | Fail the invocation. | +| Enabled ordinary reports | Fail the invocation if any required report cannot be written. | +| Debug directory or individual debug artifacts | Continue processing and submit a redacted warning through CommandKit. | +| CommandKit output and diagnostic streams | Preserve the native outcome and delivery-error behavior. | + +Ordinary reports describe incomplete work, recoverable model failures, and +repairs. When enabled, delivering that account is part of completing the run. +Debug artifacts provide supplemental investigative detail; their loss alone does +not invalidate a delivered transcript and reports. Do not add a debug strictness +option. + +Detect required-destination problems before model work wherever practical through +the [output preparation below](#output-resource-preparation-during-preflight--resolved). +If a required output fails later, retain successfully written files, fail the +invocation, and attempt to finalize the remaining reports with the failure +recorded. A report failure after transcript publication must not erase that +transcript or repeat model calls. Preserve the original processing failure when +secondary reporting or cleanup also fails. Emit the completion summary only +after processing and all enabled required outputs have succeeded, including +required final writes, closes, and file replacements. + +Use the existing redaction and resolved visibility rules for debug warnings. +In particular, `--debug --quiet` still attempts capture but suppresses warnings +about missing optional artifacts; required-output failures retain the normal +fatal-error explanation. A separate failure while writing a warning can still +make the invocation fail under CommandKit's native +[output-failure contract](https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.6.0/docs/cli.md#output-failures). +Keep that outcome under upstream ownership rather than overriding it to make +diagnostic stream failures nonfatal. + +## Output Resource Preparation During Preflight — Resolved + +Prepare output resources as part of Audita's expressly named +`internal/framework/preflight` phase, after ConfigKit handoff and before model +calls. Follow the [preflight package boundary](glossary-stages.md#empty-collections-and-audita-preflight--resolved). +Complete configuration, invocation, and domain-readiness checks for the entire +pipeline first, including the quiet-destination rule and active destination +collision checks. Only then prepare output resources, and begin execution after +required preparation succeeds. Disabled report and debug destinations remain +inactive. + +- For a regular transcript destination, create and open a temporary file in its + destination directory before processing. Preserve the existing destination + until processing and writing succeed, then replace it. This also preserves + input when the transcript intentionally replaces its own source; preparation + must not truncate the final path. +- Initialize enabled ordinary reports before processing, using the existing + `in_progress` report lifecycle and per-file replacement. An initialization + failure stops the invocation before model work. +- When debug capture is enabled, create its directory and initial artifacts + early. Create later artifacts as needed instead of opening every possible + artifact in advance. Debug preparation failures follow the nonfatal artifact + rule above. +- Use CommandKit's supplied writer for transcript stdout. Defer opening pipe and + device destinations until output delivery; in particular, opening a named pipe + during preflight could block waiting for its reader. Do not replace these + destinations with regular temporary files. + +The preflight package coordinates preparation through the output-session +component. Reuse and adapt that component to own prepared handles, cleanup, and +final publication; keep filesystem machinery out of individual readiness checks. +Close owned handles and remove unpublished temporary files on failure or +cancellation, while retaining published outputs and retained diagnostic evidence. +The caller retains ownership of supplied standard streams. + +Opening resources early establishes readiness at that point; later writes, +closes, and replacements can still fail. Apply the output-failure rules throughout +execution and finalization. Preserve per-file publication without adding a +transaction across the transcript and reports. + ## Request Preview, Validation, and Smoke Commands — Resolved Include standard ckprompt request preview, request validation, and the complete @@ -198,8 +277,11 @@ released dependency set when implementing this outline. framework around CommandKit. 4. Adapt domain process handlers to reporting/outcome boundaries, preserving transcript/artifact integrity, cancellation, and partial success. Apply the - resolved visibility and completion-summary rules above; resolve - [output-failure behavior](#output-failure-behavior) before implementation. + resolved visibility, completion-summary, and + [output-failure rules](#output-failure-behavior--resolved). Establish the + preflight package and its [output preparation](#output-resource-preparation-during-preflight--resolved) + through the output-session component. The later glossary feature extends the + same package with its collection-readiness checks before resource preparation. Standard command statuses, including comparison status 3, should not be forced into the old Audita-only status table without a domain reason. Validate quiet mode against the transcript destination before creating @@ -233,6 +315,23 @@ runs. Fatal error explanations remain visible in every mode, and a required-outp failure must not produce a successful completion summary. Cover ordinary reporting disabled as well as enabled, and document the loss of the former default summary. +Cover required-output preparation failure before model calls, late write/close/ +replacement failures, and report failure after a transcript has been published. +Verify retention of published output, preservation of the original processing +failure when reporting also fails, and attempts to finalize available reports. +Exercise optional debug-directory and artifact failures, including suppressed +warnings in quiet mode, and the native CommandKit outcome when a diagnostic +write itself fails. Keep these checks focused on Audita's integration and output +responsibilities rather than duplicating upstream reporter tests. + +Verify that readiness failures occur before output resources are created. +Check that preparing regular transcript output preserves existing content, +including an in-place transcript, and that failure or cancellation closes owned +resources and removes unpublished temporary files. Cover enabled report +initialization, inactive disabled destinations, and debug artifacts created as +needed. Confirm that preflight does not open pipe/device destinations and that +supplied standard streams remain caller-owned. + Cover omitted and empty output selections in effective quiet mode, including with reports or debug capture enabled: reject before model calls and artifact creation. Accept valid explicit file and discard destinations. Exercise native @@ -299,31 +398,3 @@ capture/mode combinations, aliases, and quiet-destination policy. The resolved graph selected the intended ConfigKit v1.1.0 and PromptKit v0.20.0 dependencies. These checks establish the upstream gate; they do not replace Audita's assembled integration acceptance. No additional upstream request was identified. - -## Open Decisions - -### Output-Failure Behavior - -This is the remaining open decision in the -[master roadmap](library-adoption.md#open-decisions). -Specify how failed transcript, report, debug-artifact, and standard reporter writes -influence the process outcome. The existing visibility rules and prohibition on -a successful completion summary after required-output failure remain selected. - -**Recommendation:** required transcript and enabled ordinary-report failures fail -the invocation. Optional debug-artifact failures remain nonfatal and use the -standard reporting path with existing redaction and visibility rules. Standard -CommandKit reporter failures follow native behavior rather than an Audita-owned -replacement policy. This preserves the distinction between delivering the -requested product and retaining supplemental diagnostic evidence, while avoiding -a second reporting implementation. Validate the exact adopted reporter behavior -and cover early and late output failures, including a report failure after the -transcript has already been written. - -**Alternative:** also make debug-artifact failures fatal when debug capture was -explicitly requested. This treats capture as a required deliverable and makes -incomplete evidence visible in the process outcome, but can fail an otherwise -successful transcript run solely because supplemental evidence could not be -saved. Standard reporter behavior would still remain upstream-owned. Choose the -artifact requirement deliberately rather than infer it from the debug reporting -mode, which remains independent of capture. diff --git a/docs/roadmap/glossary-stages.md b/docs/roadmap/glossary-stages.md index b98ebf0..a1fd980 100644 --- a/docs/roadmap/glossary-stages.md +++ b/docs/roadmap/glossary-stages.md @@ -394,7 +394,7 @@ assets, whether imported for reuse, inspected, or selected alongside populated collections. A nonempty list of selected collection names does not establish that a stage has entries to process. -Add an explicit Audita preflight phase after ConfigKit hands off the fully +Use an explicit Audita preflight phase after ConfigKit hands off the fully resolved configuration and before pipeline execution begins. Preflight evaluates execution-readiness business rules against that resolved data. It does not load sources, resolve configuration, restore defaults, or change stage inheritance. @@ -402,7 +402,8 @@ Keep the nonempty glossary-stage check in Audita preflight rather than ConfigKit configuration-validation callbacks. Implement this phase in an expressly named `internal/framework/preflight` -package. Accept fully resolved configuration and the domain inputs needed for +package, established during CommandKit adoption and extended by this feature. +Accept fully resolved configuration and the domain inputs needed for readiness checks through ordinary Go functions and concrete inputs. Return errors identifying the affected stage and reason; CLI presentation remains with the caller. Keep the package small and independent of CLI and provider transport. @@ -411,6 +412,15 @@ retains configuration resolution, and correction validators retain evaluation of proposed transcript changes. Future execution-readiness checks belong here when they fit this responsibility; no generic check-registration framework is needed. +Complete invocation and domain-readiness checks before preparing output +resources, then begin execution after required preparation succeeds. Preflight +coordinates preparation through the output-session component, which owns file +handles, cleanup, and final publication. The CommandKit outline owns the +[output preparation contract](commandkit.md#output-resource-preparation-during-preflight--resolved) +and [output-failure rules](commandkit.md#output-failure-behavior--resolved). +This extends readiness preparation after handoff without adding configuration +resolution or duplicating filesystem machinery in the glossary checks. + Preflight checks every configured glossary-correction stage. Its combined selected collections must supply at least one entry. Empty selections, null selections, and nonempty selections whose collections collectively contain no entries fail @@ -420,8 +430,9 @@ no protected vocabulary. Complete preflight for the entire pipeline before executing any stage or making processing model calls, so an empty later glossary stage cannot fail only after -earlier stages have incurred work. Report a clear readiness error identifying -the stage by position, module, and optional ID, with its selected collections. +earlier stages have incurred work. Reject it before output resource preparation. +Report a clear readiness error identifying the stage by position, module, and +optional ID, with its selected collections. Do not silently skip the stage. This is a processing preflight phase, not another operator-configured pipeline module; ordinary configuration/resource inspection can still inspect valid empty collections. @@ -782,7 +793,8 @@ do not compensate for a mismatch with a private configuration system. Completion requires durable documentation before retiring this roadmap. Record the glossary-configuration ownership boundary, typed handoff, and Audita preflight -responsibility in an ADR under the [documentation policy](../policy/documentation.md#architecture-decision-records). +responsibility, including its coordination with the output-session lifecycle, in +an ADR under the [documentation policy](../policy/documentation.md#architecture-decision-records). This is an architectural ownership decision; routine scope and clearing choices do not each require an ADR. The roadmap owns implementation status and sequencing. @@ -811,7 +823,9 @@ Implement after ConfigKit, PromptKit, CommandKit, and ReleaseKit adoption, and before final release validation and preparation. This lets the feature use the adopted configuration and CLI conventions and avoids adapting it twice. Keep the requirement visible during library design without expanding those migrations -into feature implementation. +into feature implementation. CommandKit adoption establishes the common preflight +package and output preparation; this feature adds glossary-specific readiness +checks to that phase. 1. Apply named collection selection, default file/directory underlay imports, recursive placement, and the extensible entry content contract. Apply native @@ -822,10 +836,12 @@ into feature implementation. collection identities, validate configuration contracts, and deliver final typed definitions and selections before transcript processing. Remove the redundant standalone YAML parser and local composition path. -3. Add `internal/framework/preflight` and invoke it after configuration handoff. - Reject any glossary stage whose selected collections supply no entries before - any stage executes. Return readiness errors for CLI presentation, permit empty - collection definitions, and preserve inspection access to them. +3. Extend `internal/framework/preflight`, established during CommandKit adoption, + with the glossary-stage readiness check after configuration handoff. Reject any + glossary stage whose selected collections supply no entries before output + resource preparation or stage execution. Return readiness errors for CLI + presentation, permit empty collection definitions, and preserve inspection + access to them. 4. Derive rich LLM inputs, deduplicated protection vocabulary, and glossary-stage exemptions from resolved configuration. Route the appropriate views through proposal generation and validators without mutating configuration or other @@ -936,8 +952,9 @@ Verify ConfigKit can hand off empty collections and empty effective selections and that ordinary inspection remains available. Cover a selected empty collection, several collectively empty collections, and mixed empty/populated collections. Include an empty glossary stage late in the pipeline and verify preflight fails -before any stage executes or processing model calls occur. Check useful stage -and collection attribution in the readiness error, and no silent stage skipping. +before output resources are created, any stage executes, or processing model +calls occur. Check useful stage and collection attribution in the readiness +error, and no silent stage skipping. Exercise the preflight package directly with resolved inputs and through the processing entry point. Verify it leaves configuration unchanged and returns readiness errors without provider calls or CLI presentation responsibilities. diff --git a/docs/roadmap/library-adoption.md b/docs/roadmap/library-adoption.md index 17d210e..b447356 100644 --- a/docs/roadmap/library-adoption.md +++ b/docs/roadmap/library-adoption.md @@ -2,10 +2,11 @@ **Updated:** 2026-11-09 -**Status:** Agreed direction with proposed sequencing. Initial outlines only; -no implementation or dependency changes have been made for this plan. Resolve -the remaining [output-failure decision](#open-decisions) before implementation -planning. ReleaseKit workflow and validator selection are resolved below. +**Status:** Design decisions are resolved across the master and all four child +roadmaps; ready for implementation planning. Implementation and dependency +changes remain pending. The sequence below includes common preflight and output +preparation during CommandKit adoption, followed by glossary-specific readiness +checks with the glossary feature. ## Objective and Document Ownership @@ -127,11 +128,11 @@ implemented and accepted in Audita. ## Proposed Order of Operations -1. **Resolve the target contracts across all four libraries.** Establish the +1. **Carry the agreed contracts into implementation planning.** Use the selected standard command inventory, discovery behavior, source ownership, and - reporting and release-workflow expectations before implementation. Identify - concrete upstream requests early; do not postpone CommandKit design until the - runtime is done. + reporting and release-workflow expectations to define implementation units. + Identify concrete upstream requests early; do not postpone CommandKit design + until the runtime is done. Reconcile planned changes with the current architecture policy, especially preference for standard-library CLI parsing. Backend, credential, and concurrency ownership are resolved in the [PromptKit outline](promptkit.md): @@ -156,15 +157,19 @@ implemented and accepted in Audita. 4. **Integrate the selected CommandKit core and both standard integrations.** The upstream release gate is satisfied. Complete the designed operator interface using the selected v0.6.0 set, adapting Audita domain handlers and - verifying the assembled application. Resolve concrete new integration gaps - upstream. This remains required scope for the next major release. + verifying the assembled application. Establish the named preflight package + and its [output preparation](commandkit.md#output-resource-preparation-during-preflight--resolved), + applying the [resolved output-failure rules](commandkit.md#output-failure-behavior--resolved). + Resolve concrete new integration gaps upstream. This remains required scope + for the next major release. 5. **Add stage-specific glossary inputs.** After all four library adoptions, complete the [glossary-stage roadmap](glossary-stages.md). Use the adopted configuration and CLI conventions to compose reusable glossary assets through ConfigKit, then derive Audita's runtime views for ordered stages such as a party pass followed by a spell pass. Remove the standalone glossary parsing - path. This sequencing avoids designing the feature around interfaces that - the upgrades replace. + path and extend the common preflight package with glossary-stage readiness + checks before output preparation. This sequencing avoids designing the + feature around interfaces that the upgrades replace. 6. **Validate and document the assembled application.** Complete the offline behavioral checks from the outlines and the full Audita suite. Update canonical configuration, CLI, operations, integration, and internal documentation with @@ -306,15 +311,3 @@ Refresh [attribution](../oss/attribution.md) for the exact selected dependencies and toolchain under [OSS policy](../policy/oss.md). The reviewed library grants remain LGPL-3.0-or-later; remove old catalog entries when no longer incorporated and retain shared license texts. Do not copy upstream policy inventories. - -## Open Decisions - -CommandKit release selection and ReleaseKit workflow/validator selection are -resolved above. One decision remains before implementation planning. - -### Output-Failure Behavior - -Resolve the [output-failure decision](commandkit.md#output-failure-behavior) in -the CommandKit roadmap. That document owns the recommendation, alternative, and -behavior contract. This decision completes the -remaining write-failure detail before implementing the output/reporting boundary.