From 976ed8940327b83472f18935e3f8c9857367fb6d Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Mon, 26 Oct 2026 00:00:25 +0000 Subject: [PATCH] Clarify reporting lifecycle and concurrency contracts --- docs/roadmap/reporting.md | 111 +++++++++++++++++++++++++++++++------- 1 file changed, 91 insertions(+), 20 deletions(-) diff --git a/docs/roadmap/reporting.md b/docs/roadmap/reporting.md index 96f8ad3..4e07f50 100644 --- a/docs/roadmap/reporting.md +++ b/docs/roadmap/reporting.md @@ -121,15 +121,22 @@ diagnostic delivery. ### Message prefixes and channels +Substantive output uses the result destination, conventionally stdout. Warning, +Info, Progress, Debug, and terminal explanations use the diagnostic destination, +conventionally stderr. Callers supply both destinations. + Use `WARN:`, `DEBUG:`, and `ERROR:` prefixes for simple warning, debug, and error messages respectively. Substantive output and Info messages have no prefix. For multiline diagnostics, prefix the opening line and preserve the structured detail. Integration adapters coordinate native diagnostic rendering to avoid duplicate severity labels or loss of upstream formatting. -Initially, Progress messages are unprefixed streamed lines. They use the -diagnostic channel so they do not corrupt substantive output such as JSON or -YAML. Terminal redraw and progress-bar behavior are outside this implementation. +Usage guidance belongs to the terminal usage explanation and remains visible in +quiet mode. Successful help output follows the substantive-output policy. + +Initially, Progress messages are unprefixed streamed lines. The diagnostic +destination keeps them separate from substantive output such as JSON or YAML. +Terminal redraw and progress-bar behavior are outside this implementation. ### Terminal failure model @@ -179,6 +186,17 @@ message derived from an application error cause. Native renderers can supply already-labelled content without receiving duplicate prefixes. Avoid text-based deduplication: separate events may legitimately produce identical messages. +Constructing a failure result attaches explanatory content without emitting or +independently scheduling an Error. Returning that result hands the explanation +to CommandKit for terminal delivery through the invocation's reporting facility. +The concrete API must express this distinction for both simple and structured +explanations. For example: + +| Handler action | Presentation | +| --- | --- | +| Construct a non-zero result with "Unable to export the requested report," then return it. | Construction emits nothing. CommandKit presents the message once during terminal delivery. | +| Return a failure with native diagnostic records and their source context, submitting independent warnings separately. | CommandKit presents the structured terminal explanation in every mode. Independent warnings follow their own visibility policy. | + ### Retention boundary (OD-3) CommandKit provides invocation-scoped CLI presentation. Logging and message @@ -215,10 +233,11 @@ the selection. | `--verbose --quiet --quiet=false` | Ordinary | | `--quiet=false --debug` | Debug | -Keep verbosity a presentation choice: a quiet invocation still performs its -requested operation and returns the ordinary outcome. Apply the policy through -CommandKit-managed reporting and streams. Output written directly to process -streams or external destinations remains application-owned. +Keep verbosity a presentation choice: quiet preserves the requested operation's +semantics and rendering failures. Delivery failures depend on the output actually +requested; suppressing a destination write can avoid a delivery failure. Apply +the policy through CommandKit-managed reporting and streams. Output written +directly to process streams or external destinations remains application-owned. ### Progress selection @@ -310,7 +329,7 @@ Unknown flags, invalid values, and invalid command paths remain invocation failures with error diagnostics. Start debug instrumentation with routing, alias resolution, verbosity selection, -and invocation completion. Other useful candidates can be added where they +and handler completion. Other useful candidates can be added where they explain a consequential decision: | Event | Useful detail | @@ -322,7 +341,12 @@ explain a consequential decision: | Help-route selection | A canonical command or alias-only discovery context was selected. | | Repeated singleton flag | A later occurrence replaced an earlier occurrence, without printing their values. | | Input preparation | Canonical input validation completed and the handler is starting. | -| Invocation completion | Canonical command, outcome classification, and status. | +| Handler completion | Canonical command and the handler's returned classification and status, before final delivery. | + +The handler-completion event describes the handler's result. The final returned +invocation outcome also includes delivery failures and remains authoritative. +Present this event before terminal failure presentation; do not emit a completion +message afterward. Use the final selected mode to present earlier parser decisions. Collect the relevant events during parsing and apply visibility after mode selection: @@ -343,8 +367,12 @@ Use synchronous delivery. Present immediate messages and substantive writes in submission order. After the handler returns: 1. Deliver eligible deferred messages in queue order, without sorting by severity. -2. Present any terminal failure explanation. -3. Close the invocation's reporting facility and return the final outcome. +2. Finish ordinary reporting, including any handler-completion debug message and + flushing any CommandKit-owned buffers for ordinary output. Account for failures + discovered during this delivery before preparing the terminal explanation. +3. Present any terminal failure explanation and complete its delivery, including + any CommandKit-owned buffering of that explanation. +4. Close the invocation's reporting facility and return the final outcome. Deferred messages remain eligible when the operation fails, subject to verbosity. Deferral controls timing; applications queue success summaries only after the @@ -358,6 +386,14 @@ handler returns and participates in normal reporting. Applications complete any reporting goroutines before returning. After finalization, writes and message submissions return a closed-reporter error and emit nothing. +Reporter operations within an invocation are safe to call concurrently. +Serialize access to reporting state and delivery so individual messages do not +interleave. Concurrent submissions have no predetermined order; deferred messages +follow the order in which they enter the queue. Multiple writes composing one +application report still require application coordination if they must remain +together. This contract covers operations through one invocation's reporter; +coordination with writes outside that reporter remains caller-owned. + Finalize any buffers owned by CommandKit. Caller-supplied writers remain caller-owned; CommandKit does not automatically close them or invoke arbitrary flushing interfaces. @@ -373,7 +409,21 @@ related explanatory context. Unrelated warnings and informational messages remain subject to the selected mode. Apply the selected retention boundary without accumulating a message history. -### Write failures and execution flow +### Rendering failures, write failures, and execution flow + +Distinguish who invokes a renderer and whether its destination failed: + +| Failure source | Responsibility and handling | +| --- | --- | +| An encoder or renderer called directly by the application | The application handles its returned error and chooses the outcome and explanation. | +| A rendering callback invoked by CommandKit | CommandKit handles its returned error as a presentation failure and reflects it in the invocation outcome. | +| A destination rejects a write | CommandKit records the delivery failure and stops further writes to that destination. | + +A rendering failure may occur before any write or while the destination remains +healthy. It does not by itself establish a failed destination. CommandKit can +explain its rendering failure through a working diagnostic destination. These +responsibilities also apply when rendering into a discard destination under +quiet mode. Failure to deliver requested output makes an otherwise successful invocation unsuccessful. Apply this rule to substantive output, Warning, Info, Progress, @@ -390,6 +440,10 @@ failure. If the application already returned failure, preserve its classificatio status, and explanation, and include the delivery failure separately. Reporting failures belong to CommandKit's own error-handling responsibility. +Apply the same outcome rule to rendering failures from callbacks invoked by +CommandKit. In particular, a usage failure retains its usage classification and +status when subsequent presentation fails. + Stop writing to a destination after it fails. A substantive-output failure can receive a terminal explanation through the diagnostic destination if it still works. If diagnostic delivery fails, return that information to the caller. @@ -483,6 +537,13 @@ application's recovery decisions or classify arbitrary Go errors. Applications using upstream libraries directly retain the same choice over outcomes and messages as for their other operations. +An integration may receive both a useful substantive report and an operation +failure, as PromptKit smoke testing supports. Preserve the report under the +ordinary substantive-output policy, then present the terminal explanation. +Quiet mode suppresses the report while retaining the explanation. Having a +report does not imply a successful outcome, and returning a failure does not +discard substantive output already prepared for delivery before the terminal step. + Debug output follows the same upstream disclosure and safe-rendering contracts. Enabling debug does not authorize secret disclosure or bypass owning-library rules. @@ -495,6 +556,8 @@ rules. emit no Progress messages and the distinction between Info and Progress. Cover silent recovery, an explicit Warning with or without a Go error, and a terminal failure with an explicit explanation and no Go error cause. + Include simple and structured failure construction without early emission, + and an integration returning both a substantive report and a failure. 2. Review current diagnostic batching, parsing, help, outcome presentation, and writer ownership to identify the smallest shared changes. Include invocation failures that occur before a handler receives its `Invocation`. @@ -517,17 +580,25 @@ rules. of normal reporting after terminal error presentation, including deferred and late writes under the selected OD-4 behavior. Cover immediate and deferred ordering on success and failure, handler cleanup, automatic finalization, - and ownership of caller-supplied writers. Verify ignored immediate write - errors and deferred delivery failures affect the final outcome, existing - failures retain their classification and status, and failed destinations + flushing ordinary buffers before terminal delivery, handler-completion debug + placement, and ownership of caller-supplied writers. Verify concurrent reporter + calls protect shared state and keep individual messages together, without + requiring a predetermined order among concurrent submissions. Verify ignored + immediate write errors and deferred delivery failures affect the final outcome, + existing failures retain their classification and status, and failed destinations receive no further writes. Cover diagnostic-destination failure without fallback into substantive output, quiet-mode rendering failures, and suppression without destination writes or incomplete-delivery status. -6. Update GoDoc, the CLI reference, consumer guidance, and release documentation - when implemented. Reconcile the selected error-handling boundary with - ADR 0005's existing outcome and error-identity commitments before changing - those contracts. Record any resulting architectural change in a new ADR; - exact reporting APIs and flag syntax remain API/CLI contracts. + Distinguish renderer failures from destination failures, including callbacks + that fail without writing and continued terminal delivery through a healthy + diagnostic destination. Cover usage-status preservation on presentation + failure, all category destinations, and usage guidance under quiet mode. +6. Before changing architectural contracts, reconcile the selected error-handling + boundary with ADR 0005's existing outcome and error-identity commitments and + the corresponding commitments in the [architecture policy](../policy/architecture.md). + Record architectural changes in a new ADR and align the policy. Update GoDoc, + the CLI reference, consumer guidance, and release documentation when + implemented. Exact reporting APIs and flag syntax remain API/CLI contracts. ## Remaining design work