commandkit/docs/roadmap/reporting.md

# Output policy and invocation reporting

Status: behavioral decisions selected through OD-4. Message categories, all four
output modes, uniform treatment of help and version, message prefixes, and
OD-1's flag grammar and scope (including independent Progress control) are selected.
OD-2's unified reporting direction, the terminal failure model, and application
ownership of error handling are selected.
OD-3's failure presentation and retention boundaries are selected.
OD-4's delivery lifecycle, write-failure handling, and quiet-mode rendering are
selected. Exact API names, signatures, and representations remain to be
finalized. Implementation has not started.

## Purpose

Provide consistent handling of `--quiet`, `--verbose`, `--debug`, and
`--progress` across application commands and optional integrations. Reduce
repeated output classification, diagnostic presentation, and delivery bookkeeping
where CommandKit can provide a meaningful benefit.

Ordinary Go writers and formatting functions already make application output
straightforward. Design reporting conveniences around the shared output policy
and invocation lifecycle. Applications continue to own their report content,
domain behavior, and operation outcomes.

## CLI presentation and error-handling ownership

Applications decide how to handle errors from their operations: recover, retry,
continue silently, report a Warning, or end the invocation with a failure.
A Go `error` does not determine a CLI message category. Warnings may describe
issues with or without an underlying Go error, and handling an error does not
require emitting a message.

CommandKit handles the presentation the application requests: message category,
visibility, formatting, timing, and delivery. For terminal application failures,
the application supplies a non-zero outcome and an operator-facing explanation.
Reporting does not require the underlying Go error or undertake its tracking,
recovery, or identity preservation.

Applications can explicitly present `err.Error()` as a Warning through the
ordinary message API, illustrated by `report.Warn(err.Error())`. Defer a separate
`WarnError` convenience helper; this conversion already requires little code.

CommandKit retains responsibility for handling its own errors, including
routing, parsing, and reporting failures. Integration-owned commands similarly
adapt their upstream APIs into deliberate command outcomes and presentations.
This boundary governs application error handling; ordinary Go error returns
from CommandKit APIs and writers remain available where appropriate.

## Current baseline

[`Streams` and `Invocation`](../../identity.go) expose caller-selected output
and diagnostic writers. [`Outcome` and `Diagnostic`](../../outcome.go) carry
operation causes, diagnostic writers, and delivery state. Handlers can return
`Failure(err)` or attach diagnostics to an outcome for presentation after the
handler returns. `MessageDiagnostic` currently writes the supplied message with
a newline; severity does not add a label automatically.

CommandKit coordinates final diagnostic delivery and retains delivery failures
separately from the operation cause. Custom handlers can also write directly to
the supplied streams and retain responsibility for their output. The existing
outcome and channel ownership decisions are recorded in
[ADR 0005](../adr/0005-invocation-outcomes.md).

Integration adapters already translate upstream diagnostic records into
CommandKit diagnostics while preserving native rendering and source context.
Some adapters wrap multiple records in one diagnostic writer and classify the
batch by its highest severity. This matters when designing selective visibility.

There is currently no shared verbosity policy or invocation reporter API.

## Selected output policy

Distinguish substantive application output from diagnostic message categories:

- **Substantive output (Normal):** The requested report or command result.
- **Error:** Terminal presentation of an unrecoverable invocation failure,
  resulting in a non-zero status code.
- **Warning:** Something went wrong or deserves attention, but the invocation
  can still succeed. The application chooses whether to report such an issue.
- **Info:** Informational context and useful explanation, such as which profile
  was selected or a final operation summary.
- **Progress:** Streaming updates about work underway, such as items processed.
- **Debug:** Troubleshooting detail, including intermediate decisions and
  execution-path information.

Debug mode adds troubleshooting detail to verbose output:

| Output category | `--quiet` | No flag | `--verbose` | `--debug` |
| --- | --- | --- | --- | --- |
| Substantive application output | Suppressed | Shown | Shown | Shown |
| Error | Shown | Shown | Shown | Shown |
| Warning | Suppressed | Suppressed | Shown | Shown |
| Info | Suppressed | Suppressed | Shown | Shown |
| Debug | Suppressed | Suppressed | Suppressed | Shown |
| Progress, without an explicit setting | Suppressed | Suppressed | Suppressed | Shown |

Info belongs in verbose mode; debug includes verbose messages. Normal output
remains a substantive result category, separate from diagnostic severity.
The category of a message determines its visibility; stream destination alone
cannot distinguish a warning from an error.

Progress is distinct from Info and is controlled separately as specified below.
Applications and integrations are not required to generate messages in any
category. Enabling a category adds no output when there are no messages in it;
other output remains subject to its own visibility rules.

Error presentation follows the terminal failure model below. An application
may report a recovered problem as a Warning or continue without a message.

Apply the same policy to help and version. There are no command-specific
exceptions for successful output:

```text
app help --quiet       # succeeds silently
app version --quiet    # succeeds silently
```

Usage and operational failures still report their errors in quiet mode.
The feature must distinguish successful output suppression from failure
diagnostic delivery.

### Message prefixes and channels

Use `WARN:`, `DEBUG:`, and `ERROR:` prefixes for simple warning, debug, and error
messages respectively. Substantive output and Info messages have no prefix.
For multiline diagnostics, prefix the opening line and preserve the structured
detail. Integration adapters coordinate native diagnostic rendering to avoid
duplicate severity labels or loss of upstream formatting.

Initially, Progress messages are unprefixed streamed lines. They use the
diagnostic channel so they do not corrupt substantive output such as JSON or
YAML. Terminal redraw and progress-bar behavior are outside this implementation.

### Terminal failure model

During execution, applications can produce substantive output, Progress, Info,
Debug, and Warnings. An unrecoverable failure ends the handler by returning a
failed outcome. There is no standalone `report.Error(...)` method. Detailed
error explanations accompany the returned failure and are presented as part of
that terminal step.

CommandKit finalizes reporting, presents the terminal failure explanation, and
returns the non-zero outcome to the application entry point. It accepts no
further normal output for that invocation after terminal error presentation.
The same boundary applies to core routing and parsing failures that prevent
handler execution. The application entry point retains ownership of process
exit; CommandKit ends the invocation and returns its outcome.

For example, if an export operation fails and the application decides it cannot
continue, the handler returns a non-zero outcome with an explicit explanation
such as "Unable to export the requested report." The application can choose to
include detail from a Go error in that explanation. CommandKit presents the
supplied explanation without automatically deriving text from an application
error. Exact API names and signatures remain to be finalized.

Output streamed before the failure remains visible according to its selected
mode; terminal failure does not withdraw earlier bytes. Deferred messages,
cleanup writes, and delivery failures must respect the end of normal reporting.
Their finalization and late-write behavior follow OD-4 below. A terminal
failure may have multiple explanatory records, all belonging to that failure.

### Failure explanations and presentation ownership (OD-3)

Make a non-zero outcome with an explicit message the convenient failure path.
Support structured explanations, including multiple records, through the same
presentation facility for richer application and integration reports. Neither
form requires an underlying Go error.

The application or integration explicitly identifies the context belonging to
a terminal explanation. Present that context with the failure in every mode,
including quiet. Independent warnings and other messages retain their own
visibility rules. For mixed upstream batches, adapters preserve useful native
rendering while distinguishing terminal explanatory context from independent
messages. Core does not infer those relationships from error chains or text.

Assign each submitted message one presentation path. Returning a terminal
explanation schedules its presentation once; core does not append another
message derived from an application error cause. Native renderers can supply
already-labelled content without receiving duplicate prefixes. Avoid text-based
deduplication: separate events may legitimately produce identical messages.

### Retention boundary (OD-3)

CommandKit provides invocation-scoped CLI presentation. Logging and message
history collection remain application concerns. Keep terminal explanations with
the returned outcome, and keep the temporary state needed for deferred delivery,
parser events awaiting mode selection, and delivery bookkeeping. The reporting
facility does not collect a transcript or retain other emitted or suppressed
messages for later inspection. No optional message-history facility is planned
for this implementation.

## Selected flag grammar and scope

### Mode selection and resets

Model quiet, ordinary, verbose, and debug as mutually exclusive presentation
modes. Each invocation starts in ordinary mode. Process flag occurrences in
invocation order; the last enabled mode wins. Repetition does not increment
verbosity, and debug is selected explicitly.

Accept presence and the existing explicit boolean syntax, including
`--quiet=true` and `--quiet=false`. A false occurrence clears its mode only if
that mode is currently selected, returning to ordinary mode. It does not restore
an earlier selection or disable a different mode. Every occurrence is converted;
invalid boolean values remain usage errors even when a later flag would replace
the selection.

| Flag sequence | Final mode |
| --- | --- |
| `--quiet --verbose` | Verbose |
| `--debug --quiet` | Quiet |
| `--verbose --verbose` | Verbose |
| `--verbose --verbose=false` | Ordinary |
| `--quiet --verbose=false` | Quiet |
| `--verbose --quiet --quiet=false` | Ordinary |
| `--quiet=false --debug` | Debug |

Keep verbosity a presentation choice: a quiet invocation still performs its
requested operation and returns the ordinary outcome. Apply the policy through
CommandKit-managed reporting and streams. Output written directly to process
streams or external destinations remains application-owned.

### Progress selection

`--progress` enables Progress without enabling Info, Warning, or Debug messages.
`--progress=false` explicitly disables it. Accept the existing boolean syntax;
repeated canonical or aliased progress settings use the last value. Retain
whether a setting was explicitly supplied, separately from its boolean value.

Determine effective Progress visibility from the final presentation mode and
explicit progress setting:

1. Quiet mode always suppresses Progress.
2. Otherwise, an explicit progress setting controls visibility.
3. Without an explicit setting, debug enables Progress; ordinary and verbose
   modes leave it disabled.

An explicit progress setting overrides the mode's default regardless of its
position relative to mode flags. It does not select or reset the presentation
mode. Debug includes verbose detail and defaults Progress on; verbose alone
does not enable Progress.

| Flags | Result |
| --- | --- |
| `--verbose` | Warning and Info, without Progress. |
| `--verbose --progress` | Warning, Info, and Progress. |
| `--debug` | Warning, Info, Debug, and Progress. |
| `--debug --progress=false` or `--progress=false --debug` | Debug detail without Progress. |
| `--quiet --progress` or `--progress --quiet` | Only failure diagnostics. |
| `--debug --verbose` | Verbose detail without Progress, absent an explicit progress setting. |

Passing `--progress` to an application that emits no Progress messages adds no
visible output. It does not suppress that application's substantive results or
require a progress implementation.

### Placement, availability, and aliases

Core supplies `--quiet`, `--verbose`, `--debug`, and `--progress` for application
commands, integrations, the root, canonical groups, and alias-only discovery help.
Consumers do not register them individually. Preserve the existing placement
grammar: flags follow the complete command path and may be interspersed with
operands after that path. For example:

```text
app config show --quiet
app export destination --verbose
app help config show --quiet
app version --quiet
```

Options before or between command-path components are outside this grammar.
Tokens consumed as values and tokens following `--` retain their ordinary
meaning.

Reserve the four long spellings. Applications may add `-q`, `-v`, or `-d`
through the alias API; no built-in short spellings are selected. An application
flag declaration that defines a reserved long name fails installation with a
collision error. Aliases may target these flags but may not overload canonical
spellings. Apply the existing collision and atomic installation rules, including
later registration.

Aliased and canonical occurrences participate in the same ordering. With
`-q → --quiet`, `app export --verbose -q` selects quiet mode. Document the newly
reserved canonical names as a compatibility change.

### Routing and parsing failures

Determine the mode through the normal parser before execution or successful
help/report presentation. Do not add a separate verbosity scanner. This keeps
token interpretation consistent and avoids reinterpreting values or operands.

Routing and parsing failures report their errors regardless of verbosity. For
example, `app export --quiet --unknown` still reports the unknown flag. Error
visibility follows the selected output policy independently of how the mode is
determined; avoiding a separate scanner is an implementation simplification.

## CommandKit's own diagnostics

Use the shared reporting system for CommandKit's invocation diagnostics so
parser, integration, and application messages follow the same visibility and
delivery policy. Installation errors occur before an invocation exists and
continue to be returned to the application without automatic printing.

Supported repetition and last-occurrence overrides are routine decisions.
Repeated singleton flags, including `--verbose --verbose`, may produce debug
detail; repetition alone does not warrant a warning. Repeatable flags retain
their intended accumulation behavior. Add warnings for concrete concerns an
operator can act on, such as a deprecated spelling if deprecation is introduced.
Unknown flags, invalid values, and invalid command paths remain invocation
failures with error diagnostics.

Start debug instrumentation with routing, alias resolution, verbosity selection,
and invocation completion. Other useful candidates can be added where they
explain a consequential decision:

| Event | Useful detail |
| --- | --- |
| Command-alias resolution | Entered path, intermediate redirects, and final canonical path. |
| Flag-alias resolution | Entered spelling and canonical flag name. |
| Scoped alias selection | A command-scoped mapping took precedence over an application-wide mapping. |
| Verbosity selection | A later occurrence selected a different mode; for `--verbose --debug`, debug wins because it appears last. |
| Help-route selection | A canonical command or alias-only discovery context was selected. |
| Repeated singleton flag | A later occurrence replaced an earlier occurrence, without printing their values. |
| Input preparation | Canonical input validation completed and the handler is starting. |
| Invocation completion | Canonical command, outcome classification, and status. |

Use the final selected mode to present earlier parser decisions. Collect the
relevant events during parsing and apply visibility after mode selection:
`--verbose --debug` shows the selection explanation, while `--debug --quiet`
suppresses it. This collection of parser events does not require whole-run
output buffering.

Debug messages should explain operator-relevant decisions independently of
incidental implementation steps. Use command paths, flag names, and decision
descriptions rather than raw argument or configuration-value dumps. Upstream
libraries retain ownership of their diagnostic detail and disclosure rules.

## Selected delivery lifecycle and quiet execution (OD-4)

### Ordering and finalization

Use synchronous delivery. Present immediate messages and substantive writes in
submission order. After the handler returns:

1. Deliver eligible deferred messages in queue order, without sorting by severity.
2. Present any terminal failure explanation.
3. Close the invocation's reporting facility and return the final outcome.

Deferred messages remain eligible when the operation fails, subject to verbosity.
Deferral controls timing; applications queue success summaries only after the
relevant operation succeeds. Ordering governs CommandKit's writes; separate
stdout and stderr destinations may be buffered or displayed differently by the
caller.

CommandKit finalizes automatically. Consumers do not need a separate flush or
finish call. Handler cleanup, including Go `defer` functions, runs before the
handler returns and participates in normal reporting. Applications complete any
reporting goroutines before returning. After finalization, writes and message
submissions return a closed-reporter error and emit nothing.

Finalize any buffers owned by CommandKit. Caller-supplied writers remain
caller-owned; CommandKit does not automatically close them or invoke arbitrary
flushing interfaces.

Visibility applies when output is emitted. A verbose or debug run may show
informational detail before subsequently failing. Progress and substantive
output may likewise be partially delivered before a failure. Previously written
bytes cannot be withdrawn. Whole-run buffering solely to make failed runs
contain only error output is outside this implementation.

Keep errors understandable in every mode, including source locations and
related explanatory context. Unrelated warnings and informational messages
remain subject to the selected mode. Apply the selected retention boundary
without accumulating a message history.

### Write failures and execution flow

Failure to deliver requested output makes an otherwise successful invocation
unsuccessful. Apply this rule to substantive output, Warning, Info, Progress,
Debug, and terminal explanations. Intentional suppression requests no delivery
and is not a delivery failure.

Standard writer operations return write errors normally. Immediate reporting
methods also expose delivery errors. CommandKit records bounded delivery-failure
information so an ignored write error cannot yield a successful invocation.
Deferred delivery failures appear in the final outcome.

If the application returned success, a delivery failure produces an operational
failure. If the application already returned failure, preserve its classification,
status, and explanation, and include the delivery failure separately. Reporting
failures belong to CommandKit's own error-handling responsibility.

Stop writing to a destination after it fails. A substantive-output failure can
receive a terminal explanation through the diagnostic destination if it still
works. If diagnostic delivery fails, return that information to the caller.
Do not redirect diagnostics into substantive output or recursively attempt to
report the reporting failure.

A failed write does not stop application code. The handler can inspect the error
and return immediately; otherwise, CommandKit records the failure and finalizes
when the handler returns. CommandKit does not use process exit or panic to
interrupt execution. Terminal failure presentation ends normal output.

### Quiet-mode rendering and suppression

Initially, run substantive report generation normally with its managed writer
discarding output in quiet mode. This preserves report generation, validation,
and serialization failures while changing presentation. For example, a report
encoder failure still fails the invocation under `--quiet`; the handler remains
responsible for handling errors returned by an encoder it calls.

Simple suppressed messages need no formatting or destination writes. For richer
report renderers, initially render into a discard destination. Rendering may be
skipped later where doing so preserves the command's operation and failure
semantics.

Treat suppression as intentional suppression, never as failed or incomplete
delivery. Do not retain suppressed content. Delivery needs only the deferred
queue, reporting lifecycle state, and bounded delivery-failure information,
alongside the temporary parser state and returned explanations described above.

## Selected consumer reporting API direction

Provide one reporting facility per invocation for substantive output, immediate
messages, and end-of-run diagnostics. The consumer API is chosen for its intended
long-term design; with no current consumers, preserving existing API shapes
carries no weight in this decision.

The facility exposes interfaces appropriate to each need:

| Need | Selected direction | Illustrative shape |
| --- | --- | --- |
| Substantive results | An ordinary `io.Writer`, usable by encoders and native report writers. | `inv.Reporter().Output()` |
| Immediate messages | Categorized methods applying visibility and presentation automatically. | `Info`, `Warn`, `Debug`, `Progress` |
| End-of-run diagnostics | An explicit deferred view sharing the invocation's reporting state. | `report.AtEnd().Warn(message)` |
| Rich upstream diagnostics | Typed submission preserving severity, context, and native rendering. | Exact API remains open. |

These method names illustrate the selected direction; exact names and signatures
remain to be finalized. Progress is streaming output, distinct from deferred
informational summaries. Applications need not implement categories they do not
use or check verbosity before reporting.

The reporter owns presentation: output visibility, progress settings, prefixes,
ordering, and delivery bookkeeping. Its views share invocation state, allowing
already-emitted messages, deferred messages, suppression, and write failures to
be coordinated in one place. Helpers may receive the reporter or a narrow
application-defined interface. Domain libraries can continue returning
diagnostic data for the handler to submit, independently of CommandKit.

The returned outcome owns the operation result: success, failure, or an
application domain classification. CommandKit presents a failed outcome through
the same reporting facility as the terminal step. Error explanations belong to
that returned failure, including rich upstream explanations. Typed diagnostic
submission must preserve this relationship instead of permitting independently
emitted Error messages during normal execution. Application failures carry
explicit status and presentation content; they do not require a Go error cause.

Use the reporter as the common diagnostic submission path, replacing the earlier
proposal to split submission between a reporter and `Outcome.Diagnostics`.
The returned outcome carries terminal explanations, with other retention limited
to the presentation needs described above. Caller-selected destinations belong
at the execution boundary; the handler-facing API routes output through the
reporter instead of requiring a parallel raw-stream API.

Immediate methods expose delivery errors and the reporter retains delivery
failure state for automatic finalization, as selected in OD-4. Standard writer
operations remain compatible with `io.Writer`. Exact API names, signatures, and
outcome representations will express these behaviors through the selected
consumer-facing structure.

## Upstream diagnostic ownership

ConfigKit and PromptKit remain authoritative for their diagnostic semantics,
severity, source context, and disclosure rules. Integration adapters expose
those capabilities through the shared CommandKit presentation policy.

Support consistent reporting of ConfigKit errors and warnings as its adopted
public API evolves. This roadmap does not assume that a future warning API
already exists or prescribe its upstream shape. Integration-owned commands
select the outcome and submit useful native diagnostic detail for presentation.
Core presents those records under the shared policy; it does not infer an
application's recovery decisions or classify arbitrary Go errors. Applications
using upstream libraries directly retain the same choice over outcomes and
messages as for their other operations.

Debug output follows the same upstream disclosure and safe-rendering contracts.
Enabling debug does not authorize secret disclosure or bypass owning-library
rules.

## Work needed

1. Specify the selected behavior in examples covering ordinary application output,
   immediate and deferred diagnostics, failure after partial output, help and
   version, and integration diagnostics in every mode. Include applications that
   emit no Progress messages and the distinction between Info and Progress.
   Cover silent recovery, an explicit Warning with or without a Go error, and a
   terminal failure with an explicit explanation and no Go error cause.
2. Review current diagnostic batching, parsing, help, outcome presentation, and
   writer ownership to identify the smallest shared changes. Include invocation
   failures that occur before a handler receives its `Invocation`.
3. Finalize the unified reporting API and flag contracts, including interactions
   with command and flag aliases, following OD-1 through OD-4 above. Replace split
   handler submission paths and adapt core and integration callers. Account for
   flag collisions and document changed public contracts.
4. Implement the selected behavior across core and the integrations. Preserve
   upstream rendering and semantics while applying the chosen visibility policy.
5. Verify classification, flag selection, suppression, terminal explanations,
   ordering, duplicate prevention, terminal Error/non-zero-outcome consistency, debug
   visibility, and write-failure handling with focused behavioral tests and
   maintained consumer examples. Cover mode resets, mixed canonical/alias
   ordering, help and discovery contexts, and core parser events presented
   according to the final mode. Cover explicit progress overrides, debug's
   progress default, quiet suppression, progress aliases, message prefixes,
   and preservation of native multiline diagnostics. Cover explicit explanatory
   context in quiet mode, independent warnings in mixed batches, already-labelled
   native content, and distinct submissions with identical text. Verify the end
   of normal reporting after terminal error presentation, including deferred
   and late writes under the selected OD-4 behavior. Cover immediate and deferred
   ordering on success and failure, handler cleanup, automatic finalization,
   and ownership of caller-supplied writers. Verify ignored immediate write
   errors and deferred delivery failures affect the final outcome, existing
   failures retain their classification and status, and failed destinations
   receive no further writes. Cover diagnostic-destination failure without
   fallback into substantive output, quiet-mode rendering failures, and
   suppression without destination writes or incomplete-delivery status.
6. Update GoDoc, the CLI reference, consumer guidance, and release documentation
   when implemented. Reconcile the selected error-handling boundary with
   ADR 0005's existing outcome and error-identity commitments before changing
   those contracts. Record any resulting architectural change in a new ADR;
   exact reporting APIs and flag syntax remain API/CLI contracts.

## Remaining design work

OD-1 through OD-4 are resolved. Before implementation, finalize the concrete
reporter and outcome API, including simple and structured terminal explanations,
deferred submission, and delivery-failure observations. Review complete examples
against the selected behavior and reconcile the architectural contracts noted
above.