commandkit/docs/roadmap/reporting.md

# Output policy and invocation reporting

Status: initial planning. Message categories, all four output modes, uniform
treatment of help and version, message prefixes, and OD-1's flag grammar and
scope (including independent Progress control) are selected.
Consumer API and delivery mechanics remain open. Implementation has not started.

## Purpose

Provide consistent handling of `--quiet`, `--verbose`, `--debug`, and
`--progress` across application commands and optional integrations. Reduce
repeated output classification, diagnostic presentation, and delivery bookkeeping
where CommandKit can provide a meaningful benefit.

Ordinary Go writers and formatting functions already make application output
straightforward. Design reporting conveniences around the shared output policy
and invocation lifecycle. Applications continue to own their report content,
domain behavior, and operation outcomes.

## Current baseline

[`Streams` and `Invocation`](../../identity.go) expose caller-selected output
and diagnostic writers. [`Outcome` and `Diagnostic`](../../outcome.go) carry
operation causes, diagnostic writers, and delivery state. Handlers can return
`Failure(err)` or attach diagnostics to an outcome for presentation after the
handler returns. `MessageDiagnostic` currently writes the supplied message with
a newline; severity does not add a label automatically.

CommandKit coordinates final diagnostic delivery and retains delivery failures
separately from the operation cause. Custom handlers can also write directly to
the supplied streams and retain responsibility for their output. The existing
outcome and channel ownership decisions are recorded in
[ADR 0005](../adr/0005-invocation-outcomes.md).

Integration adapters already translate upstream diagnostic records into
CommandKit diagnostics while preserving native rendering and source context.
Some adapters wrap multiple records in one diagnostic writer and classify the
batch by its highest severity. This matters when designing selective visibility.

There is currently no shared verbosity policy or invocation reporter API.

## Selected output policy

Distinguish substantive application output from diagnostic message categories:

- **Substantive output (Normal):** The requested report or command result.
- **Error:** The invocation failed and will return a non-zero status code.
- **Warning:** Something went wrong or deserves attention, but the invocation
  can still succeed. A recovered problem is a warning.
- **Info:** Informational context and useful explanation, such as which profile
  was selected or a final operation summary.
- **Progress:** Streaming updates about work underway, such as items processed.
- **Debug:** Troubleshooting detail, including intermediate decisions and
  execution-path information.

Debug mode adds troubleshooting detail to verbose output:

| Output category | `--quiet` | No flag | `--verbose` | `--debug` |
| --- | --- | --- | --- | --- |
| Substantive application output | Suppressed | Shown | Shown | Shown |
| Error | Shown | Shown | Shown | Shown |
| Warning | Suppressed | Suppressed | Shown | Shown |
| Info | Suppressed | Suppressed | Shown | Shown |
| Debug | Suppressed | Suppressed | Suppressed | Shown |
| Progress, without an explicit setting | Suppressed | Suppressed | Suppressed | Shown |

Info belongs in verbose mode; debug includes verbose messages. Normal output
remains a substantive result category, separate from diagnostic severity.
The category of a message determines its visibility; stream destination alone
cannot distinguish a warning from an error.

Progress is distinct from Info and is controlled separately as specified below.
Applications and integrations are not required to generate messages in any
category. Enabling a category adds no output when there are no messages in it;
other output remains subject to its own visibility rules.

Error presentation is tied to invocation failure. Prefer deriving error
diagnostics from a failed outcome, which may carry detailed explanations. Avoid
an unrestricted `Error(...)` reporting method that permits an error message
followed by a successful outcome. The application still controls process exit
under the existing outcome contract; the returned outcome must consistently
represent the failure and its non-zero status.

Apply the same policy to help and version. There are no command-specific
exceptions for successful output:

```text
app help --quiet       # succeeds silently
app version --quiet    # succeeds silently
```

Usage and operational failures still report their errors in quiet mode.
The feature must distinguish successful output suppression from failure
diagnostic delivery.

### Message prefixes and channels

Use `WARN:`, `DEBUG:`, and `ERROR:` prefixes for simple warning, debug, and error
messages respectively. Substantive output and Info messages have no prefix.
For multiline diagnostics, prefix the opening line and preserve the structured
detail. Integration adapters coordinate native diagnostic rendering to avoid
duplicate severity labels or loss of upstream formatting.

Streaming Progress messages use the diagnostic channel so they do not corrupt
substantive output such as JSON or YAML. Progress presentation details remain
part of the consumer API and delivery design below.

## Selected flag grammar and scope

### Mode selection and resets

Model quiet, ordinary, verbose, and debug as mutually exclusive presentation
modes. Each invocation starts in ordinary mode. Process flag occurrences in
invocation order; the last enabled mode wins. Repetition does not increment
verbosity, and debug is selected explicitly.

Accept presence and the existing explicit boolean syntax, including
`--quiet=true` and `--quiet=false`. A false occurrence clears its mode only if
that mode is currently selected, returning to ordinary mode. It does not restore
an earlier selection or disable a different mode. Every occurrence is converted;
invalid boolean values remain usage errors even when a later flag would replace
the selection.

| Flag sequence | Final mode |
| --- | --- |
| `--quiet --verbose` | Verbose |
| `--debug --quiet` | Quiet |
| `--verbose --verbose` | Verbose |
| `--verbose --verbose=false` | Ordinary |
| `--quiet --verbose=false` | Quiet |
| `--verbose --quiet --quiet=false` | Ordinary |
| `--quiet=false --debug` | Debug |

Keep verbosity a presentation choice: a quiet invocation still performs its
requested operation and returns the ordinary outcome. Apply the policy through
CommandKit-managed reporting and streams. Output written directly to process
streams or external destinations remains application-owned.

### Progress selection

`--progress` enables Progress without enabling Info, Warning, or Debug messages.
`--progress=false` explicitly disables it. Accept the existing boolean syntax;
repeated canonical or aliased progress settings use the last value. Retain
whether a setting was explicitly supplied, separately from its boolean value.

Determine effective Progress visibility from the final presentation mode and
explicit progress setting:

1. Quiet mode always suppresses Progress.
2. Otherwise, an explicit progress setting controls visibility.
3. Without an explicit setting, debug enables Progress; ordinary and verbose
   modes leave it disabled.

An explicit progress setting overrides the mode's default regardless of its
position relative to mode flags. It does not select or reset the presentation
mode. Debug includes verbose detail and defaults Progress on; verbose alone
does not enable Progress.

| Flags | Result |
| --- | --- |
| `--verbose` | Warning and Info, without Progress. |
| `--verbose --progress` | Warning, Info, and Progress. |
| `--debug` | Warning, Info, Debug, and Progress. |
| `--debug --progress=false` or `--progress=false --debug` | Debug detail without Progress. |
| `--quiet --progress` or `--progress --quiet` | Only failure diagnostics. |
| `--debug --verbose` | Verbose detail without Progress, absent an explicit progress setting. |

Passing `--progress` to an application that emits no Progress messages adds no
visible output. It does not suppress that application's substantive results or
require a progress implementation.

### Placement, availability, and aliases

Core supplies `--quiet`, `--verbose`, `--debug`, and `--progress` for application
commands, integrations, the root, canonical groups, and alias-only discovery help.
Consumers do not register them individually. Preserve the existing placement
grammar: flags follow the complete command path and may be interspersed with
operands after that path. For example:

```text
app config show --quiet
app export destination --verbose
app help config show --quiet
app version --quiet
```

Options before or between command-path components are outside this grammar.
Tokens consumed as values and tokens following `--` retain their ordinary
meaning.

Reserve the four long spellings. Applications may add `-q`, `-v`, or `-d`
through the alias API; no built-in short spellings are selected. An application
flag declaration that defines a reserved long name fails installation with a
collision error. Aliases may target these flags but may not overload canonical
spellings. Apply the existing collision and atomic installation rules, including
later registration.

Aliased and canonical occurrences participate in the same ordering. With
`-q → --quiet`, `app export --verbose -q` selects quiet mode. Document the newly
reserved canonical names as a compatibility change.

### Routing and parsing failures

Determine the mode through the normal parser before execution or successful
help/report presentation. Do not add a separate verbosity scanner. This keeps
token interpretation consistent and avoids reinterpreting values or operands.

Routing and parsing failures report their errors regardless of verbosity. For
example, `app export --quiet --unknown` still reports the unknown flag. Error
visibility follows the selected output policy independently of how the mode is
determined; avoiding a separate scanner is an implementation simplification.

## CommandKit's own diagnostics

Use the shared reporting system for CommandKit's invocation diagnostics so
parser, integration, and application messages follow the same visibility and
delivery policy. Installation errors occur before an invocation exists and
continue to be returned to the application without automatic printing.

Supported repetition and last-occurrence overrides are routine decisions.
Repeated singleton flags, including `--verbose --verbose`, may produce debug
detail; repetition alone does not warrant a warning. Repeatable flags retain
their intended accumulation behavior. Add warnings for concrete concerns an
operator can act on, such as a deprecated spelling if deprecation is introduced.
Unknown flags, invalid values, and invalid command paths remain invocation
failures with error diagnostics.

Start debug instrumentation with routing, alias resolution, verbosity selection,
and invocation completion. Other useful candidates can be added where they
explain a consequential decision:

| Event | Useful detail |
| --- | --- |
| Command-alias resolution | Entered path, intermediate redirects, and final canonical path. |
| Flag-alias resolution | Entered spelling and canonical flag name. |
| Scoped alias selection | A command-scoped mapping took precedence over an application-wide mapping. |
| Verbosity selection | A later occurrence selected a different mode; for `--verbose --debug`, debug wins because it appears last. |
| Help-route selection | A canonical command or alias-only discovery context was selected. |
| Repeated singleton flag | A later occurrence replaced an earlier occurrence, without printing their values. |
| Input preparation | Canonical input validation completed and the handler is starting. |
| Invocation completion | Canonical command, outcome classification, and status. |

Use the final selected mode to present earlier parser decisions. Collect the
relevant events during parsing and apply visibility after mode selection:
`--verbose --debug` shows the selection explanation, while `--debug --quiet`
suppresses it. This collection of parser events does not require whole-run
output buffering.

Debug messages should explain operator-relevant decisions independently of
incidental implementation steps. Use command paths, flag names, and decision
descriptions rather than raw argument or configuration-value dumps. Upstream
libraries retain ownership of their diagnostic detail and disclosure rules.

## Tentative direction

### Streaming and failure diagnostics

The proposed approach applies visibility when output is emitted. A verbose or
debug run may show informational detail before subsequently failing. A run with
Progress enabled may stream updates before failing, and an ordinary run may
write part of its substantive output before failing. Previously written bytes cannot
be withdrawn. Whole-run buffering solely to make failed runs contain only error
output is not part of the proposed direction.

Keep errors understandable in every mode, including source locations and
related explanatory context. Unrelated warnings and informational messages
remain subject to the selected mode. Retain suppressed diagnostics for
programmatic inspection rather than equating suppression with loss of evidence.
The exact representation of retained and delivered diagnostics remains open.

### Consumer reporting API

An invocation-scoped facility could classify messages, coordinate immediate and
end-of-handler presentation, and retain delivery failures. Earlier illustrative
API shapes included `inv.Reporter()`, `Info`, `Warn`, `Diagnostic`, and an
`AtEnd()` view. These names and shapes are exploratory, not selected contracts.
Reassess the necessary API around the output modes and expanded message
categories before adopting them. Debug and Progress messages need corresponding
classifications; their exact APIs remain open. Progress is a streaming category;
do not conflate it with deferred informational summaries.

Continue to support standard `io.Writer` usage for report bodies and native
report writers. Reuse existing diagnostic and outcome concepts where practical.
Any reporting facility should coordinate with returned `Outcome.Diagnostics`
and operation causes so already-presented messages are not repeated.

Whether reporting methods return write errors, retain them for finalization,
or expose an explicit delivery-error accessor remains open. The API must enforce
the selected relationship between Error diagnostics and failed outcomes,
including diagnostics supplied through generic or existing entry points.

### Upstream diagnostic ownership

ConfigKit and PromptKit remain authoritative for their diagnostic semantics,
severity, source context, and disclosure rules. Integration adapters expose
those capabilities through the shared CommandKit presentation policy.

Support consistent reporting of ConfigKit errors and warnings as its adopted
public API evolves. This roadmap does not assume that a future warning API
already exists or prescribe its upstream shape. Preserve useful native
diagnostic detail and supported error identity, and coordinate returned errors
with diagnostics that already explain them.

Debug output follows the same upstream disclosure and safe-rendering contracts.
Enabling debug does not authorize secret disclosure or bypass owning-library
rules.

## Work needed

1. Settle the open decisions with examples covering ordinary application output,
   immediate and deferred diagnostics, failure after partial output, help and
   version, and integration diagnostics in every mode. Include applications that
   emit no Progress messages and the distinction between Info and Progress.
2. Review current diagnostic batching, parsing, help, outcome presentation, and
   writer ownership to identify the smallest shared changes. Include invocation
   failures that occur before a handler receives its `Invocation`.
3. Specify the public API and flag contracts, including interactions with command
   and flag aliases, following the selected OD-1 rules above. Assess existing
   flag collisions and compatibility impact.
4. Implement the selected behavior across core and the integrations. Preserve
   upstream rendering and semantics while applying the chosen visibility policy.
5. Verify classification, flag selection, suppression, retained diagnostics,
   ordering, duplicate prevention, Error/non-zero-outcome consistency, debug
   visibility, and write-failure handling with focused behavioral tests and
   maintained consumer examples. Cover mode resets, mixed canonical/alias
   ordering, help and discovery contexts, and core parser events presented
   according to the final mode. Cover explicit progress overrides, debug's
   progress default, quiet suppression, progress aliases, message prefixes,
   and preservation of native multiline diagnostics.
6. Update GoDoc, the CLI reference, consumer guidance, and release documentation
   when implemented. Review whether any resulting architectural change needs an
   ADR; exact reporting APIs and flag syntax remain API/CLI contracts.

## Open Decisions

### OD-2: Consumer API and output classification

What is the smallest API for substantive output, streaming progress, debug and
informational messages, warnings, and failure diagnostics? How do direct writes through
`Invocation.Streams` interact with it? How can helpers report through the same
invocation? Is a dedicated immediate/deferred reporter needed, and how does it compose with existing
outcome diagnostics?
How do helpers emit Progress without requiring every application to implement
it or manage verbosity checks themselves?

### OD-3: Failure presentation and diagnostic retention

How does the API enforce that Error diagnostics accompany a failed invocation
with a non-zero status, including application domain outcomes? How should
inconsistent caller-supplied combinations be handled through existing diagnostic
and outcome APIs? How are recovered upstream problems represented as warnings
while preserving their native context?
Which contextual records belong to an error explanation? How are mixed upstream
diagnostic batches filtered while preserving useful native rendering? What is
retained when presentation is suppressed, and how is duplicate presentation
prevented without hiding unrelated failures?
How do adapters apply the selected prefixes to mixed or already-labelled native
diagnostics? What presentation should streaming Progress use?

### OD-4: Delivery lifecycle and quiet execution

How are immediate and deferred messages ordered and finalized, including when
the operation fails? How do callers observe and react to write failures? What
delivery state represents intentional suppression? Does quiet mode discard
rendered bytes or skip report rendering, and how does that choice preserve
operation semantics and failure detection? Which streaming behavior becomes
part of the public contract?
Are initial Progress messages ordinary streamed lines, or is terminal redraw
behavior needed? How do progress delivery failures and retention interact with
other messages? No progress-bar, redraw, or terminal-specific API is selected yet.