Record native reporting machine output and outcomes

This commit is contained in:
2026-11-19 18:13:08 +00:00
parent 6fa520f8f3
commit 27081bc0af

View File

@@ -22,8 +22,9 @@ this discussion.
Item 1 selects the dependency versions and standard capabilities, including
PromptKit v0.23.3 in place of the master roadmap's initial v0.23.2 target.
Item 2 selects the application command structure and input conventions. Item 3
selects native composition, shared loaders, and application preflight. Detailed
reporting, debug capture, and verification choices remain open below.
selects native composition, shared loaders, and application preflight. Item 4
selects native reporting, machine output, and outcomes. Exact run-report fields,
debug capture, and verification choices remain open below.
The PromptKit architecture and checkpoint decisions remain in force. This
discussion develops the CommandKit portion of that migration and its interactions
@@ -81,9 +82,9 @@ spending migration effort recreating existing CLI conventions.
| Receipt content, debug artifact lifecycle, and application outcome meaning | Notarius, expressed through CommandKit's reporting and outcome conventions. |
Adopt the standard capability suite selected in item 1, including upstream
commands and recommendations beyond the existing Notarius surface. Items 2–3
select application inputs and composition; the remaining discussion determines
result presentation, debug capture, and verification.
commands and recommendations beyond the existing Notarius surface. Items 2–4
select application inputs, composition, and reporting; the remaining discussion
determines debug capture, verification, and concrete record details.
Pipeline and domain code should remain independent of CommandKit. Native
PromptKit observations supply execution facts; Notarius attaches application
@@ -91,10 +92,10 @@ meaning and selects report content. CommandKit supplies presentation mechanics.
Two application needs already identified for deliberate treatment are:
- The current `run --json` publishes a receipt after output publication. Determine
the needed result information and express it through an appropriate native
output facility. The old flag, envelope, formatting, and stream conventions are
not compatibility requirements; reports must still describe the actual outcome.
- The current `run --json` publishes a receipt after output publication. Item 4
replaces its presentation with one compact application report rendered natively
for humans and machines. Exact fields/versioning remain with U9; reports must
describe the actual publication and domain outcome.
- The current `--debug` requests durable artifacts. Retain the distinction between
artifact capture and reporting verbosity, while choosing native controls and
meanings rather than requiring that flag to retain its old behavior.
@@ -499,24 +500,144 @@ Evidence: [native suite and loader ownership](../../../commandkit/docs/consumers
## 4. Reports, Machine Output, And Outcomes
**Discussion / decision:** Pending.
**Discussion / decision:** Agreed. Use native CommandKit reports for command
results, native outcomes for command completion, and a compact application-owned
run report for artifact discovery and domain status. Distinguish pipeline results,
command completion, and presentation delivery. Implementation is pending.
- How should native reports and output modes express application results? Where
does an application data requirement justify raw output rather than a standard
report, independently of the old CLI's output conventions?
- How should syntax, configuration, pipeline, and publication outcomes map to
native outcome/status conventions and user-facing explanations? How should
presentation failures remain distinguishable from the operation outcome?
- Which application scope and minimal projections connect native observations
to progress, terminal summaries, manifests, and receipts without duplicate
execution accounting?
### One Report Value For Human And Machine Output
Review the current [run-result contract](../integrations/run-result.md) and
Application commands submit ordinary values through `Reporter.Report` and use
native `--format text|yaml|json` rendering. Execution supplies one run-report
value with a concise text template for default human output; JSON and YAML render
the same structured value. Do not maintain separate human-summary and JSON-receipt
construction paths.
```sh
notarius run dnd-session --input-file transcript.json
notarius run dnd-session --input-file transcript.json --format json
```
Standard configuration and PromptKit commands retain their upstream report shapes
and behavior. Do not wrap them in a universal Notarius result envelope. No current
run-report need requires raw output; reserve native `Output`/`RenderOutput` for an
operation whose actual product needs a byte stream.
### Compact Application Run Report
CommandKit owns presentation; Notarius owns the meaning and deliberate contract
of the application run report. Retain these content categories:
| Content | Purpose |
| --- | --- |
| Application report version | Identify the report contract for consumers. |
| Run and pipeline identity | Identify completed work. |
| Published bundle directory and applicable index path | Locate artifacts without guessing filenames. |
| Accepted/normalized and rejected output counts | Summarize production results. |
| Validation completeness/status | Distinguish fully validated results from permitted incomplete validation. |
| Process-warning counts | Expose degradation independently of reporting verbosity. |
| Debug location, when applicable | Locate requested diagnostic artifacts. |
Detailed per-producer validation records, rejection reasons, and execution
provenance belong in the published bundle rather than duplicated collections in
the command report. Do not serialize the entire internal pipeline result or
complete upstream runtime types as a durable contract. U9 will select exact
fields, versioning, and durable record formats. The old CLI receipt's shape is
migration evidence, not a requirement to reproduce it.
### Publication, Streams, And Machine Consumption
The run command executes the pipeline, publishes the output bundle, completes
required run finalization, submits its successful report, and returns a native
outcome. The report describes an actual published result. Item 5 will settle the
treatment of requested debug-finalization failures.
Use one primary run-result report, with progress and diagnostics separate. Wire
CommandKit's result destination to stdout and its diagnostic destination to
stderr. Subprocess consumers request `--format json`, capture streams separately,
check the exit status, and decode the successful run report. A report-write
failure may leave partial stdout even though publication already succeeded.
These run-consumption rules do not replace the native report behavior of other
commands, including configuration commands that can report failed validation.
### Native Visibility And Message Categories
| Native facility | Application use |
| --- | --- |
| `Report` | Requested command result. |
| `Progress` | Pipeline progress. |
| `Warn` | Actionable process degradation. |
| `Info` | Supplementary operational information. |
| `Debug` | Diagnostic detail, distinct from durable artifact capture. |
| Outcome explanation | Terminal failure or usage guidance. |
Adopt native visibility rules directly: ordinary mode suppresses warning messages,
and quiet mode suppresses command results. Include validation status and
process-warning counts in the primary run report and its human summary so those
facts do not depend on warning-message visibility. Operators request `--verbose`
for warning details; machine callers expecting a report do not request `--quiet`.
Use native progress selection and failure visibility without custom bypasses.
Quiet suppression is not a delivery failure; native rendering errors can still
fail a quiet invocation.
Artifact-quality advisories retain their application classification. Do not
reclassify them as process warnings merely to make them visible.
### Native Outcomes And Domain Acceptance
Adopt the standard process-status conventions:
| Status | Meaning |
| --- | --- |
| `0` | The command completed successfully. |
| `1` | Operational failure, cancellation, publication failure, or presentation failure. |
| `2` | Invalid invocation or identifiable invalid configuration input. |
| `3` | Native `config diff --check` found differences. |
Use ckconfig's structured configuration-error classification. Do not classify
every configuration failure as operational or every preflight failure as invalid
input. Invalid operator selections and missing bundled application resources can
have different causes; preserve that distinction and native error identity.
Command completion remains separate from domain acceptance. A successfully
published run can contain recorded rejections or validation-incomplete results
allowed by application policy; those facts appear in reports and artifacts and
do not automatically fail the command. A terminal pipeline failure does fail the
command. Native status zero also does not replace reading standard PromptKit
reports for partial discovery or unsupported smoke checks.
Application handlers supply useful native outcome explanations and retain the
underlying error through native cause handling. CommandKit presents the terminal
result. Remove duplicate terminal-error printing and custom outcome wrappers.
### Delivery Failures, Progress, And Accounting
Use CommandKit's native separation of operation causes from output and diagnostic
delivery failures. If publication succeeds but stdout fails, the final command
status reflects presentation failure while the published artifacts remain a
fact. Do not rewrite pipeline history as failed execution or automatically rerun
model work because report delivery failed.
Notarius owns stage/lane/chunk progress, domain acceptance, and checkpoint reuse.
PromptKit supplies native execution observations and `ExecutionGroup` totals;
CommandKit presents the selected facts. Explicitly enroll the relevant initial,
continuation, and validator executions without double counting. Checkpoint reuse
does not add historical token usage to current execution totals, and unavailable
usage remains distinguishable from zero.
Remove bespoke format selection, parallel receipt encoders, custom verbosity
handling, duplicate execution accounting, and superseded terminal-reporting
machinery. Retain only application report content, publication facts, domain
decisions, and small deliberate durable projections.
Evidence: [native reporting](../../../commandkit/docs/consumers/guide.md#invocation-reporting),
[visibility and delivery](../../../commandkit/docs/cli.md#reporting), and
[outcome classification](../../../commandkit/docs/cli.md#outcome-status).
The current [run-result contract](../integrations/run-result.md) and
[CLI failure mapping](../internal/cli.md#failure-mapping-and-terminal-reporting)
to understand application needs and caller migration impact. Their existing CLI
shapes and status mappings do not override the native-cutover direction.
Item 6 of the [PromptKit discussion record](promptkit-architecture-discussion.md#6-inspection-and-execution-bookkeeping)
provides the bookkeeping boundary; U9 still owns concrete durable record choices.
document migration impact. Item 6 of the
[PromptKit discussion record](promptkit-architecture-discussion.md#6-inspection-and-execution-bookkeeping)
retains the bookkeeping boundary; U9 still owns concrete durable record choices.
## 5. Debug Capture And Optional Diagnostics