diff --git a/docs/roadmap/commandkit-migration-discussion.md b/docs/roadmap/commandkit-migration-discussion.md index e0a79ccb..aecd744e 100644 --- a/docs/roadmap/commandkit-migration-discussion.md +++ b/docs/roadmap/commandkit-migration-discussion.md @@ -22,8 +22,9 @@ this discussion. Item 1 selects the dependency versions and standard capabilities, including PromptKit v0.23.3 in place of the master roadmap's initial v0.23.2 target. Item 2 selects the application command structure and input conventions. Item 3 -selects native composition, shared loaders, and application preflight. Detailed -reporting, debug capture, and verification choices remain open below. +selects native composition, shared loaders, and application preflight. Item 4 +selects native reporting, machine output, and outcomes. Exact run-report fields, +debug capture, and verification choices remain open below. The PromptKit architecture and checkpoint decisions remain in force. This discussion develops the CommandKit portion of that migration and its interactions @@ -81,9 +82,9 @@ spending migration effort recreating existing CLI conventions. | Receipt content, debug artifact lifecycle, and application outcome meaning | Notarius, expressed through CommandKit's reporting and outcome conventions. | Adopt the standard capability suite selected in item 1, including upstream -commands and recommendations beyond the existing Notarius surface. Items 2–3 -select application inputs and composition; the remaining discussion determines -result presentation, debug capture, and verification. +commands and recommendations beyond the existing Notarius surface. Items 2–4 +select application inputs, composition, and reporting; the remaining discussion +determines debug capture, verification, and concrete record details. Pipeline and domain code should remain independent of CommandKit. Native PromptKit observations supply execution facts; Notarius attaches application @@ -91,10 +92,10 @@ meaning and selects report content. CommandKit supplies presentation mechanics. Two application needs already identified for deliberate treatment are: -- The current `run --json` publishes a receipt after output publication. Determine - the needed result information and express it through an appropriate native - output facility. The old flag, envelope, formatting, and stream conventions are - not compatibility requirements; reports must still describe the actual outcome. +- The current `run --json` publishes a receipt after output publication. Item 4 + replaces its presentation with one compact application report rendered natively + for humans and machines. Exact fields/versioning remain with U9; reports must + describe the actual publication and domain outcome. - The current `--debug` requests durable artifacts. Retain the distinction between artifact capture and reporting verbosity, while choosing native controls and meanings rather than requiring that flag to retain its old behavior. @@ -499,24 +500,144 @@ Evidence: [native suite and loader ownership](../../../commandkit/docs/consumers ## 4. Reports, Machine Output, And Outcomes -**Discussion / decision:** Pending. +**Discussion / decision:** Agreed. Use native CommandKit reports for command +results, native outcomes for command completion, and a compact application-owned +run report for artifact discovery and domain status. Distinguish pipeline results, +command completion, and presentation delivery. Implementation is pending. -- How should native reports and output modes express application results? Where - does an application data requirement justify raw output rather than a standard - report, independently of the old CLI's output conventions? -- How should syntax, configuration, pipeline, and publication outcomes map to - native outcome/status conventions and user-facing explanations? How should - presentation failures remain distinguishable from the operation outcome? -- Which application scope and minimal projections connect native observations - to progress, terminal summaries, manifests, and receipts without duplicate - execution accounting? +### One Report Value For Human And Machine Output -Review the current [run-result contract](../integrations/run-result.md) and +Application commands submit ordinary values through `Reporter.Report` and use +native `--format text|yaml|json` rendering. Execution supplies one run-report +value with a concise text template for default human output; JSON and YAML render +the same structured value. Do not maintain separate human-summary and JSON-receipt +construction paths. + +```sh +notarius run dnd-session --input-file transcript.json +notarius run dnd-session --input-file transcript.json --format json +``` + +Standard configuration and PromptKit commands retain their upstream report shapes +and behavior. Do not wrap them in a universal Notarius result envelope. No current +run-report need requires raw output; reserve native `Output`/`RenderOutput` for an +operation whose actual product needs a byte stream. + +### Compact Application Run Report + +CommandKit owns presentation; Notarius owns the meaning and deliberate contract +of the application run report. Retain these content categories: + +| Content | Purpose | +| --- | --- | +| Application report version | Identify the report contract for consumers. | +| Run and pipeline identity | Identify completed work. | +| Published bundle directory and applicable index path | Locate artifacts without guessing filenames. | +| Accepted/normalized and rejected output counts | Summarize production results. | +| Validation completeness/status | Distinguish fully validated results from permitted incomplete validation. | +| Process-warning counts | Expose degradation independently of reporting verbosity. | +| Debug location, when applicable | Locate requested diagnostic artifacts. | + +Detailed per-producer validation records, rejection reasons, and execution +provenance belong in the published bundle rather than duplicated collections in +the command report. Do not serialize the entire internal pipeline result or +complete upstream runtime types as a durable contract. U9 will select exact +fields, versioning, and durable record formats. The old CLI receipt's shape is +migration evidence, not a requirement to reproduce it. + +### Publication, Streams, And Machine Consumption + +The run command executes the pipeline, publishes the output bundle, completes +required run finalization, submits its successful report, and returns a native +outcome. The report describes an actual published result. Item 5 will settle the +treatment of requested debug-finalization failures. + +Use one primary run-result report, with progress and diagnostics separate. Wire +CommandKit's result destination to stdout and its diagnostic destination to +stderr. Subprocess consumers request `--format json`, capture streams separately, +check the exit status, and decode the successful run report. A report-write +failure may leave partial stdout even though publication already succeeded. +These run-consumption rules do not replace the native report behavior of other +commands, including configuration commands that can report failed validation. + +### Native Visibility And Message Categories + +| Native facility | Application use | +| --- | --- | +| `Report` | Requested command result. | +| `Progress` | Pipeline progress. | +| `Warn` | Actionable process degradation. | +| `Info` | Supplementary operational information. | +| `Debug` | Diagnostic detail, distinct from durable artifact capture. | +| Outcome explanation | Terminal failure or usage guidance. | + +Adopt native visibility rules directly: ordinary mode suppresses warning messages, +and quiet mode suppresses command results. Include validation status and +process-warning counts in the primary run report and its human summary so those +facts do not depend on warning-message visibility. Operators request `--verbose` +for warning details; machine callers expecting a report do not request `--quiet`. +Use native progress selection and failure visibility without custom bypasses. +Quiet suppression is not a delivery failure; native rendering errors can still +fail a quiet invocation. + +Artifact-quality advisories retain their application classification. Do not +reclassify them as process warnings merely to make them visible. + +### Native Outcomes And Domain Acceptance + +Adopt the standard process-status conventions: + +| Status | Meaning | +| --- | --- | +| `0` | The command completed successfully. | +| `1` | Operational failure, cancellation, publication failure, or presentation failure. | +| `2` | Invalid invocation or identifiable invalid configuration input. | +| `3` | Native `config diff --check` found differences. | + +Use ckconfig's structured configuration-error classification. Do not classify +every configuration failure as operational or every preflight failure as invalid +input. Invalid operator selections and missing bundled application resources can +have different causes; preserve that distinction and native error identity. + +Command completion remains separate from domain acceptance. A successfully +published run can contain recorded rejections or validation-incomplete results +allowed by application policy; those facts appear in reports and artifacts and +do not automatically fail the command. A terminal pipeline failure does fail the +command. Native status zero also does not replace reading standard PromptKit +reports for partial discovery or unsupported smoke checks. + +Application handlers supply useful native outcome explanations and retain the +underlying error through native cause handling. CommandKit presents the terminal +result. Remove duplicate terminal-error printing and custom outcome wrappers. + +### Delivery Failures, Progress, And Accounting + +Use CommandKit's native separation of operation causes from output and diagnostic +delivery failures. If publication succeeds but stdout fails, the final command +status reflects presentation failure while the published artifacts remain a +fact. Do not rewrite pipeline history as failed execution or automatically rerun +model work because report delivery failed. + +Notarius owns stage/lane/chunk progress, domain acceptance, and checkpoint reuse. +PromptKit supplies native execution observations and `ExecutionGroup` totals; +CommandKit presents the selected facts. Explicitly enroll the relevant initial, +continuation, and validator executions without double counting. Checkpoint reuse +does not add historical token usage to current execution totals, and unavailable +usage remains distinguishable from zero. + +Remove bespoke format selection, parallel receipt encoders, custom verbosity +handling, duplicate execution accounting, and superseded terminal-reporting +machinery. Retain only application report content, publication facts, domain +decisions, and small deliberate durable projections. + +Evidence: [native reporting](../../../commandkit/docs/consumers/guide.md#invocation-reporting), +[visibility and delivery](../../../commandkit/docs/cli.md#reporting), and +[outcome classification](../../../commandkit/docs/cli.md#outcome-status). +The current [run-result contract](../integrations/run-result.md) and [CLI failure mapping](../internal/cli.md#failure-mapping-and-terminal-reporting) -to understand application needs and caller migration impact. Their existing CLI -shapes and status mappings do not override the native-cutover direction. -Item 6 of the [PromptKit discussion record](promptkit-architecture-discussion.md#6-inspection-and-execution-bookkeeping) -provides the bookkeeping boundary; U9 still owns concrete durable record choices. +document migration impact. Item 6 of the +[PromptKit discussion record](promptkit-architecture-discussion.md#6-inspection-and-execution-bookkeeping) +retains the bookkeeping boundary; U9 still owns concrete durable record choices. ## 5. Debug Capture And Optional Diagnostics