Simplify the planned artifact output and completion report
This commit is contained in:
@@ -23,20 +23,22 @@ Item 1 selects the dependency versions and standard capabilities, including
|
||||
PromptKit v0.23.3 in place of the master roadmap's initial v0.23.2 target.
|
||||
Item 2 selects the application command structure and input conventions. Item 3
|
||||
selects native composition, shared loaders, and application preflight. Item 4
|
||||
selects native reporting, machine output, and outcomes. Item 5 selects conditional
|
||||
debug capture through the shared `--debug` flag, native diagnostics, and a run/
|
||||
operation bundle with a `latest` symlink. Its capture inventory, depth, and
|
||||
selects artifact-first output, a small native completion report, and outcomes.
|
||||
Item 5 selects conditional debug capture through the shared `--debug` flag,
|
||||
native diagnostics, and a run/operation bundle with a `latest` symlink. Its
|
||||
capture inventory, depth, and
|
||||
explicit-empty directory behavior are resolved below. Item 6 selects verification,
|
||||
migration guidance, and incorporation into the master roadmap; that incorporation
|
||||
is still pending.
|
||||
|
||||
The cache follow-up in item 4 replaces the earlier checkpoint design with an
|
||||
accepted-artifact cache keyed by initial message content. The output/report
|
||||
contract has been reopened around consumer needs; it is not merely waiting for
|
||||
field names and versions. Function signatures, bundle filenames other than
|
||||
`latest`, and similar implementation details do not
|
||||
reopen the selected architecture. Broader PromptKit choices remain with the
|
||||
master roadmap.
|
||||
direction is settled: deliver artifacts as files and render a small structured
|
||||
completion value through CommandKit, with a native text template for ordinary
|
||||
human output. Detailed diagnostics belong in optional debug capture. Function
|
||||
signatures, debug filenames other than `latest`, and similar implementation
|
||||
details do not reopen the selected architecture. Broader PromptKit choices
|
||||
remain with the master roadmap.
|
||||
|
||||
The PromptKit ownership direction remains in force. The revised
|
||||
[cache decision](promptkit-architecture-discussion.md#5-accepted-artifact-cache)
|
||||
@@ -93,13 +95,13 @@ spending migration effort recreating existing CLI conventions.
|
||||
| Standard configuration and PromptKit inspection commands | ckconfig and ckprompt, supplied with application loaders and request builders. |
|
||||
| Configuration composition and model execution | ConfigKit and PromptKit, using the native boundaries already selected. |
|
||||
| Pipeline execution and listing, pipeline-specific preflight, artifact publication, accepted-artifact caching, and domain outcomes | Notarius application handlers and framework. |
|
||||
| Receipt content, debug artifact lifecycle, and application outcome meaning | Notarius, expressed through CommandKit's reporting and outcome conventions. |
|
||||
| Completion-summary content, debug artifact lifecycle, and application outcome meaning | Notarius, expressed through CommandKit's reporting and outcome conventions. |
|
||||
|
||||
Adopt the standard capability suite selected in item 1, including upstream
|
||||
commands and recommendations beyond the existing Notarius surface. Items 2–4
|
||||
select application inputs, composition, and reporting; items 5–6 select debug
|
||||
capture principles, verification, and migration. Concrete record details remain
|
||||
open as identified below.
|
||||
capture, verification, and migration. The earlier extensive receipt/manifest
|
||||
proposal is superseded by item 4's artifact-first contract.
|
||||
|
||||
Pipeline and domain code should remain independent of CommandKit. Native
|
||||
PromptKit observations supply execution facts; Notarius attaches application
|
||||
@@ -108,9 +110,9 @@ meaning and selects report content. CommandKit supplies presentation mechanics.
|
||||
Two application needs already identified for deliberate treatment are:
|
||||
|
||||
- The current `run --json` publishes a receipt after output publication. Item 4
|
||||
records native reporting mechanics, but the output/report contract has been
|
||||
reopened around artifact delivery and optional metadata. Its earlier mandatory
|
||||
run-report proposal is subject to that discussion.
|
||||
replaces that contract with artifact files and a small completion summary,
|
||||
rendered natively as text, YAML, or JSON. No automatic metadata bundle or
|
||||
separately persisted report is required.
|
||||
- Item 5 uses native `--debug` for diagnostic verbosity and for capture when an
|
||||
effective debug directory is available, including an application default.
|
||||
Capture selection remains independent of the final reporting mode.
|
||||
@@ -202,8 +204,8 @@ implementing cache lookup; the version table remains the reviewed baseline, not
|
||||
a claim that the new hash is already available. On roadmap
|
||||
reconciliation, incorporate these versions into U1, replace U10's open standard
|
||||
capability selection with this decision, and incorporate the application wiring
|
||||
and presentation decisions from items 2–5. Preserve the remaining record choices
|
||||
explicitly as directed by item 6.
|
||||
and presentation decisions from items 2–5. Retire superseded durable-record
|
||||
requirements as directed by item 6.
|
||||
|
||||
## 2. Native Command Surface And Inputs
|
||||
|
||||
@@ -336,8 +338,7 @@ Generic effective configuration and source explanations remain available through
|
||||
`config show` and `config explain`. ConfigKit's safe inspection facilities do not
|
||||
automatically redact an arbitrary application report; select appropriate report
|
||||
content rather than dumping the raw typed configuration. Item 3 selects loader
|
||||
wiring; item 4 selects reporting conventions and identifies remaining run-report
|
||||
contract choices.
|
||||
wiring; item 4 selects the artifact-first output and completion-report contract.
|
||||
|
||||
### Removals And Remaining Work
|
||||
|
||||
@@ -350,8 +351,9 @@ native `config show`, `config profile` with `promptkit show profile`, and bespok
|
||||
Application handlers still validate meaningful input combinations and translate
|
||||
typed inputs into pipeline selections. Native `--debug` supplies reporting
|
||||
behavior and conditional artifact capture as selected in item 5.
|
||||
Items 3–4 select loader behavior, report content categories, and outcome
|
||||
presentation; exact run-report fields and durable formats remain with U9.
|
||||
Items 3–4 select loader behavior, the small completion report, and native outcome
|
||||
presentation. Reconcile U9 with these decisions rather than expanding durable
|
||||
metadata.
|
||||
|
||||
Evidence: [CommandKit command/input conventions](../../../commandkit/docs/cli.md#registered-command-syntax),
|
||||
[reusable PromptKit flags](../../../commandkit/docs/consumers/guide.md#request-builders-and-loading),
|
||||
@@ -521,14 +523,12 @@ Evidence: [native suite and loader ownership](../../../commandkit/docs/consumers
|
||||
|
||||
## 4. Reports, Machine Output, And Outcomes
|
||||
|
||||
**Discussion / decision:** Native reporting and outcome mechanics remain selected.
|
||||
The output/report contract is reopened: consumers primarily need resulting
|
||||
artifacts, metadata may be optional, failures need useful nonzero outcomes, and
|
||||
operators/developers use debug capture for detailed investigation. The earlier
|
||||
run-report and durable-metadata proposals below remain discussion material for
|
||||
that review, not requirements to preserve or expand. No new report/manifest
|
||||
schemas have been selected. Finalize the cache decision first; return to the
|
||||
consumer output/report contract afterward. Implementation is pending.
|
||||
**Discussion / decision:** Agreed. Deliver artifacts as files and use CommandKit
|
||||
to render a small structured completion report on stdout, with a native text
|
||||
template for the ordinary successful case. Additional operational diagnostics
|
||||
use stderr; detailed investigation uses optional debug capture. This supersedes
|
||||
the earlier extensive receipt, manifest, and durable-execution-record proposals.
|
||||
Keep native outcome and delivery-failure handling. Implementation is pending.
|
||||
|
||||
### Accepted-Artifact Cache And PromptKit Ownership
|
||||
|
||||
@@ -572,45 +572,82 @@ Concrete cache flags and legacy-checkpoint handling remain implementation-planni
|
||||
choices. The output/report discussion must not reintroduce cache metadata solely
|
||||
to populate a report. The master roadmap still needs this replacement policy.
|
||||
|
||||
### One Report Value For Human And Machine Output
|
||||
### Artifacts Are The Deliverable
|
||||
|
||||
Application commands submit ordinary values through `Reporter.Report` and use
|
||||
native `--format text|yaml|json` rendering. Execution supplies one run-report
|
||||
value with a concise text template for default human output; JSON and YAML render
|
||||
the same structured value. Do not maintain separate human-summary and JSON-receipt
|
||||
construction paths.
|
||||
Publish the accepted domain artifacts as files in the output directory. Their
|
||||
contents and encoding are independent of CLI report formatting: `--format`
|
||||
controls the completion summary, not the artifact payloads. Keep artifact
|
||||
discovery understandable without requiring consumers to read execution metadata.
|
||||
|
||||
```sh
|
||||
notarius run dnd-session --input-file transcript.json
|
||||
notarius run dnd-session --input-file transcript.json --format json
|
||||
Do not require a run manifest, metadata bundle, or persisted completion receipt
|
||||
alongside the artifacts. Preserve information intrinsic to the domain artifacts
|
||||
and their existing schemas; do not add model, retry, validation-history, or
|
||||
execution inventories to those payloads merely because it is available.
|
||||
Detailed operational evidence belongs in item 5's optional debug bundle.
|
||||
|
||||
### Small Structured Completion Report And Native Text Template
|
||||
|
||||
Submit one small application value through `Reporter.Report`. Use these fields
|
||||
for an ordinary successful run:
|
||||
|
||||
| Field | Meaning |
|
||||
| --- | --- |
|
||||
| `pipeline` | Selected pipeline identifier. |
|
||||
| `artifact_count` | Number of deliverable artifacts produced, including reused artifacts; excludes internal intermediate results, cache entries, and debug files. |
|
||||
| `output_directory` | Absolute path to the directory containing the published artifacts. |
|
||||
| `debug_directory` | Optional absolute path to the completed run-specific debug bundle; omitted when capture is disabled. |
|
||||
|
||||
For example, without debug capture:
|
||||
|
||||
```json
|
||||
{
|
||||
"pipeline": "dnd-session",
|
||||
"artifact_count": 6,
|
||||
"output_directory": "/work/results/run-123"
|
||||
}
|
||||
```
|
||||
|
||||
Standard configuration and PromptKit commands retain their upstream report shapes
|
||||
and behavior. Do not wrap them in a universal Notarius result envelope. No current
|
||||
run-report need requires raw output; reserve native `Output`/`RenderOutput` for an
|
||||
operation whose actual product needs a byte stream.
|
||||
Supply a short native `text/template` to render that same value as:
|
||||
|
||||
### Compact Application Run Report
|
||||
```text
|
||||
Success: dnd-session produced 6 artifacts in /work/results/run-123
|
||||
```
|
||||
|
||||
CommandKit owns presentation; Notarius owns the meaning and deliberate contract
|
||||
of the application run report. Retain these content categories:
|
||||
CommandKit automatically renders a string value as plain text. Here the value
|
||||
is structured so JSON/YAML callers receive named fields instead of a success
|
||||
sentence they must parse. The native text template provides the concise human
|
||||
presentation; without it, a structured value would default to YAML. Use native
|
||||
`--format text|yaml|json` selection and serializers, without separate encoders or
|
||||
independent human/machine report construction.
|
||||
|
||||
| Content | Purpose |
|
||||
| --- | --- |
|
||||
| Application report version | Identify the report contract for consumers. |
|
||||
| Run and pipeline identity | Identify completed work. |
|
||||
| Published bundle directory and applicable index path | Locate artifacts without guessing filenames. |
|
||||
| Accepted/normalized and rejected output counts | Summarize production results. |
|
||||
| Validation completeness/status | Distinguish fully validated results from permitted incomplete validation. |
|
||||
| Process-warning counts | Expose degradation independently of reporting verbosity. |
|
||||
| Debug location, when applicable | Locate requested diagnostic artifacts. |
|
||||
When a completed run includes rejected work or permitted incomplete validation,
|
||||
qualify the completion message clearly. Include the rejection count and/or
|
||||
incomplete-validation fact in the same structured value when applicable. Do not
|
||||
present unqualified success that conceals those outcomes, or make their visibility
|
||||
depend on verbose/debug mode. These are small outcome qualifications, not
|
||||
per-producer histories or a general metadata envelope.
|
||||
|
||||
Detailed per-producer validation records, rejection reasons, and execution
|
||||
provenance belong in the published bundle rather than duplicated collections in
|
||||
the command report. Do not serialize the entire internal pipeline result or
|
||||
complete upstream runtime types as a durable contract. U9 will select exact
|
||||
fields, versioning, and durable record formats. The old CLI receipt's shape is
|
||||
migration evidence, not a requirement to reproduce it.
|
||||
No report-version field, run-ID field, token totals, profile inventory, warning
|
||||
statistics, validation-summary collection, or durable execution ledger is required
|
||||
by this completion contract. The old receipt is migration context, not a shape
|
||||
to preserve. Standard configuration and PromptKit commands retain their native
|
||||
report values and behavior without a universal Notarius wrapper.
|
||||
|
||||
### Optional Machine Output And Saved Reports
|
||||
|
||||
Operators and subprocess callers select YAML or JSON through the native format
|
||||
flag. The completion report is enough to find the artifacts and recognize any
|
||||
qualified outcome. Consumers interested only in artifacts need no metadata file.
|
||||
|
||||
Save a report with ordinary shell redirection:
|
||||
|
||||
```sh
|
||||
notarius run dnd-session --input-file transcript.json --format json > run-report.json
|
||||
```
|
||||
|
||||
Do not add a separate report-file flag or automatically persist a duplicate report
|
||||
for this design. Keep stdout as one completion report, with additional diagnostics
|
||||
on stderr, so `--format json --verbose` still produces valid report JSON.
|
||||
|
||||
### Publication, Streams, And Machine Consumption
|
||||
|
||||
@@ -628,28 +665,37 @@ failure may leave partial stdout even though publication already succeeded.
|
||||
These run-consumption rules do not replace the native report behavior of other
|
||||
commands, including configuration commands that can report failed validation.
|
||||
|
||||
### Native Visibility And Message Categories
|
||||
### Native Visibility, Diagnostics, And Debug Location
|
||||
|
||||
| Native facility | Application use |
|
||||
| --- | --- |
|
||||
| `Report` | Requested command result. |
|
||||
| `Progress` | Pipeline progress. |
|
||||
| `Warn` | Actionable process degradation. |
|
||||
| `Info` | Supplementary operational information. |
|
||||
| `Debug` | Diagnostic detail, distinct from durable artifact capture. |
|
||||
| Outcome explanation | Terminal failure or usage guidance. |
|
||||
Wire CommandKit's result destination to stdout and diagnostic destination to
|
||||
stderr. Use `Info`, `Warn`, `Debug`, `Progress`, and native outcome explanations
|
||||
for their respective categories; do not mix these messages into a JSON/YAML
|
||||
completion report.
|
||||
|
||||
Adopt native visibility rules directly: ordinary mode suppresses warning messages,
|
||||
and quiet mode suppresses command results. Include validation status and
|
||||
process-warning counts in the primary run report and its human summary so those
|
||||
facts do not depend on warning-message visibility. Operators request `--verbose`
|
||||
for warning details; machine callers expecting a report do not request `--quiet`.
|
||||
Use native progress selection and failure visibility without custom bypasses.
|
||||
Quiet suppression is not a delivery failure; native rendering errors can still
|
||||
fail a quiet invocation.
|
||||
| Mode | Stdout | Stderr |
|
||||
| --- | --- | --- |
|
||||
| Ordinary | Brief completion summary. | Failure explanation, if needed. |
|
||||
| `--verbose` | Same completion report. | Additional operational information, warnings, and any failure explanation. |
|
||||
| `--quiet` | Nothing. | Failure explanation, if needed. |
|
||||
| `--debug` | Completion report, including capture location when enabled. | Native debug diagnostics, information, warnings, progress, and any failure explanation. |
|
||||
|
||||
Artifact-quality advisories retain their application classification. Do not
|
||||
reclassify them as process warnings merely to make them visible.
|
||||
Adopt native reporting-mode ordering, explicit progress selection, and failure
|
||||
visibility. The table describes mode defaults; `--progress` can explicitly enable
|
||||
progress outside quiet mode. Machine callers expecting a report do not request
|
||||
`--quiet`. Quiet suppression is not a delivery failure; native rendering errors
|
||||
can still fail a quiet invocation. Artifact-quality advisories keep their domain
|
||||
classification rather than becoming process warnings solely for visibility.
|
||||
|
||||
When capture completes, the text template adds a line such as:
|
||||
|
||||
```text
|
||||
Debug capture: /tmp/notarius-debug/run-123
|
||||
```
|
||||
|
||||
Use `debug_directory` in the structured value for the same location. Provide the
|
||||
path rather than mirroring the debug bundle to stdout. An explicitly empty debug
|
||||
directory preserves native debug diagnostics but creates no capture or capture
|
||||
path in the completion report. Item 5 owns capture controls and lifecycle.
|
||||
|
||||
### Native Outcomes And Domain Acceptance
|
||||
|
||||
@@ -669,14 +715,18 @@ have different causes; preserve that distinction and native error identity.
|
||||
|
||||
Command completion remains separate from domain acceptance. A successfully
|
||||
published run can contain recorded rejections or validation-incomplete results
|
||||
allowed by application policy; those facts appear in reports and artifacts and
|
||||
do not automatically fail the command. A terminal pipeline failure does fail the
|
||||
allowed by application policy; those facts qualify the completion report and do
|
||||
not automatically fail the command. A terminal pipeline failure does fail the
|
||||
command. Native status zero also does not replace reading standard PromptKit
|
||||
reports for partial discovery or unsupported smoke checks.
|
||||
|
||||
Application handlers supply useful native outcome explanations and retain the
|
||||
underlying error through native cause handling. CommandKit presents the terminal
|
||||
result. Remove duplicate terminal-error printing and custom outcome wrappers.
|
||||
result. On failure, provide a useful explanation and nonzero status rather than
|
||||
requiring a separate structured error-report protocol. `--format` controls the
|
||||
completion report, not native diagnostics. If artifacts were published before a
|
||||
later failure, include their location in the failure explanation. Remove duplicate
|
||||
terminal-error printing and custom outcome wrappers.
|
||||
|
||||
### Delivery Failures, Progress, And Accounting
|
||||
|
||||
@@ -687,16 +737,21 @@ fact. Do not rewrite pipeline history as failed execution or automatically rerun
|
||||
model work because report delivery failed.
|
||||
|
||||
Notarius owns stage/lane/chunk progress, domain acceptance, and cache reuse.
|
||||
PromptKit supplies native execution observations and `ExecutionGroup` totals;
|
||||
CommandKit presents the selected facts. Explicitly enroll the relevant initial,
|
||||
continuation, and validator executions without double counting. Cache reuse
|
||||
does not add historical token usage to current execution totals, and unavailable
|
||||
usage remains distinguishable from zero.
|
||||
PromptKit supplies native execution observations and `ExecutionGroup` totals.
|
||||
These facts support native operational diagnostics and optional debug capture;
|
||||
they do not require a permanent consumer execution report. Explicitly enroll the
|
||||
relevant initial, continuation, and validator executions without double counting.
|
||||
Cache reuse does not add historical token usage to current execution totals, and
|
||||
unavailable usage remains distinguishable from zero.
|
||||
|
||||
Remove bespoke format selection, parallel receipt encoders, custom verbosity
|
||||
handling, duplicate execution accounting, and superseded terminal-reporting
|
||||
machinery. Retain only application report content, publication facts, domain
|
||||
decisions, and small deliberate durable projections.
|
||||
machinery. Retire automatic reporting-only metadata bundles, mandatory run
|
||||
manifests, per-producer provenance inventories, and durable execution-ledger
|
||||
requirements. Do not preserve or rebuild those structures to populate the small
|
||||
completion value. Use existing application outcome facts for its contents and
|
||||
the debug overview. Artifact schemas remain domain contracts; detailed debug
|
||||
representations remain diagnostic data.
|
||||
|
||||
Evidence: [native reporting](../../../commandkit/docs/consumers/guide.md#invocation-reporting),
|
||||
[visibility and delivery](../../../commandkit/docs/cli.md#reporting), and
|
||||
@@ -705,7 +760,8 @@ The current [run-result contract](../integrations/run-result.md) and
|
||||
[CLI failure mapping](../internal/cli.md#failure-mapping-and-terminal-reporting)
|
||||
document migration impact. Item 6 of the
|
||||
[PromptKit discussion record](promptkit-architecture-discussion.md#6-inspection-and-execution-bookkeeping)
|
||||
retains the bookkeeping boundary; U9 still owns concrete durable record choices.
|
||||
retains the native bookkeeping boundary. Reconcile U9 with this simpler contract,
|
||||
including retirement of superseded durable-record requirements.
|
||||
|
||||
## 5. Debug Capture And Optional Diagnostics
|
||||
|
||||
@@ -817,7 +873,7 @@ notifications alone do not provide cache short-circuiting.
|
||||
Conversation capture required for continuation remains in-process execution
|
||||
state. Persisting its history requires enabled debug capture. Cached values are
|
||||
accepted domain artifacts, not conversation state or provenance records. The
|
||||
separate output/report contract remains under discussion.
|
||||
separate artifact-first output and completion-report contract is selected in item 4.
|
||||
|
||||
### Capture Lifecycle And Failure Policy
|
||||
|
||||
@@ -951,7 +1007,7 @@ a fabricated model execution. Historical provenance is not required in cache
|
||||
files or reconstructed solely for debug capture.
|
||||
|
||||
Retain enough available context to investigate a run that fails before publishing
|
||||
output. Do not depend entirely on a successful output manifest or automatically
|
||||
output. Do not depend entirely on successfully published artifacts or automatically
|
||||
copy every original input and published artifact. Retain complete native
|
||||
conversation snapshots for successive turns as selected below. Do not build a
|
||||
competing transcript model or deduplication system to eliminate repeated history.
|
||||
@@ -1188,8 +1244,10 @@ completeness. The phases do not require separate releases or an intermediate
|
||||
compatibility framework.
|
||||
|
||||
Resolve U1's dependency baseline and U10's CommandKit scope from items 1–4. Narrow
|
||||
U9 to the reopened consumer output/report contract, and debug implementation
|
||||
details within item 5's resolved inventory, depth, and directory controls.
|
||||
U9 to item 4's selected artifact-first output/completion contract and item 5's
|
||||
debug implementation details within the resolved inventory, depth, and controls.
|
||||
Retire earlier receipt/manifest field inventories rather than treating them as
|
||||
unresolved requirements to recreate.
|
||||
Update U1 for the upstream message-content hash and replace U8's checkpoint
|
||||
identity/provenance plan with the accepted-artifact cache decision.
|
||||
Reconcile removal inventory and acceptance criteria, including old
|
||||
|
||||
Reference in New Issue
Block a user