From 69d2ba7038333e80707643d06511e2abf1e2dd5d Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 20 Nov 2026 03:14:07 +0000 Subject: [PATCH] Allow native format selection for separate reports --- docs/roadmap/commandkit-adoption.md | 31 ++++++++----- docs/roadmap/commandkit-notepad.md | 24 +++++----- docs/roadmap/implementation.md | 71 ++++++++++++++++++----------- 3 files changed, 77 insertions(+), 49 deletions(-) diff --git a/docs/roadmap/commandkit-adoption.md b/docs/roadmap/commandkit-adoption.md index 8241ff1..f486c1d 100644 --- a/docs/roadmap/commandkit-adoption.md +++ b/docs/roadmap/commandkit-adoption.md @@ -63,8 +63,9 @@ dependencies remain subject to the existing No selected-format accessor is required for this migration. The native invocation reporter handles the main result on stdout; the existing writer API -handles the separate report using fixed JSON. The upstream accessor proposal -remains optional API work, independent of Scriptorium's adoption. +handles the separate report with automatic or explicitly selected rendering. +The upstream accessor proposal remains optional API work, independent of +Scriptorium's adoption. ## Target End State @@ -199,9 +200,9 @@ mapping, not a promise that separate invocations see an immutable snapshot. ### Generation, Reports, And Process Outcomes -Use a CommandKit `Report` whose value is PromptKit's native `RunResult` and whose -text template emits only the resulting artifact content. The template should -emit artifact bytes without presentation decoration. Default/text serves raw +Use a main CommandKit `Report` whose value is PromptKit's native `RunResult` and +whose text template emits only the resulting artifact content. The template +should emit artifact bytes without presentation decoration. Default/text serves raw artifact consumers; JSON and YAML expose the full native result. The native structured serialization, including its representation of artifact bytes, remains upstream-owned. There is no Scriptorium result envelope. @@ -211,7 +212,14 @@ remains upstream-owned. There is no Scriptorium result envelope. | Main result | Goes through the native invocation reporter to stdout using the selected format. | | `--format text` or default | Main result contains only the artifact. | | `--format json` or `--format yaml` | Main result contains the native result and metadata. | -| `--report PATH` | Additionally write the native result as JSON through CommandKit's renderer, independently of the main format and destination. | +| `--report PATH` | Additionally write the native result through CommandKit's renderer, independently of the main format and destination. Without `--report-format`, use native automatic rendering. | +| `--report-format text\|yaml\|json` | Explicitly select the separate report's native rendering format. | + +The separate report contains the same native `RunResult`, without the main +report's artifact-only text template. Omission of `--report-format` leaves the +format unset so CommandKit decides how to render it. Its current automatic +rendering for structured values is YAML; future upstream default changes flow +through. Scriptorium does not set a default report format. There is no `--output` or legacy `--out` flag. Callers save the main result by redirecting stdout in a shell or supplying a file as the subprocess stdout @@ -234,7 +242,7 @@ invocation-error envelope is part of this migration. Adopt CommandKit's quiet and verbosity semantics, including rendering and presentation failures. Console results use the invocation reporter; diagnostics use its diagnostic channel. An explicit `--report` write is an application -effect rendered as JSON through the writer API and still occurs in quiet mode. +effect rendered through the writer API and still occurs in quiet mode. Quiet does not cancel generation; it suppresses the main result even when stdout is redirected, while failure explanations remain visible. Move any retained generation summary into an appropriate native message category. @@ -296,8 +304,9 @@ documentation and examples describe the shipped interface. In particular: collections. Their upstream lifecycle differences remain visible. - Generation defaults to the artifact alone; structured output and separate reports expose the native result. Main output uses stdout and native quiet - semantics; the optional JSON report file is written independently of quiet, - with useful failure outcomes and nontransactional delivery. + semantics; the optional report file uses native automatic or explicitly + selected rendering, independently of quiet, with useful failure outcomes and + nontransactional delivery. - Narratio's linked acceptance scenarios are supported, including useful partial discovery and discovery before generated artifacts exist. - Native help, aliases, completion, and explicit smoke testing are available. @@ -309,8 +318,8 @@ documentation and examples describe the shipped interface. In particular: reflect the cutover. The CLI migration includes positional prompt selection, command replacements, literal versus file inputs, repeated mappings, overlay selection, native endpoint/timeout controls, stdout redirection in place of - output-file flags, content-bearing reports, report availability, quiet - behavior, and changed exit statuses. + output-file flags, content-bearing reports, optional report-format selection, + report availability, quiet behavior, and changed exit statuses. The [implementation plan](implementation.md) owns task sequencing, concrete adapter mappings, verification steps, and sprint status. There are no unresolved diff --git a/docs/roadmap/commandkit-notepad.md b/docs/roadmap/commandkit-notepad.md index 7c74bf8..2e5741b 100644 --- a/docs/roadmap/commandkit-notepad.md +++ b/docs/roadmap/commandkit-notepad.md @@ -100,18 +100,20 @@ is application work; deciding what constitutes a PromptKit failure is not. ### Native result reporting and separate report files -Decision recorded 2026-11-19: use a CommandKit `Report` containing PromptKit's +Decision revised 2026-11-19: use a main CommandKit `Report` containing PromptKit's native `RunResult`, with a text template that emits only generated content. Default/text output serves artifact consumers; standard JSON/YAML formats expose the native result and its metadata. This composes existing upstream APIs; ckprompt does not supply a general generation command or prescribe this template. Include an optional `--report PATH` flag requiring an explicit file destination. -Omitting it creates no separate report file. When requested, save the native -result as JSON through CommandKit's renderer, using the same representation as -inline JSON reporting. The report file's format is JSON independently of the -main result's `--format`; requesting a report does not change the main result's -representation or destination. +Omitting it creates no separate report file. When requested, pass the native +`RunResult` to CommandKit's renderer without an artifact-only text template. +Optional `--report-format text|yaml|json` selects the native rendering format. +When omitted, leave the format unset and let CommandKit choose; its current +structured-value default is YAML, and future upstream changes flow through. +There is no Scriptorium default or filename-based selection. Report rendering +is independent of the main result's `--format`, representation, and destination. Failure to write a requested report is a delivery failure and must produce a nonzero outcome and diagnostic. @@ -136,7 +138,7 @@ on stderr. Do not retain the old `--out` flag. `--format` selects the stdout representation independently of redirection: default/text emits the raw artifact, while JSON/YAML emits the native result with metadata. The filename does not select the format. `--report PATH` remains -an additional native JSON report, so callers can save the raw artifact and its +an additional native result report, so callers can save the raw artifact and its report to separate files by redirecting stdout and supplying `--report`. Adopt CommandKit's upstream verbosity and `--quiet` semantics fully, including @@ -146,8 +148,8 @@ Quiet mode does not cancel generation, and failure explanations remain visible through the native diagnostic channel. An explicit `--report` write remains an application-owned effect in quiet mode, -using the existing writer API with fixed JSON. Quiet suppresses the main result -even when stdout is redirected; callers wanting that result omit quiet. No +using the existing writer API. Quiet suppresses the main result even when stdout +is redirected; callers wanting that result omit quiet. No selected-format accessor is needed, so the upstream proposal is no longer a migration prerequisite. The [upstream reporting contract][upstream-cli] owns the shared verbosity rules. @@ -243,8 +245,8 @@ families. The CLI migration does not redesign the HTTP API. Reuse ckprompt's applicable resource, input, and execution flag bindings. Preserve native parsing, explicit-value presence, repeated mapping semantics, and PromptKit's interpretation of request values. Keep application-specific -configuration/source selection, `--report`, server options, and the -whole-operation deadline separate from framework request controls. +configuration/source selection, `--report`/`--report-format`, server options, +and the whole-operation deadline separate from framework request controls. Enable upstream convenience aliases consistently on applicable commands, including `run`. Prefer their shorter spellings in user-facing documentation, diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index 50b9d73..7ac0e60 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -52,7 +52,8 @@ expanding this sprint into another framework upgrade. No selected-format accessor is required. Main results go through the native invocation reporter to stdout, with caller-controlled redirection. The separate -`--report` file uses the existing `RenderReport(writer, report, FormatJSON)` API. +`--report` file uses the existing `RenderReport(writer, report, format)` API, +with the format unset unless `--report-format` is supplied. The upstream accessor proposal is optional and does not block any stage. Do not add a second reporting-flag parser or private-API workaround. @@ -112,7 +113,7 @@ classify every `ErrInvalidConfig` wrapper as invalid user input without native evidence. Retain safe explanations for provider failures; causes are inspectable Go errors, not permission to print credential-bearing provider payloads. -Use one `commandkit.Report` with the actual `*promptkit.RunResult` as `Value`. +Use a main `commandkit.Report` with the actual `*promptkit.RunResult` as `Value`. Its reusable text template emits `Artifact.Body` as bytes represented as a string, with no added newline, header, escaping, or summary. Native JSON/YAML serialization remains upstream-owned, including byte representation and native metadata. @@ -123,9 +124,16 @@ their subprocess API; Scriptorium keeps diagnostics on stderr. Native quiet suppresses the main result even when stdout is redirected. Consumers wanting that result omit `--quiet`. -`--report PATH` additionally renders the same native result as fixed JSON via -`commandkit.RenderReport`, regardless of the main format or stdout destination. -This application-owned file write still occurs in quiet mode. Omitting the flag +`--report PATH` additionally renders `commandkit.Report{Value: result}` through +`commandkit.RenderReport`, without the main report's artifact-only text template. +Optional `--report-format text|yaml|json` selects its native format independently +of stdout's `--format`. When omitted, pass the zero `commandkit.Format` value +(`""`) for native automatic rendering. Do not substitute JSON, YAML, or text. +CommandKit currently renders structured values as YAML in automatic and text +modes; future upstream defaults flow through without application policy changes. +Filename extensions do not select the format. + +This application-owned file write still occurs in quiet mode. Omitting `--report` creates no separate report. Use ordinary create/truncate operations (mode 0644 for newly created files, subject to umask), preserve normal OS behavior for existing files, and handle render/write/close errors. Do not create parent @@ -349,7 +357,7 @@ helpers in `internal/adapter/cli`, and run integration tests. 7. Build the native result report/text template and submit stdout results through the invocation reporter. Keep any retained generation summary as a native Info message, normally hidden at ordinary verbosity. Stage 6 adds - the separate JSON report without changing the result shape. + the separate native report without changing the result shape. **Evidence:** real-engine/controlled-provider tests exercise the assembled positional command, preferred aliases, literal/file precedence, comma-containing @@ -359,18 +367,22 @@ serialization, completed invalid content/status 4, and resultless operational failure/status 1. Include a small typed-error mapping table rather than parsing error prose. Use native config classification for representative load failures. -## Stage 6: Add Separate Native JSON Report Delivery +## Stage 6: Add Separate Native Report Delivery **Primary files:** CLI generation-delivery helpers and focused delivery tests. 1. Add `--report` as a presence-aware application file flag with path completion. - Reject an explicitly empty path before executing the model. Do not create - files during validation or add a main-output destination flag. -2. Implement the settled delivery rules using the same report as Stage 5. - Main stdout uses `Invocation.Reporter().Report`; the separate report uses - `RenderReport(..., FormatJSON)`. The invocation reporter owns main format - selection, including automatic artifact-template output. No selected-format - accessor is needed. Extensions never select formats. + Add optional `--report-format` with typed binding and completion for the native + `text`, `yaml`, and `json` formats; leave its default unset. Reject an explicitly + empty path or invalid format as usage errors before executing the model. Do + not create files during validation or add a main-output destination flag. +2. Main stdout uses Stage 5's report through `Invocation.Reporter().Report`. + Render the separate report through `RenderReport` with + `commandkit.Report{Value: result}`, sharing the native result but no text + template. Pass the empty format when `--report-format` is omitted; + otherwise pass the explicitly selected native format. The invocation reporter + owns main format selection, including automatic artifact-template output. + No selected-format accessor is needed. Extensions never select formats. 3. Keep the report file effect independent of console quiet selection. Do not temporarily change reporting mode, rerun `App.Execute`, capture stdout and parse it back, or replace native structured serialization to obtain the report. @@ -385,12 +397,15 @@ error prose. Use native config classification for representative load failures. native run result is eligible for the new CLI report file. **Evidence:** cover default/text raw bytes and JSON/YAML through stdout, including -a subprocess whose stdout is redirected to a file. Check separate JSON report -equivalence to inline native JSON and independence from the main format. Quiet -suppresses main output, including redirected stdout, while an explicitly requested -report is still written. Test empty report paths, replacement without parent -creation, stdout failure followed by report attempt, report failure after -successful main delivery, and an unchanged existing report on a resultless +a subprocess whose stdout is redirected to a file. Compare omitted and explicit +report formats against native `RenderReport` of the same value without a template; +use an empty format for the omission expectation, not hard-coded YAML. Confirm +the separate report retains the full result and metadata, independently of the +main format and filename extension. Quiet suppresses main output, including +redirected stdout, while an explicitly requested report is still written. Test +empty report paths and invalid formats before model execution, replacement +without parent creation, stdout failure followed by report attempt, report failure +after successful main delivery, and an unchanged existing report on a resultless operational failure. Use temporary files and a small failing-writer seam for stdout/render failures. Confirm status 4 survives a delivery failure while its explanation identifies the failure. No exhaustive filesystem matrix. @@ -456,7 +471,7 @@ version output. Test-local fallback deadlines prevent regressions from hanging. | Owner | Required changes | | --- | --- | - | `docs/cli.md` | New command tree, applicable flags/aliases, native selection/reporting, stdout redirection and separate report-file semantics, statuses, completion, explicit smoke tests. Link upstream for standard family details. | + | `docs/cli.md` | New command tree, applicable flags/aliases, native selection/reporting, stdout redirection, separate report files with optional format selection, statuses, completion, explicit smoke tests. Link upstream for standard family details. | | `docs/config.md` | Removed render default; native overlay-name selection; application overrides and loader authority; app/framework scopes and native comparison. Preserve collection/source/default semantics. | | `docs/integrations/subprocess.md` | Artifact-first execution, native reports and result availability, content-bearing metadata reports, stream capture, partial delivery, cancellation, and caller responsibility. Link exact CLI syntax rather than duplicate it. | | `docs/operations.md` | Updated discovery/preview/execution workflow, explicit smoke-test effects, potentially sensitive reports, file retention, server process lifecycle, unchanged HTTP limits. | @@ -468,9 +483,10 @@ version output. Test-local fallback deadlines prevent regressions from hanging. CLI/consumer overview. Cover positional prompt selection; `render` and `inspect` replacements; file versus literal inputs; repeated mappings without comma splitting; `--overlay` names; endpoint and timeout spelling/units; - stdout redirection in place of output-file flags; separate JSON reports, - native formats, and result availability; quiet behavior (omit quiet to capture - main output); status changes; and content-bearing native inspection/reports. + stdout redirection in place of output-file flags; separate native reports, + optional report-format selection and upstream automatic rendering, and result + availability; quiet behavior (omit quiet to capture main output); status + changes; and content-bearing native inspection/reports. This prepares release migration information without selecting a version, editing published notes, tagging, pushing, or adding release gates. Eventual release notes must summarize the breaking change and link the guide under @@ -509,9 +525,10 @@ current documentation links resolve. Migration examples match the built CLI. provider availability. Confirm retired commands/flags are absent or rejected except where a spelling now intentionally has its native meaning (`--input`). 4. Review the output/outcome matrix across stdout, caller redirection, report - files, quiet, completed validation failure, operational failures, presentation - errors, cancellation, and partial delivery. Verify reports are native values - with no Scriptorium wrapper, and HTTP's distinct contract remains unchanged. + files with automatic/explicit formats, quiet, completed validation failure, + operational failures, presentation errors, cancellation, and partial delivery. + Verify reports are native values with no Scriptorium wrapper, and HTTP's + distinct contract remains unchanged. 5. Run the repository baseline on the selected toolchain: ```sh