From cb7e05284c554f0da170dc17a0e2e3eac2aed0bc Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sat, 7 Nov 2026 03:17:23 +0000 Subject: [PATCH] Clarify native preview reporting and cancellation ownership --- docs/roadmap/promptkit-v0.20.0.md | 32 ++++++++++++++++++++++++++----- 1 file changed, 27 insertions(+), 5 deletions(-) diff --git a/docs/roadmap/promptkit-v0.20.0.md b/docs/roadmap/promptkit-v0.20.0.md index 4b2ff95..a9ed368 100644 --- a/docs/roadmap/promptkit-v0.20.0.md +++ b/docs/roadmap/promptkit-v0.20.0.md @@ -256,7 +256,8 @@ no CommandKit wrappers or CLI commands for those construction conveniences. [OD-2](#od-2-request-controls-and-provider-parameter-grammar) owns reusable flag collections, aliases, input forms, and precedence. Expose preview as an independent capability alongside existing suite callbacks. Call `Engine.Prepare` once and -project its snapshot; a preliminary `ValidateRequest` call is unnecessary. +hand its native snapshot to core under OD-1; a preliminary `ValidateRequest` +call is unnecessary. Upstream excludes resolved credentials from prepared settings, but rendered application content is not a redacted configuration report. @@ -409,6 +410,14 @@ the existing Reporter lifecycle for visibility, destinations, delivery, and presentation failures, preserving domain outcomes and causes. This extends the existing reporting facility rather than creating a second delivery lifecycle. +Integration handlers decide whether cancellation stops report submission or +whether already-acquired evidence should still be reported. Core renders the +submitted content through its ordinary visibility and presentation-failure +lifecycle; do not introduce a blanket invocation-cancellation check that discards +such evidence. Ordinary inspection can stop before submission, while smoke +reports and OD-6 aggregate validation preserve available evidence after +cancellation. This does not resume canceled acquisition or execution work. + ### Agreed shared representation rules **Agreed:** use Go's standard `encoding/json` serialization conventions to @@ -745,6 +754,9 @@ rather than prescribing one test per line or exact cosmetic snapshots. - [ ] Invocation and writer APIs share rendering; quiet mode, domain outcomes, cause identity, cancellation, and presentation failures retain their agreed lifecycle. Changing format does not change selected content or detail level. + Cover handler-selected submission after cancellation for acquired smoke and + aggregate-validation evidence, without imposing that behavior on ordinary + inspection or restarting canceled work. - [ ] Every path in the [output inventory](#output-implementation-inventory) is reviewed. Remove obsolete per-command format plumbing and duplicate encoders; preserve explicit raw output, help, and diagnostic boundaries. @@ -768,8 +780,9 @@ rather than prescribing one test per line or exact cosmetic snapshots. native unknown facts, supported causes, and outcomes, including no-report cases. Smoke aggregation preserves ordered results and partial failure/cancellation evidence in one report. Default tests need no live credentials or model calls. -- [ ] OD-1 preview forwards the request, calls preparation once, and projects - native evidence without a model call or retained execution handle. Help explains +- [ ] OD-1 preview forwards the request, calls preparation once, and hands native + `PreparedRun` data to core unless a concrete presentation need warrants a small + projection. It makes no model call and retains no execution handle. Help explains the prompt-ID argument, resource I/O, and rendered-content disclosure. - [ ] OD-2 collections work in standard and application-owned commands with their own typed inputs. Preserve selected subsets, omission/empty/zero values, @@ -878,8 +891,11 @@ upstream fields selected under [resource inspection](#resource-inspection). ### OD-1: Preview command and report presentation -**Agreed:** use `promptkit show prepared PROMPT_ID` with the complete relevant -`PreparedRun` projection. The structured mapping, without a dedicated text +**Agreed:** use `promptkit show prepared PROMPT_ID` with the native `PreparedRun` +handed directly to core. Its existing JSON serialization fields cover the selected +report content. Introduce a small projection only for a concrete presentation +requirement; do not maintain a parallel report structure merely to copy those +fields. The structured mapping, without a dedicated text template, selects YAML through core's automatic policy. Universal format choices and automatic selection supersede both the earlier YAML-only restriction and the command-specific default proposal. The argument names @@ -1254,6 +1270,12 @@ results and available evidence from the failing scope. Do not claim unvisited sc were validated. Apply the same aggregation to concise and detailed modes, using one ordinary report value and the shared renderers, including one JSON document. +The validation handler submits that acquired evidence even when cancellation +stopped acquisition, following the +[shared reporting cancellation boundary](#agreed-format-independent-reporting-operation). +Do not reuse an ordinary inspection helper's cancellation guard to suppress this +report. Preserve the cancellation outcome alongside any presentation failure. + This deliberately changes delivery timing: scope results are no longer written one at a time during acquisition. Output failures are observed after acquisition has stopped and cannot prevent already-completed loads. Preserve the original