diff --git a/docs/roadmap/promptkit.md b/docs/roadmap/promptkit.md index 20f489d7..1e0ccf1f 100644 --- a/docs/roadmap/promptkit.md +++ b/docs/roadmap/promptkit.md @@ -182,8 +182,9 @@ requirements discovery/validation and prepared-request preview; and explicit reachability, authentication, text, structured-output, and aggregate smoke tests. Include upstream convenience aliases and useful completion hints. -Use native typed declarations and supplied-value semantics. Defaults apply to -omission, not explicit empty/false/zero values. Preserve native command-path, +Use native typed declarations and supplied-value semantics. Preserve the +distinction between omission and explicit empty/false/zero values, then apply +each field's selected default or empty-value policy. Preserve native command-path, operand, repeated-flag, and `--` parsing conventions. Check every occurrence of a singleton flag and use the last successfully parsed value. @@ -311,6 +312,11 @@ and inspection: recursive `.yml`/`.yaml` files under `prompts/`, beside the main application configuration actually selected by ConfigKit. +These are ordered underlays of the single application load. The selected main +document and its imports, selected overlay, and explicit configuration overrides +participate through ConfigKit's native precedence. Do not introduce a second +PromptKit main document or apply overlays/overrides again during engine construction. + Complete documents place native resources and `schema_root` beneath `promptkit`. Use `SettingsInput` for wrapperless settings fragments and `CollectionInput` for collection fragments; discovered fragments need an explicit destination because @@ -537,10 +543,11 @@ The application flow is: and the resolved `max_additional_turns` as native `AdditionalTurns`. Its callback selects a repair prompt and supplies findings; PromptKit retains the initial exchange and subsequent turns. -4. Interpret every delivered revised response and rebuild from the fixed original - inputs. Run the complete configured validation chain again for changed - candidates, including validators that previously approved. Assess the final - permitted response before declaring exhaustion. +4. Apply the same native-result check, interpretation, and assessment to every + delivered revised response, rebuilding from the fixed original inputs. Run + the complete applicable validation chain even when a revision produces an + unchanged candidate, including validators that previously approved. Assess + the final permitted response before declaring exhaustion. 5. Stop on acceptance or a terminal condition. If no acceptable result is obtained, fail in strict mode or produce the module's supported fallback in permissive mode. Native `stopped` and `exhausted` outcomes do not themselves mean application @@ -961,9 +968,12 @@ prepared/result details for the actual execution target, including overrides. Standard commands retain their native reports, support/applicability distinctions, and failure behavior. Smoke status zero alone does not prove a check passed. -Use ordinary `Run` with `OnPrepared` when actual request details are needed for -diagnostics; split preparation has a separate purpose for cache lookup. Native -observations provide terminal facts even without a returned `RunResult`. +For ordinary `Run` and coordinated turns, use `OnPrepared` to retain actual +request details for diagnostics. For cacheable operations, retain the already +available `PreparedExecution.Details()` before discarding or executing the handle. +That snapshot is available on a cache hit even though no execution or `OnPrepared` +callback occurs. Debug capture must not trigger another preparation or model call. +Native observations provide terminal facts even without a returned `RunResult`. `OnPrepared` is absent on preparation/admission failures and is not a completion record. Keep synchronous callbacks lightweight, compose capture/accounting observers, and synchronize shared state where operations overlap. `OnPrepared` @@ -1467,7 +1477,7 @@ retry tests or freeze complete help text and native debug snapshots. | Commands/loaders | One composed ConfigKit load supplies engine construction and native inspection, including explicit bundled resources, source evidence, and focused paths; no duplicate PromptKit load. Required operands, native alias, supplied-value distinctions, references, and run-wide overrides reach intended handlers/roles; help/version/completion need no config; diff selections remain distinct. | | Preflight/inspection | Same definitions across execution and inspection; missing statically selected resources fail with context; partial profiles alone pass; no provider calls, transcript, or credential requirement for offline preflight. | | Domain workflow | Representative extraction and normalization preserve source/reference identity, fixed handles, safe plan application, ordered validators, artifact schemas, and stable output ordering; one assessor checks initial/corrected candidates without an extra final pass; deterministic stages and LLM validators need no producer conversation loop. | -| Prompt/guidance contracts | Domain-specific initial/correction pairs use the same response schema and default profile; explicit profile/execution overrides reach both; operator correction-profile defaults work when not overridden; all applicable validators assess the same candidate despite earlier rejection; their actionable findings reach the correction template; module interpretation defects and unavailable assessments remain distinct. | +| Prompt/guidance contracts | Domain-specific initial/correction pairs use the same response schema and default profile; explicit profile/execution overrides reach both; operator correction-profile defaults work when not overridden; all applicable validators assess the same candidate despite earlier rejection and reassess each delivered revision, including an unchanged candidate; their actionable findings reach the correction template; module interpretation defects and unavailable assessments remain distinct. | | Retry controls | `max_additional_turns` resolves binding, pipeline, then application default `2`; explicit zero disables correction; acceptance stops early and the final allowed turn is assessed. Bundled declarations/application overrides leave structural repair unspecified, native operator customization remains effective, and tests do not freeze upstream default attempt counts. | | Non-LLM failure handling | Deterministic errors propagate without generic retries; rejected deterministic candidates reach terminal policy without regeneration; cancellation stops work. Producer/validator `retries` fields and the generic outer attempt loop are retired. | | Continuation/failures | Accumulated history, full revalidation and final-turn assessment; native failed validation with a nil Go error reaches terminal handling before domain interpretation or caching in producer, continuation, and validator paths. No blanket retries beyond native exhaustion; validator failure differs from rejection; no earlier-candidate recovery or cross-turn proposal union; cancellation stops without fallback work. | @@ -1479,7 +1489,7 @@ retry tests or freeze complete help text and native debug snapshots. | Output/outcomes | Preserved run-specific domain paths, lane filename mapping, and payload schemas; optional chunk-map/evidence-context publication remains. No index/manifest/reporting sidecars. Native text/JSON/YAML summary includes positive `fallback_count` and omits zero; count emitting producer operations, not attempts, validator failures, or affected derivatives. Quiet suppression, verbose stderr, debug path, and useful nonzero failures hold. Publication survives later capture/report failure without model reruns. | | Execution observations | One native invocation group enrolls actual initial/continuation/LLM validator executions once, with application scope; native totals preserve unknown-versus-zero usage without application accumulators or overlapping-group aggregation. Result-less failures remain observable; cache hits add no executions/history; native execution success remains distinct from acceptance. | | Debug controls/lifecycle | Default/CLI/config/explicit-empty paths and native mode ordering; quiet capture; allocation before engine/preflight; start-time relative `latest`, no completion retarget; disabled runs leave it untouched; capture failures preserve primary causes and publication facts. | -| Debug evidence | Available input/source/native snapshots, readable text, all visible turns, assessments, normalization and deterministic outputs, cache hits, and failed-run evidence; no invented provider transcript; ConfigKit secrets/native credentials omitted, no heuristic content scrubbing. | +| Debug evidence | Available input/source/native snapshots, readable text, all visible turns, assessments, normalization and deterministic outputs, cache hits, and failed-run evidence. Cache hits retain prepared details without an execution event, duplicate preparation, or model call; no invented provider transcript. ConfigKit secrets/native credentials omitted, no heuristic content scrubbing. | | End-to-end migration | Maintained minimal and complete D&D workflows, session routing, reference behavior, raw artifacts, reports, and migration examples work through the new executable. No postponed CommandKit phase. | Run focused checks during implementation and the repository-wide checks for the