diff --git a/docs/roadmap/promptkit.md b/docs/roadmap/promptkit.md index 9e8d3700..5187c8cf 100644 --- a/docs/roadmap/promptkit.md +++ b/docs/roadmap/promptkit.md @@ -148,8 +148,8 @@ separate known-dependency checks. Exact argument grouping and helper signatures can be refined against representative extraction and normalization during implementation; ownership and placement are settled. U4 selects prompt roles and guidance ownership below; U5 selects retry controls and the correction budget. -U6 selects strict/permissive handling and module-owned fallbacks below; its default, -configuration scope, and concrete module fallback choices remain open. +U6 selects strict-by-default handling and module-owned fallbacks below; +configuration scope and concrete module fallback choices remain open. Implementation is pending. Retain the fixed `input -> chunk -> extract -> merge -> normalize -> output` @@ -471,8 +471,8 @@ apply U6's strict/permissive handling when no acceptable result is obtained. Ownership, response contracts, and default profile behavior are resolved. Refine exact template prose, correction IDs, and necessary operation-specific inputs while implementing representative workflows. Implementation is pending; U5 selects -semantic budgets and U6 selects terminal handling, with defaults, configuration -scope, and concrete module fallback choices still open. +semantic budgets and U6 selects strict-by-default terminal handling, with +configuration scope and concrete module fallback choices still open. #### Execution And Assessment Flow @@ -590,9 +590,9 @@ outer attempt loop. Migration guidance maps intent: a producer allowance intende for semantic correction becomes `max_additional_turns`; native transport controls and prompt-owned structural customization cover their own concerns. Remove non-LLM retry settings without a replacement generic allowance. Do not promise -equivalent total provider-attempt counts. U5 is resolved; U6 selects strict/permissive -handling, with defaults, configuration scope, and concrete module fallback choices -still open. Implementation is pending. +equivalent total provider-attempt counts. U5 is resolved; U6 selects strict-by-default +handling, with configuration scope and concrete module fallback choices still +open. Implementation is pending. #### Terminal Outcomes And Module Fallbacks (U6 Direction Resolved) @@ -607,8 +607,11 @@ The pipeline selects the mode; the module owns the fallback algorithm. Replace the failure-category policy object and separate `fail_run`, `reject_output`, and `warn_continue` choices with this selection. An empty artifact is a supported fallback where the domain permits it; no separate omit-output action is needed. -The default mode, configuration spelling/scope/precedence, and concrete module -fallback definitions remain open. Do not infer them from existing policy fields. +The application default is strict for every role, including normalization. +Modules do not declare their own defaults. Permissive handling requires an +explicit configuration selection; normalization has no automatic exception. +Configuration spelling/scope/precedence and concrete module fallback definitions +remain open. Do not infer them from existing policy fields. Attempt application-directed correction when appropriate within the selected budget before terminal handling. Neither mode extends PromptKit's native retry @@ -653,8 +656,8 @@ completion report, even when diagnostic verbosity hides individual warnings. Use the same application outcome for reporting and optional debug detail. Keep conversation state in process, outside ordinary artifact/cache payloads; explicitly requested detailed debug is separate. Implementation is pending, and -U6 remains open only for the default mode, configuration contract, and concrete -module fallback choices. +U6 remains open only for the configuration contract and concrete module fallback +choices. ### Admission And Pipeline Workers @@ -980,7 +983,7 @@ reopen the architecture. IDs retain continuity with the earlier roadmap. | ID | Remaining choice or dependency | Resolve for | | --- | --- | --- | | U1 | Review the message-content hash API/release documented upstream in v0.24.0, including its ConfigKit dependency; verify the final compatible dependency set against the selected baseline. | Track in Phase 1; required to finish cache integration in Phase 5. | -| U6 | Default strict/permissive mode; configuration spelling, scope, and override precedence; concrete module fallbacks, particularly whole-transcript versus sliced chunking. The two-mode model, module ownership, and final-response-only handling are settled. | Phases 3–4. | +| U6 | Configuration spelling, scope, and override precedence; concrete module fallbacks, particularly whole-transcript versus sliced chunking. Strict is the application default for every role; the two-mode model, module ownership, and final-response-only handling are settled. | Phases 3–4. | | U7 | Independent worker defaults; application-supplied native backend/queue limits, if any; retired `total_llm` migration and admission-failure handling by role. | Phase 4. | | U8 | Cache directory configuration and reuse/bypass/refresh controls (names, defaults, scope); old-checkpoint treatment and unusable-entry diagnostics. Hash inputs, lookup timing, accepted value, and ownership are settled. | Design with Phase 3; finish in Phase 5. | | U9 | Observer/group enrollment and application scope; concrete conditional report fields and debug filenames/serialization within the agreed contracts; publication-layout/consumer migration details preserving intrinsic artifact schemas. | Implementation details in Phases 3–5; no new receipt/manifest inventory or capture-policy decision required. | @@ -1017,8 +1020,10 @@ with its specific operation. Implement in Phases 3–4; U6 selects terminal hand replace failure-category policies. Fallbacks satisfy artifact/downstream invariants without asserting ordinary semantic acceptance; use only the final response and fixed originals. Fallback and affected downstream work remain outside accepted- -artifact caching, and completion reports disclose fallback use. Default mode, -configuration contract, and concrete module fallback choices remain open above. +artifact caching, and completion reports disclose fallback use. Strict is the +application default, including normalization; there are no module-declared +defaults. Configuration contract and concrete module fallback choices remain +open above. **U10 resolved:** CommandKit versions, capabilities, commands, loaders/preflight, output, debug policy, and coordinated cutover sequencing are selected above. @@ -1069,8 +1074,8 @@ without credentials or synthetic requests. `config diff` handles two selections. ### Phase 3: Prove A Representative Native Pipeline - Implement the resolved U3–U5 collaboration and retry policy and U6's - strict/permissive model. Settle its default mode, configuration contract, and - representative module fallbacks. + strict-by-default model. Settle its configuration contract and representative + module fallbacks. Connect module request/interpretation functions to the pipeline assessment callback through the shared producer helper; refine concrete signatures here. Implement domain-specific initial/correction prompts using the same schema and @@ -1183,7 +1188,7 @@ retry tests or freeze complete help text and native debug snapshots. | Retry controls | `max_additional_turns` resolves binding, pipeline, then application default `2`; explicit zero disables correction; acceptance stops early and the final allowed turn is assessed. Bundled declarations/application overrides leave structural repair unspecified, native operator customization remains effective, and tests do not freeze upstream default attempt counts. | | Non-LLM failure handling | Deterministic errors propagate without generic retries; rejected deterministic candidates reach terminal policy without regeneration; cancellation stops work. Producer/validator `retries` fields and the generic outer attempt loop are retired. | | Continuation/failures | Accumulated history, full revalidation and final-turn assessment; no blanket retries beyond native exhaustion; validator failure differs from rejection; no earlier-candidate recovery or cross-turn proposal union; cancellation stops without fallback work. | -| Strict/permissive outcomes | Strict failure when no acceptable result is obtained; permissive module fallback with required artifact/downstream invariants, without rerunning the ordinary full validation chain or claiming normal acceptance. Fallback construction failure fails execution; successful fallback use is disclosed in human/machine completion reports. | +| Strict/permissive outcomes | Strict is the default for every role, including normalization, without module-specific defaults. Strict failure when no acceptable result is obtained; explicitly selected permissive handling produces module fallback with required artifact/downstream invariants, without rerunning the ordinary full validation chain or claiming normal acceptance. Fallback construction failure fails execution; successful fallback use is disclosed in human/machine completion reports. | | Admission/workers | Shared backend pools across roles, independent backends, bounded pipeline work, nested validators, queue exhaustion, and cancellation without leaked capacity or a global ceiling. | | Accepted-artifact cache | Downstream recovery; initial-message keys; corrected final artifact stored late under the initial key; changed messages miss; settings/ID-only changes can hit; no model/revalidation on hits; unusable entries miss. Failed/rejected/incomplete work is not stored; fallbacks and affected downstream work skip lookup/storage while unaffected accepted predecessors remain reusable; no required history. | | Cache migration | Selected reuse/bypass/refresh and old-record treatment work; invocation configuration no longer blocks lookup; canonical chunk caching retains its distinct policy. | @@ -1222,8 +1227,8 @@ Provide practical release guidance for operators and subprocess callers: Explain retirement of all producer/validator `retries` fields and generic non-LLM retries, upstream-owned structural defaults, removal of application repair controls and bundled counts, native operator customization, accumulated-history - costs, strict/permissive selection and precedence, and each module's fallback - behavior and downstream guarantees. + costs, the universal strict default, explicit permissive selection and precedence, + and each module's fallback behavior and downstream guarantees. - Accepted-artifact caching, message-based misses, settings-only reuse, refresh/ bypass for changed interpretation or validation, fallback/affected-work exclusion, and old-checkpoint handling.