diff --git a/docs/roadmap/promptkit.md b/docs/roadmap/promptkit.md index cddd6d3..7a2022d 100644 --- a/docs/roadmap/promptkit.md +++ b/docs/roadmap/promptkit.md @@ -2,9 +2,10 @@ ## Status And Intent -Proposed on 2026-10-03. No implementation is delivered by this document. -The recommendations under Open Decisions need confirmation before preparing a -decision-complete implementation plan. Release preparation remains separate. +Drafted on 2026-10-03; source-selection and framework-inspection decisions +confirmed by the maintainer on 2026-10-04. No implementation is delivered by +this document. No significant decisions remain open within this scope; +implementation sequencing and release preparation remain separate work. Adopt PromptKit's ConfigKit-native configuration boundary while preserving Scriptorium's simple, request-independent execution model. As established in @@ -13,9 +14,10 @@ definitions for the selected PromptKit version should work without a narrower Scriptorium field-level format. Library-only APIs are considered separately; this upgrade does not turn Scriptorium into a workflow application. -The maintainer's preference for native ConfigKit semantics and a clean cutover -informs the recommended source interface below. That interface is a proposed -product change, not something required merely to compile against PromptKit. +The maintainer selected a native source cutover and full framework configuration +inspection. The latter uses upstream's native redaction in this new interface +only. These are deliberate product choices, not requirements inferred merely +from new library APIs. ## Reviewed Baseline And Upstream Target @@ -71,16 +73,16 @@ Do not independently preflight domain fields in lower-priority definitions that the final effective configuration replaces. Syntax and composition failures remain upstream loading failures. -The recommended application interface, subject to Open Decision 1, is an -ordered `promptkit_sources` sequence. Each entry selects either a `file` or a -`directory`; each entry contributes one caller underlay, from lowest to -highest priority. A directory uses `promptkit.DirectoryInputs` and strictly -composes its recursively discovered YAML files. A file uses `configkit.File` +The selected application interface is an ordered `promptkit_sources` sequence. +Each entry selects either a `file` or a `directory` and contributes one caller +underlay, from lowest to highest priority. A directory uses +`promptkit.DirectoryInputs` and strictly composes its recursively discovered +YAML files. A file uses `configkit.File` with declared imports enabled, so an entry document can compose fragments. All caller groups follow PromptKit's maintained catalogs and owned defaults. No separately exposed `Sources.Main` setting is needed for this interface. -For example, the proposed application shape is: +For example, the target application shape is: ```yaml promptkit_sources: @@ -96,7 +98,7 @@ files twice in the same group, which ConfigKit rejects. Use a selected entry file with imports for that layout; do not silently deduplicate, skip files, or convert composition conflicts into precedence. -Under this recommendation, `prompt_dir`, `profile_dir`, application `backends`, +Under this decision, `prompt_dir`, `profile_dir`, application `backends`, and their directory flags are replaced, not retained as a second configuration model. Framework backend definitions live with the framework sources rather than passing through Scriptorium's scalar backend schema. A repeatable @@ -134,7 +136,7 @@ zero controls must affect outbound requests as specified upstream. Partial profiles remain valid when request overrides can complete them. Scriptorium must not introduce a mandatory model, credential, default profile, -or directory prerequisite that prevents such a request. With the recommended +or directory prerequisite that prevents such a request. With the selected source interface, engine construction may have no prompt collection; missing request selections fail through PromptKit's ordinary public errors rather than a required `prompt_dir` check. Source selection does not imply provider access @@ -168,7 +170,7 @@ one. CLI invocations naturally construct fresh engines. No file watcher, reload endpoint, per-request server reconstruction, or durable state is added. This is not a transactional snapshot of all external files or credentials. -### Inspection, Requests, And Presentation +### Definition And Application Inspection `inspect prompt` and `inspect profile` use the same selected configuration snapshot as execution. Remove the empty-prompt-filesystem substitution. @@ -178,10 +180,52 @@ captures all selected definitions, including prompts. A partial target may be inspected without imposing preparation's completeness requirements. `config validate`, `show`, `sources`, and `explain` remain application -configuration operations without constructing an engine. Under the proposed +configuration operations without constructing an engine. Under the selected source cutover, they show source selections rather than the contents of PromptKit's separately loaded definitions. They are not framework validation. -Full framework provenance inspection is addressed in Open Decision 2. +The separate framework inspection family below supplies framework validation +and captured provenance. + +### Captured Framework Configuration Inspection + +Add the CLI command family `inspect configuration` with `validate`, `show`, +`sources`, and `explain` operations. It uses the same application settings, +source selections, and engine construction as other engine-using commands, +without requiring a prompt/profile selection or request inputs. Each invocation +constructs one engine; reporting uses that engine's captured public view rather +than loading definitions again. + +- `validate` succeeds when the engine constructs successfully and produces no + success report. It validates final definitions and captures message resources, + not provider access, credential availability, or request-specific preparation. +- `show` writes the captured effective YAML through + `ConfigurationInspection.WriteEffective`. +- `sources` presents structured JSON source records from + `ConfigurationInspection.Sources`, including the source kinds, layers, and + underlay positions needed to distinguish maintained and operator inputs. +- `explain --path JSON_POINTER` uses the native explanation view and writer + for effective values and history. Root and literal mapping-key paths follow + the same JSON Pointer conventions as application `config explain`; IDs with + slashes or punctuation must not become dotted-path lookups. + +Use PromptKit's public `InspectConfiguration` and its native writers and +records. Surface applicable load diagnostics safely on stderr; successful +reports go to stdout. Inspection does not resolve credential values, contact +a provider, or watch/reopen sources after construction. Captured effective +configuration is distinct from backend-completed `inspect profile` output and +request-specific prepared output; none substitutes for the others. + +The maintainer explicitly accepts upstream's credential-shaped provider-key +redaction for this new family, including effective and historical values. +Fields such as `token` and `password` can therefore be redacted here while +remaining visible in existing profile/prepared views. This is not blanket +redaction of `extra_params`, a change to parameters sent to providers, or a +change to existing inspection interfaces. Environment-reference names and +ordinary unclassified content can remain visible. Reports may contain prompt +text, provider data, paths, and identifiers; native redaction is not a promise +that the output is safe to disclose. + +### Requests, Presentation, And Disclosure Keep CLI/HTTP request independence, presence-aware execution overrides, generation/whole-run timeout separation, transient retries, execution reports, @@ -210,10 +254,10 @@ inspection. the applicable local license texts. Preserve unrelated attribution entries and historical release documents. - **Application settings and CLI:** implement the selected source interface - through `internal/config` and shared CLI override mapping. Under the native - recommendation, retire the backend document/runtime conversion and old - directory fields, add ordered source selection, and update command help and - prerequisites. Application validation checks its schema and selector shape; + through `internal/config` and shared CLI override mapping. Retire the backend + document/runtime conversion and old directory fields, add ordered source + selection, and update command help and prerequisites. Application validation + checks its schema and selector shape; framework acquisition and domain validation belong to engine construction. - **Shared engine wiring:** replace removed fields/options with upstream inputs and underlays, retain `SchemaDir`, `TransientRetries`, and runtime @@ -221,13 +265,19 @@ inspection. rendering, definition inspection, and serving. Preserve identifiable PromptKit and ConfigKit causes and safe diagnostics. Remove obsolete prompt-filesystem inspection scaffolding. +- **Captured framework inspection:** add the `inspect configuration` family + with `validate`, `show`, `sources`, and `explain` using the shared engine path + and its public captured inspection view. Reuse native YAML/explanation writers, + preserve useful source-layer information in JSON presentation, and keep application + `config` independent of framework loading. Apply the approved native + redaction exception only to this new family. - **Definitions and fixtures:** migrate every maintained prompt/profile file under `examples/` and `testdata/`, and inline definitions and construction helpers in CLI, HTTP, and formatting tests. Replace old inheritance syntax - and move backend examples into framework definitions if the native cutover - is selected. Review old explicit zeros for their changed meaning instead of - preserving obsolete expectations mechanically. Keep useful feature-adoption - coverage; filenames need not be renamed merely to reduce historical naming. + and move backend examples into framework definitions. Review old explicit + zeros for their changed meaning instead of preserving obsolete expectations + mechanically. Keep useful feature-adoption coverage; filenames need not be + renamed merely to reduce historical naming. - **Focused adoption coverage:** use real ConfigKit/PromptKit boundaries for representative source order, same-group conflicts, file imports, combined documents, inherited source-relative content, exact IDs/version selection, @@ -235,6 +285,11 @@ inspection. Verify eager failures for unused final definitions and unchanged captured message content after file edits on an existing engine. Retain shared-server capacity, retry, cancellation, restricted artifact, output, and error tests. + Cover framework inspection's effective YAML, source layers, literal-key + explanations, diagnostics, and effective/historical native redaction without + provider calls. Protect the distinction between application validation and + framework validation, and unchanged visibility in existing profile/prepared + inspection. Avoid snapshots of incidental writer prose. Own integration risks locally without recreating upstream's exhaustive parser or inheritance matrices; use offline clients or `httptest` for generation. - **Current documentation and migration guidance:** update owning contracts @@ -243,7 +298,8 @@ inspection. README, examples, and applicable consumer guidance. Link to v0.12.0 framework contracts rather than copying their specification. Correct statements that ConfigKit does not participate in framework loading. Explain the selected - source migration and engine-rebuild requirement, and preserve accepted ADR + source migration, framework-inspection command family and scoped redaction + exception, and engine-rebuild requirement. Preserve accepted ADR decision content. Update HTTP docs only where observable behavior changes. ## Compatibility And Release Impact @@ -251,7 +307,7 @@ inspection. This is an intentional breaking adoption, not a patch-only dependency refresh. Users must migrate definitions, inheritance syntax, source assembly, and assumptions about numeric presence, recursive maps, eager validation, and file -reload. Under the recommended native interface they must also migrate the +reload. Under the selected native interface they must also migrate the application source selectors and move application backend definitions into PromptKit source files. Provide small before/after examples and runnable offline migration checks when implementation lands; no automatic legacy parser @@ -272,9 +328,11 @@ version, notes, and roadmap retirement belong to release preparation. Adoption is complete when the released dependency graph is selected, the approved source interface reaches the canonical framework boundary without losing definition semantics, maintained examples and fixtures use v0.12.0, -and CLI/HTTP workflows and inspection meet the target above. Run the existing -development checks, affected offline examples, and link/diff checks, with -focused new integration tests for the actual migration risks. +and CLI/HTTP workflows and inspection meet the target above. The new framework +inspection family must use captured native reports with its explicitly scoped +redaction behavior, without changing existing inspection disclosure. Run the +existing development checks, affected offline examples, and link/diff checks, +with focused new integration tests for the actual migration risks. This work does not add conversations, appended/follow-up messages, prepared-handle coordination, semantic retry workflows, embedded configuration @@ -283,51 +341,32 @@ Library availability alone does not justify additional commands, endpoints, resource controls, recurring upstream probes, or release gates. Deferred interfaces are scope choices for this adoption, not permanent prohibitions. -## Open Decisions +## Confirmed Decisions -### 1. Native Source Cutover Or Retained Application Selectors? +### Native Source Cutover -**Recommendation:** adopt the ordered `promptkit_sources` file/directory -interface above and move backend definitions out of application settings. -This exposes combined definitions, explicit priority, imports, and full -presence/null semantics through their canonical owner. It removes duplicated -backend representation and avoids having to translate absent scalar fields -into new presence-aware overlays. The cost is a deliberate application schema -and CLI migration alongside the unavoidable framework format migration. +The maintainer accepted the ordered `promptkit_sources` file/directory interface +and moving backend definitions out of application settings. This exposes +combined definitions, explicit priority, imports, and full presence/null +semantics through their canonical owner. It removes duplicated backend +representation and avoids translating absent scalar fields into presence-aware +overlays. The deliberate application schema and CLI migration accompanies the +unavoidable framework format migration. -**Viable alternative:** retain `prompt_dir`, `profile_dir`, their flags, and -application `backends`. Assemble a prompt-directory underlay, then a -profile-directory underlay, then application backend overrides. Allow combined -definition documents in either directory. This reduces migration and keeps -familiar directory organization, but requires explicit backend presence/null -mapping to avoid replacing catalog values with incidental Go zeros, and a -separate file-selector design if framework imports are to be usable without -recursive-discovery repetition. It also leaves two backend configuration -surfaces and less flexible source ordering. Do not implement this alternative -as an old parser or a filter on PromptKit definition fields. +Retaining the old directories and application backend surface was considered +but not selected: it would reduce migration while retaining duplicate backend +configuration, presence/null translation, and a less flexible source interface. -### 2. Expose Full Framework Configuration Inspection In This Adoption? +### Native Framework Inspection With A Scoped Redaction Exception -**Recommendation:** defer a new framework `validate/show/sources/explain` -command family. Keep existing application and per-definition inspection, and -document their distinct scopes. PromptKit's new `InspectConfiguration` uses -fixed redaction of credential-shaped dynamic provider keys, including `token` -and `password`. That is broader than the maintainer's established API-key-only -redaction choice and cannot be disabled through the released public view. -Deferral avoids quietly changing that choice or reconstructing private -configuration to bypass upstream writers; it does not limit execution of valid -definitions. +The maintainer selected `inspect configuration` using the captured native view, +including upstream redaction in this new interface only, rather than deferring +the feature. Provenance and history make layered configuration easier to +diagnose, and native writers avoid maintaining a parallel representation. -**Viable alternative:** add an explicit `inspect configuration` command family -using the captured native view, with approval for upstream redaction in this -new interface only. Provenance and history would make layered configuration -easier to diagnose, and native writers minimize maintenance. The drawback is -different provider-parameter visibility between the new view and existing -inspection; the exception must be explicit, not applied to other commands. - -**Another viable alternative:** first obtain an upstream release with a public -consumer-selectable inspection sensitivity policy, then expose native -framework reports with API-key-only redaction. This preserves the selected -policy and source ownership, but requires separate upstream work and delays -inspection. Do not assume that API exists in v0.12.0 or make core adoption -depend on an unpublished interface. +The accepted tradeoff is different provider-parameter visibility between this +new view and existing profile/prepared inspection. Existing API-key-only +redaction choices remain otherwise unchanged. Deferral or an upstream +consumer-selectable sensitivity API could avoid that difference, but neither is +a prerequisite for this adoption. Do not reconstruct private configuration to +bypass native writers or depend on an unpublished upstream interface.