From 15e7c1aed98b494301b6933f552b6db5972ad66f Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sat, 3 Oct 2026 18:08:09 +0000 Subject: [PATCH] Plan PromptKit configuration adoption --- docs/roadmap/promptkit.md | 333 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 333 insertions(+) create mode 100644 docs/roadmap/promptkit.md diff --git a/docs/roadmap/promptkit.md b/docs/roadmap/promptkit.md new file mode 100644 index 0000000..cddd6d3 --- /dev/null +++ b/docs/roadmap/promptkit.md @@ -0,0 +1,333 @@ +# PromptKit v0.12.0 Adoption + +## Status And Intent + +Proposed on 2026-10-03. No implementation is delivered by this document. +The recommendations under Open Decisions need confirmation before preparing a +decision-complete implementation plan. Release preparation remains separate. + +Adopt PromptKit's ConfigKit-native configuration boundary while preserving +Scriptorium's simple, request-independent execution model. As established in +[ADR 0004](../adr/0004-definition-boundary.md), valid prompt and profile +definitions for the selected PromptKit version should work without a narrower +Scriptorium field-level format. Library-only APIs are considered separately; +this upgrade does not turn Scriptorium into a workflow application. + +The maintainer's preference for native ConfigKit semantics and a clean cutover +informs the recommended source interface below. That interface is a proposed +product change, not something required merely to compile against PromptKit. + +## Reviewed Baseline And Upstream Target + +The reviewed Scriptorium source is commit +`bb5006517b6c27dcbe56074f1d97140bf0999c48`; the last release is v0.15.0. +The completed [ConfigKit adoption](configkit.md) selects ConfigKit +v0.5.0. PromptKit remains v0.11.2, with the OpenRouter and Rakestrawhome `/v2` +catalog modules at v2.20260926.0. + +Application configuration already uses one ConfigKit loading, inheritance, +resolution, validation, and inspection pipeline. Framework construction still +passes `prompt_dir` and `profile_dir` into the removed PromptKit directory +fields and registers application `backends` with `WithBackend`. Profile +inspection substitutes an empty prompt filesystem to avoid loading prompts. +Maintained prompt/profile files and inline test fixtures use the old embedded +`id`, `version`, and `base_profile` syntax. + +The target is the published PromptKit v0.12.0 release at full commit +`4b8169da286917fde7875425bb32b799b12c6af3`. The published tag and downloaded +module identify that same revision. Canonical references are: + +- [Release notes](https://gitea.maximumdirect.net/eric/promptkit/src/commit/4b8169da286917fde7875425bb32b799b12c6af3/docs/releases/v0.12.0.md). +- [Framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/commit/4b8169da286917fde7875425bb32b799b12c6af3/docs/formats.md). +- [Consumer and migration guide](https://gitea.maximumdirect.net/eric/promptkit/src/commit/4b8169da286917fde7875425bb32b799b12c6af3/docs/consumers/pkg-promptkit.md). +- [Configuration ownership decision](https://gitea.maximumdirect.net/eric/promptkit/src/commit/4b8169da286917fde7875425bb32b799b12c6af3/docs/adr/0001-configuration-boundary.md). + +PromptKit now also selects ConfigKit v0.5.0, so no further ConfigKit upgrade is +needed. Go 1.25.5 remains sufficient. Its catalog selections are: + +| Module | Version | Full source commit | +| --- | --- | --- | +| `promptkit-backend-openrouter/v3` | `v3.20260930.0` | `f17d208cb7ad1fb043cda2266cd3011d11b40aa0` | +| `promptkit-backend-rakestrawhome/v3` | `v3.20260930.0` | `993443d88c451901f4c917fa00c204fd3b54d28d` | + +This review examined upstream contracts and the affected Scriptorium adapters, +configuration, presentation, examples, and tests. It did not establish that the +unmodified application builds against the new, incompatible construction API. + +## Targeted End State + +### Framework Ownership And Source Assembly + +Scriptorium constructs engines through PromptKit's public root package and +`Config.Sources`. ConfigKit owns source composition, overlays, inheritance, +presence, null values, and resource origins. PromptKit owns final definition +validation, catalog contributions, backend completion, content capture, +preparation, execution, and capacity coordination. + +An operator-selected source may contain any combination of keyed `backends`, +`profiles`, and versioned `prompts`. Do not partition or filter documents by +directory name, filename, or a Scriptorium copy of the framework schema. +Do not independently preflight domain fields in lower-priority definitions +that the final effective configuration replaces. Syntax and composition +failures remain upstream loading failures. + +The recommended application interface, subject to Open Decision 1, is an +ordered `promptkit_sources` sequence. Each entry selects either a `file` or a +`directory`; each entry contributes one caller underlay, from lowest to +highest priority. A directory uses `promptkit.DirectoryInputs` and strictly +composes its recursively discovered YAML files. A file uses `configkit.File` +with declared imports enabled, so an entry document can compose fragments. +All caller groups follow PromptKit's maintained catalogs and owned defaults. +No separately exposed `Sources.Main` setting is needed for this interface. + +For example, the proposed application shape is: + +```yaml +promptkit_sources: + - directory: ./examples/prompts + - directory: ./examples/profiles + - file: ./deployment/promptkit.yml +schema_dir: ./examples/schemas +``` + +Directories retain the helper's default of disabled imports. Enabling imports +while also recursively discovering their YAML targets would include those +files twice in the same group, which ConfigKit rejects. Use a selected entry +file with imports for that layout; do not silently deduplicate, skip files, or +convert composition conflicts into precedence. + +Under this recommendation, `prompt_dir`, `profile_dir`, application `backends`, +and their directory flags are replaced, not retained as a second configuration +model. Framework backend definitions live with the framework sources rather +than passing through Scriptorium's scalar backend schema. A repeatable +`--promptkit-source PATH` selects a replacement ordered source list for +engine-using commands; each path selects a regular file or directory using the +same rules. Only an explicitly supplied flag overrides the configured list. + +Application source discovery, its imports and selected overlay, explicit CLI +override handling, and working-directory-relative application paths remain +unchanged. `--config-overlay` still selects an application overlay; it does not +implicitly select a PromptKit layer or profile. Framework priority is explicit +in the source list, with import/resource containment supplied by upstream for +each selected root. This is not a new global filesystem sandbox. + +### Complete Definition Support + +The target accepts the keyed v0.12.0 formats without compatibility rewriting: +profile and backend IDs are mapping keys, prompt IDs contain keyed string +versions, and `extends` replaces `base_profile`. Literal IDs and references +retain their exact spelling, including punctuation and meaningful whitespace; +filenames do not establish identities. + +All supported prompt/profile fields continue to reach PromptKit, including +inputs, message roles and cache metadata, inline and file content, session +templates, output validation and repair, backend/endpoint selection, +credentials by environment name, generation controls, and provider parameters. +The upstream format reference, not this list, owns the complete contract. + +Presence must survive the boundary: omission, explicit zero/false/blank, and +null are not interchangeable. Inheritance and layers recursively combine maps; +an empty map does not clear inherited entries. Backend defaults apply after +effective profile inheritance. Changing a model or backend no longer drops +unrelated explicitly inherited settings. Profile null suppression and explicit +zero controls must affect outbound requests as specified upstream. + +Partial profiles remain valid when request overrides can complete them. +Scriptorium must not introduce a mandatory model, credential, default profile, +or directory prerequisite that prevents such a request. With the recommended +source interface, engine construction may have no prompt collection; missing +request selections fail through PromptKit's ordinary public errors rather than +a required `prompt_dir` check. Source selection does not imply provider access +or credential availability. + +### Captured Definitions And Runtime Lifecycle + +Engine construction validates every final effective definition, including +unused entries, and captures message-file content and static template syntax. +Invalid unused definitions or unreadable message resources may therefore fail +startup or inspection before a request selects them. An overwritten domain +value is not independently rejected merely because it appears in history. + +ConfigKit's structural directives now also apply to framework input mappings, +including dynamic provider maps. Document the reserved `extends` key and the +difference from literal runtime provider parameters; do not invent a quoting +escape or a second inheritance resolver. + +Content files use their effective field's source context, including imported +and inherited content. Retain those contexts by passing actual upstream +inputs; do not flatten files into synthetic mappings that lose origins. +Application paths remain working-directory-relative, whereas framework message +resources follow PromptKit's source-relative contract. Schema and request +artifact reads remain preparation-time resources; credentials and provider +operations retain their documented runtime boundaries. + +A server retains one constructed engine across its requests, with shared +backend capacity and queue accounting. Definition/message edits require a new +engine; for this scope, restarting the server is the supported way to obtain +one. CLI invocations naturally construct fresh engines. No file watcher, +reload endpoint, per-request server reconstruction, or durable state is added. +This is not a transactional snapshot of all external files or credentials. + +### Inspection, Requests, And Presentation + +`inspect prompt` and `inspect profile` use the same selected configuration +snapshot as execution. Remove the empty-prompt-filesystem substitution. +Profile inspection still does not need request inputs, schema reads, model +calls, or credential values, but constructing its engine now validates and +captures all selected definitions, including prompts. A partial target may be +inspected without imposing preparation's completeness requirements. + +`config validate`, `show`, `sources`, and `explain` remain application +configuration operations without constructing an engine. Under the proposed +source cutover, they show source selections rather than the contents of +PromptKit's separately loaded definitions. They are not framework validation. +Full framework provenance inspection is addressed in Open Decision 2. + +Keep CLI/HTTP request independence, presence-aware execution overrides, +generation/whole-run timeout separation, transient retries, execution reports, +artifact policy, and public error mapping. Preserve existing serialized results +unless an upstream change actually requires an application contract change. +Prepared/profile numeric values alone are not proof that a zero control was +sent; verify upstream presence through generation or recorded outbound requests. +The dependency change alone does not require a new presence DTO. Continue +mapping errors by public identity rather than message text. + +Environment-variable names remain visible; resolved API keys are never +printed or serialized. Ordinary `extra_params` retain their selected visibility +where those values are presented. Moving definitions out of application +settings changes what `config` can inspect, not their sensitivity classification. +Do not silently adopt broader redaction rules for existing profile/prepared +inspection. + +## Work Needed From The Baseline + +- **Dependencies and attribution:** select released PromptKit v0.12.0, tidy the + module graph, and use its selected `/v3` catalogs without direct application + imports. Remove obsolete `/v2` modules and `gopkg.in/yaml.v3` if no remaining + imports require them. Update PromptKit/catalog component entries and exact + source, COPYRIGHT, and LICENSE links in `docs/oss/attribution.md`. The reviewed + grants remain LGPL-3.0-or-later for PromptKit and MIT for the catalogs; reuse + the applicable local license texts. Preserve unrelated attribution entries + and historical release documents. +- **Application settings and CLI:** implement the selected source interface + through `internal/config` and shared CLI override mapping. Under the native + recommendation, retire the backend document/runtime conversion and old + directory fields, add ordered source selection, and update command help and + prerequisites. Application validation checks its schema and selector shape; + framework acquisition and domain validation belong to engine construction. +- **Shared engine wiring:** replace removed fields/options with upstream + inputs and underlays, retain `SchemaDir`, `TransientRetries`, and runtime + collaborator injection, and use the same construction path for execution, + rendering, definition inspection, and serving. Preserve identifiable + PromptKit and ConfigKit causes and safe diagnostics. Remove obsolete + prompt-filesystem inspection scaffolding. +- **Definitions and fixtures:** migrate every maintained prompt/profile file + under `examples/` and `testdata/`, and inline definitions and construction + helpers in CLI, HTTP, and formatting tests. Replace old inheritance syntax + and move backend examples into framework definitions if the native cutover + is selected. Review old explicit zeros for their changed meaning instead of + preserving obsolete expectations mechanically. Keep useful feature-adoption + coverage; filenames need not be renamed merely to reduce historical naming. +- **Focused adoption coverage:** use real ConfigKit/PromptKit boundaries for + representative source order, same-group conflicts, file imports, combined + documents, inherited source-relative content, exact IDs/version selection, + null suppression, explicit zero/false, and partial-profile completion. + Verify eager failures for unused final definitions and unchanged captured + message content after file edits on an existing engine. Retain shared-server + capacity, retry, cancellation, restricted artifact, output, and error tests. + Own integration risks locally without recreating upstream's exhaustive parser + or inheritance matrices; use offline clients or `httptest` for generation. +- **Current documentation and migration guidance:** update owning contracts + in `docs/config.md`, `docs/cli.md`, and `docs/operations.md`, affected internal + docs, `docs/development.md`, architecture version/ownership references, + README, examples, and applicable consumer guidance. Link to v0.12.0 framework + contracts rather than copying their specification. Correct statements that + ConfigKit does not participate in framework loading. Explain the selected + source migration and engine-rebuild requirement, and preserve accepted ADR + decision content. Update HTTP docs only where observable behavior changes. + +## Compatibility And Release Impact + +This is an intentional breaking adoption, not a patch-only dependency refresh. +Users must migrate definitions, inheritance syntax, source assembly, and +assumptions about numeric presence, recursive maps, eager validation, and file +reload. Under the recommended native interface they must also migrate the +application source selectors and move application backend definitions into +PromptKit source files. Provide small before/after examples and runnable +offline migration checks when implementation lands; no automatic legacy parser +or catalog alias layer is proposed. + +PromptKit's maintained profile IDs remain supported, not frozen to fixed model +settings. Review used external catalog IDs against the selected catalogs; do +not promise live provider availability or preserve every volatile inventory ID. +Hashes remain opaque equality values, not cross-version identity guarantees. + +Under the [release policy](../policy/release.md), the eventual release must be +a new pre-1.0 minor version. Do not select or publish a release in this work +set. The previously completed ConfigKit work may share that release; final +version, notes, and roadmap retirement belong to release preparation. + +## Completion Criteria And Scope Limits + +Adoption is complete when the released dependency graph is selected, the +approved source interface reaches the canonical framework boundary without +losing definition semantics, maintained examples and fixtures use v0.12.0, +and CLI/HTTP workflows and inspection meet the target above. Run the existing +development checks, affected offline examples, and link/diff checks, with +focused new integration tests for the actual migration risks. + +This work does not add conversations, appended/follow-up messages, +prepared-handle coordination, semantic retry workflows, embedded configuration +bundles, framework internals, or application-owned framework validation. +Library availability alone does not justify additional commands, endpoints, +resource controls, recurring upstream probes, or release gates. Deferred +interfaces are scope choices for this adoption, not permanent prohibitions. + +## Open Decisions + +### 1. Native Source Cutover Or Retained Application Selectors? + +**Recommendation:** adopt the ordered `promptkit_sources` file/directory +interface above and move backend definitions out of application settings. +This exposes combined definitions, explicit priority, imports, and full +presence/null semantics through their canonical owner. It removes duplicated +backend representation and avoids having to translate absent scalar fields +into new presence-aware overlays. The cost is a deliberate application schema +and CLI migration alongside the unavoidable framework format migration. + +**Viable alternative:** retain `prompt_dir`, `profile_dir`, their flags, and +application `backends`. Assemble a prompt-directory underlay, then a +profile-directory underlay, then application backend overrides. Allow combined +definition documents in either directory. This reduces migration and keeps +familiar directory organization, but requires explicit backend presence/null +mapping to avoid replacing catalog values with incidental Go zeros, and a +separate file-selector design if framework imports are to be usable without +recursive-discovery repetition. It also leaves two backend configuration +surfaces and less flexible source ordering. Do not implement this alternative +as an old parser or a filter on PromptKit definition fields. + +### 2. Expose Full Framework Configuration Inspection In This Adoption? + +**Recommendation:** defer a new framework `validate/show/sources/explain` +command family. Keep existing application and per-definition inspection, and +document their distinct scopes. PromptKit's new `InspectConfiguration` uses +fixed redaction of credential-shaped dynamic provider keys, including `token` +and `password`. That is broader than the maintainer's established API-key-only +redaction choice and cannot be disabled through the released public view. +Deferral avoids quietly changing that choice or reconstructing private +configuration to bypass upstream writers; it does not limit execution of valid +definitions. + +**Viable alternative:** add an explicit `inspect configuration` command family +using the captured native view, with approval for upstream redaction in this +new interface only. Provenance and history would make layered configuration +easier to diagnose, and native writers minimize maintenance. The drawback is +different provider-parameter visibility between the new view and existing +inspection; the exception must be explicit, not applied to other commands. + +**Another viable alternative:** first obtain an upstream release with a public +consumer-selectable inspection sensitivity policy, then expose native +framework reports with API-key-only redaction. This preserves the selected +policy and source ownership, but requires separate upstream work and delays +inspection. Do not assume that API exists in v0.12.0 or make core adoption +depend on an unpublished interface.