diff --git a/docs/roadmap/promptkit-v0.23.2-adoption.md b/docs/roadmap/promptkit-v0.23.2-adoption.md new file mode 100644 index 0000000..fee303b --- /dev/null +++ b/docs/roadmap/promptkit-v0.23.2-adoption.md @@ -0,0 +1,180 @@ +# PromptKit v0.23.2 Adoption + +## Status And Scope + +Accepted direction; not yet implemented. Upgrade the existing Scriptorium +product in one native cutover. Use the released PromptKit and ConfigKit APIs +for framework and configuration mechanisms that they already own. Provide +migration guidance, not a second application schema, a legacy definition +reader, or compatibility aliases. + +The [definition boundary](../adr/0004-definition-boundary.md) and +[configuration boundary](../adr/0005-configkit-boundary.md) remain in effect. +The existing CLI and HTTP surfaces should be able to use valid definitions for +the selected PromptKit release without copying its schema or maintaining a +narrower application-specific subset. This roadmap covers the changes to +existing surfaces needed for the migration. New CLI capabilities, including +[Narratio's requirements inspection](narratio-feature-request.md), concrete +request validation, inventory, backend inspection, configuration diffs, and +smoke diagnostics, are deferred to a separate roadmap. It does not add HTTP +operations for those capabilities or introduce continuation, coordination, +prepared-handle workflows, or prepared-content capture. + +## Baseline And Target + +The reviewed Scriptorium baseline is +`1b1d967fc78e9c514a478b27debf1fadbc772283`. It uses +`gitea.maximumdirect.net/eric/promptkit` v0.12.0. Application configuration +already uses native ConfigKit v1.2.0, but framework inputs, inspection +records, and diagnostics still use old-path ConfigKit v0.5.0. The old +PromptKit module also selects two external catalog modules and go-yaml +v4.0.0-rc.6. Prompt versions are selectors in both executable interfaces and +appear in prepared, inspection, and execution output. The completed +[v0.12.0 roadmap](promptkit.md) records the prior migration. + +The target is `go.maximumdirect.net/promptkit` v0.23.2, commit +`382ccfb14b9630fd8f48c8364dbe2b6159637d71`, and +`go.maximumdirect.net/configkit` v1.2.1, commit +`51ef3144158db1a09fabdbd1c7ae59908c84923b`, with stable go-yaml v4.0.0. +The existing Go 1.27.0 consumer minimum and Go 1.27.4 release toolchain +already satisfy this selection. + +- [PromptKit release notes](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/382ccfb14b9630fd8f48c8364dbe2b6159637d71/docs/releases/v0.23.2.md) +- [PromptKit migration guide](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/382ccfb14b9630fd8f48c8364dbe2b6159637d71/docs/consumers/configuration-migration.md) +- [PromptKit framework formats](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/382ccfb14b9630fd8f48c8364dbe2b6159637d71/docs/formats.md) +- [PromptKit consumer API guide](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/382ccfb14b9630fd8f48c8364dbe2b6159637d71/docs/consumers/pkg-promptkit.md) +- [ConfigKit v1.2.1 release notes](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/releases/v1.2.1.md) + +## Targeted End State + +### One Configuration Mechanism, Separate Schemas + +ConfigKit resolves the application document; Scriptorium owns its schema, +defaults, domain rules, source selection, and override mapping. The app uses +one native schema and decoder. Existing fields can remain where they still +express the right application setting, without a separate legacy mode. +PromptKit loads its framework sources separately using the same released +ConfigKit types. Application-only configuration inspection still does not +construct a PromptKit engine or require framework resources. + +Scriptorium selects ordered framework files and directories and supplies +them through PromptKit's supported public APIs. The application config file +is not itself a framework source. Preserve the application selectors, +explicit override behavior, and source grouping where they already map +directly to upstream inputs. PromptKit and ConfigKit determine how those +inputs compose and how relative framework resources resolve. Scriptorium +does not infer namespaces, rewrite IDs, flatten source origins, or supply a +second merge policy. + +Adopt PromptKit's default OpenRouter bundle as its lowest-priority underlay. +Caller definitions remain able to replace bundled records by ordinary +composition. Scriptorium does not recreate the former catalogs, maintained +profile aliases, or a local implicit profile selection. Bundle availability +is not a readiness check or a promise of a fixed provider model. + +### Existing Commands And Execution + +Execution, rendering, and prompt inspection select a prompt by exact ID, +including a qualified collection ID where appropriate. Remove the CLI +version selector and the corresponding version members from the CLI and HTTP +request, response, inspection, and report contracts where they exist. Do not +accept a legacy version selector and then ignore it. The HTTP route decision +below determines where the single new HTTP contract is served. + +Continue to distinguish framework configuration, resolved resources, and +request preparation. Framework configuration inspection reports the +upstream captured result, with native provenance, normalization, and +redaction where currently used. Profile inspection reports the resolved +profile, preserving unset versus explicitly supplied values, rather than +reconstructing backend-completed settings. Render shows request-completed +settings. Use native reports or writers when their content matches the +existing interface's purpose; retain focused projections where Scriptorium +owns the format or disclosure boundary. In particular, captured prompt +messages becoming available upstream is not reason to add them automatically +to existing prompt inspection output. + +PromptKit owns preparation, provider calls, retries, capacity, validation, +and repair. Preserve the configured shared server engine and established +backend admission behavior. Use PromptKit's native operation timeout for the +existing whole-run budget where it can preserve the current execution +boundary and error mapping. A provider-attempt timeout is still distinct +from the whole-run deadline; caller cancellation still applies. Keep the +existing context wrapper if delegation would change those semantics rather +than introducing two independent timers. CLI process behavior, HTTP +decoding, limits, restricted artifact loading, safe diagnostics, and +application-owned execution reports remain adapter responsibilities. + +## Work Needed From The Baseline + +- **Converge the module graph and engine wiring.** Move runtime and test + imports to the vanity paths. Replace old ConfigKit input, path, provenance, + and diagnostic types at the CLI construction and inspection boundaries. + Remove the obsolete diagnostic branches and the legacy module and catalog + dependencies. Keep distinctions between application and framework error + presentation where they remain useful. Use public typed causes and sentinel + categories, including for construction, deadlines, and artifact failures. + Reconcile commit-pinned attribution and license notices against the + resulting runtime graph under the [OSS policy](../policy/oss.md), + distinguishing linked dependencies from upstream maintainer tooling. +- **Migrate maintained configuration and definitions in place.** Convert + version-keyed prompts and other keyed resource maps to explicit-ID lists; + choose one definition per ID or give separately selectable former versions + distinct IDs. Convert framework source controls to `config:`, write + `required: true` where a missing import is an error, and use version 2 only + where an optional framework source header is supplied. Use sibling + `extends: {id: ...}` and update exact references when using collections. + Review same-ID whole-record replacement, backend fallback from null + profile fields, self-contained schemas, and source-local message files. Retain + existing application discovery, overlays, and path semantics where still + appropriate; they do not need to change solely to make an old file fail. + Review maintained examples and real-engine test fixtures together. +- **Adapt the existing interfaces.** Update CLI argument mapping, + formatters, captured framework source records and list-index explanation + paths, HTTP DTOs, and terminal report projections. Keep explicit zero + overrides and optional-value presence. Check that the restricted HTTP + artifact reader transfers ownership of returned artifacts and bytes under + the new public contract without changing Scriptorium's containment policy. + Preserve the established execution outcome, size-limit, cancellation, and + credential-disclosure behavior outside the intentional cutover changes. +- **Give operators a migration path in documentation.** Update the + [consumer migration guide](../consumers/upgrading-promptkit.md) with + before-and-after application and framework examples, directions for old + prompt versions and catalog references, and the changed CLI and HTTP + contracts. Explain semantic changes that can survive a syntactic + conversion, particularly replacement, null fallback, and schema + resolution. Update the canonical configuration, CLI, HTTP, subprocess, and + internal references where their implementation changes. Refresh current + contributor and architecture references to the selected upstream release, + leaving historical release records intact. Record release impact as + an intentional incompatible change under the + [release policy](../policy/release.md#versioning-and-compatibility); this + roadmap does not select or publish a release. +- **Verify the migrated boundary, not a second upstream test suite.** Use + the repository's baseline Go checks and representative offline examples + and real-engine adapter tests. Exercise native source composition, + qualified selection, source-local resources, captured inspection, + render and execution, and representative failures for obsolete formats + and selectors. Protect behavior likely to regress in this cutover: + explicit zero overrides, shared backend admission, timeout attribution, + safe output, and unchanged HTTP artifact policy. No live inference is + needed to validate the migration. + +## Open Decisions + +### HTTP Route For The ID-Only Contract + +**Recommendation:** Replace `POST /v1/runs` with a successor route such as +`POST /v2/runs` and support only the successor in the new release. Removing +`prompt_version` changes a documented request and output contract. A new route +makes that break explicit and lets a stale client fail even if it happened +not to send the removed field. Operators must update the URL as well as +their request and response handling, but there is no route forwarding or +legacy model to maintain. + +**Alternative:** Cut over `POST /v1/runs` in place as part of the incompatible +application release. This is viable if all deployed HTTP callers can migrate +in lockstep and changing the URL would add unnecessary deployment work. +The downside is that callers omitting the old field could reach a changed +contract at the same URL, while callers sending it fail strict decoding. +Either choice has one live contract; compatibility routing is not part of +this work.