Plan PromptKit configuration adoption
This commit is contained in:
333
docs/roadmap/promptkit.md
Normal file
333
docs/roadmap/promptkit.md
Normal file
@@ -0,0 +1,333 @@
|
||||
# PromptKit v0.12.0 Adoption
|
||||
|
||||
## Status And Intent
|
||||
|
||||
Proposed on 2026-10-03. No implementation is delivered by this document.
|
||||
The recommendations under Open Decisions need confirmation before preparing a
|
||||
decision-complete implementation plan. Release preparation remains separate.
|
||||
|
||||
Adopt PromptKit's ConfigKit-native configuration boundary while preserving
|
||||
Scriptorium's simple, request-independent execution model. As established in
|
||||
[ADR 0004](../adr/0004-definition-boundary.md), valid prompt and profile
|
||||
definitions for the selected PromptKit version should work without a narrower
|
||||
Scriptorium field-level format. Library-only APIs are considered separately;
|
||||
this upgrade does not turn Scriptorium into a workflow application.
|
||||
|
||||
The maintainer's preference for native ConfigKit semantics and a clean cutover
|
||||
informs the recommended source interface below. That interface is a proposed
|
||||
product change, not something required merely to compile against PromptKit.
|
||||
|
||||
## Reviewed Baseline And Upstream Target
|
||||
|
||||
The reviewed Scriptorium source is commit
|
||||
`bb5006517b6c27dcbe56074f1d97140bf0999c48`; the last release is v0.15.0.
|
||||
The completed [ConfigKit adoption](configkit.md) selects ConfigKit
|
||||
v0.5.0. PromptKit remains v0.11.2, with the OpenRouter and Rakestrawhome `/v2`
|
||||
catalog modules at v2.20260926.0.
|
||||
|
||||
Application configuration already uses one ConfigKit loading, inheritance,
|
||||
resolution, validation, and inspection pipeline. Framework construction still
|
||||
passes `prompt_dir` and `profile_dir` into the removed PromptKit directory
|
||||
fields and registers application `backends` with `WithBackend`. Profile
|
||||
inspection substitutes an empty prompt filesystem to avoid loading prompts.
|
||||
Maintained prompt/profile files and inline test fixtures use the old embedded
|
||||
`id`, `version`, and `base_profile` syntax.
|
||||
|
||||
The target is the published PromptKit v0.12.0 release at full commit
|
||||
`4b8169da286917fde7875425bb32b799b12c6af3`. The published tag and downloaded
|
||||
module identify that same revision. Canonical references are:
|
||||
|
||||
- [Release notes](https://gitea.maximumdirect.net/eric/promptkit/src/commit/4b8169da286917fde7875425bb32b799b12c6af3/docs/releases/v0.12.0.md).
|
||||
- [Framework formats](https://gitea.maximumdirect.net/eric/promptkit/src/commit/4b8169da286917fde7875425bb32b799b12c6af3/docs/formats.md).
|
||||
- [Consumer and migration guide](https://gitea.maximumdirect.net/eric/promptkit/src/commit/4b8169da286917fde7875425bb32b799b12c6af3/docs/consumers/pkg-promptkit.md).
|
||||
- [Configuration ownership decision](https://gitea.maximumdirect.net/eric/promptkit/src/commit/4b8169da286917fde7875425bb32b799b12c6af3/docs/adr/0001-configuration-boundary.md).
|
||||
|
||||
PromptKit now also selects ConfigKit v0.5.0, so no further ConfigKit upgrade is
|
||||
needed. Go 1.25.5 remains sufficient. Its catalog selections are:
|
||||
|
||||
| Module | Version | Full source commit |
|
||||
| --- | --- | --- |
|
||||
| `promptkit-backend-openrouter/v3` | `v3.20260930.0` | `f17d208cb7ad1fb043cda2266cd3011d11b40aa0` |
|
||||
| `promptkit-backend-rakestrawhome/v3` | `v3.20260930.0` | `993443d88c451901f4c917fa00c204fd3b54d28d` |
|
||||
|
||||
This review examined upstream contracts and the affected Scriptorium adapters,
|
||||
configuration, presentation, examples, and tests. It did not establish that the
|
||||
unmodified application builds against the new, incompatible construction API.
|
||||
|
||||
## Targeted End State
|
||||
|
||||
### Framework Ownership And Source Assembly
|
||||
|
||||
Scriptorium constructs engines through PromptKit's public root package and
|
||||
`Config.Sources`. ConfigKit owns source composition, overlays, inheritance,
|
||||
presence, null values, and resource origins. PromptKit owns final definition
|
||||
validation, catalog contributions, backend completion, content capture,
|
||||
preparation, execution, and capacity coordination.
|
||||
|
||||
An operator-selected source may contain any combination of keyed `backends`,
|
||||
`profiles`, and versioned `prompts`. Do not partition or filter documents by
|
||||
directory name, filename, or a Scriptorium copy of the framework schema.
|
||||
Do not independently preflight domain fields in lower-priority definitions
|
||||
that the final effective configuration replaces. Syntax and composition
|
||||
failures remain upstream loading failures.
|
||||
|
||||
The recommended application interface, subject to Open Decision 1, is an
|
||||
ordered `promptkit_sources` sequence. Each entry selects either a `file` or a
|
||||
`directory`; each entry contributes one caller underlay, from lowest to
|
||||
highest priority. A directory uses `promptkit.DirectoryInputs` and strictly
|
||||
composes its recursively discovered YAML files. A file uses `configkit.File`
|
||||
with declared imports enabled, so an entry document can compose fragments.
|
||||
All caller groups follow PromptKit's maintained catalogs and owned defaults.
|
||||
No separately exposed `Sources.Main` setting is needed for this interface.
|
||||
|
||||
For example, the proposed application shape is:
|
||||
|
||||
```yaml
|
||||
promptkit_sources:
|
||||
- directory: ./examples/prompts
|
||||
- directory: ./examples/profiles
|
||||
- file: ./deployment/promptkit.yml
|
||||
schema_dir: ./examples/schemas
|
||||
```
|
||||
|
||||
Directories retain the helper's default of disabled imports. Enabling imports
|
||||
while also recursively discovering their YAML targets would include those
|
||||
files twice in the same group, which ConfigKit rejects. Use a selected entry
|
||||
file with imports for that layout; do not silently deduplicate, skip files, or
|
||||
convert composition conflicts into precedence.
|
||||
|
||||
Under this recommendation, `prompt_dir`, `profile_dir`, application `backends`,
|
||||
and their directory flags are replaced, not retained as a second configuration
|
||||
model. Framework backend definitions live with the framework sources rather
|
||||
than passing through Scriptorium's scalar backend schema. A repeatable
|
||||
`--promptkit-source PATH` selects a replacement ordered source list for
|
||||
engine-using commands; each path selects a regular file or directory using the
|
||||
same rules. Only an explicitly supplied flag overrides the configured list.
|
||||
|
||||
Application source discovery, its imports and selected overlay, explicit CLI
|
||||
override handling, and working-directory-relative application paths remain
|
||||
unchanged. `--config-overlay` still selects an application overlay; it does not
|
||||
implicitly select a PromptKit layer or profile. Framework priority is explicit
|
||||
in the source list, with import/resource containment supplied by upstream for
|
||||
each selected root. This is not a new global filesystem sandbox.
|
||||
|
||||
### Complete Definition Support
|
||||
|
||||
The target accepts the keyed v0.12.0 formats without compatibility rewriting:
|
||||
profile and backend IDs are mapping keys, prompt IDs contain keyed string
|
||||
versions, and `extends` replaces `base_profile`. Literal IDs and references
|
||||
retain their exact spelling, including punctuation and meaningful whitespace;
|
||||
filenames do not establish identities.
|
||||
|
||||
All supported prompt/profile fields continue to reach PromptKit, including
|
||||
inputs, message roles and cache metadata, inline and file content, session
|
||||
templates, output validation and repair, backend/endpoint selection,
|
||||
credentials by environment name, generation controls, and provider parameters.
|
||||
The upstream format reference, not this list, owns the complete contract.
|
||||
|
||||
Presence must survive the boundary: omission, explicit zero/false/blank, and
|
||||
null are not interchangeable. Inheritance and layers recursively combine maps;
|
||||
an empty map does not clear inherited entries. Backend defaults apply after
|
||||
effective profile inheritance. Changing a model or backend no longer drops
|
||||
unrelated explicitly inherited settings. Profile null suppression and explicit
|
||||
zero controls must affect outbound requests as specified upstream.
|
||||
|
||||
Partial profiles remain valid when request overrides can complete them.
|
||||
Scriptorium must not introduce a mandatory model, credential, default profile,
|
||||
or directory prerequisite that prevents such a request. With the recommended
|
||||
source interface, engine construction may have no prompt collection; missing
|
||||
request selections fail through PromptKit's ordinary public errors rather than
|
||||
a required `prompt_dir` check. Source selection does not imply provider access
|
||||
or credential availability.
|
||||
|
||||
### Captured Definitions And Runtime Lifecycle
|
||||
|
||||
Engine construction validates every final effective definition, including
|
||||
unused entries, and captures message-file content and static template syntax.
|
||||
Invalid unused definitions or unreadable message resources may therefore fail
|
||||
startup or inspection before a request selects them. An overwritten domain
|
||||
value is not independently rejected merely because it appears in history.
|
||||
|
||||
ConfigKit's structural directives now also apply to framework input mappings,
|
||||
including dynamic provider maps. Document the reserved `extends` key and the
|
||||
difference from literal runtime provider parameters; do not invent a quoting
|
||||
escape or a second inheritance resolver.
|
||||
|
||||
Content files use their effective field's source context, including imported
|
||||
and inherited content. Retain those contexts by passing actual upstream
|
||||
inputs; do not flatten files into synthetic mappings that lose origins.
|
||||
Application paths remain working-directory-relative, whereas framework message
|
||||
resources follow PromptKit's source-relative contract. Schema and request
|
||||
artifact reads remain preparation-time resources; credentials and provider
|
||||
operations retain their documented runtime boundaries.
|
||||
|
||||
A server retains one constructed engine across its requests, with shared
|
||||
backend capacity and queue accounting. Definition/message edits require a new
|
||||
engine; for this scope, restarting the server is the supported way to obtain
|
||||
one. CLI invocations naturally construct fresh engines. No file watcher,
|
||||
reload endpoint, per-request server reconstruction, or durable state is added.
|
||||
This is not a transactional snapshot of all external files or credentials.
|
||||
|
||||
### Inspection, Requests, And Presentation
|
||||
|
||||
`inspect prompt` and `inspect profile` use the same selected configuration
|
||||
snapshot as execution. Remove the empty-prompt-filesystem substitution.
|
||||
Profile inspection still does not need request inputs, schema reads, model
|
||||
calls, or credential values, but constructing its engine now validates and
|
||||
captures all selected definitions, including prompts. A partial target may be
|
||||
inspected without imposing preparation's completeness requirements.
|
||||
|
||||
`config validate`, `show`, `sources`, and `explain` remain application
|
||||
configuration operations without constructing an engine. Under the proposed
|
||||
source cutover, they show source selections rather than the contents of
|
||||
PromptKit's separately loaded definitions. They are not framework validation.
|
||||
Full framework provenance inspection is addressed in Open Decision 2.
|
||||
|
||||
Keep CLI/HTTP request independence, presence-aware execution overrides,
|
||||
generation/whole-run timeout separation, transient retries, execution reports,
|
||||
artifact policy, and public error mapping. Preserve existing serialized results
|
||||
unless an upstream change actually requires an application contract change.
|
||||
Prepared/profile numeric values alone are not proof that a zero control was
|
||||
sent; verify upstream presence through generation or recorded outbound requests.
|
||||
The dependency change alone does not require a new presence DTO. Continue
|
||||
mapping errors by public identity rather than message text.
|
||||
|
||||
Environment-variable names remain visible; resolved API keys are never
|
||||
printed or serialized. Ordinary `extra_params` retain their selected visibility
|
||||
where those values are presented. Moving definitions out of application
|
||||
settings changes what `config` can inspect, not their sensitivity classification.
|
||||
Do not silently adopt broader redaction rules for existing profile/prepared
|
||||
inspection.
|
||||
|
||||
## Work Needed From The Baseline
|
||||
|
||||
- **Dependencies and attribution:** select released PromptKit v0.12.0, tidy the
|
||||
module graph, and use its selected `/v3` catalogs without direct application
|
||||
imports. Remove obsolete `/v2` modules and `gopkg.in/yaml.v3` if no remaining
|
||||
imports require them. Update PromptKit/catalog component entries and exact
|
||||
source, COPYRIGHT, and LICENSE links in `docs/oss/attribution.md`. The reviewed
|
||||
grants remain LGPL-3.0-or-later for PromptKit and MIT for the catalogs; reuse
|
||||
the applicable local license texts. Preserve unrelated attribution entries
|
||||
and historical release documents.
|
||||
- **Application settings and CLI:** implement the selected source interface
|
||||
through `internal/config` and shared CLI override mapping. Under the native
|
||||
recommendation, retire the backend document/runtime conversion and old
|
||||
directory fields, add ordered source selection, and update command help and
|
||||
prerequisites. Application validation checks its schema and selector shape;
|
||||
framework acquisition and domain validation belong to engine construction.
|
||||
- **Shared engine wiring:** replace removed fields/options with upstream
|
||||
inputs and underlays, retain `SchemaDir`, `TransientRetries`, and runtime
|
||||
collaborator injection, and use the same construction path for execution,
|
||||
rendering, definition inspection, and serving. Preserve identifiable
|
||||
PromptKit and ConfigKit causes and safe diagnostics. Remove obsolete
|
||||
prompt-filesystem inspection scaffolding.
|
||||
- **Definitions and fixtures:** migrate every maintained prompt/profile file
|
||||
under `examples/` and `testdata/`, and inline definitions and construction
|
||||
helpers in CLI, HTTP, and formatting tests. Replace old inheritance syntax
|
||||
and move backend examples into framework definitions if the native cutover
|
||||
is selected. Review old explicit zeros for their changed meaning instead of
|
||||
preserving obsolete expectations mechanically. Keep useful feature-adoption
|
||||
coverage; filenames need not be renamed merely to reduce historical naming.
|
||||
- **Focused adoption coverage:** use real ConfigKit/PromptKit boundaries for
|
||||
representative source order, same-group conflicts, file imports, combined
|
||||
documents, inherited source-relative content, exact IDs/version selection,
|
||||
null suppression, explicit zero/false, and partial-profile completion.
|
||||
Verify eager failures for unused final definitions and unchanged captured
|
||||
message content after file edits on an existing engine. Retain shared-server
|
||||
capacity, retry, cancellation, restricted artifact, output, and error tests.
|
||||
Own integration risks locally without recreating upstream's exhaustive parser
|
||||
or inheritance matrices; use offline clients or `httptest` for generation.
|
||||
- **Current documentation and migration guidance:** update owning contracts
|
||||
in `docs/config.md`, `docs/cli.md`, and `docs/operations.md`, affected internal
|
||||
docs, `docs/development.md`, architecture version/ownership references,
|
||||
README, examples, and applicable consumer guidance. Link to v0.12.0 framework
|
||||
contracts rather than copying their specification. Correct statements that
|
||||
ConfigKit does not participate in framework loading. Explain the selected
|
||||
source migration and engine-rebuild requirement, and preserve accepted ADR
|
||||
decision content. Update HTTP docs only where observable behavior changes.
|
||||
|
||||
## Compatibility And Release Impact
|
||||
|
||||
This is an intentional breaking adoption, not a patch-only dependency refresh.
|
||||
Users must migrate definitions, inheritance syntax, source assembly, and
|
||||
assumptions about numeric presence, recursive maps, eager validation, and file
|
||||
reload. Under the recommended native interface they must also migrate the
|
||||
application source selectors and move application backend definitions into
|
||||
PromptKit source files. Provide small before/after examples and runnable
|
||||
offline migration checks when implementation lands; no automatic legacy parser
|
||||
or catalog alias layer is proposed.
|
||||
|
||||
PromptKit's maintained profile IDs remain supported, not frozen to fixed model
|
||||
settings. Review used external catalog IDs against the selected catalogs; do
|
||||
not promise live provider availability or preserve every volatile inventory ID.
|
||||
Hashes remain opaque equality values, not cross-version identity guarantees.
|
||||
|
||||
Under the [release policy](../policy/release.md), the eventual release must be
|
||||
a new pre-1.0 minor version. Do not select or publish a release in this work
|
||||
set. The previously completed ConfigKit work may share that release; final
|
||||
version, notes, and roadmap retirement belong to release preparation.
|
||||
|
||||
## Completion Criteria And Scope Limits
|
||||
|
||||
Adoption is complete when the released dependency graph is selected, the
|
||||
approved source interface reaches the canonical framework boundary without
|
||||
losing definition semantics, maintained examples and fixtures use v0.12.0,
|
||||
and CLI/HTTP workflows and inspection meet the target above. Run the existing
|
||||
development checks, affected offline examples, and link/diff checks, with
|
||||
focused new integration tests for the actual migration risks.
|
||||
|
||||
This work does not add conversations, appended/follow-up messages,
|
||||
prepared-handle coordination, semantic retry workflows, embedded configuration
|
||||
bundles, framework internals, or application-owned framework validation.
|
||||
Library availability alone does not justify additional commands, endpoints,
|
||||
resource controls, recurring upstream probes, or release gates. Deferred
|
||||
interfaces are scope choices for this adoption, not permanent prohibitions.
|
||||
|
||||
## Open Decisions
|
||||
|
||||
### 1. Native Source Cutover Or Retained Application Selectors?
|
||||
|
||||
**Recommendation:** adopt the ordered `promptkit_sources` file/directory
|
||||
interface above and move backend definitions out of application settings.
|
||||
This exposes combined definitions, explicit priority, imports, and full
|
||||
presence/null semantics through their canonical owner. It removes duplicated
|
||||
backend representation and avoids having to translate absent scalar fields
|
||||
into new presence-aware overlays. The cost is a deliberate application schema
|
||||
and CLI migration alongside the unavoidable framework format migration.
|
||||
|
||||
**Viable alternative:** retain `prompt_dir`, `profile_dir`, their flags, and
|
||||
application `backends`. Assemble a prompt-directory underlay, then a
|
||||
profile-directory underlay, then application backend overrides. Allow combined
|
||||
definition documents in either directory. This reduces migration and keeps
|
||||
familiar directory organization, but requires explicit backend presence/null
|
||||
mapping to avoid replacing catalog values with incidental Go zeros, and a
|
||||
separate file-selector design if framework imports are to be usable without
|
||||
recursive-discovery repetition. It also leaves two backend configuration
|
||||
surfaces and less flexible source ordering. Do not implement this alternative
|
||||
as an old parser or a filter on PromptKit definition fields.
|
||||
|
||||
### 2. Expose Full Framework Configuration Inspection In This Adoption?
|
||||
|
||||
**Recommendation:** defer a new framework `validate/show/sources/explain`
|
||||
command family. Keep existing application and per-definition inspection, and
|
||||
document their distinct scopes. PromptKit's new `InspectConfiguration` uses
|
||||
fixed redaction of credential-shaped dynamic provider keys, including `token`
|
||||
and `password`. That is broader than the maintainer's established API-key-only
|
||||
redaction choice and cannot be disabled through the released public view.
|
||||
Deferral avoids quietly changing that choice or reconstructing private
|
||||
configuration to bypass upstream writers; it does not limit execution of valid
|
||||
definitions.
|
||||
|
||||
**Viable alternative:** add an explicit `inspect configuration` command family
|
||||
using the captured native view, with approval for upstream redaction in this
|
||||
new interface only. Provenance and history would make layered configuration
|
||||
easier to diagnose, and native writers minimize maintenance. The drawback is
|
||||
different provider-parameter visibility between the new view and existing
|
||||
inspection; the exception must be explicit, not applied to other commands.
|
||||
|
||||
**Another viable alternative:** first obtain an upstream release with a public
|
||||
consumer-selectable inspection sensitivity policy, then expose native
|
||||
framework reports with API-key-only redaction. This preserves the selected
|
||||
policy and source ownership, but requires separate upstream work and delays
|
||||
inspection. Do not assume that API exists in v0.12.0 or make core adoption
|
||||
depend on an unpublished interface.
|
||||
Reference in New Issue
Block a user