Files
scriptorium/docs/roadmap/implementation.md

359 lines
23 KiB
Markdown
Raw Normal View History

2026-11-18 18:52:40 +00:00
# Implement the PromptKit v0.23.2 adoption
The [feature roadmap](promptkit-v0.23.2-adoption.md) owns the target and scope.
Start from `247b67312ef1a0552a898c558dfe4639572b0691` plus the planning
commit. Follow the [development guide](../development.md), the applicable
repository policies, and [ADR 0004](../adr/0004-definition-boundary.md) and
[ADR 0005](../adr/0005-configkit-boundary.md). Use the pinned upstream
references in the feature roadmap, not either sibling checkout's moving HEAD.
Execute these stages in order as one sprint. Intermediate stages may leave
unresolved imports, outdated fixtures, or failing tests that a later stage
explicitly addresses. Do not add shims or split the migration into separately
buildable products. Focused checks are useful when their dependencies are
ready; the completed tree is what must pass all validation. Keep the change
to the existing product and the collection discovery example and workflow.
Do not add deferred commands, endpoints, CommandKit dependencies or
scaffolding, or conversation and coordination features.
## Stage 1: Unify dependencies and framework source wiring
1. Update `go.mod` to require `go.maximumdirect.net/promptkit v0.23.2` and
`go.maximumdirect.net/configkit v1.2.1`. Remove the old PromptKit and
ConfigKit requirements and both old catalog requirements. Replace Go
imports throughout production code and tests with the public vanity paths.
Retain Scriptorium's module path, Go 1.27.0 minimum, and Go 1.27.4 release
toolchain. Use ordinary Go module commands with `GOWORK=off` to generate
checksums and select stable `go.yaml.in/yaml/v4 v4.0.0`; complete
`go mod tidy` once code and tests have been converted. Use published
modules, with no committed replacements, workspace, or vendor tree.
2. In `internal/adapter/cli/run.go`, convert `newEngine` to
`configkit.Input` and `configkit.Underlay`. Keep one underlay per
`promptkit_sources` selection in resolved order. A file contributes
`configkit.Input{Source: configkit.File(path), Imports: true}`. For a
directory, call `promptkit.DirectoryInputs(path)` and pass the inputs as
one strict-composition group with the helper's settings unchanged.
Keep `Sources.Main` unset; neither the application YAML nor the last
selector is implicitly a framework main. Pass through schema directory,
engine retry defaults, and injected options as before. Leave the upstream
OpenRouter bundle enabled; do not supply local copies or aliases.
3. In `application_config.go` and `framework_sources.go`, retire the
old-path diagnostic branches. Use `errors.As` for
`*configkit.LoadError` and `configkit.WriteDiagnostics` for typed errors
and both warning paths. Preserve the constructor's wrapped cause and
originating command context; use a safe fallback for other errors. Keep
the distinct application and engine load operations and their warning
timing. Stage 3 handles the inspection-specific path and record types.
4. Keep the current native application schema and overrides in
`internal/config`: ordered main discovery, explicit file selection,
imports, overlay, collection policy, working-directory-relative selector
paths, presence, and schema/domain validation. No field rename or
second schema is needed: the obsolete formats are removed, while fields
already matching the target keep their meaning. Application inspection
continues to use one ConfigKit result without opening framework sources.
5. Retarget the existing dependency guard in
`internal/adapter/dependency_test.go` and its fixtures to the vanity
public/internal boundary. Keep the existing protections against direct
catalog imports and restored framework packages. The final module audit,
rather than matching historical prose or test fixtures, checks that no old
module remains selected.
## Stage 2: Migrate CLI execution, rendering, and terminal reports
1. In `internal/adapter/cli/run.go` remove `--prompt-version`, the stored
value, and associated usage text from `run` and `render`. Remove the
version from `buildRunRequestFromConfig` and the run summary. Forward
exact prompt and profile IDs, including namespaced IDs, without splitting,
qualifying, or otherwise rewriting them. Preserve existing unrelated
flags, explicit numeric zero, reasoning effort presence, session IDs,
transient retry presence, optional inputs, and profile defaults.
2. Keep `run` on `Engine.Run` and `render` on `Engine.Prepare`. For
`run`, keep parsing and nonnegative validation of the Go duration
`--run-timeout`. Put that duration in `RunRequest.OperationTimeout`
before invoking the runner and remove the adapter-created timeout
context. Pass the caller context (`context.Background()` in the current
CLI). The engine owns the budget for preparation, admission, retries,
generation, and validation; parsing, construction, and file writes stay
outside. Do not put the budget on `render` or change the per-attempt
`--timeout` setting.
3. For a failed CLI run use `errors.Is(err, promptkit.ErrOperationTimeout)`
to select the existing safe deadline diagnostic and exit 1. Do not use
a bare deadline cause as proof the whole operation timed out: provider
failures can carry one. Keep completed output, exit 2 for completed
invalid output, other runtime failures at exit 1, and the existing
artifact/report write ordering and failure handling. Do not override a
returned successful result based on a later context check. No local timer
is needed: the pinned upstream API attributes its own expiration and
fixes the outcome before terminal observation.
4. In `internal/format/prepared_run.go` keep native prepared JSON and
remove the version line from text. In `internal/report/report.go`
remove the version field and projection from the terminal report shared by
CLI and HTTP; do not add ancestry or content fields. Keep usage presence,
outcomes, retry/attempt counters, duration, and opt-in report behavior.
Use PromptKit's terminal event rather than reclassifying or synthesizing it.
5. Adapt the existing CLI, preparation-format, and report tests to assert
ID-only mapping, rejection of the removed flag, no version in current
output, explicit zero presence, deadline vs provider failure, and unchanged
report rules. Reuse existing coverage of output files and status handling.
Migrate inline framework fixtures in Stage 5.
## Stage 3: Adapt existing inspection commands in place
1. In `internal/adapter/cli/inspect.go` call
`Engine.InspectPrompt(ctx, id)`. Remove the version flag, config member,
usage text, and output field. Leave the command families, format flags,
exit behavior, and full-source engine construction in place. Do not
introduce list, requirement, validation, or diagnostic commands.
2. In `internal/format/inspection.go`, keep the current narrow prompt
projection and output names other than the retired version. Map
`PromptID`, `PromptHash`, `DefaultProfileID`, declared `Inputs`,
and the new `PromptOutputInspection` fields. Map optional fields as
optional: omit absent metadata in JSON/text rather than inventing an
empty value, while showing present empty strings, zero/false values, or
an empty input list. Do not emit captured message text, description or
session fields that were not previously projected.
3. Map `ProfileInspection` directly into the existing profile formatter:
retain `profile_id` and the separate optional `api_key_required` field.
Rename the existing nested `effective_model_params` section to
`profile_model_params` in JSON and text to avoid implying completion.
Keep the existing member names in that section, mapping upstream
`Backend` to `backend_id`. Use the upstream optional pointers; omit
nil members and show present zeros, empty strings, or an empty provider
map. Do not fill missing values from the backend or perform preparation.
No new renderer, format, or extra settings projection is needed.
4. In `framework_config.go`, use `configkit.ParsePath`,
`configkit.Path`, `configkit.SourceInfo`, and the current
`Layer`, `SourceKind`, and import location fields. Keep the small
current source DTO, its field names, and captured order; pass through
native layer values without translating them to the old generation.
Keep the native effective and explanation writers and their redaction.
Validate constructs the engine; show, sources, and explain use its
captured view. For list indexes use paths such as
`/collections/support/profiles/0/model`, not paths keyed by profile ID.
Keep existing explicit-root, invalid, and unknown pointer handling.
5. Adapt only affected tests: prompt/profile optional values, lack of
message disclosure, absence of backend completion in profile inspection,
and framework captured paths, diagnostics, and existing redaction. Do
not expand the application `config` projection or add blanket redaction
for provider parameters. Leave the CommandKit transition for later.
## Stage 4: Refactor the single-run HTTP API
1. Keep `POST /v1/runs` as the single route: the operation is still a run
and there is no parallel version to distinguish. This is a direct
contract replacement at that path, not a promise of old-client
compatibility. Update `internal/adapter/http/dto.go` and `handler.go`
rather than layering a second handler. Drop request `prompt_version`.
Retain the other current request members and their types: `prompt_id`,
`profile_id`, `session_id`, `inputs`, `vars`, `model`,
`include_raw_output`, `include_execution_report`,
`transient_retries`, and `run_timeout_seconds`. Keep input references
and model overrides in their existing nested shapes. Strictly decode a
single request, so the removed selector and raw `api_key` are unknown
fields. Leave prompt/definition validation and input necessity to
PromptKit after transport validation; do not require a profile when the
prompt defaults one or trim nonblank IDs.
2. Return the existing `artifact`, `validation`, and `metadata`
envelopes on completed runs (including failed output validation) at
200. Keep artifact body as text, not the Go library's base64 artifact
JSON. Retain opt-in `raw_model_output` and opt-in terminal
`execution` on applicable responses. In metadata omit
`prompt_version` and eliminate redundant `model_name`, `endpoint`,
`validation_mode`, `validation_status`, and `repair_attempts_used`:
consume these values from `metadata.model_params` or `validation`.
Keep `metadata.selected_backend_id` as the backend identity and drop
the duplicate `metadata.model_params.backend_id`; upstream guarantees
equality for real results and endpoint-only profiles have no backend ID.
Retain remaining DTO keys, types, timing/usage, and optional fields
for this cutover. Keep the error envelope `{error:{code,message}}`
with the existing optional terminal report where applicable. Do not
serialize raw PromptKit requests/results, causes, or resolved secrets.
3. Map validated `run_timeout_seconds` to
`RunRequest.OperationTimeout` and invoke the shared runner with
`r.Context()`, not a second timeout context. Retain int64 range and
duration-overflow checks, absence/null/zero as no new deadline, and
request/response IO outside the budget. Preserve the current status/code
mapping in the baseline [API table](../api.md#error-responses) as the
starting contract (including 400 for malformed inputs, 404 for missing
IDs and route, 405 for wrong method, 413 for size limits, 500 for
operational validation/unclassified errors, 502 for generation failure,
and 503 for capacity). Resolve these cases explicitly:
- `ErrOperationTimeout`, or a failed run wrapping
`context.DeadlineExceeded` when the parent context itself expired:
504 `deadline_exceeded`.
- A failed run wrapping `context.Canceled` when the parent context is
canceled: 408 `request_canceled`. Check the typed operation timeout
first; do not replace a completed outcome based on a later context
race. A provider-only timeout remains 502 while the parent is live.
- Specific selection, credential, and restricted-reader causes precede
broader PromptKit fallback categories. Use errors.Is/errors.As and
safe messages, not provider status passthrough or string matching.
4. Keep the one engine created by `serve`, request-local reports, current
body and complete encoded response limits, and compact 413 overflow
fallback. Keep file-root requirement, lexical containment with existing
symlink behavior, configured artifact limit, and inline inputs.
Verify that `artifact_reader.go` returns detached mutable body
storage and retains no aliases that can mutate a returned artifact;
fix only if needed by the ownership handoff. Do not implement a new
filesystem threat model or rebuild backend pools per request.
5. Update `handler_test.go`, `artifact_reader_test.go` as needed, and
`examples/http-run.json`. Exercise removed selector/duplicate output
fields, presence-sensitive mapping, safe errors, namespace selection,
limits, and unchanged file policy. Adapt the retry-wait and capacity-wait
deadline tests to exercise a real PromptKit engine and verify typed budget
attribution and capacity release. Use controlled offline doubles or
loopback, with no real keys or paid calls. Use representative canceled
parent and provider-only deadline cases; a fake runner alone is not
evidence that the engine honors a mapped operation budget.
## Stage 5: Migrate maintained definitions and existing test fixtures
1. Convert selected files under `examples/prompts/`,
`examples/profiles/`, and `examples/framework/` and inline or
on-disk framework fixtures across CLI and HTTP tests. Replace keyed
backends, profiles, and version-keyed prompts with lists of `id`
records at their existing resource destinations. Keep the repository's
existing prompt IDs where there is one intended record. For tests with
distinct formerly version-selected prompts, choose distinct IDs only
when both definitions still serve a test; delete expectations about
version ambiguity. Keep role/cache, inputs, schemas, and validation
behavior as appropriate. Leave source-local message files relative to
their owning definition file.
2. Replace framework `configkit:` controls with native `config:`,
express required imports with `required: true`, and use optional
source header `version: 2` if one is needed. Replace resource
inheritance with `extends: {id: <sibling-id>}`. Review overlays for
whole-record replacement at the same ID and replace obsolete catalog
references with an intended current bundle profile or a complete local
profile. Prefer already selected local profiles in offline fixtures;
do not infer alias mappings or a default model. Check meaningful null
endpoint/credential/map cases against new backend fallback semantics
and keep schemas self-contained. Preserve application composition and
collection selector examples (`examples/config.collections.yml`
demonstrates app source selector composition, not checkout discovery).
3. Update scripts such as `examples/render-v0.9-features.sh` and active
sample invocations to drop version selectors. Adapt remaining native
framework fixtures and assertions from the previous stages. Retain
obsolete YAML only as deliberate negative or historical material.
Add a representative adapter-boundary test for an obsolete version-keyed
definition being rejected by upstream, without matching upstream prose.
Check a valid current definition reaches preparation/execution without a
Scriptorium field filter; do not copy PromptKit's format test matrix.
## Stage 6: Demonstrate discovery of independent collection checkouts
1. Add `examples/config.discovery.yml` selecting
`./examples/discovery/framework.yml` as a file under
`promptkit_sources`. The framework file declares two **optional**
`config.imports` globs: `collections/*/collection.yml` and
`collections/*/collection.yaml`. Omit `required: true` on each.
These paths are relative to that file; `*` selects one checkout level
in this example, not a new Scriptorium layout restriction. Leave
discovery in native ConfigKit declarations: no Python or Go scanner,
implicit path search, special per-collection registration, or auto-mount
by checkout name.
2. Under `examples/discovery/collections/`, add `support-repo` with
`collection.yml` declaring `collections.support` and a separate
`review-repo` with `collection.yaml` declaring `collections.review`.
Give support at least `base` and `fast` profiles and `answer` and
`summarize` prompts, each a separate YAML record file with an `id`.
The entry file uses explicit relative `config.imports` with
`placement: sequence-item`, destinations
`[collections, support, profiles]` or
`[collections, support, prompts]`, and `required: true` for its own
named required fragments. Use `fast` extending `base`, a synthetic
endpoint/model and `default_profile: support/fast`. Give review its
own profile and prompt (reuse a local ID to demonstrate namespace
separation if useful). Have `support/answer` read a message file
beside its own definition and use the named input `transcript` with
`examples/fixtures/transcript.md`. Include unrelated YAML that would be
unsuitable as a framework input and is not selected by the imports.
3. Run the maintained config through `inspect configuration validate`
and `render --prompt support/answer --profile support/fast` with
`--input transcript=./examples/fixtures/transcript.md`; render a review
prompt too. Add a focused real-engine CLI
test using a scratch checkout tree with the same discovery declarations:
each spelling by itself works, both work together, and placing a
second checkout after one invocation makes it selectable on a new
invocation without editing config. Assert qualified selection and
rendered content from the source-local message/inherited profile.
Confirm unrelated YAML is ignored by successfully loading and preparing
the real fixture. Test through the CLI command/engine boundary, not
through a mock glob implementation. Do not promise hot reload of a
running server.
## Stage 7: Update canonical docs and component notices
1. Put before/after application and framework examples in
[upgrading-promptkit.md](../consumers/upgrading-promptkit.md). Explain
that unchanged app fields still work through the one native schema,
how to rewrite the selected framework files, how to replace old
version selectors and catalog IDs, whole-record replacement, null
fallback, and self-contained schemas. Do not add HTTP migration
instructions. Describe the final contract in [api.md](../api.md)
including the removal of redundant metadata, deadline/status distinctions,
and updated request example. Keep the HTTP example current.
2. In [config.md](../config.md) explain native discovery selections and
both optional filename globs with a link to the maintained example.
In [operations.md](../operations.md) describe checkout placement and
independent updates; in [cli.md](../cli.md) demonstrate the discovery
example and qualified rendering. Use the actual Stage 6 IDs and input
path. Update CLI flags, inspection field semantics and list-index
explanation examples, render/report details, and subprocess guidance
as affected. Do not describe deferred CommandKit behavior as current.
3. Refresh README where affected, `docs/development.md`,
`docs/internal/{overview,adapters,sources}.md`,
the Scriptorium-specific dependency text in
`docs/policy/architecture.md`, and affected current consumer guides.
Use the new public module links and one ConfigKit generation with
two schemas; distinguish changed implementation from existing policy.
Leave historical release notes, ADR decisions, and completed roadmaps
intact. Do not select, tag, or publish a Scriptorium release here.
4. Under the [OSS policy](../policy/oss.md), update
`docs/oss/attribution.md` for PromptKit and a single ConfigKit entry
at the exact selected commits from the feature roadmap. Remove obsolete
catalog and legacy ConfigKit entries. Inspect the final runtime
dependency list and upstream notice files; expect jsonschema v6.0.3,
x/text v0.44.0, and go-yaml v4.0.0. Use the selected versions'
copyright and license notices and commit-pinned links (the selected
PromptKit attribution is a starting point to verify). Remove stale
YAML v3 attribution if no runtime or incorporated material uses it.
Keep applicable verbatim local license texts and companion links.
Distinguish tool-only graph entries such as upstream ReleaseKit and its
tooling dependencies from Scriptorium's runtime. This is attribution
maintenance, not a project license or release procedure change.
## Stage 8: Verify the complete cutover
1. Use the selected Go 1.27.4 toolchain. Finish `GOWORK=off go mod tidy`
after all imports and fixtures are in place and confirm
`GOWORK=off go mod tidy -diff` is empty. Inspect `go list -m all` for
the pinned modules, stable YAML, and absence of both legacy modules
and the catalogs. Use `go list -deps ./cmd/scriptorium` to distinguish
linked packages from graph-only tooling. Confirm no replacements,
workspaces, vendored code, or dependency on unpublished sources.
2. Run the [baseline Go checks](../development.md#baseline-validation)
using prepared dependencies and the deterministic offline test suite:
`go test ./...`, `go test -race ./...`, `go vet ./...`, and
`go build ./cmd/scriptorium`. Put binaries outside the repository.
Build the two existing Linux release targets (amd64 and arm64,
`CGO_ENABLED=0`, `-trimpath`, `-ldflags '-s -w'`) without running
publication. Check `gofmt` and `git diff --check`.
3. Use a freshly built binary for maintained config and render commands,
including simple, full, composed with its local overlay, inherited,
application source-list collection/overlay, and discovered collections.
Exercise app inspection without a framework load and framework validate,
show, sources, and explain with a real current path. Verify synthetic
data only, one appropriate output on stdout, and no provider calls for
render/inspection. Covered CLI/HTTP run tests should use a deterministic
fake provider or injected client. Check shell examples and affected
links, including pinned attribution targets.
4. Review the complete diff against the feature roadmap: one native load
per domain, ID-only execution, no legacy routes or translators, no
extra inspection surface, optional discovery for both spellings,
consistent disclosure and transport boundaries, and current docs.
Check for stale active references rather than rewriting history or
deleting deliberate rejection fixtures. Report verification results and
any material blockers; do not convert these sprint checks into new
permanent release gates.