diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md new file mode 100644 index 0000000..d7f38f8 --- /dev/null +++ b/docs/roadmap/implementation.md @@ -0,0 +1,358 @@ +# Implement the PromptKit v0.23.2 adoption + +The [feature roadmap](promptkit-v0.23.2-adoption.md) owns the target and scope. +Start from `247b67312ef1a0552a898c558dfe4639572b0691` plus the planning +commit. Follow the [development guide](../development.md), the applicable +repository policies, and [ADR 0004](../adr/0004-definition-boundary.md) and +[ADR 0005](../adr/0005-configkit-boundary.md). Use the pinned upstream +references in the feature roadmap, not either sibling checkout's moving HEAD. + +Execute these stages in order as one sprint. Intermediate stages may leave +unresolved imports, outdated fixtures, or failing tests that a later stage +explicitly addresses. Do not add shims or split the migration into separately +buildable products. Focused checks are useful when their dependencies are +ready; the completed tree is what must pass all validation. Keep the change +to the existing product and the collection discovery example and workflow. +Do not add deferred commands, endpoints, CommandKit dependencies or +scaffolding, or conversation and coordination features. + +## Stage 1: Unify dependencies and framework source wiring + +1. Update `go.mod` to require `go.maximumdirect.net/promptkit v0.23.2` and + `go.maximumdirect.net/configkit v1.2.1`. Remove the old PromptKit and + ConfigKit requirements and both old catalog requirements. Replace Go + imports throughout production code and tests with the public vanity paths. + Retain Scriptorium's module path, Go 1.27.0 minimum, and Go 1.27.4 release + toolchain. Use ordinary Go module commands with `GOWORK=off` to generate + checksums and select stable `go.yaml.in/yaml/v4 v4.0.0`; complete + `go mod tidy` once code and tests have been converted. Use published + modules, with no committed replacements, workspace, or vendor tree. +2. In `internal/adapter/cli/run.go`, convert `newEngine` to + `configkit.Input` and `configkit.Underlay`. Keep one underlay per + `promptkit_sources` selection in resolved order. A file contributes + `configkit.Input{Source: configkit.File(path), Imports: true}`. For a + directory, call `promptkit.DirectoryInputs(path)` and pass the inputs as + one strict-composition group with the helper's settings unchanged. + Keep `Sources.Main` unset; neither the application YAML nor the last + selector is implicitly a framework main. Pass through schema directory, + engine retry defaults, and injected options as before. Leave the upstream + OpenRouter bundle enabled; do not supply local copies or aliases. +3. In `application_config.go` and `framework_sources.go`, retire the + old-path diagnostic branches. Use `errors.As` for + `*configkit.LoadError` and `configkit.WriteDiagnostics` for typed errors + and both warning paths. Preserve the constructor's wrapped cause and + originating command context; use a safe fallback for other errors. Keep + the distinct application and engine load operations and their warning + timing. Stage 3 handles the inspection-specific path and record types. +4. Keep the current native application schema and overrides in + `internal/config`: ordered main discovery, explicit file selection, + imports, overlay, collection policy, working-directory-relative selector + paths, presence, and schema/domain validation. No field rename or + second schema is needed: the obsolete formats are removed, while fields + already matching the target keep their meaning. Application inspection + continues to use one ConfigKit result without opening framework sources. +5. Retarget the existing dependency guard in + `internal/adapter/dependency_test.go` and its fixtures to the vanity + public/internal boundary. Keep the existing protections against direct + catalog imports and restored framework packages. The final module audit, + rather than matching historical prose or test fixtures, checks that no old + module remains selected. + +## Stage 2: Migrate CLI execution, rendering, and terminal reports + +1. In `internal/adapter/cli/run.go` remove `--prompt-version`, the stored + value, and associated usage text from `run` and `render`. Remove the + version from `buildRunRequestFromConfig` and the run summary. Forward + exact prompt and profile IDs, including namespaced IDs, without splitting, + qualifying, or otherwise rewriting them. Preserve existing unrelated + flags, explicit numeric zero, reasoning effort presence, session IDs, + transient retry presence, optional inputs, and profile defaults. +2. Keep `run` on `Engine.Run` and `render` on `Engine.Prepare`. For + `run`, keep parsing and nonnegative validation of the Go duration + `--run-timeout`. Put that duration in `RunRequest.OperationTimeout` + before invoking the runner and remove the adapter-created timeout + context. Pass the caller context (`context.Background()` in the current + CLI). The engine owns the budget for preparation, admission, retries, + generation, and validation; parsing, construction, and file writes stay + outside. Do not put the budget on `render` or change the per-attempt + `--timeout` setting. +3. For a failed CLI run use `errors.Is(err, promptkit.ErrOperationTimeout)` + to select the existing safe deadline diagnostic and exit 1. Do not use + a bare deadline cause as proof the whole operation timed out: provider + failures can carry one. Keep completed output, exit 2 for completed + invalid output, other runtime failures at exit 1, and the existing + artifact/report write ordering and failure handling. Do not override a + returned successful result based on a later context check. No local timer + is needed: the pinned upstream API attributes its own expiration and + fixes the outcome before terminal observation. +4. In `internal/format/prepared_run.go` keep native prepared JSON and + remove the version line from text. In `internal/report/report.go` + remove the version field and projection from the terminal report shared by + CLI and HTTP; do not add ancestry or content fields. Keep usage presence, + outcomes, retry/attempt counters, duration, and opt-in report behavior. + Use PromptKit's terminal event rather than reclassifying or synthesizing it. +5. Adapt the existing CLI, preparation-format, and report tests to assert + ID-only mapping, rejection of the removed flag, no version in current + output, explicit zero presence, deadline vs provider failure, and unchanged + report rules. Reuse existing coverage of output files and status handling. + Migrate inline framework fixtures in Stage 5. + +## Stage 3: Adapt existing inspection commands in place + +1. In `internal/adapter/cli/inspect.go` call + `Engine.InspectPrompt(ctx, id)`. Remove the version flag, config member, + usage text, and output field. Leave the command families, format flags, + exit behavior, and full-source engine construction in place. Do not + introduce list, requirement, validation, or diagnostic commands. +2. In `internal/format/inspection.go`, keep the current narrow prompt + projection and output names other than the retired version. Map + `PromptID`, `PromptHash`, `DefaultProfileID`, declared `Inputs`, + and the new `PromptOutputInspection` fields. Map optional fields as + optional: omit absent metadata in JSON/text rather than inventing an + empty value, while showing present empty strings, zero/false values, or + an empty input list. Do not emit captured message text, description or + session fields that were not previously projected. +3. Map `ProfileInspection` directly into the existing profile formatter: + retain `profile_id` and the separate optional `api_key_required` field. + Rename the existing nested `effective_model_params` section to + `profile_model_params` in JSON and text to avoid implying completion. + Keep the existing member names in that section, mapping upstream + `Backend` to `backend_id`. Use the upstream optional pointers; omit + nil members and show present zeros, empty strings, or an empty provider + map. Do not fill missing values from the backend or perform preparation. + No new renderer, format, or extra settings projection is needed. +4. In `framework_config.go`, use `configkit.ParsePath`, + `configkit.Path`, `configkit.SourceInfo`, and the current + `Layer`, `SourceKind`, and import location fields. Keep the small + current source DTO, its field names, and captured order; pass through + native layer values without translating them to the old generation. + Keep the native effective and explanation writers and their redaction. + Validate constructs the engine; show, sources, and explain use its + captured view. For list indexes use paths such as + `/collections/support/profiles/0/model`, not paths keyed by profile ID. + Keep existing explicit-root, invalid, and unknown pointer handling. +5. Adapt only affected tests: prompt/profile optional values, lack of + message disclosure, absence of backend completion in profile inspection, + and framework captured paths, diagnostics, and existing redaction. Do + not expand the application `config` projection or add blanket redaction + for provider parameters. Leave the CommandKit transition for later. + +## Stage 4: Refactor the single-run HTTP API + +1. Keep `POST /v1/runs` as the single route: the operation is still a run + and there is no parallel version to distinguish. This is a direct + contract replacement at that path, not a promise of old-client + compatibility. Update `internal/adapter/http/dto.go` and `handler.go` + rather than layering a second handler. Drop request `prompt_version`. + Retain the other current request members and their types: `prompt_id`, + `profile_id`, `session_id`, `inputs`, `vars`, `model`, + `include_raw_output`, `include_execution_report`, + `transient_retries`, and `run_timeout_seconds`. Keep input references + and model overrides in their existing nested shapes. Strictly decode a + single request, so the removed selector and raw `api_key` are unknown + fields. Leave prompt/definition validation and input necessity to + PromptKit after transport validation; do not require a profile when the + prompt defaults one or trim nonblank IDs. +2. Return the existing `artifact`, `validation`, and `metadata` + envelopes on completed runs (including failed output validation) at + 200. Keep artifact body as text, not the Go library's base64 artifact + JSON. Retain opt-in `raw_model_output` and opt-in terminal + `execution` on applicable responses. In metadata omit + `prompt_version` and eliminate redundant `model_name`, `endpoint`, + `validation_mode`, `validation_status`, and `repair_attempts_used`: + consume these values from `metadata.model_params` or `validation`. + Keep `metadata.selected_backend_id` as the backend identity and drop + the duplicate `metadata.model_params.backend_id`; upstream guarantees + equality for real results and endpoint-only profiles have no backend ID. + Retain remaining DTO keys, types, timing/usage, and optional fields + for this cutover. Keep the error envelope `{error:{code,message}}` + with the existing optional terminal report where applicable. Do not + serialize raw PromptKit requests/results, causes, or resolved secrets. +3. Map validated `run_timeout_seconds` to + `RunRequest.OperationTimeout` and invoke the shared runner with + `r.Context()`, not a second timeout context. Retain int64 range and + duration-overflow checks, absence/null/zero as no new deadline, and + request/response IO outside the budget. Preserve the current status/code + mapping in the baseline [API table](../api.md#error-responses) as the + starting contract (including 400 for malformed inputs, 404 for missing + IDs and route, 405 for wrong method, 413 for size limits, 500 for + operational validation/unclassified errors, 502 for generation failure, + and 503 for capacity). Resolve these cases explicitly: + - `ErrOperationTimeout`, or a failed run wrapping + `context.DeadlineExceeded` when the parent context itself expired: + 504 `deadline_exceeded`. + - A failed run wrapping `context.Canceled` when the parent context is + canceled: 408 `request_canceled`. Check the typed operation timeout + first; do not replace a completed outcome based on a later context + race. A provider-only timeout remains 502 while the parent is live. + - Specific selection, credential, and restricted-reader causes precede + broader PromptKit fallback categories. Use errors.Is/errors.As and + safe messages, not provider status passthrough or string matching. +4. Keep the one engine created by `serve`, request-local reports, current + body and complete encoded response limits, and compact 413 overflow + fallback. Keep file-root requirement, lexical containment with existing + symlink behavior, configured artifact limit, and inline inputs. + Verify that `artifact_reader.go` returns detached mutable body + storage and retains no aliases that can mutate a returned artifact; + fix only if needed by the ownership handoff. Do not implement a new + filesystem threat model or rebuild backend pools per request. +5. Update `handler_test.go`, `artifact_reader_test.go` as needed, and + `examples/http-run.json`. Exercise removed selector/duplicate output + fields, presence-sensitive mapping, safe errors, namespace selection, + limits, and unchanged file policy. Adapt the retry-wait and capacity-wait + deadline tests to exercise a real PromptKit engine and verify typed budget + attribution and capacity release. Use controlled offline doubles or + loopback, with no real keys or paid calls. Use representative canceled + parent and provider-only deadline cases; a fake runner alone is not + evidence that the engine honors a mapped operation budget. + +## Stage 5: Migrate maintained definitions and existing test fixtures + +1. Convert selected files under `examples/prompts/`, + `examples/profiles/`, and `examples/framework/` and inline or + on-disk framework fixtures across CLI and HTTP tests. Replace keyed + backends, profiles, and version-keyed prompts with lists of `id` + records at their existing resource destinations. Keep the repository's + existing prompt IDs where there is one intended record. For tests with + distinct formerly version-selected prompts, choose distinct IDs only + when both definitions still serve a test; delete expectations about + version ambiguity. Keep role/cache, inputs, schemas, and validation + behavior as appropriate. Leave source-local message files relative to + their owning definition file. +2. Replace framework `configkit:` controls with native `config:`, + express required imports with `required: true`, and use optional + source header `version: 2` if one is needed. Replace resource + inheritance with `extends: {id: }`. Review overlays for + whole-record replacement at the same ID and replace obsolete catalog + references with an intended current bundle profile or a complete local + profile. Prefer already selected local profiles in offline fixtures; + do not infer alias mappings or a default model. Check meaningful null + endpoint/credential/map cases against new backend fallback semantics + and keep schemas self-contained. Preserve application composition and + collection selector examples (`examples/config.collections.yml` + demonstrates app source selector composition, not checkout discovery). +3. Update scripts such as `examples/render-v0.9-features.sh` and active + sample invocations to drop version selectors. Adapt remaining native + framework fixtures and assertions from the previous stages. Retain + obsolete YAML only as deliberate negative or historical material. + Add a representative adapter-boundary test for an obsolete version-keyed + definition being rejected by upstream, without matching upstream prose. + Check a valid current definition reaches preparation/execution without a + Scriptorium field filter; do not copy PromptKit's format test matrix. + +## Stage 6: Demonstrate discovery of independent collection checkouts + +1. Add `examples/config.discovery.yml` selecting + `./examples/discovery/framework.yml` as a file under + `promptkit_sources`. The framework file declares two **optional** + `config.imports` globs: `collections/*/collection.yml` and + `collections/*/collection.yaml`. Omit `required: true` on each. + These paths are relative to that file; `*` selects one checkout level + in this example, not a new Scriptorium layout restriction. Leave + discovery in native ConfigKit declarations: no Python or Go scanner, + implicit path search, special per-collection registration, or auto-mount + by checkout name. +2. Under `examples/discovery/collections/`, add `support-repo` with + `collection.yml` declaring `collections.support` and a separate + `review-repo` with `collection.yaml` declaring `collections.review`. + Give support at least `base` and `fast` profiles and `answer` and + `summarize` prompts, each a separate YAML record file with an `id`. + The entry file uses explicit relative `config.imports` with + `placement: sequence-item`, destinations + `[collections, support, profiles]` or + `[collections, support, prompts]`, and `required: true` for its own + named required fragments. Use `fast` extending `base`, a synthetic + endpoint/model and `default_profile: support/fast`. Give review its + own profile and prompt (reuse a local ID to demonstrate namespace + separation if useful). Have `support/answer` read a message file + beside its own definition and use the named input `transcript` with + `examples/fixtures/transcript.md`. Include unrelated YAML that would be + unsuitable as a framework input and is not selected by the imports. +3. Run the maintained config through `inspect configuration validate` + and `render --prompt support/answer --profile support/fast` with + `--input transcript=./examples/fixtures/transcript.md`; render a review + prompt too. Add a focused real-engine CLI + test using a scratch checkout tree with the same discovery declarations: + each spelling by itself works, both work together, and placing a + second checkout after one invocation makes it selectable on a new + invocation without editing config. Assert qualified selection and + rendered content from the source-local message/inherited profile. + Confirm unrelated YAML is ignored by successfully loading and preparing + the real fixture. Test through the CLI command/engine boundary, not + through a mock glob implementation. Do not promise hot reload of a + running server. + +## Stage 7: Update canonical docs and component notices + +1. Put before/after application and framework examples in + [upgrading-promptkit.md](../consumers/upgrading-promptkit.md). Explain + that unchanged app fields still work through the one native schema, + how to rewrite the selected framework files, how to replace old + version selectors and catalog IDs, whole-record replacement, null + fallback, and self-contained schemas. Do not add HTTP migration + instructions. Describe the final contract in [api.md](../api.md) + including the removal of redundant metadata, deadline/status distinctions, + and updated request example. Keep the HTTP example current. +2. In [config.md](../config.md) explain native discovery selections and + both optional filename globs with a link to the maintained example. + In [operations.md](../operations.md) describe checkout placement and + independent updates; in [cli.md](../cli.md) demonstrate the discovery + example and qualified rendering. Use the actual Stage 6 IDs and input + path. Update CLI flags, inspection field semantics and list-index + explanation examples, render/report details, and subprocess guidance + as affected. Do not describe deferred CommandKit behavior as current. +3. Refresh README where affected, `docs/development.md`, + `docs/internal/{overview,adapters,sources}.md`, + the Scriptorium-specific dependency text in + `docs/policy/architecture.md`, and affected current consumer guides. + Use the new public module links and one ConfigKit generation with + two schemas; distinguish changed implementation from existing policy. + Leave historical release notes, ADR decisions, and completed roadmaps + intact. Do not select, tag, or publish a Scriptorium release here. +4. Under the [OSS policy](../policy/oss.md), update + `docs/oss/attribution.md` for PromptKit and a single ConfigKit entry + at the exact selected commits from the feature roadmap. Remove obsolete + catalog and legacy ConfigKit entries. Inspect the final runtime + dependency list and upstream notice files; expect jsonschema v6.0.3, + x/text v0.44.0, and go-yaml v4.0.0. Use the selected versions' + copyright and license notices and commit-pinned links (the selected + PromptKit attribution is a starting point to verify). Remove stale + YAML v3 attribution if no runtime or incorporated material uses it. + Keep applicable verbatim local license texts and companion links. + Distinguish tool-only graph entries such as upstream ReleaseKit and its + tooling dependencies from Scriptorium's runtime. This is attribution + maintenance, not a project license or release procedure change. + +## Stage 8: Verify the complete cutover + +1. Use the selected Go 1.27.4 toolchain. Finish `GOWORK=off go mod tidy` + after all imports and fixtures are in place and confirm + `GOWORK=off go mod tidy -diff` is empty. Inspect `go list -m all` for + the pinned modules, stable YAML, and absence of both legacy modules + and the catalogs. Use `go list -deps ./cmd/scriptorium` to distinguish + linked packages from graph-only tooling. Confirm no replacements, + workspaces, vendored code, or dependency on unpublished sources. +2. Run the [baseline Go checks](../development.md#baseline-validation) + using prepared dependencies and the deterministic offline test suite: + `go test ./...`, `go test -race ./...`, `go vet ./...`, and + `go build ./cmd/scriptorium`. Put binaries outside the repository. + Build the two existing Linux release targets (amd64 and arm64, + `CGO_ENABLED=0`, `-trimpath`, `-ldflags '-s -w'`) without running + publication. Check `gofmt` and `git diff --check`. +3. Use a freshly built binary for maintained config and render commands, + including simple, full, composed with its local overlay, inherited, + application source-list collection/overlay, and discovered collections. + Exercise app inspection without a framework load and framework validate, + show, sources, and explain with a real current path. Verify synthetic + data only, one appropriate output on stdout, and no provider calls for + render/inspection. Covered CLI/HTTP run tests should use a deterministic + fake provider or injected client. Check shell examples and affected + links, including pinned attribution targets. +4. Review the complete diff against the feature roadmap: one native load + per domain, ID-only execution, no legacy routes or translators, no + extra inspection surface, optional discovery for both spellings, + consistent disclosure and transport boundaries, and current docs. + Check for stale active references rather than rewriting history or + deleting deliberate rejection fixtures. Report verification results and + any material blockers; do not convert these sprint checks into new + permanent release gates.