Files
scriptorium/docs/roadmap/implementation.md

23 KiB

Implement the PromptKit v0.23.2 adoption

The feature roadmap owns the target and scope. Start from 247b67312ef1a0552a898c558dfe4639572b0691 plus the planning commit. Follow the development guide, the applicable repository policies, and ADR 0004 and ADR 0005. Use the pinned upstream references in the feature roadmap, not either sibling checkout's moving HEAD.

Execute these stages in order as one sprint. Intermediate stages may leave unresolved imports, outdated fixtures, or failing tests that a later stage explicitly addresses. Do not add shims or split the migration into separately buildable products. Focused checks are useful when their dependencies are ready; the completed tree is what must pass all validation. Keep the change to the existing product and the collection discovery example and workflow. Do not add deferred commands, endpoints, CommandKit dependencies or scaffolding, or conversation and coordination features.

Stage 1: Unify dependencies and framework source wiring

Status: Complete.

  1. Update go.mod to require go.maximumdirect.net/promptkit v0.23.2 and go.maximumdirect.net/configkit v1.2.1. Remove the old PromptKit and ConfigKit requirements and both old catalog requirements. Replace Go imports throughout production code and tests with the public vanity paths. Retain Scriptorium's module path, Go 1.27.0 minimum, and Go 1.27.4 release toolchain. Use ordinary Go module commands with GOWORK=off to generate checksums and select stable go.yaml.in/yaml/v4 v4.0.0; complete go mod tidy once code and tests have been converted. Use published modules, with no committed replacements, workspace, or vendor tree.
  2. In internal/adapter/cli/run.go, convert newEngine to configkit.Input and configkit.Underlay. Keep one underlay per promptkit_sources selection in resolved order. A file contributes configkit.Input{Source: configkit.File(path), Imports: true}. For a directory, call promptkit.DirectoryInputs(path) and pass the inputs as one strict-composition group with the helper's settings unchanged. Keep Sources.Main unset; neither the application YAML nor the last selector is implicitly a framework main. Pass through schema directory, engine retry defaults, and injected options as before. Leave the upstream OpenRouter bundle enabled; do not supply local copies or aliases.
  3. In application_config.go and framework_sources.go, retire the old-path diagnostic branches. Use errors.As for *configkit.LoadError and configkit.WriteDiagnostics for typed errors and both warning paths. Preserve the constructor's wrapped cause and originating command context; use a safe fallback for other errors. Keep the distinct application and engine load operations and their warning timing. Stage 3 handles the inspection-specific path and record types.
  4. Keep the current native application schema and overrides in internal/config: ordered main discovery, explicit file selection, imports, overlay, collection policy, working-directory-relative selector paths, presence, and schema/domain validation. No field rename or second schema is needed: the obsolete formats are removed, while fields already matching the target keep their meaning. Application inspection continues to use one ConfigKit result without opening framework sources.
  5. Retarget the existing dependency guard in internal/adapter/dependency_test.go and its fixtures to the vanity public/internal boundary. Keep the existing protections against direct catalog imports and restored framework packages. The final module audit, rather than matching historical prose or test fixtures, checks that no old module remains selected.

Stage 2: Migrate CLI execution, rendering, and terminal reports

Status: Complete.

  1. In internal/adapter/cli/run.go remove --prompt-version, the stored value, and associated usage text from run and render. Remove the version from buildRunRequestFromConfig and the run summary. Forward exact prompt and profile IDs, including namespaced IDs, without splitting, qualifying, or otherwise rewriting them. Preserve existing unrelated flags, explicit numeric zero, reasoning effort presence, session IDs, transient retry presence, optional inputs, and profile defaults.
  2. Keep run on Engine.Run and render on Engine.Prepare. For run, keep parsing and nonnegative validation of the Go duration --run-timeout. Put that duration in RunRequest.OperationTimeout before invoking the runner and remove the adapter-created timeout context. Pass the caller context (context.Background() in the current CLI). The engine owns the budget for preparation, admission, retries, generation, and validation; parsing, construction, and file writes stay outside. Do not put the budget on render or change the per-attempt --timeout setting.
  3. For a failed CLI run use errors.Is(err, promptkit.ErrOperationTimeout) to select the existing safe deadline diagnostic and exit 1. Do not use a bare deadline cause as proof the whole operation timed out: provider failures can carry one. Keep completed output, exit 2 for completed invalid output, other runtime failures at exit 1, and the existing artifact/report write ordering and failure handling. Do not override a returned successful result based on a later context check. No local timer is needed: the pinned upstream API attributes its own expiration and fixes the outcome before terminal observation.
  4. In internal/format/prepared_run.go keep native prepared JSON and remove the version line from text. In internal/report/report.go remove the version field and projection from the terminal report shared by CLI and HTTP; do not add ancestry or content fields. Keep usage presence, outcomes, retry/attempt counters, duration, and opt-in report behavior. Use PromptKit's terminal event rather than reclassifying or synthesizing it.
  5. Adapt the existing CLI, preparation-format, and report tests to assert ID-only mapping, rejection of the removed flag, no version in current output, explicit zero presence, deadline vs provider failure, and unchanged report rules. Reuse existing coverage of output files and status handling. Migrate inline framework fixtures in Stage 5.

Stage 3: Adapt existing inspection commands in place

Status: Complete.

  1. In internal/adapter/cli/inspect.go call Engine.InspectPrompt(ctx, id). Remove the version flag, config member, usage text, and output field. Leave the command families, format flags, exit behavior, and full-source engine construction in place. Do not introduce list, requirement, validation, or diagnostic commands.
  2. In internal/format/inspection.go, keep the current narrow prompt projection and output names other than the retired version. Map PromptID, PromptHash, DefaultProfileID, declared Inputs, and the new PromptOutputInspection fields. Map optional fields as optional: omit absent metadata in JSON/text rather than inventing an empty value, while showing present empty strings, zero/false values, or an empty input list. Do not emit captured message text, description or session fields that were not previously projected.
  3. Map ProfileInspection directly into the existing profile formatter: retain profile_id and the separate optional api_key_required field. Rename the existing nested effective_model_params section to profile_model_params in JSON and text to avoid implying completion. Keep the existing member names in that section, mapping upstream Backend to backend_id. Use the upstream optional pointers; omit nil members and show present zeros, empty strings, or an empty provider map. Do not fill missing values from the backend or perform preparation. No new renderer, format, or extra settings projection is needed.
  4. In framework_config.go, use configkit.ParsePath, configkit.Path, configkit.SourceInfo, and the current Layer, SourceKind, and import location fields. Keep the small current source DTO, its field names, and captured order; pass through native layer values without translating them to the old generation. Keep the native effective and explanation writers and their redaction. Validate constructs the engine; show, sources, and explain use its captured view. For list indexes use paths such as /collections/support/profiles/0/model, not paths keyed by profile ID. Keep existing explicit-root, invalid, and unknown pointer handling.
  5. Adapt only affected tests: prompt/profile optional values, lack of message disclosure, absence of backend completion in profile inspection, and framework captured paths, diagnostics, and existing redaction. Do not expand the application config projection or add blanket redaction for provider parameters. Leave the CommandKit transition for later.

Stage 4: Refactor the single-run HTTP API

Status: Complete.

  1. Keep POST /v1/runs as the single route: the operation is still a run and there is no parallel version to distinguish. This is a direct contract replacement at that path, not a promise of old-client compatibility. Update internal/adapter/http/dto.go and handler.go rather than layering a second handler. Drop request prompt_version. Retain the other current request members and their types: prompt_id, profile_id, session_id, inputs, vars, model, include_raw_output, include_execution_report, transient_retries, and run_timeout_seconds. Keep input references and model overrides in their existing nested shapes. Strictly decode a single request, so the removed selector and raw api_key are unknown fields. Leave prompt/definition validation and input necessity to PromptKit after transport validation; do not require a profile when the prompt defaults one or trim nonblank IDs.
  2. Return the existing artifact, validation, and metadata envelopes on completed runs (including failed output validation) at 200. Keep artifact body as text, not the Go library's base64 artifact JSON. Retain opt-in raw_model_output and opt-in terminal execution on applicable responses. In metadata omit prompt_version and eliminate redundant model_name, endpoint, validation_mode, validation_status, and repair_attempts_used: consume these values from metadata.model_params or validation. Keep metadata.selected_backend_id as the backend identity and drop the duplicate metadata.model_params.backend_id; upstream guarantees equality for real results and endpoint-only profiles have no backend ID. Retain remaining DTO keys, types, timing/usage, and optional fields for this cutover. Keep the error envelope {error:{code,message}} with the existing optional terminal report where applicable. Do not serialize raw PromptKit requests/results, causes, or resolved secrets.
  3. Map validated run_timeout_seconds to RunRequest.OperationTimeout and invoke the shared runner with r.Context(), not a second timeout context. Retain int64 range and duration-overflow checks, absence/null/zero as no new deadline, and request/response IO outside the budget. Preserve the current status/code mapping in the baseline API table as the starting contract (including 400 for malformed inputs, 404 for missing IDs and route, 405 for wrong method, 413 for size limits, 500 for operational validation/unclassified errors, 502 for generation failure, and 503 for capacity). Resolve these cases explicitly:
    • ErrOperationTimeout, or a failed run wrapping context.DeadlineExceeded when the parent context itself expired: 504 deadline_exceeded.
    • A failed run wrapping context.Canceled when the parent context is canceled: 408 request_canceled. Check the typed operation timeout first; do not replace a completed outcome based on a later context race. A provider-only timeout remains 502 while the parent is live.
    • Specific selection, credential, and restricted-reader causes precede broader PromptKit fallback categories. Use errors.Is/errors.As and safe messages, not provider status passthrough or string matching.
  4. Keep the one engine created by serve, request-local reports, current body and complete encoded response limits, and compact 413 overflow fallback. Keep file-root requirement, lexical containment with existing symlink behavior, configured artifact limit, and inline inputs. Verify that artifact_reader.go returns detached mutable body storage and retains no aliases that can mutate a returned artifact; fix only if needed by the ownership handoff. Do not implement a new filesystem threat model or rebuild backend pools per request.
  5. Update handler_test.go, artifact_reader_test.go as needed, and examples/http-run.json. Exercise removed selector/duplicate output fields, presence-sensitive mapping, safe errors, namespace selection, limits, and unchanged file policy. Adapt the retry-wait and capacity-wait deadline tests to exercise a real PromptKit engine and verify typed budget attribution and capacity release. Use controlled offline doubles or loopback, with no real keys or paid calls. Use representative canceled parent and provider-only deadline cases; a fake runner alone is not evidence that the engine honors a mapped operation budget.

Stage 5: Migrate maintained definitions and existing test fixtures

  1. Convert selected files under examples/prompts/, examples/profiles/, and examples/framework/ and inline or on-disk framework fixtures across CLI and HTTP tests. Replace keyed backends, profiles, and version-keyed prompts with lists of id records at their existing resource destinations. Keep the repository's existing prompt IDs where there is one intended record. For tests with distinct formerly version-selected prompts, choose distinct IDs only when both definitions still serve a test; delete expectations about version ambiguity. Keep role/cache, inputs, schemas, and validation behavior as appropriate. Leave source-local message files relative to their owning definition file.
  2. Replace framework configkit: controls with native config:, express required imports with required: true, and use optional source header version: 2 if one is needed. Replace resource inheritance with extends: {id: <sibling-id>}. Review overlays for whole-record replacement at the same ID and replace obsolete catalog references with an intended current bundle profile or a complete local profile. Prefer already selected local profiles in offline fixtures; do not infer alias mappings or a default model. Check meaningful null endpoint/credential/map cases against new backend fallback semantics and keep schemas self-contained. Preserve application composition and collection selector examples (examples/config.collections.yml demonstrates app source selector composition, not checkout discovery).
  3. Update scripts such as examples/render-v0.9-features.sh and active sample invocations to drop version selectors. Adapt remaining native framework fixtures and assertions from the previous stages. Retain obsolete YAML only as deliberate negative or historical material. Add a representative adapter-boundary test for an obsolete version-keyed definition being rejected by upstream, without matching upstream prose. Check a valid current definition reaches preparation/execution without a Scriptorium field filter; do not copy PromptKit's format test matrix.

Stage 6: Demonstrate discovery of independent collection checkouts

  1. Add examples/config.discovery.yml selecting ./examples/discovery/framework.yml as a file under promptkit_sources. The framework file declares two optional config.imports globs: collections/*/collection.yml and collections/*/collection.yaml. Omit required: true on each. These paths are relative to that file; * selects one checkout level in this example, not a new Scriptorium layout restriction. Leave discovery in native ConfigKit declarations: no Python or Go scanner, implicit path search, special per-collection registration, or auto-mount by checkout name.
  2. Under examples/discovery/collections/, add support-repo with collection.yml declaring collections.support and a separate review-repo with collection.yaml declaring collections.review. Give support at least base and fast profiles and answer and summarize prompts, each a separate YAML record file with an id. The entry file uses explicit relative config.imports with placement: sequence-item, destinations [collections, support, profiles] or [collections, support, prompts], and required: true for its own named required fragments. Use fast extending base, a synthetic endpoint/model and default_profile: support/fast. Give review its own profile and prompt (reuse a local ID to demonstrate namespace separation if useful). Have support/answer read a message file beside its own definition and use the named input transcript with examples/fixtures/transcript.md. Include unrelated YAML that would be unsuitable as a framework input and is not selected by the imports.
  3. Run the maintained config through inspect configuration validate and render --prompt support/answer --profile support/fast with --input transcript=./examples/fixtures/transcript.md; render a review prompt too. Add a focused real-engine CLI test using a scratch checkout tree with the same discovery declarations: each spelling by itself works, both work together, and placing a second checkout after one invocation makes it selectable on a new invocation without editing config. Assert qualified selection and rendered content from the source-local message/inherited profile. Confirm unrelated YAML is ignored by successfully loading and preparing the real fixture. Test through the CLI command/engine boundary, not through a mock glob implementation. Do not promise hot reload of a running server.

Stage 7: Update canonical docs and component notices

  1. Put before/after application and framework examples in upgrading-promptkit.md. Explain that unchanged app fields still work through the one native schema, how to rewrite the selected framework files, how to replace old version selectors and catalog IDs, whole-record replacement, null fallback, and self-contained schemas. Do not add HTTP migration instructions. Describe the final contract in api.md including the removal of redundant metadata, deadline/status distinctions, and updated request example. Keep the HTTP example current.
  2. In config.md explain native discovery selections and both optional filename globs with a link to the maintained example. In operations.md describe checkout placement and independent updates; in cli.md demonstrate the discovery example and qualified rendering. Use the actual Stage 6 IDs and input path. Update CLI flags, inspection field semantics and list-index explanation examples, render/report details, and subprocess guidance as affected. Do not describe deferred CommandKit behavior as current.
  3. Refresh README where affected, docs/development.md, docs/internal/{overview,adapters,sources}.md, the Scriptorium-specific dependency text in docs/policy/architecture.md, and affected current consumer guides. Use the new public module links and one ConfigKit generation with two schemas; distinguish changed implementation from existing policy. Leave historical release notes, ADR decisions, and completed roadmaps intact. Do not select, tag, or publish a Scriptorium release here.
  4. Under the OSS policy, update docs/oss/attribution.md for PromptKit and a single ConfigKit entry at the exact selected commits from the feature roadmap. Remove obsolete catalog and legacy ConfigKit entries. Inspect the final runtime dependency list and upstream notice files; expect jsonschema v6.0.3, x/text v0.44.0, and go-yaml v4.0.0. Use the selected versions' copyright and license notices and commit-pinned links (the selected PromptKit attribution is a starting point to verify). Remove stale YAML v3 attribution if no runtime or incorporated material uses it. Keep applicable verbatim local license texts and companion links. Distinguish tool-only graph entries such as upstream ReleaseKit and its tooling dependencies from Scriptorium's runtime. This is attribution maintenance, not a project license or release procedure change.

Stage 8: Verify the complete cutover

  1. Use the selected Go 1.27.4 toolchain. Finish GOWORK=off go mod tidy after all imports and fixtures are in place and confirm GOWORK=off go mod tidy -diff is empty. Inspect go list -m all for the pinned modules, stable YAML, and absence of both legacy modules and the catalogs. Use go list -deps ./cmd/scriptorium to distinguish linked packages from graph-only tooling. Confirm no replacements, workspaces, vendored code, or dependency on unpublished sources.
  2. Run the baseline Go checks using prepared dependencies and the deterministic offline test suite: go test ./..., go test -race ./..., go vet ./..., and go build ./cmd/scriptorium. Put binaries outside the repository. Build the two existing Linux release targets (amd64 and arm64, CGO_ENABLED=0, -trimpath, -ldflags '-s -w') without running publication. Check gofmt and git diff --check.
  3. Use a freshly built binary for maintained config and render commands, including simple, full, composed with its local overlay, inherited, application source-list collection/overlay, and discovered collections. Exercise app inspection without a framework load and framework validate, show, sources, and explain with a real current path. Verify synthetic data only, one appropriate output on stdout, and no provider calls for render/inspection. Covered CLI/HTTP run tests should use a deterministic fake provider or injected client. Check shell examples and affected links, including pinned attribution targets.
  4. Review the complete diff against the feature roadmap: one native load per domain, ID-only execution, no legacy routes or translators, no extra inspection surface, optional discovery for both spellings, consistent disclosure and transport boundaries, and current docs. Check for stale active references rather than rewriting history or deleting deliberate rejection fixtures. Report verification results and any material blockers; do not convert these sprint checks into new permanent release gates.