Files
scriptorium/docs/roadmap/implementation.md

26 KiB

ConfigKit v1.2.0 Interim Implementation Plan

Status, Baseline, And Authority

Planned on 2026-11-17 for a Luna coding agent against Scriptorium a5d8394313c4cbdee8e8618844e65364c70189e9. Stage completion is recorded below. The feature roadmap owns the target, rationale, native-semantics policy, and confirmed interim sequencing decision. This document owns the ordered implementation work.

This replaces the completed PromptKit v0.12.0 implementation plan, preserved at a5d8394313c4cbdee8e8618844e65364c70189e9:docs/roadmap/implementation.md. Do not execute that earlier plan again.

Adopt application go.maximumdirect.net/configkit v1.2.0 at c1c7e87f2cdf77a0cbf3327831ac05578e756b9e. Retain gitea.maximumdirect.net/eric/ConfigKit v0.5.0 at dc09959cca4e105f49ec316c6d4ba585bd2340bd where the unchanged gitea.maximumdirect.net/eric/promptkit v0.12.0 public API needs its types. Keep go-yaml v4.0.0-rc.6 and the existing PromptKit catalog selections.

Use ConfigKit's pinned consumer guide, configuration contract, and public root-package declarations. The sibling checkout currently contains v1.2.1; read the selected tag or downloaded v1.2.0 module, not unqualified sibling HEAD. Keep PromptKit behavior tied to its existing tagged contract.

Planning evidence: a temporary module resolved both ConfigKit identities, PromptKit v0.12.0, and the existing YAML prerelease. Package compilation with go test -run '^$' passed for all three libraries on Go 1.27.4. This does not constitute an application migration or test pass. Stable YAML cannot be selected in this interim graph because old ConfigKit imports its removed plugin/limit package.

Sprint Execution Rules

Implement Stage 1 through Stage 7 in order as one end-to-end sprint. Intermediate stages may fail compilation or tests and need not be independently deployable. Do not add compatibility layers, temporary alternate loaders, skips, or weakened tests just to make an intermediate stage green. Use focused checks as useful feedback, record failures accurately, and continue when the remaining work is expressly assigned to later stages. Investigate unrelated failures. Only the completed end state must pass the full validation.

Start with working-tree inspection and the task reading guides in docs/development.md. Follow the agent policy and applicable architecture, testing, documentation, OSS, and release policies. Use the graph for code discovery. Keep completed-stage evidence here concise and accurate; commit focused completed units under the agent policy.

The work is native ConfigKit adoption, not preservation of old processing semantics. Keep application schema, domain defaults, normalization, validation, path interpretation, and CLI selection choices with Scriptorium under ADR 0005. Imports, composition, inheritance, decoding, inspection, and processing order belong to the selected library. Do not introduce new restrictions, limits, identity rules, sensitivity heuristics, or guarantees because they are convenient to assert.

This sprint does not upgrade PromptKit, migrate its module path, select ConfigKit v1.2.1 or stable YAML, add configuration commands/endpoints, change resource paths to source-relative interpretation, or adopt ReleaseKit. HTTP request independence, shared backend capacity, artifact handling, credential disclosure, execution controls, and existing request/response contracts are not redesign targets. The existing restriction on committed workspaces, replacements, vendoring, and framework internals remains applicable.

Use real temporary files and the existing assembled collaborators for tests; provider interactions use existing fakes or loopback HTTP servers. Default tests stay offline and credential-independent. Prefer typed errors, diagnostic codes, effective values, and meaningful history over native prose snapshots or fixed source IDs. Reuse unchanged successful checks; do not run the entire baseline after every intermediate edit. Final validation is in Stage 7.

Stage 1: Select The Interim Dependencies And Go Toolchain

Complete. Go 1.27.4 and the selected dependency graph were provisioned from the local cache. Offline go list -m all confirmed both ConfigKit versions, PromptKit v0.12.0, the existing catalogs, and YAML v4.0.0-rc.6.

Work

  1. Select Go 1.27.4 for release validation and builds. It is available from the official Go downloads, and the official golang:1.27.4 image supports the existing Linux targets. Provision the toolchain and dependency cache before offline validation.
  2. Set the consumer directive in go.mod to go 1.27.0, as required by ConfigKit v1.2.0. Add the exact vanity module version without removing the old ConfigKit requirement or changing PromptKit, catalogs, or YAML. The application module path remains unchanged.
  3. Use ordinary Go module commands to update required sums. Defer final go mod tidy until source imports are migrated, so the newly selected dependency is not removed merely because it is not used yet. Do not add a tool entry for upstream maintainer tools or require Go 1.27.4 from all consumers through a new toolchain directive.
  4. Change only the Go build image in .woodpecker/release.yml from 1.26.11 to 1.27.4. Retain GOTOOLCHAIN=local, build targets, publication configuration, and asynchronous delivery behavior. No release-script gate or workflow redesign is needed.

Verification focus

Inspect go list -m all for the exact selected modules above. Tool-only graph requirements may appear without being runtime imports; inspect actual selection rather than delete them or upgrade unrelated modules speculatively. Compile failures from the remaining application migration belong to subsequent stages. Do not publish, tag, push, or invoke the release script.

Stage 2: Move Application Loading And Resolution To Native v1.2.0

Complete. Application loading uses native search and v1.2.0 resolution; go build ./internal/config passed with Go 1.27.4. Adapter and test migration remain assigned below.

Primary files: internal/config/document.go, load.go, and settings.go.

Work

  1. Change application ConfigKit imports to go.maximumdirect.net/configkit. Keep Document, its field schema, LoadOptions, AppSettings, and Settings responsibilities. Preserve fresh defaults, selector order, nullable retry presence, domain validation, and working-directory-relative application resource strings. Add no framework-resource acquisition to application validation.

  2. Replace the os.Stat loop in Load with source selection only:

    • explicit selection: configkit.File(options.Path), even for a blank explicit argument, leaving its failure to ConfigKit;
    • implicit selection: configkit.SearchConfig("scriptorium", configkit.SearchPaths(defaultConfigSearchPaths...)).

    Retain the existing ordered two-location slice and test substitution seam. Construct the search for each load. Remove the fileBacked bookkeeping, filesystem probing, and hand-written discovery error formatting.

  3. Call ordinary configkit.Load once with the existing definition, selected overlay, explicit overrides, and Imports: true. Leave import defaults, filesystem context, source-version policy, and sequence registrations unset. Optional discovery exhaustion follows native absent-main behavior.

  4. Keep the existing contextual load-error wrapper with %w so the new *configkit.LoadError remains discoverable. Do not recover by searching another file after a selected candidate fails.

  5. In resolveDocument, keep field-level Normalize calls. Remove the sourcePathsChanged aggregation and redundant generic whole-list resolution.Record call. Simplify the local path-normalization helper's boolean result if it has no other caller. Use RecordNormalization only for actual direct mutations requiring it; no replacement container record is necessary for today's same-length list-field edits.

  6. Preserve ordinary native control and collection handling. Add neither legacy control aliases nor application filtering of declared imports, inference of collection identity, or a second composition pass.

Verification focus

Review the loader against the selected public lifecycle and the resolver against v1.1+ leaf recording. Existing unit tests still require import/syntax changes in Stage 4. Do not broaden LoadOptions or introduce a filesystem/merge framework to preserve the old test setup.

Stage 3: Separate Application And Framework Types In CLI Adapters

Primary files: internal/adapter/cli/application_config.go, config.go, run.go, inspect.go, and framework_config.go.

Use configkit for the new application package and legacyconfigkit for the old package where both are imported.

Boundary ConfigKit owner
Application overrides, paths, results, stored diagnostics, and writers Vanity v1.2.0
newEngine input/underlay construction and promptkit.DirectoryInputs values Old-path v0.5.0
Captured framework paths, native source records, diagnostics, and writers Old-path v0.5.0

Work

  1. Migrate applicationOverrides, loadApplicationSettings, and writeApplicationWarnings to new public types and writers. Migrate stored application diagnostics in run, serve, common-command, prompt/profile inspection, and framework-command selection structs. The latter's overrides and pre-engine diagnostics are still application-owned.
  2. Keep newEngine's Input, Underlay, and File calls old-typed. Continue passing promptkit.DirectoryInputs output directly to that old boundary. Keep source ordering, underlay grouping, retry defaults, and shared server engine construction unchanged.
  3. configCommand uses new paths and writers. Its existing configurationSource DTO uses the new Layer type, yielding main for application file sources. Keep its JSON shape and empty-array behavior.
  4. Framework explain parses paths with legacyconfigkit.ParsePath and passes them to the existing view. Keep frameworkConfigurationSource, frameworkConfigurationSources, Layer, and SourceKind old-typed; it is already a separate DTO, so no new shared conversion type is needed.
  5. In inspectFrameworkConfigurationCommand, retain the new application warning call for cfg.diagnostics. Send view.Diagnostics() directly to legacyconfigkit.WriteDiagnostics, propagating writer failure through the existing runtime-error exit. Keep native framework output and its established scoped redaction.
  6. Preserve structured error presentation for both generations. In the existing common writeApplicationError helper, check for the new *LoadError and then the old *legacyconfigkit.LoadError with errors.As, sending each one's diagnostics to its matching writer. Retain current handling of diagnostic-write failure and the fallback for non-load errors. This small explicit dispatch preserves errors from engine construction as well as application loading; it is not a cross-version data-conversion layer.
  7. Audit actual ConfigKit references by owner rather than globally replacing imports. No casts, YAML/JSON round trips, or duplicate configuration result representations are needed between these generations.

Verification focus

Check both native writer signatures and all CLI construction paths. Compilation provides useful feedback, but stale tests/fixtures remain assigned below. Neither successful application configuration validation nor its inspection should construct a PromptKit engine.

Stage 4: Migrate And Extend Application Configuration Tests

Primary files: internal/config/document_test.go and load_test.go.

Work

  1. Migrate their imports, error assertions, paths, and LayerBase references to the new package and LayerMain. Change application input controls from configkit to config, marking indispensable imported fixtures required: true. Do not edit framework-definition fixtures as if they were application documents.
  2. Retain and adjust existing discovery/defaults tests. Cover the first candidate, a missing first candidate selecting the second, optional exhaustion, explicit missing/blank selection, and a malformed or nonregular selected candidate that does not fall through to a valid second file. Reuse useConfigSearchPaths; tests that mutate that package state remain nonparallel and restore it through cleanup.
  3. Update TestLoadDiscoveryErrorQuotesPath to inspect a typed load error's rendered diagnostics, not demand that LoadError.Error() contain a path. Keep the real protection: terminal control characters in a failing path are escaped by the native writer. A deterministic nonregular candidate can be used instead of requiring host symlink support.
  4. Rename/reframe TestLoadRejectsOverlayWithoutBase around this application's absent-main/no-context selection. It should still fail for an active overlay, while default selects none. It must not claim ConfigKit universally prohibits overlays without a main or imports on an absent main.
  5. Add focused import coverage: missing optional input succeeds with omitted requiredness, the corresponding required input fails, and the removed optional field fails. Use one dir import with an authored YAML pattern and destination: [server] to mount a fragment containing addr. Verify final application values and an applicable origin, not a complete upstream traversal/filter matrix.
  6. Demonstrate native collection policy on the existing selector list with YAML config.sequences, destination: [promptkit_sources], key: file, and mode: upsert. Use only file-selector records in that fixture. Distinct imported peers combine; a duplicate authored identity conflicts; an overlay appends a new identity while retaining lower records. Contrast unkeyed whole-list replacement and explicit CLI whole-list replacement. These are authored policies, not Go identity registrations.
  7. Cover changed CLI timing with a paired case: a source value with a decoding type error still fails despite a replacing override; a decodable value failing application validation can be corrected by an override. Retain explicit zero/false/empty/allowed-null tests where applicable to the schema and callbacks; create no new fields solely to assert all categories.
  8. Keep effective-value, source-order, snapshot, and normalization coverage. For a changed selector leaf, assert its normalized result and recorded normalization history; for an unchanged leaf, assert no invented operation. Avoid testing incidental history counts or private callback choreography.

Verification focus

Run go test ./internal/config when useful. Preserve distinct regression protection already present; consolidate overlapping assertions rather than clone ConfigKit's complete format suite.

Stage 5: Migrate CLI Integration Fixtures And Maintain Runnable Examples

Primary files: CLI configuration/execution/inspection tests and examples/. Review HTTP/report test helpers only for their ConfigKit owner.

Work

  1. In CLI tests, use new ConfigKit for application-load errors/diagnostics and old ConfigKit for framework engine setup and captured view records. Specifically review run_test.go, framework_sources_test.go, and framework_config_test.go. Keep the old imports in the HTTP newHandlerEngineFromDirectories helper and report engine-construction test, because they construct PromptKit sources rather than load application settings.

  2. Migrate application control literals in config_test.go and run_test.go to config.imports, with explicit requiredness when absence should fail. Retain framework controls in the generation selected by PromptKit v0.12.0. Keep pre-existing backend-inheritance coverage in the framework, where those backend definitions now live.

  3. Exercise the Stage 3 dispatch using real failed new application loads and failed old framework loads. Keep stdout clean on failures, native diagnostics on stderr, and diagnostic/output writer-failure coverage. Existing framework show, sources, and explain tests protect the old path, records, and redaction; update only expectations actually owned by new application loading.

  4. Protect an assembled path that loads new application imports and an overlay, then renders through unchanged PromptKit definitions. Include a framework file with its supported old import controls so blanket control renaming would be detected. Continue application-only inspection coverage with nonexistent framework resources to prove it does not acquire them.

  5. Change examples/config.composed.yml to root config.imports and add required: true to its source-selection fragment. Other ordinary application and framework examples change only when their actual owner requires it.

  6. Add this concrete, small collection example using existing framework files:

    • examples/config.collections.yml: ordinary application main; import directory config-fragments/collections with pattern: "*.yml" and required: true; declare promptkit_sources identity file and mode: upsert under root config.sequences; set schema_dir: ./examples/schemas.
    • In that imported directory, 10-backends.yml, 20-profile.yml, and 30-prompt.yml each contribute one file selector, respectively ./examples/framework/backends.yml, ./examples/profiles/local-fast.yaml, and ./examples/prompts/generic.markdown_summary.yaml.
    • examples/overlays/collection.yml adds one file selector for ./examples/profiles/local-gpu.yml. The inherited upsert policy retains the three lower selectors and appends the new one.

    No new id field, default registration, runtime merge logic, or copied framework definition is needed. Application path strings remain relative to the repository invocation directory.

  7. Exercise the maintained collection main and overlay through the application load and at least one no-generation render with the transcript and glossary fixtures. Compare effective selector order/retention and a real prepared result, not a giant native-text golden. Reuse tests from Stage 4 where their risk is already covered.

  8. Finish go mod tidy after the imports are in place. Confirm the exact interim versions and distinguish tool-only module requirements from the runtime graph. Leave implementation of v1.2.1 and the upcoming PromptKit roadmap to the separate work set.

Verification focus

Run affected CLI tests and the existing HTTP/report suites as warranted by changed imports. All fixtures must eventually compile with both identities. New source-list composition must reach the old framework boundary in order; do not sort or deduplicate it to repair a test.

Stage 6: Update Canonical Documentation And Both ConfigKit Attributions

Work

  1. Update docs/config.md for application ConfigKit v1.2.0 and native discovery. Describe the actual processing order, optional import default, explicit required, expanded selectors/placement, authored collection policy, and CLI timing. Correct unconditional statements that overlays always replace lists and defaults-only loading cannot enable imports. Explain that this CLI supplies no independent overlay root; the context limitation is not a universal library prohibition.
  2. Clearly distinguish application root config controls from the unchanged PromptKit v0.12.0 framework contract. Keep working-directory-relative resource values, strict application schema, and current credential handling. Link to the pinned upstream contract for detailed grammar instead of copying it.
  3. Update docs/cli.md for application main provenance and changed application loading behavior. Keep source JSON fields, command flags, streams, and exits otherwise unchanged; framework inspection continues to use its own old native types and writers. Add no new source-detail or diff command.
  4. Update README example navigation and the owned configuration examples. Add copyable inspection and render invocations for the collection example. Use --config-overlay collection for its added selector and both existing input fixtures for generic.markdown_summary; no provider connection or credential is needed for these render checks.
  5. In docs/development.md, document Go 1.27.0 as the consumer floor because of application ConfigKit, and point release builds to Go 1.27.4. Update docs/release.md's selected toolchain, installation path example, and version output to 1.27.4, matching the image from Stage 1. Leave release commands, gates, footer conventions, and publication completion unchanged.
  6. Update docs/internal/sources.md, docs/internal/adapters.md, and the overview as affected: native discovery, two ConfigKit generations, leaf resolution recording, warning/error dispatch, and actual selector conversion. Remove the obsolete claim that application backend IDs are sorted. Update only the architecture policy's affected dependency/boundary description and local date; preserve its base version, upstream provenance, and adopted decisions. Accepted ADR content stays unchanged.
  7. In docs/oss/attribution.md, retain a distinct entry for old-path ConfigKit v0.5.0 used by PromptKit and add application ConfigKit v1.2.0. Use factual headings/identification to distinguish them, the actual copyright and LGPL-3.0-or-later grant, and shared local LGPL/GPL text links. New ConfigKit source/notice links use forge commit c1c7e87f2cdf77a0cbf3327831ac05578e756b9e; legacy ConfigKit uses its retained commit dc09959cca4e105f49ec316c6d4ba585bd2340bd at that same canonical ConfigKit repository. Verify notice files at both revisions.
  8. Preserve unchanged PromptKit, catalog, YAML prerelease, and other dependency notices. Inspect newly selected runtime material and add notices only if actually incorporated or distributed; upstream ReleaseKit, Goldmark, and Go module tooling requirements alone are not runtime inclusion. No project relicense, new standard-text copies, or tool adoption follows automatically.
  9. Keep the feature roadmap as target/context rather than a duplicate step log. Preserve the explicit v1.2.1/stable-YAML/old-module-removal handoff to the next PromptKit roadmap. Do not overwrite the completed historical PromptKit roadmap or invent a release version to create a versioned release note.

Verification focus

Verify current facts against implementation, affected local links and pinned source paths, license-text fidelity, and changed example commands. Preserve published release notes and tags. The eventual new minor release and its note remain separate preparation; this stage does not authorize publication.

Stage 7: Validate The End State And Record The Handoff

Work

  1. Review the entire diff for unintended contract strengthening or obsolete compatibility logic: native search rather than probes; default optional imports; no default selector identity; post-decode CLI assignment; leaf normalization; no new redaction, filesystem policy, or framework processing. Confirm all imports, paths, diagnostics, and writers belong to their owner.

  2. Inspect go list -m all and go list -deps ./cmd/scriptorium under the selected release toolchain. Verify both ConfigKit identities at their exact versions, unchanged PromptKit/catalogs/YAML, no replacements/workspaces, and the intended runtime versus tool-only dependency distinction. Do not freeze the entire dependency inventory into a new permanent test.

  3. Format changed Go code and run the existing development baseline once against the final source: go test ./..., go test -race ./..., go vet ./..., and a native go build of ./cmd/scriptorium with output outside the repository. Use existing offline/provider-independent fixtures; loopback test servers are local test infrastructure, not live probes.

  4. Check both release targets with the existing build settings. With Go 1.27.4 already selected and the dependency cache provisioned, the following are validation commands, not publication:

    validation_dir=$(mktemp -d)
    export GOWORK=off GOTOOLCHAIN=local GOFLAGS=-mod=readonly
    export GOPROXY=off GONOPROXY=none GOSUMDB=off
    CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -ldflags '-s -w' \
      -o "$validation_dir/scriptorium-linux-amd64" ./cmd/scriptorium
    CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -trimpath -ldflags '-s -w' \
      -o "$validation_dir/scriptorium-linux-arm64" ./cmd/scriptorium
    

    Keep products outside the worktree. Do not invoke scripts/release.sh, push, tag, poll hosted delivery, or add a new release check.

  5. Run the maintained ordinary, composed, inheritance, and new collection no-generation examples; check collection overlay retention, representative application inspection, and old framework inspection. Reuse equivalent unchanged successful checks. Run git diff --check and validate affected documentation links and workflow YAML/shell syntax.

  6. Audit every roadmap completion criterion against code, tests, examples, metadata, and documentation. Fix remaining gaps instead of marking a partly migrated but green subset complete. Do not carry old PromptKit stage records forward as evidence for this sprint.

  7. Record concise actual validation results and mark stages complete only when their work is done. Update the feature roadmap status after the complete audit, retaining its scope and later-upgrade handoff. Commit remaining scoped changes with a plain-English message, review status, and report residual uncommitted work or blocked validation accurately.

Completion means application ConfigKit v1.2.0 is native and operator-visible where already supported, PromptKit v0.12.0 still works through old ConfigKit, the shared YAML prerelease remains selected, and all seven stages are delivered. It does not mean ConfigKit v1.2.1, stable YAML, or a new Scriptorium release has been delivered. No significant product questions remain open for this sprint.