Files
scriptorium/docs/roadmap/implementation.md

385 lines
22 KiB
Markdown

# ConfigKit v0.5.0 Implementation Plan
## Status, Baseline, And Authority
Planned on 2026-10-03 against Scriptorium
`8b6c12f912b28ac00016f047f9846798a5de95d9`. All five stages are pending.
The [feature roadmap](configkit.md) owns scope, intent, and the target state.
This plan gives a Luna coding agent concrete implementation steps and acceptance
checks. Implement Stage 1 through Stage 5 in order, one stage per work unit.
The baseline selects ConfigKit v0.4.2 and Promptkit v0.11.2. The target is the
published ConfigKit v0.5.0 at
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`.
Use its [configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/config.md)
and [consumer guide](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/consumers/README.md).
Inspect this exact revision through the downloaded module, a verified checkout,
or pinned upstream links. Do not adopt unpublished sibling changes.
The previous content of this filename is the completed policy-alignment plan:
`git show 8b6c12f912b28ac00016f047f9846798a5de95d9:docs/roadmap/implementation.md`.
Do not rerun it. The initial ConfigKit cutover is already delivered; this work
adds upstream inheritance to the existing integration.
## Execution Rules And Fixed Decisions
Before each stage, inspect status and relevant diffs, read
`docs/development.md` and `docs/policy/agents.md`, and select other policies
using the development guide. Architecture, testing, and OSS govern dependency
and integration work; documentation governs examples and prose; release governs
compatibility and the release handoff. Reuse unchanged instructions already in
context. Prefer the repository's code graph for symbol discovery.
Keep the following agreed choices throughout:
- Retain ordinary `configkit.Load`, the current `Document` schema and
definition callbacks, and the shared execution/inspection result. ConfigKit
already expands inheritance before decoding. Do not add an `Extends` Go
field, application merge pass, opt-in, keyword escape, or compatibility loader.
- Keep current discovery, explicit imports, selected overlay, CLI override
presence, working-directory-relative application paths, and normalization.
Backend conversion still uses resolved values and sorted IDs.
- A parent backend remains an ordinary configured backend. It is registered
and validated with the child. Inheritance copies settings; it does not create
a shared capacity pool across distinct backend IDs or abstract templates.
Existing sharing of each backend's capacity across server requests remains.
- Keep `api_key_env` as a visible environment-name string passed to Promptkit.
Do not resolve it through ConfigKit or mark it secret. Effective and historical
ordinary `extra_params` values stay visible under the maintainer's policy.
Existing raw-key restrictions and credential ownership remain.
- ConfigKit reserves `extends` across application input mappings, including
dynamic provider data and mapping names. Accept that upstream compatibility
change. It does not apply to separately parsed Promptkit definitions or HTTP
payloads.
- Underlays, native filesystem/mapping source APIs, source-relative resources,
schema versioning, secret bootstrap, and hot reload are deferred. Do not add
flags, HTTP configuration routes, source DTO fields, or APIs for them here.
- Keep Promptkit and other dependency versions, public HTTP behavior, accepted
ADR decisions, policy template baselines, release automation, and published
release notes unchanged except for the selected ConfigKit integration.
A concrete upstream defect should be reported with a reproducer; do not copy
its implementation into Scriptorium or assume a different unreleased target.
Production code changes are expected only if validation exposes a real
integration defect. Prefer the existing implementation when it already works.
Use small behavioral tests with real ConfigKit and Promptkit collaborators,
temporary files, and existing helpers. Do not add parser/inheritance matrices,
full prose snapshots, live provider calls, or test frameworks.
Each stage must leave a coherent, reviewable state. Record concise completion
evidence under that stage and commit its files with a short plain-English
message under the agent policy. Preserve unrelated user changes. Do not push,
tag, publish, or invoke the release command as part of this implementation.
For stages changing modules, Go code, or tests, run the development baseline
before committing: `go test ./...`, `go test -race ./...`,
`go vet ./...`, and `go build ./cmd/scriptorium`. Also run each stage's focused
checks and `git diff --check`; format changed Go files with `gofmt`.
Reuse successful checks for unchanged content. Documentation-only changes need
affected examples and link checks, not unrelated new runtime tests.
## Stage 1: Adopt The Published Dependency And Its Compatibility Notice
Status: Complete.
Completion evidence: selected ConfigKit v0.5.0 at
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`, updated its attribution and current
configuration compatibility documentation, and retained the single `Load`
pipeline. `go test ./...`, `go test -race ./...`, `go vet ./...`,
`go build ./cmd/scriptorium`, affected upstream link checks, and
`git diff --check` passed.
Files: `go.mod`, `go.sum`, `docs/oss/attribution.md`,
`docs/config.md`, `docs/development.md`, and
`docs/policy/architecture.md`.
1. Review the current `internal/config.Load`, `documentDefinition`,
`resolveDocument`, `validateDocument`, and `Settings`, plus CLI
`loadApplicationSettings` and `newEngine`. Confirm that the existing
ConfigKit lifecycle owns input assembly and no second merge is introduced.
2. Select only the published ConfigKit release:
```sh
go get gitea.maximumdirect.net/eric/ConfigKit@v0.5.0
go mod tidy
```
Review module and checksum changes. The YAML dependency remains
`go.yaml.in/yaml/v4 v4.0.0-rc.6`; keep Promptkit v0.11.2 and the selected
catalogs. Investigate incidental changes rather than silently including them.
Do not commit a workspace, replacement, or vendored dependency.
3. Inspect the selected upstream COPYRIGHT, LICENSE, and notices. Update only
ConfigKit's attribution entry to v0.5.0 and the full target commit in its
COPYRIGHT, LICENSE, and Source links. Retain its copyright statement,
LGPL-3.0-or-later expression, local LGPL/GPL links, and House Format.
Existing license texts remain applicable; no new policy-document copying
or attribution entry is needed merely to use the library.
4. Update the current ConfigKit version references in development, architecture,
and configuration documentation. Preserve policy Base Policy Version and
Upstream Source; update architecture's Last Updated date when editing its
local dependency statement. Do not revise shared policy language.
5. In `docs/config.md`, add a concise application inheritance subsection
linked to the pinned upstream contract. Explain `extends: sibling`,
inherited settings, explicit replacements, post-overlay/override expansion,
and `extends: null` cancellation. State that parents remain configured
backends, and the keyword applies beyond just the `backends` collection.
Include the reserved-key compatibility notice now so this dependency commit
does not leave accepted syntax incorrectly documented.
6. Explain that quoting does not escape `extends`. Distinguish renameable
application names from literal provider parameters whose names cannot be
changed without provider support. The current target has no escape or opt-out;
a concrete need should be handled upstream. Keep the Promptkit definition
boundary explicit. Stage 4 will add the example and remaining presentation.
Validation: run the development baseline, review module/version/notice changes,
and check affected documentation links. Existing tests are expected to pass;
update an expectation only when it encodes the selected upstream change.
Acceptance: ConfigKit v0.5.0 is selected, attribution identifies its actual
revision, and current documentation acknowledges the newly reserved syntax.
The application retains the existing single loading pipeline.
## Stage 2: Protect Application Loading, Presence, And Runtime Conversion
Status: Complete. Depends on Stage 1.
Completion evidence: configuration tests pass for inherited settings and
presence, `Origin.Inheritance` provenance, sorted runtime conversion, imports
and overlays before inheritance, CLI overrides, working-directory-relative
paths, cancellation, and structured reserved-key diagnostics. Focused and full
tests, race tests, vet, build, and `git diff --check` passed.
Files: primarily `internal/config/load_test.go` and
`internal/config/document_test.go`; production configuration files only if a
specific adoption defect requires a fix.
Use existing `loadDocumentBytes` and `writeApplicationFile` helpers and
`TestLoadComposesExplicitImportsAndSelectedOverlay` as patterns. Add the
following bounded scenarios rather than a general fixture framework.
1. Add a backend-inheritance test using an actual `Document`:
- Parent `base` has an endpoint, an environment-reference name,
concurrency limit 2, queue capacity 4, and dynamic parameters
`enabled: true`, `shared: visible`, and `count: 7`.
- Child `child` extends `base`, sets queue capacity 0, and overrides
`enabled` to false. Assert inherited endpoint/reference/limit and
retained shared/count values, plus explicit zero and false replacements.
- A second child extends `base` and sets queue capacity to null.
Assert the nullable pointer remains nil rather than inheriting 4.
- Convert through `Settings(result.Config)`. Assert that all declared
backend IDs, including `base`, survive in the existing sorted order,
inherited settings survive conversion, zero capacity has a non-nil pointer,
and the dynamic integer uses the existing `int64` representation.
2. Query `result.Metadata()` for the inherited child endpoint with
`configkit.Keys("backends", "child", "endpoint")`.
Check `IsExplicit` and `Lookup`, then verify a public
`Origin.Inheritance` step links the parent field, child field, and controlling
`/backends/child/extends` path. Do not assert exact location numbers,
internal traversal counts, or all history records.
3. Add a file-backed assembly test:
- Main input imports a fragment defining `base` and declares a child
extending it. Put a relative `prompt_dir` value in the fragment.
- A selected overlay changes the parent's endpoint and gives the child an
explicit queue capacity of 0. Supply an ordinary explicit CLI override
for `server.addr` through `LoadOptions.Overrides`.
- Assert the child inherits the overlay-adjusted parent endpoint, its own
zero wins, the CLI address wins, and the relative prompt directory keeps
the current cleaning/working-directory-relative representation rather
than becoming relative to the fragment.
- Reuse the same fixture with a cancelling overlay that sets
`extends: null` and gives the child its own valid endpoint. Check that
the child no longer inherits the parent's reference or capacity.
4. Add one reserved-key failure case through `Load`: under a valid backend,
set `extra_params.extends` to a missing sibling name. Require a nil result,
use `errors.As` to reach `*configkit.LoadError`, and find
`CodeInheritanceMissing` at the directive path in `Diagnostics()`.
This protects propagation of the compatibility change without re-testing
upstream's cycle, nesting, grammar, or expansion-limit matrices.
Validation: `go test ./internal/config`, development baseline, formatting,
and diff checks.
Acceptance: inherited input and explicit replacements retain their application
types and presence, source assembly precedes inheritance, structured failures
survive application wrapping, and `Settings` performs no hidden merge.
No new field ranges, secret handling, or source/path policies appear.
## Stage 3: Protect CLI Inspection And Promptkit Engine Adoption
Status: Pending. Depends on Stage 2.
Files: `internal/adapter/cli/config_test.go` and
`internal/adapter/cli/run_test.go`; CLI production code only if an actual
boundary defect is demonstrated.
1. Add a small file-backed parent/child fixture for CLI inspection, borrowing
existing `runCLICommand`, `writeAppConfigFile`, and inspection test
patterns. Omit framework source directories and run validation with the
referenced environment variable absent; use a synthetic name and value with
`t.Setenv` for the inspection checks. Check:
- `config validate` succeeds without credentials or a Promptkit engine.
- `config show` exposes resolved inherited endpoint/reference/ordinary
parameters and the child's false/zero values, and omits the structural
`extends` directive from effective backend data.
- `config explain --path /backends/child/endpoint` identifies parent,
destination, controlling directive, and effective endpoint. Use stable
semantic fragments rather than a complete text snapshot.
- Explanation of a replaced child parameter keeps the inherited and
replacement values visible. The environment reference remains visible;
synthetic API-key bytes never appear in successful output.
- `config sources` still uses its current JSON DTO; a one-file fixture
has one acquired file source and no fabricated inheritance source.
Parse structured output where useful; do not add output fields or promise
byte-stable upstream YAML/explanation wording.
2. Check the reserved-key failure from Stage 2 at the CLI boundary once:
`config validate` returns `ExitRuntimeError`, successful stdout is empty,
and stderr identifies the inheritance code/path through existing safe
diagnostic rendering. Avoid a complete error-string assertion.
3. Add a render integration test alongside
`TestRenderCommandUsesConfiguredCustomBackend`. Use a minimal synthetic
prompt/profile selecting a child backend. Define its endpoint on the parent
and only `extends` plus explicit replacements on the child.
Verify preparation succeeds and reports the child's backend ID and inherited
endpoint using the established prepared output contract. Stage 2 owns
detailed assertions on capacity and dynamic value conversion.
4. Demonstrate the parent is not treated as an abstract template: in a focused
variant, give the parent an invalid endpoint and the child its own valid
endpoint. Application loading may succeed, but existing `newEngine`
initialization must return an error identifiable as
`promptkit.ErrInvalidConfig`. Follow
`TestConfiguredBackendValidationComesFromPromptkit` for that boundary.
Do not add endpoint validation to ConfigKit or suppress parent registration.
Use render/preparation and real local libraries; no live provider calls or
generation fixture is needed. Reuse existing shared-server capacity tests
rather than adding a new concurrency matrix for copied backend settings.
Validation: `go test ./internal/adapter/cli`, development baseline,
formatting, and diff checks.
Acceptance: inspection and engine construction consume the same inherited
configuration, provenance is understandable, intentional visibility is retained,
and all configured backends follow existing registration rules.
## Stage 4: Deliver A Maintained Example And Finish Current Documentation
Status: Pending. Depends on Stage 3.
Files: new `examples/config.inheritance.yml`, `README.md`,
`docs/config.md`, `docs/cli.md`, `docs/internal/sources.md`, and
`docs/internal/adapters.md` only where its description needs correction.
1. Create `examples/config.inheritance.yml` with the existing example prompt,
profile, and schema directories. Define:
- `local-base`: localhost OpenAI-compatible endpoint, reference
`LOCAL_GPU_API_KEY`, concurrency limit 2, queue capacity 4, and an
illustrative `extra_params.provider_option: true`.
- `local-gpu`: `extends: local-base`, queue capacity 0, and
`extra_params.provider_option: false`.
The existing `examples/profiles/local-gpu.yml` selects the child ID.
Keep both parent and child valid ordinary backend definitions. Add a short
comment that application paths are relative to the repository invocation
directory. No new template collection or credential fixture is needed.
2. Link the new example from README's Examples list and the application
inheritance subsection in `docs/config.md`. Keep complete YAML in the
maintained file, not duplicated in prose.
3. In `docs/config.md`, finish the inheritance description started in Stage 1:
explicit zero/false/empty/null replacement subject to the schema, sibling
lookup, ordinary parent registration, eager resolution of effective
relationships, cancellation, and inspection. Link the pinned ConfigKit
contract for detailed rules. Retain migration guidance for literal keys
and named mapping entries, without implying a universal rename solution.
4. Add structural inheritance to `docs/cli.md`'s explanation-history
description. Commands and flags remain unchanged, and `sources` retains
its documented fields. Do not copy inheritance semantics into the CLI owner.
5. Update `docs/internal/sources.md` to describe post-assembly, pre-decode
inheritance and downstream runtime conversion. Link the owning config
reference. If needed, clarify native writer use in adapter internals; do
not duplicate the configuration contract.
6. Check current ConfigKit version references throughout nonhistorical docs.
Preserve old release records and explicitly historical cutover references.
Keep scope distinctions clear: ordinary application path values and
credential references retain their current meaning; Promptkit formats and
HTTP bodies do not acquire ConfigKit directives.
Run these example commands from the repository root:
```sh
go run ./cmd/scriptorium config validate --config examples/config.inheritance.yml
go run ./cmd/scriptorium config show --config examples/config.inheritance.yml
go run ./cmd/scriptorium config sources --config examples/config.inheritance.yml
go run ./cmd/scriptorium config explain --config examples/config.inheritance.yml --path /backends/local-gpu/endpoint
go run ./cmd/scriptorium render --config examples/config.inheritance.yml --prompt generic.markdown_summary --profile local-gpu --input transcript=./examples/fixtures/transcript.md --input glossary=./examples/fixtures/glossary.yml
```
The render command prepares output without calling a model. Inspect that its
selected backend is the child and the effective endpoint is inherited. The
sample parameter is illustrative, not a claim of support by a particular
provider. Validate changed local/upstream links and run diff checks.
Acceptance: a complete example works without credentials or provider access,
users can discover and inspect it, and current documentation describes
implemented native behavior without inventing stronger guarantees.
## Stage 5: Verify The Complete Work Set And Record The Handoff
Status: Pending. Depends on Stage 4.
Files: this plan and `docs/roadmap/configkit.md`; the historical-reference
portion of `docs/roadmap/policy.md`; other previously affected files only if
a correction is needed within this scope.
1. Review the complete diff against the baseline and check each roadmap
completion criterion. Confirm the dependency/notice revision, ordinary
`Load`, application presence/path/credential handling, inherited backend
mapping, inspection behavior, and maintained example.
2. Confirm no new wrappers, source APIs, redaction rules, HTTP contracts,
abstract backend behavior, cross-ID capacity pooling, unrelated dependency
upgrades, permanent validation gates, or release automation slipped in.
Correct actual divergence rather than weakening the selected tests.
3. Run final development validation if not already successful for the unchanged
Go/module/test content:
```sh
go test ./...
go test -race ./...
go vet ./...
go build ./cmd/scriptorium
git diff --check
```
Reuse Stage 4's successful commands and links when those files are unchanged.
Record actual checks and outcomes; do not claim the exploratory review
replaced final validation. No live API, CI polling, cross-platform expansion,
or publication command is needed to complete this work set.
4. Repair the completed policy roadmap's current `implementation.md` link:
replace it with the historical Git reference
`git show 8b6c12f912b28ac00016f047f9846798a5de95d9:docs/roadmap/implementation.md`.
This is a navigation correction, not a new policy adoption.
5. Mark stages complete here only after their acceptance checks pass.
Mark the ConfigKit feature roadmap delivered, retaining target-state rationale
and deferred scope. Do not put the ordered steps into that roadmap.
6. Record a concise release handoff in this plan: ConfigKit v0.5.0, backend
inheritance, globally reserved application `extends`, no keyword escape,
parent registration, and unchanged application path/credential ownership.
The next intended minor is currently v0.16.0. Version availability checks,
its checked-in release note with COPYRIGHT | LICENSE | ATTRIBUTION links,
roadmap retirement, tagging, and publication belong to separate release
preparation under `docs/release.md`. Do not create or publish a release here.
7. Review status and staged changes, commit the completed handoff, and report
completion, verification, commit IDs, and any remaining uncommitted work.
The implementation is complete independently of release publication.
Acceptance: all five stages have truthful completion evidence, the feature
roadmap reflects delivery, historical navigation remains accurate, and the
final application work matches the agreed scope.
## Open Questions
None within the accepted scope. Native inheritance and its reserved-key
compatibility change, visible ordinary parameters and reference names, existing
source/path policies, and deferral of new source APIs are already selected.
A future concrete requirement to pass literal provider `extends` data needs
upstream consideration; it is not an unresolved implementation decision for
this work set.