Files
scriptorium/docs/roadmap/configkit.md

355 lines
23 KiB
Markdown
Raw Normal View History

2026-11-17 20:27:12 +00:00
# ConfigKit v1.2.0 Interim Adoption
## Status And Intent
2026-11-17 20:27:12 +00:00
Drafted on 2026-11-17. The maintainer selected an interim ConfigKit v1.2.0
upgrade, followed by ConfigKit v1.2.1 as part of the upcoming PromptKit upgrade.
This roadmap describes that interim target and the changes needed to reach it;
it is not an implementation plan or a claim that the upgrade has landed.
2026-10-03 17:16:02 +00:00
The maintainer is adopting ConfigKit across an application suite to standardize
2026-11-17 20:27:12 +00:00
configuration semantics. Scriptorium should use the selected ConfigKit's native
loading path, rather than reproduce older processing rules for compatibility.
Configuration migration is acceptable. Application schema and deployment choices
remain Scriptorium's responsibilities under
[ADR 0005](../adr/0005-configkit-boundary.md).
This replaces the completed v0.5.0 roadmap, preserved at
`ba24b7a879c95e2b306e0d53eb0b45e043e1b29a:docs/roadmap/configkit.md`.
The existing [implementation document](implementation.md) concerns the previous
PromptKit work set and is not the implementation plan for this upgrade.
## Reviewed Baseline And Target
The reviewed Scriptorium revision is
`ba24b7a879c95e2b306e0d53eb0b45e043e1b29a`. Its selected modules are
`gitea.maximumdirect.net/eric/ConfigKit v0.5.0` and
`gitea.maximumdirect.net/eric/promptkit v0.12.0`, with
`go.yaml.in/yaml/v4 v4.0.0-rc.6`. The consumer Go minimum is 1.25.5; release
validation and hosted builds select Go 1.26.11.
Application configuration already uses ConfigKit through `internal/config`:
- `Document`, a fresh defaults factory, a recorded resolver, and a validator
define application settings. `Settings` converts the resolved result without
another merge.
- `Load` selects an explicit file or runs a local `os.Stat` loop over two system
configuration paths, falling back to `NoSource`. File-backed loads enable
imports, one selected overlay, and explicit application CLI overrides.
- Application inspection reuses the retained result. `show` and `explain` use
native writers; `sources` projects `id`, `label`, `layer`, and available
`path` into a Scriptorium-owned JSON array.
- Application path values are working-directory-relative. Framework source
selectors are translated separately into ConfigKit values accepted by
PromptKit's current public API.
The interim target is `go.maximumdirect.net/configkit v1.2.0` at full commit
`c1c7e87f2cdf77a0cbf3327831ac05578e756b9e`. The locally available v1.2.0 tag and
remotely verified published tag identify that same commit. Canonical references are:
- [Consumer adoption and migration guide](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/consumers/README.md).
- [Configuration contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/config.md).
- [v1.2.0 release note](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/releases/v1.2.0.md).
- [v1.0.0 breaking-change inventory](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/releases/v1.0.0.md).
The interim target requires Go 1.27.0 and retains go-yaml v4.0.0-rc.6. PromptKit
stays at v0.12.0 and continues selecting old-path ConfigKit v0.5.0. ConfigKit's
code grant remains LGPL-3.0-or-later. This review examined release notes, public
contracts, Scriptorium's integration, and a focused dependency compatibility
experiment; it did not migrate or validate the full application against v1.2.0.
## Intervening Release Review
All published releases after the selected v0.5.0 through v1.2.1 were reviewed.
Notes through v1.2.0 are linked at the interim target revision. The v1.2.1 note
is linked at its own commit and describes deliberately deferred work.
| Release | Consumer-facing change and adoption consequence |
| --- | --- |
| [v0.6.0](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/releases/v0.6.0.md) | Loading/inspection improvements and Go 1.26.0 minimum. Obtain improvements through the dependency; no Scriptorium performance guarantee or benchmark gate follows. |
| [v0.7.0](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/releases/v0.7.0.md) | Effective-configuration `Diff` with combined sensitivity protection. Useful future operator functionality, not required for native loading. |
| [v0.7.1](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/releases/v0.7.1.md) | Maintainer-tool update; no application runtime or configuration migration. |
| [v0.8.0](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/releases/v0.8.0.md) | Input destinations, sequence-item placement, shared source-version checks, and initial identified sequences. Authored imports can use applicable placement; application version policy and Go identity registrations remain opt-in. |
| [v0.9.0](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/releases/v0.9.0.md) | Expanded imports/defaults, author-declared collection operations, identified-record inheritance, and dependency-ready ordering. Imports become optional by default; CLI assignment moves after inheritance and typed decoding. Migrate old syntax and lifecycle assumptions. |
| [v0.10.0](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/releases/v0.10.0.md) | Native configuration discovery and expanded absent-main loading. Replace application filesystem probing with a ConfigKit search description. |
| [v0.10.1](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/releases/v0.10.1.md) | Internal consolidation without public migration; inherit the fixes rather than reproduce the machinery. |
| [v0.10.2](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/releases/v0.10.2.md) | Acquisition, cleanup, projection-limit, and collection fixes. Accept corrected upstream diagnostics without adding application limits or recovery machinery. |
| [v1.0.0](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/releases/v1.0.0.md) | New module identity, Go 1.27.0, source-root `config` controls, API cleanup, `main` provenance, source lookup, and inspection fixes. These require migration. |
| [v1.1.0](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/releases/v1.1.0.md) | Field-level resolution recording within lists and identity-aware history for registered collections. Remove redundant whole-list normalization recording when leaf records cover all edits. |
| [v1.2.0](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/releases/v1.2.0.md) | Main-relative layered companions and derived/fallback overlay contexts. Available for future multi-source application needs; ordinary `Load` remains appropriate today. |
| [v1.2.1](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/releases/v1.2.1.md) | Stable go-yaml v4 and loader-plugin adaptation. Deferred to the PromptKit upgrade because of the shared dependency obstacle below. |
Upstream release-tool changes do not require Scriptorium to adopt ReleaseKit or
copy upstream publication procedures.
## Dependency Boundary And Sequencing
There are two distinct compatibility issues, not just import renaming:
1. Old and vanity ConfigKit paths have different named Go types. `newEngine`
currently constructs old-path `Input` and `Underlay` values for PromptKit.
New-path application overrides and results cannot simply replace those
types at that boundary.
2. Both paths select the same go-yaml v4 module through Go's module graph.
ConfigKit v1.2.1 requires v4.0.0. Old ConfigKit v0.5.0 imports
`go.yaml.in/yaml/v4/plugin/limit`, which is absent from stable v4.
An isolated temporary module requiring only old ConfigKit v0.5.0 and go-yaml
v4.0.0 failed a package compile check:
`go test -mod=mod -run '^$' gitea.maximumdirect.net/eric/ConfigKit`.
Package setup failed at that missing import; the selected module graph was
verified. This is a compile-time dependency conflict, not evidence of a
Scriptorium runtime failure. Repository source and dependency files were untouched.
Retaining old-path adapter imports alone cannot bridge v1.2.1, but the selected
v1.2.0 target still declares the same YAML prerelease as the existing stack.
The interim graph deliberately contains both ConfigKit module identities:
- Application loading, overrides, callbacks, diagnostics, and application
inspection use `go.maximumdirect.net/configkit v1.2.0`.
- Framework construction and captured framework inspection retain
`gitea.maximumdirect.net/eric/ConfigKit v0.5.0` wherever PromptKit v0.12.0's
public API requires its types.
- Both share `go.yaml.in/yaml/v4 v4.0.0-rc.6`. Validate the complete selected
graph, including transitive requirements; agreement of direct requirements
alone is not proof that the application works.
Use clear local import aliases where both packages are needed. Build each
owner's values directly with its own public types; serialization or a new
cross-version facade is unnecessary. Framework JSON Pointer parsing and writers
must use the old owner's types, while application diagnostics use the new
owner's types. Review shared source DTOs for named layer-type assignments.
Runtime `AppSettings` and application-owned selector values can continue to
cross the existing adapter boundary without transporting a ConfigKit result
into PromptKit.
The upcoming PromptKit roadmap owns migration to a compatible framework release,
ConfigKit v1.2.1, stable YAML, and removal of old ConfigKit imports and notices
when no longer used. PromptKit v0.23.2 already demonstrates that dependency
combination, but selecting and assessing its full migration remains later work.
This interim upgrade uses published dependencies without local replacements or
copied loaders under the existing architecture policy.
The two loaders deliberately have different format generations during this
interval. Migrate application documents and their imports to v1.2.0 controls;
framework files selected for PromptKit remain governed by its v0.12.0 contract,
including any old ConfigKit import controls it supports. Do not bulk-rename
controls across all YAML files or parse framework documents with the new
application loader.
2026-10-03 17:16:02 +00:00
## Targeted End State
2026-11-17 20:27:12 +00:00
### One Native Application Loading Path
`internal/config` consumes the public `go.maximumdirect.net/configkit` package.
All command families use one definition and one completed application load per
selection. Runtime settings come from `result.Config`; inspection uses that
result's captured public views.
Ordinary `configkit.Load` remains the entry point. A main file with imports and
an overlay does not require artificial underlay groups, embedded defaults, or a
new configuration abstraction. Fresh typed defaults, recorded normalization,
and final domain validation remain definition callbacks.
### ConfigKit-Owned Discovery
An explicit `--config` selects `configkit.File` exclusively. Omission selects
`SearchConfig` with ordered `SearchPaths` for
`/usr/local/etc/scriptorium/config.yml` and `/etc/scriptorium/config.yml`.
Delete the `os.Stat` loop and hand-written discovery errors; ConfigKit owns
selection, acquisition, absence, and structured diagnostics.
These locations are application-selected deployment conventions, not a
compatibility implementation of library processing. ConfigKit does not mandate
one search convention. Its directory presets consider additional filenames and,
in some cases, platform constraints; adopting those or adding user/home/current
directory discovery is a separate application choice.
Use an optional search and native defaults. Exhaustion contributes no fabricated
source; a selected-file error terminates loading. Enable imports directly:
an absent main has no declarations to traverse, so no existence precheck is
necessary.
Keep the overlay-name selector and delegate its processing. The current CLI has
no independent overlay filesystem root; without a selected main or explicit
context, an active overlay cannot be located. Report ConfigKit's outcome rather
than create a permanent application prohibition on overlays without a main.
The library supports that combination when an appropriate context is supplied.
### Native Source Controls, Imports, And Collections
Application sources use root `config` controls, including fragments and selected
overlays where their controls are allowed. Rename genuine old control blocks in
maintained application examples and fixtures. Do not introduce aliases or
runtime rewriting. Nested `config`, and former `configkit` and `_configkit`
names, follow ordinary schema rules rather than a global reserved-key rule.
Leave `ImportDefaults` unspecified unless a future application requirement
warrants a choice. In particular:
- Imports are optional by default. Indispensable maintained fragments declare
`required: true`; intentionally optional imports may omit it. Remove old
`optional` declarations rather than translate them at runtime.
- Native `path`, `dir`, and `glob` selectors, lists, filters, recursion,
destinations, and item placement are available wherever ConfigKit allows.
Directory filename selection is unrestricted by default; YAML-only examples
should author a pattern. Hidden/symlink discovery follows native defaults,
not application filtering.
- Strict peer composition, dependency-ready assembly, inheritance, and
collection policy remain upstream responsibilities. Sources can declare
`config.sequences` for native identity, append, upsert, and replacement.
- `promptkit_sources` has no application identity field. Add no identity
registration, inferred path identity, duplicate prohibition, or forced
replacement solely for this upgrade. Ordinary unkeyed lists replace across
precedence boundaries; authored collection policy may change that behavior.
CLI lists remain whole-list replacements after decoding.
Import and overlay containment remain library behavior. Application resource
strings such as `schema_dir`, framework selectors, and HTTP artifact roots are
not source-control paths; ConfigKit does not automatically rebase them. Keep
their working-directory-relative interpretation in this scope. Source-relative
resource behavior would require a separate application choice using public
origin/context APIs. No broader filesystem sandbox is introduced.
### Current Resolution And CLI Lifecycle
Adopt assembly, inheritance, typed defaults/decoding, explicit CLI assignment,
recorded resolution, credential processing where applicable, and validation in
the current upstream order. Source syntax, composition, inheritance, and decoding
failures cannot be repaired by later CLI overrides. Valid decoded values can
still be replaced before application validation.
Continue mapping only explicitly visited flags to public `Override` records,
with safe labels and supported built-in values. Preserve explicit empty, false,
zero, and allowed null values according to the application schema. Recorded
normalization may change those values through native helpers.
Migrate used removed symbols, especially the resolver's generic `Record` call,
to current APIs. Leaf `Normalize` calls now cover individual edits within the
equal-length `promptkit_sources` list; a second whole-list record is unnecessary.
Do not add identity merely to obtain field auditing. Future membership/order
changes use the applicable native container-recording rules.
The v1 inventory also includes `EntryInfo`, `Sensitive`,
`SourceVersionModeMainOnly`, `SourceVersionEvidence`,
`EmptyImportSelections`, `DestinationIsFallback`, `SearchPaths`,
`FollowSymlinks`/`IncludeHidden`, `LayerMain`, and
`CodeHistoryLimitExceeded`. Update actual uses and tests; this list does not
require implementing features that Scriptorium does not use.
### Inspection And Disclosure
Keep `config validate`, `show`, `sources`, and `explain` as the application
inspection surface. Native writers own effective YAML and explanation semantics,
including improved list-field history. Retain the small application source DTO;
its main layer now reports `main`, not `base`. Acquisition order is not
composition order, and load-local source IDs are not durable identities.
Source lookup, empty optional-selection evidence, and placement metadata remain
available through public results for future focused operator enhancements.
Adoption does not require dumping every source field into the CLI schema.
Existing stream separation, writer-error handling, and application-versus-framework
inspection boundaries remain applicable.
The application document has no raw API-key field. Migration adds no sensitivity
heuristics, redaction of environment names, or redaction of ordinary provider
parameters. API-key values remain the secret requiring protection; PromptKit
owns framework credential handling. The accepted scoped native redaction of
framework configuration inspection is separate from application inspection.
The renamed `Sensitive` hook does not establish a broader disclosure policy.
## Required Changes And Supporting Material
- Update dependency/import identity, used symbols, discovery construction,
normalization records, diagnostics, and application/CLI fixtures. Keep the
PromptKit-owned old types isolated from new application types and retain the
shared YAML prerelease in the selected graph.
- Raise `go.mod`'s consumer minimum to 1.27.0 and select a supported Go 1.27
release patch for validation and builds. Update current 1.26.11 release
instructions and the hosted image together; no higher consumer patch minimum
or extra platform matrix is implied.
- Migrate application composition examples to `config.imports`, with explicit
requiredness where they depend on a fragment. Demonstrate one useful native
collection composition on the selector list without adding a schema identity
field or universal registration.
- Update configuration/CLI contracts, contributor/toolchain guidance,
architecture dependency descriptions, source/adapter internals, and affected
README/example navigation. Correct obsolete source-internals text about
sorting application backend IDs: those backends were removed and selector
order is meaningful. Preserve accepted ADR decisions and historical notes.
- Update ConfigKit attribution to identify both the new application module at
v1.2.0 and the old module at v0.5.0 still used with PromptKit. Preserve each
revision's applicable notices and exact-commit links, using separate factual
component entries where needed. Leave unchanged PromptKit and YAML selections
and notices accurate; do not prematurely claim stable YAML adoption. Inspect
other affected selected runtime dependencies under the OSS policy.
- Preserve existing faithful LGPL/GPL and other applicable standard texts;
verify changed upstream project notices under the OSS policy. Maintainer
tool requirements may affect module selection without becoming runtime
imports. Their presence alone requires neither Scriptorium tool adoption nor
attribution for tools not incorporated into its distributed material.
The eventual release should be a new minor version under the adopted release
policy: consumer minimum and configuration contracts change. Notes should
explain control-block migration, required imports, CLI ordering, provenance
naming, and the intentional application/framework version split. Exact version
selection and publication remain separate preparation work.
## Validation And Completion Criteria
The eventual implementation is complete when:
- The published graph selects application ConfigKit v1.2.0, old-path ConfigKit
v0.5.0 for PromptKit v0.12.0, and go-yaml v4.0.0-rc.6. It builds with each
boundary using its owner's public types and without a cross-version shim.
- Explicit-file, native discovery, exhaustion/defaults-only, and selected-file
error cases use one native load. Inspection and execution share the
application definition and resolution boundary.
- Representative application tests cover new controls, optional/required
imports, an applicable expanded selector/placement, native collection and
overlay behavior, and changed CLI timing. Distinguish decode failures from
domain validation that CLI values can legitimately affect.
- Leaf normalization, explicit zero/false/null handling, `main` provenance,
and selector-list explanation work through the public result and CLI without
duplicating upstream's exhaustive test matrix.
- Render and representative CLI/HTTP execution continue through PromptKit
v0.12.0, including shared capacity and existing artifact handling. Application
v1.2.0 controls and inspection do not reinterpret framework configuration as
if its independent loader had also been upgraded.
- Documentation, examples, attribution, and toolchains match the implemented
graph and semantics. Run applicable development checks and existing release
target builds. Add no recurring synchronization checks, scanners, provider
probes, performance promises, or publication gates merely for this upgrade.
No implementation or full target-graph test pass is claimed by this review.
## Plausible Future Work Outside This Adoption
The upcoming PromptKit upgrade includes ConfigKit v1.2.1 and stable go-yaml,
unifies ConfigKit's module identity, and retires this temporary framework
boundary. Completing this v1.2.0 roadmap does not require that later work.
`config diff` could expose native effective-value comparison for two files or
an overlay's impact using safe snapshots, not rendered-text comparison. It is
useful operator work, but not necessary to standardize existing loading semantics
and remains a separately scoped enhancement.
Layered application underlays, main-relative companions, bundled defaults,
additional discovery locations, source-relative resources, source-detail
inspection, and source-version policy remain available for concrete future
needs. This scope also leaves HTTP configuration inspection, hot reload, and
secrets-directory bootstrap for separate consideration. None is permanently
prohibited by this roadmap.
## Confirmed Sequencing Decision
The maintainer selected v1.2.0 as the complete target of this work set so native
application adoption can precede the larger PromptKit migration. The tradeoff is
temporary coexistence of two ConfigKit generations and deferred stable YAML.
A coordinated cutover to both latest dependencies was considered; it avoids
that intermediate state but would combine the larger framework API migration
with application configuration work. The selected scope keeps those work sets
separate and carries an explicit retirement handoff to the PromptKit roadmap.
No significant product decisions remain open within this interim scope.