From 8b6c12f912b28ac00016f047f9846798a5de95d9 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sat, 3 Oct 2026 17:16:02 +0000 Subject: [PATCH] Plan ConfigKit inheritance adoption --- docs/roadmap/configkit.md | 388 +++++++++++++++++++------------------- 1 file changed, 194 insertions(+), 194 deletions(-) diff --git a/docs/roadmap/configkit.md b/docs/roadmap/configkit.md index fade91c..2646509 100644 --- a/docs/roadmap/configkit.md +++ b/docs/roadmap/configkit.md @@ -1,232 +1,232 @@ -# ConfigKit Adoption +# ConfigKit v0.5.0 Adoption ## Status And Intent -Implemented feature direction, drafted on 2026-09-26 and delivered through the -historical plan at -`git show 8452b54809b99425a97940da485daaa48c77aeb8:docs/roadmap/implementation.md`. -This roadmap records the target behavior and changes from the reviewed baseline; -that Git record owns sequencing, concrete interfaces, and verification evidence. +Accepted feature direction, recorded on 2026-10-03; implementation is pending. +This roadmap replaces the completed initial ConfigKit adoption roadmap. That +earlier work and its resolved decisions remain available in Git at +`9849c6b:docs/roadmap/configkit.md`. -The user is adopting ConfigKit across an application suite to standardize -configuration semantics. Scriptorium will make a clean cutover to ConfigKit's -native conventions, including intentional breaking changes. Preserving the -legacy loader's behavior is not a goal: there will be no parallel legacy loader, -compatibility mode, or recreation of the old merging rules. +The maintainer is adopting ConfigKit across an application suite to standardize +configuration semantics. This update continues that native integration by +adopting ConfigKit v0.5.0 and exposing its automatic sibling inheritance. +An intentional configuration compatibility change is accepted. The target +retains Scriptorium's existing discovery and application path policies rather +than expanding the configuration interface merely because upstream added +library APIs. -Alignment applies to application configuration, not to Promptkit-owned prompt, -profile, or schema definitions. Departures from ConfigKit conventions require an -application-specific reason or a genuine ownership boundary, not backward -compatibility alone. The [architecture policy](../policy/architecture.md) and -[ADR 0004](../adr/0004-definition-boundary.md) remain the scope boundaries. +This document owns the desired end state and required changes. A later +implementation plan will own ordered implementation instructions. The +[architecture policy](../policy/architecture.md), +[configuration contract](../config.md), and +[ADR 0004](../adr/0004-definition-boundary.md) establish the existing application +and framework ownership boundaries. -## Reviewed Baseline (Historical) +## Reviewed Baseline And Upstream Target -Scriptorium consumes Promptkit v0.11.0. Its `internal/config` package uses -yaml.v3 directly and manually applies defaults, file settings, and CLI overrides. -It has a parser-specific integer decoder, ignores empty string overrides, and -treats an empty configuration file as defaults-only input. It has no application -configuration imports, selected overlays, or provenance/inspection interface. -The [configuration reference](../config.md), [CLI reference](../cli.md), and -[source internals](../internal/sources.md) own the current behavior. +The baseline is Scriptorium v0.15.0 at +`9849c6b`, selecting ConfigKit v0.4.2 and Promptkit v0.11.2. +Application configuration already uses a shared ConfigKit-native +`Document`, defaults factory, recorded resolver, validator, and `Load` +boundary. Runtime conversion uses the resolved document without a second merge. +Execution and the `config validate`, `show`, `sources`, and `explain` +commands share that pipeline. -The reviewed ConfigKit release is v0.4.0 at commit -`e6ced0d3d1194b58e0b3dc8970089471ba175258`. Adopt that published release as the -initial target, rather than a sibling checkout or unpublished replacement. -Its canonical references are: +Current source selection is an explicit `--config` file, the first existing +documented system configuration file, or defaults-only loading. File-backed +loads enable explicit imports and one named overlay; explicit application CLI +flags supply override records. Application path values remain relative to the +process working directory. Backend definitions are independent mappings, with +no application-owned inheritance implementation. -- [Configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/e6ced0d3d1194b58e0b3dc8970089471ba175258/docs/config.md). -- [Consumer adoption guide](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/e6ced0d3d1194b58e0b3dc8970089471ba175258/docs/consumers/README.md). -- [Release notes](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/e6ced0d3d1194b58e0b3dc8970089471ba175258/docs/releases/v0.4.0.md). +The upstream target is the published ConfigKit v0.5.0 release at full commit +`dc09959cca4e105f49ec316c6d4ba585bd2340bd`. Canonical references are: -ConfigKit requires Go 1.25, within Scriptorium's existing toolchain requirement. -It selects `go.yaml.in/yaml/v4 v4.0.0-rc.6`. Removing Scriptorium's direct parser -usage does not remove yaml.v3 from the dependency graph: Promptkit still uses it. -The initial review passed ConfigKit's default test suite and Scriptorium's -configuration and CLI tests; that is baseline evidence, not migration validation. +- [Release notes](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/releases/v0.5.0.md). +- [Configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/config.md). +- [Sibling inheritance](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/config.md#sibling-inheritance). +- [Consumer examples](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/consumers/README.md). -## Target Ownership And Loading Model +The release retains the Go 1.25 minimum, the existing +`go.yaml.in/yaml/v4 v4.0.0-rc.6` dependency, and the library's +LGPL-3.0-or-later grant. No Promptkit upgrade is needed for this work. -| Owner | Responsibility | -| --- | --- | -| ConfigKit | Application YAML policy, source acquisition, composition, overlays, typed decoding, explicit override application, provenance, and redacted inspection. | -| Scriptorium | Source discovery, application schema and defaults, CLI parsing, recorded application normalization, semantic validation, command prerequisites, and runtime conversion. | -| Promptkit | Framework definition formats, backend semantics, credential lookup, engine defaults, and prompt execution. | +An isolated review using a temporary module file passed Scriptorium's existing +test suite and build against v0.5.0. CLI checks confirmed effective backend +inheritance, explicit zero and false replacements, and inheritance explanation. +They also confirmed that a literal `extra_params.extends` accepted by v0.4.2 +fails under v0.5.0. This evidence supports adoption; it does not establish +completion of the planned regression coverage, race testing, or vet checks. -`internal/config` becomes a small application-specific ConfigKit integration, -not a second general-purpose configuration framework. Execution and inspection -share one configuration definition and loading pipeline. Defaults, selected base -and imports, selected overlay, and explicit application CLI overrides participate -in ConfigKit's lifecycle; resolution and validation follow its upstream rules. -Normalization that changes inspected values is recorded through its resolver. -Runtime conversion does not perform a hidden second merge or apply new defaults -after the inspection snapshot has been created. +## Targeted End State -Scriptorium retains its application field names and domain meanings unless a -specific change below requires otherwise. ConfigKit's public root API replaces -direct YAML decoding and parser-specific hooks. Application defaults remain -Scriptorium-owned; omitted framework settings still delegate to Promptkit. -Backend conversion retains deterministic registration and engine-scoped shared -capacity state. This work does not change the HTTP request contract or move -configuration loading into each HTTP request. +Scriptorium selects ConfigKit v0.5.0 through its public root package and keeps +the existing single configuration loading lifecycle. Native sibling inheritance +is available in application configuration wherever upstream permits it, without +an application-specific opt-in, collection registry, merging implementation, +or compatibility preprocessing. -## Native Configuration Semantics +The primary user benefit is sharing settings among named backend definitions. +A child backend may inherit its parent's endpoint, API-key environment +reference, provider parameters, concurrency limit, and queue capacity, then +replace selected values. The resolved backend retains its own ID and reaches +Promptkit through the existing deterministic runtime conversion and registration. -The target accepts ConfigKit's upstream YAML grammar, schema binding, limits, -composition, and value-presence rules without a legacy preprocessing layer. -Migration-visible changes include: +ConfigKit owns structural inheritance, value presence, provenance, history, +and explanation. Scriptorium owns its schema, discovery, normalization, +application validation, and runtime mapping. Promptkit continues to own backend +semantics, credential lookup, framework definitions, and execution. -- An existing empty file is invalid; `{}` expresses an empty application mapping. - Defaults-only operation with no discovered file uses `NoSource` instead. -- Explicit empty strings, false values, and zeroes replace defaults. They are - not silently treated as omission. Application validation still decides whether - a resulting value is meaningful for its field or selected command. -- Null follows the application's declared nullable types. It is not a universal - spelling for omission. Scalars and structs do not accept null merely because - the former parser tolerated it. -- Anchors, aliases, merge keys, explicit tags, additional documents, unsupported - scalar spellings, and coercions rejected by ConfigKit are rejected here too. -- ConfigKit's reserved `configkit` key and resource limits apply, including in - dynamic application data. These constraints do not apply to independently - loaded Promptkit definitions or HTTP request bodies. -- Dynamic integers in backend `extra_params` use ConfigKit's `int64` - representation. Runtime mapping preserves their supported value semantics - rather than reserializing through YAML or demanding the old Go scalar type. +### Inheritance Semantics And Application Consequences -Presence remains meaningful at the framework boundary: omitted/null -`transient_retries` delegates to Promptkit, while explicit zero disables retries; -omitted backend queue capacity differs from explicit zero. Application HTTP -size defaults and zero-as-unlimited behavior remain. No new value ranges or -backend restrictions are introduced merely to adopt the library. +The target uses upstream semantics directly; the linked upstream contract +remains the canonical detailed specification: -## Discovery, Imports, Overlays, And Paths +- An input mapping can use `extends: NAME` to inherit from one named immediate + sibling mapping. Relationships resolve after imports, overlays, and explicit + CLI overrides have assembled the input, and before typed decoding. +- Child mappings recursively combine with their resolved parents. Explicit + zero, false, empty-string, sequence, and null values replace inherited values + according to ConfigKit's rules. Normal schema constraints still apply; null + is not newly accepted for every application field. +- `extends: null` cancels a relationship. Invalid references and cycles fail + through upstream structured diagnostics. Quoting the key does not make it + ordinary data. +- Inherited values count as explicit input. ConfigKit retains inheritance + derivation and displaced history for inspection; Scriptorium's normalization + continues to be recorded after decoding. +- All effective relationships resolve, including definitions a particular + invocation does not select. Inheritance is not lazy backend selection. +- Parent backend entries remain configured backends. They are not abstract + templates and receive the same existing Promptkit registration and validation + treatment as other backends. This adoption adds no template-only collection, + registration suppression, or validation exemption. -Source discovery remains Scriptorium-owned. The target retains the documented -explicit `--config` selection and ordered system-location search. An explicitly -empty config path is invalid rather than an implicit request for a default file. -If neither implicit candidate exists, loading uses defaults only; a selected -file that fails loading is not silently replaced with defaults. No implicit -working-directory configuration search, environment overlay, or directory scan -is added. +The backend example should demonstrate reusable complete parent settings and +a child with meaningful explicit replacements, including zero queue capacity +and a false dynamic parameter. Parameter names and values must be illustrative +and contain no credentials. -Application configuration supports ConfigKit's explicit path and glob imports. -Imports compose disjoint definitions; repeated non-mapping definitions are -conflicts, not last-writer-wins overrides. There is no implicit `conf.d` scan. +### Inspection And Credential Handling -An optional `--config-overlay` selects ConfigKit's one named overlay using its -standard `overlays/.yml` location, naming rules, and merge behavior. No -overlay is selected by default. Selecting an active overlay without a base -source follows ConfigKit's rejection of overlays with `NoSource`; Scriptorium -does not invent a root. Application overlays do not replace `--profile`, -`profile_dir`, or Promptkit's execution-profile selection. +`config show` presents the final resolved settings; `config explain` exposes +upstream inheritance origins and history, including consumed directive history +where supported. Both continue to use ConfigKit's native writers and the same +result used by execution. Inheritance creates no additional acquired source: +`config sources` retains its existing application DTO and source-list contract. +No new JSON fields or source layer are needed solely for inheritance. -Use the selected file's canonical directory as ConfigKit's default import and -overlay root. This root governs configuration source traversal only. Values of -`prompt_dir`, `profile_dir`, `schema_dir`, and `server.artifact_root` retain -working-directory-relative interpretation; absolute values remain absolute. -This is an explicit application path policy, not an upstream requirement or a -legacy parsing mode. ConfigKit's consumer examples demonstrate alternatives but -do not mandate source-file-relative application paths. This adoption does not -extend import containment to application data or alter HTTP artifact containment. +The maintainer's existing inspection policy remains: ordinary `extra_params`, +including historical values, and `api_key_env` names remain visible. +Scriptorium does not resolve API-key values through ConfigKit, add secret tags +to environment-reference names, or introduce sensitivity heuristics. +Promptkit resolves the selected credential values. Inheriting an environment +reference does not eagerly read it or change credential ownership. -## CLI Overrides And Configuration Inspection +### Existing Application Contracts -Only flags explicitly supplied by the user become ConfigKit application -override records. Parser defaults do not overwrite file or overlay settings. -Explicit empty values are not dropped. The existing invocation-specific prompt, -profile, input, model, timeout, output, and reporting options remain invocation -state rather than new application configuration fields. In particular, the -run/render retry flag remains a request override; it does not mutate the engine's -application retry setting or backend state. +The target retains current configuration discovery, defaults, explicit imports, +one selected overlay, explicit CLI override mapping, and command prerequisites. +It continues using ordinary `configkit.Load`; the new `LoadLayered` API is +not required to obtain inheritance. -Expose application configuration validation, effective redacted YAML, source -provenance, and explanation of an individual configuration path. These are -included features, not merely library capabilities available for future work. -They use ConfigKit diagnostics and inspection records from the shared loading -pipeline, including defaults, imports, overlays, explicit CLI values, and -recorded normalization. No separate inspection-only interpretation is maintained. +Application path values, including inherited values, retain their documented +working-directory-relative meaning. The new source-resource contexts do not +implicitly rebase `prompt_dir`, `profile_dir`, `schema_dir`, or +`server.artifact_root`. -Configuration inspection does not execute prompts, make provider calls, or -resolve API-key values. Application validation is not a promise that every -Promptkit backend or definition is valid or that a provider is reachable. -Command-specific execution prerequisites remain with the relevant command; -inspection can explain an incomplete application configuration without requiring -an executable prompt selection. The interface is a CLI configuration command -family, not a second CLI framework or an HTTP configuration endpoint; the -implementation plan specifies its concrete commands and mappings. +Promptkit prompt, profile, and schema files continue to be parsed by Promptkit. +ConfigKit's new directive has no effect on those independently loaded formats +or on HTTP request payloads. Engine-scoped backend capacity coordination and +request independence remain governed by the existing architecture. -Safe diagnostics must not restore raw parser errors or configuration values -that ConfigKit deliberately suppresses. Effective output and explanation use -the upstream redacted renderers; they are inspection output, not guaranteed -reloadable configuration. Under the maintainer's selected policy, backend -`extra_params` are visible in effective output and explanation, including -historical values. There is no blanket container masking, sensitive-key heuristic, -or new selective-redaction configuration surface. +## Compatibility And Release Impact -## Credential Boundary And Deferred Features +ConfigKit now reserves `extends` in every application input mapping, including +dynamic backend `extra_params` and nested provider data. A previously literal +key can fail inheritance validation or be consumed and change effective data. +The reservation can also collide with a named mapping entry such as a backend +ID equal to `extends`. This is a real configuration compatibility change, +not merely a new optional syntax. -Existing `api_key_env` values remain environment-name strings passed to Promptkit. -They do not become ConfigKit `Secret` fields. ConfigKit eagerly resolves effective -typed references and requires nonempty values; that would change Promptkit's -optional/required and selected-credential behavior. This departure from the -secret-reference feature is an ownership decision, not compatibility plumbing. -Raw API-key fields remain unsupported under the existing architecture policy. +Migration guidance must distinguish structural configuration names from +provider-defined parameter names. Application-owned names can be renamed +where appropriate. A provider-required literal parameter cannot generally be +renamed without changing the provider contract; quoting the key is not an +escape. The selected upstream version has no inheritance opt-out or literal-key +escape. A concrete need for such a parameter should be addressed upstream, +rather than by silently adding a second Scriptorium configuration parser. +This limitation applies to application configuration, not to Promptkit-owned +profile definitions. -API-key values are the only secrets requiring ConfigKit redaction in this -adoption. The application configuration holds environment-variable names, not -those values, so inspection neither resolves nor contains the actual keys. -Reference names remain visible. Do not tag `api_key_env` as secret merely to -hide its name, add an eager key lookup, or mask unrelated application settings. -The existing raw-key prohibition and Promptkit boundary provide the protection -without introducing a redundant credential-bearing configuration field. +The reviewed maintained examples and fixtures contain no `extends` keys. +Ordinary configurations without the newly reserved key should retain their +meaning, subject to validation of the integration. -Secrets-directory bootstrap and application schema versioning are deferred until -there is a concrete application need. Their availability in ConfigKit does not -make them mandatory. Hot reload, runtime resource replacement, remote imports, -automatic migrations, and general environment-variable overlays are outside -this adoption. These are scope limits for this feature, not new permanent -prohibitions on future explicitly requested work. +The intended release is a new minor version, currently v0.16.0, under the +[release policy](../policy/release.md). Its release note should identify the +inheritance feature and reserved-key migration impact. Final version selection +and publication remain release preparation work; this roadmap adds no +publication gate or recurring upstream probe. -## Required Changes And Completion Criteria +## Work Needed From The Baseline -- Replace manual decoding/default/override machinery with the shared ConfigKit - integration and retain only application-specific discovery, rules, and mapping. -- Support imports and selected overlays across commands that load application - configuration, and deliver the configuration inspection capabilities above. -- Update application configuration types, presence handling, CLI explicit-set - mapping, and diagnostics to match the selected native semantics. -- Update maintained examples and canonical configuration, CLI, operations, - architecture, and internal documentation when behavior lands. Upstream generic - contracts remain linked rather than copied into a competing specification. -- Add proportionate integration coverage for the cutover: defaults-only input, - explicit values and nulls, imports/conflicts, overlays, CLI precedence, - inspection agreement and API-key non-disclosure, visible dynamic backend - parameters, unchanged credential ownership, and representative - execution/server startup. - ConfigKit owns exhaustive parser and composition tests; Scriptorium tests its - adoption and application boundaries. -- Pin the published dependency and maintain ConfigKit and YAML attribution under - the [OSS policy](../policy/oss.md), using imported-commit source/notice links - and applicable standard and companion license texts. -- Document migration in a release appropriate for the intentional breaking - configuration changes under the [release policy](../policy/release.md). - Release selection and publication remain separate preparation work; this - roadmap adds no release gate, network probe, or new distribution format. +- **Dependency and attribution maintenance:** update `go.mod` and `go.sum` + to the published v0.5.0 tag. Update ConfigKit's version and exact-commit + source, COPYRIGHT, and LICENSE links in `docs/oss/attribution.md` under the + [OSS policy](../policy/oss.md). Existing standard license texts remain + applicable; check the selected upstream notices. Leave unrelated dependencies + and historical release records alone. +- **Integration review:** verify the existing `internal/config` definition, + loading, normalization, and runtime conversion preserve resolved inherited + values and explicit presence. Make only changes needed for that integration; + upstream already implements the inheritance machinery. +- **Application coverage:** add representative tests at the configuration and + CLI boundaries for inherited backend settings, explicit zero/false and + nullable replacements, interaction with an imported parent and selected + overlay, explainable derivation, and reserved-key failure. Verify inherited + backend settings reach existing runtime mapping or registration. Use small + fixtures and behavior assertions rather than duplicating ConfigKit's + exhaustive inheritance, parser, or metadata matrix. +- **Maintained examples:** add a runnable application configuration example + under `examples/` showing backend inheritance, and link it from the owning + configuration documentation. Validate it through the real application + loading and inspection path without credentials or provider calls. +- **Current documentation:** update `docs/config.md` with the user-facing + inheritance support and compatibility guidance. Update relevant dependency + references in `docs/development.md` and `docs/policy/architecture.md`; + reflect integration mechanics in `docs/internal/sources.md` and adapter + documentation where needed. CLI syntax is unchanged; update CLI documentation + only where its inspection description needs clarification. Link the pinned + upstream contract rather than copying its full specification. +- **Validation and release handoff:** run the existing deterministic suite, + race suite, vet, build, diff checks, and affected example/link checks. Prepare + accurate minor-release migration information when preparing the release. + Routine checks remain governed by the existing development and release + policies, rather than becoming new requirements introduced by this roadmap. -Completion means one ConfigKit-native application configuration path serves -execution and inspection, the selected user-facing features work, and migrated -examples and regression coverage demonstrate the documented semantics. It does -not require adopting every ConfigKit feature or changing Promptkit definitions. +## Deferred Scope -## Resolved Inspection Policy +Ordered source underlays, caller-owned filesystem sources, mapping sources, +and source-resource contexts remain available library capabilities rather than +new Scriptorium interfaces in this adoption. The application has no current +requirement for bundled defaults, extra defaults-file selection, alternative +configuration media, or source-relative application resources. -On 2026-09-26 the maintainer selected no redaction for `extra_params` and limited -the relevant secret to the API key. Inspection therefore preserves parameter -values and environment-reference names, while actual API-key values remain -outside the application configuration pipeline. Visibility is intentional; -this is not a promise to detect secrets placed inside arbitrary parameter values. +This work does not add underlay flags, embedded configuration bundles, +schema versioning, secret-directory bootstrap, hot reload, or new HTTP +configuration endpoints. These are scope choices for this work set, not +permanent prohibitions on future requested features. -The delivered `config show` and `config explain` commands use ConfigKit's -inspection writers without secret tags or a second redaction path. Both -effective and historical parameter values remain visible; `api_key_env` names -remain visible while Promptkit alone resolves selected credential values. +## Completion Criteria + +Adoption is complete when the published dependency is selected; native +inheritance works through the shared execution and inspection pipeline; +backend values and explicit replacements reach the existing runtime boundary; +focused tests and a maintained example demonstrate the integration; and +current documentation and attribution describe the selected version accurately. + +The reserved-key compatibility impact must be explicit, and established +credential visibility, path interpretation, discovery, and framework ownership +must remain intact. No significant decisions remain open within this agreed +scope.