Files
notarius/docs/roadmap/configkit.md

289 lines
17 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Adopt Current ConfigKit Configuration Semantics
## Purpose
Adopt `go.maximumdirect.net/configkit` v1.2.0 and its native consumer workflow.
The objective is consistent configuration semantics across ConfigKit-consuming
applications. Existing Notarius composition behavior is a migration baseline,
not a compatibility requirement to reproduce through adapters.
This work set retains PromptKit v0.12.0 and go-yaml v4.0.0-rc.6. ConfigKit
v1.2.1 and stable go-yaml v4.0.0 belong to the subsequent PromptKit upgrade.
[ADR-0020](../adr/0020-delegate-configuration-mechanics-to-configkit.md) owns the
boundary: ConfigKit delivers the final resolved application document; Notarius
supplies application schema, defaults, normalization, validation, and subsequent
pipeline resolution. This roadmap defines the target and required work. A later
implementation plan will own sequencing. Release publication is separate work.
## Reviewed Baseline And Sources
Reviewed on 2026-11-17 against Notarius `b9899ea`:
- ConfigKit v0.5.0 under `gitea.maximumdirect.net/eric/ConfigKit`;
- PromptKit v0.12.0, which consumes that same old ConfigKit package;
- Go 1.25.5 and go-yaml v4.0.0-rc.6;
- Notarius configuration version 5, imports, overlays, sibling inheritance,
declaration-relative application paths, and effective/explanation CLI reports.
The target v1.2.0 tag in `../ConfigKit` identifies source commit
`c1c7e87f2cdf77a0cbf3327831ac05578e756b9e`. Review references:
- [Consumer adoption guide](https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.2.0/docs/consumers/README.md)
- [Configuration contract](https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.2.0/docs/config.md)
- [Release notes](https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.2.0/docs/releases/)
All intervening release notes were read in the local checkout. The important
changes are:
| Releases | Relevance to Notarius |
| --- | --- |
| v0.6.0 | Loading/inspection optimizations; Go minimum becomes 1.26.0. No promised application-wide speedup. |
| v0.7.0–v0.7.1 | Frozen effective-configuration diffs; maintainer tooling update. |
| v0.8.0 | Input destinations, sequence-item placement, source-version checks, and identified sequences. |
| v0.9.0 | Expanded imports; imports optional by default; author-declared replace/append/upsert; identified-record inheritance; CLI assignments move after inheritance and decoding. |
| v0.10.0–v0.10.2 | Explicit configuration discovery; broader absent-main lifecycle; loading, ownership, acquisition, and history fixes. |
| v1.0.0 | New module path, source-root `config` controls, public API cleanup, source lookup, and Go 1.27.0 minimum. |
| v1.1.0 | Field-level normalization inside lists; identity-aware provenance/history across record movement. |
| v1.2.0 | Main-relative companion inputs and layered overlay context selection. |
| v1.2.1 (reviewed, deferred) | Stable go-yaml v4.0.0 and loader-plugin adaptation; adopted with the subsequent PromptKit upgrade. |
### Current Integration Areas
`internal/core/config/load.go` supplies ConfigKit's definition and callbacks.
`loaded_config.go` privately retains its inspection result and publishes a
detached runtime value. `document_paths.go`, `pipeline_document.go`, and
`document_normalizer.go` implement application normalization and source tracking.
The normalizer currently records whole-list coverage even for leaf edits.
`internal/cli/configkit_load.go` and `run.go` own selection and diagnostics.
`discoverConfigPath` manually probes files before `config.Load` calls ConfigKit.
The complete example imports its pipeline through root `configkit`. Current
documentation describes unconditionally replacing whole step lists.
ConfigKit types also appear inside the PromptKit adapter's native source and
inspection APIs. That is a distinct integration boundary with the dependency
constraint below; a repository-wide import replacement would be incorrect.
## Target End State
### Native Loading And Ownership
Use the new public package and the ordinary `Load` workflow for the application
configuration. Defaults, `Resolve`, `Validate`, `VersionRule`, and sensitivity
classification remain supported application extension points. The handoff
contains fully composed, decoded, normalized, and validated application values.
Runtime conversion and catalog-dependent pipeline preparation retain their
application ownership.
ConfigKit owns import acquisition, ordering, inheritance, collection operations,
CLI assignment semantics, limits, source evidence, and frozen inspection. Pass
its results and structured failures through the established adapter boundary.
Application validation checks the resulting domain configuration; it does not
audit superseded contributions or replay structural directives.
Keep configuration `version: 5`: its typed application shape is unchanged.
Explain the deliberate ConfigKit syntax/semantic migration in operator guidance
and the eventual minor-release note. Existing files require migration rather
than a compatibility pre-parser or legacy control aliases.
### Source Controls, Imports, And Composition
Use source-root `config` controls in Notarius configuration inputs. Update real
control blocks and stored declaration pointers, including imported files,
overlays, mapping fixtures, and examples. Nested application data named `config`
retains native treatment. Former `configkit`/`_configkit` keys are ordinary data;
ordinary typed validation governs them. No custom legacy-control scanner is
needed.
This control migration applies to Notarius's application loader. PromptKit
operator files and adapter fixtures follow the version of PromptKit that reads
them; retain their old control spelling while that separate engine remains old.
Enable ConfigKit imports and accept its native path/list, directory, glob,
destination, placement, exclusion, recursion, hidden-file, symlink, and
requiredness rules. Leave `ImportDefaults` unspecified unless an application
choice is explicitly approved. In particular, do not restore former implicit
requiredness or impose a YAML-only directory filter globally. Required deployed
fragments, such as the maintained complete pipeline import, should say
`required: true` in their declaration so an incomplete installation fails there.
Optional imports remain genuinely optional.
Use the native main-only source-version default; fragments do not acquire a new
mandatory version declaration as a side effect of this upgrade.
Expose native identified sequence operations and inheritance through the source
format. Without identity, lists retain native peer-conflict/whole-replacement
behavior. Identified lists support strict peer composition and native
replace/append/upsert across tiers. Upsert replaces a complete matching record,
retaining its position; it is not a recursive merge of that record. Clearing an
append/upsert collection requires explicit replacement with an empty list.
Dependency-ready assembly order, not a Notarius sorting or append pass, controls
composition. Final Notarius checks still validate stage order, supported modules,
reference dependencies, and unique application identifiers.
Register `id` for `/pipelines/<any-mapping-key>/steps` through
`SequenceIdentityMatching`. Step IDs already carry application meaning and
uniqueness requirements. Native identity permits independently imported steps
and supplies the ordinary upsert default for overlays and inheritance. Authors
can explicitly select replacement or append. Other identified collections
remain available through author declarations; no broad automatic `id` inference
or universal registration is required. Demonstrate a composed step list and a
derived pipeline using native controls in maintained examples or small canonical
snippets, with behavioral coverage.
CLI overrides apply after inheritance and typed decoding, before application
resolution. Preserve explicit zero/false/empty values. Structural errors cannot
be repaired by CLI assignments; document this instead of rebuilding the former
override order. Notarius's binding-profile and validation-policy rules remain
domain semantics applied to the final configuration.
### Native Source Selection And Resources
Replace manual acquisition/probing with ConfigKit source descriptions. Explicit
`--config` and, when no CLI selection exists, `NOTARIUS_CONFIG` select exclusive
`File` sources: a missing selected file fails instead of searching elsewhere.
For default discovery use `SearchConfig` with `Searches` composing these native
presets in order:
1. `WorkingDirectory()`;
2. `UserConfigDirectory()`; and
3. `UnixSystemDirectories()`.
Together with the explicit selections above, precedence is `--config`, then
`NOTARIUS_CONFIG`, then working-directory, user-configuration, and system searches.
Retain each preset's native filename groups and ordering. In particular, the
working-directory preset selects `notarius.yml`, then `notarius.yaml`, rather
than generic `config.yml`. The separate `HomeDirectory()` preset is not enabled.
Discovery selects one main file; it does not merge files from all search
locations. Missing candidates advance the search under ConfigKit's rules. Access
errors and failures loading a selected file are terminal rather than reasons to
try a lower-priority configuration. Document working-directory-dependent
selection and recommend explicit `--config` for services and subprocess callers
that want a fixed selection.
Use a required search for commands that require a configuration file, preserving
their need for a selected configuration. This does not impose requiredness on
all imports or on separate defaults-only programmatic loads.
Report the actual selected main file using retained source information; update
load/error plumbing rather than making a second discovery pass. Construction of
search descriptions remains side-effect-free, and acquisition errors follow
ConfigKit's native selected-file behavior.
Retain the deliberately selected application path policy: file-authored values
use their declaring file, CLI paths use the invocation directory, and relative
application defaults use the canonical main file. Use `EntryInfo`, source records,
and public resource contexts where appropriate. Native destinations and source
coordinates are distinct: a mounted or inherited value keeps its source base.
No new target-existence, confinement, symlink, or permission policy is added to
application resource paths. ConfigKit's own input-root rules remain upstream.
Ordinary `Load` is sufficient for current application composition. `LoadLayered`,
`MainRelative`, and overlay fallbacks are available when actual bundled/companion
inputs need them; the upgrade does not require an artificial extra input layer.
### Lean Application Resolution And Inspection
Replace `InputInfo` with `EntryInfo`, `Definition.Secret` with `Sensitive`,
generic `Record` calls with the corresponding named recording methods, and
`LayerBase`/`"base"` with `LayerMain`/`"main"` where used. Audit all affected
symbols and diagnostic strings against the v1 migration table. Use assignment-
and-record helpers when appropriate; named recording methods alone do not mutate
the application value.
Use v1.1 field-level recording for list leaf normalization. Remove the old
blanket whole-sequence coverage wrapper where its only purpose was satisfying
the old audit. Record actual structural mutations at their proper scope. Resolve
source-relative values before moving records; fixed entry positions, final
positions, historical evidence, and effective origins serve different purposes.
Keep map-key relocation bookkeeping only where actual application identifier
normalization still needs it. Do not add an independent history-replay engine.
Effective/explanation commands delegate to ConfigKit's captured inspection,
retaining redaction, presence, source history, and detached ownership. Source
lookup APIs support useful diagnostics, including empty import selections when
needed, without turning ordinary optional absence into warnings. ConfigKit's
`Diff` is available for a later operator-facing comparison command; a new CLI
diff or secrets-management workflow is not required for this adoption.
## Dependency And Toolchain Boundary
The target is ConfigKit v1.2.0, go-yaml v4.0.0-rc.6, and a Go consumer minimum
of 1.27.0. Update the selected release-validation toolchain, setup guidance, and
actual configured build/CI environments consistently. Minimum and validation
patch remain separate choices. Include Go and configuration migration impact in
the eventual release note; patch releases cannot raise the Go minimum.
PromptKit v0.12.0 exposes **old-path ConfigKit types** through `Config.Sources`,
its source helpers, and inspection. The approved intermediate architecture
retains old-path ConfigKit v0.5.0 for that dependency while ordinary Notarius
configuration uses new-path ConfigKit v1.2.0. Keep each package's types inside
its owning integration boundary. Prefer neutral scalar/domain handoffs over
translating dependency-owned graphs. Maintain the old PromptKit format for its
operator files and fixtures during this work set.
The selected intermediate versions share YAML rc.6. A direct v1.2.1 upgrade
would introduce a shared-dependency incompatibility:
- ConfigKit v1.2.1 requires stable go-yaml v4.0.0.
- Old-path ConfigKit v0.5.0 imports `go.yaml.in/yaml/v4/plugin/limit`.
- Stable go-yaml contains `plugin/loaderlimits` and no `plugin/limit`.
- Go selects one version of this shared YAML module, despite ConfigKit itself
having two distinct module paths.
Targeting v1.2.0 avoids that specific package-removal conflict using released
dependencies, without maintaining an upstream compatibility patch. Verify the
mixed dependency graph and native build early in implementation; no trial build
has yet established full compatibility. This work set is complete at v1.2.0
when its native semantics and integration pass acceptance, independently of the
later PromptKit upgrade.
## Required Repository Work And Acceptance
- Migrate application-side dependency imports, APIs, diagnostics, source
declarations, inspection, and discovery. Verify coexistence of new-path
ConfigKit v1.2.0, old-path v0.5.0, PromptKit v0.12.0, and YAML rc.6 without
dependency forks or replacements. Native builds must pass at completion.
- Update real-file loading, imports, overlays, inheritance, CLI assignment,
path-origin, failure, and ownership tests under `internal/core/config` and
`internal/cli`. Add representative collection composition and step-reference
normalization cases, including imports in distinct directories and leaf
histories after list composition. Test native semantics, not the old behavior.
- Retain two top-level configuration examples. Migrate the complete example's
required import and demonstrate useful native composition without adding a
second application merger or an unnecessary example framework.
- Revise `docs/config.md`, `docs/cli.md`, `docs/internal/configuration.md`, and
affected operations/subprocess guidance for syntax, search, imports, collections,
CLI ordering, origins, and inspection. Update README/toolchain/release guidance
and affected dependency integration references. Preserve accepted ADR history;
ADR-0020 already supplies the durable ownership rationale.
- Update `docs/oss/attribution.md` from actual selected module revisions and
notices, including both selected ConfigKit module identities, retained YAML rc.6,
and newly selected transitive dependencies. Preserve the application grant.
Upstream ReleaseKit is maintainer tooling; adopting it as Notarius release
machinery or adding its gates is outside this feature.
- Verify focused configuration/CLI/LLM tests, full offline tests, vet, a native
build, maintained examples and report commands without provider calls, tidy
consistency, selected module identities, and affected links. No production
inference, fixed performance improvement, new release gate, or broader
filesystem/security guarantee is part of acceptance.
Completion means current native configuration semantics are usable and documented
on a coherent dependency graph, with final domain validation and no unresolved
directives at runtime. The old ConfigKit 0.5 roadmap is replaced by this feature
scope; unrelated completed plans and review findings remain separate records.
## Deferred To The PromptKit Upgrade
The next work set upgrades PromptKit together with ConfigKit v1.2.1 and stable
go-yaml v4.0.0. Its roadmap will cover PromptKit's module/API and source-format
migration, replace the temporary old-path ConfigKit integration, and remove old
dependencies when no longer used. It will also update the relevant tests,
inspection/checkpoint integration, documentation, and attribution for the final
selected graph. That later work does not gate completion of this v1.2.0 scope.
The dependency approach, ordered native discovery presets, and application step
identity registration are approved decisions. No significant open decisions
remain for this feature roadmap.