# ConfigKit 1.2.0 Implementation Plan ## Scope And Working Rules Implement [the feature roadmap](configkit.md), whose approved choices govern this plan. The baseline is `2c8b631`: old-path ConfigKit v0.5.0, PromptKit v0.12.0, Go 1.25.5, and YAML v4.0.0-rc.6. Target new-path ConfigKit v1.2.0 at `c1c7e87f2cdf77a0cbf3327831ac05578e756b9e`, Go minimum 1.27.0, and the same PromptKit/YAML versions. Use `git show v1.2.0:` in `../ConfigKit` for pinned guidance; its moving checkout may contain v1.2.1's incompatible YAML changes. During planning, the configuration, CLI, and LLM suites passed with the existing Go 1.25.5 graph. No upgraded-graph trial was performed; these baseline results do not establish compatibility with the target dependencies. Each numbered stage is one prompt for Luna, executed in order in a single end-to-end sprint. Intermediate revisions may fail compilation or tests. Record those failures and continue with the owning stage; avoid transitional parsers, type-translation layers, or duplicate machinery just to keep intermediate commits green. Completion requires all stages and the final acceptance checks. For every stage: - Read `docs/development.md`, `docs/policy/agents.md`, the feature roadmap, and applicable architecture/testing/documentation/OSS/release policies. Reuse unchanged context. Prefer the code graph for discovery. - Keep ConfigKit responsible for source mechanics and final inspection, per ADR-0020. Notarius owns application rules and runtime pipeline resolution. - Preserve configuration version 5, artifact/subprocess contracts, LLM behavior, cache permissions, and existing retry/scheduling policy. Native composition, discovery, CLI ordering, and control syntax intentionally change. - Use `GOWORK=off`. Keep dependencies released and selected through ordinary Go module resolution. ConfigKit v1.2.1, stable YAML, PromptKit migration, release publication, and unrelated remediation findings are outside this sprint. - Test real loading with temporary files. Keep tests offline and credential-free; inject only external LLM/HTTP boundaries. Reuse coverage instead of mirroring the entire upstream suite or snapshotting incidental diagnostics. - Review the diff and commit nonempty units under agent policy. Record actual validation and deferred checks. No empty commits, pushing, tagging, roadmap retirement, or changes to accepted ADR bodies. ## Settled Implementation Details ### Package Identity And Loading Handoff Application configuration and CLI configuration overrides/diagnostics use `go.maximumdirect.net/configkit` v1.2.0. PromptKit v0.12.0's source helpers and inspection continue to use `gitea.maximumdirect.net/eric/ConfigKit` v0.5.0 inside the LLM adapter and direct integration fixtures. These named types are distinct. Do not globally replace imports or pass a new-package path/source to old PromptKit. Keep `config.Load(path, options)` as an exclusive file convenience. Add `config.LoadSource(source configkit.Source, options LoadOptions)` for a production source supplied by the composition boundary, with imports enabled. Both reach the same definition/callback/load path. Private byte/reader test helpers may retain their existing explicit import choice. This configuration package is the explicit ConfigKit integration boundary; upstream values stay out of runtime pipeline/module contracts. The CLI selects a source without filesystem probes: 1. Trim a nonblank `--config` value and use `File` exclusively. 2. Otherwise use a nonblank `NOTARIUS_CONFIG` via the existing injected `LookupEnv`, selecting `File` exclusively. 3. Otherwise construct `SearchConfig("notarius", Searches(WorkingDirectory(), UserConfigDirectory(), UnixSystemDirectories()), SearchOptions{Required:true})`. Pass that source into `LoadSource` exactly once. Add a neutral `LoadedConfig.MainSourcePath()` accessor based on retained sources: select the non-imported main file (`LayerMain`, `SourceKindFile`, nil `ImportedFrom`). Return its recorded path, or empty for a non-file private/programmatic load. Do not infer the main file from source IDs, a particular field's surviving origin, or another filesystem search. Keep `loadConfigKit`'s selected-path string return and populate it from this accessor. Debug/invocation provenance then describes the actual selected main. Its existing flag/discovered category need not expand. Native search filename groups, missing-candidate behavior, selected-file failure, and directory-helper timing remain upstream behavior. WorkingDirectory searches `notarius.yml`, then `notarius.yaml`; generic filenames belong only to the presets that include them. HomeDirectory is not added. Files are selected, not automatically layered. NoSource remains a separate explicit programmatic choice. ### Native Collections And Application Resolution Register precisely this identity on the application definition: ```go configkit.SequenceIdentityMatching(configkit.SequenceSelector{ configkit.SequenceLiteralKey("pipelines"), configkit.SequenceAnyMappingKey(), configkit.SequenceLiteralKey("steps"), }, "id") ``` Use native upsert by default for those identified lists. A matching step is replaced as a whole, not field-merged; order and clearing follow native rules. Source authors may select replace/append explicitly and declare identities for other lists. Do not register validator chains or infer identities elsewhere. Application validation still checks actual domain constraints after resolution. Leave import defaults and source-version mode at native defaults. Maintained required fragments say `required: true`; other absent imports remain optional. Keep version 5 main validation without requiring version headers on every fragment. Do not impose global filename filtering, hidden-file, symlink, or recursion rules. CLI assignments occur after typed decode/inheritance and before Resolve. Replace blanket list recording with leaf records for actual leaf edits. Keep whole-container records for actual map-key or list structure changes. Normalize paths and nested reference keys while records still have entry order; use EntryInfo to follow the source of the original scalar, then record the final field. ConfigKit composition has already assembled step positions before Resolve. Step-ID trimming can change correspondence: it does not justify an extra custom identity matcher or a promise that every transformed record retains a file origin. Test changed identities with native auditing and record an ancestor when needed. Map-key normalization remains application behavior. Retain original-path tracking where a renamed pipeline/lane/reference key requires it; reduce redundant state only after the updated origin tests prove it unnecessary. Preserve file-relative, CLI-invocation-relative, default-main-relative, absolute, and empty-cache path semantics. No new resource-existence or access policy is introduced. ## Stage 1 — Select The Mixed Dependency Graph And Toolchain **Complete.** Go 1.27.0 was verified using the locally installed toolchain; both ConfigKit packages build with YAML v4.0.0-rc.6, and the existing config, CLI, and LLM packages compile with the selected graph. Application API migration and full behavioral checks remain in their respective stages. Goal: establish the approved build inputs without migrating PromptKit. Inspect `go.mod`, `go.sum`, pinned ConfigKit/PromptKit module files, the v1.2.0 consumer migration guide, and `docs/release.md`. Work: - Set the Go consumer minimum to 1.27.0. Use Go 1.27.0 as the selected validation toolchain for this sprint and update its release-runbook selection. Later validation-patch updates remain separate from the consumer minimum. - Add `go.maximumdirect.net/configkit` v1.2.0 while retaining old-path v0.5.0, PromptKit v0.12.0, its current catalogs, and YAML rc.6. Tidy after real imports land; unused new requirements may disappear before then. Retain needed graph changes, not arbitrary unrelated upgrades. Do not adopt ReleaseKit as a tool. - Verify both ConfigKit module paths and shared YAML selection. Record API compile errors for Stage 2 rather than repairing them with replacements or forks. Acceptance: required released versions are obtainable and their graph has no stable-YAML plugin-removal conflict. An unexpected dependency incompatibility is reported with evidence rather than changing the target or upgrading PromptKit. Checks: `GOTOOLCHAIN=go1.27.0 GOWORK=off go version`, selected module inspection, and a compilation attempt. Network/dependency preparation precedes offline tests. Prompt size: dependency/toolchain preparation only. ## Stage 2 — Migrate Application APIs And Separate Old PromptKit Types **Complete.** Application and CLI APIs now use the new module; PromptKit keeps the old module at its boundary. The native build, package compilation, and source-handoff test pass. Broader attempts exposed old control fixtures and CLI assignment expectations owned by later stages; CLI/LLM tests using local HTTP listeners also encountered this environment's socket restriction. Goal: move the application loader and CLI to v1 APIs. Inspect `internal/core/config/{load,loaded_config,operational_document, pipeline_document,document_paths,document_normalizer}.go`, CLI ConfigKit imports, and direct PromptKit setup under `internal/framework/llm` and test support. Work: - Migrate application-side imports. Update InputInfo to EntryInfo, Secret to Sensitive, LayerBase to LayerMain, removed diagnostic names, and generic Record calls to named normalization/derived-default methods. Audit stored strings too. - Temporarily retain sequence ancestor records where the wrapper currently emits them; simplify scope in Stage 3. Named recording methods record mutations already performed; Default/Normalize helpers both assign and record. - Keep old types in PromptKit-facing sources, profile inspection, and integration fixtures. CLI ordinary configuration errors use new LoadError/WriteDiagnostics; PromptKit errors remain adapted through its existing neutral facade. - Introduce the shared LoadSource path and MainSourcePath accessor specified above. Keep frozen inspection and detached runtime conversion. Acceptance: production imports have explicit owners, and no mixed named types cross the boundary. Any remaining failures are classified as fixtures, collection policy, or discovery work. No upstream types enter module/runtime contracts. Checks: native build to a `mktemp -d` location; focused compile/test attempts for `./internal/core/config ./internal/cli ./internal/framework/llm`; diff review. Prompt size: API/handoff migration; no discovery implementation or broad fixtures. ## Stage 3 — Simplify Recorded Normalization And Preserve Source Bases **Complete.** Leaf edits are recorded directly and reference paths are resolved before key normalization moves their source coordinates. Focused path, inheritance, discovered-main, and leaf-history tests pass. The full config suite's remaining override-order expectation is owned by fixture migration. Goal: use v1.1 field-level auditing instead of old blanket sequence coverage. Inspect document_normalizer.go, document_paths.go, pipeline_document.go, reference normalization helpers, and existing path/inheritance tests. Work: - Remove `inSequence`/whole-list coverage for ordinary leaf edits. Use the public recording helpers directly or retain only a genuinely useful thin helper. - Preserve explicitness and source lookup through key normalization. Resolve reference paths before key/record movement obscures entry addresses; retain only needed relocation state. Preserve true structural mutation records. - Verify canonical main-directory initialization against public source evidence and native version-path protection, including a discovered main. Retain the existing version-origin lookup where that contract makes it valid; avoid a second search. Preserve private non-file test-source handling separately. - Improve existing tests for imported/overlaid/inherited source bases, paths inside steps, whitespace-normalized IDs, CLI paths, and leaf explanation history. Fixtures must use distinct directories so incorrect bases cannot also pass. Acceptance: leaf-only normalization is accurately recorded; real structural edits remain covered; runtime paths match effective output and source history. No new filesystem constraint or custom provenance-replay mechanism appears. Checks: focused config normalization/path/inheritance tests, then all core/config tests when migrated fixtures allow them. Expected old control-format failures belong to Stage 5. Prompt size: one resolution/provenance boundary. ## Stage 4 — Register Steps And Exercise Native Collection Semantics **Complete.** Registered step IDs and verified real-file peer composition, upsert, replacement, append, clearing validation, inherited records, CLI list assignment, source-relative references, duplicate peers, and unregistered lists. Focused config and framework resolution tests pass. Identity-changing normalization records the affected list, while unchanged identities retain leaf history. Goal: implement the accepted step identity and composition policy. Inspect documentDefinition, pipeline step validation/conversion, ConfigKit's pinned sequence/placement contract, and inheritance tests. Work: - Add the exact wildcard registration above. Keep validation/domain IDs unchanged; ConfigKit owns structural identity comparison, assembly, inheritance, and history. - Add small real-file cases for distinct imported steps, strict duplicate peers, inherited/overlaid full-record upsert, replacement/append, and empty-list clearing. Include one identified-record inheritance case and an unregistered list showing that broad identity registration has not leaked elsewhere. - Assert resulting order, full-record replacement, and actual pipeline compatibility. Cover step-local references declared in different imported files and native CLI whole-list assignment. Do not add list merging to Notarius. - Use native leaf histories after composition to expose any Stage 3 defects. Reuse domain validation for invalid or duplicated normalized step identifiers. Acceptance: native composition produces the intended typed pipeline and existing domain rules still apply. Default upsert and explicit replace differ as documented. Checks: config collection/inheritance tests and focused framework resolution tests. Prompt size: one definition registration and its focused behavior coverage. ## Stage 5 — Migrate Application Fixtures And Restore Configuration Coverage **Complete.** Application fixtures and expectations now use native controls, post-decode overrides, and identified step operations; PromptKit fixtures remain on its old format. Config and LLM suites pass. CLI tests compile and pass apart from maintained-example cases reserved for the example migration. Goal: translate existing tests to the selected source contract. Inspect config document/operational/loaded/paths/inheritance tests and CLI ConfigKit/command/production tests. Discover literals and helpers by consumer. Work: - Rename genuine application-source controls to `config` and update stored pointer and layer strings. Required file fixtures explicitly declare requiredness; tests of absent optional imports adopt the native default. Keep old PromptKit sources and their fixtures under the old reader's syntax and APIs. - Update tests deliberately for post-decode CLI assignment and identified step upsert. A CLI override does not repair decoding/structural errors. An explicitly replaced empty list, null, omission, and empty append/upsert have distinct meanings. - Preserve version checks, application field validation, dynamic integer option conversion, redaction, cancellation, and detached runtime/inspection ownership. Include nested ordinary `config` data in dynamic options to protect control scope. - Consolidate small fixtures only when it reduces duplication without disguising the two loader versions. No generic legacy-to-new YAML converter belongs in tests. Acceptance: existing application configuration coverage exercises v1.2.0 with intentional semantic changes documented in tests; LLM fixtures still use old types. Checks: `GOWORK=off go test ./internal/core/config ./internal/framework/llm` plus CLI compile/tests, noting discovery assertions assigned to Stages 6–7. Prompt size: mechanical fixture migration; avoid unrelated production refactoring. ## Stage 6 — Delegate CLI Discovery And Publish The Actual Selected File **Complete.** The shared CLI loader selects one ConfigKit source and reports the retained main path; manual probing was removed. Focused load, selection, diagnostic, and report tests and a native temporary build pass. The expanded discovery matrix is covered separately below. Goal: replace filesystem probing with the settled native source selection. Inspect loadConfigKit, discoverConfigPath/requireConfigFile, command dispatch, Options.LookupEnv, and invocation/debug use of loadedConfigPath. Work: - Construct exclusive File or ordered required SearchConfig sources as specified. Pass one source to LoadSource. Remove obsolete stat/probe loops and single-default constants once unused. Keep blank optional flag/environment handling consistent. - Populate the selected-path return from MainSourcePath. Preserve canonical source-relative application defaults and the actual path in requested debug data. - Route load failures through new ConfigKit structured diagnostics/source legends. Keep ordinary syntax exit 2 and acquisition/validation exit 1 behavior; do not parse diagnostic prose. Retain old PromptKit error handling for its own reports. - Update all run/config/report/list callers through the shared boundary. Do not resolve pipelines or call providers merely to discover/report configuration. Acceptance: no manual file search precedes ConfigKit; every config-consuming command uses the same selected source and its retained provenance. Checks: focused CLI load/discovery/report tests and native build. Prompt size: shared CLI loading boundary, not the full discovery test matrix. ## Stage 7 — Verify Discovery, Inspection, And Failure Semantics **Complete.** Controlled discovery tests cover precedence, filename ordering, terminal failures, canonical symlinks, required exhaustion, reports, and the selected path recorded by an injected-client run. Focused tests pass; the broader CLI suite still awaits the maintained-example migration. Goal: protect the new operator contract with bounded offline coverage. Inspect CLI command_contract/configkit_load/configkit_commands tests and core/config loaded/source tests. Use real CWD/user-directory behavior in tests that control process state and are not parallel. Work: - Cover CLI over environment, environment over discovery, CWD over user selection, native named-file ordering, and the absence of generic CWD config.yml selection. Bad explicitly selected files and invalid selected CWD files fail without fallback. - Isolate user directories with controlled platform environment. Avoid tests that write /etc or depend on the host's system configuration. Rely on upstream for preset directory expansion; test required-search exhaustion with a controlled SearchConfig passed to LoadSource rather than assuming host directories are empty. - Verify selected-file path and declaring-directory defaults after discovery, source-local overlays/imports, canonical symlink source behavior, and CLI paths. - Exercise effective/explain/validate/list and a recorded run with an injected external client. Assert safe source diagnostics, origins, provider-free reports, ordinary optional absence without warnings, and unchanged PromptKit scoped reports. Acceptance: selection precedence and terminal failures are observable through commands without a second acquisition pass or host-dependent fixtures. Checks: `GOWORK=off go test ./internal/core/config ./internal/cli ./internal/framework/llm`. Prompt size: discovery and cross-boundary behavioral verification. ## Stage 8 — Maintain Runnable Native Composition Examples **Complete.** The complete example imports its first step as a required native sequence item; existing order, handoff, and inheritance assertions pass. The CLI and module-integration suites and credential-free minimal, complete, and development validation pass. An old missing-source diagnostic assertion was also updated to match the delegated loader. Goal: demonstrate the approved features in the existing two top-level examples. Inspect examples/dnd-complete.config.yml, pipelines/dnd-session.yml, development overlay, profile file, and internal/cli/example_contract_test.go. Work: - Change the complete root's import to `config` and make it explicitly required. - Move its describe-session step into a new sibling fragment `examples/pipelines/dnd-session.describe-session.yml`, containing the step mapping. The pipeline fragment imports it with `required: true`, `destination: [pipelines, dnd-session, steps]`, and `placement: sequence-item`; retain extract-events in the declaring fragment's steps list. Native dependency- ready composition places the imported first step before the declaring body. - Keep the new fragment beside the old one so existing file-relative references retain their bases. Preserve lane IDs, module options, generated references, pipeline behavior, and the derived development profile. Leave the old PromptKit deployment document's syntax unchanged. - Update example tests to resolve the complete/development pipelines, assert actual step order and handoffs, and validate without credentials/inference. Retain exactly the existing two top-level examples; the added fragment is supporting content. Acceptance: maintained configurations load natively and demonstrate composed steps plus inheritance with no duplicated complete pipeline or altered D&D execution policy. Checks: existing CLI example tests and module-integration tests; credential-free CLI validation of minimal, complete, and development selections. Prompt size: one supporting fragment and existing example coverage. ## Stage 9 — Update Operator And Consumer Documentation Goal: document implemented native behavior and migration in canonical homes. Work: - Update docs/config.md for selected-source order/filename groups, root controls, optional imports/explicit requiredness, step identity/upsert/replace/append, record inheritance, destinations, native CLI timing, and path bases. Remove obsolete universal whole-list-replacement/step-inheritance limitations. - Update docs/cli.md discovery and report behavior, linking to config's field rules. Keep explicit --config advice first-class in subprocess/operations guidance. - Explain the two configuration formats at the temporary boundary: Notarius uses new controls, PromptKit operator documents still follow its old engine. Show concise migration snippets and link the maintained complete files. - Update README's consumer-minimum rationale for the newly selected ConfigKit requirement and release setup to Go 1.27.0. Keep source-only publication and existing release gates. Update actual CI toolchain configuration if present; create no new CI workflow or publication mechanism. Acceptance: operator guidance explains real new semantics and intentional migration without introducing extra import restrictions or promising unchanged composition. Checks: relevant example tests, link/command review, and `git diff --check`. Prompt size: public references and concise consumer/workflow adjustments. ## Stage 10 — Update Integration References And Attribution Goal: make ownership and the temporary mixed graph accurately documented. Work: - Update docs/internal/configuration.md and docs/internal/cli.md for LoadSource, selected-path provenance, field-level resolution, discovery, and source parity. Remove stale blanket-list-recording and manual search descriptions. - Update docs/integrations/pkg-configkit.md to v1.2.0/new module/pinned contracts. Explain the temporary old dependency in the PromptKit/LLM integration references without documenting its entire future migration. ADR-0020 already owns the decision. - Update docs/oss/attribution.md for new ConfigKit at the full source commit, retain its old-path dependency's notice, and review actually selected transitive modules/tool requirements. Inspect released COPYRIGHT/LICENSE/notices; reuse unchanged standard texts. Preserve the project grant and policy provenance. - Keep upstream release tooling and its gates upstream. Dependency attribution does not adopt ReleaseKit as Notarius's release implementation. Acceptance: current docs and factual notices match the selected mixed graph; historical ADRs/releases remain untouched. No policy or license grant is expanded. Checks: selected module inspection, pinned/local link and notice review, whitespace check. Reuse unchanged example validation rather than adding documentation snapshots. Prompt size: focused internal/integration docs and dependency notices. ## Stage 11 — Verify The Entire Target And Finish The Sprint Goal: prove all feature-roadmap acceptance on the completed implementation. Work: - Compare final code, examples, tests, and docs against each target section. Check new ConfigKit v1.2.0, old v0.5.0 for PromptKit, YAML rc.6, unchanged PromptKit v0.12.0/catalogs, Go minimum/toolchain, native controls, discovery, step composition, sources/paths, leaf histories, and frozen inspection. Correct small residual defects. - Review remaining old imports/control strings by owner, not a blanket grep ban. Old PromptKit APIs and historical release text legitimately remain. Remove obsolete application wrappers/probing and contradictions; retain meaningful behavioral tests. - Run full offline tests, vet, tidy diff, native temporary build, and source/version inspection. Smoke-test config validate on all maintained selections and both ordinary/PromptKit inspection reports. Use explicit fixture input only for offline injected-client run tests; do not invoke a production pipeline for verification. - Review accidental strengthening: no extra permissions, mandatory secret lookup, fragment-version requirement, stricter import defaults, hidden list merger, compatibility shim, or new release gate. Leave unrelated prior findings and roadmap retirement to their own work sets. - Report actual checks and remaining limitations. Upgrade completion requires a buildable graph and passing target behavior, not only a successful go get. Validation commands, with dependencies prepared and Go 1.27.0 selected: ```sh GOWORK=off go test ./... GOWORK=off go vet ./... GOWORK=off go mod tidy -diff GOWORK=off go list -m go.maximumdirect.net/configkit gitea.maximumdirect.net/eric/ConfigKit gitea.maximumdirect.net/eric/promptkit go.yaml.in/yaml/v4 git diff --check ``` Build ./cmd/notarius beneath a mktemp directory outside the repository. These are sprint verification steps, not added permanent release gates. No file changes means no empty final commit. Release numbering, notes/publication, ConfigKit v1.2.1/stable YAML, and the full PromptKit upgrade remain separate work. Prompt size: final evidence review and small cleanup; significant newly discovered scope requires an explicit decision rather than silently narrowing acceptance. ## Open Questions None. The intermediate version, source selection, step identity, and native configuration policies are approved. The mixed graph remains an early technical verification task; report a concrete incompatibility if one appears instead of silently selecting another version or adding compatibility machinery.