Plan the ConfigKit upgrade and workspace development
This commit is contained in:
319
docs/roadmap/implementation.md
Normal file
319
docs/roadmap/implementation.md
Normal file
@@ -0,0 +1,319 @@
|
||||
# ConfigKit v0.5.0 implementation plan
|
||||
|
||||
## Scope and authority
|
||||
|
||||
Implement [the ConfigKit roadmap](config.md). It owns the target, compatibility
|
||||
choices, and rationale. This plan covers Narratio's upgrade from v0.4.2 to
|
||||
v0.5.0 and integration with native sibling inheritance. Audita adoption and
|
||||
workspace reorganization remain subsequent, separately planned work.
|
||||
|
||||
The reviewed upstream revision is
|
||||
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`, tagged `v0.5.0`.
|
||||
Use the released module, not a local replacement. The pinned
|
||||
[configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/config.md)
|
||||
and [consumer guide](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/consumers/README.md)
|
||||
own upstream semantics. Read their sibling-inheritance and metadata sections
|
||||
when working on the relevant stages; source may also be inspected in the
|
||||
selected Go module cache. The sibling checkout is optional reference material,
|
||||
not a build or test dependency.
|
||||
|
||||
## Working instructions
|
||||
|
||||
The audience is a Luna coding agent. Complete the numbered stages in order.
|
||||
Each stage is a bounded unit with its own verification and completion criteria.
|
||||
Inspect current code before editing; the paths below identify owners rather
|
||||
than requiring changes to every listed file.
|
||||
|
||||
- Start with `docs/development.md`, `docs/policy/agents.md`, and `config.md` in
|
||||
this directory. Follow the development policy-reading guide. Architecture,
|
||||
testing, documentation, and OSS policies apply to this work; release policy
|
||||
applies if discussing compatibility or eventual release preparation.
|
||||
- Use Go 1.26. Prefer the repository's graph tools for code discovery and
|
||||
inspect focused tests at each package boundary.
|
||||
- Keep ordinary `Load`, typed defaults, explicit imports, one selected overlay,
|
||||
domain validation, and current path-resolution rules. Native inheritance is
|
||||
available wherever ConfigKit and Narratio's schema permit it, with no new
|
||||
collection allowlist, opt-in flag, or Narratio-owned inheritance engine.
|
||||
- Distinguish original declarations from inherited contributors. Preserve
|
||||
genuine duplicate-base rejection and truthful provenance. Parent entries
|
||||
remain ordinary entries subject to existing domain rules.
|
||||
- Keep invocation-owned secrets bootstrap and shared exact-name credential
|
||||
forwarding unchanged. Do not introduce a second base load, file reparse, or
|
||||
historical-secret lookup to recover declaration information.
|
||||
- Tests use temporary files, controlled synthetic secrets, and real internal
|
||||
collaborators. No downstream executable, sibling checkout, cloud service,
|
||||
or paid provider is needed. Protect integration behavior rather than
|
||||
duplicating ConfigKit's generic algorithm tests or freezing incidental output.
|
||||
- Commit each completed nonempty unit with a short plain-English message after
|
||||
suitable validation. Report blockers and incomplete work accurately. If a
|
||||
review-only task needs no source fix, retain useful behavioral coverage and
|
||||
report that result rather than making a cosmetic rewrite.
|
||||
|
||||
Do not adopt layered loading, new source types, source-relative rebasing,
|
||||
abstract artifacts, automatic YAML conversion, new execution selection rules,
|
||||
new output collision policy, or changes to subprocess credentials and workspace
|
||||
paths in this feature. Existing domain validation and reuse rules still apply;
|
||||
do not add prohibitions, hardening, or guarantees merely because they can be
|
||||
tested. This plan does not authorize a release, push, tag, ADR acceptance, or
|
||||
retirement of pending roadmaps.
|
||||
|
||||
## Stage 1: Select the released dependency and update attribution
|
||||
|
||||
Purpose: establish the actual dependency baseline before integration work.
|
||||
|
||||
Read `go.mod`, `go.sum`, `docs/oss/attribution.md`, the OSS policy, and the
|
||||
selected release's notes, `COPYRIGHT`, and `LICENSE`.
|
||||
|
||||
1. Select `gitea.maximumdirect.net/eric/ConfigKit@v0.5.0` with the normal Go
|
||||
module workflow. Keep Go 1.26 and avoid unrelated dependency upgrades.
|
||||
2. Update ConfigKit's attribution entry to v0.5.0 and use the full reviewed
|
||||
commit for its Source, COPYRIGHT, and LICENSE links. Preserve upstream
|
||||
copyright wording, SPDX expression, and applicable local license links.
|
||||
Its license and YAML dependency are unchanged; no new license copy is needed.
|
||||
3. Inspect module changes for unintended churn or a local `replace`. Existing
|
||||
configuration loaders should compile without adopting optional APIs.
|
||||
|
||||
Verify with `go test ./internal/config ./internal/app ./internal/doccheck`
|
||||
and `go mod tidy -diff`. Confirm
|
||||
`go list -m gitea.maximumdirect.net/eric/ConfigKit` selects v0.5.0 without a
|
||||
replacement.
|
||||
|
||||
Complete when the released dependency and notices agree and current focused
|
||||
tests pass. The cross-file inheritance defect is addressed in the next stage;
|
||||
this stage does not claim that new inheritance behavior is fully integrated.
|
||||
|
||||
## Stage 2: Correct direct declaration accounting
|
||||
|
||||
Purpose: allow distinct artifact and family keys to inherit across imported
|
||||
files while preserving existing duplicate-declaration rejection.
|
||||
|
||||
Read `internal/config/pipeline_configkit.go`, particularly
|
||||
`pipelineMetadataFromConfigKit` and `rejectDuplicateBaseDefinitions`, and
|
||||
`internal/config/pipeline_configkit_test.go`. Inspect the public ConfigKit
|
||||
`Metadata.History`, `HistoricalPaths`, `Origin`, and `ChangeKind` contracts.
|
||||
|
||||
1. Add package-boundary regression fixtures with a valid parent in one imported
|
||||
file and a child using `extends` in another. Cover ordinary artifacts and
|
||||
artifact families. Give parent and child distinct valid output paths or
|
||||
family patterns, and provide canonical-party context for expanded families.
|
||||
Both keys remain ordinary domain entries; child IDs must stay their own.
|
||||
2. Adjust declaration accounting using retained public metadata. In v0.5.0,
|
||||
original base-input events have `LayerBase` and no origin operation;
|
||||
projected transfers carry `ChangeInheritance`. Count original base
|
||||
declarations, not these transfers or subsequent decoding/resolution events.
|
||||
Consumed structural `extends` paths are not artifact declarations.
|
||||
3. Keep the historical-path check, including original declarations erased by
|
||||
a selected overlay. Do not switch to surviving effective keys only or count
|
||||
all displaced origins as fresh declarations. Reuse a small package-local
|
||||
helper only if it makes the next stage clearer.
|
||||
4. Retain `TestPipelineConfigKitRejectsDuplicateBaseDeclarations`, including
|
||||
disjoint fields supplied to the same key, container removal, and removal of
|
||||
Scriptorium by an overlay. An overlay may still modify an existing entry.
|
||||
Add only any missing regression case needed to exercise the new filter.
|
||||
|
||||
Verify with `go test ./internal/config`. The positive cross-file regression
|
||||
should expose the baseline defect before the fix. Assert loaded values and
|
||||
identities, or the duplicate category and relevant sources, without matching an
|
||||
entire diagnostic or ConfigKit's private event ordering.
|
||||
|
||||
Complete when cross-file inheritance loads and expands correctly, and actual
|
||||
duplicates still fail even when the overlay hides them. Do not use the direct
|
||||
declaration filter to remove contributor evidence from inspection.
|
||||
|
||||
## Stage 3: Preserve contributor provenance and family ownership
|
||||
|
||||
Purpose: keep inspection useful when one entry contains values from several
|
||||
files, without confusing its defining file with those contributors.
|
||||
|
||||
Read `internal/config/artifact_catalog.go`,
|
||||
`internal/config/artifact_families.go`, `internal/config/effective_pipeline.go`,
|
||||
and metadata-to-ownership projection in `pipeline_configkit.go`. Inspect the
|
||||
focused catalog, family, and effective-source tests.
|
||||
|
||||
1. Exercise the inherited configurations from Stage 2 through
|
||||
`EffectiveArtifactCatalog` and `EffectivePipelineSources`. Check inherited
|
||||
values against their actual parent contributors and child overrides against
|
||||
the child/overlay contributors. Keep historical definition evidence distinct
|
||||
from effective field ownership.
|
||||
2. Preserve `DefinitionSources` aggregation, including inherited contributors.
|
||||
Do not relabel all inherited values as authored in the child's file.
|
||||
3. Correct `artifactFamilySource` where its first inherited field currently
|
||||
selects the parent's file. For an explicitly declared child family, identify
|
||||
its original declaration using retained metadata: use its base defining file
|
||||
when present, or its overlay defining file for an overlay-only family.
|
||||
Overlay edits to a base family do not relocate its base definition.
|
||||
4. Retain the existing fallback for entries without a direct declaration or
|
||||
programmatically constructed configuration. Do not restrict valid enclosing
|
||||
inheritance merely to force every entry into a single-file ownership model.
|
||||
5. Keep logical `SourceID`, artifact keys, family/character/party provenance,
|
||||
existing source roles, and inspection interfaces. Update projections only
|
||||
where inherited evidence is otherwise lost or incorrectly attributed.
|
||||
|
||||
Verify with `go test ./internal/config`. Cover an explicitly declared child
|
||||
family inheriting from another file and a selected-overlay contribution. Check
|
||||
meaningful records and source identities, not a complete formatted snapshot.
|
||||
|
||||
Complete when contributor reporting remains truthful and generated members
|
||||
identify their child family definition rather than an arbitrary inherited field.
|
||||
|
||||
## Stage 4: Verify inherited presence, domain resolution, and path bases
|
||||
|
||||
Purpose: prevent Narratio's application defaults or normalization from undoing
|
||||
the library's inherited explicit input.
|
||||
|
||||
Read `pipelineConfigDefaults`, `resolvePipelineConfig`, and
|
||||
`internal/config/load.go`, including `finalizeLoadedPipeline` and
|
||||
`resolveNotariusPaths`. Inspect the current presence/defaulting tests and
|
||||
the relevant domain validators.
|
||||
|
||||
1. Add representative inheritance cases for explicit false, zero, empty,
|
||||
and null input versus genuinely absent input. Include a pointer boolean
|
||||
and an entry-level absence-based default where Narratio consumes metadata.
|
||||
For example, inherited `render_debug: false` can exercise artifact presence;
|
||||
inherited empty Scriptorium timeout can exercise an existing invalid value
|
||||
that must not silently become its default.
|
||||
2. Preserve the existing domain outcome for each chosen field. Valid explicit
|
||||
values retain their value; explicit values already invalid must still fail
|
||||
ordinary validation rather than be refilled. An empty mapping merges under
|
||||
ConfigKit's rules and must not be described as universal container removal.
|
||||
3. Exercise overlay composition before inheritance: modify a parent's field
|
||||
in the selected overlay and confirm the child's inherited result uses that
|
||||
effective value. A child override still wins over its parent.
|
||||
4. Verify a known root-relative field, such as enabled Notarius `config_path`
|
||||
or `working_directory`, when inherited from an imported declaration.
|
||||
Resolve it exactly as the existing field contract does. Do not use passive
|
||||
SourceContext to rebase it to the parent's or child's source directory, or
|
||||
newly normalize other path fields merely to make their behavior uniform.
|
||||
5. Fix any demonstrated Narratio integration defect at its existing defaulting
|
||||
or domain-resolution owner. Use ConfigKit presence and recorded operations;
|
||||
do not recreate structural inheritance or weaken ordinary validation.
|
||||
|
||||
Verify with `go test ./internal/config`. Keep coverage representative; generic
|
||||
missing-parent, cycle, resource-context, and expansion-limit matrices belong to
|
||||
ConfigKit. Parents must remain valid Narratio entries, not special templates.
|
||||
|
||||
Complete when inherited explicit input and absent defaults have distinct,
|
||||
correct domain outcomes and existing path bases are unchanged.
|
||||
|
||||
## Stage 5: Verify safe effective projections and inherited credentials
|
||||
|
||||
Purpose: ensure inheritance changes neither semantic comparison nor the
|
||||
existing secret boundary.
|
||||
|
||||
Read `internal/config/effective_pipeline.go`, `effective_digest.go`,
|
||||
`effective_projection.go`, `scriptorium_credentials_test.go`, and the effective
|
||||
projection tests in `load_validate_test.go`. For diff composition, read
|
||||
`internal/app/config_diff.go`. Reuse
|
||||
`TestPipelineOverlayDoesNotRequireUnusedBaseCredentials` and
|
||||
`TestCredentialValuesNeverEnterEffectiveProjectionOrDigest` coverage.
|
||||
|
||||
1. Load two configurations that produce the same effective domain values:
|
||||
one uses inheritance, the other writes the expanded values explicitly.
|
||||
Include ordinary artifacts and representative resolved family members.
|
||||
Verify equal safe effective mappings and digests and no effective diff.
|
||||
Do not compare private provenance as part of semantic equality.
|
||||
2. Add a representative typed credential inheritance case using shared fields
|
||||
of valid Narratio sibling mappings and controlled synthetic environment
|
||||
variables. Verify resolved credentials retain their exact configured names;
|
||||
serialization and inspection expose those identities without values.
|
||||
3. Verify overlay removal/replacement of a credential before inheritance does
|
||||
not resolve the superseded reference. Reuse the existing selected-overlay
|
||||
regression rather than creating a second loader or bootstrap algorithm.
|
||||
4. Check that changing only synthetic secret values leaves semantic projections
|
||||
and digest unchanged. Fix only demonstrated projection defects; preserve
|
||||
the existing safe typed rendering boundary and semantic reuse evidence.
|
||||
|
||||
Verify with `go test ./internal/config ./internal/app`. Existing shared adapter
|
||||
credential-forwarding tests remain the owner of subprocess forwarding; do not
|
||||
rewrite adapters or duplicate those tests simply because input can be inherited.
|
||||
|
||||
Complete when resolved values, not source layout or secret bytes, determine
|
||||
effective comparison, and inherited credentials obey the existing safe boundary.
|
||||
|
||||
## Stage 6: Document inheritance and maintain a small example
|
||||
|
||||
Purpose: make the adopted behavior understandable and usable without expanding
|
||||
Narratio's configuration surface.
|
||||
|
||||
Read `docs/config.md`, `docs/internal/configuration.md`, `examples/README.md`,
|
||||
the production/testing bundle, and `TestExamplesLoadAndValidate` in
|
||||
`internal/config/load_validate_test.go`. Follow the documentation policy.
|
||||
|
||||
1. Add native sibling inheritance guidance to the configuration reference.
|
||||
Explain exact sibling lookup, overlay-before-inheritance resolution,
|
||||
explicit presence, ordinary parent validation, distinct child identity,
|
||||
and cross-file declaration/contributor distinctions. Link to the pinned
|
||||
upstream contract for detailed syntax and structural rules.
|
||||
2. Explain the reserved `extends` key throughout ConfigKit inputs, including
|
||||
dynamic artifact IDs and prompt-variable mappings. Operators must rename
|
||||
existing ordinary-data keys with that name; there is no opt-in or automatic
|
||||
conversion. Campaign, session, and party inputs also use ConfigKit.
|
||||
3. Clarify the existing statement that overlays cannot inherit: overlay files
|
||||
cannot inherit or stack, but the selected overlay may supply native sibling
|
||||
directives within its values. Keep imports and overlay selection unchanged.
|
||||
4. Update configuration internals for inheritance before typed decoding,
|
||||
original-declaration counting, contributor projection, and the safe
|
||||
effective boundary. Describe implemented mechanisms, linking to the public
|
||||
reference for their operator-facing meaning.
|
||||
5. In `examples/production-testing/overlays/testing.yml`, add
|
||||
`extends: session_recap` to the existing `testing_notes` artifact. Keep its
|
||||
disabled status and distinct output override. It inherits the imported
|
||||
parent's prompt/input and the parent's testing profile after overlay
|
||||
application. The parent remains an ordinary artifact.
|
||||
6. Update the example guide and extend existing example validation to check
|
||||
those inherited fields. Preserve the bundle's current identities and
|
||||
artifact count; no new example root or template convention is necessary.
|
||||
|
||||
Verify with `go test ./internal/config -run '^TestExamplesLoadAndValidate$'`
|
||||
and `go test ./internal/doccheck`. Review examples for synthetic-only content
|
||||
and validate local links. Keep unimplemented Audita/workspace behavior in its
|
||||
roadmaps. Do not create a release note under an invented Narratio version;
|
||||
reserved-key compatibility impact remains an explicit release-preparation item.
|
||||
|
||||
Complete when canonical documentation and a tested copyable example describe
|
||||
the implemented adoption, without promoting new restrictions to policy.
|
||||
|
||||
## Stage 7: Verify the assembled inspection workflow and close the feature
|
||||
|
||||
Purpose: establish that real command composition consumes the integrated
|
||||
configuration correctly and finish the upgrade as a reviewable baseline.
|
||||
|
||||
Read `internal/app/config_commands_test.go`, `config_artifacts_test.go`,
|
||||
and existing read-only inspection workflow coverage. Reuse the preceding
|
||||
package coverage rather than repeating its entire matrix at the CLI boundary.
|
||||
|
||||
1. Add or extend a representative command-level temporary imported bundle with
|
||||
inheritance. Exercise validation, effective show, source reporting, artifact
|
||||
catalog, and diff through existing command entry points. Use a campaign/party
|
||||
fixture for family inspection, and paired overlays with equivalent inherited
|
||||
and explicitly expanded values when testing command-level diff equality.
|
||||
2. Check semantic output: resolved child values, expected contributor/defining
|
||||
sources, logical artifact identities, and no secret values. Keep inspection
|
||||
read-only with respect to session/runtime state, as existing tests establish.
|
||||
Do not invoke downstream applications or add CLI modes or columns.
|
||||
3. Review the combined diff against `config.md` and this plan. Confirm that
|
||||
ConfigKit v0.5.0 is selected, attribution links use the reviewed revision,
|
||||
no optional loading feature was adopted, and no Audita/workspace change or
|
||||
compatibility shim slipped into scope.
|
||||
4. Run `go test ./...`, `go test -race ./...`, `go vet ./...`,
|
||||
`go build ./...`, `go mod tidy -diff`, and `git diff --check` against the
|
||||
final source. These reuse existing repository checks, not new release gates.
|
||||
The full suite includes example and documentation-link validation; focused
|
||||
checks need not be repeated solely for this stage if unchanged.
|
||||
5. Commit the completed unit and report the dependency revision, checks,
|
||||
compatibility impact, and any unfinished work. Leave the pending Audita and
|
||||
workspace plans and Proposed ADRs intact. Retire completed planning files
|
||||
only during separately requested cleanup/release preparation, repairing links.
|
||||
|
||||
Complete when the feature roadmap's completion criteria are evidenced by the
|
||||
released dependency, integrated behavior, maintained documentation/examples,
|
||||
and passing final checks. The next implementation scope is Audita; its concrete
|
||||
plan is prepared separately. Release version selection and publication are
|
||||
outside this implementation pass.
|
||||
|
||||
## Open Questions
|
||||
|
||||
None required for this adoption. Existing decisions select native inheritance,
|
||||
current loading/path/credential semantics, and ConfigKit before Audita before
|
||||
workspace work. Eventual release preparation must account for the reserved-key
|
||||
compatibility impact under the release policy; no version is selected here.
|
||||
Reference in New Issue
Block a user