Files
configkit/docs/roadmap/implementation.md

345 lines
21 KiB
Markdown
Raw Normal View History

# Implement Document-Root Configuration Controls
Status: ready for implementation by a Luna coding agent.
Implementation baseline: `b27fe0598a9540ad05eefe2868a5a243eb28111e`.
The [feature roadmap](config-directives.md) owns the target and its rationale;
this document owns execution order and verification.
## Working rules and settled decisions
Follow the [development guide](../development.md), [agent policy](../policy/agents.md),
[architecture policy](../policy/architecture.md), [testing policy](../policy/testing.md),
and [documentation policy](../policy/documentation.md). This is one end-to-end
sprint. Intermediate stages may leave compilation or existing fixtures broken;
do not introduce compatibility code, skip tests, or weaken assertions to make
an intermediate stage green. Use focused checks as their fixtures become ready,
then run the complete development check at the end.
The following decisions are settled:
| Input or operation | Target behavior |
| --- | --- |
| Root `config` in any acquired source document | Parse as the document's control mapping before placement, preserving the existing context-specific fields, validation, and import opt-in. |
| Nested application `config` | Ordinary data, including inside sequence records and selected overlays. Children named `imports` or `sequences` do not activate controls. |
| A source mounted beneath an application destination | Interpret that source's own controls first; its mounting location does not change its document root. |
| Application-root `/config` produced by placement, defaults, CLI, or resolution | Ordinary data. Do not re-extract controls from assembled or typed values. |
| `configkit` and `_configkit` application keys | Ordinary data at any depth, with no control alias or special legacy-key error. Normal schema and other structural rules still apply. |
| `extends` | Preserve its existing inheritance semantics and restrictions, including CLI rejection. |
| Source versions | Protect actual source-local controls; retain existing header modes, conversion, main ownership, consumption, and disclosure rules. A matching application-path spelling alone is not control overlap. |
Keep exported signatures, the module/package name, `configkit:"secret"`,
diagnostic identifiers, and the `configkit:` library error prefix. Preserve
composition, ordering, limits, ownership, and secret-handling behavior. Do not
add keyword configuration, control discovery after acquisition, public record
fields, runtime watchers, dependencies, or release gates.
When migrating literals, distinguish root control syntax and control-evidence
paths from package imports, struct tags, error text, ordinary application keys,
and application paths. A repository-wide replacement of `configkit` or
`configkit:` is incorrect. Keep useful old-spelling inputs as positive ordinary
data regressions. A former rejection test needs a deliberate new expectation;
renaming its input while retaining the old rejection is not sufficient.
## Stage 1 — Record the agreed architectural boundary
**Status:** Complete.
**Scope:** one decision record, without changing current-behavior documentation.
1. Create `docs/adr/0020-document-root-controls.md`, using the repository's ADR
format, status Accepted, and the date of authorship. Record the decisions in
the table above: per-source root recognition, ordinary nested and assembled
root data, and one canonical spelling.
2. Explain the source-local versus application-coordinate distinction and why
placement and later lifecycle operations do not create another parsing pass.
Keep the decision focused on ConfigKit. Discuss rejected choices only under
Alternatives Considered; avoid framing the decision around downstream apps.
3. Identify the reservation-scope portion of ADR 0011 as amended, while retaining
its placement decisions. Update only that earlier ADR's status section with
a link to the new decision and a statement that its other decisions remain
in force. Leave accepted decision bodies and historical release notes intact.
**Complete when:** the ADR records the approved scope, including ordinary
application-root data, without claiming implementation is finished. Add its
development-guide navigation with the final documentation stage.
## Stage 2 — Implement source-root control extraction and guards
**Status:** Complete. Focused document-root control and extraction tests pass.
**Owners:** [directives.go](../../internal/engine/directives.go), its callers in
[acquisition.go](../../internal/engine/acquisition.go) and
[selected_overlay.go](../../internal/engine/selected_overlay.go), and focused
directive/declaration tests.
1. Change the engine's control-key constant to `config`. Keep the existing
`parseRootDirectiveBlock` grammar and context distinctions. Root extraction
must clone/remove the selected block as before and return the remaining
application body without recursively rejecting names in that body.
2. Remove `nestedDirectiveDiagnostics` and `disabledNestedDirectiveDiagnostics`
once their callers no longer need them. Make both imports-disabled guard
paths inspect only the original document's root. Preserve the distinction
between ordinary/layered sequence declarations and the private imports-only
extraction context. Root imports with imports disabled still fail.
3. Make `overlayDirectiveDiagnostics` inspect only the original overlay root.
Preserve its current root acceptance rules and diagnostics, including the
allowed collection-only block and rejected imports. Nested application
`config` values must not trigger overlay-directive errors.
4. Check acquisition call sites so these guards receive acquired source trees,
not placed or composed application trees. Preserve extraction before
mounting and existing processing-result reuse; add no second extraction
pass. Leave source-version overlap changes to Stage 4.
5. In a focused `internal/engine/directive_scope_test.go`, cover a valid new
root block; nested `config` with control-looking children in a mapping and
record; ordinary old spellings; malformed/unknown root controls; disabled
imports; and overlay root restrictions. Reuse existing declaration fixtures
for their detailed grammar and migrate the literals needed by these tests.
**Complete when:** these focused cases distinguish source roots from data, with
unchanged error families for still-invalid controls. Removing a recursive guard
must not remove validation of fields inside the actual control block.
## Stage 3 — Permit application paths and CLI data
**Status:** Complete. Focused destination, registration, and override tests pass.
**Owners:** [destination.go](../../internal/engine/destination.go),
[sequence_registration.go](../../internal/engine/sequence_registration.go),
[override.go](../../internal/engine/override.go), and their existing tests.
1. Remove the `configkit` name ban from `normalizeDestination`; do not replace
it with a `config` ban or an application-root prefix check. Retain path
validity, mapping-key normalization, explicit-index rejection, and the
existing `extends` restriction. This applies to root and nested destinations
and to relative suffixes beginning with `config`.
2. Let exact and wildcard identity registrations use literal `config` and
`configkit` segments through the shared validation. Preserve wildcard
matching, identity normalization, conflicting-registration checks, and
final sequence validation. YAML declarations inherit the same path change.
3. Remove `pathContainsDirective` and its CLI path rejection. Preserve the
independent inheritance-path check. Narrow recursive override-value control
detection to its continuing `extends` responsibility; `config` and old
spellings are ordinary data anywhere in a CLI value. Simplify private names
and parameters where the removed responsibility makes them misleading.
4. Keep CLI type checking, root-path/index restrictions, overlap detection,
protected-version checks, labels, accounting, and assignment order intact.
Ordinary root data at `Keys("config")` is not the empty/root override path.
5. Revise destination, registration, and override tests. Move formerly invalid
`configkit` cases into positive data cases. Where a table still needs a
reserved-key rejection, use the continuing `extends` rule rather than
treating `config` as reserved application data. Cover root and nested
`config`, relative suffixes, exact/wildcard registrations, and CLI values
containing ordinary `imports`/`sequences` fields. Retain meaningful negative
cases for indexes, invalid paths, inheritance controls, and version writes.
**Complete when:** literal application keys work through the shared path APIs
without a new per-caller exception or configurable reservation policy. Typed
defaults and resolution must not gain a new schema/key rejection.
## Stage 4 — Verify source-version and acquisition integration
**Owners:** [source_version.go](../../internal/engine/source_version.go),
`processAcquiredInputVersion` in acquisition, import processing, and existing
source-version/item-policy tests.
1. Use the new control constant in actual source-root overlap detection. Keep
`sourceVersionHeaderControlOverlap` tied to an observed source-local header
and the original source tree. Preserve malformed-control validation before
a checker can consume that content, along with `extends` overlap handling.
2. Trace ordinary, layered, imported, sequence-item, and overlay processing.
Retain the distinction between local version lookup and effective main
ownership after mounting. Do not add a preflight ban on a version path merely
because it contains or begins with `config`.
3. Migrate the source-version test inputs and source-local expected paths.
Preserve absent/null behavior, callback order, consumption of accepted
non-main headers, conservative error/callback redaction, and existing codes.
4. Add focused cases for a header under an ordinary nested `config`; a real
header inside root `config.imports` being rejected before its checker; and
an enabled-mode main with local `version: 4` mounted at `[config]`. For the
mounted case, check the local header, final `/config/version`, and rejection
of an override targeting that protected application path. Include an
assembled-root case under the default mode using its existing effective-path
rule, rather than changing MainOnly semantics to match enabled modes.
**Complete when:** application-root data and source-root controls are not
confused by version checking. Existing version behavior needs no new mode,
schema exception, or record format. Run focused SourceVersion tests once their
affected fixtures have been migrated.
## Stage 5 — Migrate engine source and acquisition fixtures
**Scope:** remaining engine source/discovery/import/placement/resource-context
tests and acquisition-related benchmarks, excluding structural/history fixtures
assigned to Stage 6.
1. Migrate genuine root control blocks in import specs, import discovery and
evidence tests, filesystem/equivalence tests, discovery-load tests,
destination/import lifecycle tests, and standalone/grouped acquisition
tests. Cover escaped Go strings, raw multiline YAML, and mapping-source
keys; do not rely on a root-line text replacement alone.
2. Update expected declaration/empty-selection paths and coordinates to the
new source-local spelling. Preserve source labels and IDs, physical paths,
namespace identity, acquisition order, and effective destinations.
3. Convert nested-name rejection cases into ordinary-data cases where required.
Keep the file/containment/permission/repetition/limit tests testing their
original risks with valid new control syntax.
4. Include an acquired fragment with its own `config` controls mounted below a
nested destination or as a record. Include a payload mounted at root
`[config]` and a transitive relative destination starting with `config`.
Check successful assembly and source/resource attribution using existing
filesystem test patterns, without a mock traversal-count requirement.
**Complete when:** these tests reach the behavior they intend to exercise with
the new grammar. Run the relevant Source/Import/Glob/Discovery/Destination/
Resource tests; defer unrelated stale structural fixtures to Stage 6.
## Stage 6 — Migrate engine structural and inspection fixtures
**Scope:** remaining engine composition, collection, inheritance, lifecycle,
history, provenance, explanation, sensitivity, and rendering tests/benchmarks.
1. Migrate genuine root controls and control-path expectations in the remaining
engine fixtures, including collection declarations, overlay policy,
inheritance-policy propagation, and item-local policy cases. Preserve
application keys and Go secret tags. Rename fixture comments when they
describe the changed syntax, not the library identity.
2. Retain the existing strict-peer, precedence, collection-operation, and
inheritance assertions. Upsert remains whole-record replacement; the
namespace change does not authorize a different combination algorithm.
3. Extend existing behavioral owners to cover ordinary `config` data surviving
strict composition, a higher-tier override, and sibling inheritance. Verify
explicit false/zero and representative permitted empty/null values at these
paths; reuse existing presence tests rather than constructing a Cartesian
product of every input kind and lifecycle operation.
4. Verify value inspection and source-local control evidence can coexist even
when path text shares a `/config/...` prefix. Use the existing source IDs,
locations, and record contexts; do not invent a path-kind flag or format.
Preserve historical redaction and snapshot ownership through existing test
coverage, adding a scope-specific case only where that coverage leaves a gap.
**Complete when:** the engine suite passes with no obsolete global-name
rejections or control aliases. Run `go test ./internal/engine`; use the final
stage for the complete repository check.
## Stage 7 — Update facade fixtures and public integration coverage
**Scope:** root facade tests and executable package examples.
1. Migrate root-level test and example inputs that intentionally use controls,
including source equivalence, import-default evidence, mounted collections,
version modes, diagnostics, and underlay-origin fixtures. Update expected
source-local paths, preserving package names, error prefixes, and secret tags.
2. Add public API regressions in `facade_config_directives_test.go`, using the
external `configkit_test` package and independently authored expectations:
- A normal load with a new root control and nested typed `config` data.
- A declared import mounted at `[config]`, with ordinary data children named
`imports` or `sequences`. It must not attempt another import pass.
- Defaults-only loading with a typed root `config` field, an explicit CLI
false/zero assignment below it, and a recorded resolution change. Check
entry-state versus final presence/origins and effective values through the
existing public APIs.
- Old spellings as normal data, including a control-looking old block whose
nonexistent import is not executed. Use a permissive schema for this
positive case; separately retain strict unknown-field failures.
3. Use the mounted case to verify import-declaration evidence, final data
lookup, explanation, and effective output describe their proper contexts.
Include a field tagged `configkit:"secret"` under application `config` and
verify ordinary hiding still applies. Do not compare full incidental error
wording or add duplicate engine tests for every parser shape.
4. Keep root facade conversions and exported signatures unchanged unless a
genuine existing conversion bug is exposed; this feature requires no new
bridge API or inspection fields.
**Complete when:** `go test .` passes, including executable API examples and
public/import-boundary checks, with working external use of root and nested
application `config` data.
## Stage 8 — Migrate maintained consumer fixtures and show the boundary
**Scope:** `examples/consumers`, its YAML fixtures, and their executable tests.
1. Migrate genuine root controls in consumer fixtures and inline generated
documents. Update expected declaration paths or displayed errors as needed;
retain domain behavior, application-owned profile concepts, and the existing
source/resource ownership assertions.
2. Add `Example_documentRootConfig` in
`examples/consumers/config_controls_test.go`. Use a small `fstest.MapFS` main
document with `config.imports`, importing a fragment containing ordinary
settings at destination `[config]`. Decode into an application type with a
`yaml:"config"` field and show the resulting value. The example demonstrates
that the source control is consumed while the fragment becomes root data.
Keep the example offline, credential-free, and small enough to copy.
3. Preserve existing complete consumer examples instead of adding a parallel
loader abstraction. Use existing imports, destinations, and recorded-helper
conventions without a second application-side merge.
**Complete when:** `go test ./examples/consumers` passes, including the new
executable example. Existing scenarios still exercise their original features.
## Stage 9 — Update current documentation and public comments
**Scope:** canonical behavior, local architectural descriptions, consumer
migration guidance, and navigation after the implementation is in place.
1. Update the configuration reference's imports, overlays, destinations,
collection declarations/registrations, CLI, version overlap, and earlier
directive spelling sections as one consistency pass. Replace the obsolete
nested-key prohibition with the per-source root boundary. Explain ordinary
application-root data and the existing non-reloadability of inspection YAML.
2. Show a source-root control beside a nested application `config`, and link
the maintained mounted-root example. Explain that every imported file has
its own root, even when mounted below another value or as a sequence item.
3. Add consumer migration guidance: rename genuine old control blocks; move,
rename, or separately mount source-root application fields formerly named
`config`; update stored `/configkit/...` declaration paths to `/config/...`.
Old blocks can decode as data in permissive schemas and no longer execute
controls. Preserve historical release notes and accepted ADR bodies.
4. Update relevant public Go comments and local architecture-policy wording
for source-local controls versus application data. Keep exact YAML rules
in the configuration reference. Explain only affected implementation flow
in internal docs; add the ADR and example to existing development/consumer
navigation. Review README examples and links for any obsolete control syntax.
5. Leave unrelated policy language, licensing, module identity, Go minimum,
dependencies, release commands, and release gates unchanged. Do not prepare
v1.0.0 notes or publish anything during this sprint.
**Complete when:** current docs, declarations, and examples agree on both the
spelling and the boundary, with historical records clearly historical. Keep
implementation progress in this plan; the feature roadmap remains an end-state
document.
## Stage 10 — Complete the integration and scope review
1. Search current production code, tests, examples, and docs for old root
controls, recursive name bans, and stale declaration paths. Review each
remaining old spelling in context; retained package names, tags, error
prefixes, ordinary data tests, and historical records are intentional.
2. Review the complete result against the settled-decision table. In particular,
prove new source controls work across supported source forms; nested and
constructed root data are accepted; controls are interpreted before
placement; CLI/collection/version checks use the right coordinates; and
`extends`/protected-version restrictions remain effective. Use the owning
tests from prior stages instead of adding another duplicate integration suite.
3. Review for unintended scope changes: no source-root data fallback, old-key
compatibility alias, configurable keyword, post-acquisition control parsing,
application-root schema ban, new metadata format, or changes to composition,
defaults, ownership, secret handling, limits, or ordering. Fix discrepancies
rather than altering the approved target to match the implementation.
4. Run `./scripts/check.sh` with dependencies already available. It supplies
formatting/whitespace, all tests and examples, local links, vet, and build.
Reuse successful focused checks for unchanged work; broaden testing only for
concrete unresolved risks. No benchmark, CI wait, macOS gate, or live service
check is required for this migration.
5. Review the final diff and preserve unrelated work and ignored credentials.
Record actual completion/validation in this plan and update the feature
roadmap's overall status when the whole feature is implemented, without
adding stage checklists there. Commit validated, reviewable work under the
agent policy with plain-English messages; no
empty commits, push, release tag, hosted delivery, or consumer dependency
update is part of this implementation sprint.
**Complete when:** the targeted behavior is implemented and documented, the
complete check passes, and the final review finds no remaining gap against the
feature roadmap. There are no unresolved design decisions in this plan.