configkit/docs/roadmap/implementation.md

Pre-v1 Public API Cleanup Implementation Plan

Status: in progress; Stages 1–8 are complete and Stages 9–11 are pending. Baseline: 07e4ae5d9236935389e81b207d4a8f00543fafce. Audience: a Luna coding agent executing the stages sequentially.

Scope And Execution

Implement the accepted changes in the API roadmap. That document owns the target, rationale, and behavioral boundaries. The revision-pinned catalog and additional review preserve evidence and alternatives; they are not additional work lists. The preceding completed interface-remediation plan is preserved at 07e4ae5:docs/roadmap/implementation.md. References in older audit records to that completed plan refer to its historical version, not this sprint.

This is one end-to-end sprint. Intermediate stages need not compile or leave a coherent repository. Do not create compatibility aliases, temporary public adapters, or intermediate releases to make individual stages green. Run focused checks when buildable and record deferred checks until they can actually run. Commit focused nonempty units under the agent policy, recording actual completion and validation below. A stage is not complete merely because its commands passed against the unchanged baseline.

Read AGENTS.md, the development guide, and applicable architecture, testing, and documentation policies. Use graph-first discovery and check current references before editing. Preserve unrelated work.

The sprint changes Go names, removes selected public state, adds two passive queries, and changes exactly the two selected public strings. It does not change loading/composition, YAML syntax, version policy, callback auditing, history ownership, sensitivity, source acquisition, or secrets-bootstrap semantics. Do not add performance promises, restrictions, dependencies, release gates, or tests for incidental mechanics. Public path comparison (A2) remains deferred. Do not select a release version, write version-specific release notes, publish, or update external consumers in this sprint.

Coverage Map

Accepted item	Implementation owner
D1: EntryInfo; D2: Sensitive	Stage 1
D3: version policy/evidence vocabulary	Stage 2
D4: empty-selection records; D5: destination fallback name	Stage 3
N6: descriptive Go import defaults	Stage 4
N1: SearchPaths; D7: LoadSecretsIntoEnvironment	Stage 5
D6/N4: recording methods and invalid-kind diagnostic removal	Stage 6
N2: main-tier provenance name and string	Stage 7
N3: history-limit name/string; D8: unproduced diagnostic removal	Stage 8
A1/N5: successful and failed source lookup	Stage 9
Current guidance, migration information, integrated completion	Stages 10–11

Private Implementation Choices

• Keep the public facade root-owned, with explicit forwarding and conversion. For facade-only renames, retain existing engine names unless steps below require a change. Root Sensitive can forward to engine Secret, and root EntryInfo can call engine InputInfo. Engine source-version records, import-default fields, discovery/bootstrap functions, and destination-fallback helpers need no cosmetic renames.
• Remove empty-selection outcome state through its small private transport path. Keep candidate selection and admission decisions distinct from empty-selection evidence.
• Replace root and engine kind-selecting Resolution.Record entry points with fixed-intent methods. Keep common addRecord and assignment helpers; remove the unreachable invalid-kind category after tracing remaining routes.
• Use one engine-local slice lookup helper for new result/error queries. Scan the retained inventory and clone only a matching source with existing cloneSourceInfo; do not clone Sources() in full to search it. Keep the callback's existing indexed entry-state lookup unchanged. This is a private implementation choice, not a public complexity guarantee.
• Rename main-layer and history-limit constants in both root and engine, so producers, conversion, and rendering use the selected strings directly.

Validation Conventions

Commands run from the repository root using an installed compatible Go toolchain and already acquired dependencies. For Go checks use:

export GOTOOLCHAIN=local
export GOPROXY=off
export GOTELEMETRY=off

Keep disposable inventories outside the repository. Reuse behavioral tests for mechanical renames. Do not add negative-compilation fixtures or a permanent exported-symbol inventory test simply to prove old names disappeared; Stage 11 performs a one-time review. Focused commands name existing test families and the proposed new lookup/recording cases. Include differently named new tests explicitly. A no-test match is not validation.

Update affected declaration comments and maintained Go callers, including examples, with each change. Stage 10 owns the integrated prose/migration sweep; it is not a reason to leave obvious neighboring comments wrong.

Stage 1 — Clarify Resolution Entry Metadata And Sensitivity

Status: complete. Implements D1 and D2.

1. In lifecycle.go, rename root Resolution.InputInfo to EntryInfo and Definition.Secret to Sensitive, without old aliases. Update toEngineDefinition to adapt the new predicate into engine Secret. Both loading entry points already share this conversion; retain that route.
2. Keep EntryInfo's existing (FieldInfo, bool) forwarding and silent unknown/invalid/shape-mismatch behavior. Its fixed entry view includes retained typed defaults and CLI assignments. Do not substitute final metadata or change IsExplicit's separate schema-path validation behavior.
3. Migrate root tests and maintained consumer calls. Update inactive-context cases in facade_context_test.go. Leave engine InputInfo, private sensitivity names, the configkit:"secret" tag, and typed Secret references unchanged. Do not broadly replace every identifier Secret.
4. Reuse facade_resolution_presence_test.go, facade_candidate_test.go, facade_context_test.go, and existing secret/provenance owners. Preserve ordinary/container classification and its union with tags/dependencies.

Focused validation:

go test . -run 'Resolution|InactiveContexts|FacadeLoad|Secret|Inspection|Diff' -count=1
go test ./internal/engine -run 'Resolution|Provenance|Secret|Sensitivity' -count=1

Completion: selected root names are in use; entry queries, lifecycle guards, and classification behavior are unchanged.

Stage 2 — Rename Version Policy And Evidence

Status: complete. Implements D3.

1. Replace root SourceVersionModeDefault with SourceVersionModeMainOnly in the same zero slot in lifecycle.go. Preserve numeric values and behavior of IfPresent/Required and the numeric engine bridge used by ordinary/layered loads.
2. In records.go, rename SourceVersionCheck to SourceVersionEvidence and SourceInfo.VersionCheck to VersionEvidence. Update conversion.go; name its root-returning helper fromEngineSourceVersionEvidence, still accepting engine SourceVersionCheck. Preserve fields, nil handling, detached paths/locations, and sanitized text.
3. Migrate Go callers/tests/examples and version comments together. MainOnly checks the effective path only with a configured rule; broader modes use source-local paths and require a usable rule. Evidence means observation, not acceptance or checker invocation. Add neither acceptance flags nor rules.
4. Reuse records_test.go and facade_source_version_mode_test.go: zero-mode compatibility, missing-main checker invocation, absent non-main skipping, NoSource, explicit null, consumption, mounted/item coordinates, redaction, failure evidence, and detachment. Rename test names directly naming the replaced declaration; do not duplicate their case matrix.

Focused validation:

go test . -run 'SourceVersion|VersionCheckConversion|VersionEvidenceConversion|Facade.*Version' -count=1
go test ./internal/engine -run 'SourceVersion' -count=1

Completion: all three replacements cross every source-record exposure correctly, with existing version behavior intact.

Stage 3 — Simplify Empty-Selection And Destination Evidence

Status: complete. Implements D4 and D5.

1. Replace root ImportSelection with EmptyImportSelection and SourceInfo.ImportSelections with EmptyImportSelections. Retain Path, Location, and Selector; remove Outcome, ImportSelectionOutcome, and ImportSelectionNone. Keep ImportSelector and its three constants.
2. Update root conversion/tests. In engine records.go, remove Outcome from ImportSelectionForFacade, the private evidence outcome field, and importSelectionOutcomeString. Remove its initializer in import_acquisition.go and the now-unused selection-outcome enum in import_selection.go after checking references. Its unused importSelectionHasCandidates member does not justify creating evidence. Preserve actual admission outcomes such as importAdmissionOptionalMissing, candidate handling, and accounting. Existing private transport/helper names may stay unchanged.
3. Rename root SourceInfo.DestinationUnresolved to DestinationIsFallback. Conversion still reads existing private fallback evidence. Keep destination coordinates and placement modes; do not derive exact item coordinates where existing evidence does not contain them.
4. Migrate root/internal assertions and maintained Go callers. Preserve empty selection scope/order, declaration-local paths, and copies on results, callbacks, and errors. Remove assertions of the retired outcome only, keeping real selection assertions. Reuse nested-item fallback/partial-failure cases; do not invent new placement states.

Focused validation:

go test . -run 'ImportSelection|EmptyImport|Destination|SourceInfo|Fallback|RelativeImport' -count=1
go test ./internal/engine -run 'ImportSelection|ImportEvidence|Import|Destination|Item' -count=1

Completion: source records use the reduced record/new fields; selection, placement, partial evidence, and detachment retain established behavior.

Stage 4 — Make Go Import Defaults Self-Describing

Status: complete. Implements N6.

1. Replace root ImportDefaults.Follow with FollowSymlinks and Hidden with IncludeHidden, preserving *bool and the other three fields. Update comments and toEngineImportDefaults to map to existing engine Follow/Hidden.
2. Migrate root option literals, examples, and tests by type/context. Do not rename YAML follow/hidden, declaration parser fields, or unrelated hidden-file controls. Do not snapshot earlier in the facade: existing engine preflight owns the once-per-load snapshot before acquisition/callbacks.
3. Reuse facade_import_defaults_test.go for nil, explicit false/true, transitive/underlay application, declaration overrides, pattern selection, and callback-mutation isolation. Preserve traversal/search scope and nil versus non-nil empty Pattern.

Focused validation:

go test . -run 'ImportDefaults|ImportSelection|EmptyImport' -count=1
go test ./internal/engine -run 'ImportDefaults|Import.*(Hidden|Follow|Symlink|Pattern)' -count=1

Completion: descriptive Go fields feed unchanged YAML policy and snapshots in ordinary/layered loading.

Stage 5 — Clarify Discovery And Environment Bootstrap Names

Status: complete. Implements N1 and D7.

1. In root discovery.go, rename Paths to SearchPaths. Forward the same literal candidates to existing engine machinery. Migrate callers without touching semantic Path, Keys, or Searches. Retain order, deferred acquisition, and selected-file terminal failures.
2. In root secrets.go, rename LoadSecretsDir to LoadSecretsIntoEnvironment, preserving arguments, return values, error conversion, and engine bootstrap call. Comments should plainly describe explicit process-environment mutation, already-set/empty preservation, reports, and existing partial-application behavior.
3. Migrate maintained calls, affected executable example identifiers, and nearby documentation. Do not couple bootstrap to Load, rename report/error types, or add filesystem/transactional guarantees. Process-mutating tests remain serial and restore state. Private engine function names may remain unchanged.

Focused validation:

go test . -run 'Search|Discovery|SecretsBootstrap|LoadSecrets|Example_weatherStartup' -count=1
go test ./internal/engine -run 'Search|Discovery|LoadSecretsDir' -count=1
go test ./examples/consumers -run 'Discovery|WorkingDirectoryPreset|SequenceCatalog|Weather' -count=1

Completion: old root functions are replaced outright; acquisition and safe bootstrap behavior remain unchanged.

Stage 6 — Replace Kind-Selecting Recording With Intent Methods

Status: complete. Implements D6 and N4.

1. Replace root and engine Resolution.Record with:

   RecordDerivedDefault(path Path, reason string, dependencies ...Path)
   RecordNormalization(path Path, reason string, dependencies ...Path)
   

   Root methods forward through existing live-context guards/path conversion. Engine methods call activeState().addRecord with their fixed ChangeDerivedDefault or ChangeNormalization and no helper expected value. Share existing validation/coalescing/auditing; neither method assigns values or tests absence.

2. Migrate manual callers according to their existing supported kind in root tests, engine tests, examples, and current snippets. Leave Default and Normalize as assignment-and-recording helpers using existing addRecord routes. Retain ChangeKind and all inspection constants.

3. Check every remaining addRecord caller supplies a supported fixed kind. Then remove the invalid-kind guard and CodeResolveInvalidKind from both root/engine. Remove only the obsolete arbitrary-kind case in resolution_test.go; retain invalid path, target, reason, dependency, projected-value, and unrecorded-change cases. If a legitimate supported public trigger remains, report it before changing its behavior.

4. Update nil/zero/closed-context coverage for both methods. Add/adapt a small root case, TestFacadeResolutionRecordingIntents, proving each method's resulting origin/history kind after application assignment. Include explicit input changed and recorded as a derived default, so this cannot accidentally become the absence-checking helper. Reuse container coverage, overlapping records, repeated/equal intent, dependencies, and net-change tests elsewhere.

Focused validation:

go test . -run 'Resolution|InactiveContexts|FacadeLoad|Historical|History' -count=1
go test ./internal/engine -run 'Resolution|Provenance|Callback|History' -count=1

Completion: root exposes only the selected manual recording methods; supported input cannot produce resolve.invalid_kind. Other auditing/lifetime behavior is unchanged.

Stage 7 — Use Main Consistently In Provenance

Status: complete. Implements N2.

1. Replace LayerBase = "base" with LayerMain = "main" in root/engine records. Migrate producer references across acquisition, composition, decoding, inheritance origins, overrides, discovery, and history/rendering. Keep existing tier assignments.
2. Review literal "base" provenance comparisons/rendered expectations, not just Go references. Update only the main-tier category. Filesystem base directories, domain values, fixture labels, and unrelated words are unchanged. Retain other layer spellings and source IDs/order.
3. Update existing assertions in root, engine, and examples. Add a focused root records assertion for string(LayerMain) == "main"; reuse a public provenance/rendering fixture to verify main/imported-main attribution reaches consumers with that label. Imports retain their owning tier; other categories, precedence, and acquisition order are unchanged.

Focused validation:

go test . -run 'Facade|Historical|History|Layer|PublicValues' -count=1
go test ./internal/engine -run 'Compose|Overlay|Decode|Provenance|History|Explain|Grouped|Discovery' -count=1

Completion: main-tier evidence/rendering consistently uses the selected constant and spelling without attribution/precedence changes.

Stage 8 — Rename The History Diagnostic And Remove The Unused Code

Status: complete. Implements N3 and D8.

1. Replace root/engine CodeExplainLimitExceeded with CodeHistoryLimitExceeded = "history.limit_exceeded". Update every producer, including executeLoadRequest in engine load.go and inheritanceHistoryBudgetDiagnostic in inheritance_walk.go, plus readers/literal assertions. Other explanation errors are unchanged.
2. Reuse history_budget_test.go and inheritance-budget cases to preserve exact retained-node accounting and failure before effective secret lookup/application validation. Change expected code, not quota, fixtures, accounting, or timing. Add the new literal to focused root constant-value assertions, not a general API snapshot test.
3. Reconfirm CodeOptionsUnavailable/options.unavailable has no producer or documented supported/reserved role. Baseline references are the two declarations and planning/review evidence. Remove both declarations. If a concrete supported role is discovered, bring it back for discussion rather than remove behavior or create a trigger. Other diagnostic codes stay intact.

Focused validation:

go test ./internal/engine -run 'HistoryBudget|RetainedNode|InheritanceExpansionEnforcesRetainedHistoryBudget' -count=1
go test . -run 'PublicValues|LoadReturnsFacadeLoadError|Diagnostics|Historical' -count=1

Completion: history failures carry the new code under unchanged conditions; D8's role check is resolved and its unproduced declaration is removed.

Stage 9 — Add Detached Source Lookup On Results And Load Errors

Status: pending. Implements A1 and N5.

1. Add engine-local lookupSourceInfo(sources []SourceInfo, id string) (SourceInfo, bool) beside source cloning helpers. Return zero/false for empty/unknown IDs; otherwise scan the retained slice and clone the match. Add nil-safe engine Inspection.Source and LoadError.Source using it. Do not change inventories, query-time evidence, or the callback's source map.

2. Add the selected root methods in lifecycle.go and diagnostic.go:

   func (r *Result[T]) Source(id string) (SourceInfo, bool)
   func (e *LoadError) Source(id string) (SourceInfo, bool)
   

   Return zero/false for nil or zero receivers/absent delegated state. Convert a found engine record through the Stage 2–3 fromEngineSourceInfo path. Keep enumeration, passive SourceContext, and callback Source intact.

3. Document passive lookup, load-local IDs, detachment, and partial failed-load evidence. Read LoadError's already-sanitized retained sources, not raw acquisition/version state. Do not complete placement, reinterpret absence as acceptance, open resources, or perform credential lookup.

4. Add focused external-package tests TestResultSourceLookup and TestLoadErrorSourceLookup, or extend equivalent facade owners. Compare found records to enumeration; check empty/unknown IDs and nil/zero receivers. Mutate nested returned evidence and verify repeated lookup/enumeration are unchanged. Reuse import/version fixtures for ImportedFrom, VersionEvidence.Location, and empty-selection slices. Cover representative layered sources and failure evidence with redaction/partial placement. Reuse counting-resource fixtures where useful to demonstrate no query I/O; do not add a full load-mode matrix, timing assertions, or concurrency promises.

Focused validation:

go test . -run 'SourceLookup|SourceVersionEvidence|ImportSelection|EmptyImport|SourceContext|Fallback' -count=1
go test ./internal/engine -run 'SourceVersion|ImportSelection|Resource|ResolutionInput' -count=1

Completion: both lookups agree with enumeration, return detached records, and preserve success/failure evidence without new load activity.

Stage 10 — Complete Guidance And Consumer Migration Information

Status: pending. Covers all accepted changes' documentation and maintained usage.

1. Review declaration comments and config.md's owning sections: discovery, import defaults/evidence, source versions, layered metadata, sensitivity, bootstrap, resolution, and history limits. Update relevant names and strings while preserving semantics. Keep detailed Go contracts beside declarations rather than reproducing them throughout prose.
2. Migrate README, consumer guidance, root executable examples, and maintained examples/ code/fixtures. Use new source lookup where an existing example enumerates solely to find an ID; retain enumeration where that is the task. Use new intent methods for manual recording while retaining assignment helpers where they do the work.
3. Add a concise Pre-v1 public API cleanup subsection to the consumer guide's existing migration section. Include every old-to-new name from the roadmap, removed empty-selection outcome field/type/constant and diagnostic constants, both Record call rewrites, new lookups, and "base" → "main" / "explain.limit_exceeded" → "history.limit_exceeded". Explain unchanged YAML follow/hidden and configkit:"secret"; literal comparisons and serialized expectations may need migration as well as Go code. Do not claim external migrations or select a version for this breaking pre-v1 work.
4. Update internal docs only for changed descriptions. Do not rewrite accepted ADR decisions, pinned reviews, or historical release notes. If retained completed reports link to this overwritten plan, make their old-plan reference revision-specific rather than implying it concerns this sprint. Preserve unrelated proposals/status.

Validation:

go test ./examples/consumers -count=1
go test . -run '^Example|TestLocalDocumentationLinks' -count=1
git diff --check

Completion: current guidance/examples use the selected API, migration information is complete, and historical evidence remains accurate.

Stage 11 — Verify The Integrated Surface And Finish The Sprint

Status: pending.

1. Compare actual root exports/comments against the roadmap's complete accepted table using go doc -all . and, if needed, a disposable AST/type inventory outside the checkout. Check fields/method sets/constants as well as top-level names. Confirm both recording and lookup signatures and no removed root names/compatibility aliases.

2. Search active Go callers/current documentation for old names and the two old strings. Review matches by context: retained private engine names, domain text, migration instructions, pinned reviews, and historical records are valid. Do not globally rewrite base, Secret, Paths, Follow, Hidden, or Record, or enforce a blanket zero-match rule. Recheck D6/N4 and D8 reference evidence against the final tree.

3. Run deferred focused tests and existing boundary/link checks:

   go test . -run 'TestPublicFacadeBoundary|TestDependencyImportBoundary|TestLocalDocumentationLinks' -count=1
   ./scripts/check.sh
   

   The everyday check already runs the full default suite, vet, and build, including examples. No additional full-suite pass or benchmarks are needed solely for this plan. If implementation changed mutable ownership/concurrency beyond forwarding/existing clone paths, follow the development guide with go test -race ./...; record the reason/result. This introduces no permanent release gate and requires no CI waiting.

4. Inspect the final diff for unselected semantics, ownership changes, dependencies, artifacts, and strengthened guarantees. Record actual stage completion/validation here. Mark the API roadmap implemented only after its selected surface and behavior preservation are verified. Keep completed planning/review records until normal release preparation. Commit completed nonempty units with plain-English messages; report remaining work accurately.

Completion: all accepted items are implemented, diagnostic role/reachability checks are resolved, applicable checks pass, and current docs/examples agree. No unselected public or semantic changes remain. This is implementation completion, not release publication or external consumer verification.

Decision Status

No open design question is required to execute this plan. D8's role check and D6/N4's reachability check are verification tasks with an explicit discussion path if unexpected supported use is found. Private helper naming may be adjusted for cohesion without changing the selected surface or behavior.