base-layer/configkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 3 Wiki Activity

Plan main-relative companion inputs

Browse Source
This commit is contained in:
Eric Rakestraw
2026-11-09 18:39:25 +00:00
parent c51bea9b4f
commit ae0c94d3c6
3 changed files with 322 additions and 70 deletions
Download Patch File Download Diff File Expand all files Collapse all files

81
docs/roadmap/audita-request.md Normal file
View File

@@ -0,0 +1,81 @@
# Audita request: main-relative companion resources
Consumer requirements against ConfigKit v1.1.0. These complement the proposed
[main-relative underlays and overlays](main-relative-inputs.md); that roadmap
owns upstream API design and implementation planning.
## What Audita is doing
Audita is a transcript-processing CLI adopting ConfigKit, PromptKit, and
CommandKit's native loading and inspection workflows. ConfigKit should own
discovery, imports, composition, overlays, and configuration resolution. Audita
should receive the resolved configuration and then apply transcript-specific
business logic.
The main application configuration is either an explicit file or the result of
native `SearchConfig` discovery. Companion resources should follow the directory
containing the selected main file, regardless of the process working directory:
| Resource | Intended location and use |
| --- | --- |
| Glossaries | Optional `glossary.yml` and recursive `glossary/` imports, mounted at `glossary_collections` in an application underlay. |
| PromptKit definitions | Optional `prompts.yml` and recursive `prompts/` imports, in an operator tier above embedded Audita prompts and PromptKit's bundled catalog. |
| Selected named overlay | Beside the main file, using native `overlays/<name>.yml` lookup. |
Both recursive imports select `*.yml` and `*.yaml`. Audita supplies their import
declarations, so operators need not repeat discovery boilerplate in every main
file or create a manifest just to use a resource directory. Each pair composes
as peers within its tier; native precedence and diagnostics remain authoritative.
Embedded assets retain their separate filesystem contexts.
For example, selecting `/srv/campaign/audita.yml` while running from `/tmp`
should use `/srv/campaign/glossary/`, `/srv/campaign/prompts/`, and
`/srv/campaign/overlays/<name>.yml`.
An aggregate `overlays.yml` was also suggested for symmetry. It is not supported
by today's named-overlay contract and is a separate possible enhancement, not
necessary to solve the directory-binding requirement described here.
## Missing API support
In the current [layered API](../config.md#layered-loading), `SearchConfig`
selects the main during loading, while underlay filesystem contexts and
`OverlayFilesystem` must be supplied beforehand. Audita cannot bind these to
the selected directory without duplicating discovery or performing a preliminary
load. Imports in the main file would be peers, not the required lower-priority
underlay.
1. **Bind an underlay import context to the selected main directory.** Audita
needs this for an application-supplied mapping or bytes containing native
import declarations, including optional file and recursive-directory imports
and their destinations. A constructor for a required main-relative file alone
would not cover this workflow. Resolve the relationship within one layered
load, then let existing import machinery acquire and compose the resources.
2. **Derive the selected overlay context during that same load.** Support an
explicit request for main-relative lookup with either an explicit or discovered
main file. Preserve native overlay naming, precedence, and errors. Audita's
intended lookup base is the main file's directory; distinguish that choice
from a potentially broader filesystem containment root.
3. **Make absent-main behavior explicit.** Audita permits optional discovery to
find no main file and still run with sufficient defaults and CLI settings.
The API should allow an explicit policy for dependent resources in that case,
without manufacturing a main source or guessing from unsuccessful candidates.
A caller-supplied fallback directory is one useful option; whether Audita
chooses the working directory or skips automatic companion imports remains
an application decision. Failures reading a selected main must remain terminal.
PromptKit's separate engine can be constructed after application loading and use
the selected main's recorded source context. Existing context inspection may
already suffice there; this request does not require combining the two schemas
or adding a second discovery API.
## Completion from Audita's perspective
One native application load can discover the main, bind its companion imports
and selected overlay, and return normal source evidence and errors. Changing
only the invocation directory does not relocate companions of the same selected
main. Processing and inspection share this behavior, with no Audita directory
walker, configuration parser, discovery pass, or preliminary full load.
The API shape is ConfigKit's choice. The important addition is native context
binding for both declared underlay imports and selected overlays.

239
docs/roadmap/implementation.md Normal file
View File

@@ -0,0 +1,239 @@
# Implement main-relative companion inputs
The [feature roadmap](main-relative-inputs.md) owns the target behavior and the
four settled decisions. The [Audita request](audita-request.md) supplies the
consumer workflow. Implement in order as one sprint. Intermediate stages may
leave code or tests incomplete; run focused checks when their dependencies are
in place, and require a coherent, validated tree at the end. Stage boundaries
are handoffs for Luna, not required commit boundaries.
Start with the [development guide](../development.md) and its applicable
agent, architecture, testing, and documentation policies. Keep the public
facade root-owned. Keep ordinary loading, file source semantics, and existing
explicit layered overlays compatible. No dependency change, new YAML syntax,
second discovery pass, application callback for context selection, or new
release gate is part of this work.
## Stage 1 — Define the Go API and preflight rules
In root and engine `layered.go`, add matching descriptions and carry them into
`loadInput` and `loadRequest` in the engine. Use these public names:
```go
type MainRelativeContext struct {
Fallback *FilesystemContext
RequireBase bool
}
func MainRelative() *MainRelativeContext
```
`MainRelative()` returns a fresh pointer to a zero-valued description. Add
`Input.MainRelative *MainRelativeContext` and
`LayeredOptions[T].OverlayFallback *FilesystemContext`. The latter only needs
a concrete context: a selected overlay already requires a base. Use keyed
literals in new examples. Copy the description and its concrete fallback into
the private request using the existing filesystem-context conversion in
`lifecycle.go`; preserve the caller's native namespace and reader ownership
rules. Store resolved state only in the current load.
In layered preflight, apply the concrete-context override before checking
binding scope. Reject an active main binding and active bindings on `File`,
`Filesystem.File`, and `SearchConfig` underlays with `CodeOptionsInvalid`.
Keep ordinary source validation, including zero-source rejection. A valid
`NoSource` underlay remains absent; binding does not make it contribute content
or evidence, and `RequireBase` still applies to an active binding. Suppress the
synthetic-source missing-context preflight error only for a valid active
underlay binding. Binding does not enable imports.
Check the static shape of a fallback that can participate, using the existing
non-file context rules (both base and root, host absolute paths or native
namespace paths). Defer existence, directory, access, and resolved containment
checks until selection. Ignore the shadowed binding entirely when
`Input.Filesystem` is supplied; likewise ignore the overlay fallback when the
overlay is inactive or an explicit `OverlayFilesystem` wins. Keep overlay
name and explicit-context preflight checks. Move the missing-context error for
active layered overlays to Stage 5, where selection can see the main. Do not
change preflight for ordinary `Load`.
Cover active binding scope, explicit overrides (including invalid explicit
contexts), and fallback shape through public option tests. Reuse existing
option diagnostic codes; avoid promising order among unrelated errors.
## Stage 2 — Bind companion inputs with a usable context
In `assembleLayeredInput`, pass the main's `assembledInput.resolvedFilesystem`
to `acquireUnderlayGroupScopes`. Reuse the acquisition handoff in
`acquisition.go` and resolution helpers in `filesystem_context.go`; do not
select or open the main again. Verify the handoff covers a file or selected
search without imports, a native file, explicit non-file contexts, and
`NoSource` or exhausted optional search with explicit context. Preserve the
absent host root-only rule and the native context requirements. A main
acquisition/context error is terminal, never an absent-base signal.
For each underlay, use its explicit filesystem first; otherwise, for active
bindings, use the resolved main context, then its fallback. For main-derived
contexts copy the resolved base and root and keep the native namespace object
intact. Clear `mainPath` in the copy: the companion is still a synthetic
source, not a second copy of the main. Use the receiving tier's existing host
identity token and visited state; the main's host location and scope carry
over, not its tier bookkeeping. A selected fallback uses the same
resolution/validation rules as a concrete non-file context; pass it through
the normal acquisition path where possible. Validate selected contexts even
when imports are inactive as required by the existing context contract;
do not probe the fallback when main derivation wins.
Carry a pre-resolved derived context through the private acquisition request
to `acquireImportedInput` and the standalone/context completion path. Use
existing import access, traversal and closure; keep root/base checks, parse
and version processing, budget accounting, and source evidence. Bound
synthetic sources can expose the same passive context as explicitly located
synthetic sources but keep their original kind, label and empty physical path.
The imported files use their own paths and directory bases.
Make the existing native namespace anchor selection in
`acquireUnderlayGroupScopes` use the **selected** namespace, including
derived or fallback contexts. Determine anchors from selected descriptors
without opening directories or validating later inputs out of turn. Keep the
first input position anchor for each namespace and the existing host and
synthetic anchoring and tier boundaries. Preserve independent sources and
repetition semantics, including two different native namespaces with the
same label and paths.
Cover successful bindings for host and native contexts, an unrelated working
directory, canonical host symlink target directory, a wider root that permits
contained parent-relative imports, and reuse of one description across loads.
Test selected and unused fallbacks. Defer the no-base path to Stages 3–4.
## Stage 3 — Share import declaration preparation
In `import_acquisition.go`, extract the context-independent first pass of
`walkDeclarations` so both normal traversal and the contextless path can
use it. Prepare from the already parsed declaration specs, apply
`resolveImportPolicy` with the load-wide defaults, and retain existing
placement checks and diagnostic sites. For an input placed as a sequence
item, prepare against its local mapping scope and keep the existing diagnostic
path projection. Use the already processed root directives from
`processAcquiredInputVersion`; do not reparse YAML, repeat version checks,
or infer a filesystem path from a source label.
Keep selection and admission in the existing normal walk. This stage changes
no bound import behavior. Reuse import specification, defaults, placement,
selection, and diagnostic tests for verification.
## Stage 4 — Finish companion handling when no base exists
Handle the no-base outcome only for an active underlay binding with no
override, usable main context, or fallback. `RequireBase` fails with
`CodeOptionsInvalid` regardless of import enablement or declaration count.
Otherwise retain normal source parsing, version handling, disabled-import
validation, and contribution assembly. When imports are enabled, use Stage 3
to validate their declarations even if none will be expanded.
For required declarations use `CodeImportMissing` at the first named target
site for `path`, or the selector field site for `dir` and `glob`. Say that no
base is available; do not report a file search that did not happen. A required named-path declaration with nonempty
targets fails, as do required directory or glob selections. Empty
`path: []` has no targets even if required. Explicit per-declaration
requiredness continues to override the application default. Skip optional
expansion without calling a filesystem selector.
Assemble the declaring payload once, with its original source record,
placement, sequence declarations, and version evidence. Share the existing
root contribution assembly and accounting where needed for imports-enabled
payloads, including the pending local scope for sequence-item placement.
Do not turn imports off to bypass validation or introduce a dummy access
object, dummy filesystem, acquired child, resource context, or
`EmptyImportSelections` entry. Charge the declaring payload as usual; charge
no traversal or child work for skipped imports. Normal searches with a
selected context still produce their normal empty-selection evidence.
Test absent optional discovery and explicit absence, mixed payload with
optional and required declarations, application defaults and explicit
overrides, empty paths under both requiredness values, require-base with and
without enabled imports, malformed declarations and invalid placement,
source version policy, mounted payloads, and absence of synthetic search
evidence. Required discovery and failed selected main processing must not
activate this path.
## Stage 5 — Derive layered overlay context and lookup base
In the existing overlay branch of `assembleLayeredInput`, select in order:
explicit `OverlayFilesystem`, usable main context, `OverlayFallback`.
Skip selection entirely for an inactive name. For an active name without a
base, return `CodeOptionsInvalid`; if context validation fails, propagate
its normal diagnostics without trying another context. Keep overlay selection
after source assembly and main version handling, and preserve existing
failures for missing files, invalid overlays, and repeated sources.
In `selected_overlay.go`, pass a lookup directory separately to the existing
selected overlay loader (for example on `selectedOverlayRequest`). Ordinary
`Load` and explicit layered contexts use the resolved root as before.
Derived and fallback contexts use the resolved base. Open access at the
containment root; join the lookup directory with `overlays/<name>.yml`
using the selected namespace's path operations, then use normal
canonicalization and containment. Keep the namespace, source identities,
version processing, limits, source contexts, and handle ownership on the
existing path. Do not use the lookup directory as a replacement root or
construct a new namespace.
Cover canonical host symlink targets, wider roots with distinct overlay
files at base and root, native filesystems, context on a non-file or absent
main, inactive overlay with an invalid fallback, explicit override with no
fallback on error, base-relative fallback, and missing selected overlays
without recovery. Retain ordinary and explicit root-relative regression
coverage, including existing namespace and repeated-source cases.
## Stage 6 — Exercise the combined workflow
Add a maintained executable example in `examples/consumers` for a single
layered load using `SearchConfig`, a mapping companion with optional named
files and recursive directory imports, an independently located embedded
tier, and one selected overlay. Use actual fixture files and the existing
`*.yml` and `*.yaml` pattern list; show destination placement, transitive
imports, and strict peers. Demonstrate precedence from embedded catalog
through operator underlay, main, and overlay. Include a registered identity
case for whole-record upsert across tiers and a same-tier duplicate failure.
No change to Audita or PromptKit is needed.
## Stage 7 — Close remaining integration and evidence gaps
Fill remaining public integration gaps without repeating every engine test:
first and later search candidates, explicit file and mapping/bytes/reader
sources, only the file, only the directory, both, neither and an empty
directory, invalid or inaccessible selected fragments (use a deterministic
filesystem error where permissions are unreliable), and a selected main
failure that cannot resume search or use the fallback. Use focused fixtures
to check layer/index, physical paths, passive resource contexts, normal
source-version evidence, shared acquisition budgets and cleanup on failure.
Check that callbacks execute only in the existing lifecycle, and that a
reused description binds anew. Preserve existing ownership and detached
inspection checks. Do not require new global diagnostic ordering or test
incidental call counts.
## Stage 8 — Document and verify the completed sprint
Update the public declaration comments and the discovery, layered loading,
imports, resource context and selected overlay sections of
[the configuration contract](../config.md). Document lookup versus
containment, canonical host symlink behavior, override and fallback
precedence, and absent-base behavior. Link the maintained examples from
consumer guidance. Update the [internal overview](../internal/overview.md)
and [import internals](../internal/imports.md) for the actual handoff,
contextless path, and ownership.
Record the deferred context decision in the next numbered ADR, extending
[ADR 0019](../adr/0019-explicit-configuration-discovery.md) without rewriting
it. Clarify in the architecture policy that the layered loader binds
application-selected dependent locations after main selection; applications
still own search conventions, and an explicit overlay selection is required.
Preserve existing scope boundaries. Use the documentation policy's canonical
owners rather than restating the full contract in every document.
Run the focused engine, facade, and consumer tests added or affected above,
then the everyday `./scripts/check.sh`. Run the race suite for the
load-local ownership and description reuse changes. Check changed Markdown
links, public facade boundaries and examples. Use the declared Go toolchain
and existing dependencies. No release or benchmark is part of completion.
Report validation and any remaining limitations; commit the completed sprint
under the agent policy.

72
docs/roadmap/main-relative-inputs.md
View File

@@ -1,7 +1,7 @@
# Main-relative companion imports and overlays
Status: all four design decisions settled; not implemented. Final API spelling
and implementation planning remain. Baseline: `191015e`.
Status: design settled; not implemented. The [implementation plan](implementation.md)
owns the concrete API spelling, execution order, and verification. Baseline: `191015e`.
## Purpose
@@ -205,74 +205,6 @@ architectural exclusions. The design follows
follow-up for deferred context ownership without rewriting accepted decisions.
No release version or additional release gate is selected here.
## Work from the baseline
1. **Finalize API spelling and examples.** Translate the settled decisions below
into concrete declarations and prepare Go examples for Audita's mapping/bytes
workflow, explicit overrides, non-file main contexts, and independently located
embedded inputs.
2. **Carry binding intent through the public facade and load request.** Update
the relevant input/options declarations and conversions in `layered.go` and
`lifecycle.go`, retaining root-owned public types and normal ownership rules.
Distinguish a deferred relationship from a concrete filesystem context.
3. **Bind underlay import contexts after main context resolution.** Extend
preparation and acquisition in `internal/engine/load_request.go`, `load_input.go`, and
`acquisition.go` as needed. Reuse the main's resolved base, root, and namespace,
apply the settled missing-base policy, and delegate selection to existing import
machinery. Preserve independently supplied contexts.
4. **Derive overlay context and separate lookup from containment.** Remove the
unconditional explicit-context requirement for the new layered default.
Preserve explicit and ordinary lookup while using the main context's resolved
base inside a potentially wider root. Reuse selected-overlay loading and its
failure and resource-ownership behavior.
5. **Verify the complete workflow and document it.** Add maintained consumer
examples, public integration coverage, and focused engine tests. Update the
discovery, layered-loading, import-context, and selected-overlay contracts,
declaration comments, and internal ownership documentation when implemented.
This is dependency order for a later implementation plan, not a required commit
layout or a requirement to expose intermediate stages to consumers.
## Validation and completion
Validate behavior through the public loading boundary, including:
- Explicit and discovered main files, early and later search candidates, and an
unrelated invocation directory. Companions follow the same selected main.
- Native file mains, bytes/readers/mappings with explicit host or native contexts,
and absent mains with explicit contexts. Cover absent host root-only contexts,
missing and invalid contexts, and context reuse without enabling main imports.
- Mapping, bytes, and reader underlays containing optional named-file and recursive
directory imports with `*.yml`/`*.yaml` filtering and mounted destinations.
- Receiving-input scope: active bindings on main inputs and file-backed underlays
fail with option errors; concrete context overrides retain their precedence.
- File only, directory only, both, neither, empty directories, and invalid or
unreadable selected fragments. Both forms compose as peers when present.
- Operator prompt overrides above embedded catalogs using registered identity;
same-tier duplicate IDs remain conflicts. Main and overlay precedence survive.
- Mixed embedded and companion inputs, multiple underlay tiers, transitive
imports, placement, source-version checks, and load-wide import defaults.
- Automatic derived overlays and explicit overrides, active and inactive names,
broader containment roots, and missing selected overlays.
- Optional discovery exhaustion, required discovery, fallback and skip policies,
mixed payload/required-import cases from OD-2, and terminal selected-main errors.
- Unused fallback paths need not exist or be accessible; selected fallback paths
receive normal validation. Inactive overlays ignore fallback settings.
- Empty `path: []` with either requiredness value when no base exists, including
the independent effect of `RequireBase`, and skipped imports without fabricated
completed-search evidence.
- Symlinks, eligible namespaces, parent-relative paths within the selected scope,
context validity, and namespace identity under OD-3 and OD-4.
- Normal source evidence, diagnostics, passive resource contexts, cleanup, and
budget accounting; no duplicate lifecycle callbacks or preliminary full load.
- Existing ordinary and explicit layered workflows and description reuse.
Run focused checks during implementation and applicable normal repository checks
before completion. No benchmark campaign is required. Completion means the
application can discover its main, bind companion imports and the selected
overlay, and receive normal resolved configuration and inspection in one load,
without an application directory walker, parser, or extra discovery pass.
## Settled Decisions
### OD-1: Public representation of deferred context binding — settled