configkit/docs/consumers/README.md

# Consumer Adoption Guide

ConfigKit loads configuration into your application's Go types. You provide
those types, defaults, and application rules. ConfigKit reads and combines the
selected inputs, checks their types, and returns values with source information.
Your application still owns CLI parsing, resource reads, remote acquisition, and
runtime initialization.

Import `"go.maximumdirect.net/configkit"`; the package name is `configkit`.
Start with one file, then choose the tasks your application needs:

| Task | Start here |
| --- | --- |
| Load one file | [Happy path](#happy-path) and the [README example](../../README.md#load-a-configuration-file) |
| Supply fixed defaults and check application rules | [Defaults and application rules](#defaults-and-application-rules) |
| Add bundled defaults beneath application settings | [Native filesystem and layered input](#native-filesystem-and-layered-input) |
| Select files or an overlay | [Discovery](#explicit-main-file-discovery), [import defaults](#import-defaults-and-selection-inspection), and the [overlay reference](../config.md#selected-overlays) |
| Reuse settings or combine record lists | [Inheritance](#named-sibling-inheritance) and [identified sequences](#identified-prompt-sequences) |
| Handle credentials | [Typed-secret reference](../config.md#typed-secret-references) and the [startup example](../../examples/consumers/consumers_test.go) |
| Inspect sources, earlier values, or changes | [Inspect a completed load](#inspect-a-completed-load) and [diffs](#effective-configuration-diffs) |
| Upgrade an existing consumer | [Migration guidance](#intentional-migration-changes) |

## Choosing a loading entry point

- Use [`Load`](../../lifecycle.go) with one source for an ordinary configuration.
  [`SearchConfig`](../../discovery.go) is also a source; pass it to `Load` or use
  it as the main input to [`LoadLayered`](../../layered.go).
- Use `LoadLayered` when several inputs should supply lower-priority defaults
  beneath the main source. Each underlay group combines its inputs strictly.
  Each [`Input`](../../layered.go) owns its source, filesystem, destination, and
  import choice. The overlay filesystem context is selected independently in
  [`LayeredOptions`](../../layered.go).
- Load-wide rules belong to [`Options`](../../lifecycle.go) or
  [`LayeredOptions`](../../layered.go): the application definition (including
  version policy), import defaults, and explicit overrides. These do not turn
  on imports or discovery by themselves. See the
  [configuration contract](../config.md#layered-loading) and
  [discovery contract](../config.md#configuration-discovery) for exact rules.

The root [README example](../../README.md) shows the small `Load` path.
[`layered.go`](../../examples/consumers/layered.go) demonstrates the bundled
file and layered cases. For import-focused adoption, see the
[prompt catalog](../../examples/consumers/prompt_catalog.go); for explicit
discovery composed with sequences, see the
[sequence catalog](../../examples/consumers/sequence_catalog.go).

## Happy path

Define the Go configuration type, choose one source, and call `Load`. Handle a
load error before starting the application with `result.Config`. The
[README example](../../README.md#load-a-configuration-file) shows this complete
minimal loading function. An explicitly missing file is an error; use `NoSource`
when you deliberately want defaults without a main document.

### Defaults and application rules

Using the README's `Config` type, an application can add reusable rules:

```go
definition := configkit.Definition[Config]{
    Defaults: func() Config { return Config{Enabled: true, Workers: 2} },
    Validate: func(value *Config, validation *configkit.Validation) {
        if value.Workers < 0 {
            validation.Error(configkit.Keys("workers"), "must be nonnegative")
        }
    },
}
```

Pass this definition in `Options[Config]{Definition: definition}` when calling
`Load`. Supplied `enabled: false` or `workers: 0` replaces those defaults.
Use `Resolve` when a value must be derived or normalized before validation, and
record that change with the provided helpers. The
[callback reference](../config.md#recorded-application-resolution) compares the
helpers; [executable examples](../../example_test.go) show full definitions.

### Inspect a completed load

Keep the result when you need to explain configuration. Use
`result.Metadata().Lookup(configkit.Keys("workers"))` to inspect whether input
supplied the value and which source or application step supplied it. Follow an
origin's source ID with `result.Source(id)` when you need its source record.

Use `Explain` for earlier values, `WriteExplanation` for a readable explanation,
and `WriteEffective` for safe YAML describing the final saved state. These
operations describe the completed load even if your application later changes
`result.Config`. The [inspection reference](../config.md#layered-metadata)
explains returned values, copies, and hidden information.

Add imports, a selected overlay, or explicitly set CLI values through options
only when needed. An overlay changes configuration; choosing an LLM, report, or
execution profile remains application logic.

## Explicit main-file discovery

Applications choose the search conventions and their precedence, then pass the
resulting `SearchConfig` source to the same `Load` or `LoadLayered` entry
point. For example, the maintained sequence catalog puts a caller-selected
file before a custom directory fallback while keeping its underlay and overlay
choices independent. See
[`sequence_catalog.go`](../../examples/consumers/sequence_catalog.go) and its
[behavior tests](../../examples/consumers/sequence_catalog_test.go), plus the
[working-directory executable
example](../../examples/consumers/discovery_test.go). Together, the [discovery
API](../../discovery.go) and [configuration discovery
contract](../config.md#configuration-discovery) explain the order of attempted
files, what happens when no file is found, and why a selected-file failure
stops the search. [ADR 0019](../adr/0019-explicit-configuration-discovery.md)
records why discovery is explicitly selected.

## Import defaults and selection inspection

`Options.ImportDefaults` and `LayeredOptions.ImportDefaults` apply one
load-wide policy to enabled main, underlay, and transitive imports. Each field
is resolved independently: a declaration's explicit value wins, then the
application default, then the library default. `Required`, `FollowSymlinks`,
`IncludeHidden`, and `Recursive` use nil pointers for unspecified values.
`Pattern` applies to
directory searches: nil leaves filenames unrestricted, while a non-nil empty
slice selects no files. A declaration's own `pattern` replaces the application
default. The defaults do not enable imports or change selected-overlay behavior.
The Go fields `FollowSymlinks` and `IncludeHidden` correspond to the unchanged
YAML keys `follow` and `hidden`.

`Result.Sources`, `LoadError.Sources`, and `Resolution.Source` expose each
declaring source's `EmptyImportSelections`. `Result.Source(id)` and
`LoadError.Source(id)` retrieve one retained record directly when its load-local
ID is known. These records identify the source-local declaration path, its
authored location and selector;
they do not include candidate names or a guessed reason for absence. Returned
records are detached snapshots. The [configuration contract](../config.md#explicit-imports)
owns selector applicability and exact absence semantics.

## Shared source-version checks

[`LoadVersionedService`](../../examples/consumers/source_version.go) demonstrates
if-present and required modes with one `Definition.Version` rule across a mounted
main input and an underlay. The example reads only sanitized
`SourceInfo.VersionEvidence` fields; its executable test is in
[`layered_test.go`](../../examples/consumers/layered_test.go). The
[configuration contract](../config.md#shared-source-version-checks) defines
participation, source-local lookup and diagnostics, non-main header consumption,
main-path protection, default behavior, and evidence disclosure. Accepted
declarations do not replace final assembled decoding or application validation.

## Effective configuration diffs

[`CompareFiles`](../../examples/consumers/diff.go) loads two caller-selected files
with one application definition, handles each load error, calls `configkit.Diff`,
and presents the safe records through consumer-owned `FormatDifferences`.
[`CompareRolloutOverlay`](../../examples/consumers/diff.go) loads the same base
once without an overlay and once with the selected overlay, so omitted lower
priority values remain present. Its resolver records a derived endpoint URL after
overlay application, and its nested typed secret reference is classified before
the resulting records are presented.

The maintained [behavior tests and executable
examples](../../examples/consumers/diff_test.go) use temporary files and
disposable secret values. Presenters should use `ValueSnapshot.Present` for
absence, record membership for change, and `HiddenChanged` for a withheld
change indicator. Do not parse the redaction marker or add separately rendered
full contexts that lack combined sensitivity. The [configuration
contract](../config.md#effective-configuration-diffs) owns shared equality,
granularity, and disclosure semantics.

## Named sibling inheritance

[`LoadPresets`](../../examples/consumers/inheritance.go) loads the maintained
[`presets.yml`](../../examples/consumers/testdata/inheritance/presets.yml) in one
call. `custom` inherits `enabled: false` from `base`, while its nested
`settings.primary` changes the inherited `settings.secondary` result at the
custom destination. The application struct has no `extends` field; ConfigKit
consumes the relationship before strict typed decoding.

The [executable example](../../examples/consumers/consumers_test.go) reads
`Metadata.IsExplicit` for the inherited false value, then follows
`Origin.Inheritance` back to the parent path while `Origin.Locations` identifies
the declaring source. Applications still choose which preset to execute and
validate domain meaning. The [configuration contract](../config.md#sibling-inheritance)
owns exact lookup, precedence, cancellation, and migration rules.

## Native filesystem and layered input

[`LoadBundled`](../../examples/consumers/layered.go) keeps the simple `Load` shape:
it selects a file in an application-owned embedded filesystem and needs no
underlay setup. [`LoadCatalog`](../../examples/consumers/layered.go) uses
`LoadLayered` with an imported embedded-defaults group, a separate
`Mapping` group, and a host main file supplied by its caller. The groups
compose strictly within themselves and recursively across precedence boundaries;
the main file's explicit `false` and `0` replace lower values without being
mistaken for omissions. Its validator owns allowed modes and resource-name policy.

[`ReadWelcome`](../../examples/consumers/layered.go) follows the effective field's
origin location to a source ID, queries `Result.SourceContext`, and opens the
resource relative to that context's `BaseDir`. It does not assume the main file
owns a retained underlay value. A composite origin needs an application choice;
ConfigKit does not guess a directory. The context query is passive, while later
resource reads observe the caller-owned filesystem or a freshly opened host root
and may fail independently of the completed configuration load. The
[configuration contract](../config.md#source-resource-contexts) owns exact
lifetime and precedence rules.

[`LoadPromptBundle`](../../examples/consumers/layered.go) is the ordinary-root
counterpart: its entry opts into imports through `Load`, imports metadata at the
root, and places a prompt beneath `prompts` using YAML. The executable example
and behavior test are in
[`layered_test.go`](../../examples/consumers/layered_test.go).

## Input destinations and mounted collections

Main and underlay inputs use the same destination field. An import without a
`destination` inherits its declaring document's effective destination; an
explicit sequence of key strings appends a relative suffix. Nested imports
append to their immediate parent's effective destination, while filesystem
paths remain relative to the declaring file. Destinations are application
mapping paths, independent of filesystem context. The configuration contract
owns the complete behavior and validation rules.

The small [`Example_inputDestination`](../../examples/consumers/destinations_test.go)
mounts one mapping source beneath literal collection/prompt keys and loads a
second input without a destination to show that root placement remains the
default. Consumers pass paths, not filenames or namespace strings for ConfigKit
to interpret.

[`LoadCollectionBundle`](../../examples/consumers/destinations.go) assembles an
embedded collection manifest mounted beneath a literal collection path. The
entry's YAML places nested metadata imports at the collection root and prompt
and profile peers beneath their own keys. A higher-priority input changes only
one prompt's ordinary `version` field. The main document alone supplies the
application schema version. A prompt's local `extends` relationship resolves
after all tiers, so `custom` inherits the final `brief` content and fields while
retaining its own profile selection.

The resolver follows each effective `content_file` origin to its declaring
source context. This also works for `custom`'s inherited resource. The
application-owned [`ReadCollectionPromptContent`](../../examples/consumers/destinations.go)
demonstrates the same lookup after loading; ConfigKit does not open or interpret
the resource. [`destinations_test.go`](../../examples/consumers/destinations_test.go)
checks source destinations and authored coordinates, presence, inherited values,
relative resources, and strict peer conflicts. The
[configuration contract](../config.md#input-destinations) owns shared placement,
import, source-location, and ownership behavior.

## Identified prompt sequences

[`LoadPromptCatalog`](../../examples/consumers/prompt_catalog.go) shows the
smaller import-focused adoption path: a registered `prompts.id` identity,
directory and glob selection, a helper import that supplies an item's ID, and
resource lookup through the effective source context. Its executable examples
and tests are in
[`prompt_catalog_test.go`](../../examples/consumers/prompt_catalog_test.go), with
complete [catalog](../../examples/consumers/testdata/prompt-catalog/config.yml)
and [required-file](../../examples/consumers/testdata/prompt-catalog/simple.yml)
fixtures.

[`LoadSequenceCatalog`](../../examples/consumers/sequence_catalog.go) integrates
an underlay declaration, imported main fragments, identified-record inheritance,
selected overlay `append`, `upsert`, and `replace`, a typed CLI override, and
recorded application resolution. Its tests and runnable example demonstrate
stable output order, whole-record upsert, provenance/history inspection,
resource lookup for inherited records, and secret-safe explanation in
[`sequence_catalog_test.go`](../../examples/consumers/sequence_catalog_test.go).
[`LoadNamespacedMapCatalog`](../../examples/consumers/sequence_catalog.go) adds
an exact root registration and a wildcard namespace registration, showing that
each concrete sequence has an independent identity scope. The same tests show
that a YAML identity declaration without an application registration does not
create final-result identity enforcement.

The examples keep prompt meaning and domain checks in the consuming application.
The [configuration contract](../config.md#strict-document-composition) owns
declaration, identity, operation, ownership, and final-validation semantics;
[sibling inheritance](../config.md#sibling-inheritance) defines record
relationships, and the [CLI contract](../config.md#explicit-cli-overrides) owns
override ordering.

Known module catalogs should use canonical object entries with a module identifier
and typed options. A small union struct is sufficient when the available option
fields are known; validation selects which fields apply to each module. Runtime-
selected catalogs may retain `map[string]any`, whose integers are `int64`; applications
validate keys and ranges and convert only at their runtime boundary. Neither shape
needs YAML reserialization, a parser hook, a ConfigKit registry, or a generic subtree
conversion API.

## Source-aware application path policy

[`LoadNotariusPaths`](../../examples/consumers/consumers.go) is the maintained
path-bearing example. Its application resolver uses `Resolution.EntryInfo` and
`Resolution.Source` before normalizing: relative CLI input uses an explicitly
supplied invocation directory, retained defaults use the canonical main source,
and a file-authored scalar uses its one declaring source. Absolute paths remain
unchanged. Its executable test checks the resulting YAML and explanation
history. When the configuration filename is a symlink, this example uses the
actual target file's directory as the base for file-authored paths.

The selected example policy leaves a relative value unchanged and emits a warning
when a CLI base is absent or file provenance is missing or composite. That is an
application choice, not a ConfigKit fallback: consumers can instead reject it or
choose another explicit base. The example intentionally performs neither target
existence checks nor confinement checks.

## Intentional migration changes

When upgrading, start with the changes relevant to your dependency and inputs:

1. Update the module requirement and Go imports together as described below.
2. Replace old public names using the old/new table.
3. Update any layer or diagnostic-code strings stored or compared by your code.
4. Review YAML and startup changes that apply to your application, then rebuild
   and run its configuration tests.

The maintained examples illustrate these configuration changes:

- Explicit empty strings, lists, zeroes, false values, and nulls remain intentional.
  ConfigKit accepts an empty list structurally; WeatherReporter-shaped validation
  rejects an explicitly or programmatically supplied empty module/template list as a
  domain rule. Loaded and constructed values use the same validator.
- Canonical module and object bindings replace string shorthand. Unknown fields and
  incompatible option shapes fail at their semantic paths.
- Schema versions remain string-backed in application types. A configured
  `VersionRule` accepts string or strict decimal-integer YAML at its exact path and
  supplies the spelling to both the checker and a plain string destination.
- Integers inside dynamic options are portable `int64` values. Runtime `int`
  conversion is explicit and range checked.
- Strict YAML intentionally rejects duplicate keys, aliases, merge keys, unsupported
  tags/scalars, trailing documents, and unknown typed fields. Diagnostics expose
  stable classifications and safe paths, not copied parser text.
- General environment-variable configuration overlays retire. Application CLI
  parsers translate only explicitly set configuration flags into `Override` records.
  Separate invocation state remains outside configuration.
- Credential fields migrate to environment-name `Secret` references. Applications
  explicitly call `LoadSecretsIntoEnvironment` before `Load` when they choose
  file bootstrap;
  blocked empty variables remain blocked, and ordinary `Load` never changes the
  process environment.

### Vanity module path migration

ConfigKit's module identifier changed from
`gitea.maximumdirect.net/eric/ConfigKit` to `go.maximumdirect.net/configkit` before
v1.0.0. Update the dependency requirement in your application's `go.mod` and its
Go imports together, selecting a revision or release that declares the new module
path. The package name remains `configkit`; the configuration model and public
symbol names are unchanged by this identity migration.

Go treats the old and new import paths as different packages, with different
named types. For example, if another library returns a ConfigKit `Path` or
`Secret` using the old import, your application cannot use it as the new
package's type just by changing a module requirement. Coordinate migration
with dependencies that pass ConfigKit values to or from your application. The
maintained examples use the new path, but other repositories must manage their
own migrations.

Historical tags keep their original module declarations and contents. Older
releases such as v0.10.2 still use the old path and cannot be installed under the
vanity identifier simply because discovery can find their repository. A matching
migrated revision is needed; no old-path compatibility module is supplied here.

### Pre-v1 public API cleanup

The pre-v1 API cleanup replaces these names outright; it does not retain
compatibility aliases:

| Previous API | Current API |
| --- | --- |
| `Resolution.InputInfo(path)` | `Resolution.EntryInfo(path)` |
| `Definition.Secret` | `Definition.Sensitive` |
| `SourceVersionModeDefault` | `SourceVersionModeMainOnly` |
| `SourceVersionCheck` | `SourceVersionEvidence` |
| `SourceInfo.VersionCheck` | `SourceInfo.VersionEvidence` |
| `ImportSelection` | `EmptyImportSelection` |
| `SourceInfo.ImportSelections` | `SourceInfo.EmptyImportSelections` |
| `SourceInfo.DestinationUnresolved` | `SourceInfo.DestinationIsFallback` |
| `ImportDefaults.Follow` / `.Hidden` | `FollowSymlinks` / `IncludeHidden` |
| `Paths(paths...)` | `SearchPaths(paths...)` |
| `LoadSecretsDir(directory)` | `LoadSecretsIntoEnvironment(directory)` |
| `LayerBase` | `LayerMain` |
| `CodeExplainLimitExceeded` | `CodeHistoryLimitExceeded` |
| `CodeResolveInvalidKind` / `CodeOptionsUnavailable` | Removed; no replacement |
| `ImportSelection.Outcome`, `ImportSelectionOutcome`, `ImportSelectionNone` | Removed; empty-selection records no longer carry an outcome |
| `Resolution.Record(path, ChangeDerivedDefault, reason, deps...)` | `Resolution.RecordDerivedDefault(path, reason, deps...)` |
| `Resolution.Record(path, ChangeNormalization, reason, deps...)` | `Resolution.RecordNormalization(path, reason, deps...)` |
| Enumerating `Result.Sources()` / `LoadError.Sources()` to find one ID | `Result.Source(id)` / `LoadError.Source(id)` |
| Provenance string `"base"` | `"main"` |
| History diagnostic string `"explain.limit_exceeded"` | `"history.limit_exceeded"` |

The YAML import keys `follow` and `hidden`, the `configkit:"secret"` tag, and
typed `Secret` references are unchanged. If your application compares or
stores layer names or diagnostic codes as strings, update those strings
alongside Go references. This repository guidance does not claim that
downstream applications have migrated and does not select a release version
for this breaking pre-v1 change. `ChangeKind` and its inspection constants
remain; use `Default` and `Normalize` when the helper should both assign and
record a value.

### Selected-overlay migration

To adopt the current selected-overlay API, rename `Options.Profile`,
`LayerProfile`, and `CodeProfile*` references to their `Overlay` equivalents; move
selected files from `profiles/<name>.yml` to `overlays/<name>.yml`; update any
handling of the `overlay` layer and `overlay.*` diagnostic strings; then rebuild.
The [configuration contract](../config.md#selected-overlays) is the canonical
behavioral reference. Application-owned profile concepts, such as `model_profile`,
are unchanged, and any CLI flag rename remains the application's decision.

### Background on the examples

These examples demonstrate migration shapes. They do not claim that Narratio,
Notarius, WeatherReporter, or Seriatim has been migrated. Updating those
repositories, their deployed YAML, CLI wiring, runtime catalogs, and
operational documentation remains separate downstream work.

## Example background

The maintained [`examples/consumers`](../../examples/consumers) package demonstrates
sanitized shapes using only ConfigKit and the Go standard library:

| Shape | Demonstrated fit | Application code deliberately retained |
| --- | --- | --- |
| Artifact defaults and normalization (Narratio-like) | Fresh defaults, explicit partial imports, artifact maps, recorded inherited defaults, normalization, and source ownership | Artifact meaning and the separate `model_profile` domain field |
| Versions and dynamic options (Notarius-like) | Canonical object bindings, a string-backed schema version accepting string or integer YAML, strict dynamic options, and `int64` dynamic integers | Catalog selection, allowed option keys, range-checked conversion to runtime `int` |
| Module validation and credential startup (WeatherReporter-like) | Defaults-only loading, report maps, canonical module objects, typed option unions, explicit CLI records, and typed secret startup | Report/module compatibility, rejection of explicitly empty domain lists, CLI parsing, and startup ordering |
| Independent ordered rule files (Seriatim-like) | Independent strict ordered speaker and autocorrect rule files | First-match and ordered replacement algorithms; no central application configuration file |
| Bundled file | Simple native `fs.FS` source through ordinary `Load` | Selection and lifetime of the application-owned embedded filesystem |
| Layered catalog | Embedded defaults with an import, a mapping underlay, an independent host main file, explicit false/zero input, and source-relative resource access | Domain validation and opening/interpreting the template resource |
| Mounted collections | Entry-declared relative destinations, nested metadata/prompt/profile imports, local inheritance, and a version-only higher-tier override | Collection IDs, profile meaning, and opening/interpreting prompt resources |
| Named presets | One-load sibling inheritance, explicit false, nested destination resolution, and source/derivation inspection | Choosing and executing a preset |
| Effective configuration diff | Same-type frozen effective comparisons for two selected files or base-versus-overlay impact | Source/options selection and consumer-owned safe presentation |
| Shared source-version checks | One `Definition.Version` policy for accepted declarations across participating source documents, with optional or required declarations and sanitized evidence | Choosing which declarations the application accepts; final assembled schema and domain validation |
| Sequence catalogs | Declared collection operations, record inheritance, typed CLI and resolution, inspection, exact and wildcard application registrations | Catalog meaning and final domain validation |

The YAML under [`testdata`](../../examples/consumers/testdata) is loaded by the
executable examples and ordinary tests. It contains no private application data,
credentials, network calls, or dependency on a sibling checkout.

The Notarius-shaped fixtures deliberately load both `version: 4` and
`version: "4"` into the same string-backed application field. Its version checker
still owns the supported identifier; fractional numeric input and unsupported
integer identifiers remain negative examples.

## Maintained API examples

Every nontrivial initial API sketch has a compiled example or a
focused behavioral owner:

| Roadmap surface | Maintained owner |
| --- | --- |
| `Load`, `NoSource`, validation, versions, resolution, `SecretEnv`, overrides, and explanation | Executable examples in [`example_test.go`](../../example_test.go) |
| Source/import/selected-overlay pipeline rules and artifact defaults | Narratio-shaped executable example and fixture |
| Canonical module objects and dynamic option handling | WeatherReporter- and Notarius-shaped examples and migration tests |
| `LoadSecretsIntoEnvironment`, bootstrap reporting, typed lookup, CLI explanation | `Example_weatherStartup` |
| Native filesystem, grouped underlays, programmatic mapping, retained resource context | `Example_nativeFilesystem`, `Example_layeredCatalog`, and their focused consumer tests |
| Literal input destinations, root-default compatibility, and mounted peer collections | `Example_inputDestination`, `LoadCollectionBundle`, and `destinations_test.go` |
| Named sibling inheritance, presence, and derivation | `Example_inheritedPresets` and `TestPresetCatalogOneLoadAndDerivation` |
| Effective configuration diffs and consumer-owned presentation | `ExampleDiff`, `ExampleCompareFiles`, and `ExampleCompareRolloutOverlay` |
| Results, metadata, diagnostics, effective output, and writer failures | Focused root-package load, provenance, rendering, and explanation tests |

The package-level import-boundary test covers the consumer examples too: supported
usage names no YAML dependency, parser type, parser option, or parser error chain.