Make consumer documentation easier to follow

This commit is contained in:
2026-11-01 19:04:48 +00:00
parent e74c26a4b3
commit 6db441893d
12 changed files with 1065 additions and 399 deletions

View File

@@ -1,27 +1,66 @@
# Configuration Input Contract
This document owns ConfigKit's currently implemented source-input, YAML syntax,
schema, typed-decoding, loading, metadata, redaction, rendering, and ownership
contracts. The parser, owned document representation, and decoder remain private
behind the public loading lifecycle.
This reference explains the YAML ConfigKit accepts, how inputs combine, and what
an application receives from a load. For a first working example, start with the
[README](../README.md#load-a-configuration-file) or the
[consumer guide](consumers/README.md).
Configuration authors can start with [YAML values](#scalars),
[combination rules](#strict-document-composition), [imports](#explicit-imports),
and [inheritance](#sibling-inheritance). Go API users can start with
[sources](#sources), [defaults and decoding](#defaults-and-typed-decoding),
[application callbacks](#recorded-application-resolution), and
[inspection](#layered-metadata). [Credentials](#typed-secret-references) and
[secrets-file startup](#explicit-secrets-directory-bootstrap) have separate roles.
## Terms used in this reference
A **mapping** is a group of named YAML values; a **sequence** is a list. A
**scalar** is a single value such as a string, number, or boolean. For example:
```yaml
service: # mapping
enabled: false # scalar
names: [primary, spare] # sequence
```
A **source** supplies an input document, such as a file or an in-memory mapping.
A **tier** is a group of inputs at the same priority. **Precedence** describes
which tier takes priority when several tiers supply a setting. Inputs within
one tier follow strict combination rules.
A **configuration path** names a value inside the configuration, rather than a
file on disk. For example, `/service/names/0` identifies the first name in the
YAML above. Go callers can construct it with `Keys("service", "names").Index(0)`.
Its text uses JSON Pointer escaping: `~0` represents `~` and `~1` represents `/`
inside a key. Explicit mapping keys and sequence indexes remain distinct even
when their pointer text looks the same.
**Metadata** describes a value's presence and **origin**: where it came from,
including application processing that changed it. A returned **detached** record
is a copy you can change without changing ConfigKit's stored record. A **snapshot**
is saved configuration state used for inspection after loading finishes.
Borrowed filesystems have their own [resource lifetime](#source-resource-contexts).
## Sources
ConfigKit exposes opaque sources through these constructors:
A source describes the input to pass to `Load` or `LoadLayered`. Choose the
constructor that matches where your configuration comes from:
| Constructor | Current contract |
| --- | --- |
| `File(path)` | Requires a nonempty path, opens a regular file for each consumption, and closes only the library-owned file handle. Missing, inaccessible, unreadable, directory, and other nonregular inputs fail. |
| `File(path)` | Requires a nonempty path, opens a regular file for each load, and closes only the library-owned file handle. Missing, inaccessible, unreadable, directory, and other nonregular inputs fail. |
| `SearchConfig(application, search, options...)` | Creates an explicit, reusable search source. Loading selects one host file or follows the absent-input lifecycle when an optional search is exhausted. |
| `NewFilesystem(label, filesystem).File(name)` | Borrows a caller-owned `fs.FS` under a safe nonempty display label. `name` must be a non-root slash-relative `fs.ValidPath`. Each load reopens the selected regular file; copies of the `Filesystem` preserve namespace identity. Separate constructors create separate namespaces even for the same label or filesystem. |
| `Bytes(label, data)` | Requires a nonempty, non-secret diagnostic label and snapshots bytes during construction. Later caller mutation cannot affect the source. |
| `Reader(label, input)` | Requires a nonempty, non-secret label and non-nil reader. The reader is claimed once across all copies of the source, consumed but never closed, and cannot be reused. |
| `Mapping(label, values)` | Requires a safe nonempty label and a non-nil root `map[string]any`. Snapshots supported built-in values into an owned mapping document at construction; `{}` is valid. Constructor conversion failures become safe deferred diagnostics. |
| `Reader(label, input)` | Requires a nonempty, non-secret label and non-nil reader. Copies of the source share one read attempt. ConfigKit never closes the reader; you remain responsible for closing it if needed. |
| `Mapping(label, values)` | Requires a safe nonempty label and a non-nil root `map[string]any`. Snapshots supported built-in values into an owned mapping document at construction; `{}` is valid. Conversion failures are reported when loading uses the source. |
| `NoSource()` | Selects defaults-only loading, creates no source record, and marks no paths explicit. |
The zero `Source` is invalid. Constructor argument failures are retained inside the
opaque source and become safe structured diagnostics when the source is consumed.
File and filesystem sources can observe changes between consumptions; byte and
The zero `Source` is invalid. Constructors store argument failures so loading
can report them as structured diagnostics. A source's private representation is
not part of the API.
File and filesystem sources can observe changes between loads; byte and
mapping sources cannot. Mapping accepts nil, built-in string, boolean, signed or
unsigned integer, finite float, `[]any`, and `map[string]any` values recursively.
Typed nil maps and slices represent explicit null; empty collections remain empty.
@@ -51,12 +90,22 @@ group contribute no candidates. An explicitly supplied empty path or filename
is invalid. Slugs and custom filenames must be usable single host filename
components; ConfigKit preserves their case and spelling.
Search expansion is ordered concatenation. For each directory search, ConfigKit
tries its filename groups and names before advancing to the next directory.
When multiple directories use the same filename list, candidates follow the
directory-first Cartesian product: every filename in one directory is tried
before the next directory. Explicit file candidates stay at their declared
positions among directory searches.
Search descriptions are tried in the order supplied. Within a directory search,
ConfigKit tries every selected filename before moving to the next directory.
For example, two directories using `NamedFilenames` for application `demo` are
searched in this order:
```text
first/demo.yml
first/demo.yaml
second/demo.yml
second/demo.yaml
```
Explicit file candidates keep their declared positions among directory searches.
If no candidate exists, an optional search continues as absent input and a
required search fails. Once a file is selected, a failure to load it is an error;
ConfigKit does not continue looking for another configuration file.
The presets expand to these ordered directories and filename groups:
@@ -102,6 +151,8 @@ candidates create no records. `SearchConfig` can be used as the main source of
ordinary or layered loading, including an underlay input, and does not add a new
precedence tier.
### Input limits
Each source is limited to 8 MiB. A load using imports or an active overlay is limited
to 64 MiB across all accepted sources, 1,024 accepted sources, 64 active import
edges, and 1,000,000 input nodes in aggregate. Glob expansion separately permits
@@ -113,15 +164,19 @@ overrides, and the converted tree uses the same node and depth bounds.
## Single-source loading
`Load[T](source, options...)` accepts zero or one `Options[T]`. The configuration
root `T` must satisfy the application schema contract below. With no options it
starts from the Go zero value. A `Definition[T].Defaults` factory, when supplied,
runs exactly once after successful source parsing, schema construction, optional
schema-version checking, and selected-overlay loading. Its result is detached before
input is applied. Resolver-provided mutable values are detached before secret lookup
and validation, and a final detached copy prevents equal-valued aliases introduced
by validators from escaping into the returned configuration. The publication copy
preserves already validated secret state without rereading the environment.
`Load[T](source, options...)` returns a configuration of your Go type `T` plus
metadata for inspection. It accepts zero or one `Options[T]`. The root type must
follow the [schema rules](#application-schema-binding). With no options, loading
starts from the Go zero value. To supply defaults, provide a
`Definition[T].Defaults` function that returns a fresh configuration.
That function runs exactly once after successful source parsing, schema checks,
optional version checking, and selected-overlay loading. ConfigKit copies its
result before applying input. Mutable values supplied by a resolver are also
copied before secret lookup and validation. A final copy prevents even
equal-valued replacements made by validators from leaving shared mutable storage
in the returned configuration. It preserves validated secret state without
reading credentials again.
The current lifecycle is:
@@ -136,11 +191,12 @@ validate options → parse the selected source → extract source controls and
→ snapshot configuration and metadata
```
`File`, `Filesystem.File`, `Bytes`, `Reader`, and `Mapping` each contribute the first base source with deterministic
ID `source-1`; enabled imports continue in depth-first order as `source-2` onward.
`NoSource` skips parsing, supplies an absent version to a configured version checker,
contributes no source, and continues through defaults, decode, and projection. A
missing file remains an error; it is never interpreted as no input.
`File`, `Filesystem.File`, `Bytes`, `Reader`, and `Mapping` each contribute
the first base source with deterministic ID `source-1`; enabled imports
continue in depth-first order as `source-2` onward. `NoSource` skips parsing,
supplies an absent version to a configured version checker, contributes no
source, and continues through defaults, decode, and projection. A missing file
remains an error; it is never interpreted as no input.
`Options` accepts the application definition, ordinary-source `Destination` and
`PlacementMode`, explicit imports, one selected overlay, filesystem
@@ -161,16 +217,23 @@ when imports are disabled. An active ordinary overlay requires this explicit
context. `NoSource` accepts overrides and remains absent rather than contributing
an empty document.
Every source, options, schema, version, defaults, decode, resolution, or validation
failure returns a nil result and a `*LoadError`. Successful results contain caller-owned `Config`, detached
`Metadata`, a defensive source list, and defensive nonfatal diagnostics. Each source
record's `Kind` identifies file, filesystem, bytes, reader, mapping, or CLI acquisition
independently of its precedence `Layer`. Host file paths remain canonical;
filesystem paths are slash-relative within their namespace; synthetic inputs have
no path. Mutating
the returned configuration or inspection records cannot alter ConfigKit's private
snapshot. ConfigKit performs no logging. `Result.WriteEffective` exposes only the
frozen redacted projection.
### Results and failures
You can change `Result.Config` after loading. Inspection methods continue to
describe the configuration as it was when the load finished. Returned source,
diagnostic, and metadata records are copies; changing them cannot change later
queries. Separate loads return independently owned configuration and records.
Every source, options, schema, version, defaults, decode, resolution, or
validation failure returns a nil result and a `*LoadError`. Successful results
contain caller-owned `Config`, detached `Metadata`, a defensive source list,
and defensive nonfatal diagnostics. Each source record's `Kind` identifies
file, filesystem, bytes, reader, mapping, or CLI acquisition independently of
its precedence `Layer`. Host file paths remain canonical; filesystem paths are
slash-relative within their namespace; synthetic inputs have no path. Mutating
the returned configuration or inspection records cannot alter ConfigKit's
private snapshot. ConfigKit performs no logging. `Result.WriteEffective`
exposes only the frozen redacted projection.
`Result.Source(id)` returns the detached record for an acquired source already
listed by `Result.Sources()`. Source IDs are local to one load; empty and unknown
@@ -186,7 +249,7 @@ or unknown IDs return false.
### Completion and resource lifetime
`Load` is synchronous and complete on return. Success means input assembly,
`Load` finishes its work before returning. Success means input assembly,
defaulting, typed decoding, configured application resolution, secret lookup,
validation, and configuration/metadata snapshotting have completed. A reported
load failure returns diagnostics and a nil result, as described above.
@@ -204,9 +267,9 @@ when explicitly called and use the retained snapshot. Applications normally load
once near startup, but may explicitly load again using sources that satisfy their
documented reuse rules. Additional loads produce independently owned results.
Completion is not a transaction over external state. Files and environment
variables are observed during processing rather than as one simultaneous external
snapshot. ConfigKit does not roll back application callback effects or manage work
Files and environment variables are read during processing; they are not all
captured at one instant. If your callback changes something outside the working
configuration, ConfigKit does not undo that effect when loading fails or manage work
or resources created by application callbacks. The explicit secrets-directory
bootstrap remains a separate operation invoked by the application.
@@ -221,6 +284,23 @@ nothing. Underlay groups are listed from lowest to highest priority. Repeating
a file within one group remains an import/repetition error; the same reusable
source may be selected again in another tier.
### Priority and processing order
The input priority order is shown below, from lowest to highest. Mappings combine
by key across these boundaries; identified lists use their collection operation.
```text
typed defaults → underlay groups (first to last) → main + imports → overlay → CLI
```
This is a priority order, rather than a schedule of processing steps. Defaults
have low priority but are applied during typed decoding, after source assembly
and inheritance. Application resolution runs after CLI assignment and can make
recorded changes. Secret lookup and validation then assess the resulting values.
See the [load timeline](#single-source-loading) for the processing sequence.
### Source order and version checks
ConfigKit acquires the main input first and constructs the schema. It then
acquires and strictly composes underlay groups from lowest to highest priority,
passing each group's accumulated lower collection policy into the next group.
@@ -260,7 +340,42 @@ is the sequence container. After assembly assigns an index, nested source
destinations are projected through that item. Physical source locations and
resource contexts continue to identify the declaring source.
#### Input destinations
### Input destinations
A destination controls where settings appear in the configuration. It is
independent of where the source file lives on disk. Consider this file layout:
```text
/etc/demo/
config.yml
fragments/
prompt.yml
welcome.txt
```
With imports enabled, `config.yml` can contain:
```yaml
configkit:
imports:
- path: fragments/prompt.yml
destination: [prompts, welcome]
```
If `prompt.yml` contains `template: welcome.txt`, the assembled input is:
```yaml
prompts:
welcome:
template: welcome.txt
```
The import filename is relative to `config.yml`; the destination places its
contents beneath `/prompts/welcome`. A later application read of `welcome.txt`
can use the imported source's context, as described under
[resource contexts](#source-resource-contexts).
#### Destination rules and inspection evidence
`Input.Destination` places the source's application mapping beneath a path
relative to the configuration root. Its zero or empty value means root, so
@@ -303,15 +418,52 @@ acquisition identity, physical source locations, or passive resource contexts.
Introduced ancestor mappings use the contributing document's root location as a
source fallback; authored content retains its original source coordinates.
The maintained [layered consumer example](consumers/README.md#native-filesystem-and-layered-input)
shows a main host file, two underlay tiers, embedded defaults, a mapping source,
and explicit resource access without a second configuration pass.
The accepted rationale for input placement and its relative-import extension
is recorded in [ADR 0011](adr/0011-input-destinations.md) and
[ADR 0012](adr/0012-relative-import-destinations.md).
The maintained [layered consumer
example](consumers/README.md#native-filesystem-and-layered-input) shows a main
host file, two underlay tiers, embedded defaults, a mapping source, and
explicit resource access without a second configuration pass. The accepted
rationale for input placement and its relative-import extension is recorded in
[ADR 0011](adr/0011-input-destinations.md) and [ADR
0012](adr/0012-relative-import-destinations.md).
### Sibling inheritance
Inheritance lets one entry reuse another entry's settings. A sibling is another
entry in the same enclosing mapping. For example:
```yaml
presets:
base:
width: 10
length: 20
custom:
extends: base
width: 30
```
The resulting `custom` has `width: 30` and `length: 20`; `base` remains an
ordinary entry. ConfigKit resolves this after combining source layers, so the
child uses the parent's settings that remain after that combination.
`extends: null` cancels a relationship and keeps the child's own settings.
An identified list uses a mapping reference instead of a sibling name:
```yaml
configkit:
sequences:
- destination: [presets]
key: id
presets:
- {id: base, width: 10, length: 20}
- {id: custom, extends: {id: base}, width: 30}
```
Here the child keeps `id: custom` and receives the parent's other settings.
It has `width: 30` and `length: 20`. Every parent remains subject to ordinary
application decoding and validation.
#### Lookup, processing order, and limits
Any input mapping may declare `extends: name` to inherit from the mapping named
`name` among its immediate siblings. Names are exact keys, including the empty
string; lookup does not search ancestor or unrelated scopes. A mapping inside a
@@ -390,6 +542,21 @@ usage.
### Source resource contexts
Use a source context when a configuration value names another file, such as a
template. It tells the application which filesystem and directory belonged to
the source that supplied the value. Obtaining the context does not open files.
For the [placement example](#input-destinations), inspect the origin of
`/prompts/welcome/template` and take its source ID. `result.SourceContext(id)`
provides a filesystem rooted at `/etc/demo` and a `BaseDir` of `fragments`.
Joining that base with `welcome.txt` gives `fragments/welcome.txt`, which the
application can open through the returned `FS`. Close the returned file after
reading it. The maintained [ReadWelcome example](../examples/consumers/layered.go)
shows this workflow. If a value has several contributing sources, the application
must choose the intended source according to its own rules.
#### Context queries and lifetime
`Result.SourceContext(id)` and callback-scoped `Resolution.SourceContext(id)`
return a passive `ResourceContext` for a source with filesystem context, or
false for unknown and contextless IDs. The result query is nil-safe; the
@@ -423,6 +590,12 @@ a caller-owned filesystem closes. This is not a snapshot or universal sandbox.
### Application schema versions
Your application chooses which version field to check and which identifiers it
accepts. ConfigKit passes the declared value to that checker; it does not assign
meaning to a version number. For a rule at `version`, both `version: 4` and
`version: "4"` supply text `4`. A plain string field can receive that spelling
under the conversion rules below.
`VersionRule` declares a semantic path and non-nil checker. Before defaults or typed
decoding, the checker receives a `Version` preserving absent, explicit null, string,
or strict decimal-integer input. Strings and integers supply their exact spelling in
@@ -455,15 +628,43 @@ successful result. Callback panics remain programming failures.
#### Shared source-version checks
By default, version checking concerns the assembled main input. Enable a shared
source-version mode when the application needs to check declarations in individual
main, imported, underlay, and selected-overlay documents with the same rule.
With a configured usable version rule, the main cases are:
| Situation | `MainOnly` (default) | `IfPresent` | `Required` |
| --- | --- | --- | --- |
| Header absent in an actual main document | Call the checker with an absent version. | Call the checker with an absent version. | Reject before calling the checker. |
| Header absent in another participating document | No source-local check; existing version-path protection applies. | Skip its checker call. | Reject the document. |
| Main header is explicit null | Call the checker with `Null` true. | Call the checker with `Null` true. | Call the checker with `Null` true. |
| Main input is `NoSource()` | Keep the absent-main checker call. | Keep the absent-main checker call. | Keep the absent-main checker call. |
The short column labels stand for the `SourceVersionMode` constants below.
In enabled modes, an explicit null in another participating document also reaches
the checker. The application's rule decides whether null is accepted.
For example, a main document with `version: 4` placed at destination `[service]`
is checked at its own `/version` path in enabled modes. Its accepted value remains
at `/service/version` in the application configuration, and protection applies
there. Accepted non-main headers are removed before documents combine, so they
cannot overwrite the main header. The maintained
[versioned-service example](../examples/consumers/source_version.go) demonstrates
this placement. Source evidence records what was observed; its existence alone
is not proof that a checker ran or accepted a declaration.
##### Mode rules and version evidence
`Options.SourceVersionMode` and `LayeredOptions.SourceVersionMode` select how
one `Definition.Version` policy applies across source documents. The zero value
`SourceVersionModeMainOnly` (the zero value) preserves the existing main-only
check at its effective path and existing protected-path behavior.
`SourceVersionModeIfPresent` checks each participating document that declares a
version and permits absent non-main declarations. `SourceVersionModeRequired` rejects a participating
document without a declaration. Enabled modes require a usable `VersionRule`
and checker; an unknown mode or unusable rule is an options error before source
access or callbacks.
one `Definition.Version` policy applies across source documents. The zero
value `SourceVersionModeMainOnly` preserves the existing
main-only check at its effective path and existing protected-path behavior.
`SourceVersionModeIfPresent` checks each participating document that declares
a version and permits absent non-main declarations.
`SourceVersionModeRequired` rejects a participating document without a
declaration. Enabled modes require a usable `VersionRule` and checker; an
unknown mode or unusable rule is an options error before source access or
callbacks.
In enabled modes, `Definition.Version.Path` is interpreted in each acquired
document's own coordinates, before placement and composition. The main document,
@@ -516,7 +717,27 @@ the completed implementation and remediation records in Git history.
### Layered metadata
`Result.Metadata` describes every final canonical path. Queries retain explicit
Use inspection to understand a completed load without reading the files again:
| Need | API to use |
| --- | --- |
| Check whether input supplied a final value, including false or zero | `Metadata().Lookup` or `IsExplicit` |
| Find the sources that supplied it | The field's `Origin`, then `Source(id)` or `Sources()` |
| Inspect attribution for earlier definitions | `Metadata().History` |
| Inspect earlier values as safe YAML | `Explain` or `WriteExplanation` |
| Show the saved final configuration as safe YAML | `WriteEffective` |
| Compare final values from two loads | `Diff` |
For example, suppose a typed default sets `workers` to `2`, the main input
supplies `workers: 3`, and an overlay supplies `workers: 4`. The final value is
`4`, it is explicit, and its origin identifies the overlay. History retains the
earlier default and main contributions. `History` returns attribution records;
`Explain` adds their before/after values. Changing `result.Config.Workers` later
does not change any of these saved inspection results.
#### Presence, origins, and historical paths
`Result.Metadata()` describes every final configuration path. Queries retain explicit
key/index distinctions using actual effective and historical container shapes;
only parsed numeric tokens are resolved contextually. A descendant that never
existed is not historical merely because an atomic ancestor was replaced. Exact
@@ -572,6 +793,18 @@ reports it absent. Queries return detached records.
### Secret classification and effective rendering
Choose which configuration values to hide when displaying a load. A
`configkit:"secret"` field tag or `Definition.Sensitive` predicate marks a path
as sensitive. Marking a container also hides its descendants. This choice is
separate from using a typed `Secret` to read a credential, described below.
The saved output continues to hide sensitive historical values even when a
higher layer replaces them. When a resolver records a dependency on a sensitive
value, that sensitivity also protects the derived target. The detailed rules
below explain how classification spreads and how output is formatted.
#### Classification and output details
Secret classification combines `configkit:"secret"` field tags with the optional
pure `Definition.Sensitive(Path) bool` predicate. Tags on concrete structs inside
supported dynamic values are retained in typed projections, including superseded
@@ -617,6 +850,15 @@ prose.
### Historical-value explanation
An explanation shows the final value and the earlier contributions that led to
it. In the `workers` example above, a caller can inspect `/workers` to see the
default, main input, and overlay along with their source attribution. The
returned values are YAML strings in copies of the saved records; editing them
cannot change a later explanation. Sensitive values are hidden before those
strings are returned.
#### Value representations and history
`Result.Explain(path)` returns a detached `Explanation` for any known effective or
historical path. Its effective value and each chronological before/after value are
`ValueSnapshot` records containing deterministic YAML text rather than mutable Go or
@@ -700,6 +942,23 @@ kind, or null/container replacements report their affected region. Records use
the existing comparator's canonical pointer ordering. `Present=false` with empty
YAML means absence; explicit null is present and renders as `null`.
The result fields help distinguish absence from hidden output:
| Field | Meaning |
| --- | --- |
| `ValueSnapshot.Present` | False means the value is absent. Explicit null is present and renders as `null`. |
| `ValueSnapshot.Redacted` | This value, or something inside it, was hidden. |
| `Difference.HiddenChanged` | A hidden value or its presence actually changed. Redaction alone does not set this flag. |
For example, a sensitive string changing from one value to another can produce
the same `'[REDACTED]'` text on both sides. The difference record still exists and
`HiddenChanged` is true. A literal input string that looks like that marker is
ordinary data; use the fields rather than parsing marker text. A typed secret's
reference name is compared, so rotating its environment credential without
changing that name does not produce a difference.
### Combined protection and comparison details
Before either side is returned, sensitivity from both loads is applied to both
snapshots, including historical and dependency-derived classification. A
classification on either side protects the corresponding values on both sides.
@@ -722,7 +981,24 @@ its normal nil-result/`LoadError` behavior, which callers handle before invoking
## Typed secret references
Credential-bearing fields use the opaque `Secret` type and store an environment
There are three separate tasks when working with credentials:
| Task | Mechanism |
| --- | --- |
| Read a credential from an environment variable | A typed `Secret` field whose YAML value is the variable's name |
| Hide a configuration value in inspection output | A `configkit:"secret"` tag or `Definition.Sensitive` |
| Populate environment variables from local files before loading | An explicit `LoadSecretsIntoEnvironment` call |
For example, YAML `api_key: DEMO_API_KEY` names an environment variable. Its
credential is available to trusted application code through `Secret.Value()`
after loading succeeds. The name `DEMO_API_KEY` remains visible in inspection
unless the path is separately classified as sensitive. The credential itself is
never retained in inspection history. The optional file helper below populates
the environment; ordinary configuration loading does not write it.
### Reference requirements and credential lifetime
Fields that need credentials use the `Secret` type and store an environment
variable name in YAML, such as `api_key: OPENROUTER_API_KEY`. `SecretEnv(name)`
constructs the same unresolved reference for application defaults or recorded
resolution. Names must match `[A-Za-z_][A-Za-z0-9_]*`; they are public
@@ -770,6 +1046,34 @@ reference state without printing its value.
## Explicit secrets-directory bootstrap
If an application stores credentials in files, it can call this helper before
loading configuration. A file named `DEMO_API_KEY` containing the dummy text
`example-value` supplies environment variable `DEMO_API_KEY`. YAML can then
refer to that variable with `api_key: DEMO_API_KEY`. Applications that already
receive environment variables do not need this file step.
Existing environment variables take priority over files. If `DEMO_API_KEY` is
already set to an empty string, its file is skipped and it stays empty. A later
required `Secret` reference will still fail on that empty value. File bootstrap
accepting an empty file is different from a required reference resolving to an
empty credential.
Read the report even when the helper returns an error:
| Status | How to interpret it |
| --- | --- |
| `loaded` | This file supplied the environment value. |
| `skipped_existing` | The variable was already set; `ExistingEmpty` says whether that existing value was empty. |
| `failed` | This entry could not be processed; inspect the safe diagnostics. |
| `not_applied` | No value from this eligible entry was applied, for example because preflight failed or an earlier write failed. |
Render the returned `SecretsReport` separately from a later configuration load.
It contains filenames, paths, and statuses, not credential contents. The
[maintained startup example](../examples/consumers/consumers_test.go) shows how to
keep bootstrap and configuration reports together in the application.
### Preflight, limits, and partial writes
`LoadSecretsIntoEnvironment(directory)` is a separate, opt-in startup operation.
It is never called by `Load`, is not a YAML option, and does not participate in
configuration validation. Applications choose the native directory path and
@@ -809,6 +1113,44 @@ files when appropriate; ConfigKit stores no implicit global report.
### Recorded application resolution
Application rules fit into three tasks:
| Task | Definition field |
| --- | --- |
| Supply fixed starting values | `Defaults`, which returns a fresh configuration |
| Derive or normalize values while recording why they changed | `Resolve` |
| Report domain errors and warnings without changing values | `Validate` |
Within `Resolve`, choose a helper according to whether it should assign a value:
| Helper | Assigns a value? | Records the change? |
| --- | --- | --- |
| `Default` | Only when input did not explicitly supply the path. | When it assigns. |
| `Normalize` | Always. | Yes. |
| `RecordDerivedDefault` | No; the application has already performed the mutation. | Yes, including for explicit input. |
| `RecordNormalization` | No; the application has already performed the mutation. | Yes. |
For a configuration with `Workers` and `QueueLimit`, a resolver can use this call:
```go
configkit.Default(resolution, configkit.Keys("queue_limit"),
&configuration.QueueLimit, configuration.Workers*2,
"size queue from workers", configkit.Keys("workers"))
```
If input omits `queue_limit`, this derives it from `workers` and records the
reason and dependency. An explicitly supplied `queue_limit: 0` is preserved.
Dependency paths explain where a derived value came from and support sensitivity
propagation. Use value-free reasons. See the [executable API examples](../example_test.go)
for complete callback definitions.
Use a `Resolution` or `Validation` context only inside its callback. Do not retain
it or use it concurrently. Returned metadata records are copies you can keep.
`EntryInfo` keeps describing the configuration at the start of resolution,
including defaults and CLI assignments, even after your resolver changes values.
#### Resolution recording and failure rules
An optional `Definition.Resolve` callback runs after successful defaults, typed
decoding, and explicit CLI assignment. It receives the working configuration and a
per-load `Resolution` context. The context's
@@ -871,6 +1213,11 @@ snapshots.
### Semantic validation
Use validation for application rules such as a nonnegative worker count or a
required combination of fields. Report errors and warnings through the supplied
context. Make any normalization or derived assignment in `Resolve` first;
validation examines the final configuration without changing it.
An optional `Definition.Validate` callback runs after successful resolution and
receives the resolved configuration plus a per-load `Validation` context. It can
collect multiple independent `validate.error` and `validate.warning` reports.
@@ -898,6 +1245,24 @@ live services as a substitute for configuration validation.
## Strict document composition
### Combining mappings
The main document and its imports are peers in one tier. For example, a main
file containing `service: {workers: 2}` and an import containing
`service: {enabled: false}` produce:
```yaml
service:
workers: 2
enabled: false
```
If the import also defined `workers`, strict composition would fail, even if its
value were also `2`. An overlay containing `service: {workers: 4}` intentionally
changes the result to `workers: 4`, retaining `enabled: false`. This shows the
difference between combining peers and applying a higher-priority layer.
Identified lists have their own rules below.
Ordinary `Load` applies strict document composition to its parsed source scope;
`Options.Imports` adds imported documents as peers in that scope. Application
defaults do not participate. With imports disabled, the single source still has
@@ -920,6 +1285,46 @@ Sequences without effective identity conflict when repeated and are never joined
by index. Disjoint entries under shared mappings compose, making stable keyed
mappings the natural representation for independently contributed entries.
### Identified lists: a worked example
A record is a mapping inside a list. An identity field such as `id` lets ConfigKit
tell which records refer to the same entry. Declare the identity and desired
operation at the source root, for example:
```yaml
configkit:
sequences:
- destination: [records]
key: id
mode: upsert
```
Suppose a lower layer supplies
`records: [{id: a, width: 10, length: 20}, {id: c, width: 50}]`, and
its higher layer supplies `records: [{id: a, width: 30}, {id: b, width: 40}]`.
The result during source assembly depends on the selected operation:
| Operation | Result for this example |
| --- | --- |
| `replace` | `[{id: a, width: 30}, {id: b, width: 40}]` |
| `append` | Error: both layers contain identity `a`. |
| `upsert` | `[{id: a, width: 30}, {id: c, width: 50}, {id: b, width: 40}]` |
Upsert replaces the complete matching record, so `length` from the lower `a`
record is gone. The unmatched lower `c` record stays at its original position;
new identities append. If the higher layer supplied only `[{id: b, width: 40}]`,
append would succeed with
`[{id: a, width: 10, length: 20}, {id: c, width: 50}, {id: b, width: 40}]`.
Typed defaults and decoding happen later.
Within a strict tier, distinct identities combine and duplicate identities fail
under every operation. For example, two imported peers may each supply different
records, but cannot both define record `a`. Across layers, the operation table
below applies. Without identity, repeated peer lists conflict and higher-layer
lists replace whole lower lists.
### Application registrations and identity details
`Definition.SequenceIdentities` contains registrations created by
`SequenceIdentityAt` or `SequenceIdentityMatching`. The exact form accepts a
mapping-only `Path`; the matching form accepts literal-key and any-mapping-key
@@ -976,6 +1381,8 @@ headers are consumed before strict composition and cannot supply a registered
record identity. Final typed decoding and application validation still run on the
completed configuration.
### YAML declarations and policy lifetime
Ordinary sources, layered inputs, and selected overlays may declare collection
behavior under `configkit.sequences` at the document root. Each entry has a
required literal `destination` and may specify `key` as one string or a nonempty
@@ -1167,6 +1574,52 @@ zero.
## Explicit imports
Imports include other documents as peers of the declaring document. Enable
imports in the application (`Options.Imports` or the layered input's `Imports`)
and choose one selector per declaration. These are alternatives for selecting
files, not three declarations that must be used together:
Named file:
```yaml
configkit:
imports:
- path: fragments/service.yml
```
Directory search, restricted here to YAML filenames:
```yaml
configkit:
imports:
- dir: fragments
pattern: "*.yml"
```
Path pattern:
```yaml
configkit:
imports:
- glob: "fragments/*.yml"
```
Each selected document follows the same strict combination rules. A missing
optional input can be absent; invalid declarations, unreadable selected files,
and conflicting settings still fail loading.
| Option | `path` | `dir` | `glob` |
| --- | --- | --- | --- |
| `required`, `exclude`, `destination`, `placement` | Supported | Supported | Supported |
| `pattern`, `recursive`, `depth` | Not supported | Supported (`depth` needs effective recursion) | Not supported |
| `follow`, `hidden` | Not supported | Supported | Supported for wildcard-discovered entries |
Each applicable option is chosen independently: its declaration value wins over
an application import default, which wins over the library default. The detailed
rules below explain absence, patterns, placement, and ordering.
### Declaration and selection rules
Imports are opt-in. With `Options.Imports` enabled, a document may use its root
`configkit` mapping for `imports` and `sequences` controls. Ordinary `Load` may use
`sequences` without enabling imports, but an `imports` field still requires
@@ -1184,12 +1637,6 @@ reads them as strings. Null, scalar destinations, and non-string members are
key below the root is invalid. With imports disabled, root imports and nested
`configkit` keys remain `import.disabled`.
The directive key is `configkit`, not the `_configkit` spelling used in v0.1.0.
There is no legacy alias: `_configkit` is ordinary application data, subject to
normal schema validation, and does not trigger imports. When upgrading, rename
directive keys in root files and imported fragments, and rename any application
fields previously named `configkit` to avoid the newly reserved key.
Path declarations use slash-separated relative paths. Absolute paths and network
URLs are invalid. Each path resolves from its declaring file's canonical directory.
An empty `path: []` selects nothing; it succeeds even when `required: true` because
@@ -1238,6 +1685,17 @@ Sequence identities can be supplied either by these YAML declarations or by
application registrations. The [strict composition contract](#strict-document-composition)
defines their agreement, operation, and lifetime rules.
### Ordering and inspection
Reading sources and combining their settings are separate steps. Source IDs
follow the reading order: declarations in the order written, with selected
paths in lexical order. Combination waits for dependencies. If `main.yml`
imports `extra.yml`, `extra.yml` is ready before the body of `main.yml`.
Among all ready contributions, ConfigKit uses its stable source-key ordering;
independent branches can therefore interleave. This ordering determines list
assembly and reporting, but never allows conflicting peer values to overwrite
one another.
Acquisition visits declarations in authored order and each declaration's selected
paths in namespace-path lexical order, preserving source-record and version-callback
order. Ordinary mapping contributions in a main tier are assembled dependency-first:
@@ -1306,30 +1764,31 @@ Physical paths continue to resolve from the file that declares each import.
An actual empty mapping contributes an empty item, while an unmatched declaration
with default or explicit `required: false` contributes nothing.
For host files, the configuration root defaults to the selected file's canonical
directory. A host file may supply `FilesystemContext.Root` to widen that root but
may not supply `BaseDir` or a native `FS`. Byte, reader, and mapping inputs with host context
must provide both `BaseDir` and `Root` as absolute existing directories.
For `Filesystem.File`, the default root is `.` in its namespace and the base is
the selected file's directory. An explicit context must use the same `Filesystem`
namespace and may set `Root` to contain the file and its base; it may not supply
`BaseDir`. Byte, reader, and mapping inputs with native context provide both `BaseDir` and
`Root` as slash-relative `fs.ValidPath` directories in `FS` (`.` means its root),
with the base inside the root. Context paths are checked even when imports and
overlays are inactive. Diagnostic labels, including URI-looking labels, never
imply filesystem paths. Nonregular targets are rejected before opening to avoid blocking on FIFOs;
post-open regular-file checks remain in place under the stable-input assumption.
ConfigKit canonicalizes symlinks,
checks containment with relative paths, and opens the canonical root-relative target
through a library-owned root handle. Contained relative or absolute symlink targets
are accepted; escapes are rejected. Canonical paths define identity, so hard links at
distinct paths remain distinct sources. Native filesystems use slash-path
operations. Where the supplied `fs.FS` implements `fs.ReadLinkFS`, ConfigKit uses
that capability to inspect and resolve discovered links; otherwise it follows
the filesystem's ordinary `fs.FS` behavior. ConfigKit does not require every
native filesystem to expose link inspection, and native filesystem behavior is
not a general sandbox guarantee. Imports stay in their selected namespace and
root.
For host files, the configuration root defaults to the selected file's
canonical directory. A host file may supply `FilesystemContext.Root` to widen
that root but may not supply `BaseDir` or a native `FS`. Byte, reader, and
mapping inputs with host context must provide both `BaseDir` and `Root` as
absolute existing directories. For `Filesystem.File`, the default root is `.`
in its namespace and the base is the selected file's directory. An explicit
context must use the same `Filesystem` namespace and may set `Root` to contain
the file and its base; it may not supply `BaseDir`. Byte, reader, and mapping
inputs with native context provide both `BaseDir` and `Root` as slash-relative
`fs.ValidPath` directories in `FS` (`.` means its root), with the base inside
the root. Context paths are checked even when imports and overlays are
inactive. Diagnostic labels, including URI-looking labels, never imply
filesystem paths. Nonregular targets are rejected before opening to avoid
blocking on FIFOs; post-open regular-file checks remain in place under the
stable-input assumption. ConfigKit canonicalizes symlinks, checks containment
with relative paths, and opens the canonical root-relative target through a
library-owned root handle. Contained relative or absolute symlink targets are
accepted; escapes are rejected. Canonical paths define identity, so hard links
at distinct paths remain distinct sources. Native filesystems use slash-path
operations. Where the supplied `fs.FS` implements `fs.ReadLinkFS`, ConfigKit
uses that capability to inspect and resolve discovered links; otherwise it
follows the filesystem's ordinary `fs.FS` behavior. ConfigKit does not require
every native filesystem to expose link inspection, and native filesystem
behavior is not a general sandbox guarantee. Imports stay in their selected
namespace and root.
An active-stack repeat is `import.cycle` and reports the complete cycle's import
sites. A previously visited canonical path is `import.repeated` and reports both
@@ -1339,9 +1798,39 @@ application's schema-version path or replace one of its ancestors; the version m
come from the selected source. No directory is scanned implicitly, including one
named `conf.d`, and ConfigKit performs no remote fetching.
### Earlier directive spelling
The directive key is `configkit`, not the `_configkit` spelling used in v0.1.0.
There is no legacy alias: `_configkit` is ordinary application data, subject to
normal schema validation, and does not trigger imports. When upgrading, rename
directive keys in root files and imported fragments, and rename any application
fields previously named `configkit` to avoid the newly reserved key.
## YAML documents
The private YAML boundary currently accepts exactly one nonempty document whose root
A configuration document is a YAML mapping at the top level. For example, these
values can be loaded into Go fields of the matching types:
```yaml
enabled: false
wait: 5s
title: "yes"
```
```go
type Settings struct {
Enabled bool `yaml:"enabled"`
Wait time.Duration `yaml:"wait"`
Title string `yaml:"title"`
}
```
Here `5s` is a string that decodes into `time.Duration`; ordinary text `yes`
needs quotes because the unquoted legacy boolean spelling is rejected. Use
lowercase `true` and `false` for booleans. The following reference lists the
complete accepted and rejected forms.
ConfigKit accepts exactly one nonempty YAML document whose root
is a mapping. Comments and block or flow collections are supported. It rejects:
- empty input or an empty document;
@@ -1386,9 +1875,8 @@ keys that otherwise classify as a boolean, number, or null.
## Application schema binding
ConfigKit builds a private reflection descriptor graph for each schema operation;
it keeps no global mutable descriptor cache. A configuration root must be a struct
or string-keyed map. Nested supported shapes are:
Your Go configuration type defines the accepted fields and value types. The
root must be a struct or a map with string keys. Nested supported shapes are:
- booleans, strings, signed and unsigned integers, and finite floating-point types;
- `time.Duration`, typed `Secret` references, and paired
@@ -1396,6 +1884,8 @@ or string-keyed map. Nested supported shapes are:
- pointers, structs, string-keyed maps, slices, fixed arrays, and `any`; and
- recursive type definitions whose eventual application values are acyclic.
### Advanced type and field rules
Named scalar types retain their Go identity. `uintptr`, complex numbers, functions,
channels, unsafe pointers, non-string map keys, and nonempty interfaces are not
configuration types. A custom text scalar must provide both directions on its value
@@ -1415,16 +1905,16 @@ ConfigKit field tag and marks its bound field or container for secret classifica
and redacted inspection.
Schema path traversal uses these same bindings. Explicit key segments address
mappings, explicit index segments address sequences, and parsed numeric pointer
tokens become indices only in a sequence. Sequence indices must use canonical
nonnegative decimal spelling: no signs or leading zeroes except `0` itself.
Tokens such as `+0` and `-0` remain literal keys in mappings but do not identify
sequence elements. Fixed-array indices are bounds checked. Dynamic `any` and inline-map entries
remain schema-valid for application-defined descendants, but dynamic values cannot
introduce typed secret references.
mappings, explicit index segments address sequences, and parsed numeric
pointer tokens become indices only in a sequence. Sequence indices must use
canonical nonnegative decimal spelling: no signs or leading zeroes except `0`
itself. Tokens such as `+0` and `-0` remain literal keys in mappings but do
not identify sequence elements. Fixed-array indices are bounds checked.
Dynamic `any` and inline-map entries remain schema-valid for
application-defined descendants, but dynamic values cannot introduce typed
secret references.
Descriptors are reused by typed decoding, projection, and cloning so those operations
cannot acquire different field bindings.
Decoding, inspection output, and copied values use the same field bindings.
### Canonical module options
@@ -1446,8 +1936,24 @@ examples.
## Defaults and typed decoding
The private decoder starts from an independently owned clone of the application's
typed defaults and applies only explicitly present input nodes. Defaults therefore
Defaults are starting values for omitted settings. Suppose your defaults are
`enabled: true`, `workers: 2`, `names: [primary]`, and a non-nil pointer value
for `optional`. The following independent examples illustrate the effect of input:
| Input | Result |
| --- | --- |
| `enabled` omitted | Retain `true`; it is not explicit input. |
| `enabled: false` | Use `false` and mark the path explicit. |
| `workers: 0` | Use `0` and mark the path explicit. |
| `names: []` | Use an empty list instead of the default list. |
| `optional: null` | Use nil and suppress the pointer default. |
Null requires a field type that allows nil. For example, `enabled: null` fails
for a non-pointer boolean; it does not mean "use the default." The full table
below also covers partial mappings, new entries, and pointer objects.
Decoding starts from a copy of the application's typed defaults and applies only
settings that input supplied. Defaults therefore
never become explicit, while equal-valued input remains explicit. Successful decode
produces detached typed state, baseline and final canonical trees, an input-presence
index, and default/input origins for the later public loading lifecycle.
@@ -1492,13 +1998,30 @@ conversion API.
## Typed projection and ownership
ConfigKit can internally project a schema-valid application value directly into its
owned canonical tree without YAML serialization. Projection includes every bound
field, including ordinary zero values; `omitempty` has no effect. Struct fields keep
binding order, dynamic map keys sort by decoded string, and sequences keep their
order. Integers retain exact base-10 values, finite floats use their destination
precision, durations use `time.Duration.String`, and paired text scalars use their
canonical marshaled text.
ConfigKit uses a consistent representation of typed values for inspection and
comparison, called a projection. It includes every field bound by the schema,
even ordinary zero values. `omitempty` does not remove these fields from
output. For example, a nil slice displays as `null`, while a non-nil empty
slice displays as `[]`. Creating this representation does not serialize and
reparse YAML. Struct fields keep binding order, dynamic map keys sort by
decoded string, and sequences keep their order. Integers retain exact base-10
values, finite floats use their destination precision, durations use
`time.Duration.String`, and paired text scalars use their canonical marshaled
text.
For example, loading `{}` into this type leaves `Workers` at zero and ignores
`Internal`:
```go
type OutputSettings struct {
Workers int `yaml:"workers,omitempty"`
Internal string `yaml:"-"`
}
```
The effective YAML includes `workers: 0`; `omitempty` does not hide it. There is
no `Internal` field in that output. Ignored fields are zeroed in the returned
configuration, including when the defaults factory supplied a value for them.
Nil pointers, maps, slices, and dynamic values project as null. Non-nil empty maps
and slices remain empty containers, preserving the distinction. Recursive acyclic
@@ -1523,6 +2046,15 @@ and rendering. They are not a public generic conversion or cloning API.
## Safe failures
A failed load returns a `LoadError` with structured diagnostics. Inspect the
code to identify the kind of problem, the path to identify the setting, and the
source location to find it in the input. For example, misspelling a declared
`enabled` field as `enabeld` produces `decode.unknown_field` at `/enabeld`, with
available source coordinates. Human-readable messages explain the problem;
application code can compare the diagnostic code instead of parsing that text.
See [WriteDiagnostics](../diagnostic.go) and the
[maintained error-handling example](../example_test.go).
Source failures use stable `source.*` codes and YAML failures use stable `yaml.*`
codes declared by the root package. Diagnostics contain safe classifications,
semantic paths when applicable, and available source coordinates. Raw parser,