diff --git a/README.md b/README.md index 3c45356..295df2c 100644 --- a/README.md +++ b/README.md @@ -1,56 +1,14 @@ # ConfigKit -An opinionated Go library for applications that use YAML configuration files. +ConfigKit loads YAML into your application's Go configuration type, combines the +inputs you choose, and lets you inspect where the final values came from. You +supply the types, defaults, and application rules; ConfigKit handles loading, +combination, strict decoding, and inspection. Linux is release-validated; macOS support is best-effort. See the -[platform support policy](docs/policy/release.md#platform-support) for details. +[platform support policy](docs/policy/release.md#platform-support). -ConfigKit's simple `Load` accepts one strict YAML source (or explicit -defaults-only input). `LoadLayered` additionally combines explicit lower-priority -source groups with an independently selected main input. Sources can be host -files, native `fs.FS` files, labeled bytes or readers, or snapshotted mappings. -Enabled explicit imports compose strictly within their group; precedence -between groups combines mappings recursively. Both entry points return a complete, -validated configuration with detached metadata and safe diagnostics. Successful -results and load errors also provide passive `Source(id)` lookups alongside source -enumeration. -Applications may perform audited defaulting and normalization through a resolver, -then report domain errors and warnings through read-only semantic validation. Secret -tags and a path predicate drive deterministic redacted effective YAML. Typed secret -references resolve environment names before validation, and explicit -secrets-directory bootstrap is available as a separate startup operation. Explicit -CLI override records apply after inheritance and typed decoding, before application -resolution. Value explanation is available through -detached structured records and deterministic redacted output. Built-in -`extends` relationships let a mapping inherit from a named sibling during the -same load, retaining explicit presence and source/derivation history. The root -`Diff` API compares frozen effective values from two successful same-type loads; -the [configuration contract](docs/config.md#effective-configuration-diffs) and -[consumer guide](docs/consumers/README.md#effective-configuration-diffs) describe -its safe changed-region records and maintained workflows. - -Sources can also be explicitly placed as sequence items. Configuration -declarations and application registrations can give selected sequences scalar -identities and choose how they combine across precedence and inheritance. -Strict peers combine distinct records, while higher-priority collections can -replace, append, or upsert whole records. See the -[sequence contract](docs/config.md#strict-document-composition), -[maintained catalog examples](docs/consumers/README.md#identified-prompt-sequences), -and [ADR 0017](docs/adr/0017-collection-sequence-policy.md). - -Imports are also explicit: a source may name files, search declared directories, -or select files with path globs. Applications can set load-wide import defaults -while keeping declaration-specific exceptions. Files are acquired in authored -declaration order; dependency-ready composition uses a shared stable scheduler. -ConfigKit does not search conventional application paths implicitly. See the -[import contract](docs/config.md#explicit-imports), [consumer examples](docs/consumers/README.md), -and [ADR 0015](docs/adr/0015-shared-dependency-ready-ordering.md) and -[ADR 0016](docs/adr/0016-independent-import-controls.md). - -Main-file discovery is explicit too: applications choose and order `Search` -descriptions and pass `SearchConfig` to the existing load APIs. See the -[discovery contract](docs/config.md#configuration-discovery), [consumer guide](docs/consumers/README.md#explicit-main-file-discovery), -and [ADR 0019](docs/adr/0019-explicit-configuration-discovery.md). +## Load a configuration file ```go import "go.maximumdirect.net/configkit" @@ -72,48 +30,54 @@ func ReadConfig(path string) (Config, error) { Existing consumers should follow the [module-path migration guidance](docs/consumers/README.md#vanity-module-path-migration). +## Add the features you need + +Start with one file. Optional features use the same loading and inspection model: + +- [Choose inputs](docs/consumers/README.md#choosing-a-loading-entry-point): read + files, an application-owned `fs.FS`, bytes, a reader, or a Go mapping. Use + explicit search descriptions when the application should look in several places. +- [Supply lower-priority settings](docs/consumers/README.md#native-filesystem-and-layered-input): + use `LoadLayered` for bundled defaults that application configuration can override. +- [Combine configuration](docs/config.md#explicit-imports): add declared imports, + select one overlay, or pass values explicitly set by your CLI parser. +- [Reuse settings](docs/config.md#sibling-inheritance): let one named entry inherit + from a sibling. [Identified lists](docs/config.md#strict-document-composition) + also support record inheritance and replace, append, or upsert operations. +- [Apply application rules](docs/config.md#recorded-application-resolution): derive + or normalize values in a resolver, then check the final configuration in a validator. +- [Handle credentials](docs/config.md#typed-secret-references): use environment-name + references and choose which configuration fields to hide in inspection output. +- [Inspect a load](docs/config.md#layered-metadata): examine sources, presence, + history, safe YAML output, or [changes between two loads](docs/config.md#effective-configuration-diffs). + +The [consumer guide](docs/consumers/README.md) walks through these tasks and links +to maintained examples. The [configuration reference](docs/config.md) defines the +exact YAML, combination, and lifecycle rules. + ## Design philosophy -ConfigKit favors one coherent, predictable configuration model over flexibility -for reproducing each consumer's conventions. Its goal is to let applications -load once, inspect where final values came from, and distinguish explicit values -from omissions with minimal configuration bookkeeping. Applications retain their -domain rules and execution responsibilities. +ConfigKit favors consistent, predictable rules across applications. A load returns +the final configuration and metadata that distinguish supplied values such as +`false` and `0` from omissions. Applications keep their domain rules and runtime +initialization. -Loading is synchronous and complete on return. Applications normally load once -near startup and then use the final configuration and metadata; ConfigKit stays -inactive until explicitly called again. Applications may initiate additional -loads as needed. See the [loading lifecycle](docs/config.md#completion-and-resource-lifetime) -for completion and ownership details. +Within one layer, independent contributions combine strictly: conflicting +values are errors. Across layers, higher-priority settings can replace +lower-priority settings, while mappings combine by key. Identified lists have +their declared combination rules. See the [worked combination +examples](docs/config.md#strict-document-composition) for the details and +exceptions. -Within a configuration tier, independent contributions compose strictly: shared -mappings combine, while overlapping non-mapping definitions are errors. Between -precedence tiers, mappings combine recursively, retaining omitted lower-priority -children; other values replace lower-priority values. The main document and its -imports form one strictly composed tier. Explicit underlay groups sit below it -and above typed defaults; a selected overlay applies before inheritance, while -explicit CLI overrides apply after decoding and before application resolution. -The [configuration contract](docs/config.md#strict-document-composition) owns -the detailed combination rules and input-specific constraints. Declared or -registered collections combine distinct identified records within strict tiers -and use their selected operation at precedence boundaries; other sequences keep -their strict-conflict and whole-value-replacement behavior. +Loading finishes before the call returns. ConfigKit then stays inactive until you +call it again; inspection uses the saved result. See +[completion and resource lifetime](docs/config.md#completion-and-resource-lifetime). -The [architecture goals](docs/policy/architecture.md#success-criteria) explain -these priorities and how genuine new needs can justify extending the shared -model. The [future feature list](docs/roadmap/future.md) preserves candidates for -consideration in future releases. - -The [configuration contract](docs/config.md) describes implemented behavior, -including [input destinations](docs/config.md#input-destinations). The -[consumer adoption guide](docs/consumers/README.md) provides maintained simple, -layered, mounted-collection, and identified-prompt examples alongside the -migration shapes. Input placement decisions are recorded in -[ADR 0011](docs/adr/0011-input-destinations.md) and sequence placement and -composition in [ADR 0014](docs/adr/0014-sequence-placement-and-composition.md). -See the [development guide](docs/development.md) for repository orientation and -the [release procedure](docs/release.md) for maintainer-only candidate and -publication steps. +The [architecture goals](docs/policy/architecture.md#success-criteria) and +[ADRs](docs/adr) explain design decisions, including shared import and inheritance +ordering. [Future work](docs/roadmap/future.md) lists possible extensions. +Contributors can start with the [development guide](docs/development.md); +release maintainers use the [release procedure](docs/release.md). ConfigKit code is licensed under the [LGPL v3.0 or later](COPYRIGHT). diff --git a/diff.go b/diff.go index 1a392f3..a704edb 100644 --- a/diff.go +++ b/diff.go @@ -2,14 +2,13 @@ package configkit import "go.maximumdirect.net/configkit/internal/engine" -// Difference describes one canonical changed region between two successful -// loads. Left is the earlier value and Right is the later value. Each snapshot -// is detached; Present distinguishes absence from an explicit null, and -// Redacted reports whether combined sensitivity affected that side. Mappings -// are compared recursively, while sequences are reported as atomic values. -// Records are ordered by canonical JSON pointer. HiddenChanged indicates that -// a value concealed by combined sensitivity changed, without revealing which -// concealed member changed. +// Difference describes one changed value or subtree between two successful loads. +// Left is the earlier value and Right is the later value. Snapshots are copies; +// Present distinguishes absence from explicit null. Redacted reports hiding +// based on sensitivity from either load. Mappings compare by key; a changed +// sequence is reported as a whole value. Records are ordered by canonical JSON +// pointer. HiddenChanged reports an actual hidden change, without identifying +// the hidden member. Redaction alone does not set it. type Difference struct { Path Path Left ValueSnapshot @@ -17,8 +16,9 @@ type Difference struct { HiddenChanged bool } -// Diff compares the frozen effective values of two successful loads of the -// same configuration type. It does not reread sources, environment variables, +// Diff compares the saved final values of two successful loads of the same +// configuration type, independently of later changes to Result.Config. +// It does not reread sources, environment variables, // or rerun defaults or resolvers. Equal values produce no records even when // their sources, defaults, or resolution histories differ. Credential contents // are not compared: typed secret references compare by reference name only. diff --git a/doc.go b/doc.go index d150bee..6be733b 100644 --- a/doc.go +++ b/doc.go @@ -1,9 +1,17 @@ -// Package configkit provides deterministic, application-owned configuration -// loading for Go programs. +// Package configkit loads YAML into an application's Go configuration type and +// lets the application inspect where the final values came from. // -// Load supports file, byte, reader, and explicit no-input sources with strict YAML, -// typed defaults, imports, overlays, CLI overrides, schema-version checks, detached -// metadata, recorded application resolution, typed secret references, safe effective -// rendering, read-only semantic validation, and structured load errors. The -// configuration contract is documented in docs/config.md. +// Start with Load and a File, Bytes, Reader, Mapping, or Filesystem.File source. +// Use NoSource for deliberately absent input or SearchConfig for an explicitly +// chosen search. Applications supply types, defaults, version rules, and callbacks +// for deriving and validating values. LoadLayered adds lower-priority input groups. +// +// A successful load returns the final configuration and saved inspection state. +// Metadata tracks supplied values, including false and zero; explanations retain +// safe earlier values. WriteEffective and Diff use the saved state even if the +// application later changes Result.Config. Secret references resolve credentials +// during loading, while inspection follows the configured hiding rules. +// +// The consumer guide is in docs/consumers/README.md. The exact YAML, combination, +// lifecycle, and ownership rules are documented in docs/config.md. package configkit diff --git a/docs/config.md b/docs/config.md index 7d62f80..3750fb6 100644 --- a/docs/config.md +++ b/docs/config.md @@ -1,27 +1,66 @@ # Configuration Input Contract -This document owns ConfigKit's currently implemented source-input, YAML syntax, -schema, typed-decoding, loading, metadata, redaction, rendering, and ownership -contracts. The parser, owned document representation, and decoder remain private -behind the public loading lifecycle. +This reference explains the YAML ConfigKit accepts, how inputs combine, and what +an application receives from a load. For a first working example, start with the +[README](../README.md#load-a-configuration-file) or the +[consumer guide](consumers/README.md). + +Configuration authors can start with [YAML values](#scalars), +[combination rules](#strict-document-composition), [imports](#explicit-imports), +and [inheritance](#sibling-inheritance). Go API users can start with +[sources](#sources), [defaults and decoding](#defaults-and-typed-decoding), +[application callbacks](#recorded-application-resolution), and +[inspection](#layered-metadata). [Credentials](#typed-secret-references) and +[secrets-file startup](#explicit-secrets-directory-bootstrap) have separate roles. + +## Terms used in this reference + +A **mapping** is a group of named YAML values; a **sequence** is a list. A +**scalar** is a single value such as a string, number, or boolean. For example: + +```yaml +service: # mapping + enabled: false # scalar + names: [primary, spare] # sequence +``` + +A **source** supplies an input document, such as a file or an in-memory mapping. +A **tier** is a group of inputs at the same priority. **Precedence** describes +which tier takes priority when several tiers supply a setting. Inputs within +one tier follow strict combination rules. + +A **configuration path** names a value inside the configuration, rather than a +file on disk. For example, `/service/names/0` identifies the first name in the +YAML above. Go callers can construct it with `Keys("service", "names").Index(0)`. +Its text uses JSON Pointer escaping: `~0` represents `~` and `~1` represents `/` +inside a key. Explicit mapping keys and sequence indexes remain distinct even +when their pointer text looks the same. + +**Metadata** describes a value's presence and **origin**: where it came from, +including application processing that changed it. A returned **detached** record +is a copy you can change without changing ConfigKit's stored record. A **snapshot** +is saved configuration state used for inspection after loading finishes. +Borrowed filesystems have their own [resource lifetime](#source-resource-contexts). ## Sources -ConfigKit exposes opaque sources through these constructors: +A source describes the input to pass to `Load` or `LoadLayered`. Choose the +constructor that matches where your configuration comes from: | Constructor | Current contract | | --- | --- | -| `File(path)` | Requires a nonempty path, opens a regular file for each consumption, and closes only the library-owned file handle. Missing, inaccessible, unreadable, directory, and other nonregular inputs fail. | +| `File(path)` | Requires a nonempty path, opens a regular file for each load, and closes only the library-owned file handle. Missing, inaccessible, unreadable, directory, and other nonregular inputs fail. | | `SearchConfig(application, search, options...)` | Creates an explicit, reusable search source. Loading selects one host file or follows the absent-input lifecycle when an optional search is exhausted. | | `NewFilesystem(label, filesystem).File(name)` | Borrows a caller-owned `fs.FS` under a safe nonempty display label. `name` must be a non-root slash-relative `fs.ValidPath`. Each load reopens the selected regular file; copies of the `Filesystem` preserve namespace identity. Separate constructors create separate namespaces even for the same label or filesystem. | | `Bytes(label, data)` | Requires a nonempty, non-secret diagnostic label and snapshots bytes during construction. Later caller mutation cannot affect the source. | -| `Reader(label, input)` | Requires a nonempty, non-secret label and non-nil reader. The reader is claimed once across all copies of the source, consumed but never closed, and cannot be reused. | -| `Mapping(label, values)` | Requires a safe nonempty label and a non-nil root `map[string]any`. Snapshots supported built-in values into an owned mapping document at construction; `{}` is valid. Constructor conversion failures become safe deferred diagnostics. | +| `Reader(label, input)` | Requires a nonempty, non-secret label and non-nil reader. Copies of the source share one read attempt. ConfigKit never closes the reader; you remain responsible for closing it if needed. | +| `Mapping(label, values)` | Requires a safe nonempty label and a non-nil root `map[string]any`. Snapshots supported built-in values into an owned mapping document at construction; `{}` is valid. Conversion failures are reported when loading uses the source. | | `NoSource()` | Selects defaults-only loading, creates no source record, and marks no paths explicit. | -The zero `Source` is invalid. Constructor argument failures are retained inside the -opaque source and become safe structured diagnostics when the source is consumed. -File and filesystem sources can observe changes between consumptions; byte and +The zero `Source` is invalid. Constructors store argument failures so loading +can report them as structured diagnostics. A source's private representation is +not part of the API. +File and filesystem sources can observe changes between loads; byte and mapping sources cannot. Mapping accepts nil, built-in string, boolean, signed or unsigned integer, finite float, `[]any`, and `map[string]any` values recursively. Typed nil maps and slices represent explicit null; empty collections remain empty. @@ -51,12 +90,22 @@ group contribute no candidates. An explicitly supplied empty path or filename is invalid. Slugs and custom filenames must be usable single host filename components; ConfigKit preserves their case and spelling. -Search expansion is ordered concatenation. For each directory search, ConfigKit -tries its filename groups and names before advancing to the next directory. -When multiple directories use the same filename list, candidates follow the -directory-first Cartesian product: every filename in one directory is tried -before the next directory. Explicit file candidates stay at their declared -positions among directory searches. +Search descriptions are tried in the order supplied. Within a directory search, +ConfigKit tries every selected filename before moving to the next directory. +For example, two directories using `NamedFilenames` for application `demo` are +searched in this order: + +```text +first/demo.yml +first/demo.yaml +second/demo.yml +second/demo.yaml +``` + +Explicit file candidates keep their declared positions among directory searches. +If no candidate exists, an optional search continues as absent input and a +required search fails. Once a file is selected, a failure to load it is an error; +ConfigKit does not continue looking for another configuration file. The presets expand to these ordered directories and filename groups: @@ -102,6 +151,8 @@ candidates create no records. `SearchConfig` can be used as the main source of ordinary or layered loading, including an underlay input, and does not add a new precedence tier. +### Input limits + Each source is limited to 8 MiB. A load using imports or an active overlay is limited to 64 MiB across all accepted sources, 1,024 accepted sources, 64 active import edges, and 1,000,000 input nodes in aggregate. Glob expansion separately permits @@ -113,15 +164,19 @@ overrides, and the converted tree uses the same node and depth bounds. ## Single-source loading -`Load[T](source, options...)` accepts zero or one `Options[T]`. The configuration -root `T` must satisfy the application schema contract below. With no options it -starts from the Go zero value. A `Definition[T].Defaults` factory, when supplied, -runs exactly once after successful source parsing, schema construction, optional -schema-version checking, and selected-overlay loading. Its result is detached before -input is applied. Resolver-provided mutable values are detached before secret lookup -and validation, and a final detached copy prevents equal-valued aliases introduced -by validators from escaping into the returned configuration. The publication copy -preserves already validated secret state without rereading the environment. +`Load[T](source, options...)` returns a configuration of your Go type `T` plus +metadata for inspection. It accepts zero or one `Options[T]`. The root type must +follow the [schema rules](#application-schema-binding). With no options, loading +starts from the Go zero value. To supply defaults, provide a +`Definition[T].Defaults` function that returns a fresh configuration. + +That function runs exactly once after successful source parsing, schema checks, +optional version checking, and selected-overlay loading. ConfigKit copies its +result before applying input. Mutable values supplied by a resolver are also +copied before secret lookup and validation. A final copy prevents even +equal-valued replacements made by validators from leaving shared mutable storage +in the returned configuration. It preserves validated secret state without +reading credentials again. The current lifecycle is: @@ -136,11 +191,12 @@ validate options → parse the selected source → extract source controls and → snapshot configuration and metadata ``` -`File`, `Filesystem.File`, `Bytes`, `Reader`, and `Mapping` each contribute the first base source with deterministic -ID `source-1`; enabled imports continue in depth-first order as `source-2` onward. -`NoSource` skips parsing, supplies an absent version to a configured version checker, -contributes no source, and continues through defaults, decode, and projection. A -missing file remains an error; it is never interpreted as no input. +`File`, `Filesystem.File`, `Bytes`, `Reader`, and `Mapping` each contribute +the first base source with deterministic ID `source-1`; enabled imports +continue in depth-first order as `source-2` onward. `NoSource` skips parsing, +supplies an absent version to a configured version checker, contributes no +source, and continues through defaults, decode, and projection. A missing file +remains an error; it is never interpreted as no input. `Options` accepts the application definition, ordinary-source `Destination` and `PlacementMode`, explicit imports, one selected overlay, filesystem @@ -161,16 +217,23 @@ when imports are disabled. An active ordinary overlay requires this explicit context. `NoSource` accepts overrides and remains absent rather than contributing an empty document. -Every source, options, schema, version, defaults, decode, resolution, or validation -failure returns a nil result and a `*LoadError`. Successful results contain caller-owned `Config`, detached -`Metadata`, a defensive source list, and defensive nonfatal diagnostics. Each source -record's `Kind` identifies file, filesystem, bytes, reader, mapping, or CLI acquisition -independently of its precedence `Layer`. Host file paths remain canonical; -filesystem paths are slash-relative within their namespace; synthetic inputs have -no path. Mutating -the returned configuration or inspection records cannot alter ConfigKit's private -snapshot. ConfigKit performs no logging. `Result.WriteEffective` exposes only the -frozen redacted projection. +### Results and failures + +You can change `Result.Config` after loading. Inspection methods continue to +describe the configuration as it was when the load finished. Returned source, +diagnostic, and metadata records are copies; changing them cannot change later +queries. Separate loads return independently owned configuration and records. + +Every source, options, schema, version, defaults, decode, resolution, or +validation failure returns a nil result and a `*LoadError`. Successful results +contain caller-owned `Config`, detached `Metadata`, a defensive source list, +and defensive nonfatal diagnostics. Each source record's `Kind` identifies +file, filesystem, bytes, reader, mapping, or CLI acquisition independently of +its precedence `Layer`. Host file paths remain canonical; filesystem paths are +slash-relative within their namespace; synthetic inputs have no path. Mutating +the returned configuration or inspection records cannot alter ConfigKit's +private snapshot. ConfigKit performs no logging. `Result.WriteEffective` +exposes only the frozen redacted projection. `Result.Source(id)` returns the detached record for an acquired source already listed by `Result.Sources()`. Source IDs are local to one load; empty and unknown @@ -186,7 +249,7 @@ or unknown IDs return false. ### Completion and resource lifetime -`Load` is synchronous and complete on return. Success means input assembly, +`Load` finishes its work before returning. Success means input assembly, defaulting, typed decoding, configured application resolution, secret lookup, validation, and configuration/metadata snapshotting have completed. A reported load failure returns diagnostics and a nil result, as described above. @@ -204,9 +267,9 @@ when explicitly called and use the retained snapshot. Applications normally load once near startup, but may explicitly load again using sources that satisfy their documented reuse rules. Additional loads produce independently owned results. -Completion is not a transaction over external state. Files and environment -variables are observed during processing rather than as one simultaneous external -snapshot. ConfigKit does not roll back application callback effects or manage work +Files and environment variables are read during processing; they are not all +captured at one instant. If your callback changes something outside the working +configuration, ConfigKit does not undo that effect when loading fails or manage work or resources created by application callbacks. The explicit secrets-directory bootstrap remains a separate operation invoked by the application. @@ -221,6 +284,23 @@ nothing. Underlay groups are listed from lowest to highest priority. Repeating a file within one group remains an import/repetition error; the same reusable source may be selected again in another tier. +### Priority and processing order + +The input priority order is shown below, from lowest to highest. Mappings combine +by key across these boundaries; identified lists use their collection operation. + +```text +typed defaults → underlay groups (first to last) → main + imports → overlay → CLI +``` + +This is a priority order, rather than a schedule of processing steps. Defaults +have low priority but are applied during typed decoding, after source assembly +and inheritance. Application resolution runs after CLI assignment and can make +recorded changes. Secret lookup and validation then assess the resulting values. +See the [load timeline](#single-source-loading) for the processing sequence. + +### Source order and version checks + ConfigKit acquires the main input first and constructs the schema. It then acquires and strictly composes underlay groups from lowest to highest priority, passing each group's accumulated lower collection policy into the next group. @@ -260,7 +340,42 @@ is the sequence container. After assembly assigns an index, nested source destinations are projected through that item. Physical source locations and resource contexts continue to identify the declaring source. -#### Input destinations +### Input destinations + +A destination controls where settings appear in the configuration. It is +independent of where the source file lives on disk. Consider this file layout: + +```text +/etc/demo/ + config.yml + fragments/ + prompt.yml + welcome.txt +``` + +With imports enabled, `config.yml` can contain: + +```yaml +configkit: + imports: + - path: fragments/prompt.yml + destination: [prompts, welcome] +``` + +If `prompt.yml` contains `template: welcome.txt`, the assembled input is: + +```yaml +prompts: + welcome: + template: welcome.txt +``` + +The import filename is relative to `config.yml`; the destination places its +contents beneath `/prompts/welcome`. A later application read of `welcome.txt` +can use the imported source's context, as described under +[resource contexts](#source-resource-contexts). + +#### Destination rules and inspection evidence `Input.Destination` places the source's application mapping beneath a path relative to the configuration root. Its zero or empty value means root, so @@ -303,15 +418,52 @@ acquisition identity, physical source locations, or passive resource contexts. Introduced ancestor mappings use the contributing document's root location as a source fallback; authored content retains its original source coordinates. -The maintained [layered consumer example](consumers/README.md#native-filesystem-and-layered-input) -shows a main host file, two underlay tiers, embedded defaults, a mapping source, -and explicit resource access without a second configuration pass. -The accepted rationale for input placement and its relative-import extension -is recorded in [ADR 0011](adr/0011-input-destinations.md) and -[ADR 0012](adr/0012-relative-import-destinations.md). +The maintained [layered consumer +example](consumers/README.md#native-filesystem-and-layered-input) shows a main +host file, two underlay tiers, embedded defaults, a mapping source, and +explicit resource access without a second configuration pass. The accepted +rationale for input placement and its relative-import extension is recorded in +[ADR 0011](adr/0011-input-destinations.md) and [ADR +0012](adr/0012-relative-import-destinations.md). ### Sibling inheritance +Inheritance lets one entry reuse another entry's settings. A sibling is another +entry in the same enclosing mapping. For example: + +```yaml +presets: + base: + width: 10 + length: 20 + custom: + extends: base + width: 30 +``` + +The resulting `custom` has `width: 30` and `length: 20`; `base` remains an +ordinary entry. ConfigKit resolves this after combining source layers, so the +child uses the parent's settings that remain after that combination. +`extends: null` cancels a relationship and keeps the child's own settings. + +An identified list uses a mapping reference instead of a sibling name: + +```yaml +configkit: + sequences: + - destination: [presets] + key: id +presets: + - {id: base, width: 10, length: 20} + - {id: custom, extends: {id: base}, width: 30} +``` + +Here the child keeps `id: custom` and receives the parent's other settings. +It has `width: 30` and `length: 20`. Every parent remains subject to ordinary +application decoding and validation. + +#### Lookup, processing order, and limits + Any input mapping may declare `extends: name` to inherit from the mapping named `name` among its immediate siblings. Names are exact keys, including the empty string; lookup does not search ancestor or unrelated scopes. A mapping inside a @@ -390,6 +542,21 @@ usage. ### Source resource contexts +Use a source context when a configuration value names another file, such as a +template. It tells the application which filesystem and directory belonged to +the source that supplied the value. Obtaining the context does not open files. + +For the [placement example](#input-destinations), inspect the origin of +`/prompts/welcome/template` and take its source ID. `result.SourceContext(id)` +provides a filesystem rooted at `/etc/demo` and a `BaseDir` of `fragments`. +Joining that base with `welcome.txt` gives `fragments/welcome.txt`, which the +application can open through the returned `FS`. Close the returned file after +reading it. The maintained [ReadWelcome example](../examples/consumers/layered.go) +shows this workflow. If a value has several contributing sources, the application +must choose the intended source according to its own rules. + +#### Context queries and lifetime + `Result.SourceContext(id)` and callback-scoped `Resolution.SourceContext(id)` return a passive `ResourceContext` for a source with filesystem context, or false for unknown and contextless IDs. The result query is nil-safe; the @@ -423,6 +590,12 @@ a caller-owned filesystem closes. This is not a snapshot or universal sandbox. ### Application schema versions +Your application chooses which version field to check and which identifiers it +accepts. ConfigKit passes the declared value to that checker; it does not assign +meaning to a version number. For a rule at `version`, both `version: 4` and +`version: "4"` supply text `4`. A plain string field can receive that spelling +under the conversion rules below. + `VersionRule` declares a semantic path and non-nil checker. Before defaults or typed decoding, the checker receives a `Version` preserving absent, explicit null, string, or strict decimal-integer input. Strings and integers supply their exact spelling in @@ -455,15 +628,43 @@ successful result. Callback panics remain programming failures. #### Shared source-version checks +By default, version checking concerns the assembled main input. Enable a shared +source-version mode when the application needs to check declarations in individual +main, imported, underlay, and selected-overlay documents with the same rule. +With a configured usable version rule, the main cases are: + +| Situation | `MainOnly` (default) | `IfPresent` | `Required` | +| --- | --- | --- | --- | +| Header absent in an actual main document | Call the checker with an absent version. | Call the checker with an absent version. | Reject before calling the checker. | +| Header absent in another participating document | No source-local check; existing version-path protection applies. | Skip its checker call. | Reject the document. | +| Main header is explicit null | Call the checker with `Null` true. | Call the checker with `Null` true. | Call the checker with `Null` true. | +| Main input is `NoSource()` | Keep the absent-main checker call. | Keep the absent-main checker call. | Keep the absent-main checker call. | + +The short column labels stand for the `SourceVersionMode` constants below. +In enabled modes, an explicit null in another participating document also reaches +the checker. The application's rule decides whether null is accepted. + +For example, a main document with `version: 4` placed at destination `[service]` +is checked at its own `/version` path in enabled modes. Its accepted value remains +at `/service/version` in the application configuration, and protection applies +there. Accepted non-main headers are removed before documents combine, so they +cannot overwrite the main header. The maintained +[versioned-service example](../examples/consumers/source_version.go) demonstrates +this placement. Source evidence records what was observed; its existence alone +is not proof that a checker ran or accepted a declaration. + +##### Mode rules and version evidence + `Options.SourceVersionMode` and `LayeredOptions.SourceVersionMode` select how -one `Definition.Version` policy applies across source documents. The zero value -`SourceVersionModeMainOnly` (the zero value) preserves the existing main-only -check at its effective path and existing protected-path behavior. -`SourceVersionModeIfPresent` checks each participating document that declares a -version and permits absent non-main declarations. `SourceVersionModeRequired` rejects a participating -document without a declaration. Enabled modes require a usable `VersionRule` -and checker; an unknown mode or unusable rule is an options error before source -access or callbacks. +one `Definition.Version` policy applies across source documents. The zero +value `SourceVersionModeMainOnly` preserves the existing +main-only check at its effective path and existing protected-path behavior. +`SourceVersionModeIfPresent` checks each participating document that declares +a version and permits absent non-main declarations. +`SourceVersionModeRequired` rejects a participating document without a +declaration. Enabled modes require a usable `VersionRule` and checker; an +unknown mode or unusable rule is an options error before source access or +callbacks. In enabled modes, `Definition.Version.Path` is interpreted in each acquired document's own coordinates, before placement and composition. The main document, @@ -516,7 +717,27 @@ the completed implementation and remediation records in Git history. ### Layered metadata -`Result.Metadata` describes every final canonical path. Queries retain explicit +Use inspection to understand a completed load without reading the files again: + +| Need | API to use | +| --- | --- | +| Check whether input supplied a final value, including false or zero | `Metadata().Lookup` or `IsExplicit` | +| Find the sources that supplied it | The field's `Origin`, then `Source(id)` or `Sources()` | +| Inspect attribution for earlier definitions | `Metadata().History` | +| Inspect earlier values as safe YAML | `Explain` or `WriteExplanation` | +| Show the saved final configuration as safe YAML | `WriteEffective` | +| Compare final values from two loads | `Diff` | + +For example, suppose a typed default sets `workers` to `2`, the main input +supplies `workers: 3`, and an overlay supplies `workers: 4`. The final value is +`4`, it is explicit, and its origin identifies the overlay. History retains the +earlier default and main contributions. `History` returns attribution records; +`Explain` adds their before/after values. Changing `result.Config.Workers` later +does not change any of these saved inspection results. + +#### Presence, origins, and historical paths + +`Result.Metadata()` describes every final configuration path. Queries retain explicit key/index distinctions using actual effective and historical container shapes; only parsed numeric tokens are resolved contextually. A descendant that never existed is not historical merely because an atomic ancestor was replaced. Exact @@ -572,6 +793,18 @@ reports it absent. Queries return detached records. ### Secret classification and effective rendering +Choose which configuration values to hide when displaying a load. A +`configkit:"secret"` field tag or `Definition.Sensitive` predicate marks a path +as sensitive. Marking a container also hides its descendants. This choice is +separate from using a typed `Secret` to read a credential, described below. + +The saved output continues to hide sensitive historical values even when a +higher layer replaces them. When a resolver records a dependency on a sensitive +value, that sensitivity also protects the derived target. The detailed rules +below explain how classification spreads and how output is formatted. + +#### Classification and output details + Secret classification combines `configkit:"secret"` field tags with the optional pure `Definition.Sensitive(Path) bool` predicate. Tags on concrete structs inside supported dynamic values are retained in typed projections, including superseded @@ -617,6 +850,15 @@ prose. ### Historical-value explanation +An explanation shows the final value and the earlier contributions that led to +it. In the `workers` example above, a caller can inspect `/workers` to see the +default, main input, and overlay along with their source attribution. The +returned values are YAML strings in copies of the saved records; editing them +cannot change a later explanation. Sensitive values are hidden before those +strings are returned. + +#### Value representations and history + `Result.Explain(path)` returns a detached `Explanation` for any known effective or historical path. Its effective value and each chronological before/after value are `ValueSnapshot` records containing deterministic YAML text rather than mutable Go or @@ -700,6 +942,23 @@ kind, or null/container replacements report their affected region. Records use the existing comparator's canonical pointer ordering. `Present=false` with empty YAML means absence; explicit null is present and renders as `null`. +The result fields help distinguish absence from hidden output: + +| Field | Meaning | +| --- | --- | +| `ValueSnapshot.Present` | False means the value is absent. Explicit null is present and renders as `null`. | +| `ValueSnapshot.Redacted` | This value, or something inside it, was hidden. | +| `Difference.HiddenChanged` | A hidden value or its presence actually changed. Redaction alone does not set this flag. | + +For example, a sensitive string changing from one value to another can produce +the same `'[REDACTED]'` text on both sides. The difference record still exists and +`HiddenChanged` is true. A literal input string that looks like that marker is +ordinary data; use the fields rather than parsing marker text. A typed secret's +reference name is compared, so rotating its environment credential without +changing that name does not produce a difference. + +### Combined protection and comparison details + Before either side is returned, sensitivity from both loads is applied to both snapshots, including historical and dependency-derived classification. A classification on either side protects the corresponding values on both sides. @@ -722,7 +981,24 @@ its normal nil-result/`LoadError` behavior, which callers handle before invoking ## Typed secret references -Credential-bearing fields use the opaque `Secret` type and store an environment +There are three separate tasks when working with credentials: + +| Task | Mechanism | +| --- | --- | +| Read a credential from an environment variable | A typed `Secret` field whose YAML value is the variable's name | +| Hide a configuration value in inspection output | A `configkit:"secret"` tag or `Definition.Sensitive` | +| Populate environment variables from local files before loading | An explicit `LoadSecretsIntoEnvironment` call | + +For example, YAML `api_key: DEMO_API_KEY` names an environment variable. Its +credential is available to trusted application code through `Secret.Value()` +after loading succeeds. The name `DEMO_API_KEY` remains visible in inspection +unless the path is separately classified as sensitive. The credential itself is +never retained in inspection history. The optional file helper below populates +the environment; ordinary configuration loading does not write it. + +### Reference requirements and credential lifetime + +Fields that need credentials use the `Secret` type and store an environment variable name in YAML, such as `api_key: OPENROUTER_API_KEY`. `SecretEnv(name)` constructs the same unresolved reference for application defaults or recorded resolution. Names must match `[A-Za-z_][A-Za-z0-9_]*`; they are public @@ -770,6 +1046,34 @@ reference state without printing its value. ## Explicit secrets-directory bootstrap +If an application stores credentials in files, it can call this helper before +loading configuration. A file named `DEMO_API_KEY` containing the dummy text +`example-value` supplies environment variable `DEMO_API_KEY`. YAML can then +refer to that variable with `api_key: DEMO_API_KEY`. Applications that already +receive environment variables do not need this file step. + +Existing environment variables take priority over files. If `DEMO_API_KEY` is +already set to an empty string, its file is skipped and it stays empty. A later +required `Secret` reference will still fail on that empty value. File bootstrap +accepting an empty file is different from a required reference resolving to an +empty credential. + +Read the report even when the helper returns an error: + +| Status | How to interpret it | +| --- | --- | +| `loaded` | This file supplied the environment value. | +| `skipped_existing` | The variable was already set; `ExistingEmpty` says whether that existing value was empty. | +| `failed` | This entry could not be processed; inspect the safe diagnostics. | +| `not_applied` | No value from this eligible entry was applied, for example because preflight failed or an earlier write failed. | + +Render the returned `SecretsReport` separately from a later configuration load. +It contains filenames, paths, and statuses, not credential contents. The +[maintained startup example](../examples/consumers/consumers_test.go) shows how to +keep bootstrap and configuration reports together in the application. + +### Preflight, limits, and partial writes + `LoadSecretsIntoEnvironment(directory)` is a separate, opt-in startup operation. It is never called by `Load`, is not a YAML option, and does not participate in configuration validation. Applications choose the native directory path and @@ -809,6 +1113,44 @@ files when appropriate; ConfigKit stores no implicit global report. ### Recorded application resolution +Application rules fit into three tasks: + +| Task | Definition field | +| --- | --- | +| Supply fixed starting values | `Defaults`, which returns a fresh configuration | +| Derive or normalize values while recording why they changed | `Resolve` | +| Report domain errors and warnings without changing values | `Validate` | + +Within `Resolve`, choose a helper according to whether it should assign a value: + +| Helper | Assigns a value? | Records the change? | +| --- | --- | --- | +| `Default` | Only when input did not explicitly supply the path. | When it assigns. | +| `Normalize` | Always. | Yes. | +| `RecordDerivedDefault` | No; the application has already performed the mutation. | Yes, including for explicit input. | +| `RecordNormalization` | No; the application has already performed the mutation. | Yes. | + +For a configuration with `Workers` and `QueueLimit`, a resolver can use this call: + +```go +configkit.Default(resolution, configkit.Keys("queue_limit"), + &configuration.QueueLimit, configuration.Workers*2, + "size queue from workers", configkit.Keys("workers")) +``` + +If input omits `queue_limit`, this derives it from `workers` and records the +reason and dependency. An explicitly supplied `queue_limit: 0` is preserved. +Dependency paths explain where a derived value came from and support sensitivity +propagation. Use value-free reasons. See the [executable API examples](../example_test.go) +for complete callback definitions. + +Use a `Resolution` or `Validation` context only inside its callback. Do not retain +it or use it concurrently. Returned metadata records are copies you can keep. +`EntryInfo` keeps describing the configuration at the start of resolution, +including defaults and CLI assignments, even after your resolver changes values. + +#### Resolution recording and failure rules + An optional `Definition.Resolve` callback runs after successful defaults, typed decoding, and explicit CLI assignment. It receives the working configuration and a per-load `Resolution` context. The context's @@ -871,6 +1213,11 @@ snapshots. ### Semantic validation +Use validation for application rules such as a nonnegative worker count or a +required combination of fields. Report errors and warnings through the supplied +context. Make any normalization or derived assignment in `Resolve` first; +validation examines the final configuration without changing it. + An optional `Definition.Validate` callback runs after successful resolution and receives the resolved configuration plus a per-load `Validation` context. It can collect multiple independent `validate.error` and `validate.warning` reports. @@ -898,6 +1245,24 @@ live services as a substitute for configuration validation. ## Strict document composition +### Combining mappings + +The main document and its imports are peers in one tier. For example, a main +file containing `service: {workers: 2}` and an import containing +`service: {enabled: false}` produce: + +```yaml +service: + workers: 2 + enabled: false +``` + +If the import also defined `workers`, strict composition would fail, even if its +value were also `2`. An overlay containing `service: {workers: 4}` intentionally +changes the result to `workers: 4`, retaining `enabled: false`. This shows the +difference between combining peers and applying a higher-priority layer. +Identified lists have their own rules below. + Ordinary `Load` applies strict document composition to its parsed source scope; `Options.Imports` adds imported documents as peers in that scope. Application defaults do not participate. With imports disabled, the single source still has @@ -920,6 +1285,46 @@ Sequences without effective identity conflict when repeated and are never joined by index. Disjoint entries under shared mappings compose, making stable keyed mappings the natural representation for independently contributed entries. +### Identified lists: a worked example + +A record is a mapping inside a list. An identity field such as `id` lets ConfigKit +tell which records refer to the same entry. Declare the identity and desired +operation at the source root, for example: + +```yaml +configkit: + sequences: + - destination: [records] + key: id + mode: upsert +``` + +Suppose a lower layer supplies +`records: [{id: a, width: 10, length: 20}, {id: c, width: 50}]`, and +its higher layer supplies `records: [{id: a, width: 30}, {id: b, width: 40}]`. +The result during source assembly depends on the selected operation: + +| Operation | Result for this example | +| --- | --- | +| `replace` | `[{id: a, width: 30}, {id: b, width: 40}]` | +| `append` | Error: both layers contain identity `a`. | +| `upsert` | `[{id: a, width: 30}, {id: c, width: 50}, {id: b, width: 40}]` | + +Upsert replaces the complete matching record, so `length` from the lower `a` +record is gone. The unmatched lower `c` record stays at its original position; +new identities append. If the higher layer supplied only `[{id: b, width: 40}]`, +append would succeed with +`[{id: a, width: 10, length: 20}, {id: c, width: 50}, {id: b, width: 40}]`. +Typed defaults and decoding happen later. + +Within a strict tier, distinct identities combine and duplicate identities fail +under every operation. For example, two imported peers may each supply different +records, but cannot both define record `a`. Across layers, the operation table +below applies. Without identity, repeated peer lists conflict and higher-layer +lists replace whole lower lists. + +### Application registrations and identity details + `Definition.SequenceIdentities` contains registrations created by `SequenceIdentityAt` or `SequenceIdentityMatching`. The exact form accepts a mapping-only `Path`; the matching form accepts literal-key and any-mapping-key @@ -976,6 +1381,8 @@ headers are consumed before strict composition and cannot supply a registered record identity. Final typed decoding and application validation still run on the completed configuration. +### YAML declarations and policy lifetime + Ordinary sources, layered inputs, and selected overlays may declare collection behavior under `configkit.sequences` at the document root. Each entry has a required literal `destination` and may specify `key` as one string or a nonempty @@ -1167,6 +1574,52 @@ zero. ## Explicit imports +Imports include other documents as peers of the declaring document. Enable +imports in the application (`Options.Imports` or the layered input's `Imports`) +and choose one selector per declaration. These are alternatives for selecting +files, not three declarations that must be used together: + +Named file: + +```yaml +configkit: + imports: + - path: fragments/service.yml +``` + +Directory search, restricted here to YAML filenames: + +```yaml +configkit: + imports: + - dir: fragments + pattern: "*.yml" +``` + +Path pattern: + +```yaml +configkit: + imports: + - glob: "fragments/*.yml" +``` + +Each selected document follows the same strict combination rules. A missing +optional input can be absent; invalid declarations, unreadable selected files, +and conflicting settings still fail loading. + +| Option | `path` | `dir` | `glob` | +| --- | --- | --- | --- | +| `required`, `exclude`, `destination`, `placement` | Supported | Supported | Supported | +| `pattern`, `recursive`, `depth` | Not supported | Supported (`depth` needs effective recursion) | Not supported | +| `follow`, `hidden` | Not supported | Supported | Supported for wildcard-discovered entries | + +Each applicable option is chosen independently: its declaration value wins over +an application import default, which wins over the library default. The detailed +rules below explain absence, patterns, placement, and ordering. + +### Declaration and selection rules + Imports are opt-in. With `Options.Imports` enabled, a document may use its root `configkit` mapping for `imports` and `sequences` controls. Ordinary `Load` may use `sequences` without enabling imports, but an `imports` field still requires @@ -1184,12 +1637,6 @@ reads them as strings. Null, scalar destinations, and non-string members are key below the root is invalid. With imports disabled, root imports and nested `configkit` keys remain `import.disabled`. -The directive key is `configkit`, not the `_configkit` spelling used in v0.1.0. -There is no legacy alias: `_configkit` is ordinary application data, subject to -normal schema validation, and does not trigger imports. When upgrading, rename -directive keys in root files and imported fragments, and rename any application -fields previously named `configkit` to avoid the newly reserved key. - Path declarations use slash-separated relative paths. Absolute paths and network URLs are invalid. Each path resolves from its declaring file's canonical directory. An empty `path: []` selects nothing; it succeeds even when `required: true` because @@ -1238,6 +1685,17 @@ Sequence identities can be supplied either by these YAML declarations or by application registrations. The [strict composition contract](#strict-document-composition) defines their agreement, operation, and lifetime rules. +### Ordering and inspection + +Reading sources and combining their settings are separate steps. Source IDs +follow the reading order: declarations in the order written, with selected +paths in lexical order. Combination waits for dependencies. If `main.yml` +imports `extra.yml`, `extra.yml` is ready before the body of `main.yml`. +Among all ready contributions, ConfigKit uses its stable source-key ordering; +independent branches can therefore interleave. This ordering determines list +assembly and reporting, but never allows conflicting peer values to overwrite +one another. + Acquisition visits declarations in authored order and each declaration's selected paths in namespace-path lexical order, preserving source-record and version-callback order. Ordinary mapping contributions in a main tier are assembled dependency-first: @@ -1306,30 +1764,31 @@ Physical paths continue to resolve from the file that declares each import. An actual empty mapping contributes an empty item, while an unmatched declaration with default or explicit `required: false` contributes nothing. -For host files, the configuration root defaults to the selected file's canonical -directory. A host file may supply `FilesystemContext.Root` to widen that root but -may not supply `BaseDir` or a native `FS`. Byte, reader, and mapping inputs with host context -must provide both `BaseDir` and `Root` as absolute existing directories. -For `Filesystem.File`, the default root is `.` in its namespace and the base is -the selected file's directory. An explicit context must use the same `Filesystem` -namespace and may set `Root` to contain the file and its base; it may not supply -`BaseDir`. Byte, reader, and mapping inputs with native context provide both `BaseDir` and -`Root` as slash-relative `fs.ValidPath` directories in `FS` (`.` means its root), -with the base inside the root. Context paths are checked even when imports and -overlays are inactive. Diagnostic labels, including URI-looking labels, never -imply filesystem paths. Nonregular targets are rejected before opening to avoid blocking on FIFOs; -post-open regular-file checks remain in place under the stable-input assumption. -ConfigKit canonicalizes symlinks, -checks containment with relative paths, and opens the canonical root-relative target -through a library-owned root handle. Contained relative or absolute symlink targets -are accepted; escapes are rejected. Canonical paths define identity, so hard links at -distinct paths remain distinct sources. Native filesystems use slash-path -operations. Where the supplied `fs.FS` implements `fs.ReadLinkFS`, ConfigKit uses -that capability to inspect and resolve discovered links; otherwise it follows -the filesystem's ordinary `fs.FS` behavior. ConfigKit does not require every -native filesystem to expose link inspection, and native filesystem behavior is -not a general sandbox guarantee. Imports stay in their selected namespace and -root. +For host files, the configuration root defaults to the selected file's +canonical directory. A host file may supply `FilesystemContext.Root` to widen +that root but may not supply `BaseDir` or a native `FS`. Byte, reader, and +mapping inputs with host context must provide both `BaseDir` and `Root` as +absolute existing directories. For `Filesystem.File`, the default root is `.` +in its namespace and the base is the selected file's directory. An explicit +context must use the same `Filesystem` namespace and may set `Root` to contain +the file and its base; it may not supply `BaseDir`. Byte, reader, and mapping +inputs with native context provide both `BaseDir` and `Root` as slash-relative +`fs.ValidPath` directories in `FS` (`.` means its root), with the base inside +the root. Context paths are checked even when imports and overlays are +inactive. Diagnostic labels, including URI-looking labels, never imply +filesystem paths. Nonregular targets are rejected before opening to avoid +blocking on FIFOs; post-open regular-file checks remain in place under the +stable-input assumption. ConfigKit canonicalizes symlinks, checks containment +with relative paths, and opens the canonical root-relative target through a +library-owned root handle. Contained relative or absolute symlink targets are +accepted; escapes are rejected. Canonical paths define identity, so hard links +at distinct paths remain distinct sources. Native filesystems use slash-path +operations. Where the supplied `fs.FS` implements `fs.ReadLinkFS`, ConfigKit +uses that capability to inspect and resolve discovered links; otherwise it +follows the filesystem's ordinary `fs.FS` behavior. ConfigKit does not require +every native filesystem to expose link inspection, and native filesystem +behavior is not a general sandbox guarantee. Imports stay in their selected +namespace and root. An active-stack repeat is `import.cycle` and reports the complete cycle's import sites. A previously visited canonical path is `import.repeated` and reports both @@ -1339,9 +1798,39 @@ application's schema-version path or replace one of its ancestors; the version m come from the selected source. No directory is scanned implicitly, including one named `conf.d`, and ConfigKit performs no remote fetching. +### Earlier directive spelling + +The directive key is `configkit`, not the `_configkit` spelling used in v0.1.0. +There is no legacy alias: `_configkit` is ordinary application data, subject to +normal schema validation, and does not trigger imports. When upgrading, rename +directive keys in root files and imported fragments, and rename any application +fields previously named `configkit` to avoid the newly reserved key. + ## YAML documents -The private YAML boundary currently accepts exactly one nonempty document whose root +A configuration document is a YAML mapping at the top level. For example, these +values can be loaded into Go fields of the matching types: + +```yaml +enabled: false +wait: 5s +title: "yes" +``` + +```go +type Settings struct { + Enabled bool `yaml:"enabled"` + Wait time.Duration `yaml:"wait"` + Title string `yaml:"title"` +} +``` + +Here `5s` is a string that decodes into `time.Duration`; ordinary text `yes` +needs quotes because the unquoted legacy boolean spelling is rejected. Use +lowercase `true` and `false` for booleans. The following reference lists the +complete accepted and rejected forms. + +ConfigKit accepts exactly one nonempty YAML document whose root is a mapping. Comments and block or flow collections are supported. It rejects: - empty input or an empty document; @@ -1386,9 +1875,8 @@ keys that otherwise classify as a boolean, number, or null. ## Application schema binding -ConfigKit builds a private reflection descriptor graph for each schema operation; -it keeps no global mutable descriptor cache. A configuration root must be a struct -or string-keyed map. Nested supported shapes are: +Your Go configuration type defines the accepted fields and value types. The +root must be a struct or a map with string keys. Nested supported shapes are: - booleans, strings, signed and unsigned integers, and finite floating-point types; - `time.Duration`, typed `Secret` references, and paired @@ -1396,6 +1884,8 @@ or string-keyed map. Nested supported shapes are: - pointers, structs, string-keyed maps, slices, fixed arrays, and `any`; and - recursive type definitions whose eventual application values are acyclic. +### Advanced type and field rules + Named scalar types retain their Go identity. `uintptr`, complex numbers, functions, channels, unsafe pointers, non-string map keys, and nonempty interfaces are not configuration types. A custom text scalar must provide both directions on its value @@ -1415,16 +1905,16 @@ ConfigKit field tag and marks its bound field or container for secret classifica and redacted inspection. Schema path traversal uses these same bindings. Explicit key segments address -mappings, explicit index segments address sequences, and parsed numeric pointer -tokens become indices only in a sequence. Sequence indices must use canonical -nonnegative decimal spelling: no signs or leading zeroes except `0` itself. -Tokens such as `+0` and `-0` remain literal keys in mappings but do not identify -sequence elements. Fixed-array indices are bounds checked. Dynamic `any` and inline-map entries -remain schema-valid for application-defined descendants, but dynamic values cannot -introduce typed secret references. +mappings, explicit index segments address sequences, and parsed numeric +pointer tokens become indices only in a sequence. Sequence indices must use +canonical nonnegative decimal spelling: no signs or leading zeroes except `0` +itself. Tokens such as `+0` and `-0` remain literal keys in mappings but do +not identify sequence elements. Fixed-array indices are bounds checked. +Dynamic `any` and inline-map entries remain schema-valid for +application-defined descendants, but dynamic values cannot introduce typed +secret references. -Descriptors are reused by typed decoding, projection, and cloning so those operations -cannot acquire different field bindings. +Decoding, inspection output, and copied values use the same field bindings. ### Canonical module options @@ -1446,8 +1936,24 @@ examples. ## Defaults and typed decoding -The private decoder starts from an independently owned clone of the application's -typed defaults and applies only explicitly present input nodes. Defaults therefore +Defaults are starting values for omitted settings. Suppose your defaults are +`enabled: true`, `workers: 2`, `names: [primary]`, and a non-nil pointer value +for `optional`. The following independent examples illustrate the effect of input: + +| Input | Result | +| --- | --- | +| `enabled` omitted | Retain `true`; it is not explicit input. | +| `enabled: false` | Use `false` and mark the path explicit. | +| `workers: 0` | Use `0` and mark the path explicit. | +| `names: []` | Use an empty list instead of the default list. | +| `optional: null` | Use nil and suppress the pointer default. | + +Null requires a field type that allows nil. For example, `enabled: null` fails +for a non-pointer boolean; it does not mean "use the default." The full table +below also covers partial mappings, new entries, and pointer objects. + +Decoding starts from a copy of the application's typed defaults and applies only +settings that input supplied. Defaults therefore never become explicit, while equal-valued input remains explicit. Successful decode produces detached typed state, baseline and final canonical trees, an input-presence index, and default/input origins for the later public loading lifecycle. @@ -1492,13 +1998,30 @@ conversion API. ## Typed projection and ownership -ConfigKit can internally project a schema-valid application value directly into its -owned canonical tree without YAML serialization. Projection includes every bound -field, including ordinary zero values; `omitempty` has no effect. Struct fields keep -binding order, dynamic map keys sort by decoded string, and sequences keep their -order. Integers retain exact base-10 values, finite floats use their destination -precision, durations use `time.Duration.String`, and paired text scalars use their -canonical marshaled text. +ConfigKit uses a consistent representation of typed values for inspection and +comparison, called a projection. It includes every field bound by the schema, +even ordinary zero values. `omitempty` does not remove these fields from +output. For example, a nil slice displays as `null`, while a non-nil empty +slice displays as `[]`. Creating this representation does not serialize and +reparse YAML. Struct fields keep binding order, dynamic map keys sort by +decoded string, and sequences keep their order. Integers retain exact base-10 +values, finite floats use their destination precision, durations use +`time.Duration.String`, and paired text scalars use their canonical marshaled +text. + +For example, loading `{}` into this type leaves `Workers` at zero and ignores +`Internal`: + +```go +type OutputSettings struct { + Workers int `yaml:"workers,omitempty"` + Internal string `yaml:"-"` +} +``` + +The effective YAML includes `workers: 0`; `omitempty` does not hide it. There is +no `Internal` field in that output. Ignored fields are zeroed in the returned +configuration, including when the defaults factory supplied a value for them. Nil pointers, maps, slices, and dynamic values project as null. Non-nil empty maps and slices remain empty containers, preserving the distinction. Recursive acyclic @@ -1523,6 +2046,15 @@ and rendering. They are not a public generic conversion or cloning API. ## Safe failures +A failed load returns a `LoadError` with structured diagnostics. Inspect the +code to identify the kind of problem, the path to identify the setting, and the +source location to find it in the input. For example, misspelling a declared +`enabled` field as `enabeld` produces `decode.unknown_field` at `/enabeld`, with +available source coordinates. Human-readable messages explain the problem; +application code can compare the diagnostic code instead of parsing that text. +See [WriteDiagnostics](../diagnostic.go) and the +[maintained error-handling example](../example_test.go). + Source failures use stable `source.*` codes and YAML failures use stable `yaml.*` codes declared by the root package. Diagnostics contain safe classifications, semantic paths when applicable, and available source coordinates. Raw parser, diff --git a/docs/consumers/README.md b/docs/consumers/README.md index 5b0f4f9..87d62bd 100644 --- a/docs/consumers/README.md +++ b/docs/consumers/README.md @@ -1,24 +1,35 @@ # Consumer Adoption Guide -ConfigKit removes parser setup, strict decoding, presence bookkeeping, import and -selected-overlay traversal, precedence, provenance, secret lookup, and safe inspection from -applications. Consumers still own their configuration structs, defaults, semantic -validation, schema-version policy, CLI parsing, runtime matching and conversion, -resource checks, remote acquisition, and service initialization. +ConfigKit loads configuration into your application's Go types. You provide +those types, defaults, and application rules. ConfigKit reads and combines the +selected inputs, checks their types, and returns values with source information. +Your application still owns CLI parsing, resource reads, remote acquisition, and +runtime initialization. -Import the public root package as `"go.maximumdirect.net/configkit"`; its package -name remains `configkit`. +Import `"go.maximumdirect.net/configkit"`; the package name is `configkit`. +Start with one file, then choose the tasks your application needs: + +| Task | Start here | +| --- | --- | +| Load one file | [Happy path](#happy-path) and the [README example](../../README.md#load-a-configuration-file) | +| Supply fixed defaults and check application rules | [Defaults and application rules](#defaults-and-application-rules) | +| Add bundled defaults beneath application settings | [Native filesystem and layered input](#native-filesystem-and-layered-input) | +| Select files or an overlay | [Discovery](#explicit-main-file-discovery), [import defaults](#import-defaults-and-selection-inspection), and the [overlay reference](../config.md#selected-overlays) | +| Reuse settings or combine record lists | [Inheritance](#named-sibling-inheritance) and [identified sequences](#identified-prompt-sequences) | +| Handle credentials | [Typed-secret reference](../config.md#typed-secret-references) and the [startup example](../../examples/consumers/consumers_test.go) | +| Inspect sources, earlier values, or changes | [Inspect a completed load](#inspect-a-completed-load) and [diffs](#effective-configuration-diffs) | +| Upgrade an existing consumer | [Migration guidance](#intentional-migration-changes) | ## Choosing a loading entry point - Use [`Load`](../../lifecycle.go) with one source for an ordinary configuration. - [`SearchConfig`](../../discovery.go) is also a source, so its explicit - discovery result can be passed to `Load` or used as the main input to - [`LoadLayered`](../../layered.go). -- Use `LoadLayered` when inputs must be composed in underlay groups before the - main source. Each [`Input`](../../layered.go) owns its source, filesystem, - destination, and import choice. The overlay filesystem context is selected - independently in [`LayeredOptions`](../../layered.go). + [`SearchConfig`](../../discovery.go) is also a source; pass it to `Load` or use + it as the main input to [`LoadLayered`](../../layered.go). +- Use `LoadLayered` when several inputs should supply lower-priority defaults + beneath the main source. Each underlay group combines its inputs strictly. + Each [`Input`](../../layered.go) owns its source, filesystem, destination, and + import choice. The overlay filesystem context is selected independently in + [`LayeredOptions`](../../layered.go). - Load-wide rules belong to [`Options`](../../lifecycle.go) or [`LayeredOptions`](../../layered.go): the application definition (including version policy), import defaults, and explicit overrides. These do not turn @@ -33,68 +44,68 @@ file and layered cases. For import-focused adoption, see the discovery composed with sequences, see the [sequence catalog](../../examples/consumers/sequence_catalog.go). -The maintained [`examples/consumers`](../../examples/consumers) package demonstrates -sanitized shapes using only ConfigKit and the Go standard library: - -| Shape | Demonstrated fit | Application code deliberately retained | -| --- | --- | --- | -| Narratio-like | Fresh defaults, explicit partial imports, artifact maps, recorded inherited defaults, normalization, and source ownership | Artifact meaning and the separate `model_profile` domain field | -| Notarius-like | Canonical object bindings, a string-backed schema version accepting string or integer YAML, strict dynamic options, and `int64` dynamic integers | Catalog selection, allowed option keys, range-checked conversion to runtime `int` | -| WeatherReporter-like | Defaults-only loading, report maps, canonical module objects, typed option unions, explicit CLI records, and typed secret startup | Report/module compatibility, rejection of explicitly empty domain lists, CLI parsing, and startup ordering | -| Seriatim-like | Independent strict ordered speaker and autocorrect rule files | First-match and ordered replacement algorithms; no central application configuration file | -| Bundled file | Simple native `fs.FS` source through ordinary `Load` | Selection and lifetime of the application-owned embedded filesystem | -| Layered catalog | Embedded defaults with an import, a mapping underlay, an independent host main file, explicit false/zero input, and source-relative resource access | Domain validation and opening/interpreting the template resource | -| Mounted collections | Entry-declared relative destinations, nested metadata/prompt/profile imports, local inheritance, and a version-only higher-tier override | Collection IDs, profile meaning, and opening/interpreting prompt resources | -| Named presets | One-load sibling inheritance, explicit false, nested destination resolution, and source/derivation inspection | Choosing and executing a preset | -| Effective configuration diff | Same-type frozen effective comparisons for two selected files or base-versus-overlay impact | Source/options selection and consumer-owned safe presentation | -| Shared source-version checks | One `Definition.Version` policy for accepted declarations across participating source documents, with optional or required declarations and sanitized evidence | Choosing which declarations the application accepts; final assembled schema and domain validation | -| Sequence catalogs | Declared collection operations, record inheritance, typed CLI and resolution, inspection, exact and wildcard application registrations | Catalog meaning and final domain validation | - -The YAML under [`testdata`](../../examples/consumers/testdata) is loaded by the -executable examples and ordinary tests. It contains no private application data, -credentials, network calls, or dependency on a sibling checkout. - -The Notarius-shaped fixtures deliberately load both `version: 4` and -`version: "4"` into the same string-backed application field. Its version checker -still owns the supported identifier; fractional numeric input and unsupported -integer identifiers remain negative examples. - ## Happy path -An application defines ordinary types and passes one explicit source. Optional -features appear only when the application needs them: +Define the Go configuration type, choose one source, and call `Load`. Handle a +load error before starting the application with `result.Config`. The +[README example](../../README.md#load-a-configuration-file) shows this complete +minimal loading function. An explicitly missing file is an error; use `NoSource` +when you deliberately want defaults without a main document. + +### Defaults and application rules + +Using the README's `Config` type, an application can add reusable rules: ```go -result, err := configkit.Load[Config](configkit.File(path), configkit.Options[Config]{ - Definition: rules, - Imports: useDeclaredImports, - Overlay: selectedConfigOverlay, - Overrides: explicitlySetFlags, -}) -if err != nil { - return err +definition := configkit.Definition[Config]{ + Defaults: func() Config { return Config{Enabled: true, Workers: 2} }, + Validate: func(value *Config, validation *configkit.Validation) { + if value.Workers < 0 { + validation.Error(configkit.Keys("workers"), "must be nonnegative") + } + }, } -return start(result.Config) ``` -Simple consumers use neither imports nor selected overlays. A complex consumer can opt into -both without redefining their syntax. ConfigKit overlays are configuration overlays; -they do not select or replace an application's LLM, model, pipeline, report, or -execution profile concepts. +Pass this definition in `Options[Config]{Definition: definition}` when calling +`Load`. Supplied `enabled: false` or `workers: 0` replaces those defaults. +Use `Resolve` when a value must be derived or normalized before validation, and +record that change with the provided helpers. The +[callback reference](../config.md#recorded-application-resolution) compares the +helpers; [executable examples](../../example_test.go) show full definitions. + +### Inspect a completed load + +Keep the result when you need to explain configuration. Use +`result.Metadata().Lookup(configkit.Keys("workers"))` to inspect whether input +supplied the value and which source or application step supplied it. Follow an +origin's source ID with `result.Source(id)` when you need its source record. + +Use `Explain` for earlier values, `WriteExplanation` for a readable explanation, +and `WriteEffective` for safe YAML describing the final saved state. These +operations describe the completed load even if your application later changes +`result.Config`. The [inspection reference](../config.md#layered-metadata) +explains returned values, copies, and hidden information. + +Add imports, a selected overlay, or explicitly set CLI values through options +only when needed. An overlay changes configuration; choosing an LLM, report, or +execution profile remains application logic. ## Explicit main-file discovery Applications choose the search conventions and their precedence, then pass the -resulting `SearchConfig` source to the same `Load` or `LoadLayered` entry point. -For example, the maintained sequence catalog puts a caller-selected file before -a custom directory fallback while keeping its underlay and overlay choices -independent. See [`sequence_catalog.go`](../../examples/consumers/sequence_catalog.go) -and its [behavior tests](../../examples/consumers/sequence_catalog_test.go), plus -the [working-directory executable example](../../examples/consumers/discovery_test.go). -Together, the root-owned [discovery API](../../discovery.go) and [configuration -discovery contract](../config.md#configuration-discovery) define ordered -concatenation, directory-first Cartesian expansion, optional exhaustion, and -selected-file behavior. [ADR 0019](../adr/0019-explicit-configuration-discovery.md) +resulting `SearchConfig` source to the same `Load` or `LoadLayered` entry +point. For example, the maintained sequence catalog puts a caller-selected +file before a custom directory fallback while keeping its underlay and overlay +choices independent. See +[`sequence_catalog.go`](../../examples/consumers/sequence_catalog.go) and its +[behavior tests](../../examples/consumers/sequence_catalog_test.go), plus the +[working-directory executable +example](../../examples/consumers/discovery_test.go). Together, the [discovery +API](../../discovery.go) and [configuration discovery +contract](../config.md#configuration-discovery) explain the order of attempted +files, what happens when no file is found, and why a selected-file failure +stops the search. [ADR 0019](../adr/0019-explicit-configuration-discovery.md) records why discovery is explicitly selected. ## Import defaults and selection inspection @@ -143,13 +154,14 @@ priority values remain present. Its resolver records a derived endpoint URL afte overlay application, and its nested typed secret reference is classified before the resulting records are presented. -The maintained [behavior tests and executable examples](../../examples/consumers/diff_test.go) -use temporary files and disposable secret values. Presenters should use -`ValueSnapshot.Present` for absence, record membership for change, and -`HiddenChanged` for a withheld change indicator. Do not parse the redaction marker -or add separately rendered full contexts that lack combined sensitivity. The -[configuration contract](../config.md#effective-configuration-diffs) owns shared -equality, granularity, and disclosure semantics. +The maintained [behavior tests and executable +examples](../../examples/consumers/diff_test.go) use temporary files and +disposable secret values. Presenters should use `ValueSnapshot.Present` for +absence, record membership for change, and `HiddenChanged` for a withheld +change indicator. Do not parse the redaction marker or add separately rendered +full contexts that lack combined sensitivity. The [configuration +contract](../config.md#effective-configuration-diffs) owns shared equality, +granularity, and disclosure semantics. ## Named sibling inheritance @@ -276,10 +288,9 @@ path-bearing example. Its application resolver uses `Resolution.EntryInfo` and `Resolution.Source` before normalizing: relative CLI input uses an explicitly supplied invocation directory, retained defaults use the canonical main source, and a file-authored scalar uses its one declaring source. Absolute paths remain -unchanged. Its executable test covers the matching effective YAML and explanation -history, source-file ownership, and a symlink opened through an alias; canonical -target-file directories, rather than the alias directory or process working -directory, are the selected bases. +unchanged. Its executable test checks the resulting YAML and explanation +history. When the configuration filename is a symlink, this example uses the +actual target file's directory as the base for file-authored paths. The selected example policy leaves a relative value unchanged and emits a warning when a CLI base is absent or file provenance is missing or composite. That is an @@ -289,7 +300,15 @@ existence checks nor confinement checks. ## Intentional migration changes -The examples preserve domain meaning while adopting one configuration convention: +When upgrading, start with the changes relevant to your dependency and inputs: + +1. Update the module requirement and Go imports together as described below. +2. Replace old public names using the old/new table. +3. Update any layer or diagnostic-code strings stored or compared by your code. +4. Review YAML and startup changes that apply to your application, then rebuild + and run its configuration tests. + +The maintained examples illustrate these configuration changes: - Explicit empty strings, lists, zeroes, false values, and nulls remain intentional. ConfigKit accepts an empty list structurally; WeatherReporter-shaped validation @@ -323,11 +342,13 @@ Go imports together, selecting a revision or release that declares the new modul path. The package name remains `configkit`; the configuration model and public symbol names are unchanged by this identity migration. -Old and new import paths identify distinct Go packages, including distinct named -types. Coordinate adoption with any dependencies that exchange ConfigKit values -with your application; changing a requirement alone does not make those types -interchangeable. The maintained examples use the new path, but other repositories -must manage their own migrations. +Go treats the old and new import paths as different packages, with different +named types. For example, if another library returns a ConfigKit `Path` or +`Secret` using the old import, your application cannot use it as the new +package's type just by changing a module requirement. Coordinate migration +with dependencies that pass ConfigKit values to or from your application. The +maintained examples use the new path, but other repositories must manage their +own migrations. Historical tags keep their original module declarations and contents. Older releases such as v0.10.2 still use the old path and cannot be installed under the @@ -363,12 +384,13 @@ compatibility aliases: | History diagnostic string `"explain.limit_exceeded"` | `"history.limit_exceeded"` | The YAML import keys `follow` and `hidden`, the `configkit:"secret"` tag, and -typed `Secret` references are unchanged. Consumers that compare literal strings -or keep serialized diagnostic/provenance expectations may need to update those -alongside Go references. This repository guidance does not claim that downstream -applications have migrated and does not select a release version for this -breaking pre-v1 change. `ChangeKind` and its inspection constants remain; use -`Default` and `Normalize` when the helper should both assign and record a value. +typed `Secret` references are unchanged. If your application compares or +stores layer names or diagnostic codes as strings, update those strings +alongside Go references. This repository guidance does not claim that +downstream applications have migrated and does not select a release version +for this breaking pre-v1 change. `ChangeKind` and its inspection constants +remain; use `Default` and `Normalize` when the helper should both assign and +record a value. ### Selected-overlay migration @@ -380,10 +402,40 @@ The [configuration contract](../config.md#selected-overlays) is the canonical behavioral reference. Application-owned profile concepts, such as `model_profile`, are unchanged, and any CLI flag rename remains the application's decision. -These examples are migration evidence, not completed changes to Narratio, Notarius, -WeatherReporter, or Seriatim. Updating those repositories, their deployed YAML, CLI -wiring, runtime catalogs, and operational documentation remains separate downstream -work. +### Background on the examples + +These examples demonstrate migration shapes. They do not claim that Narratio, +Notarius, WeatherReporter, or Seriatim has been migrated. Updating those +repositories, their deployed YAML, CLI wiring, runtime catalogs, and +operational documentation remains separate downstream work. + +## Example background + +The maintained [`examples/consumers`](../../examples/consumers) package demonstrates +sanitized shapes using only ConfigKit and the Go standard library: + +| Shape | Demonstrated fit | Application code deliberately retained | +| --- | --- | --- | +| Artifact defaults and normalization (Narratio-like) | Fresh defaults, explicit partial imports, artifact maps, recorded inherited defaults, normalization, and source ownership | Artifact meaning and the separate `model_profile` domain field | +| Versions and dynamic options (Notarius-like) | Canonical object bindings, a string-backed schema version accepting string or integer YAML, strict dynamic options, and `int64` dynamic integers | Catalog selection, allowed option keys, range-checked conversion to runtime `int` | +| Module validation and credential startup (WeatherReporter-like) | Defaults-only loading, report maps, canonical module objects, typed option unions, explicit CLI records, and typed secret startup | Report/module compatibility, rejection of explicitly empty domain lists, CLI parsing, and startup ordering | +| Independent ordered rule files (Seriatim-like) | Independent strict ordered speaker and autocorrect rule files | First-match and ordered replacement algorithms; no central application configuration file | +| Bundled file | Simple native `fs.FS` source through ordinary `Load` | Selection and lifetime of the application-owned embedded filesystem | +| Layered catalog | Embedded defaults with an import, a mapping underlay, an independent host main file, explicit false/zero input, and source-relative resource access | Domain validation and opening/interpreting the template resource | +| Mounted collections | Entry-declared relative destinations, nested metadata/prompt/profile imports, local inheritance, and a version-only higher-tier override | Collection IDs, profile meaning, and opening/interpreting prompt resources | +| Named presets | One-load sibling inheritance, explicit false, nested destination resolution, and source/derivation inspection | Choosing and executing a preset | +| Effective configuration diff | Same-type frozen effective comparisons for two selected files or base-versus-overlay impact | Source/options selection and consumer-owned safe presentation | +| Shared source-version checks | One `Definition.Version` policy for accepted declarations across participating source documents, with optional or required declarations and sanitized evidence | Choosing which declarations the application accepts; final assembled schema and domain validation | +| Sequence catalogs | Declared collection operations, record inheritance, typed CLI and resolution, inspection, exact and wildcard application registrations | Catalog meaning and final domain validation | + +The YAML under [`testdata`](../../examples/consumers/testdata) is loaded by the +executable examples and ordinary tests. It contains no private application data, +credentials, network calls, or dependency on a sibling checkout. + +The Notarius-shaped fixtures deliberately load both `version: 4` and +`version: "4"` into the same string-backed application field. Its version checker +still owns the supported identifier; fractional numeric input and unsupported +integer identifiers remain negative examples. ## Maintained API examples diff --git a/docs/internal/overview.md b/docs/internal/overview.md index 26103b3..d538455 100644 --- a/docs/internal/overview.md +++ b/docs/internal/overview.md @@ -179,3 +179,9 @@ reflection behavior by `schema_test.go`, `projection_test.go`, and `decode_test.go`. The [development test map](../development.md#task-to-test-map) links focused commands. [`clone.go`](../../internal/engine/clone.go) owns typed copying, using the shared descriptors and per-operation traversal helpers. + +Each schema operation builds its private reflection descriptor graph with +operation-local caching; there is no global mutable descriptor cache. Decoding, +projection, and cloning reuse these descriptors so field bindings agree. The +public reference describes the resulting type, output, and ownership behavior; +this descriptor storage is an implementation detail. diff --git a/docs/release.md b/docs/release.md index 93a14c8..9e2e25d 100644 --- a/docs/release.md +++ b/docs/release.md @@ -1,9 +1,16 @@ # ConfigKit Release Procedure This is the maintainer runbook for ConfigKit's [release policy](policy/release.md). -ConfigKit publishes a source-only Go module. The immutable annotated tag and -matching checked-in release note are the deliverables; there is no binary, -container image, generated package, or separate registry publication. +A release publishes the Go source under a new Git tag, with a matching release +note committed in the repository. The tag is annotated: it includes the release +note text and points to the released commit. Published tags and contents are +immutable. There is no binary, container image, generated package, or separate +registry publication. + +A **candidate** is the committed revision you intend to release. The +**authoritative remote** is the repository where ConfigKit publishes releases; +its exact push URL is listed below. The **consumer minimum** is the oldest Go +version supported for applications using ConfigKit. ## Release identity and support @@ -20,12 +27,14 @@ container image, generated package, or separate registry publication. - Best-effort targets: macOS amd64 and arm64, under the [platform support policy](policy/release.md#platform-support) -Release validation runs one offline race-enabled Go suite, one full vet pass, -and CGO-disabled package compilation for both Linux targets. macOS and CI are +Release validation runs the offline Go test suite once with the race detector, +then a full vet pass and package compilation for both Linux targets with cgo +disabled (`CGO_ENABLED=0`). macOS and CI are not release gates; do not wait for or poll CI. -The vanity module path identifies Go imports. Source/tag browser links and public -Go retrieval use the Forge HTTPS repository independently of SSH publication. +The vanity module path is the name applications put in Go imports. Source/tag +browser links and public Go retrieval use the Forge HTTPS repository +independently of SSH publication. ## 1. Select Go and prepare the tool @@ -36,10 +45,10 @@ For a patch release, retain the series recorded in `go.mod` and leave its consum minimum unchanged. Follow the [toolchain selection policy](policy/release.md#configkit-toolchain-selection). -Put the selected series' latest locally available compatible patch first on -`PATH`, ensuring it meets `go.mod`'s minimum. The same `go` executable must build -ReleaseKit and run its child validation commands. Set local toolchain selection -and verify it: +Choose the latest compatible patch installed locally in that Go series. Put +its `go` executable first on `PATH` and ensure it meets the minimum in +`go.mod`. The same `go` executable must build ReleaseKit and run its child +validation commands. Set local toolchain selection and verify it: ```sh export GOTOOLCHAIN=local @@ -64,8 +73,17 @@ use that required version instead and document the reason. When the series is unchanged, leave the minimum unchanged unless a minor or major release deliberately changes it under the [consumer minimum policy](policy/release.md#consumer-minimum-and-release-toolchain). -For example, validating with Go 1.26.8 leaves `go 1.26.0` unchanged. There is no -separate requirement to validate with Go 1.26.0. +For example: + +| Release situation | Go used for validation | Consumer minimum | +| --- | --- | --- | +| Patch release in the 1.26 series | Latest compatible patch installed locally, such as 1.26.8 | Keep the existing `go.mod` minimum unchanged. | +| Minor/major release staying in 1.26 | Latest compatible patch installed locally | Normally retain the minimum; any deliberate change follows the linked policy. | +| Minor/major release adopting a newer installed stable series | Latest compatible patch installed locally in that series | Normally declare that series' initial stable patch, or the higher patch required by a dependency or identified fix. | + +Using Go 1.26.8 for validation can therefore leave `go 1.26.0` unchanged. Users +need the declared minimum, while maintainers validate with their selected patch. +There is no separate requirement to validate with Go 1.26.0. Prepare dependencies whether or not the minimum changes: @@ -73,7 +91,8 @@ Prepare dependencies whether or not the minimum changes: go mod download ``` -Build the tool selected by `go.mod` into temporary storage outside the checkout. +Build the ReleaseKit version recorded in `go.mod` into temporary storage outside +the checkout. Ensure `$TMPDIR` names a system temporary location, not the repository. These commands never install another ReleaseKit version: @@ -85,8 +104,8 @@ go list -m gitea.maximumdirect.net/eric/ReleaseKit "$releasekit_bin" version ``` -The module-selected executable is ReleaseKit v0.4.2. Use the built executable -for publication to preserve its exit status and publication-state diagnostics. +This repository selects ReleaseKit v0.4.2. Use the compiled program above for +publication so its exit status and publication report reach you unchanged. `go run` maps a nonzero child exit status to 1. ReleaseKit's child Go commands use the `go` found on `PATH` with `GOTOOLCHAIN=local`; keep the selected executable first on `PATH` through the publication command. @@ -102,10 +121,10 @@ not authorize a push or release. ## 2. Prepare and commit the candidate -Choose an unused stable version locally and on the authoritative remote. Compare -the candidate with the previous tag and review exported Go signatures and -ownership, YAML and resolution semantics, diagnostic contracts, minimum Go -version, and platform support. Review maintained consumer examples and the +Choose a stable version that has not been used locally or on the publication +remote. Compare the candidate with the previous tag. Review public Go APIs and +value ownership, YAML and resolution rules, diagnostic contracts, the minimum +Go version, and platform support. Review maintained consumer examples and the migration guidance. Explain any consumer-minimum increase and required consumer action under the [consumer minimum policy](policy/release.md#consumer-minimum-and-release-toolchain). @@ -142,32 +161,40 @@ candidate to `main`, and invoke validation again. ## 4. Publish the committed candidate -With the authorized candidate already on authoritative `main`, invoke the -module-selected native executable exactly once: +Once publication is authorized and the candidate is on the publication remote's +`main`, run the compiled ReleaseKit program once to validate and publish it: ```sh "$releasekit_bin" publish vMAJOR.MINOR.PATCH ``` Replace the placeholder with the selected version. ReleaseKit checks the -effective push URL against `expected_push_url`, verifies the candidate on +push URL Git will actually use, after URL rewrites, against `expected_push_url`. +It verifies the candidate on `main`, and checks the release-note H1 for the product and version. It then runs the selected race-enabled standard suite once with test-time vet disabled, one full vet pass, and CGO-disabled package compilation for Linux amd64 and arm64. It publishes only the annotated tag for that candidate. Confirmed publication of the intended tag completes the release. The tag points to the captured commit and uses the complete checked-in note as its annotation message. -The optional `check` -command is available for diagnosis but is not a separate required pass: +Use the optional `check` command for diagnosis when useful. It validates without +publishing and is not required before `publish`, which performs its own checks: ```sh "$releasekit_bin" check vMAJOR.MINOR.PATCH ``` -Status 0 confirms publication, status 1 reports a known publication failure or -conflict, status 2 reports usage, profile, or version errors, and status 3 means -the push outcome remains unknown. Preserve both the native exit status and its -diagnostics. +For `publish`, interpret the program's exit status as follows: + +| Status | Meaning | Next action | +| --- | --- | --- | +| `0` | Publication succeeded. | The release is complete. Optional verification can follow. | +| `1` | A known publication failure or conflict was reported. | Read the report and correct the stated problem. Preserve any published tag. | +| `2` | The command, profile, or version is invalid. | Correct the command or configuration before trying again. | +| `3` | The tag push outcome is still unknown. | Follow the recovery procedure below before retrying or declaring success. | + +Keep both the program's exit status and its diagnostic report. Using `go run` +would lose the distinction between its nonzero exit statuses. The command does not push or repair `main`, wait for CI, or publish binaries or a hosted release page. A successful intended-tag publication does not require @@ -176,12 +203,17 @@ remote read-back or consumer retrieval; those checks remain optional. ## 5. Recover from a failed or uncertain tag push The executable reports the publication state, phase, version, candidate commit, -tag-object ID, effective endpoint, and error on standard error. It performs one -bounded authoritative reconciliation after a failed tag push. Exit status 3 +tag-object ID, effective endpoint, and error on standard error. After a failed +tag push, it makes one bounded check of the exact publication remote to find out +whether the tag arrived. Exit status 3 means the tag outcome remains unknown; do not blindly retry or move a tag. -For an uncertain result, use the exact endpoint, candidate commit, version, and -tag-object ID from the report. Copy the captured endpoint into the variable +An annotated tag is a Git object containing the release note and a pointer to +the source commit. Its tag-object ID identifies that complete object. Checking +only the commit ID would not establish that the intended tag object arrived. + +For an uncertain result, use the exact endpoint (remote URL), candidate commit, +version, and tag-object ID from the report. Copy the captured endpoint into the variable below using normal shell quoting. Check that Git still resolves it to itself, then query the authoritative refs once: @@ -198,7 +230,8 @@ cannot establish absence at the captured endpoint. Compare the `main` object ID with the reported candidate commit and the tag object ID with the reported tag-object ID. `--refs` returns the annotated tag -object itself, not merely its peeled commit. If the tag object matches, the +object itself. A peeled commit is the source commit reached by following that +tag; it is a different identity. If the tag object matches, the release is published. If the query fails, the result is conflicting, or the endpoint or candidate differs, stop and resolve the state before retrying. If the endpoint is reachable, the tag is confirmed absent, and `main` still equals diff --git a/docs/roadmap/documentation.md b/docs/roadmap/documentation.md index 48f6dd2..00cf362 100644 --- a/docs/roadmap/documentation.md +++ b/docs/roadmap/documentation.md @@ -2,10 +2,34 @@ Reviewed revision: `08bb3a20c6a565646d98f68688d268c7bae39573`. -Status: proposed revisions only. This catalog identifies opportunities to make -existing documentation easier to understand. None of the proposals has been -applied to its target page. The previous consistency audit remains in Git -history; its two findings were corrected in `94d1e33`. +Status: complete. All D01–D18 revisions have been applied and reviewed together +for consistency. The catalog below preserves the proposals that guided the +work; the completion record identifies their implemented locations. The previous +consistency audit remains in Git history; its two findings were corrected in +`94d1e33`. + +## Completion record + +| Proposal | Implemented result | +| --- | --- | +| D01 | [README](../../README.md) leads with the loading outcome and minimal example, followed by optional tasks and platform/licensing links. | +| D02 | [Consumer guide](../consumers/README.md) starts with task routes, defaults, validation, and inspection; example background and the test inventory follow usage. | +| D03 | [Reference introduction](../config.md) supplies topic routes, YAML illustrations, path definitions, and common-versus-detailed subheadings. | +| D04 | [Sources](../config.md#sources), [results](../config.md#results-and-failures), and [resource lifetime](../config.md#completion-and-resource-lifetime) explain copying, reuse, saved inspection, and caller-owned resources. | +| D05 | [Priority](../config.md#priority-and-processing-order), [strict combination](../config.md#combining-mappings), and [defaults](../config.md#defaults-and-typed-decoding) have visible examples and retain collection exceptions. | +| D06 | [Discovery](../config.md#configuration-discovery) shows the four attempted paths and distinguishes optional absence, required absence, and selected-file failure. | +| D07 | [Imports](../config.md#explicit-imports) introduce the three selectors, option applicability, requiredness, ordering, and earlier syntax separately. | +| D08 | [Placement](../config.md#input-destinations) illustrates files and assembled YAML; [contexts](../config.md#source-resource-contexts) explain later resource access and ownership. | +| D09 | [Identified lists](../config.md#identified-lists-a-worked-example) show replace, append, and whole-record upsert before registration and policy-lifetime details. | +| D10 | [Inheritance](../config.md#sibling-inheritance) starts with parent/child settings, then cancellation and identified records before exact scheduling rules. | +| D11 | [Versions](../config.md#application-schema-versions) introduce application acceptance, a mode table, mounted headers, and the limits of observed evidence. | +| D12 | [Callbacks](../config.md#recorded-application-resolution) have task and helper tables, a derived-default example, and entry-state/lifetime guidance. | +| D13 | [Secrets](../config.md#typed-secret-references) separate lookup, hiding, and bootstrap; [bootstrap](../config.md#explicit-secrets-directory-bootstrap) explains empty existing variables and partial reports. | +| D14 | [Inspection](../config.md#layered-metadata) starts with reader needs and a default/main/overlay example; [diffs](../config.md#effective-configuration-diffs) explain snapshot flags and hidden changes. | +| D15 | [YAML](../config.md#yaml-documents), [defaults](../config.md#defaults-and-typed-decoding), [projection](../config.md#typed-projection-and-ownership), and [failures](../config.md#safe-failures) have practical examples; descriptor caching moved to the [internal overview](../internal/overview.md). | +| D16 | [Package documentation](../../doc.go) and the targeted public declarations explain purpose, paths, returned copies, and saved state while retaining precise contracts. | +| D17 | [Migration guidance](../consumers/README.md#intentional-migration-changes) leads with numbered consumer actions and explains package type identity alongside the retained old/new table. | +| D18 | [Release procedure](../release.md) defines operational terms and explains Go selection, publication statuses, and tag-object recovery without changing command blocks or gates. | ## Audience and Scope @@ -400,14 +424,34 @@ The highest-impact starting points are the README and consumer entry routes (D13–D14). The other proposals develop a selected task or reference topic. These priorities are editorial recommendations, not additional release gates. -Later rewrites should preserve existing contracts, diagnostic identifiers, +These revisions preserve existing contracts, diagnostic identifiers, defaults, limits, security boundaries, and exceptions. Keep precise material available after its plain-language introduction. A simpler explanation must not imply universal last-value replacement, universal null support, transactional loading, refreshed inspection after caller mutation, or stronger filesystem isolation. -This pass changes only this catalog. No user-facing page, Go comment, example, -policy, command, test, or implementation has been revised. Later implementation -can validate changed examples and commands at their existing owners and inspect -rendered tables and navigation under the documentation policy. +Implementation changes consumer-facing prose and Go declaration comments, with +the relocated descriptor-caching explanation in the existing internal overview. +Policies, accepted ADRs, release history, license notices, dependencies, and +production behavior remain unchanged. Validation includes exercising the revised +illustrations against the library, checking local links and Markdown tables, +comparing Go tokens to confirm comment-only edits, and verifying that release +command blocks remain identical to the implementation baseline `e74c26a`. + +Completed validation with Go 1.26.10 and dependency downloads disabled: + +- `./scripts/check.sh` passed formatting, whitespace, all tests (including local + links and executable examples), vet, and build. +- Temporary programs outside the repository exercised the revised README, + callbacks, YAML, discovery, imports, collection operations, placement, + inheritance, version modes, inspection, secret bootstrap, defaults, and + projection illustrations against the current library; all passed. +- A Go scanner comparison confirmed identical non-comment tokens in all six + changed Go files. All 12 release command blocks are byte-for-byte unchanged. +- The six affected Markdown pages rendered successfully with the project's + Markdown parser; all 27 tables have consistent column counts. Existing + navigation anchors remain available. + +The final consistency pass checked terminology, worked results, public comments, +cross-page links, documentation ownership, and the preserved release workflow. diff --git a/inspection.go b/inspection.go index c4bdc63..7eaa36d 100644 --- a/inspection.go +++ b/inspection.go @@ -6,15 +6,18 @@ import ( "go.maximumdirect.net/configkit/internal/engine" ) -// ValueSnapshot is a detached, deterministic YAML representation of one value. -// Present distinguishes absence or deletion from an explicit null value. +// ValueSnapshot describes a saved value as consistently formatted YAML text. +// It is a copy; changing it cannot affect later inspection. Present is false for +// absence or deletion, while explicit null is present. Redacted reports whether +// the value or a descendant was hidden. type ValueSnapshot struct { Present bool Redacted bool YAML string } -// ValueChange describes one explicit definition or audited domain operation. +// ValueChange shows values before and after an input contribution or recorded +// application operation, together with its origin, reasons, and dependencies. type ValueChange struct { Before ValueSnapshot After ValueSnapshot @@ -29,7 +32,8 @@ type SecretReferenceInfo struct { Name string } -// Explanation is a detached safe account of one effective or historical path. +// Explanation describes a path's final value and earlier contributions, with +// sensitive values hidden. It is a copy that cannot alter later inspection. type Explanation struct { Path Path Effective ValueSnapshot @@ -38,9 +42,10 @@ type Explanation struct { SecretReference *SecretReferenceInfo } -// Explain returns safe effective and chronological values for a known final or -// historical path. Structural inheritance appears between source applications -// and typed reconciliation. Unknown and malformed paths return false. +// Explain returns the saved final value and earlier values for a known current +// or historical path, hiding sensitive information. History follows processing +// order: inheritance appears after source combination and before the transition +// to typed values. Unknown and malformed paths return false. func (r *Result[T]) Explain(path Path) (Explanation, bool) { if r == nil { return Explanation{}, false @@ -49,8 +54,9 @@ func (r *Result[T]) Explain(path Path) (Explanation, bool) { return fromEngineExplanation(value), ok } -// WriteEffective writes deterministic redacted YAML from the frozen resolved -// snapshot. The output is intended for inspection and is not promised reloadable. +// WriteEffective writes consistently formatted YAML for the saved final +// configuration, with sensitive values hidden. Later changes to Result.Config +// do not change the output. It is for inspection and is not promised reloadable. // Writer errors, including short writes, are returned and may leave partial output. func (r *Result[T]) WriteEffective(output io.Writer) error { if r == nil { @@ -61,7 +67,7 @@ func (r *Result[T]) WriteEffective(output io.Writer) error { } // WriteExplanation writes a deterministic human-readable rendering of Explain. -// Unknown paths produce no output. Writer failures may leave partial output. +// Unknown paths return an error without output. Writer failures may leave partial output. func (r *Result[T]) WriteExplanation(output io.Writer, path Path) error { if r == nil { var inspection *engine.Inspection diff --git a/lifecycle.go b/lifecycle.go index 527f9fd..2b2b28c 100644 --- a/lifecycle.go +++ b/lifecycle.go @@ -6,14 +6,19 @@ import ( "go.maximumdirect.net/configkit/internal/engine" ) -// Definition contains reusable application rules for a configuration type. -// Sensitive is a pure predicate that can add sensitivity to semantic paths. -// Definitions should be treated as read-only after being shared. +// Definition supplies the defaults and application rules used when loading T. +// Treat a definition as read-only after sharing it. Sensitive is a pure predicate +// that can mark additional configuration paths for hiding in inspection output. type Definition[T any] struct { - Defaults func() T - Resolve func(*T, *Resolution) error - Validate func(*T, *Validation) - Version *VersionRule + // Defaults returns starting values for settings omitted from input. + Defaults func() T + // Resolve derives or normalizes values and records those changes. + Resolve func(*T, *Resolution) error + // Validate reports application errors and warnings without changing values. + Validate func(*T, *Validation) + // Version chooses a version path and the application's acceptance rule. + Version *VersionRule + // Sensitive adds paths to those already hidden by configkit:"secret" tags. Sensitive func(Path) bool // SequenceIdentities declares the scalar fields used to combine records and // resolve record inheritance at selected sequence destinations. Registration @@ -132,9 +137,10 @@ type Override struct { Flag string } -// Resolution exposes entry-state input provenance, surviving-input presence, and -// audited application resolution changes. A resolver must not retain or -// concurrently use the context. +// Resolution lets a resolver inspect supplied values and record derived defaults +// or normalization. Its input queries describe the state at callback entry, +// even after the resolver changes the configuration. Use the context only during +// the callback; do not retain it or use it concurrently. type Resolution struct{ value *engine.Resolution } // VersionRule declares the input path and application checker for a schema version. @@ -155,21 +161,27 @@ type Version struct { Text string } -// Validation collects safe application diagnostics during a supported callback. -// A callback must not retain the context or use it concurrently. +// Validation lets an application report value-free errors and warnings during +// a version or validation callback. Use it only during the callback; do not +// retain the context or use it concurrently. type Validation struct{ value *engine.Validation } -// Result is a successful independently owned configuration load. +// Result contains a successful load's configuration and saved inspection state. +// The application owns Config and may change it. Inspection continues to describe +// the completed load; it does not follow later changes to Config. Separate loads +// return independently owned results. type Result[T any] struct { Config T inspection *engine.Inspection } -// Load parses and strictly decodes zero or one selected source, explicitly enabled -// imports, and one selected overlay. After source precedence, built-in sibling -// `extends` relationships resolve before defaults and typed decoding. It accepts -// at most one Options value and returns no result when any error is reported. +// Load reads the selected input and returns a configuration of type T with +// inspection metadata. NoSource deliberately omits the main document. Enabled +// imports and one selected overlay participate in loading. Sources combine first, +// then sibling extends relationships resolve before defaults and strict typed +// decoding. Load accepts at most one Options value and returns a nil result when +// any error is reported. See docs/config.md for the complete processing order. func Load[T any](source Source, supplied ...Options[T]) (*Result[T], error) { options := make([]engine.Options[T], len(supplied)) for index := range supplied { @@ -190,7 +202,8 @@ func fromEngineLoaded[T any](loaded *engine.Loaded[T], err error) (*Result[T], e return &Result[T]{Config: loaded.Config, inspection: loaded.Inspection}, nil } -// Metadata returns detached effective presence, origin, and history records. +// Metadata provides queries for final input presence, value origins, and history. +// Records returned by its queries are copies that cannot change the saved load. func (r *Result[T]) Metadata() Metadata { if r == nil || r.inspection == nil { return Metadata{} @@ -198,7 +211,8 @@ func (r *Result[T]) Metadata() Metadata { return Metadata{value: r.inspection.Metadata()} } -// Sources returns a detached list of sources used by the load. +// Sources returns a copied list of sources used by the load. Changing the list +// or its nested records cannot change later queries. func (r *Result[T]) Sources() []SourceInfo { if r == nil || r.inspection == nil { return nil diff --git a/path.go b/path.go index a265963..fc94991 100644 --- a/path.go +++ b/path.go @@ -4,8 +4,12 @@ import ( "go.maximumdirect.net/configkit/internal/engine" ) -// Path identifies a configuration value using YAML field names. Its textual form -// is a JSON Pointer: the root is empty, ~ escapes as ~0, and / escapes as ~1. +// Path names a value inside configuration. For example, Keys("servers").Index(0) +// names the first server, while Keys("servers", "0") names a literal mapping key. +// They have the same pointer text but distinct key/index intent. +// +// Its textual form is a JSON Pointer: the root is empty, ~ escapes as ~0, and / +// escapes as ~1. // Paths are semantic rather than filesystem or URL paths: empty, dot, dot-dot, // percent-containing, and numeric keys retain their literal meaning. Parsed // numeric tokens are resolved only when traversal knows their containing type. diff --git a/records.go b/records.go index 3d41432..683773f 100644 --- a/records.go +++ b/records.go @@ -180,10 +180,10 @@ type Location struct { Column int } -// SourceInfo describes a source discovered during a load. Kind identifies its -// acquisition, independent of Layer. Path is canonical for a host file and +// SourceInfo describes one input acquired during loading. Kind says how it was +// read; Layer says its priority role. Path is canonical for a host file and // namespace-relative for a filesystem file; synthetic sources have no Path. -// Destination is the effective mapping path where this source contributes. +// Destination says where the source contributes in the assembled configuration. // UnderlayIndex is the one-based original underlay position, or zero for other // sources. ImportedFrom identifies the declaration that included this source. type SourceInfo struct { @@ -240,12 +240,14 @@ type EmptyImportSelection struct { Selector ImportSelector } -// SourceVersionEvidence describes a source-local version declaration observed -// during an enabled load. A non-nil record with Present false records observed -// absence, but does not imply that a checker ran or accepted a value. Text is -// empty when sensitivity classification or conservative callback-time/failed- -// load redaction withholds it; Present and Null retain their meaning. Copies -// returned by source queries are detached. See docs/config.md for disclosure +// SourceVersionEvidence records an observed version declaration in one input +// document when source-version checks are enabled. A non-nil record with +// Version.Present false records absence; it does not mean a checker ran or +// accepted a value. Version.Text is empty when final sensitivity classification +// or conservative masking during callbacks or failed loads hides it. +// Version.Present and Version.Null retain their meaning. +// Source queries return copies you can change without changing stored evidence. +// See docs/config.md for disclosure // and lifecycle details. type SourceVersionEvidence struct { Path Path @@ -265,9 +267,9 @@ type Diagnostic struct { Related []Location } -// InheritanceStep identifies a concrete payload transfer. From and To name -// semantic value paths; Reference names the controlling destination `extends` -// path. Locations belong to that directive, not the transferred value. +// InheritanceStep identifies values copied from From to To by inheritance. +// These are configuration paths. Reference names the controlling destination +// extends path; Locations identify that directive rather than the copied value. type InheritanceStep struct { From Path To Path @@ -275,10 +277,10 @@ type InheritanceStep struct { Locations []Location } -// Origin describes where an effective definition came from. Multiple locations -// represent a composite origin; resolution origins are synthetic and identify their -// operation instead of a file location. Inheritance records the concrete -// derivation path while Layer and Locations continue to identify value sources. +// Origin describes where a final value came from. Multiple locations identify +// contributors to a combined value. Resolution identifies an application operation +// instead of a file location. Inheritance describes the copying steps, while +// Layer and Locations still identify the value's sources. type Origin struct { Layer Layer Locations []Location @@ -286,7 +288,8 @@ type Origin struct { Inheritance []InheritanceStep } -// FieldInfo describes an effective configuration path. +// FieldInfo describes a final configuration path: whether input supplied it, +// whether it is null, and where it came from. type FieldInfo struct { Explicit bool Null bool