Document and validate configured secrets bootstrap
This commit is contained in:
129
docs/config.md
129
docs/config.md
@@ -1019,7 +1019,7 @@ There are three separate tasks when working with credentials:
|
||||
| --- | --- |
|
||||
| Read a credential from an environment variable | A typed `Secret` field whose YAML value is the variable's name |
|
||||
| Hide a configuration value in inspection output | A `configkit:"secret"` tag or `Definition.Sensitive` |
|
||||
| Populate environment variables from local files before loading | An explicit `LoadSecretsIntoEnvironment` call |
|
||||
| Populate environment variables from local files before loading | An explicit [secrets bootstrap](#explicit-secrets-directory-bootstrap) call |
|
||||
|
||||
For example, YAML `api_key: DEMO_API_KEY` names an environment variable. Its
|
||||
credential is available to trusted application code through `Secret.Value()`
|
||||
@@ -1051,8 +1051,8 @@ its effective source location for diagnostics. Each unique name is read with
|
||||
`os.LookupEnv` once per load. Validation can inspect the credential through the
|
||||
deliberately named `Value()` accessor and distinguish success with `IsResolved()`.
|
||||
Later loads read the environment again, and cloned defaults are rebuilt from names
|
||||
so a previously resolved result cannot retain a stale credential. `Load` never
|
||||
writes to the environment.
|
||||
so a previously resolved result cannot retain a stale credential. `Load` and
|
||||
`LoadLayered` never write to the environment.
|
||||
|
||||
Credential bytes are held only in the per-load lookup state and the trusted returned
|
||||
configuration, and are returned only by `Value`. Secret formatting uses a safe
|
||||
@@ -1078,17 +1078,80 @@ reference state without printing its value.
|
||||
|
||||
## Explicit secrets-directory bootstrap
|
||||
|
||||
If an application stores credentials in files, it can call this helper before
|
||||
loading configuration. A file named `DEMO_API_KEY` containing the dummy text
|
||||
If an application stores credentials in files, it can explicitly bootstrap
|
||||
them before loading configuration. A file named `DEMO_API_KEY` containing the dummy text
|
||||
`example-value` supplies environment variable `DEMO_API_KEY`. YAML can then
|
||||
refer to that variable with `api_key: DEMO_API_KEY`. Applications that already
|
||||
receive environment variables do not need this file step.
|
||||
|
||||
Existing environment variables take priority over files. If `DEMO_API_KEY` is
|
||||
already set to an empty string, its file is skipped and it stays empty. A later
|
||||
By default existing environment variables take priority over files. If
|
||||
`DEMO_API_KEY` is already set to an empty string, its file is skipped and it stays empty. A later
|
||||
required `Secret` reference will still fail on that empty value. File bootstrap
|
||||
accepting an empty file is different from a required reference resolving to an
|
||||
empty credential.
|
||||
empty credential. `LoadSecretsIntoEnvironmentWithOptions` accepts
|
||||
`SecretsOptions{Overwrite: true}` to replace existing values explicitly.
|
||||
|
||||
For a directory selected by the main file, call
|
||||
`LoadSecretsIntoEnvironmentFromConfig` with a host `File` or `SearchConfig`
|
||||
source before `Load` or `LoadLayered`. The main document may declare:
|
||||
|
||||
```yaml
|
||||
config:
|
||||
secrets_dir: secrets
|
||||
secrets_overwrite: false
|
||||
```
|
||||
|
||||
`secrets_dir` is a nonempty relative host path, interpreted literally relative
|
||||
to the selected main file's canonical directory. Its canonical target must be
|
||||
inside the main filesystem root; a contained directory symlink is allowed.
|
||||
Parent-relative paths can reach other directories inside an explicitly wider
|
||||
root. Rooted, absolute, and host volume-qualified paths are invalid. `secrets_overwrite`
|
||||
is a Boolean and defaults to false. A non-nil Go `Overwrite` option wins over
|
||||
the YAML value, including an explicit false. Invalid declarations fail even
|
||||
when Go supplies an override. Overwrite without a directory is a no-op.
|
||||
|
||||
Only the main document root owns these controls. Imports and underlays reject
|
||||
them with source-local import diagnostics; selected overlays reject them as
|
||||
overlay directives. Nested application `config` mappings and data assembled at
|
||||
the application root by placement, defaults, CLI, or resolution remain ordinary
|
||||
data. Normal `Load` and `LoadLayered` validate and consume main controls whether
|
||||
imports are enabled or disabled, but never bootstrap or write the environment.
|
||||
Each load resolves typed secret names from the environment at the usual point.
|
||||
|
||||
The companion accepts at most one `SecretsFromConfigOptions`, including an
|
||||
optional host `FilesystemContext`. It selects and parses the main once, validates
|
||||
the bootstrap controls, and completes directory bootstrap before returning a
|
||||
prepared source. It does not run import acquisition, version checks, decoding,
|
||||
resolution, or validation. It loads every eligible directory entry, including
|
||||
names unused by the application's typed schema. A main with no directory still
|
||||
returns a prepared source and empty report. `NoSource` or an exhausted optional
|
||||
search returns `NoSource` and an empty report. Explicit absent-input context
|
||||
continues to follow the normal absent-host-context rules. The helper also accepts
|
||||
a previously prepared source, reusing its captured main and explicitly performing
|
||||
bootstrap again.
|
||||
|
||||
The prepared source is a reusable snapshot of the selected main document,
|
||||
including its unconsumed controls, byte count, canonical file identity, base,
|
||||
and containment root. A later load neither rediscovers nor rereads the main
|
||||
and does not repeat bootstrap; it reads imports and resources when accessed and
|
||||
looks up secrets anew. The main's source record retains `SourceKindFile`. Each
|
||||
load clones the document, creates its own source IDs
|
||||
and records, and applies normal input accounting. No file handle or secret-file
|
||||
contents remain in the prepared source. With no later context, loading uses the
|
||||
captured context. A supplied later host context must resolve to the same root,
|
||||
base, and main path; equivalent path spellings and root symlink aliases are
|
||||
accepted. Native contexts, a file `BaseDir`, and different roots fail with
|
||||
`options.invalid`. A caller needing another root prepares the original source
|
||||
again. These rules apply to ordinary options and layered main inputs.
|
||||
|
||||
The helper returns a non-nil report on every attempt and a zero, unusable source
|
||||
on error. Discovery, source, parse, control, and preparation-context failures
|
||||
return `*LoadError` with available source records and source-local diagnostics.
|
||||
Directory inspection, reads, and environment writes return `*SecretsError`.
|
||||
An invalid directory declaration uses `secrets.invalid_path`; a non-Boolean
|
||||
overwrite declaration uses `secrets.invalid_value`. Later loading failures do
|
||||
not undo bootstrap effects or change its report. Multiple options fail before
|
||||
acquisition or writes.
|
||||
|
||||
Read the report even when the helper returns an error:
|
||||
|
||||
@@ -1101,18 +1164,20 @@ Read the report even when the helper returns an error:
|
||||
|
||||
Render the returned `SecretsReport` separately from a later configuration load.
|
||||
It contains filenames, paths, and statuses, not credential contents. The
|
||||
[maintained startup example](../examples/consumers/consumers_test.go) shows how to
|
||||
keep bootstrap and configuration reports together in the application.
|
||||
[configured startup example](../examples/consumers/secrets.go) prepares a source
|
||||
for either loading entry point; its [test](../examples/consumers/secrets_test.go)
|
||||
shows both handoffs. The [direct-directory example](../examples/consumers/consumers_test.go)
|
||||
shows how to keep bootstrap and configuration reports together.
|
||||
|
||||
### Preflight, limits, and partial writes
|
||||
|
||||
`LoadSecretsIntoEnvironment(directory)` is a separate, opt-in startup operation.
|
||||
It is never called by `Load`, is not a YAML option, and does not participate in
|
||||
configuration validation. Applications choose the native directory path and
|
||||
should invoke the operation once before starting concurrent work. The selected
|
||||
path may itself be a symlink; ConfigKit resolves it to an existing canonical
|
||||
directory before examining
|
||||
its immediate entries.
|
||||
All bootstrap helpers are separate, opt-in startup operations. They are never
|
||||
called by a configuration load and do not perform configuration validation.
|
||||
Applications own startup ordering and should invoke bootstrap before concurrent
|
||||
work or other process-environment mutation. The direct helpers accept a native
|
||||
directory path; the configured helper resolves the declared path as described
|
||||
above. The selected path may itself be a symlink; ConfigKit resolves it to an
|
||||
existing canonical directory before examining its immediate entries.
|
||||
|
||||
The bootstrap enumerates at most 1,024 entries, including dot-prefixed entries.
|
||||
Entries whose names begin with `.` (files, directories, or symlinks, such as `.git`
|
||||
@@ -1120,16 +1185,18 @@ and `.gitignore`) are ignored before name or metadata validation, are never open
|
||||
and are omitted from the report. Remaining entries are sorted by filename; each
|
||||
must have an ASCII environment-variable name and be a regular, nonsymlink
|
||||
file. Metadata for all remaining entries is validated before file content is read
|
||||
or process state is changed. Existing environment variables win, including variables whose
|
||||
value is empty; their files are not opened. Each eligible file is limited to 1 MiB,
|
||||
and eligible content is limited to 8 MiB in total. ConfigKit removes exactly one
|
||||
or process state is changed. In preservation mode existing environment variables
|
||||
win, including variables whose value is empty; their files are not opened. In
|
||||
overwrite mode their files are read and applied as well. Each eligible file is
|
||||
limited to 1 MiB, and eligible content is limited to 8 MiB in total. ConfigKit removes exactly one
|
||||
terminal LF or CRLF, preserves all other whitespace, accepts an empty value, and
|
||||
rejects NUL bytes.
|
||||
|
||||
All eligible files are read successfully before application begins. Presence is
|
||||
checked again immediately before each `os.Setenv`: a variable populated since
|
||||
preflight is skipped. A write failure stops application without rollback, so prior
|
||||
writes remain and later eligible entries are `not_applied`. This operation is not
|
||||
All required files are read successfully before application begins. In
|
||||
preservation mode presence is checked again immediately before each `os.Setenv`:
|
||||
a variable populated since preflight is skipped. Overwrite mode applies the
|
||||
staged value regardless of presence. A write failure stops application without
|
||||
rollback, so prior writes remain and later eligible entries are `not_applied`. This operation is not
|
||||
atomic and does not provide compare-and-set semantics for concurrent environment
|
||||
mutation.
|
||||
|
||||
@@ -1137,9 +1204,11 @@ mutation.
|
||||
records with `loaded`, `skipped_existing`, `not_applied`, or `failed` status.
|
||||
`ExistingEmpty` distinguishes an empty existing winner. `SecretsReport.WriteTo`
|
||||
renders only quoted names, canonical paths, statuses, and that empty-state flag;
|
||||
it never retains or renders file contents. Failures return the non-nil partial
|
||||
report with a `*SecretsError` containing detached `secrets.*` diagnostics and no
|
||||
raw filesystem or environment error chain. Applications should retain the report
|
||||
it never retains or renders file contents. Directory and environment failures
|
||||
return the non-nil partial report with a `*SecretsError` containing detached
|
||||
`secrets.*` diagnostics and no raw filesystem or environment error chain.
|
||||
Configured main-preparation failures return `*LoadError` as described above.
|
||||
Applications should retain the report
|
||||
alongside the later configuration result and explicitly present blocked and failed
|
||||
files when appropriate; ConfigKit stores no implicit global report.
|
||||
|
||||
@@ -1674,8 +1743,10 @@ rules below explain absence, patterns, placement, and ordering.
|
||||
### Declaration and selection rules
|
||||
|
||||
Imports are opt-in. With `Options.Imports` enabled, a document may use its root
|
||||
`config` mapping for `imports` and `sequences` controls. Ordinary `Load` may use
|
||||
`sequences` without enabling imports, but an `imports` field still requires
|
||||
`config` mapping for `imports` and `sequences` controls. The main root may also
|
||||
declare the [secrets bootstrap controls](#explicit-secrets-directory-bootstrap).
|
||||
Ordinary `Load` may use `sequences` and main bootstrap controls without enabling
|
||||
imports, but an `imports` field still requires
|
||||
`Options.Imports`. ConfigKit removes recognized root controls before application
|
||||
decoding. Each imports sequence item is a mapping with exactly one `path`,
|
||||
`dir`, or `glob` selector. `path` and `dir` accept a string or a list of strings;
|
||||
|
||||
Reference in New Issue
Block a user