Revise secrets roadmap around explicit configuration bootstrap

This commit is contained in:
2026-11-17 22:28:16 +00:00
parent 20a1087e25
commit e1336f74e5

View File

@@ -9,34 +9,49 @@ argument uses the process working directory. Using a directory beside the
discovered file therefore requires the application to take over main-file discovered file therefore requires the application to take over main-file
selection or perform a preliminary load. [Configuration discovery](../config.md#configuration-discovery) · [Secrets bootstrap](../config.md#explicit-secrets-directory-bootstrap) selection or perform a preliminary load. [Configuration discovery](../config.md#configuration-discovery) · [Secrets bootstrap](../config.md#explicit-secrets-directory-bootstrap)
ConfigKit already carries the selected main's filesystem context into layered ConfigKit already retains filesystem context for acquired main files and binds
loading. It resolves effective typed `Secret` references after application companion inputs to that context. Typed `Secret` references read the environment
resolution and before final validation. Those two points provide the base for after application resolution and before final validation. A companion bootstrap
the directory and the set of credential names to read. [Main-relative contexts](../adr/0022-main-relative-context.md) · [Typed secret references](../config.md#typed-secret-references) helper can use the main's context before loading while preserving this
processing order. [Main-relative contexts](../adr/0022-main-relative-context.md) · [Typed secret references](../config.md#typed-secret-references)
## Target behavior ## Target behavior
### Declaration and path resolution ### Declaration and path resolution
An application opts in through Go when calling `Load` or `LoadLayered`. The An application explicitly calls a companion to
selected main document supplies one relative directory through its root control `LoadSecretsIntoEnvironment` before `Load` or `LoadLayered`. The selected main
document supplies the directory and overwrite policy through its root control
mapping. For example, the following declaration in `/etc/demo/config.yml` mapping. For example, the following declaration in `/etc/demo/config.yml`
selects `/etc/demo/secrets` when discovery selects that main file. selects `/etc/demo/secrets` and authorizes replacement of existing variables
when the companion helper is called.
```yaml ```yaml
config: config:
secrets_dir: secrets secrets_dir: secrets
secrets_overwrite: true
api_key: DEMO_API_KEY api_key: DEMO_API_KEY
``` ```
Interpret the declaration at the main document's root before placement and Interpret these fields at the main document's root before placement and
composition. Diagnose a declaration used without opt-in or in an unsupported composition. They are bootstrap settings consumed as document controls.
source-control context. Nested application data follows the existing `Load` and `LoadLayered` recognize the controls and perform their usual
`config` rules. The location comes from the main document even when an import, environment lookup without invoking bootstrap. The companion helper call is
overlay, CLI override, default, or application resolution supplies the final the explicit authorization for environment writes.
credential name. Environment lookup alone applies when there is no declaration,
including when an optional search finds no main file. Only the main document selects the bootstrap directory and YAML overwrite
policy. Diagnose these controls in unsupported source-control contexts.
Nested application data follows the existing `config` rules. Imports,
underlays, overlays, CLI overrides, and application resolution retain their
ordinary effects on application configuration. They do not redirect this
bootstrap operation.
`secrets_overwrite` is an optional Boolean with a default of `false`.
An explicitly supplied Go overwrite choice takes precedence over YAML,
including when that choice is `false`. If the main supplies no directory,
bootstrap is a no-op and the prepared main can still be loaded. An optional
search that finds no main preserves the existing absent-main lifecycle.
Resolve the directory from the selected main file's canonical base within its Resolve the directory from the selected main file's canonical base within its
existing filesystem root. For a discovered symlink, the base is the selected existing filesystem root. For a discovered symlink, the base is the selected
@@ -45,102 +60,115 @@ containment rule as other main-relative inputs. The declaration must be a
nonempty relative path and requires a usable main-file context. Keep the nonempty relative path and requires a usable main-file context. Keep the
declaration and any diagnostics in source-local coordinates. [Document-root controls](../adr/0020-document-root-controls.md) · [Layered loading](../config.md#layered-loading) declaration and any diagnostics in source-local coordinates. [Document-root controls](../adr/0020-document-root-controls.md) · [Layered loading](../config.md#layered-loading)
### Lookup within a load ### Companion helper and prepared main source
The usual opt-in mode checks the environment first. A present variable wins, The companion accepts the main `Source`, including a `SearchConfig` source,
including an empty one, which produces the existing empty-secret failure. For and returns a secrets report and a prepared main source. The public name and
an unset variable, read the file whose name matches the effective typed option type below are illustrative.
`Secret` reference. The directory is accessed only if a reference needs a file.
An application can also select file-first lookup for a load. An absent file ```go
allows environment lookup. A file that is present but unreadable or invalid main := configkit.SearchConfig("demo", search)
produces a safe diagnostic, and an empty file follows the existing rule that a report, preparedMain, err := configkit.LoadSecretsIntoEnvironmentFromConfig(main)
resolved credential must be nonempty. If neither source supplies a credential, if err != nil {
report the failure at the affected configuration field. // Handle the bootstrap failure and inspect report.
return err
}
result, err := configkit.Load[AppConfig](preparedMain)
```
Resolve surviving references at the existing secret-resolution step and share For layered loading, supply the prepared source as the main `Input.Source`.
the lookup result for repeated names within that load. Use the existing The helper resolves discovery once, reads the selected main document, extracts
secret-file checks, content rules, and per-file and cumulative byte limits for the bootstrap controls, and performs bootstrap. It returns the captured main
the files read. Exact-name lookup leaves unrelated directory entries alone. contents with the selected file's identity and filesystem context so the later
Close library-owned handles during the load. Final validation sees the load consumes that same main document. Keep its remaining controls, including
resolved credentials, with byte access through `Secret.Value()`. Inspection imports and collection declarations, available to normal loading.
can distinguish the environment and a file as the credential provider while
keeping credential bytes outside diagnostics, history, and effective YAML.
### Explicit startup export The helper prepares only the main document. Composition, import acquisition,
inheritance, typed decoding, application resolution, secret lookup, and
validation run through the subsequent ordinary or layered load. Main
preparation must retain the applicable source limits and root constraints.
Source records, diagnostics, placements, imports, and main-relative companions
use the selected main's identity and context.
Applications that start SDKs or subprocesses which read the environment can Release library-owned file handles before the helper returns, including on
explicitly request file-first loading and environment export. This is a error. Prepared sources retain captured document data and passive filesystem
startup operation distinct from the ordinary load entry points. Go code context and follow the source reuse contract without sharing mutable load
authorizes writes and overwrites. The main-file declaration selects the state. Later imports and resources observe the filesystem when accessed. A
directory. later load reads the environment at its normal secret-resolution step.
[Complete-on-return loading](../adr/0008-complete-on-return-loading.md) · [Source resource contexts](../config.md#source-resource-contexts)
Complete configuration loading and validation before writing to the process ### Shared bootstrap and overwrite behavior
environment. Export the values of the file-backed effective references used
by that successful load, once per name. Each successful write sets the
variable to the same value used by the returned configuration. If a file is
absent and lookup uses an existing environment value, leave that variable alone.
Invoke this operation during startup before starting concurrent work. It runs
synchronously and reports successful writes and partial failures without
credential bytes, following the existing bootstrap reporting model. ConfigKit
does not roll back earlier writes when a later write fails.
Ordinary `Load` and `LoadLayered` resolve credentials for their own results Extend the existing direct-directory helper with an optional named Boolean
without writing to the environment. The existing option, keeping existing calls valid. The option type name in this example is
`LoadSecretsIntoEnvironment(directory)` remains the explicit startup helper provisional.
for callers supplying a directory independently. [Complete-on-return loading](../adr/0008-complete-on-return-loading.md) · [Secret references and explicit bootstrap](../adr/0004-secret-references-and-explicit-bootstrap.md)
## Work needed ```go
report, err := configkit.LoadSecretsIntoEnvironment("/etc/demo/secrets")
report, err = configkit.LoadSecretsIntoEnvironment(
"/etc/demo/secrets",
configkit.SecretsOptions{Overwrite: true},
)
```
- Extend main-document control parsing and the public load options to capture Both helpers use the same directory-bootstrap implementation. With overwrite
the opted-in declaration and bind it to the main context already obtained disabled, existing variables win, including empty ones, with presence checked
during acquisition. Carry the binding into secret resolution without another again immediately before application as today. With overwrite enabled,
search or parse. eligible files are read even for existing variables and their values replace
- Reuse the secret-file reading and validation rules for exact-name lookup. those variables. Variables without a corresponding file are left alone.
Extend the operation-local lookup to report value-free filesystem failures An applied empty file supplies an empty value in either mode.
and retain the provider for inspection separately from configuration value
provenance.
- Add the explicit startup operation around the shared load lifecycle. Use the
resolved file values for export and reuse the bootstrap reporting semantics
where they fit.
- Verify discovery from a different working directory, canonical main paths,
declaration scope, both lookup orders, and effective references after
composition and application resolution. Cover unused and missing files,
empty values, read errors, safe inspection, and startup export after
validation, including partial writes and absence of writes on a failed load.
Use disposable credential values and deterministic failure seams in line
with the [testing policy](../policy/testing.md).
- Document the resulting contract and a consumer startup example. Record the
architectural decision to add file-backed resolution and explicit startup
export alongside ADR 0004, and align the current architecture and component
documentation when the feature lands.
## Open Decisions Bootstrap processes the directory's eligible entries, including credentials
used by SDKs or subprocesses outside typed `Secret` fields. Reuse the existing
dot-entry handling, file checks, content rules, limits, and filename ordering.
Validate metadata and read all content needed by the chosen overwrite policy
before beginning writes.
Directory and file failures produce the existing safe reporting behavior.
The secrets report identifies writes, skips, and partial failures without
retaining credential bytes.
### Startup API and export failure reporting Call bootstrap during startup before starting concurrent work. Successful
bootstrap precedes configuration loading and validation. A later load failure
leaves the completed environment writes in place. A bootstrap write failure
stops further writes, leaves earlier writes in place, and reports remaining
entries as `not_applied`. The existing lack of rollback and concurrent
environment transaction guarantees continues to apply.
**Recommendation.** Provide explicit startup entry points corresponding to Keep bootstrap reporting separate from the subsequent configuration's secret
ordinary and layered loading, backed by the same load implementation. Return lookup evidence. Each operation describes what it observed and performed.
the configuration outcome and the export report so callers can distinguish a [Secrets bootstrap](../config.md#explicit-secrets-directory-bootstrap) · [Secret references and explicit bootstrap](../adr/0004-secret-references-and-explicit-bootstrap.md)
failed load from a successful load followed by an environment write failure.
The successful load's private file values can be handed directly to export
within the call, which keeps the application flow short and the lifetime of
those values local to the startup operation.
**Alternative.** Export from a successfully returned result through a separate
explicit call. This gives applications control over when to perform the writes
and avoids a parallel pair of load entry points. It requires a result contract
for retaining the original file-backed assignments or defining how later
application edits to the result affect export.
### Filesystem scope ### Filesystem scope
**Recommendation.** Support discovered and explicitly selected host files Support discovered and explicitly selected host files, including their existing
first, including their existing main-root options. These cover the requested main-root options. This covers the requested deployment pattern and reuses the
deployment pattern and allow reuse of the existing native secret-file checks. native secret-file checks. Caller-supplied filesystem support is a possible
later extension requiring decisions about file-type and symlink capabilities.
**Alternative.** Support caller-supplied filesystems with a usable main-file ## Work needed
context in the same change. This aligns with companion inputs and supports
virtual filesystems. It requires an explicit decision about file-type and - Add `config.secrets_dir` and `config.secrets_overwrite` to the main-document
symlink handling where a caller filesystem exposes different metadata or control grammar and consume them during ordinary loading without triggering
access capabilities. bootstrap. Retain source-local positions for diagnostics.
- Factor main-file selection and capture for use by the companion helper and
normal acquisition. Add a prepared-source handoff with retained main contents,
identity, context, and ownership. Select an API for supplying applicable
main-root constraints and representing an absent or explicit Go overwrite
choice, including `false`.
- Add the optional overwrite setting to direct-directory bootstrap and reuse
that operation from the companion helper, with matching reports and failures.
- Verify discovery from a different working directory, canonical main paths,
optional search exhaustion, declaration scope, default and explicit
overwrite choices, Go-over-YAML precedence, and both helper interfaces.
Cover empty values, dot entries, read failures before writes, and partial
write failures. Verify bootstrap includes entries unused by typed fields and
that later load failures preserve bootstrap effects.
- Verify prepared-source reuse, handle cleanup, normal loading without
environment writes, and agreement with selected-file provenance, placement,
relative imports, and main-relative companions. Use disposable credential
values and deterministic failure seams in line with the
[testing policy](../policy/testing.md).
- Document the startup workflow and add a consumer example. Record the prepared
main-source handoff and optional overwrite policy in an ADR referencing
ADR 0004, and update the canonical configuration contract and affected
architecture and component documentation when implemented.