Revise secrets roadmap around explicit configuration bootstrap
This commit is contained in:
@@ -9,34 +9,49 @@ argument uses the process working directory. Using a directory beside the
|
|||||||
discovered file therefore requires the application to take over main-file
|
discovered file therefore requires the application to take over main-file
|
||||||
selection or perform a preliminary load. [Configuration discovery](../config.md#configuration-discovery) · [Secrets bootstrap](../config.md#explicit-secrets-directory-bootstrap)
|
selection or perform a preliminary load. [Configuration discovery](../config.md#configuration-discovery) · [Secrets bootstrap](../config.md#explicit-secrets-directory-bootstrap)
|
||||||
|
|
||||||
ConfigKit already carries the selected main's filesystem context into layered
|
ConfigKit already retains filesystem context for acquired main files and binds
|
||||||
loading. It resolves effective typed `Secret` references after application
|
companion inputs to that context. Typed `Secret` references read the environment
|
||||||
resolution and before final validation. Those two points provide the base for
|
after application resolution and before final validation. A companion bootstrap
|
||||||
the directory and the set of credential names to read. [Main-relative contexts](../adr/0022-main-relative-context.md) · [Typed secret references](../config.md#typed-secret-references)
|
helper can use the main's context before loading while preserving this
|
||||||
|
processing order. [Main-relative contexts](../adr/0022-main-relative-context.md) · [Typed secret references](../config.md#typed-secret-references)
|
||||||
|
|
||||||
## Target behavior
|
## Target behavior
|
||||||
|
|
||||||
### Declaration and path resolution
|
### Declaration and path resolution
|
||||||
|
|
||||||
An application opts in through Go when calling `Load` or `LoadLayered`. The
|
An application explicitly calls a companion to
|
||||||
selected main document supplies one relative directory through its root control
|
`LoadSecretsIntoEnvironment` before `Load` or `LoadLayered`. The selected main
|
||||||
|
document supplies the directory and overwrite policy through its root control
|
||||||
mapping. For example, the following declaration in `/etc/demo/config.yml`
|
mapping. For example, the following declaration in `/etc/demo/config.yml`
|
||||||
selects `/etc/demo/secrets` when discovery selects that main file.
|
selects `/etc/demo/secrets` and authorizes replacement of existing variables
|
||||||
|
when the companion helper is called.
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
config:
|
config:
|
||||||
secrets_dir: secrets
|
secrets_dir: secrets
|
||||||
|
secrets_overwrite: true
|
||||||
|
|
||||||
api_key: DEMO_API_KEY
|
api_key: DEMO_API_KEY
|
||||||
```
|
```
|
||||||
|
|
||||||
Interpret the declaration at the main document's root before placement and
|
Interpret these fields at the main document's root before placement and
|
||||||
composition. Diagnose a declaration used without opt-in or in an unsupported
|
composition. They are bootstrap settings consumed as document controls.
|
||||||
source-control context. Nested application data follows the existing
|
`Load` and `LoadLayered` recognize the controls and perform their usual
|
||||||
`config` rules. The location comes from the main document even when an import,
|
environment lookup without invoking bootstrap. The companion helper call is
|
||||||
overlay, CLI override, default, or application resolution supplies the final
|
the explicit authorization for environment writes.
|
||||||
credential name. Environment lookup alone applies when there is no declaration,
|
|
||||||
including when an optional search finds no main file.
|
Only the main document selects the bootstrap directory and YAML overwrite
|
||||||
|
policy. Diagnose these controls in unsupported source-control contexts.
|
||||||
|
Nested application data follows the existing `config` rules. Imports,
|
||||||
|
underlays, overlays, CLI overrides, and application resolution retain their
|
||||||
|
ordinary effects on application configuration. They do not redirect this
|
||||||
|
bootstrap operation.
|
||||||
|
|
||||||
|
`secrets_overwrite` is an optional Boolean with a default of `false`.
|
||||||
|
An explicitly supplied Go overwrite choice takes precedence over YAML,
|
||||||
|
including when that choice is `false`. If the main supplies no directory,
|
||||||
|
bootstrap is a no-op and the prepared main can still be loaded. An optional
|
||||||
|
search that finds no main preserves the existing absent-main lifecycle.
|
||||||
|
|
||||||
Resolve the directory from the selected main file's canonical base within its
|
Resolve the directory from the selected main file's canonical base within its
|
||||||
existing filesystem root. For a discovered symlink, the base is the selected
|
existing filesystem root. For a discovered symlink, the base is the selected
|
||||||
@@ -45,102 +60,115 @@ containment rule as other main-relative inputs. The declaration must be a
|
|||||||
nonempty relative path and requires a usable main-file context. Keep the
|
nonempty relative path and requires a usable main-file context. Keep the
|
||||||
declaration and any diagnostics in source-local coordinates. [Document-root controls](../adr/0020-document-root-controls.md) · [Layered loading](../config.md#layered-loading)
|
declaration and any diagnostics in source-local coordinates. [Document-root controls](../adr/0020-document-root-controls.md) · [Layered loading](../config.md#layered-loading)
|
||||||
|
|
||||||
### Lookup within a load
|
### Companion helper and prepared main source
|
||||||
|
|
||||||
The usual opt-in mode checks the environment first. A present variable wins,
|
The companion accepts the main `Source`, including a `SearchConfig` source,
|
||||||
including an empty one, which produces the existing empty-secret failure. For
|
and returns a secrets report and a prepared main source. The public name and
|
||||||
an unset variable, read the file whose name matches the effective typed
|
option type below are illustrative.
|
||||||
`Secret` reference. The directory is accessed only if a reference needs a file.
|
|
||||||
|
|
||||||
An application can also select file-first lookup for a load. An absent file
|
```go
|
||||||
allows environment lookup. A file that is present but unreadable or invalid
|
main := configkit.SearchConfig("demo", search)
|
||||||
produces a safe diagnostic, and an empty file follows the existing rule that a
|
report, preparedMain, err := configkit.LoadSecretsIntoEnvironmentFromConfig(main)
|
||||||
resolved credential must be nonempty. If neither source supplies a credential,
|
if err != nil {
|
||||||
report the failure at the affected configuration field.
|
// Handle the bootstrap failure and inspect report.
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
result, err := configkit.Load[AppConfig](preparedMain)
|
||||||
|
```
|
||||||
|
|
||||||
Resolve surviving references at the existing secret-resolution step and share
|
For layered loading, supply the prepared source as the main `Input.Source`.
|
||||||
the lookup result for repeated names within that load. Use the existing
|
The helper resolves discovery once, reads the selected main document, extracts
|
||||||
secret-file checks, content rules, and per-file and cumulative byte limits for
|
the bootstrap controls, and performs bootstrap. It returns the captured main
|
||||||
the files read. Exact-name lookup leaves unrelated directory entries alone.
|
contents with the selected file's identity and filesystem context so the later
|
||||||
Close library-owned handles during the load. Final validation sees the
|
load consumes that same main document. Keep its remaining controls, including
|
||||||
resolved credentials, with byte access through `Secret.Value()`. Inspection
|
imports and collection declarations, available to normal loading.
|
||||||
can distinguish the environment and a file as the credential provider while
|
|
||||||
keeping credential bytes outside diagnostics, history, and effective YAML.
|
|
||||||
|
|
||||||
### Explicit startup export
|
The helper prepares only the main document. Composition, import acquisition,
|
||||||
|
inheritance, typed decoding, application resolution, secret lookup, and
|
||||||
|
validation run through the subsequent ordinary or layered load. Main
|
||||||
|
preparation must retain the applicable source limits and root constraints.
|
||||||
|
Source records, diagnostics, placements, imports, and main-relative companions
|
||||||
|
use the selected main's identity and context.
|
||||||
|
|
||||||
Applications that start SDKs or subprocesses which read the environment can
|
Release library-owned file handles before the helper returns, including on
|
||||||
explicitly request file-first loading and environment export. This is a
|
error. Prepared sources retain captured document data and passive filesystem
|
||||||
startup operation distinct from the ordinary load entry points. Go code
|
context and follow the source reuse contract without sharing mutable load
|
||||||
authorizes writes and overwrites. The main-file declaration selects the
|
state. Later imports and resources observe the filesystem when accessed. A
|
||||||
directory.
|
later load reads the environment at its normal secret-resolution step.
|
||||||
|
[Complete-on-return loading](../adr/0008-complete-on-return-loading.md) · [Source resource contexts](../config.md#source-resource-contexts)
|
||||||
|
|
||||||
Complete configuration loading and validation before writing to the process
|
### Shared bootstrap and overwrite behavior
|
||||||
environment. Export the values of the file-backed effective references used
|
|
||||||
by that successful load, once per name. Each successful write sets the
|
|
||||||
variable to the same value used by the returned configuration. If a file is
|
|
||||||
absent and lookup uses an existing environment value, leave that variable alone.
|
|
||||||
Invoke this operation during startup before starting concurrent work. It runs
|
|
||||||
synchronously and reports successful writes and partial failures without
|
|
||||||
credential bytes, following the existing bootstrap reporting model. ConfigKit
|
|
||||||
does not roll back earlier writes when a later write fails.
|
|
||||||
|
|
||||||
Ordinary `Load` and `LoadLayered` resolve credentials for their own results
|
Extend the existing direct-directory helper with an optional named Boolean
|
||||||
without writing to the environment. The existing
|
option, keeping existing calls valid. The option type name in this example is
|
||||||
`LoadSecretsIntoEnvironment(directory)` remains the explicit startup helper
|
provisional.
|
||||||
for callers supplying a directory independently. [Complete-on-return loading](../adr/0008-complete-on-return-loading.md) · [Secret references and explicit bootstrap](../adr/0004-secret-references-and-explicit-bootstrap.md)
|
|
||||||
|
|
||||||
## Work needed
|
```go
|
||||||
|
report, err := configkit.LoadSecretsIntoEnvironment("/etc/demo/secrets")
|
||||||
|
report, err = configkit.LoadSecretsIntoEnvironment(
|
||||||
|
"/etc/demo/secrets",
|
||||||
|
configkit.SecretsOptions{Overwrite: true},
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
- Extend main-document control parsing and the public load options to capture
|
Both helpers use the same directory-bootstrap implementation. With overwrite
|
||||||
the opted-in declaration and bind it to the main context already obtained
|
disabled, existing variables win, including empty ones, with presence checked
|
||||||
during acquisition. Carry the binding into secret resolution without another
|
again immediately before application as today. With overwrite enabled,
|
||||||
search or parse.
|
eligible files are read even for existing variables and their values replace
|
||||||
- Reuse the secret-file reading and validation rules for exact-name lookup.
|
those variables. Variables without a corresponding file are left alone.
|
||||||
Extend the operation-local lookup to report value-free filesystem failures
|
An applied empty file supplies an empty value in either mode.
|
||||||
and retain the provider for inspection separately from configuration value
|
|
||||||
provenance.
|
|
||||||
- Add the explicit startup operation around the shared load lifecycle. Use the
|
|
||||||
resolved file values for export and reuse the bootstrap reporting semantics
|
|
||||||
where they fit.
|
|
||||||
- Verify discovery from a different working directory, canonical main paths,
|
|
||||||
declaration scope, both lookup orders, and effective references after
|
|
||||||
composition and application resolution. Cover unused and missing files,
|
|
||||||
empty values, read errors, safe inspection, and startup export after
|
|
||||||
validation, including partial writes and absence of writes on a failed load.
|
|
||||||
Use disposable credential values and deterministic failure seams in line
|
|
||||||
with the [testing policy](../policy/testing.md).
|
|
||||||
- Document the resulting contract and a consumer startup example. Record the
|
|
||||||
architectural decision to add file-backed resolution and explicit startup
|
|
||||||
export alongside ADR 0004, and align the current architecture and component
|
|
||||||
documentation when the feature lands.
|
|
||||||
|
|
||||||
## Open Decisions
|
Bootstrap processes the directory's eligible entries, including credentials
|
||||||
|
used by SDKs or subprocesses outside typed `Secret` fields. Reuse the existing
|
||||||
|
dot-entry handling, file checks, content rules, limits, and filename ordering.
|
||||||
|
Validate metadata and read all content needed by the chosen overwrite policy
|
||||||
|
before beginning writes.
|
||||||
|
Directory and file failures produce the existing safe reporting behavior.
|
||||||
|
The secrets report identifies writes, skips, and partial failures without
|
||||||
|
retaining credential bytes.
|
||||||
|
|
||||||
### Startup API and export failure reporting
|
Call bootstrap during startup before starting concurrent work. Successful
|
||||||
|
bootstrap precedes configuration loading and validation. A later load failure
|
||||||
|
leaves the completed environment writes in place. A bootstrap write failure
|
||||||
|
stops further writes, leaves earlier writes in place, and reports remaining
|
||||||
|
entries as `not_applied`. The existing lack of rollback and concurrent
|
||||||
|
environment transaction guarantees continues to apply.
|
||||||
|
|
||||||
**Recommendation.** Provide explicit startup entry points corresponding to
|
Keep bootstrap reporting separate from the subsequent configuration's secret
|
||||||
ordinary and layered loading, backed by the same load implementation. Return
|
lookup evidence. Each operation describes what it observed and performed.
|
||||||
the configuration outcome and the export report so callers can distinguish a
|
[Secrets bootstrap](../config.md#explicit-secrets-directory-bootstrap) · [Secret references and explicit bootstrap](../adr/0004-secret-references-and-explicit-bootstrap.md)
|
||||||
failed load from a successful load followed by an environment write failure.
|
|
||||||
The successful load's private file values can be handed directly to export
|
|
||||||
within the call, which keeps the application flow short and the lifetime of
|
|
||||||
those values local to the startup operation.
|
|
||||||
|
|
||||||
**Alternative.** Export from a successfully returned result through a separate
|
|
||||||
explicit call. This gives applications control over when to perform the writes
|
|
||||||
and avoids a parallel pair of load entry points. It requires a result contract
|
|
||||||
for retaining the original file-backed assignments or defining how later
|
|
||||||
application edits to the result affect export.
|
|
||||||
|
|
||||||
### Filesystem scope
|
### Filesystem scope
|
||||||
|
|
||||||
**Recommendation.** Support discovered and explicitly selected host files
|
Support discovered and explicitly selected host files, including their existing
|
||||||
first, including their existing main-root options. These cover the requested
|
main-root options. This covers the requested deployment pattern and reuses the
|
||||||
deployment pattern and allow reuse of the existing native secret-file checks.
|
native secret-file checks. Caller-supplied filesystem support is a possible
|
||||||
|
later extension requiring decisions about file-type and symlink capabilities.
|
||||||
|
|
||||||
**Alternative.** Support caller-supplied filesystems with a usable main-file
|
## Work needed
|
||||||
context in the same change. This aligns with companion inputs and supports
|
|
||||||
virtual filesystems. It requires an explicit decision about file-type and
|
- Add `config.secrets_dir` and `config.secrets_overwrite` to the main-document
|
||||||
symlink handling where a caller filesystem exposes different metadata or
|
control grammar and consume them during ordinary loading without triggering
|
||||||
access capabilities.
|
bootstrap. Retain source-local positions for diagnostics.
|
||||||
|
- Factor main-file selection and capture for use by the companion helper and
|
||||||
|
normal acquisition. Add a prepared-source handoff with retained main contents,
|
||||||
|
identity, context, and ownership. Select an API for supplying applicable
|
||||||
|
main-root constraints and representing an absent or explicit Go overwrite
|
||||||
|
choice, including `false`.
|
||||||
|
- Add the optional overwrite setting to direct-directory bootstrap and reuse
|
||||||
|
that operation from the companion helper, with matching reports and failures.
|
||||||
|
- Verify discovery from a different working directory, canonical main paths,
|
||||||
|
optional search exhaustion, declaration scope, default and explicit
|
||||||
|
overwrite choices, Go-over-YAML precedence, and both helper interfaces.
|
||||||
|
Cover empty values, dot entries, read failures before writes, and partial
|
||||||
|
write failures. Verify bootstrap includes entries unused by typed fields and
|
||||||
|
that later load failures preserve bootstrap effects.
|
||||||
|
- Verify prepared-source reuse, handle cleanup, normal loading without
|
||||||
|
environment writes, and agreement with selected-file provenance, placement,
|
||||||
|
relative imports, and main-relative companions. Use disposable credential
|
||||||
|
values and deterministic failure seams in line with the
|
||||||
|
[testing policy](../policy/testing.md).
|
||||||
|
- Document the startup workflow and add a consumer example. Record the prepared
|
||||||
|
main-source handoff and optional overwrite policy in an ADR referencing
|
||||||
|
ADR 0004, and update the canonical configuration contract and affected
|
||||||
|
architecture and component documentation when implemented.
|
||||||
|
|||||||
Reference in New Issue
Block a user