Align configuration documentation with native loading

This commit is contained in:
2026-11-17 22:42:53 +00:00
parent ff4bdaa483
commit 413b0d7134
16 changed files with 188 additions and 319 deletions

View File

@@ -37,6 +37,8 @@ Restore planner action kinds:
Planner behavior:
- prepared campaign, session, and party inputs are restored from their declared
immutable run objects to their canonical local input paths;
- a new-protocol restore uses only the selected commit's declared artifact set;
each action carries that artifact's immutable key, checksum, size, and
generation;

View File

@@ -1,210 +1,81 @@
# Configuration Internals
User-visible fields, defaults, and selection behavior belong in the
[Configuration Reference](../config.md). This document describes the internal
pipeline-loading boundary implemented by `internal/config`.
User-visible fields, defaults, discovery, and selection belong in the
[Configuration Reference](../config.md). This document describes the
`internal/config` loading and resolution boundary.
## Pipeline Loading
`LoadPipeline` assembles and validates a pipeline in this order:
Commands bootstrap secrets once before loading, then select an explicit file or
native required discovery. ConfigKit loads typed defaults, source-root imports,
and the selected overlay, resolves inheritance, and strictly decodes the
result. Narratio applies recorded defaults, normalization, resource resolution,
and effective-domain validation. The selected root, source contexts, metadata,
and authoring result are retained for provenance. Family expansion and checks
requiring the campaign occur after campaign and party resolution.
1. Give ConfigKit a fresh typed `PipelineConfig` default value, the root file,
enabled imports, and the caller's overlay selection. Omitted, empty, and
exact `default` selections use the base; named overlays use the fixed
`overlays/<name>.yml` convention.
2. ConfigKit parses the root and declared imports, additively merges base
sources, applies only the selected overlay, resolves native sibling
inheritance, and strictly decodes the result.
3. The definition resolver applies recorded entry-level and inherited defaults,
runs deterministic domain normalization, records derived publish destinations,
and resolves ordinary paths relative to the root pipeline file.
4. The definition validator performs read-only checks that require only the
pipeline file. Checks requiring canonical-party family expansion remain at
the existing explicit pipeline/campaign resolution boundary.
5. Narratio retains public ConfigKit metadata for source ownership, attaches
runtime-only state, and computes the normalized effective digest. Artifact
and family validation applies to the composed effective entries.
This ordering preserves monolithic configuration behavior. Moving a field to
an imported fragment changes its source ownership, not its path base, default,
or schema semantics.
Resource resolution consults native entry origins and source contexts before
normalization. An effective source supplies the base when usable; otherwise the
main source, then the captured invocation directory, is the fallback. Explicit
CLI file references use the invocation directory. Derived values record their
dependencies. See the [configuration reference](../config.md#pipeline-imports-and-overlays)
for the resource classifications and native authoring contract.
## Loaded Context Resolution
`LoadedPipelineCampaign` carries one already composed pipeline and its selected
campaign into session resolution. `LoadSessionWithPipelineCampaignOptions`
loads a local session against that context, while
`ResolveLoadedPipelineCampaign` also accepts an already loaded remote session
or no session while a caller retrieves one. Compatibility loaders route through
these functions after their initial pipeline and campaign reads.
`LoadedPipelineCampaign` carries the selected pipeline and campaign into local
or remote session resolution. Local campaign, session, and party loads enable
imports through their own ConfigKit definitions; only the pipeline selects an
overlay. Native optional search distinguishes an absent local session from an
invalid selected file. Remote session bytes have no incidental download-directory
context, and retain object provenance. Effective session strings are checked for
unresolved template tokens, and campaign/session identity and stable-input rules
remain Narratio-owned.
Application commands own pipeline and campaign discovery, campaign-file versus
registry selection, and the corresponding mutual-exclusion rules. Once they
have a `LoadedPipelineCampaign`, local session discovery and remote-session
download retain that exact pipeline object and its private provenance. Removing
a temporary downloaded session file therefore cannot invalidate the resolved
pipeline or campaign context.
Campaign and session loaders retain detached document values after their own
resolution. The session document is captured before campaign defaults are copied
into runtime session state. Configuration-owned renderers produce standalone
campaign and session YAML, preserving omitted session overrides. Prepare uses
those renderers and canonical party rendering for materialization and resume
comparison.
Campaign and session files use independent single-source ConfigKit definitions
with imports and selected overlays disabled; native sibling inheritance remains
available within each document. Canonical-party input follows the same
single-source rule. Campaign files normally load directly from their source;
session files are first read through the 8 MiB bounded-read boundary because the
application must reject unresolved template placeholders before passing an exact
byte snapshot to ConfigKit. Local and remotely downloaded session bytes then use
the same typed loader and retain the existing post-load identity expectations,
stable-input precedence, and source-relative path semantics.
## Inspection And Provenance
The application also has a separate read-only inspection resolver for `config
validate`, `config show`, `config sources`, and `config artifacts`. It uses the
same production root, overlay, and campaign selection functions, but never routes
through session discovery,
remote-session download, adapter composition, workspace
initialization, manifest access, or cleanup. A pipeline with retained artifact
family declarations must resolve its selected campaign before ordinary pipeline
validation, which expands its canonical-party members and generated publish
rules. A pipeline without those declarations may be validated by itself.
Secret-directory bootstrap remains invocation-scoped and happens before the
ordinary ConfigKit load because explicit typed references must resolve even for
inspection.
Read-only inspection shares production pipeline and campaign selection without
session discovery, workspace initialization, adapters, or manifest access. After
applicable campaign and family resolution, a safe typed final-domain DTO contains
concrete artifacts and generated publish rules. It stores credential reference
names rather than typed secrets. A defaults-backed native result freezes this
DTO without resolving credentials again; `WriteEffective` supplies show and
prepared pipeline YAML, and deterministic JSON of the same DTO supplies the
versioned semantic digest. Resolved resources participate in that digest, while
source layout and secret values do not.
`MarshalEffectivePipeline` is the configuration-owned projection for `config
show`. It serializes one explicit typed safe projection with the existing YAML
encoder. The projection has no slots for resolution-only artifact family
declarations or private provenance. The result has one trailing newline;
commands do not marshal runtime objects directly.
`EffectivePipelineSources` and `EffectiveCampaignSources` provide the separate
safe provenance projection for `config sources`. Pipeline ownership begins with
the complete logical field paths projected from the schema and classifies
each contributor as root, import, overlay, or centralized default. It reads
ConfigKit's effective origins and inheritance locations when building each
projection, so inherited values retain their contributors and controlling
directives without a second cached leaf-ownership table. The
projection replaces generated concrete member paths with paired family and
canonical-party records, and does the same for generated publish rules.
Campaign records identify campaign-owned fields and party inputs; canonical
derived players point to the party source. The application command only joins these sorted
records with selection metadata and never reparses configuration files.
`EffectiveArtifactCatalog` projects the same finalized concrete artifact map for
`config artifacts`. It derives source identity from each map key, carries sorted
effective dependencies, aggregates ConfigKit definition contributors, and adds
family/character/party provenance for generated members. Family source evidence
can name multiple contributing files. It does not inspect files named after
artifacts or infer another runtime graph.
`config diff` uses a thin paired loader that performs two independent ConfigKit
loads. When campaign
resolution is needed, the command loads one selected campaign and party and
expands both effective pipelines from that same party value. The configuration
owner projects each normalized effective mapping directly from that same typed
projection into sorted logical paths using JSON number preservation;
mapping leaves are compared individually while sequence values remain atomic.
Values are compact deterministic JSON representations for command output, not
raw YAML fragments, ownership records, or secret material. A differing digest
with no projected difference is treated as an internal consistency error.
Campaign context construction reads the campaign-owned party source through
`ParseParty`. The required canonical party retains its raw bytes and normalized
roster in runtime-only `ResolvedParty` provenance. Canonical resolution creates
a deterministic `derived_from_party` players projection. The party is the only
configured roster authority; player and session-party fields are unsupported.
Authoring results separately supply native origins, source histories, and empty
optional-import evidence to source and artifact inspection. Family and party
context identifies generated entries without reparsing sources. Diff fully
resolves each side against the same selected campaign and party, then uses
ConfigKit `Diff` on the safe snapshots. The command presents native JSON Pointer
paths and safe compact values.
## Canonical Party Domain
`ParseParty` is the package-owned boundary for parsing a party source. Its
single-source ConfigKit definition disables imports and overlays, requires
`schema_version` through `VersionRule`, and strictly decodes the
character-keyed typed schema before applying Narratio's party-domain checks.
Successful metadata coordinates recover authored character-map order without a
second YAML parse; class and alias slice order comes directly from typed decode.
The canonical value retains a separate exact byte copy of its source so
consumers can materialize the authored party document without reserializing it.
Its `PlayersYAML` method deterministically derives the ID-sorted, versioned
players-only projection.
An unversioned source is rejected before resolution, family expansion, or
prepare-stage materialization.
File-backed party loading enables imports; `ParseParty` accepts contextless bytes.
The shared definition checks the main document's version and Narratio validates
the effective character roster. Native acquisition order and source coordinates
establish roster order, favoring each child's own declaration. A deterministic
renderer writes standalone canonical party YAML from that roster; the players
projection and family expansion use the same resolved roster. The external format
is defined in the [party contract](../integrations/party.md).
## Validation Ownership
Configuration validation remains within `internal/config`, with the public
entry points and pipeline-domain sequencing separated from the private owners
for core pipeline/session rules, external adapters, publish/storage, Notarius,
Scriptorium, and shared scalar/path checks. These are package-local ownership
boundaries rather than extension points or a general validation framework.
ConfigKit supplies composition, inheritance, presence, typed-secret, and
provenance mechanics; Narratio supplies the domain rules applied to the
effective decoded entries. There is no separate authoring-history acceptance
gate: contributors and declaration history remain inspection evidence, not an
additional validity condition.
The [configuration reference](../config.md#pipeline-imports-and-overlays)
documents sibling syntax, reserved-key migration, and the operator-visible
inspection behavior.
Validation is read-only, fails fast, and does not define which of several
independent invalid fields is reported first. Map-backed declarations are
nevertheless traversed deterministically. Reusable domain normalization is an
explicit preceding operation for both loaded and programmatically constructed
pipelines. Notarius normalization commits canonical scalar, reference, and output
values only after the complete normalization succeeds; its validator then checks
that candidate without changing it. Scriptorium validates artifact declarations
and outputs, dependencies and inputs, variables, referenced outputs, and the final
dependency graph in deterministic artifact order.
Diagnostics identify the relevant semantic field. Merge conflicts also
retain every claiming source needed to correct an additive ownership conflict.
Complete prose, punctuation, formatting, and private validation choreography are
not compatibility contracts.
## Diagnostics And Runtime Metadata
Syntax, duplicate-key, merge, overlay, and schema failures include the
relevant source name and full field path. Additive conflicts report every
claiming source so operators can repair the split without repeatedly
rediscovering additional conflicts.
The loaded pipeline retains private runtime metadata for the absolute root
path, ordered imports, selected overlay name, source, and path, original
ConfigKit metadata and source snapshots, and effective digest. Leaf source
records are projected from that retained native metadata when requested. This
metadata does not participate in YAML decoding or alter the public
configuration model.
The effective digest is SHA-256 over a versioned deterministic JSON encoding of
the same typed safe projection used by show and diff. Runtime Notarius paths remain absolute for
execution, but the digest substitutes their normalized logical values captured
before root-relative resolution, so relocating an equivalent configuration
bundle does not change provenance. Because ConfigKit and resolution metadata
are private, the digest excludes source layout, overlay name, and ownership.
Configuration stores credentials as ConfigKit typed references. A successful
load resolves their values inside the opaque typed boundary, while this safe
projection serializes only `EnvName()` identities. Credential-value changes are
therefore never hashed; reference-name changes remain ordinary provenance.
The optional Notarius credential follows the same path even for disabled or
resumable extraction; normalization copies the opaque reference without reading
or resolving it again. Direct YAML serialization has a configuration-owned safe
projection so only the reference name can leave this boundary.
`recomputePipelineEffectiveDigest` is the single package-owned refresh point
for later runtime expansion.
## Test Surfaces
`pipeline_configkit_test.go` exercises real imports and fixed-path overlays,
base selection, unselected-overlay isolation, source attribution, cross-file
inheritance, and effective composition. Existing focused configuration
tests protect defaults, explicit-value semantics, strict decoding, validation,
metadata, and root-relative path behavior. Application configuration-loader tests protect the
single-read boundary by changing the pipeline file after its initial load and
confirming local session resolution retains the original pipeline. Other
configuration tests continue to protect defaults and validation after assembly.
`party_test.go` protects the versioned typed party schema, domain invariants,
authored character order, retained bytes, and deterministic players projection
without involving campaign or runtime wiring.
`party_resolution_test.go` protects campaign-owned party loading, canonical
input restrictions, source provenance, and virtual players input selection.
ConfigKit owns composition, inheritance, presence, typed-secret resolution, and
provenance. Narratio owns validation and normalization of effective domain
values. Source histories are inspection evidence, not an additional acceptance
gate. Focused configuration and application tests exercise imports, resource
contexts, snapshots, secret-safe inspection, party resolution, and materialized
inputs through the production boundaries.
## Artifact Family Resolution

View File

@@ -107,7 +107,7 @@ so any earlier replacement can stale a prior cleanup result.
## Focused Documentation
- [Configuration Internals](configuration.md): pipeline imports and overlays,
field ownership, decoding, and root-relative path semantics.
field ownership, decoding, and source-relative resource semantics.
- [Adapter Internals](adapters.md): external adapter boundaries, composition,
failure behavior, and test surfaces.
- [Artifact Internals](artifacts.md): source identities, runtime catalog,

View File

@@ -39,8 +39,8 @@ Materialize canonical current-session inputs before processing stages.
- materializes a configured spell catalog with checksum and provenance, or
safely removes an obsolete canonical spell catalog and its manifest record
when the effective input is omitted.
- copies the validated canonical party bytes unchanged and deterministically
generates the prepared players projection.
- renders standalone effective campaign, session, and canonical party documents,
and deterministically generates the prepared players projection.
- scans enabled configured artifact inputs for `narratio.previous_session.artifact.*` requirements.
- clears managed `previous/` state on every invocation, then, when requirements exist:
- resolves the pointer-selected previous source through the shared resolver;
@@ -71,15 +71,15 @@ previous-artifact requirement set. A change reruns prepare and
stales its fixed descendants. Existing successful records without this
evidence rerun once when selected.
Workspace, spool, and cache placement and absolute source relocation are not
semantic when logical selection, canonical names, and bytes are equivalent.
Workspace, spool, and cache placement are not semantic when logical selection,
canonical names, and effective bytes are equivalent.
The fingerprint deliberately does not read or rehash large audio. Prepared
input checksums remain the content provenance. Before reusing success, prepare
validates every durable prepared copy and compares current stable-input bytes,
canonical party and derived-player bytes, local audio membership/checksums, or
S3 key/size/entity-tag identity with that provenance. Source relocation with
equivalent names and bytes remains reusable; changed or unavailable evidence
causes a normal prepare rerun.
effective campaign/session, canonical party and derived-player bytes, local audio
membership/checksums, or S3 key/size/entity-tag identity with that provenance.
Source relocation with equivalent effective documents, names, and bytes remains
reusable; changed or unavailable evidence causes a normal prepare rerun.
## Related Contracts And Tests

View File

@@ -15,8 +15,8 @@ Upload run/session outputs to object storage and atomically advance remote curre
## Outputs
- uploaded session manifest, declared run-local diagnostics, and selected
publish outputs;
- uploaded session manifest, prepared campaign/session/party inputs, declared
run-local diagnostics, and selected publish outputs;
- uploaded durable previous-session cache files when present;
- immutable run-scoped commit manifest; and
- current commit pointer, written last.
@@ -34,7 +34,9 @@ and optional rules, destination resolution, and collision inputs live in
remote verification live in `publish_transfer.go`; committed-manifest,
immutable-commit, and current-pointer encoding and writes live in
`publish_commit.go`. `publish_output_plan.go` owns the deterministic, I/O-free
five-outcome model and its secret-free metadata projection.
five-outcome model and its secret-free metadata projection. Prepared input
objects are committed under the immutable run; remote session loading selects
the committed standalone session through the current pointer.
- when publishing or run upload is disabled, completes successfully with no
outputs and records explanatory metadata. This is not an explicit self-skip: