Align configuration documentation with native loading

This commit is contained in:
2026-11-17 22:42:53 +00:00
parent ff4bdaa483
commit 413b0d7134
16 changed files with 188 additions and 319 deletions

View File

@@ -59,8 +59,8 @@ Rules:
base configuration, while an unknown named overlay fails. The same selection base configuration, while an unknown named overlay fails. The same selection
applies to all common-flag commands, including `regenerate-artifacts`. applies to all common-flag commands, including `regenerate-artifacts`.
- `--secrets-dir` selects ConfigKit credential-file bootstrap explicitly. A - `--secrets-dir` selects ConfigKit credential-file bootstrap explicitly. A
relative value uses caller CWD. Without the flag, a present `secrets/` relative value uses caller CWD. Without the flag, Narratio uses the optional
directory beside the resolved root pipeline is used; an absent one is skipped. `/usr/local/etc/narratio/secrets` directory.
- `clean --all` cannot be combined with campaign/session selectors. - `clean --all` cannot be combined with campaign/session selectors.
- notification delivery is currently limited to the configured `noop` mode; see - notification delivery is currently limited to the configured `noop` mode; see
the [configuration reference](./config.md#notifications). the [configuration reference](./config.md#notifications).

View File

@@ -12,10 +12,20 @@ Narratio resolves three YAML documents:
### `pipeline.yml` ### `pipeline.yml`
When `--config` is omitted, search order is: An explicit `--config` selects exactly that file, including a custom filename.
Otherwise ConfigKit searches these locations in order:
1. `/usr/local/etc/narratio/pipeline.yml` 1. The working directory, for `narratio.yml` then `narratio.yaml`.
2. `/etc/narratio/pipeline.yml` 2. The Go user-config directory's `narratio/` subdirectory, for those named
files followed by `config.yml`, `config.yaml`, `main.yml`, and `main.yaml`.
3. The Go user-config directory itself, for the named files.
4. `/usr/local/etc/narratio/` and then `/etc/narratio/`, each for named files
followed by the generic filenames above.
5. `/etc/`, for the named files.
On Linux, the user-config directory normally follows `XDG_CONFIG_HOME`, or
`~/.config` when it is unset. A selected invalid file fails loading; discovery
does not continue to another candidate.
### `campaign.yml` ### `campaign.yml`
@@ -32,15 +42,12 @@ When `--session` is omitted, local search order is:
1. `/usr/local/etc/narratio/session.yml` 1. `/usr/local/etc/narratio/session.yml`
2. `/etc/narratio/session.yml` 2. `/etc/narratio/session.yml`
If local session discovery fails and a `session_id` is known, Narratio attempts remote session loading from: If no local session exists and a `session_id` is known, Narratio loads the
session from configured object storage. It selects the prepared session in the
- `{root_prefix}/campaigns/{campaign}/sessions/{session_id}/session.yml` current committed run when available; if no current pointer exists or an older
commit has no prepared session, it falls back to the session-init `session.yml`
using configured object storage. object. Downloaded session bytes are contextless snapshots, with remote object
provenance retained independently of locally materialized checksums.
The downloaded remote session file is command-scoped: Narratio removes it after
the command finishes and records only the remote object provenance alongside
the durable copied session input.
### Read-only effective pipeline inspection ### Read-only effective pipeline inspection
@@ -72,9 +79,10 @@ several files for one family member. The [CLI reference](cli.md#config-validate-
owns command syntax and output conventions. owns command syntax and output conventions.
`config diff` compares normalized field values rather than YAML text or source `config diff` compares normalized field values rather than YAML text or source
ownership. It emits sorted `added`, `removed`, and `changed` records, uses ownership. It emits native ordered `added`, `removed`, and `changed` records with
compact deterministic JSON values, treats lists atomically, and reports `no JSON Pointer paths and compact JSON values. Lists are compared atomically, and
differences` when the complete effective configurations are equal. Concrete changed regions may contain whole values. It reports `no differences` when the
complete effective configurations are equal. Concrete
family members and generated publish rules participate after expansion; moving family members and generated publish rules participate after expansion; moving
an equal value between eligible root/import sources does not create a an equal value between eligible root/import sources does not create a
difference. difference.
@@ -105,23 +113,14 @@ remote state with an unsafe legacy identity must be migrated before use.
- ConfigKit performs strict typed YAML decoding and accepts exactly one document; - ConfigKit performs strict typed YAML decoding and accepts exactly one document;
unknown fields, duplicate keys, and trailing documents fail load. unknown fields, duplicate keys, and trailing documents fail load.
- A pipeline file may explicitly import additive YAML fragments through the - Local pipeline, campaign, session, and party documents may declare native
root-only `configkit.imports` list. Imported files contribute fields to one imports and sibling inheritance. Only the pipeline selects an overlay.
logical pipeline document; they do not override fields supplied by the root - An omitted overlay, empty overlay option, and exact `default` select the base;
or another import. another name selects `overlays/<name>.yml` relative to the pipeline root.
- An omitted overlay, an empty overlay option, and the exact name `default`
select the base configuration. Any other name selects
`overlays/<name>.yml` relative to the pipeline root. Move previously selected
`profiles/<name>.yml` files there; the old directory is not searched, and
`--profile` is no longer accepted for Narratio pipeline selection. Explicit
imports from a directory named `profiles` remain valid.
- Campaign, session, and canonical-party documents are strict single-source
ConfigKit schemas. They support sibling inheritance but do not enable imports,
selected overlays, or `configkit` directives.
- Configured timeout and retry-delay durations must be positive. An omitted - Configured timeout and retry-delay durations must be positive. An omitted
artifact timeout continues to inherit its configured Scriptorium timeout. artifact timeout continues to inherit its configured Scriptorium timeout.
- Session files must be concrete; unresolved `{{ ... }}` placeholders fail load. - Session files must be concrete; unresolved `{{ ... }}` placeholders fail load.
- Session and party inputs inspected before typed loading are limited to 8 MiB. - ConfigKit applies its acquisition limits to configuration sources and imports.
- Pipeline defaults are applied before validation. - Pipeline defaults are applied before validation.
- Defaults apply only to absent fields. Explicit empty strings, zero values, - Defaults apply only to absent fields. Explicit empty strings, zero values,
`false`, and `null` retain ConfigKit's typed meaning and are then checked by `false`, and `null` retain ConfigKit's typed meaning and are then checked by
@@ -143,82 +142,74 @@ remote state with an unsafe legacy identity must be migrated before use.
### Pipeline imports and overlays ### Pipeline imports and overlays
Large pipeline configurations may be split into explicitly named fragments and Local pipeline, campaign, session, and party files can use source-root
may use fixed-path selected overlays: `config.imports` and `config.sequences`. Every acquired document has its own
source-root controls, including imported fragments. Nested `config` keys are
ordinary data. `extends` is reserved for native sibling inheritance; parents
remain ordinary entries subject to Narratio's effective-domain validation.
```yaml ```yaml
config: config:
imports: imports:
- path: config/storage.yml - path: artifacts/session-recap.yml
- path: config/integrations.yaml destination: [scriptorium, artifacts, session_recap]
required: true
campaigns: - dir: artifacts/families
root: /usr/local/share/narratio/campaigns pattern: "*.yml"
``` ```
The maintained [production/testing bundle](../examples/production-testing/pipeline.yml) Imports support `path`, `dir`, and `glob` selectors, applicable filters and
is a complete copyable example of this structure, including canonical-party relative destinations. Paths are relative to the declaring source. Missing or
artifact families. empty selections are optional by default; use `required: true` when a selection
is essential. Selected invalid input fails. Destinations place content under
literal mapping keys, independent of filenames; nested imports can declare
relative destinations. Strict peer contributions must satisfy ConfigKit's
composition rules, and Narratio validates the resulting effective entries.
Imports are resolved relative to the directory containing the root pipeline A selected pipeline overlay is applied after base composition. It may declare
file and are loaded in declaration order. Narratio does not scan directories or root `config.sequences` but cannot import other files. Ordinary overlay lists
infer fragments. Each import must be a confined regular `.yml` or `.yaml` file: replace the lower list; declared collections can use native `replace`, `append`,
absolute paths, traversal, symlinks, directories, duplicate files, and an or `upsert`. Upsert replaces a whole matching record; an empty append or upsert
import of the root pipeline itself are rejected. Imported files cannot declare does not clear the lower list. Mapping and identified-list inheritance follow
another `configkit` control section. native sibling rules. See the pinned [ConfigKit configuration contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/config.md)
and [consumer guide](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/consumers/README.md)
for full selection, composition, collection, and reserved-key semantics.
The [maintained bundle](../examples/production-testing/pipeline.yml) illustrates
these controls, family expansion, and a split roster.
Base merging is additive. A map may be extended by multiple files when every Relative host-resource paths, including integration configuration files,
leaf is distinct, but a scalar, list, or map/list/scalar kind cannot be claimed workspace/campaign/spool/cache roots, campaign template and stable inputs, and
more than once, even when the repeated values are identical. Conflict errors session stable inputs and local audio references, resolve from the effective
name the full field path and every source that claimed it. These rules apply at value's defining source. Inherited values follow their origin. When no distinct
every map path, including artifact and artifact-family entries, so distinct usable source context exists, the main source is the fallback; if it too is
fields of one entry can compose across imports. Narratio validates the resulting contextless, the invocation directory is used. Explicit CLI-relative file paths
effective entry using its ordinary domain rules. use the invocation directory. Artifact outputs, publish destinations, logical
IDs, and binary names retain their domain meanings.
After base sources are composed and the selected overlay is applied, ConfigKit The effective digest excludes source layout, overlay identity, and secret values;
resolves native sibling inheritance before typed decoding. Any mapping may use credential reference names and resolved resource paths participate. Prepared
`extends: <sibling-key>` to inherit from an exact key in the same enclosing map; campaign, session, and party documents contain effective standalone values rather
lookup does not search parent maps or unrelated scopes. The child keeps its own than authored import directives. Session fields omitted by the author remain
key and overlays its explicit fields on the resolved parent. Mappings combine eligible to inherit the receiving campaign's values.
recursively, while explicit scalar, list, false, zero, empty, and null values
remain the child's values. Parent entries remain ordinary configuration entries
and must satisfy Narratio's normal effective-domain validation; inheritance does
not create templates exempt from domain rules. Inherited explicit input counts
as present for absence-based defaults.
Use [`config sources`](./cli.md#config-validate-config-show-config-sources-config-artifacts-and-config-diff)
to inspect native effective contributors and inheritance origins; composite
artifact definitions may report multiple source paths.
The key `extends` is reserved structural input wherever ConfigKit is used, ### Upgrade from earlier configuration
including pipeline roots, imported files, selected overlay values, dynamic
artifact IDs, prompt-variable mappings, campaigns, sessions, and canonical-party
inputs. Rename any existing ordinary-data key named `extends`;
there is no opt-in or automatic conversion. See the pinned [ConfigKit v0.5.0
configuration contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/config.md#sibling-inheritance)
for exact lookup, composition, presence, and provenance semantics.
Overlay names are case-sensitive and use ConfigKit's safe fixed-path naming - Rename source-root `configkit:` controls to `config:`. Nested `config` data
rules. Only the selected overlay is opened; a malformed unselected overlay does and the Go `configkit` package and secret tags have different meanings.
not affect a base or different-overlay load. Maps merge recursively, overlay - Rename or move pipeline files to the discovery names above, or retain a custom
scalars replace base scalars, and overlay lists replace base lists completely. filename such as `pipeline.yml` and pass it with `--config`.
Explicit `false`, zero, empty strings, empty containers, and null values remain - Move conventional secrets to `/usr/local/etc/narratio/secrets`, supply them
explicit inputs and are not refilled by defaults. Overlay files cannot inherit through the environment, or pass the previous directory with `--secrets-dir`.
from or stack on other overlay files, import files, or declare another ConfigKit - Review relative resource references moved into fragments: their defining
control section. Values in the selected overlay may use native sibling source now supplies the base. Mark necessary imports `required: true`.
`extends` directives; those relationships resolve after overlay application. - Review lists and use `config.sequences` where append or whole-record upsert is
intended. Verify the effective result with the inspection commands.
- Prepared documents contain effective values; retain authored roots and their
companions separately for editing. Separately installed downstream tools own
their own configuration migrations.
During ConfigKit loading, Narratio applies fresh typed defaults, recorded These control-key changes require a major version for the eventual Narratio
entry-level defaults and normalization, read-only file-local validation, and release. This migration does not itself publish a release.
ordinary path resolution before computing a deterministic effective
configuration digest. The digest represents the normalized, secret-free
runtime pipeline mapping; it excludes ConfigKit directives, source
provenance, overlay identity, and raw environment secret values. Equivalent
effective mappings therefore have the same digest regardless of how fields are
split among the root and imports.
An imported field has the same meaning it would have in a monolithic root
pipeline. In particular, ordinary relative pipeline paths continue to resolve
from the root pipeline directory, not from the importing fragment's directory.
## Minimal Working Configuration ## Minimal Working Configuration
@@ -263,8 +254,8 @@ inputs:
stage range, or eligible for resume. Omit the field or set it to `null` when stage range, or eligible for resume. Omit the field or set it to `null` when
that deployment should not require the credential. that deployment should not require the credential.
- Every command that loads a pipeline accepts `--secrets-dir <path>`. When the - Every command that loads a pipeline accepts `--secrets-dir <path>`. When the
flag is omitted, Narratio uses a present `secrets/` directory beside the flag is omitted, Narratio uses `/usr/local/etc/narratio/secrets` when present;
resolved root `pipeline.yml`; an absent conventional directory is ignored. an absent conventional directory is ignored.
Explicit relative paths are resolved from the caller's working directory, Explicit relative paths are resolved from the caller's working directory,
and a missing or empty explicit selection is an error. and a missing or empty explicit selection is an error.
- Each immediate regular file whose name is a valid environment-variable name - Each immediate regular file whose name is a valid environment-variable name
@@ -320,7 +311,7 @@ Rules:
| Field | Type | Required | Default / Rule | | Field | Type | Required | Default / Rule |
| --- | --- | --- | --- | | --- | --- | --- | --- |
| `configkit.imports[].path` | string | No | explicit additive pipeline fragment relative to the root pipeline directory; `.yml` or `.yaml` regular files only | | `config.imports` | list | No | native source-root import selections; see [imports and overlays](#pipeline-imports-and-overlays) |
| `pipeline.workspace.root` | string | No | `/var/lib/narratio` | | `pipeline.workspace.root` | string | No | `/var/lib/narratio` |
| `pipeline.workspace.cleanup_after_publish` | bool | No | `false`; selects the published run's workspace target for final cleanup and requires effective `pipeline.publish.enabled=true` and `pipeline.publish.upload_run=true` | | `pipeline.workspace.cleanup_after_publish` | bool | No | `false`; selects the published run's workspace target for final cleanup and requires effective `pipeline.publish.enabled=true` and `pipeline.publish.upload_run=true` |
| `pipeline.campaigns.root` | string | No | `/usr/local/share/narratio/campaigns` | | `pipeline.campaigns.root` | string | No | `/usr/local/share/narratio/campaigns` |
@@ -385,10 +376,10 @@ Rules:
| `pipeline.notarius.binary` | string | No | `notarius` | | `pipeline.notarius.binary` | string | No | `notarius` |
| `pipeline.notarius.credentials[]` | list[secret reference] | No | empty; exact environment-variable names forwarded to Notarius | | `pipeline.notarius.credentials[]` | list[secret reference] | No | empty; exact environment-variable names forwarded to Notarius |
| `pipeline.notarius.config_overlay` | string | No | unset; exact Notarius ConfigKit overlay selector | | `pipeline.notarius.config_overlay` | string | No | unset; exact Notarius ConfigKit overlay selector |
| `pipeline.notarius.config_path` | string | Conditional | required when enabled; relative paths resolve from the pipeline file directory | | `pipeline.notarius.config_path` | string | Conditional | required when enabled; relative paths resolve from their defining source |
| `pipeline.notarius.pipeline_id` | string | Conditional | required when enabled | | `pipeline.notarius.pipeline_id` | string | Conditional | required when enabled |
| `pipeline.notarius.timeout` | duration | No | `3h`; must be positive | | `pipeline.notarius.timeout` | duration | No | `3h`; must be positive |
| `pipeline.notarius.working_directory` | string | No | directory containing resolved `config_path`; relative paths resolve from the pipeline file directory | | `pipeline.notarius.working_directory` | string | No | directory containing resolved `config_path`; relative paths resolve from their defining source |
| `pipeline.notarius.resume` | bool | No | `false`; request compatible Notarius checkpoint reuse during ordinary executed extraction | | `pipeline.notarius.resume` | bool | No | `false`; request compatible Notarius checkpoint reuse during ordinary executed extraction |
| `pipeline.notarius.chunk_cache` | string | No | empty; when set, exactly `auto`, `bypass`, or `refresh` | | `pipeline.notarius.chunk_cache` | string | No | empty; when set, exactly `auto`, `bypass`, or `refresh` |
| `pipeline.notarius.references` | map[string]string | No | empty; maps normalized Notarius selectors to supported prepared Narratio source IDs; maximum 256 entries | | `pipeline.notarius.references` | map[string]string | No | empty; maps normalized Notarius selectors to supported prepared Narratio source IDs; maximum 256 entries |
@@ -697,7 +688,7 @@ integration.
| `inputs.speakers_file` | string | Yes | stable input default | | `inputs.speakers_file` | string | Yes | stable input default |
| `inputs.autocorrect_file` | string | Yes | stable input default | | `inputs.autocorrect_file` | string | Yes | stable input default |
| `inputs.glossary_file` | string | Yes | stable input default | | `inputs.glossary_file` | string | Yes | stable input default |
| `inputs.party_file` | string | Yes | stable campaign party source; relative paths resolve from `campaign.yml` | | `inputs.party_file` | string | Yes | stable campaign party source; relative paths resolve from their defining source |
| `inputs.spell_catalog_file` | string | No | optional spell-catalog overlay default; required when a Notarius reference selects `narratio.input.spell_catalog` | | `inputs.spell_catalog_file` | string | No | optional spell-catalog overlay default; required when a Notarius reference selects `narratio.input.spell_catalog` |
### Session ### Session

View File

@@ -1,8 +1,8 @@
# Canonical Party Input # Canonical Party Input
`party.yml` is a campaign-owned roster input. It must be a single, versioned `party.yml` is a campaign-owned roster input. Its main document must declare
`narratio.party.v1` document when Narratio resolves a pipeline, campaign, and `narratio.party.v1` when Narratio resolves the roster. Local party files may
session together. import fragments; fragments need no version header.
```yaml ```yaml
schema_version: narratio.party.v1 schema_version: narratio.party.v1
@@ -48,8 +48,8 @@ The configuration rules and source-relative path behavior are defined in the
## Derived players document ## Derived players document
During `prepare`, Narratio copies the canonical party source bytes unchanged During `prepare`, Narratio renders the effective roster as standalone canonical
to `inputs/party.yml` and writes this deterministic players-only projection to `inputs/party.yml` and writes this deterministic players-only projection to
`inputs/players.yml`: `inputs/players.yml`:
```yaml ```yaml

View File

@@ -37,6 +37,8 @@ Restore planner action kinds:
Planner behavior: Planner behavior:
- prepared campaign, session, and party inputs are restored from their declared
immutable run objects to their canonical local input paths;
- a new-protocol restore uses only the selected commit's declared artifact set; - a new-protocol restore uses only the selected commit's declared artifact set;
each action carries that artifact's immutable key, checksum, size, and each action carries that artifact's immutable key, checksum, size, and
generation; generation;

View File

@@ -1,210 +1,81 @@
# Configuration Internals # Configuration Internals
User-visible fields, defaults, and selection behavior belong in the User-visible fields, defaults, discovery, and selection belong in the
[Configuration Reference](../config.md). This document describes the internal [Configuration Reference](../config.md). This document describes the
pipeline-loading boundary implemented by `internal/config`. `internal/config` loading and resolution boundary.
## Pipeline Loading ## Pipeline Loading
`LoadPipeline` assembles and validates a pipeline in this order: Commands bootstrap secrets once before loading, then select an explicit file or
native required discovery. ConfigKit loads typed defaults, source-root imports,
and the selected overlay, resolves inheritance, and strictly decodes the
result. Narratio applies recorded defaults, normalization, resource resolution,
and effective-domain validation. The selected root, source contexts, metadata,
and authoring result are retained for provenance. Family expansion and checks
requiring the campaign occur after campaign and party resolution.
1. Give ConfigKit a fresh typed `PipelineConfig` default value, the root file, Resource resolution consults native entry origins and source contexts before
enabled imports, and the caller's overlay selection. Omitted, empty, and normalization. An effective source supplies the base when usable; otherwise the
exact `default` selections use the base; named overlays use the fixed main source, then the captured invocation directory, is the fallback. Explicit
`overlays/<name>.yml` convention. CLI file references use the invocation directory. Derived values record their
2. ConfigKit parses the root and declared imports, additively merges base dependencies. See the [configuration reference](../config.md#pipeline-imports-and-overlays)
sources, applies only the selected overlay, resolves native sibling for the resource classifications and native authoring contract.
inheritance, and strictly decodes the result.
3. The definition resolver applies recorded entry-level and inherited defaults,
runs deterministic domain normalization, records derived publish destinations,
and resolves ordinary paths relative to the root pipeline file.
4. The definition validator performs read-only checks that require only the
pipeline file. Checks requiring canonical-party family expansion remain at
the existing explicit pipeline/campaign resolution boundary.
5. Narratio retains public ConfigKit metadata for source ownership, attaches
runtime-only state, and computes the normalized effective digest. Artifact
and family validation applies to the composed effective entries.
This ordering preserves monolithic configuration behavior. Moving a field to
an imported fragment changes its source ownership, not its path base, default,
or schema semantics.
## Loaded Context Resolution ## Loaded Context Resolution
`LoadedPipelineCampaign` carries one already composed pipeline and its selected `LoadedPipelineCampaign` carries the selected pipeline and campaign into local
campaign into session resolution. `LoadSessionWithPipelineCampaignOptions` or remote session resolution. Local campaign, session, and party loads enable
loads a local session against that context, while imports through their own ConfigKit definitions; only the pipeline selects an
`ResolveLoadedPipelineCampaign` also accepts an already loaded remote session overlay. Native optional search distinguishes an absent local session from an
or no session while a caller retrieves one. Compatibility loaders route through invalid selected file. Remote session bytes have no incidental download-directory
these functions after their initial pipeline and campaign reads. context, and retain object provenance. Effective session strings are checked for
unresolved template tokens, and campaign/session identity and stable-input rules
remain Narratio-owned.
Application commands own pipeline and campaign discovery, campaign-file versus Campaign and session loaders retain detached document values after their own
registry selection, and the corresponding mutual-exclusion rules. Once they resolution. The session document is captured before campaign defaults are copied
have a `LoadedPipelineCampaign`, local session discovery and remote-session into runtime session state. Configuration-owned renderers produce standalone
download retain that exact pipeline object and its private provenance. Removing campaign and session YAML, preserving omitted session overrides. Prepare uses
a temporary downloaded session file therefore cannot invalidate the resolved those renderers and canonical party rendering for materialization and resume
pipeline or campaign context. comparison.
Campaign and session files use independent single-source ConfigKit definitions ## Inspection And Provenance
with imports and selected overlays disabled; native sibling inheritance remains
available within each document. Canonical-party input follows the same
single-source rule. Campaign files normally load directly from their source;
session files are first read through the 8 MiB bounded-read boundary because the
application must reject unresolved template placeholders before passing an exact
byte snapshot to ConfigKit. Local and remotely downloaded session bytes then use
the same typed loader and retain the existing post-load identity expectations,
stable-input precedence, and source-relative path semantics.
The application also has a separate read-only inspection resolver for `config Read-only inspection shares production pipeline and campaign selection without
validate`, `config show`, `config sources`, and `config artifacts`. It uses the session discovery, workspace initialization, adapters, or manifest access. After
same production root, overlay, and campaign selection functions, but never routes applicable campaign and family resolution, a safe typed final-domain DTO contains
through session discovery, concrete artifacts and generated publish rules. It stores credential reference
remote-session download, adapter composition, workspace names rather than typed secrets. A defaults-backed native result freezes this
initialization, manifest access, or cleanup. A pipeline with retained artifact DTO without resolving credentials again; `WriteEffective` supplies show and
family declarations must resolve its selected campaign before ordinary pipeline prepared pipeline YAML, and deterministic JSON of the same DTO supplies the
validation, which expands its canonical-party members and generated publish versioned semantic digest. Resolved resources participate in that digest, while
rules. A pipeline without those declarations may be validated by itself. source layout and secret values do not.
Secret-directory bootstrap remains invocation-scoped and happens before the
ordinary ConfigKit load because explicit typed references must resolve even for
inspection.
`MarshalEffectivePipeline` is the configuration-owned projection for `config Authoring results separately supply native origins, source histories, and empty
show`. It serializes one explicit typed safe projection with the existing YAML optional-import evidence to source and artifact inspection. Family and party
encoder. The projection has no slots for resolution-only artifact family context identifies generated entries without reparsing sources. Diff fully
declarations or private provenance. The result has one trailing newline; resolves each side against the same selected campaign and party, then uses
commands do not marshal runtime objects directly. ConfigKit `Diff` on the safe snapshots. The command presents native JSON Pointer
paths and safe compact values.
`EffectivePipelineSources` and `EffectiveCampaignSources` provide the separate
safe provenance projection for `config sources`. Pipeline ownership begins with
the complete logical field paths projected from the schema and classifies
each contributor as root, import, overlay, or centralized default. It reads
ConfigKit's effective origins and inheritance locations when building each
projection, so inherited values retain their contributors and controlling
directives without a second cached leaf-ownership table. The
projection replaces generated concrete member paths with paired family and
canonical-party records, and does the same for generated publish rules.
Campaign records identify campaign-owned fields and party inputs; canonical
derived players point to the party source. The application command only joins these sorted
records with selection metadata and never reparses configuration files.
`EffectiveArtifactCatalog` projects the same finalized concrete artifact map for
`config artifacts`. It derives source identity from each map key, carries sorted
effective dependencies, aggregates ConfigKit definition contributors, and adds
family/character/party provenance for generated members. Family source evidence
can name multiple contributing files. It does not inspect files named after
artifacts or infer another runtime graph.
`config diff` uses a thin paired loader that performs two independent ConfigKit
loads. When campaign
resolution is needed, the command loads one selected campaign and party and
expands both effective pipelines from that same party value. The configuration
owner projects each normalized effective mapping directly from that same typed
projection into sorted logical paths using JSON number preservation;
mapping leaves are compared individually while sequence values remain atomic.
Values are compact deterministic JSON representations for command output, not
raw YAML fragments, ownership records, or secret material. A differing digest
with no projected difference is treated as an internal consistency error.
Campaign context construction reads the campaign-owned party source through
`ParseParty`. The required canonical party retains its raw bytes and normalized
roster in runtime-only `ResolvedParty` provenance. Canonical resolution creates
a deterministic `derived_from_party` players projection. The party is the only
configured roster authority; player and session-party fields are unsupported.
## Canonical Party Domain ## Canonical Party Domain
`ParseParty` is the package-owned boundary for parsing a party source. Its File-backed party loading enables imports; `ParseParty` accepts contextless bytes.
single-source ConfigKit definition disables imports and overlays, requires The shared definition checks the main document's version and Narratio validates
`schema_version` through `VersionRule`, and strictly decodes the the effective character roster. Native acquisition order and source coordinates
character-keyed typed schema before applying Narratio's party-domain checks. establish roster order, favoring each child's own declaration. A deterministic
Successful metadata coordinates recover authored character-map order without a renderer writes standalone canonical party YAML from that roster; the players
second YAML parse; class and alias slice order comes directly from typed decode. projection and family expansion use the same resolved roster. The external format
The canonical value retains a separate exact byte copy of its source so is defined in the [party contract](../integrations/party.md).
consumers can materialize the authored party document without reserializing it.
Its `PlayersYAML` method deterministically derives the ID-sorted, versioned
players-only projection.
An unversioned source is rejected before resolution, family expansion, or
prepare-stage materialization.
## Validation Ownership ## Validation Ownership
Configuration validation remains within `internal/config`, with the public ConfigKit owns composition, inheritance, presence, typed-secret resolution, and
entry points and pipeline-domain sequencing separated from the private owners provenance. Narratio owns validation and normalization of effective domain
for core pipeline/session rules, external adapters, publish/storage, Notarius, values. Source histories are inspection evidence, not an additional acceptance
Scriptorium, and shared scalar/path checks. These are package-local ownership gate. Focused configuration and application tests exercise imports, resource
boundaries rather than extension points or a general validation framework. contexts, snapshots, secret-safe inspection, party resolution, and materialized
inputs through the production boundaries.
ConfigKit supplies composition, inheritance, presence, typed-secret, and
provenance mechanics; Narratio supplies the domain rules applied to the
effective decoded entries. There is no separate authoring-history acceptance
gate: contributors and declaration history remain inspection evidence, not an
additional validity condition.
The [configuration reference](../config.md#pipeline-imports-and-overlays)
documents sibling syntax, reserved-key migration, and the operator-visible
inspection behavior.
Validation is read-only, fails fast, and does not define which of several
independent invalid fields is reported first. Map-backed declarations are
nevertheless traversed deterministically. Reusable domain normalization is an
explicit preceding operation for both loaded and programmatically constructed
pipelines. Notarius normalization commits canonical scalar, reference, and output
values only after the complete normalization succeeds; its validator then checks
that candidate without changing it. Scriptorium validates artifact declarations
and outputs, dependencies and inputs, variables, referenced outputs, and the final
dependency graph in deterministic artifact order.
Diagnostics identify the relevant semantic field. Merge conflicts also
retain every claiming source needed to correct an additive ownership conflict.
Complete prose, punctuation, formatting, and private validation choreography are
not compatibility contracts.
## Diagnostics And Runtime Metadata
Syntax, duplicate-key, merge, overlay, and schema failures include the
relevant source name and full field path. Additive conflicts report every
claiming source so operators can repair the split without repeatedly
rediscovering additional conflicts.
The loaded pipeline retains private runtime metadata for the absolute root
path, ordered imports, selected overlay name, source, and path, original
ConfigKit metadata and source snapshots, and effective digest. Leaf source
records are projected from that retained native metadata when requested. This
metadata does not participate in YAML decoding or alter the public
configuration model.
The effective digest is SHA-256 over a versioned deterministic JSON encoding of
the same typed safe projection used by show and diff. Runtime Notarius paths remain absolute for
execution, but the digest substitutes their normalized logical values captured
before root-relative resolution, so relocating an equivalent configuration
bundle does not change provenance. Because ConfigKit and resolution metadata
are private, the digest excludes source layout, overlay name, and ownership.
Configuration stores credentials as ConfigKit typed references. A successful
load resolves their values inside the opaque typed boundary, while this safe
projection serializes only `EnvName()` identities. Credential-value changes are
therefore never hashed; reference-name changes remain ordinary provenance.
The optional Notarius credential follows the same path even for disabled or
resumable extraction; normalization copies the opaque reference without reading
or resolving it again. Direct YAML serialization has a configuration-owned safe
projection so only the reference name can leave this boundary.
`recomputePipelineEffectiveDigest` is the single package-owned refresh point
for later runtime expansion.
## Test Surfaces
`pipeline_configkit_test.go` exercises real imports and fixed-path overlays,
base selection, unselected-overlay isolation, source attribution, cross-file
inheritance, and effective composition. Existing focused configuration
tests protect defaults, explicit-value semantics, strict decoding, validation,
metadata, and root-relative path behavior. Application configuration-loader tests protect the
single-read boundary by changing the pipeline file after its initial load and
confirming local session resolution retains the original pipeline. Other
configuration tests continue to protect defaults and validation after assembly.
`party_test.go` protects the versioned typed party schema, domain invariants,
authored character order, retained bytes, and deterministic players projection
without involving campaign or runtime wiring.
`party_resolution_test.go` protects campaign-owned party loading, canonical
input restrictions, source provenance, and virtual players input selection.
## Artifact Family Resolution ## Artifact Family Resolution

View File

@@ -107,7 +107,7 @@ so any earlier replacement can stale a prior cleanup result.
## Focused Documentation ## Focused Documentation
- [Configuration Internals](configuration.md): pipeline imports and overlays, - [Configuration Internals](configuration.md): pipeline imports and overlays,
field ownership, decoding, and root-relative path semantics. field ownership, decoding, and source-relative resource semantics.
- [Adapter Internals](adapters.md): external adapter boundaries, composition, - [Adapter Internals](adapters.md): external adapter boundaries, composition,
failure behavior, and test surfaces. failure behavior, and test surfaces.
- [Artifact Internals](artifacts.md): source identities, runtime catalog, - [Artifact Internals](artifacts.md): source identities, runtime catalog,

View File

@@ -39,8 +39,8 @@ Materialize canonical current-session inputs before processing stages.
- materializes a configured spell catalog with checksum and provenance, or - materializes a configured spell catalog with checksum and provenance, or
safely removes an obsolete canonical spell catalog and its manifest record safely removes an obsolete canonical spell catalog and its manifest record
when the effective input is omitted. when the effective input is omitted.
- copies the validated canonical party bytes unchanged and deterministically - renders standalone effective campaign, session, and canonical party documents,
generates the prepared players projection. and deterministically generates the prepared players projection.
- scans enabled configured artifact inputs for `narratio.previous_session.artifact.*` requirements. - scans enabled configured artifact inputs for `narratio.previous_session.artifact.*` requirements.
- clears managed `previous/` state on every invocation, then, when requirements exist: - clears managed `previous/` state on every invocation, then, when requirements exist:
- resolves the pointer-selected previous source through the shared resolver; - resolves the pointer-selected previous source through the shared resolver;
@@ -71,15 +71,15 @@ previous-artifact requirement set. A change reruns prepare and
stales its fixed descendants. Existing successful records without this stales its fixed descendants. Existing successful records without this
evidence rerun once when selected. evidence rerun once when selected.
Workspace, spool, and cache placement and absolute source relocation are not Workspace, spool, and cache placement are not semantic when logical selection,
semantic when logical selection, canonical names, and bytes are equivalent. canonical names, and effective bytes are equivalent.
The fingerprint deliberately does not read or rehash large audio. Prepared The fingerprint deliberately does not read or rehash large audio. Prepared
input checksums remain the content provenance. Before reusing success, prepare input checksums remain the content provenance. Before reusing success, prepare
validates every durable prepared copy and compares current stable-input bytes, validates every durable prepared copy and compares current stable-input bytes,
canonical party and derived-player bytes, local audio membership/checksums, or effective campaign/session, canonical party and derived-player bytes, local audio
S3 key/size/entity-tag identity with that provenance. Source relocation with membership/checksums, or S3 key/size/entity-tag identity with that provenance.
equivalent names and bytes remains reusable; changed or unavailable evidence Source relocation with equivalent effective documents, names, and bytes remains
causes a normal prepare rerun. reusable; changed or unavailable evidence causes a normal prepare rerun.
## Related Contracts And Tests ## Related Contracts And Tests

View File

@@ -15,8 +15,8 @@ Upload run/session outputs to object storage and atomically advance remote curre
## Outputs ## Outputs
- uploaded session manifest, declared run-local diagnostics, and selected - uploaded session manifest, prepared campaign/session/party inputs, declared
publish outputs; run-local diagnostics, and selected publish outputs;
- uploaded durable previous-session cache files when present; - uploaded durable previous-session cache files when present;
- immutable run-scoped commit manifest; and - immutable run-scoped commit manifest; and
- current commit pointer, written last. - current commit pointer, written last.
@@ -34,7 +34,9 @@ and optional rules, destination resolution, and collision inputs live in
remote verification live in `publish_transfer.go`; committed-manifest, remote verification live in `publish_transfer.go`; committed-manifest,
immutable-commit, and current-pointer encoding and writes live in immutable-commit, and current-pointer encoding and writes live in
`publish_commit.go`. `publish_output_plan.go` owns the deterministic, I/O-free `publish_commit.go`. `publish_output_plan.go` owns the deterministic, I/O-free
five-outcome model and its secret-free metadata projection. five-outcome model and its secret-free metadata projection. Prepared input
objects are committed under the immutable run; remote session loading selects
the committed standalone session through the current pointer.
- when publishing or run upload is disabled, completes successfully with no - when publishing or run upload is disabled, completes successfully with no
outputs and records explanatory metadata. This is not an explicit self-skip: outputs and records explanatory metadata. This is not an explicit self-skip:

View File

@@ -626,6 +626,8 @@ Publish commit model:
- uploads eligible run files under `{session_prefix}/runs/{run_id}/`, excluding - uploads eligible run files under `{session_prefix}/runs/{run_id}/`, excluding
audio and the run-local Notarius staging bundle; audio and the run-local Notarius staging bundle;
- uploads prepared campaign, session, and party documents under the run's
`prepare/inputs/` prefix, when recorded by the manifest;
- uploads configured published outputs and `previous/**` cache files into the - uploads configured published outputs and `previous/**` cache files into the
same immutable run scope, including only explicitly configured extraction same immutable run scope, including only explicitly configured extraction
lanes; lanes;
@@ -714,8 +716,8 @@ plan, so it is not a network-free operation.
Default restore scope: Default restore scope:
- the committed session manifest and the committed transcript/artifact objects - the committed session manifest, prepared campaign/session/party documents,
declared by the selected remote commit and transcript/artifact objects declared by the selected remote commit
- `previous/**` when needed by configured previous-session artifact inputs; - `previous/**` when needed by configured previous-session artifact inputs;
each is selected from its exact committed `published_output` declaration each is selected from its exact committed `published_output` declaration
@@ -821,7 +823,7 @@ ConfigKit.
Every pipeline-loading command bootstraps credentials once, before the first Every pipeline-loading command bootstraps credentials once, before the first
configuration load and before any adapter is constructed. `--secrets-dir` configuration load and before any adapter is constructed. `--secrets-dir`
selects an explicit directory (relative paths use caller CWD). Otherwise a selects an explicit directory (relative paths use caller CWD). Otherwise a
present `secrets/` beside the resolved root pipeline is used, while an absent present `/usr/local/etc/narratio/secrets` directory is used, while an absent
conventional directory is skipped. Explicit missing/empty selections and conventional directory is skipped. Explicit missing/empty selections and
invalid existing directories fail the command. ConfigKit follows a symlink used invalid existing directories fail the command. ConfigKit follows a symlink used
as the selected directory, but immediate entries must be regular non-symlink as the selected directory, but immediate entries must be regular non-symlink

View File

@@ -183,7 +183,7 @@ fail-fast validation of the effective configuration.
values remain present and are evaluated by Narratio's domain rules. values remain present and are evaluated by Narratio's domain rules.
- Validation rejects invalid configuration before stage execution where - Validation rejects invalid configuration before stage execution where
practical. practical.
- Root-owned imports and one selected overlay resolve deterministically through - Local document imports and one selected pipeline overlay resolve through
the configuration owner; commands do not implement their own merge rules. the configuration owner; commands do not implement their own merge rules.
- Native ConfigKit metadata and supported resolution callbacks provide presence, - Native ConfigKit metadata and supported resolution callbacks provide presence,
inheritance, and provenance information without a parallel loading or inheritance, and provenance information without a parallel loading or

View File

@@ -365,7 +365,7 @@ Load examples through production definitions. Check dependent logical IDs,
concrete family members, testing differences, and standalone prepared inputs. concrete family members, testing differences, and standalone prepared inputs.
Keep examples synthetic and credential-free. Keep examples synthetic and credential-free.
## Stage 11: Align canonical documentation and project-specific policy ## Stage 11: Align canonical documentation and project-specific policy [x]
Read documentation/OSS/release policies, affected references, and the approved Read documentation/OSS/release policies, affected references, and the approved
roadmap. Finish current-behavior documentation for the assembled implementation. roadmap. Finish current-behavior documentation for the assembled implementation.

View File

@@ -90,12 +90,12 @@ Symptom:
Likely cause: Likely cause:
- the campaign `party.yml` is an unversioned legacy document, or names a - the campaign party main document is unversioned, or names a
schema other than `narratio.party.v1`. schema other than `narratio.party.v1`.
Safe fix: Safe fix:
- replace the file with one exact, single-document - replace the main file with a versioned
[`narratio.party.v1` party](./integrations/party.md); do not add a version [`narratio.party.v1` party](./integrations/party.md); do not add a version
marker to an otherwise incompatible legacy structure. The versioned party is marker to an otherwise incompatible legacy structure. The versioned party is
the only configured roster authority. the only configured roster authority.
@@ -118,7 +118,8 @@ narratio config sources --config /path/pipeline.yml --overlay testing
Add `--campaign` or `--campaign-file` when the pipeline has party-driven Add `--campaign` or `--campaign-file` when the pipeline has party-driven
artifact families. The output identifies each effective logical field's root, artifact families. The output identifies each effective logical field's root,
import, overlay, default, campaign, party, or family source without printing import, overlay, default, campaign, party, or family source without printing
the field value or credential contents. the field value or credential contents. Empty optional import selections are
reported separately; selected invalid files still fail.
Safe fix: Safe fix:
@@ -958,7 +959,7 @@ Symptom:
Likely causes: Likely causes:
- missing, unreadable, or invalid explicit `--secrets-dir`; - missing, unreadable, or invalid explicit `--secrets-dir`;
- invalid entries in the conventional `secrets/` directory beside the root pipeline; - invalid entries in `/usr/local/etc/narratio/secrets`;
- missing credential environment variables; - missing credential environment variables;
- invalid S3 endpoint/bucket settings. - invalid S3 endpoint/bucket settings.

View File

@@ -222,7 +222,7 @@ func parseInspectionFlags(command string, args []string, out io.Writer) (inspect
flags.overlay.name = "overlay" flags.overlay.name = "overlay"
fs.Var(&flags.overlay, "overlay", "named pipeline overlay") fs.Var(&flags.overlay, "overlay", "named pipeline overlay")
flags.secretsDir.name = "secrets-dir" flags.secretsDir.name = "secrets-dir"
fs.Var(&flags.secretsDir, "secrets-dir", "secrets directory (optional; defaults beside pipeline.yml)") fs.Var(&flags.secretsDir, "secrets-dir", "secrets directory (optional; defaults to /usr/local/etc/narratio/secrets)")
fs.Usage = func() { fs.Usage = func() {
fmt.Fprintf(out, "Usage: narratio %s [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--overlay <name>] [--secrets-dir <path>]\n\n", command) fmt.Fprintf(out, "Usage: narratio %s [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--overlay <name>] [--secrets-dir <path>]\n\n", command)
fmt.Fprintln(out, "Resolves configuration without a session, workspace, manifest, adapters, or external services.") fmt.Fprintln(out, "Resolves configuration without a session, workspace, manifest, adapters, or external services.")

View File

@@ -105,7 +105,7 @@ func parseConfigDiffRequest(args []string, out io.Writer) (configDiffRequest, er
fs.Var(&request.flags.pipelinePath, "config", "path to pipeline.yml (optional; defaults searched)") fs.Var(&request.flags.pipelinePath, "config", "path to pipeline.yml (optional; defaults searched)")
fs.Var(&request.flags.campaignPath, "campaign", "campaign ID") fs.Var(&request.flags.campaignPath, "campaign", "campaign ID")
fs.Var(&request.flags.campaignFilePath, "campaign-file", "path to campaign.yml") fs.Var(&request.flags.campaignFilePath, "campaign-file", "path to campaign.yml")
fs.Var(&request.flags.secretsDir, "secrets-dir", "secrets directory (optional; defaults beside pipeline.yml)") fs.Var(&request.flags.secretsDir, "secrets-dir", "secrets directory (optional; defaults to /usr/local/etc/narratio/secrets)")
fs.Usage = func() { fs.Usage = func() {
fmt.Fprintln(out, "Usage: narratio config diff <left-overlay> <right-overlay> [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--secrets-dir <path>]") fmt.Fprintln(out, "Usage: narratio config diff <left-overlay> <right-overlay> [--config <pipeline.yml>] [--campaign <id> | --campaign-file <campaign.yml>] [--secrets-dir <path>]")
fmt.Fprintln(out) fmt.Fprintln(out)

View File

@@ -37,7 +37,7 @@ func addCommonConfigFlags(fs *flag.FlagSet, flags *commonConfigFlags) {
flags.overlay.name = "overlay" flags.overlay.name = "overlay"
fs.Var(&flags.overlay, "overlay", "named pipeline overlay") fs.Var(&flags.overlay, "overlay", "named pipeline overlay")
flags.secretsDir.name = "secrets-dir" flags.secretsDir.name = "secrets-dir"
fs.Var(&flags.secretsDir, "secrets-dir", "secrets directory (optional; defaults beside pipeline.yml)") fs.Var(&flags.secretsDir, "secrets-dir", "secrets directory (optional; defaults to /usr/local/etc/narratio/secrets)")
} }
func (f commonConfigFlags) sessionOptions() config.SessionLoadOptions { func (f commonConfigFlags) sessionOptions() config.SessionLoadOptions {

View File

@@ -43,7 +43,7 @@ func sessionInitWithDependencies(ctx context.Context, args []string, out io.Writ
overlay.name = "overlay" overlay.name = "overlay"
fs.Var(&overlay, "overlay", "named pipeline overlay") fs.Var(&overlay, "overlay", "named pipeline overlay")
secretsDir.name = "secrets-dir" secretsDir.name = "secrets-dir"
fs.Var(&secretsDir, "secrets-dir", "secrets directory (optional; defaults beside pipeline.yml)") fs.Var(&secretsDir, "secrets-dir", "secrets directory (optional; defaults to /usr/local/etc/narratio/secrets)")
fs.BoolVar(&remote, "remote", false, "write session.yml to S3 session prefix") fs.BoolVar(&remote, "remote", false, "write session.yml to S3 session prefix")
fs.BoolVar(&force, "force", false, "overwrite existing target") fs.BoolVar(&force, "force", false, "overwrite existing target")
if err := parseSessionAwareFlags("session init", fs, args, &sessionID); err != nil { if err := parseSessionAwareFlags("session init", fs, args, &sessionID); err != nil {