Plan the native ConfigKit migration

This commit is contained in:
2026-11-17 20:41:34 +00:00
parent 4a0e3c83db
commit 65d0e518e5
2 changed files with 383 additions and 317 deletions

View File

@@ -7,10 +7,10 @@ authoring and inspection features useful to operators. Align configuration
semantics with other ConfigKit consumers. Existing Narratio assembly conventions
are migration inputs rather than compatibility requirements for this work.
This roadmap defines the target and required changes. A subsequent implementation
plan will define staged work for the agreed target. This pass adds documentation
only. The earlier [v0.5.0 roadmap](config.md) and
[implementation plan](implementation.md) describe the completed baseline.
This roadmap defines the target, user intent, and required changes. The
[implementation plan](implementation.md) defines the work needed to reach that
target. The earlier [v0.5.0 roadmap](config.md) records the completed adoption
that forms the current baseline.
The ownership boundary in [ADR 0003](../adr/0003-assign-configuration-ownership.md)
continues to apply: ConfigKit resolves configuration before its final handoff;
@@ -200,7 +200,7 @@ does not update rendering or diff. Integrate this distinction explicitly:
record domain resolution within a supported resolver when its inputs are
available there; where later campaign resolution remains appropriate, create
a completed native inspection result from the finalized safe domain projection
using the public mapping-source API. That result describes application-derived
using ConfigKit's public loading APIs. That result describes application-derived
values; retained authoring results supply actual file provenance.
Use native comparison rather than maintaining a second general mapping-diff

View File

@@ -1,365 +1,431 @@
# ConfigKit v0.5.0 implementation plan
# ConfigKit v1.2.1 implementation plan
## Scope and authority
Implement [the ConfigKit roadmap](config.md). It owns the target, compatibility
choices, and rationale. This plan covers Narratio's upgrade from v0.4.2 to
v0.5.0 and integration with native sibling inheritance. Audita adoption and
workspace reorganization remain subsequent, separately planned work.
Implement [the approved ConfigKit roadmap](configkit.md). It owns the target,
native-semantics intent, and four approved product decisions. This plan replaces
the completed v0.5.0 plan. Its audience is a Luna coding agent implementing one
end-to-end sprint in the numbered order below.
ConfigKit owns source assembly and native provenance. Narratio validates the
effective domain configuration and exposes the library's metadata to operators.
This revision replaces the earlier plan to repair declaration-history counting
with removal of that additional acceptance rule. It retains supported callbacks,
presence-sensitive defaults, party ordering, and provenance inspection.
The reviewed upstream revision is
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`, tagged `v0.5.0`.
Use the released module, not a local replacement. The pinned
[configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/config.md)
and [consumer guide](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/docs/consumers/README.md)
own upstream semantics. Read their sibling-inheritance and metadata sections
when working on the relevant stages; source may also be inspected in the
selected Go module cache. The sibling checkout is optional reference material,
not a build or test dependency.
Select `go.maximumdirect.net/configkit v1.2.1`, upstream commit
`51ef3144158db1a09fabdbd1c7ae59908c84923b`. The pinned
[consumer guide](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/consumers/README.md)
and [configuration contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/config.md)
own upstream semantics. Read the sections identified by each stage and public
declarations in the released module as needed. The sibling checkout is optional
reference material, not a build or test prerequisite.
## Working instructions
The audience is a Luna coding agent. Complete the numbered stages in order.
Each stage is a bounded unit with its own verification and completion criteria.
Inspect current code before editing; the paths below identify owners rather
than requiring changes to every listed file.
- Start with `docs/development.md` and `docs/policy/agents.md`. Architecture,
testing, documentation, OSS, and release policies apply; reuse unchanged
policy text already read. Prefer graph tools for code discovery.
- Implement the full native configuration workflow. Discovery, destinations,
collection controls, resource bases, secondary imports, materialization, and
native inspection are part of this sprint.
- Stages are work boundaries, not deployment releases. Intermediate stages may
fail to compile or have dependent tests awaiting later work. Record that state
accurately. Avoid temporary adapters or extra abstractions merely to make
every intermediate commit green. Final validation covers the assembled system.
- Commit completed nonempty units with short plain-English messages. Add focused
behavioral tests with their owner; run meaningful available checks. The final
stage runs repository-wide validation against the finished source.
- Use ConfigKit's public loading, presence, provenance, recording, rendering,
and comparison APIs. Narratio supplies domain meaning and runtime use.
Retire obsolete source-authoring checks and tests rather than reproducing them.
- Leave import defaults unspecified so ConfigKit supplies its defaults.
Explicit false/zero/empty/null and collection semantics follow the native
contract and actual Narratio domain rules.
- Use temporary inputs, real ConfigKit, and controlled synthetic credentials.
The default tests remain offline and independent of installed subprocesses.
- Write documentation in concise, affirmative, factual language. The roadmap
owns intent; this file owns sequencing and implementation choices.
- Start with `docs/development.md`, `docs/policy/agents.md`, and `config.md` in
this directory. Follow the development policy-reading guide. Architecture,
testing, documentation, and OSS policies apply to this work; release policy
applies if discussing compatibility or eventual release preparation.
- Use Go 1.26. Prefer the repository's graph tools for code discovery and
inspect focused tests at each package boundary.
- Keep ordinary `Load`, typed defaults, explicit imports, one selected overlay,
domain validation, and current path-resolution rules. Native inheritance is
available wherever ConfigKit and Narratio's schema permit it, with no new
collection allowlist, opt-in flag, or Narratio-owned inheritance engine.
- Trust ConfigKit's structural acceptance rules. Remove Narratio's
one-base-file-per-entry check rather than making it inheritance-aware. Validate
resulting domain values; source history is diagnostic evidence, not another
acceptance gate. Parent entries that survive composition remain ordinary
entries subject to applicable domain rules.
- Use native presence and provenance APIs for defaults and operator inspection.
Keep supported callbacks and party ordering. Do not impose post-`Load` callback
relocation, relaxed empty/null semantics, or retirement of `config sources`.
- Keep invocation-owned secrets bootstrap and shared exact-name credential
forwarding unchanged. Do not introduce a second base load, file reparse, or
historical-secret lookup to reconstruct provenance or validate old input.
- Tests use temporary files, controlled synthetic secrets, and real internal
collaborators. No downstream executable, sibling checkout, cloud service,
or paid provider is needed. Protect integration behavior rather than
duplicating ConfigKit's generic algorithm tests or freezing incidental output.
- Commit each completed nonempty unit with a short plain-English message after
suitable validation. Report blockers and incomplete work accurately. If a
review-only task needs no source fix, retain useful behavioral coverage and
report that result rather than making a cosmetic rewrite.
- Update affected canonical documentation with each implemented behavior change.
In particular, Stage 2 updates the removed authoring restrictions and Stage 3
updates changed inspection meanings. Stage 6 consolidates the policy review,
inheritance guidance, and example; it is not a reason to leave earlier
completed commits describing obsolete behavior.
- For documentation updates, state the policy affirmatively using concise,
factual language. Avoid negative parallelism.
Preserve stage ordering, artifact dependency meaning, the canonical party schema,
exact-name credential forwarding, publish commit semantics, and the existing
Linux amd64/arm64 plus best-effort macOS arm64 targets. This sprint does not
implement pending Audita/workspace features, migrate old sessions, rewrite
downstream configurations, add fragment version mandates or new path checks,
adopt ReleaseKit for Narratio, or publish a release.
Do not adopt layered loading, new source types, source-relative rebasing,
abstract artifacts, automatic YAML conversion, new execution selection rules,
new output collision policy, or changes to subprocess credentials and workspace
paths in this feature. Existing domain validation and reuse rules still apply;
do not add prohibitions, hardening, or guarantees merely because they can be
tested. This plan does not authorize a release, push, tag, ADR acceptance, or
retirement of pending roadmaps.
## Concrete integration choices
## Stage 1: Select the released dependency and update attribution [x]
- Consumer Go minimum: **1.27.0**. Development/release validation and build
toolchain: official **Go 1.27.4**, an available patch on that line. Selecting
it is not a new publication gate.
- Pipeline source when no explicit file is supplied:
`SearchConfig("narratio", Searches(WorkingDirectory(), UserConfigDirectory(),
UnixSystemDirectories()), SearchOptions{Required: true})`.
An explicit `--config` uses `File` exclusively.
- Bootstrap: optional `/usr/local/etc/narratio/secrets`, or explicit
`--secrets-dir`, through `LoadSecretsIntoEnvironment` before typed loading.
Existing environment values win. The implicit sibling directory retires.
- Local campaign/session/party loading enables imports. Only the pipeline gets
its selected overlay. Party version checking remains main-only; fragments
need no new version header. Remote session bytes are contextless snapshots.
- Existing campaign registry selection and local session search locations remain
domain conventions. Use native optional `SearchConfig` with `SearchPaths`
for local session candidates. Only actual absence permits remote fallback.
- Final-domain inspection uses a safe DTO and a defaults-backed
`Load(NoSource(), Definition.Defaults)` snapshot after domain resolution.
Credential fields contain reference names, not typed secrets that would
resolve again. Original results supply actual file provenance. Stages 8–9
explain this boundary.
Purpose: establish the actual dependency baseline before integration work.
## Stage 1: Establish the dependency and toolchain baseline
Read `go.mod`, `go.sum`, `docs/oss/attribution.md`, the OSS policy, and the
selected release's notes, `COPYRIGHT`, and `LICENSE`.
Read `go.mod`, `go.sum`, `.woodpecker/*.yml`, release scripts/runbook, and
attribution. Read the upstream v1.0.0 migration table and v1.2.1 release note.
1. Select `gitea.maximumdirect.net/eric/ConfigKit@v0.5.0` with the normal Go
module workflow. Keep Go 1.26 and avoid unrelated dependency upgrades.
2. Update ConfigKit's attribution entry to v0.5.0 and use the full reviewed
commit for its Source, COPYRIGHT, and LICENSE links. Preserve upstream
copyright wording, SPDX expression, and applicable local license links.
Its license and YAML dependency are unchanged; no new license copy is needed.
3. Inspect module changes for unintended churn or a local `replace`. Existing
configuration loaders should compile without adopting optional APIs.
1. Select Go 1.27.0 in `go.mod` and ConfigKit v1.2.1 under its vanity path.
Migrate all production/test imports together and remove the old requirement.
Use the released module rather than a filesystem replacement.
2. Replace APIs actually used here: `LoadSecretsDir` →
`LoadSecretsIntoEnvironment`, `LayerBase` → `LayerMain`, and generic
`Resolution.Record` calls with `RecordDerivedDefault` or
`RecordNormalization`. Audit other removed public names and compared layer
or diagnostic strings. Use keyed expanded options/definition literals.
3. Rename genuine source-root `configkit:` controls to `config:` throughout
fixtures and examples. Keep the package name, `configkit:"secret"` Go tag,
and ordinary data keys distinct from source controls.
4. Align Go CI images and local/release instructions with Go 1.27.4. Preserve
platform targets and publication mechanics. Update incorporated Go notices.
5. Refresh ConfigKit and stable go-yaml v4.0.0 attribution from selected source
notices. Review actual module graph churn and tool-only dependencies. Keep
Narratio's separate YAML v3 use unless another scoped change needs it.
Verify with `go test ./internal/config ./internal/app ./internal/doccheck`
and `go mod tidy -diff`. Confirm
`go list -m gitea.maximumdirect.net/eric/ConfigKit` selects v0.5.0 without a
replacement.
Check module identity and attribution. Attempt focused config/app tests; record
failures assigned to later stages. Complete when the dependency/API baseline is
established, without preserving old behavioral expectations as compatibility.
Complete when the released dependency and notices agree and current focused
tests pass. The cross-file inheritance defect is addressed in the next stage;
this stage does not claim that new inheritance behavior is fully integrated.
## Stage 2: Integrate native pipeline discovery and independent bootstrap
## Stage 2: Remove source-authoring acceptance gates [x]
Read `internal/app/pipeline_config_path.go`, `secrets_bootstrap.go`,
`command_dependencies.go`, `loadPipelineConfigWithDependencies` and callers,
inspection loaders, and `internal/config/pipeline_configkit.go`.
Read upstream discovery and secrets-bootstrap contracts.
Purpose: accept structurally composed configuration from ConfigKit and reject
only invalid effective Narratio domain configuration.
1. Have the configuration owner accept an explicit file or the native required
discovery source above. Retain explicit-file package entry points where useful.
Commands load through one composition path instead of first probing with
`os.Stat` to obtain a default pathname.
2. Bootstrap once before typed loading or adapter creation from the independent
directory. Preserve explicit-path error behavior, safe partial reports, and
existing-empty environment precedence.
3. Obtain the actual selected root and import/overlay evidence from ConfigKit.
Carry that root into campaign selection, provenance, and command summaries.
Remove dependencies/search helpers used only for old pipeline preselection.
4. Apply this composition to run/range/alias commands, session init/plan/status/
restore, inspection, and remote-session workflows. A selected invalid source
fails rather than falling through to another pipeline candidate.
5. For overlay diff, load the left through native selection and the right from
that selected root with its own overlay. Share invocation bootstrap; neither
side performs a preliminary base-only load.
Read `internal/config/pipeline_configkit.go`, particularly
`pipelineMetadataFromConfigKit` and `rejectDuplicateBaseDefinitions`,
`pipeline_configkit_test.go`, and `expandFamilyPublishRules` and its tests in
`internal/config/artifact_families.go` and `artifact_families_test.go`.
Test precedence and exclusive explicit selection with controlled working/user
directories and temporary native search descriptions. Test selected invalid
input, no source, bootstrap override/default behavior, and diff bootstrap once.
Keep tests isolated from the operator's system configuration and secrets.
1. Delete `rejectDuplicateBaseDefinitions` and its invocation. Do not replace
it with filtering of inheritance events, direct-declaration counting, or
another source-history validator.
2. Replace `TestPipelineConfigKitRejectsDuplicateBaseDeclarations` with behavior
coverage showing that disjoint contributions to one artifact/family compose
successfully when the resulting domain entry is valid. Check selected-overlay
replacement/removal without Narratio rejecting superseded declarations.
3. Add cross-file artifact and family inheritance regressions. Supply valid
surviving parent and child entries with distinct output destinations, and
campaign/party context for family expansion. Verify child identities and
effective fields; the inheritance regression must fail on the old check.
4. Remove the `publishDeclared` gate in `expandFamilyPublishRules`, its metadata
population, and state retained only for that gate. Validate family publication
against the effective `Publish` value, whether authored or supplied by defaults.
Preserve the missing-effective-publish check, ordinary enablement behavior,
destination validation, and conflicts between effective publish rules.
Update the test that currently fabricates an explicit-declaration flag.
5. Retain effective-domain rejection, including invalid artifacts, unresolved
dependencies, and destination collisions under current rules. Reuse existing
tests where adequate. Retain a representative structural-conflict check at
the loader boundary; ConfigKit owns the full conflict/precedence matrix.
## Stage 3: Resolve pipeline resources from native origin context
Verify with `go test ./internal/config`. Assert effective outcomes and safe
error categories, not entire diagnostic text or private metadata event order.
Do not run a separate base-only load to validate overwritten input.
Read `resolvePipelineConfig`, `resolveNotariusPaths`,
`finalizeLoadedPipeline`, `effective_digest.go`, and the upstream source-aware
path example. Resolve paths before normalization erases entry origins.
Complete when valid cross-file composition/inheritance succeeds and invalid
effective configuration still fails before execution, without authored-source
acceptance gates. Keep native metadata available for the next stage.
1. Introduce one small config-owned host-resource base helper using
`Resolution.EntryInfo`, `Resolution.Source`, and
`Resolution.SourceContext`. Use the value's effective origin rather than
its child's inheritance directive. A distinct usable source context supplies
its canonical base; missing or composite context falls back to the main source.
Explicit CLI-origin file references use the captured invocation directory.
If the main input is contextless too, use that invocation directory rather
than interpreting a diagnostic label as a filesystem base. This covers remote
byte snapshots without turning their temporary filenames into resource roots.
2. Normalize nonempty relative references with recorded helpers. Absolute values
stay absolute. Derive Notarius working directory from its already resolved
config path and record that dependency. Preserve domain validation of relevant
empty/disabled values.
3. Apply this classification:
## Stage 3: Simplify provenance inspection around native APIs [x]
| Fields | Interpretation |
| --- | --- |
| Audita/Scriptorium/Notarius `config_path`; Notarius `working_directory` | File/directory resource based on effective origin. |
| Relative `workspace.root`, `campaigns.root`, `spool.root`, `cache.root` | Host directory reference based on effective origin; absolute defaults stay unchanged. |
| Campaign template/stable-input paths; session stable-input/audio paths | Same resource policy, completed in Stage 4. |
| Artifact output paths, family output patterns, publish destinations, source IDs, prompt/profile IDs, schema identifiers, binary commands, Notarius reference bindings and chunk-cache modes | Existing domain interpretation, not file-resource rebasing. |
Purpose: preserve useful operator inspection without a parallel provenance
system or a unique-defining-file contract for composite entries.
4. Remove duplicate pipeline-root rebasing and logical-Notarius path caches once
callers use resolved domain values. Show/digest describe those domain values,
rather than reconstructing an alternative authoring-path view.
5. Keep resource access at existing consumer boundaries; base selection adds no
target-existence check, root restriction, or history acceptance gate.
Read `internal/config/artifact_catalog.go`, `artifact_families.go`,
`effective_pipeline.go`, `effective_digest.go`, and the metadata projection in
`pipeline_configkit.go`. Inspect `internal/app/config_commands_test.go`,
`config_artifacts_test.go`, and ConfigKit's public metadata contracts.
Test imported, inherited, overlay, absent-default, absolute, and ambiguous-origin
cases. Equal resolved resources have equal effective semantics across layouts;
relative text resolving to a different resource is a meaningful change.
1. Keep `config sources` and provenance-bearing `config artifacts` output.
Use public ConfigKit sources, effective origins, history, and explanations
for the information they own. Keep presence queries and recorded callbacks
where needed; retaining returned metadata is not duplicating its implementation.
2. Review existing helper/state layers for duplicated tracking. Remove redundant
cached field-ownership reconstruction where the same result is available
directly from retained metadata. Keep a small projection where needed for
Narratio's generated family/character/party context and output formatting.
Do not add a provenance service, custom ancestry graph, or second source parse.
3. Exercise Stage 2's inherited and multi-file entries through
`EffectiveArtifactCatalog` and `EffectivePipelineSources`. Report actual
contributors and child overrides. Keep historical definition evidence
distinguishable from effective field origins as exposed by ConfigKit.
4. Replace `artifactFamilySource`'s arbitrary first-contributor ownership with
native source/origin evidence. Do not infer a unique base defining file or
implement a new declaration-accounting algorithm. Composite origins may have
several sources; preserve them using existing repeated source records and
comma-separated catalog source lists. Adjust the runtime-only representation
minimally where a single-source field cannot carry that evidence.
5. Preserve logical artifact IDs, family/character identities, party provenance,
inspection command names, and read-only behavior. Retain existing headers
where their meanings remain accurate; explain that family source evidence may
be composite instead of promising a unique defining file. Do not change
native-metadata-based party ordering merely to avoid library APIs.
## Stage 4: Load campaign/session imports with correct contexts
Verify with `go test ./internal/config ./internal/app`. Cover a child family
inheriting across files, an entry composed from disjoint files, and an overlay
override. Check meaningful source records, logical identities, and absence of
secret values rather than full formatted snapshots or upstream event ordering.
Read `document_configkit.go`, `load.go`, `pathresolve.go`,
`party_resolution.go`, app session discovery/remote-session composition, and
session init. Read native import and resource-context contracts.
Complete when native provenance remains useful and truthful, redundant tracking
is removed where practical, and no acceptance rule depends on contributor count
or an inferred defining file. Application derivation context remains available.
1. Use `File` with imports enabled for local campaign/session inputs. Share
definitions with byte loading and retain successful results/source contexts.
ConfigKit owns per-source and aggregate acquisition limits.
2. Replace local session candidate probing with native optional discovery over
the existing candidate paths. Permit absent input solely for remote fallback;
distinguish absence through native source evidence. Structure callbacks to
allow that discovery-only no-source result. Acquired invalid files still
receive ordinary validation and fail without remote fallback.
3. Downloaded session bytes receive no context inferred from their temporary
filename. Retain object provenance and current explicit session/previous-ID
checks. Use native contextless-import behavior; no remote import fetcher or
custom structural rejection layer is added.
4. Resolve campaign `session_template_file`, stable-input paths, and session
overrides/audio references with Stage 3's helper. Resolve each
`audio_files` element at its entry index before ordering. Keep campaign/
session stable-input precedence and campaign ownership of party input.
Provenance identifies the effective supplying configuration source.
5. Check unresolved template tokens in effective session string values, including
imported values. Session init still reads an authored template for intentional
substitution; runtime loading stays a configuration operation.
6. Retain detached campaign and session document values after their own loading
and resource resolution. Capture the session document before campaign-inherited
fields are copied into runtime session state. Stage 6 serializes these values.
## Stage 4: Verify inherited presence, domain resolution, and path bases [x]
Test local imports, resource bases, effective placeholder rejection, identity
mismatches, imported session-init template paths, absence versus selected-source
failure, and remote snapshots independent of temporary directories. Add no
secondary overlay flags.
Purpose: prevent Narratio's application defaults or normalization from undoing
the library's inherited explicit input.
## Stage 5: Resolve imported party definitions and canonical roster output
Read `pipelineConfigDefaults`, `resolvePipelineConfig`, and
`internal/config/load.go`, including `finalizeLoadedPipeline` and
`resolveNotariusPaths`. Inspect the current presence/defaulting tests and
the relevant domain validators.
Read `party.go`, `party_resolution.go`, `artifact_families.go`, the party
integration contract, and focused party/family tests.
1. Add representative inheritance cases for explicit false, zero, empty,
and null input versus genuinely absent input. Include a pointer boolean
and an entry-level absence-based default where Narratio consumes metadata.
For example, inherited `render_debug: false` can exercise artifact presence;
inherited empty Scriptorium timeout can exercise an existing invalid value
that must not silently become its default.
2. Preserve the existing domain outcome for each chosen field. Valid explicit
values retain their value; explicit values already invalid must still fail
ordinary validation rather than be refilled. An empty mapping merges under
ConfigKit's rules and must not be described as universal container removal.
3. Exercise overlay composition before inheritance: modify a parent's field
in the selected overlay and confirm the child's inherited result uses that
effective value. A child override still wins over its parent.
4. Verify a known root-relative field, such as enabled Notarius `config_path`
or `working_directory`, when inherited from an imported declaration.
Resolve it exactly as the existing field contract does. Do not use passive
SourceContext to rebase it to the parent's or child's source directory, or
newly normalize other path fields merely to make their behavior uniform.
5. Fix any demonstrated Narratio integration defect at its existing defaulting
or domain-resolution owner. Use ConfigKit presence and recorded operations;
retain supported defaults/resolver/validator callbacks. There is no required
move to post-`Load` functions. Do not recreate structural inheritance, inspect
old input to add acceptance rules, or relax empty/null semantics to avoid
using the library's presence API.
1. Add a file-backed party entry point with imports enabled for campaign
resolution. Keep `ParseParty(bytes)` contextless for snapshots and tests.
Reuse the definition, main-only version rule, and domain validator.
2. Preserve `narratio.party.v1`, existing character/name/class/alias validation,
and campaign ownership. Native destinations allow split roster contributions;
no fragment version mandate or abstract/template record type is added.
3. Use native metadata for roster order and evidence, preserving authored order
within a source. Extend the current ordering helper using source acquisition
order, then line/column and ID as deterministic ties across sources. Prefer a
child's own declaration/inheritance location over its parent's location.
This is a presentation convention, not a new source acceptance rule.
4. Render canonical party YAML from the effective roster: schema version, keyed
characters, singular external `alias`, classes and optional levels. Preserve
meaningful roster/list order. A small typed DTO/YAML-node writer is suitable
for this Narratio-owned external format.
5. Keep players derivation and family expansion based on that same roster.
Replace output/checksum uses of `CanonicalParty.Raw` with canonical rendering;
retain raw data only where an actual diagnostic caller needs it.
Verify with `go test ./internal/config`. Keep coverage representative; generic
missing-parent, cycle, resource-context, and expansion-limit matrices belong to
ConfigKit. Parents must remain valid Narratio entries, not special templates.
Test split/inherited rosters, aliases/classes, effective duplicate-name rejection,
order, reloadable standalone canonical output, players, and family identities.
Use native evidence rather than a second source parse to reconstruct order.
Complete when inherited explicit input and absent defaults have distinct,
correct domain outcomes and existing path bases are unchanged.
## Stage 6: Materialize effective inputs and align prepare resume checks
## Stage 5: Verify safe effective projections and inherited credentials [x]
Read `internal/stage/prepare.go`, `prepare_party.go`, `prepare_resume.go`,
prepared-input descriptors, and prepare semantic evidence/tests.
Purpose: ensure inheritance changes neither semantic comparison nor the
existing secret boundary.
1. Add configuration-owned deterministic campaign/session renderers from Stage
4's detached document values. Serialize public schemas and resolved file
references, excluding native directives and runtime/provenance state.
2. Preserve ownership: campaign-inherited stable inputs must not become explicit
session overrides. A published session that omitted `speakers_file` must
still inherit the receiving campaign's value. Explicit session overrides
remain explicit. Include actual session identity values where application
resolution supplies them, without freezing campaign defaults into the session.
3. Prepare writes these documents and Stage 5's party bytes through existing
artifact-store helpers. Keep filenames, input kinds, provenance fields, and
write/checksum behavior. Resume uses the same renderers instead of comparing
prepared effective output with a raw imported root.
4. Ensure imported effective changes reach existing stage-owned semantic/input
evidence. Equivalent domain values remain equivalent. Full pipeline digests,
source mtimes, or every authoring file are not new blanket invalidation keys.
5. Preserve ordinary stable-file copying/audio handling. Already resolved paths
must not be rebased again against the session root. Retire raw campaign/session
file requirements used solely for copying the old authoring documents.
Read `internal/config/effective_pipeline.go`, `effective_digest.go`,
`effective_projection.go`, `scriptorium_credentials_test.go`, and the effective
projection tests in `load_validate_test.go`. For diff composition, read
`internal/app/config_diff.go`. Reuse
`TestPipelineOverlayDoesNotRequireUnusedBaseCredentials` and
`TestCredentialValuesNeverEnterEffectiveProjectionOrDigest` coverage.
Test standalone prepared documents, imported changes, equivalent layouts, and
remote-origin metadata. Canonical-byte format changes can cause a normal prepare
rerun; no session migration or new manifest schema is needed for serialization.
1. Load two configurations that produce the same effective domain values:
one uses inheritance, the other writes the expanded values explicitly.
Include ordinary artifacts and representative resolved family members.
Verify equal safe effective mappings and digests and no effective diff.
Do not compare private provenance as part of semantic equality.
2. Add a representative typed credential inheritance case using shared fields
of valid Narratio sibling mappings and controlled synthetic environment
variables. Verify resolved credentials retain their exact configured names;
serialization and inspection expose those identities without values.
3. Verify overlay removal/replacement of a credential before inheritance does
not resolve the superseded reference. Reuse the existing selected-overlay
regression rather than creating a second loader or bootstrap algorithm.
4. Check that changing only synthetic secret values leaves semantic projections
and digest unchanged. Fix only demonstrated projection defects; preserve
the existing safe typed rendering boundary and semantic reuse evidence.
## Stage 7: Close publication, restore, and remote-session round trips
Verify with `go test ./internal/config ./internal/app`. Existing shared adapter
credential-forwarding tests remain the owner of subprocess forwarding; do not
rewrite adapters or duplicate those tests simply because input can be inherited.
Read publish input selection, remote-session loading, restore contracts, and
existing in-memory storage integration tests.
Complete when resolved values, not source layout or secret bytes, determine
effective comparison, and inherited credentials obey the existing safe boundary.
1. Trace the actual bytes published as remote `session.yml`. Publish the canonical
prepared session rather than the original root or command-scoped download.
Preserve immutable object/commit-pointer ordering and lock behavior.
2. Confirm campaign/party prepared products flow through existing selections
with their identities intact and without unresolved companion imports.
3. Verify a published canonical session loads through the existing contextless
path after authoring fragments and temporary files disappear. Restore uses
recorded contents and existing safety/locking. No remote import graph or
producer-host filesystem search is introduced.
4. Verify omitted session stable fields inherit the receiving campaign's values.
Keep remote object identity evidence separate from checksums of locally
materialized canonical bytes.
## Stage 6: Align policy, document inheritance, and maintain an example [x]
Use fake/in-memory storage for prepare → publish selection → remote load and
relevant restore coverage. Fix actual producers/consumers rather than adding a
parallel publication path. Existing atomic commit tests remain authoritative.
Purpose: make the adopted behavior understandable and usable without expanding
Narratio's configuration surface.
## Stage 8: Build native final-domain inspection snapshots
Read `docs/config.md`, `docs/internal/configuration.md`, `examples/README.md`,
`docs/policy/architecture.md`, the production/testing bundle, and
`TestExamplesLoadAndValidate` in
`internal/config/load_validate_test.go`. Follow the documentation policy.
Read `effective_projection.go`, `effective_digest.go`,
`effective_pipeline.go`, pipeline finalization/family expansion, and native
completion, defaults, rendering, and typed-secret contracts.
1. Revise only the project-specific Configuration section of the architecture
policy to record the implemented boundary: ConfigKit owns loading, assembly,
inheritance, presence, secrets, and provenance; Narratio owns domain meaning
and fail-fast effective validation. Permit native metadata inspection and
supported callbacks. Align the ambiguous empty-value statement with existing
explicit-presence defaults, not a new blank/null policy. Leave general
upstream agent, testing, documentation, and OSS policy wording unchanged.
2. Add native sibling inheritance guidance to the configuration reference.
Explain exact sibling lookup, overlay-before-inheritance resolution,
explicit presence, ordinary parent validation, distinct child identity,
and native contributor inspection. Remove the one-base-file restriction and
describe acceptance of ConfigKit-composed entries. Update family-publish
guidance for effective configuration rather than explicit authoring. Link to
the pinned upstream contract for detailed syntax and structural rules.
3. Explain the reserved `extends` key throughout ConfigKit inputs, including
dynamic artifact IDs and prompt-variable mappings. Operators must rename
existing ordinary-data keys with that name; there is no opt-in or automatic
conversion. Campaign, session, and party inputs also use ConfigKit.
4. Clarify the existing statement that overlays cannot inherit: overlay files
cannot inherit or stack, but the selected overlay may supply native sibling
directives within its values. Keep imports and overlay selection unchanged.
5. Update configuration internals for inheritance before typed decoding,
removal of authoring gates, thin native contributor projection, and the safe
effective boundary. Describe implemented mechanisms, linking to the public
reference for their operator-facing meaning. Update `docs/cli.md` where source
output now reports composite evidence rather than a unique defining file.
6. In `examples/production-testing/overlays/testing.yml`, add
`extends: session_recap` to the existing `testing_notes` artifact. Keep its
disabled status and distinct output override. It inherits the imported
parent's prompt/input and the parent's testing profile after overlay
application. The parent remains an ordinary artifact.
7. Update the example guide and extend existing example validation to check
those inherited fields. Preserve the bundle's current identities and
artifact count; no new example root or template convention is necessary.
1. Keep one safe schema-specific final-domain DTO containing concrete expanded
artifacts and generated publish outputs. Omit unexpanded family declarations
and bookkeeping. Audit all credentials, including S3 fields and Audita/
Notarius/Scriptorium lists: use exact reference names, with no resolved
`Secret` fields in the inspection DTO.
2. Build a native DTO result with `Load(NoSource(), Options{Definition:
Definition{Defaults: ...}})`, supplying a fresh safe DTO and no external
credential lookup or application callback rerun. ConfigKit copies and freezes
these already resolved values. Using defaults avoids interpreting final
`config`/`extends` data as fresh directives or converting precise numbers
through a generic mapping-source reconstruction.
3. Build/refresh the snapshot at existing domain-finalization boundaries, after
applicable path/dependency/family resolution. Keep original authoring results
separately. Synthetic default origins describe this derived snapshot, not
original file provenance. No general snapshot service is needed.
4. Route effective show and `pipeline.resolved.yml` rendering through native
`WriteEffective`. Use the same DTO's deterministic JSON for Narratio's
semantic digest. Preserve exact integers, domain null/empty values, and
credential identities. Update the digest discriminator if its format changes.
5. Remove flattening/serialization machinery used only to duplicate native
show/diff. Keep narrow path enumeration and family/party context genuinely
needed by catalog/source presentation and stage-owned reuse evidence.
Verify with `go test ./internal/config -run '^TestExamplesLoadAndValidate$'`
and `go test ./internal/doccheck`. Review examples for synthetic-only content
and validate local links. Keep unimplemented Audita/workspace behavior in its
roadmaps. Do not create a release note under an invented Narratio version;
reserved-key compatibility impact remains an explicit release-preparation item.
Test final family/publish expansion, inherited/expanded equivalence, large
integer variables, null/empty values, safe rendering, credential-reference
changes versus rotation, and no credential resolution during snapshot creation.
Complete when canonical documentation and a tested copyable example describe
the implemented adoption, without promoting new restrictions to policy.
## Stage 9: Adopt native diff and truthful operator source inspection
## Stage 7: Verify the assembled inspection workflow and close the feature [x]
Read `internal/app/config_diff.go`, config command loaders/tests,
`artifact_catalog.go`, and native Diff/source/history contracts.
Purpose: establish that real command composition consumes the integrated
configuration correctly and finish the upgrade as a reviewable baseline.
1. Fully resolve each overlay against the same selected campaign/party before
comparing Stage 8 snapshots with `configkit.Diff`. Remove
`diffEffectivePipelineValues`, its general map comparison, and consistency
checks predicated on the old flattened-leaf change model.
2. Present canonical JSON Pointer paths in native order. Derive added/removed/
changed from snapshot `Present`, not nil or redaction text. Use compact JSON
for safe values and expose `HiddenChanged` if present. Equal values produce
`no differences`. Document pointer notation and native changed-region/list
granularity instead of emulating the old engine.
3. Source/catalog evidence comes from authoring results through public source
lookup and metadata history/origins. Add clearly labeled empty optional-import
selection evidence to `config sources`, without guessed candidates or
reasons for absence.
4. Preserve artifact/family/character IDs and generated party context. Distinguish
normalization origins from retained definition evidence. Use native `main`
where an upstream layer is reported; root/import presentation roles may remain
where they express actual application context.
Read `internal/app/config_commands_test.go`, `config_artifacts_test.go`,
and existing read-only inspection workflow coverage. Reuse the preceding
package coverage rather than repeating its entire matrix at the CLI boundary.
Test changed family members and publish rules, equivalent layouts, composite
contributors, optional empty imports, atomic lists, and pointer escaping in
allowed dynamic mapping keys. Inspection remains secret-safe and read-only,
without source reparsing or provenance-based acceptance gates.
1. Add or extend a representative command-level temporary imported bundle with
inheritance. Exercise validation, effective show, source reporting, artifact
catalog, and diff through existing command entry points. Use a campaign/party
fixture for family inspection, and paired overlays with equivalent inherited
and explicitly expanded values when testing command-level diff equality.
2. Check semantic output: resolved child values, native contributor/origin
sources, logical artifact identities, and no secret values. Keep inspection
read-only with respect to session/runtime state, as existing tests establish.
Do not invoke downstream applications or introduce new inspection modes.
3. Review the combined diff against `config.md` and this plan. Confirm that
ConfigKit v0.5.0 is selected, attribution links use the reviewed revision,
authoring-history acceptance checks were removed, native inspection and
presence semantics remain, and policy reflects that boundary. Confirm no
optional loading feature, Audita/workspace change, or compatibility shim
slipped into scope.
4. Run `go test ./...`, `go test -race ./...`, `go vet ./...`,
`go build ./...`, `go mod tidy -diff`, and `git diff --check` against the
final source. These reuse existing repository checks, not new release gates.
The full suite includes example and documentation-link validation; focused
checks need not be repeated solely for this stage if unchanged.
5. Commit the completed unit and report the dependency revision, checks,
compatibility impact, and any unfinished work. Leave the pending Audita and
workspace plans and Proposed ADRs intact. Retire completed planning files
only during separately requested cleanup/release preparation, repairing links.
## Stage 10: Exercise native authoring and maintain ergonomic examples
Complete when the feature roadmap's completion criteria are evidenced by the
released dependency, integrated behavior, maintained documentation/examples,
and passing final checks. The next implementation scope is Audita; its concrete
plan is prepared separately. Release version selection and publication are
outside this implementation pass.
Read the production/testing bundle, artifact/family tests, native destinations
and collection rules, and maintained example tests.
1. Convert artifact/family files to destination-based definitions. IDs remain
explicit destination keys or keyed payloads; filenames do not imply IDs.
2. Demonstrate a declared directory import with `pattern: "*.yml"`, essential
`required: true` input, an optional selection, inheritance, and a useful
testing-overlay list customization through `config.sequences`.
3. Cover append/upsert/replace on an existing list using an appropriate authored
identity, such as publish output `source`. Whole-record upsert and empty-list
behavior follow ConfigKit. No application identity registration is required
merely to enable authored collection controls.
4. Demonstrate a split roster and local secondary imports using the sample
campaign, with companion resource paths following source-origin policy.
Explain that full-root and disjoint definitions remain accepted too.
5. Use ordinary `Load` for this authored-import workflow. Layered loading is
useful for actual lower-priority sources, not required for defaults alone.
If such a source is genuinely needed, use `LoadLayered` and `MainRelative`
directly with focused coverage instead of another merge/search mechanism.
Load examples through production definitions. Check dependent logical IDs,
concrete family members, testing differences, and standalone prepared inputs.
Keep examples synthetic and credential-free.
## Stage 11: Align canonical documentation and project-specific policy
Read documentation/OSS/release policies, affected references, and the approved
roadmap. Finish current-behavior documentation for the assembled implementation.
1. Update config/CLI references, operations/bootstrap guidance, troubleshooting,
configuration internals, prepare internals, and party integration docs for the
new discovery, import, resource-base, materialization, and inspection behavior.
2. Replace obsolete import restrictions with an accurate native summary and
pinned upstream links. Explain optional imports, `required`, filters,
destinations, source-local root controls, list modes, and reserved-key scope.
Keep effective domain validation Narratio-owned.
3. Align project-specific architecture configuration/startup statements and the
current raw-party-copy description in its actual owner. Preserve accepted
ADR 0003 and generic adopted policies. Routine integration details do not
require new ADRs.
4. Write manual upgrade guidance for control names, native discovery filenames/
precedence, explicit custom filenames, secrets override, relative path changes,
required imports where desired, and list operations. Explain effective prepared
documents versus authored roots; downstream migrations remain separate.
5. Finish Go prerequisites, CI/runbook toolchain alignment, and attribution from
Stage 1. Record impact for eventual major-release preparation without inventing
a release version or changing published notes. Leave pending Audita/workspace
roadmaps and Proposed ADRs intact.
Run doccheck and maintained example validation. Each volatile contract has one
canonical owner. Retire completed roadmap files only when separately requested.
## Stage 12: Verify the assembled migration and close the sprint
1. Extend a command-level temporary bundle with native discovery, explicit
secrets override, imported campaign/party/session resources, destination-based
artifacts, family expansion, and a testing overlay. Reuse preceding focused
coverage instead of repeating ConfigKit's structural matrix.
2. Exercise validate/show/sources/artifacts/diff through real composition. Check
selected root, resources, generated values, optional-import evidence, safe
credentials, and absence of runtime workspace creation by inspection. Exercise
plan or prepare with fake external seams to establish execution parity.
3. Trace remaining callers of old search, rebasing, raw-copy, and diff helpers.
Remove dead code/test seams. Review intentional test expectation changes
against the roadmap and preserve existing consequential domain/safety tests.
4. With Go 1.27.4 and dependencies prepared, run `go test ./...`,
`go test -race ./...`, `go vet ./...`, `go build ./...`,
`go mod tidy -diff`, and `git diff --check`. Use the existing release asset
builder with a temporary output directory and its existing `v0.0.0-test` label
to check the three targets; this invokes no publication or new release note.
5. Inspect `go list -m` for the exact release, absence of the old module path
and filesystem replacements, and expected notices. Review every roadmap
capability and all four approved decisions; native structural behavior must
not be narrowed by new Narratio acceptance gates.
6. Commit the finished unit and report checks, compatibility changes, revision,
and any genuine unfinished work. Leave a clean workspace. Actual release
publication and the next feature require separate authorization.
Complete when the native workflow works end to end, prepared inputs survive
loss of authoring companions, operator inspection covers final domain values,
and final validation passes.
## Open Questions
None required for this adoption. Existing decisions select native inheritance,
native metadata inspection and callbacks, effective-domain validation instead
of authoring-history gates, current path/credential/presence semantics, and
ConfigKit before Audita before workspace work. Eventual release preparation must
account for the reserved-key compatibility impact and inspection representation
changes under the release policy; no version is selected here.
None required for this sprint. The four product choices are approved. This plan
selects concrete native snapshot, source-context, materialization, and toolchain
approaches without adding compatibility shims, structural restrictions, or
stronger runtime guarantees.