Remove completed ConfigKit planning documents
All checks were successful
ci/woodpecker/push/verify Pipeline was successful

This commit is contained in:
2026-11-18 14:33:12 +00:00
parent 4dae8e0d36
commit fa9ed78a6d
4 changed files with 0 additions and 1448 deletions

View File

@@ -1,243 +0,0 @@
# Adopt ConfigKit v0.5.0
## Purpose
Upgrade Narratio from ConfigKit v0.4.2 to v0.5.0 and integrate its native
sibling inheritance with Narratio's configuration validation and inspection.
Keep the upgrade focused on the existing loading model and the adjustments
needed to consume the new release correctly. Rely on ConfigKit's assembly and
metadata features; validate Narratio's effective domain configuration without
adding acceptance rules about how that configuration was authored.
This adoption precedes [Audita adoption](audita.md) and the
[workspace reorganization](workspace.md). This document defines the target
and required changes; [the implementation plan](implementation.md) defines
the steps for reaching that target.
## Reviewed Upstream Release
ConfigKit v0.5.0 identifies commit
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`. The
[release notes](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/releases/v0.5.0.md),
[configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/config.md),
and [consumer guide](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/consumers/README.md)
own the upstream behavior.
The release introduces native `extends` relationships between sibling mappings,
layered loading with ordered underlays, caller-owned filesystem and mapping
sources, and passive source-resource contexts. Inheritance preserves explicit
presence, provenance, and secret classification.
The existing loading API remains available. ConfigKit's minimum Go line, YAML
dependency, and LGPL-3.0-or-later license are unchanged. Narratio can retain its
current Go toolchain and ordinary loading calls.
## Current Baseline
The [configuration reference](../config.md) owns Narratio's current public
semantics; [configuration internals](../internal/configuration.md) describe
loading and inspection.
- Pipeline loading uses ConfigKit v0.5.0 with explicit additive imports and one
selected overlay, typed defaults, recorded resolution, and domain validation.
- Campaign, session, and canonical-party inputs use separate single-source
loads. Secret-directory bootstrap remains an explicit invocation operation.
- Fields with root-relative pipeline path semantics retain that base when their
declarations move into imports. For example, enabled Notarius configuration
and working-directory paths resolve from the root pipeline directory.
- Distinct fields of one artifact or artifact-family entry can compose across
base files under ConfigKit's structural rules. Narratio validates the resulting
effective entry, and the selected overlay may replace or remove those values.
- Source inspection and artifact catalogs consume retained ConfigKit metadata.
Effective show, diff, and digest projections describe the resolved domain
configuration rather than source layout.
The released v0.5.0 module is now selected directly, and its attribution points
to the reviewed commit. Cross-file artifact and family inheritance and disjoint
entry composition now pass through Narratio's effective configuration checks.
The remaining integration work verifies native provenance, explicit presence,
safe projections, and the assembled inspection workflow described below.
## Targeted End State
### Native inheritance
Use ConfigKit's built-in sibling inheritance wherever its input contract and
Narratio's domain schema permit it. ConfigKit resolves `extends` after source
composition and overlay precedence, before typed decoding and application
resolution. Narratio receives the resulting typed configuration.
Inheritance can reduce repetition between artifacts and artifact families,
including definitions in separate imported files. Distinct child keys retain
their identities. Each resulting entry undergoes ordinary Narratio validation,
family expansion, dependency resolution, and execution selection.
Parent entries remain ordinary domain entries. They must satisfy applicable
validation and participate in the effective configuration. This adoption does
not introduce abstract artifacts or a separate template registry. Operators
must override inherited fields where needed to keep the resulting configuration
valid, such as output destinations.
Inherited false, zero, empty, and null input retains ConfigKit's explicit
presence semantics. Narratio's absence-based defaults must not refill such
values. Normal domain validation still determines whether a resulting value
is acceptable.
### Effective configuration and native provenance
Remove Narratio's one-base-file-per-artifact/family restriction. If ConfigKit
accepts composition of one entry from several files, Narratio accepts that
entry when its effective domain values are valid. ConfigKit continues to reject
structural conflicts under its own contract. Narratio does not reconstruct
declaration history to add another conflict rule or reject domain values that
were removed or replaced before the final configuration handoff.
Domain validation concerns the effective configuration, including application
resolution and family expansion. Family publication requires an applicable
effective publish configuration, not separate evidence that the section was
explicitly authored rather than defaulted. Existing checks on destinations,
dependencies, identities, and effective publish rules remain.
Retain `config sources` and useful provenance in `config artifacts`. Consume
ConfigKit's public source, origin, presence, history, and explanation APIs as
intended. A thin formatting/projection layer may add Narratio-specific family,
character, and party context. It must not duplicate upstream source tracking,
inheritance analysis, or derivation bookkeeping.
An entry can have several contributors. Report the evidence ConfigKit provides
without inventing a unique defining-file guarantee or choosing an arbitrary
contributor as the owner. Keep logical artifact IDs independent of source files.
Small representation changes needed for truthful composite provenance belong
to this adoption; retiring inspection commands does not.
Effective configuration, diffs, and digests continue to reflect resolved
domain values. Equivalent inherited and explicitly authored configurations
should have equivalent effective projections, regardless of their provenance.
Stage reuse continues to use existing semantic evidence.
### Loading and ownership boundaries
Retain ordinary ConfigKit loading, centralized typed defaults, imports, selected
overlays, domain resolution, credential bootstrap, and exact-name credential
forwarding. Preserve current path bases and read-only inspection workflows.
ConfigKit owns parsing, source composition, structural inheritance, presence,
provenance, and secret resolution. Narratio owns its schema, domain defaults,
semantic normalization and validation, campaign/family resolution, and runtime
use. Supported defaults, resolver, and validator callbacks remain appropriate;
there is no requirement to move them after `Load` merely to establish this
ownership boundary. ConfigKit presence APIs remain suitable for absence-based
defaults, and native metadata remains suitable for operator inspection and
existing party ordering.
Layered loading, embedded configuration sources, programmatic mapping sources,
and source-relative resource rebasing are available upstream but are outside
this focused adoption. Do not relax empty/null default semantics or change
party ordering just to avoid using useful native APIs.
## Required Changes
### Dependency and attribution
Update `go.mod` and `go.sum` to ConfigKit v0.5.0. Update its entry in
[attribution](../oss/attribution.md) to the selected version and the reviewed
commit's source and notice links. Existing applicable license texts can remain.
### Configuration integration
Delete the duplicate-definition check and its obsolete declaration-accounting
tests. Remove the family-publish explicit-declaration gate and metadata retained
only for that gate. Use existing final-value domain checks instead.
Simplify source and artifact-definition projections around ConfigKit's public
metadata. Remove duplicated tracking or inferred ownership where native evidence
already provides the needed information. Retain application context and
formatting that actually serve operators. Use the completed load's evidence
rather than reparsing sources or performing a second base-only load that could
resolve superseded credential references.
Keep effective rendering and digest construction grounded in the existing safe
typed projection, with credential identities rather than resolved values.
Use ConfigKit's native operation directly. No application-owned inheritance
engine, collection registration, alternative keyword, or compatibility loader
is needed.
### Policy alignment
Revise the Configuration section of the
[architecture policy](../policy/architecture.md) to record this responsibility
boundary: native loading and metadata belong to ConfigKit; Narratio owns domain
meaning and fail-fast effective validation. Explicitly permit supported
callbacks and native provenance inspection. Remove wording inconsistent with
the established explicit-presence/defaulting semantics rather than introducing
a new empty-value policy. General agent, testing, documentation, and OSS
policies need no broad revisions for this feature.
### Compatibility and operator guidance
Document that `extends` is now reserved structural input throughout ConfigKit
loads, including dynamic mappings such as artifact IDs and prompt variables.
Operators using that key as ordinary data must rename it. Inheritance is active
without an opt-in switch; imports and overlay-file selection remain separately
controlled as before.
Explain native sibling lookup, inherited explicit presence, and the requirement
that parent and child entries satisfy Narratio's domain rules. Link to the
pinned upstream contract for detailed inheritance semantics. Clarify that a
selected overlay may contribute sibling inheritance directives; this does not
add inheritance or stacking between overlay files.
Replace documentation of the one-base-file restriction with ConfigKit's
composition contract and Narratio's effective-value validation boundary.
Describe any inspection representation changes without promising a unique
defining file for a composite entry.
Update the configuration reference, internal configuration guide, and a small
maintained example when implementation lands. Preserve each field's current
path-resolution rule when its value is inherited. Record the reserved-key
compatibility impact in the eventual release note; automatic configuration
conversion is outside this feature.
### Verification
Add focused Narratio coverage for cross-file artifact/family inheritance and
valid disjoint contributions to one entry. Verify that effective invalid
configuration still fails before execution and that structural conflicts still
surface as ConfigKit errors. Old source-history rejection tests are retired or
rewritten for the new contract, including overlay replacement/removal cases.
Verify inherited explicit values against Narratio's defaulting rules, credential
safety, native contributor reporting, and equivalent effective projections.
Concentrate tests on Narratio's integration behavior. ConfigKit's own tests own
its generic inheritance algorithm and source lifecycle. Narratio's default
tests remain offline, use temporary configuration bundles and controlled secret
values, and do not depend on the sibling ConfigKit checkout. Run the existing
suite against the selected released module before completing the upgrade.
## Completion Criteria
Narratio selects the released ConfigKit v0.5.0 module without a local checkout
replacement. Cross-file artifact and family inheritance works through the
ordinary loader; valid composition is not rejected by Narratio's declaration
history checks. Effective domain errors fail before execution. Native provenance
remains inspectable without unique-file assumptions; effective projections,
explicit-presence defaulting, and credential handling retain their semantics.
Maintained guidance and examples explain native inheritance and the reserved
`extends` key, and the architecture policy reflects the ConfigKit/Narratio
boundary. Attribution identifies the selected upstream revision. Focused
regression coverage and the existing suite pass without implementing Audita or
workspace changes as part of this upgrade.
## Relationship To Subsequent Work
ConfigKit adoption provides a verified configuration baseline for the later
Audita and workspace features. It makes no changes to subprocess destinations,
session layout, or cleanup behavior.
Audita adoption then migrates downstream profiles and adds adapter support for
explicit report/debug destinations. Audita owns its separate ConfigKit and
PromptKit dependencies. Shared latest-attempt links, final workspace paths, and
retention protection remain part of the workspace reorganization, so those
facilities are implemented once at their shared owner.

View File

@@ -1,315 +0,0 @@
# Adopt ConfigKit v1.2.1 and native configuration ergonomics
## Purpose
Adopt current ConfigKit as Narratio's configuration owner and make its native
authoring and inspection features useful to operators. Align configuration
semantics with other ConfigKit consumers. Existing Narratio assembly conventions
are migration inputs rather than compatibility requirements for this work.
This roadmap defines the target, user intent, and required changes. The
[implementation plan](implementation.md) defines the work needed to reach that
target. The earlier [v0.5.0 roadmap](config.md) records the completed adoption
that forms the current baseline.
The ownership boundary in [ADR 0003](../adr/0003-assign-configuration-ownership.md)
continues to apply: ConfigKit resolves configuration before its final handoff;
Narratio owns domain validation, application derivation, and runtime use.
Supported callbacks and native metadata provide the integration boundary.
## Reviewed baseline and releases
Narratio currently selects `gitea.maximumdirect.net/eric/ConfigKit v0.5.0`,
commit `dc09959cca4e105f49ec316c6d4ba585bd2340bd`, with Go 1.26.0 as its
consumer minimum. Release builds currently select Go 1.26.7.
The adoption target is `go.maximumdirect.net/configkit v1.2.1`, commit
`51ef3144158db1a09fabdbd1c7ae59908c84923b`. The published tag and the sibling
checkout agree on that revision. ConfigKit requires Go 1.27.0 and uses
`go.yaml.in/yaml/v4 v4.0.0`. Its code license remains LGPL-3.0-or-later.
The pinned [configuration contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/config.md)
and [consumer guide](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/consumers/README.md)
own upstream semantics. The intervening releases supply the following relevant
changes:
| Release | Adoption relevance |
| --- | --- |
| v0.6.0 | Loading and metadata optimizations; Go minimum moves to 1.26.0. |
| v0.7.0 | Native safe effective-configuration `Diff`. |
| v0.7.1 | Maintainer-tool update; runtime behavior unchanged. |
| v0.8.0 | Input/import destinations, sequence-item placement, shared source-version checks, and initial identified-sequence composition. |
| v0.9.0 | Directory and expanded import selection; optional-by-default imports; identified-list replace/append/upsert and inheritance; CLI assignments move after decoding and inheritance. Its final collection APIs replace the initial v0.8 registrations. |
| v0.10.0 | Native ordered configuration discovery through `SearchConfig`. |
| v0.10.1 | Loading consolidation and import bookkeeping improvements. |
| v0.10.2 | Correctness fixes for decoding, callback/resource lifetime, discovery, inspection budgets, and collection operations. |
| v1.0.0 | Vanity module path, source-root `config` controls, public API cleanup, source lookup, provenance and sensitivity fixes, and Go 1.27.0 minimum. |
| v1.1.0 | Field-level list resolution records and identity-aware inspection across list movement. |
| v1.2.0 | Main-relative companion inputs and overlay context for layered loading; overlay filesystem fixes. |
| v1.2.1 | Stable go-yaml v4.0.0 dependency; maintainer release-tool update. |
The corresponding `docs/releases/<version>.md` files at the pinned revision
provide the migration details. Select v1.2.1 directly; intermediate releases
provide context rather than required deployment stops.
## Current Narratio integration
The [configuration reference](../config.md) and
[configuration internals](../internal/configuration.md) describe current behavior.
- Commands select a pipeline path through Narratio's search loop, bootstrap
credentials from an explicit directory or a sibling `secrets/` directory,
and then call ConfigKit.
- Pipeline loading enables explicit path/glob imports and one selected overlay.
Campaign, session, and canonical-party documents use separate loads. Session
loading also supports command-scoped downloaded input snapshots.
- Typed defaults, recorded normalization, domain validation, and native
provenance are integrated. Additional path/dependency resolution and
campaign-owned family expansion occur after the pipeline load.
- Artifact files currently repeat enclosing `scriptorium.artifacts` mappings.
Logical IDs come from map keys. Families expand against canonical party data.
- Show, diff, and digest use a safe finalized-domain projection. Narratio owns
the current mapping-diff algorithm. Native load metadata supplies source and
artifact-catalog evidence.
Some public import descriptions still state restrictions from older ConfigKit
behavior. Replace these with current native semantics during adoption rather
than recreating them as Narratio checks.
## Targeted end state
### Native assembly and operator choice
ConfigKit owns source discovery, acquisition, composition, inheritance, presence,
typed secrets, provenance, and safe native inspection. Narratio supplies its
schema, domain defaults and validation, supported application resolution, and
operator-selected inputs.
Use ordinary `Load` for an ordinary configuration. Use public native facilities
where they serve the operator workflow. Layered loading is appropriate when
actual lower-priority inputs are needed; typed defaults remain appropriate for
simple defaults. Main-relative companion inputs allow such underlays to follow
a discovered main without Narratio searching for it again.
Expose the library's authored `config.imports` and `config.sequences` semantics
where applicable. Imports use ConfigKit defaults, including optional absence,
unless the operator explicitly requests other supported behavior. An invalid
selected source still fails. Operators mark essential imports `required: true`.
Directory imports can select YAML with an explicit `pattern`; unrestricted
filenames remain the library default.
Native destinations, nested imports, discovery filters, source-local relative
import paths, collection operations, and inheritance work under ConfigKit's
rules. Narratio validates the final effective domain entries. It adds no
fragment-count rule, mandatory fragment version, import-mode allowlist, or
parallel composition engine.
### Ergonomic artifact files and list customization
Support an artifact definition without repeated enclosing mappings through an
explicit import destination. For example:
```yaml
config:
imports:
- path: artifacts/session-recap.yml
destination: [scriptorium, artifacts, session_recap]
required: true
```
The imported file contains the artifact fields such as `prompt_id`, `inputs`,
and `output_path`. The literal destination defines the reusable key
`session_recap`, with logical source `narratio.artifact.session_recap`.
The filename is an organizational choice. Keyed files mounted at
`[scriptorium, artifacts]` support declared directory or glob imports when many
definitions should be discovered together. Apply the same pattern to families.
Keep keyed artifact/family mappings and campaign-owned character data. They
already fit native inheritance and stable identity. Full-root and disjoint
definitions remain valid native inputs; a one-artifact-per-file organization is
guidance rather than a validity requirement.
Allow authored native collection declarations for existing list fields where
their final shapes satisfy Narratio's schema. Operators can choose replacement,
append, or upsert and identified-record inheritance. Explain that upsert
replaces whole matching records and that an empty append/upsert contribution
does not clear a list. Register application-owned sequence identities only
where Narratio has a real domain identity requirement; generic library
capabilities do not establish new uniqueness requirements for every list.
### Discovery, credentials, and resource paths
Replace Narratio-owned local candidate probing with ConfigKit search descriptions
and `SearchConfig`. Use the native `WorkingDirectory`, `UserConfigDirectory`,
and `UnixSystemDirectories` presets in that order with application name
`narratio`. Use their documented filenames. Explicit `--config` selection
remains an exclusive `File` input. Report the source actually selected by the
library. This provides convenient development-local configuration and standard
ConfigKit naming. Existing `pipeline.yml` files remain explicitly selectable;
migration guidance explains the new discovery names and precedence.
Bootstrap uses `LoadSecretsIntoEnvironment` once per invocation before typed
credential resolution and concurrent work. Existing environment variables win,
including explicitly empty variables. Use the invocation-owned conventional
directory `/usr/local/etc/narratio/secrets`, with `--secrets-dir` selecting
another directory. A missing conventional directory remains optional; an
explicit selection follows the existing explicit-selection behavior. Retain
safe partial-report handling and exact-name forwarding to subprocesses.
The independent directory lets bootstrap precede ConfigKit's first and only
configuration selection. Service environments can provide credentials directly;
development invocations can choose a project-local secrets directory explicitly.
The implicit sibling `secrets/` convention retires with this cutover.
Resolve relative file-resource references from their effective defining source's
ConfigKit resource context. Inherited values follow their effective origin;
CLI-relative paths use the invocation directory. Application-derived paths
record their dependencies. Use the main source as a documented fallback for
origins without a distinct usable resource base. This makes imported definitions
portable across bundles.
Review actual path-bearing fields explicitly. Artifact destinations, command
names, session-relative products, and remote object keys retain their domain
meanings. This policy introduces no new target-existence or root-containment
checks. ConfigKit's consumer examples illustrate application policy; Narratio
owns the selected interpretation.
### Imports in secondary documents
Enable native explicit imports for locally sourced campaign, party, and session
documents as well as pipelines. This supports split rosters and shared metadata
with consistent library semantics. Campaign selection, party schema validation,
and session domain rules remain application-owned. Pipeline overlay selection
remains associated with the pipeline.
Materialize effective documents where raw root-file copying would leave
unresolved imports in downstream inputs or published session data. Remote
session snapshots retain their explicit source context; resolving imports
against an incidental download directory would give them an incorrect base.
Remote companion-file retrieval remains separate work.
### Final-domain inspection through native APIs
Keep validate, show, sources, artifact catalog, and diff useful for the operator's
fully resolved pipeline, including campaign-driven members and publish rules.
Use native effective rendering and `Diff` where they own the operation, with
Narratio formatting and generated-domain context at the presentation boundary.
A raw pipeline `Result` is a frozen load snapshot. Mutating its `Config` later
does not update rendering or diff. Integrate this distinction explicitly:
record domain resolution within a supported resolver when its inputs are
available there; where later campaign resolution remains appropriate, create
a completed native inspection result from the finalized safe domain projection
using ConfigKit's public loading APIs. That result describes application-derived
values; retained authoring results supply actual file provenance.
Use native comparison rather than maintaining a second general mapping-diff
engine. Preserve final-domain coverage and credential identities. Narratio's
digest and stage-owned reuse evidence remain application responsibilities.
Source-layout changes alone do not change equivalent domain values. Source
inspection can report composite origins, empty optional-import selections, and
application derivation without inventing a unique defining-file contract.
### Scope and compatibility
This is a clean native cutover. Update source-root `configkit:` controls to
`config:` and remove old-module/API references in maintained code and fixtures.
Nested `config` data remains ordinary data under ConfigKit's source-root rules.
The `configkit` package name and `configkit:"secret"` Go tag retain their
upstream spelling. Mapping and identified-list `extends` follow current native
rules.
This roadmap deliberately permits revised configuration semantics. Update
tests and documentation for selected changes rather than preserving historical
acceptance rules through compatibility loaders or keyword aliases.
Application identity, artifact dependency meaning, runtime output destinations,
campaign-owned party authority, remote session retrieval, and stage reuse remain
Narratio domain concerns. This scope does not redesign the runtime workspace,
change subprocess adapters, adopt ReleaseKit for Narratio, or add external
provider probes. Audita and workspace work retain their separate roadmaps.
## Required changes
### Dependency, toolchain, and notices
- Select the released `go.maximumdirect.net/configkit v1.2.1` module and migrate
all imports and exchanged named types together. Remove the old requirement;
use the released dependency rather than a filesystem replacement.
- Raise `go.mod` to Go 1.27.0 and select an available official Go 1.27 patch for
development/release validation and release builds. Align CI, runbook setup,
and incorporated Go attribution with that selection.
- Update ConfigKit source/license links to the reviewed commit and YAML v4
attribution to the selected stable release. Check the actual resulting module
graph. Maintainer tool dependencies are not automatically Narratio runtime
components. Narratio's separate direct YAML use need not be rewritten merely
because ConfigKit changed its internal parser dependency.
### Integration and simplification
- Apply the v1 API replacements actually used here, including
`LoadSecretsIntoEnvironment`, `LayerMain`, `RecordDerivedDefault`, and
`RecordNormalization`. Review other public names against the migration guide.
- Use keyed expanded options and definition literals. Keep metadata projections
compatible with current source evidence, native layer names, and safe errors.
- Replace broad list-record workarounds with supported field-level recording
when the application changes only fields. Record whole-list changes where
order or membership changes warrant them.
- Adopt native discovery, destination-based artifact authoring, authored
collection operations, and native final-domain rendering/comparison. Remove
replaced search, comparison, and redundant metadata machinery.
- Apply the agreed source-relative resource bases and secondary-document imports.
Share definitions and loading context between execution and inspection so an
operator validates the configuration the pipeline will actually consume.
- Review secondary-document input materialization and publication as well as
parsing. A copied root file can retain imports whose
companions are absent at its new location. Materialize the effective domain
document where a subprocess or restored session consumes it, retaining native
authoring evidence separately. Review placeholder checks against the effective
session input and preserve required external party/session formats.
### Documentation and maintained examples
Update the configuration reference, CLI reference, operations/bootstrap guidance,
configuration internals, contributor prerequisites, and release procedure for
the implemented semantics. Keep canonical facts in their respective owners.
Update project-specific architecture wording where selected path/bootstrap
decisions change it; preserve ADR 0003's ownership boundary.
Migrate the production/testing bundle to `config:`. Demonstrate destination-based
artifact and family files, a native directory import restricted to YAML, an
essential required import, native inheritance, and a useful selected-overlay
list customization. Keep illustrative optional choices labeled as choices.
Show how the declared artifact key is referenced by dependent artifacts.
Provide concise manual upgrade instructions for existing Narratio configurations:
control-key/API changes, optional import defaults, collection behavior, source
discovery, and selected path/bootstrap changes. Separately installed applications
own their configuration versions; Narratio's migration does not automatically
rewrite their files.
The eventual Narratio release requires a major version because existing valid
configuration control blocks change. The Go minimum increase alone is permitted
in a minor release under the adopted release policy. This roadmap does not
authorize tagging, publication, or retrospective release-note changes.
### Verification
Use temporary inputs and controlled synthetic credentials. Exercise native
discovery and the chosen bootstrap convention together, explicit file selection,
selected-source errors, optional and required imports, import destinations,
inherited artifact identities, native list customization, and final domain
validation. Add coverage at Narratio boundaries rather than duplicating
ConfigKit's complete structural test matrix.
Verify inherited presence/defaulting, party order, final campaign/family
expansion, resource paths, source attribution, and secret-safe inspection.
Diff/show must include derived members and publish rules; compare equal effective
configurations across different source arrangements. Credential rotation behind
the same reference name must remain absent from semantic diffs and digests.
Update tests that encode intentionally retired syntax or policies. Run focused
configuration/command/example/documentation checks and the established full
test, race, vet, build, and module checks on the adopted Go line. Preserve the
current supported build targets. No new live-service or CI-completion gate is
introduced by this migration.

View File

@@ -1,431 +0,0 @@
# ConfigKit v1.2.1 implementation plan
## Scope and authority
Implement [the approved ConfigKit roadmap](configkit.md). It owns the target,
native-semantics intent, and four approved product decisions. This plan replaces
the completed v0.5.0 plan. Its audience is a Luna coding agent implementing one
end-to-end sprint in the numbered order below.
Select `go.maximumdirect.net/configkit v1.2.1`, upstream commit
`51ef3144158db1a09fabdbd1c7ae59908c84923b`. The pinned
[consumer guide](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/consumers/README.md)
and [configuration contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/config.md)
own upstream semantics. Read the sections identified by each stage and public
declarations in the released module as needed. The sibling checkout is optional
reference material, not a build or test prerequisite.
## Working instructions
- Start with `docs/development.md` and `docs/policy/agents.md`. Architecture,
testing, documentation, OSS, and release policies apply; reuse unchanged
policy text already read. Prefer graph tools for code discovery.
- Implement the full native configuration workflow. Discovery, destinations,
collection controls, resource bases, secondary imports, materialization, and
native inspection are part of this sprint.
- Stages are work boundaries, not deployment releases. Intermediate stages may
fail to compile or have dependent tests awaiting later work. Record that state
accurately. Avoid temporary adapters or extra abstractions merely to make
every intermediate commit green. Final validation covers the assembled system.
- Commit completed nonempty units with short plain-English messages. Add focused
behavioral tests with their owner; run meaningful available checks. The final
stage runs repository-wide validation against the finished source.
- Use ConfigKit's public loading, presence, provenance, recording, rendering,
and comparison APIs. Narratio supplies domain meaning and runtime use.
Retire obsolete source-authoring checks and tests rather than reproducing them.
- Leave import defaults unspecified so ConfigKit supplies its defaults.
Explicit false/zero/empty/null and collection semantics follow the native
contract and actual Narratio domain rules.
- Use temporary inputs, real ConfigKit, and controlled synthetic credentials.
The default tests remain offline and independent of installed subprocesses.
- Write documentation in concise, affirmative, factual language. The roadmap
owns intent; this file owns sequencing and implementation choices.
Preserve stage ordering, artifact dependency meaning, the canonical party schema,
exact-name credential forwarding, publish commit semantics, and the existing
Linux amd64/arm64 plus best-effort macOS arm64 targets. This sprint does not
implement pending Audita/workspace features, migrate old sessions, rewrite
downstream configurations, add fragment version mandates or new path checks,
adopt ReleaseKit for Narratio, or publish a release.
## Concrete integration choices
- Consumer Go minimum: **1.27.0**. Development/release validation and build
toolchain: official **Go 1.27.4**, an available patch on that line. Selecting
it is not a new publication gate.
- Pipeline source when no explicit file is supplied:
`SearchConfig("narratio", Searches(WorkingDirectory(), UserConfigDirectory(),
UnixSystemDirectories()), SearchOptions{Required: true})`.
An explicit `--config` uses `File` exclusively.
- Bootstrap: optional `/usr/local/etc/narratio/secrets`, or explicit
`--secrets-dir`, through `LoadSecretsIntoEnvironment` before typed loading.
Existing environment values win. The implicit sibling directory retires.
- Local campaign/session/party loading enables imports. Only the pipeline gets
its selected overlay. Party version checking remains main-only; fragments
need no new version header. Remote session bytes are contextless snapshots.
- Existing campaign registry selection and local session search locations remain
domain conventions. Use native optional `SearchConfig` with `SearchPaths`
for local session candidates. Only actual absence permits remote fallback.
- Final-domain inspection uses a safe DTO and a defaults-backed
`Load(NoSource(), Definition.Defaults)` snapshot after domain resolution.
Credential fields contain reference names, not typed secrets that would
resolve again. Original results supply actual file provenance. Stages 8–9
explain this boundary.
## Stage 1: Establish the dependency and toolchain baseline [x]
Read `go.mod`, `go.sum`, `.woodpecker/*.yml`, release scripts/runbook, and
attribution. Read the upstream v1.0.0 migration table and v1.2.1 release note.
1. Select Go 1.27.0 in `go.mod` and ConfigKit v1.2.1 under its vanity path.
Migrate all production/test imports together and remove the old requirement.
Use the released module rather than a filesystem replacement.
2. Replace APIs actually used here: `LoadSecretsDir` →
`LoadSecretsIntoEnvironment`, `LayerBase` → `LayerMain`, and generic
`Resolution.Record` calls with `RecordDerivedDefault` or
`RecordNormalization`. Audit other removed public names and compared layer
or diagnostic strings. Use keyed expanded options/definition literals.
3. Rename genuine source-root `configkit:` controls to `config:` throughout
fixtures and examples. Keep the package name, `configkit:"secret"` Go tag,
and ordinary data keys distinct from source controls.
4. Align Go CI images and local/release instructions with Go 1.27.4. Preserve
platform targets and publication mechanics. Update incorporated Go notices.
5. Refresh ConfigKit and stable go-yaml v4.0.0 attribution from selected source
notices. Review actual module graph churn and tool-only dependencies. Keep
Narratio's separate YAML v3 use unless another scoped change needs it.
Check module identity and attribution. Attempt focused config/app tests; record
failures assigned to later stages. Complete when the dependency/API baseline is
established, without preserving old behavioral expectations as compatibility.
## Stage 2: Integrate native pipeline discovery and independent bootstrap [x]
Read `internal/app/pipeline_config_path.go`, `secrets_bootstrap.go`,
`command_dependencies.go`, `loadPipelineConfigWithDependencies` and callers,
inspection loaders, and `internal/config/pipeline_configkit.go`.
Read upstream discovery and secrets-bootstrap contracts.
1. Have the configuration owner accept an explicit file or the native required
discovery source above. Retain explicit-file package entry points where useful.
Commands load through one composition path instead of first probing with
`os.Stat` to obtain a default pathname.
2. Bootstrap once before typed loading or adapter creation from the independent
directory. Preserve explicit-path error behavior, safe partial reports, and
existing-empty environment precedence.
3. Obtain the actual selected root and import/overlay evidence from ConfigKit.
Carry that root into campaign selection, provenance, and command summaries.
Remove dependencies/search helpers used only for old pipeline preselection.
4. Apply this composition to run/range/alias commands, session init/plan/status/
restore, inspection, and remote-session workflows. A selected invalid source
fails rather than falling through to another pipeline candidate.
5. For overlay diff, load the left through native selection and the right from
that selected root with its own overlay. Share invocation bootstrap; neither
side performs a preliminary base-only load.
Test precedence and exclusive explicit selection with controlled working/user
directories and temporary native search descriptions. Test selected invalid
input, no source, bootstrap override/default behavior, and diff bootstrap once.
Keep tests isolated from the operator's system configuration and secrets.
## Stage 3: Resolve pipeline resources from native origin context [x]
Read `resolvePipelineConfig`, `resolveNotariusPaths`,
`finalizeLoadedPipeline`, `effective_digest.go`, and the upstream source-aware
path example. Resolve paths before normalization erases entry origins.
1. Introduce one small config-owned host-resource base helper using
`Resolution.EntryInfo`, `Resolution.Source`, and
`Resolution.SourceContext`. Use the value's effective origin rather than
its child's inheritance directive. A distinct usable source context supplies
its canonical base; missing or composite context falls back to the main source.
Explicit CLI-origin file references use the captured invocation directory.
If the main input is contextless too, use that invocation directory rather
than interpreting a diagnostic label as a filesystem base. This covers remote
byte snapshots without turning their temporary filenames into resource roots.
2. Normalize nonempty relative references with recorded helpers. Absolute values
stay absolute. Derive Notarius working directory from its already resolved
config path and record that dependency. Preserve domain validation of relevant
empty/disabled values.
3. Apply this classification:
| Fields | Interpretation |
| --- | --- |
| Audita/Scriptorium/Notarius `config_path`; Notarius `working_directory` | File/directory resource based on effective origin. |
| Relative `workspace.root`, `campaigns.root`, `spool.root`, `cache.root` | Host directory reference based on effective origin; absolute defaults stay unchanged. |
| Campaign template/stable-input paths; session stable-input/audio paths | Same resource policy, completed in Stage 4. |
| Artifact output paths, family output patterns, publish destinations, source IDs, prompt/profile IDs, schema identifiers, binary commands, Notarius reference bindings and chunk-cache modes | Existing domain interpretation, not file-resource rebasing. |
4. Remove duplicate pipeline-root rebasing and logical-Notarius path caches once
callers use resolved domain values. Show/digest describe those domain values,
rather than reconstructing an alternative authoring-path view.
5. Keep resource access at existing consumer boundaries; base selection adds no
target-existence check, root restriction, or history acceptance gate.
Test imported, inherited, overlay, absent-default, absolute, and ambiguous-origin
cases. Equal resolved resources have equal effective semantics across layouts;
relative text resolving to a different resource is a meaningful change.
## Stage 4: Load campaign/session imports with correct contexts [x]
Read `document_configkit.go`, `load.go`, `pathresolve.go`,
`party_resolution.go`, app session discovery/remote-session composition, and
session init. Read native import and resource-context contracts.
1. Use `File` with imports enabled for local campaign/session inputs. Share
definitions with byte loading and retain successful results/source contexts.
ConfigKit owns per-source and aggregate acquisition limits.
2. Replace local session candidate probing with native optional discovery over
the existing candidate paths. Permit absent input solely for remote fallback;
distinguish absence through native source evidence. Structure callbacks to
allow that discovery-only no-source result. Acquired invalid files still
receive ordinary validation and fail without remote fallback.
3. Downloaded session bytes receive no context inferred from their temporary
filename. Retain object provenance and current explicit session/previous-ID
checks. Use native contextless-import behavior; no remote import fetcher or
custom structural rejection layer is added.
4. Resolve campaign `session_template_file`, stable-input paths, and session
overrides/audio references with Stage 3's helper. Resolve each
`audio_files` element at its entry index before ordering. Keep campaign/
session stable-input precedence and campaign ownership of party input.
Provenance identifies the effective supplying configuration source.
5. Check unresolved template tokens in effective session string values, including
imported values. Session init still reads an authored template for intentional
substitution; runtime loading stays a configuration operation.
6. Retain detached campaign and session document values after their own loading
and resource resolution. Capture the session document before campaign-inherited
fields are copied into runtime session state. Stage 6 serializes these values.
Test local imports, resource bases, effective placeholder rejection, identity
mismatches, imported session-init template paths, absence versus selected-source
failure, and remote snapshots independent of temporary directories. Add no
secondary overlay flags.
## Stage 5: Resolve imported party definitions and canonical roster output [x]
Read `party.go`, `party_resolution.go`, `artifact_families.go`, the party
integration contract, and focused party/family tests.
1. Add a file-backed party entry point with imports enabled for campaign
resolution. Keep `ParseParty(bytes)` contextless for snapshots and tests.
Reuse the definition, main-only version rule, and domain validator.
2. Preserve `narratio.party.v1`, existing character/name/class/alias validation,
and campaign ownership. Native destinations allow split roster contributions;
no fragment version mandate or abstract/template record type is added.
3. Use native metadata for roster order and evidence, preserving authored order
within a source. Extend the current ordering helper using source acquisition
order, then line/column and ID as deterministic ties across sources. Prefer a
child's own declaration/inheritance location over its parent's location.
This is a presentation convention, not a new source acceptance rule.
4. Render canonical party YAML from the effective roster: schema version, keyed
characters, singular external `alias`, classes and optional levels. Preserve
meaningful roster/list order. A small typed DTO/YAML-node writer is suitable
for this Narratio-owned external format.
5. Keep players derivation and family expansion based on that same roster.
Replace output/checksum uses of `CanonicalParty.Raw` with canonical rendering;
retain raw data only where an actual diagnostic caller needs it.
Test split/inherited rosters, aliases/classes, effective duplicate-name rejection,
order, reloadable standalone canonical output, players, and family identities.
Use native evidence rather than a second source parse to reconstruct order.
## Stage 6: Materialize effective inputs and align prepare resume checks [x]
Read `internal/stage/prepare.go`, `prepare_party.go`, `prepare_resume.go`,
prepared-input descriptors, and prepare semantic evidence/tests.
1. Add configuration-owned deterministic campaign/session renderers from Stage
4's detached document values. Serialize public schemas and resolved file
references, excluding native directives and runtime/provenance state.
2. Preserve ownership: campaign-inherited stable inputs must not become explicit
session overrides. A published session that omitted `speakers_file` must
still inherit the receiving campaign's value. Explicit session overrides
remain explicit. Include actual session identity values where application
resolution supplies them, without freezing campaign defaults into the session.
3. Prepare writes these documents and Stage 5's party bytes through existing
artifact-store helpers. Keep filenames, input kinds, provenance fields, and
write/checksum behavior. Resume uses the same renderers instead of comparing
prepared effective output with a raw imported root.
4. Ensure imported effective changes reach existing stage-owned semantic/input
evidence. Equivalent domain values remain equivalent. Full pipeline digests,
source mtimes, or every authoring file are not new blanket invalidation keys.
5. Preserve ordinary stable-file copying/audio handling. Already resolved paths
must not be rebased again against the session root. Retire raw campaign/session
file requirements used solely for copying the old authoring documents.
Test standalone prepared documents, imported changes, equivalent layouts, and
remote-origin metadata. Canonical-byte format changes can cause a normal prepare
rerun; no session migration or new manifest schema is needed for serialization.
## Stage 7: Close publication, restore, and remote-session round trips [x]
Read publish input selection, remote-session loading, restore contracts, and
existing in-memory storage integration tests.
1. Trace the actual bytes published as remote `session.yml`. Publish the canonical
prepared session rather than the original root or command-scoped download.
Preserve immutable object/commit-pointer ordering and lock behavior.
2. Confirm campaign/party prepared products flow through existing selections
with their identities intact and without unresolved companion imports.
3. Verify a published canonical session loads through the existing contextless
path after authoring fragments and temporary files disappear. Restore uses
recorded contents and existing safety/locking. No remote import graph or
producer-host filesystem search is introduced.
4. Verify omitted session stable fields inherit the receiving campaign's values.
Keep remote object identity evidence separate from checksums of locally
materialized canonical bytes.
Use fake/in-memory storage for prepare → publish selection → remote load and
relevant restore coverage. Fix actual producers/consumers rather than adding a
parallel publication path. Existing atomic commit tests remain authoritative.
## Stage 8: Build native final-domain inspection snapshots [x]
Read `effective_projection.go`, `effective_digest.go`,
`effective_pipeline.go`, pipeline finalization/family expansion, and native
completion, defaults, rendering, and typed-secret contracts.
1. Keep one safe schema-specific final-domain DTO containing concrete expanded
artifacts and generated publish outputs. Omit unexpanded family declarations
and bookkeeping. Audit all credentials, including S3 fields and Audita/
Notarius/Scriptorium lists: use exact reference names, with no resolved
`Secret` fields in the inspection DTO.
2. Build a native DTO result with `Load(NoSource(), Options{Definition:
Definition{Defaults: ...}})`, supplying a fresh safe DTO and no external
credential lookup or application callback rerun. ConfigKit copies and freezes
these already resolved values. Using defaults avoids interpreting final
`config`/`extends` data as fresh directives or converting precise numbers
through a generic mapping-source reconstruction.
3. Build/refresh the snapshot at existing domain-finalization boundaries, after
applicable path/dependency/family resolution. Keep original authoring results
separately. Synthetic default origins describe this derived snapshot, not
original file provenance. No general snapshot service is needed.
4. Route effective show and `pipeline.resolved.yml` rendering through native
`WriteEffective`. Use the same DTO's deterministic JSON for Narratio's
semantic digest. Preserve exact integers, domain null/empty values, and
credential identities. Update the digest discriminator if its format changes.
5. Remove flattening/serialization machinery used only to duplicate native
show/diff. Keep narrow path enumeration and family/party context genuinely
needed by catalog/source presentation and stage-owned reuse evidence.
Test final family/publish expansion, inherited/expanded equivalence, large
integer variables, null/empty values, safe rendering, credential-reference
changes versus rotation, and no credential resolution during snapshot creation.
## Stage 9: Adopt native diff and truthful operator source inspection [x]
Read `internal/app/config_diff.go`, config command loaders/tests,
`artifact_catalog.go`, and native Diff/source/history contracts.
1. Fully resolve each overlay against the same selected campaign/party before
comparing Stage 8 snapshots with `configkit.Diff`. Remove
`diffEffectivePipelineValues`, its general map comparison, and consistency
checks predicated on the old flattened-leaf change model.
2. Present canonical JSON Pointer paths in native order. Derive added/removed/
changed from snapshot `Present`, not nil or redaction text. Use compact JSON
for safe values and expose `HiddenChanged` if present. Equal values produce
`no differences`. Document pointer notation and native changed-region/list
granularity instead of emulating the old engine.
3. Source/catalog evidence comes from authoring results through public source
lookup and metadata history/origins. Add clearly labeled empty optional-import
selection evidence to `config sources`, without guessed candidates or
reasons for absence.
4. Preserve artifact/family/character IDs and generated party context. Distinguish
normalization origins from retained definition evidence. Use native `main`
where an upstream layer is reported; root/import presentation roles may remain
where they express actual application context.
Test changed family members and publish rules, equivalent layouts, composite
contributors, optional empty imports, atomic lists, and pointer escaping in
allowed dynamic mapping keys. Inspection remains secret-safe and read-only,
without source reparsing or provenance-based acceptance gates.
## Stage 10: Exercise native authoring and maintain ergonomic examples [x]
Read the production/testing bundle, artifact/family tests, native destinations
and collection rules, and maintained example tests.
1. Convert artifact/family files to destination-based definitions. IDs remain
explicit destination keys or keyed payloads; filenames do not imply IDs.
2. Demonstrate a declared directory import with `pattern: "*.yml"`, essential
`required: true` input, an optional selection, inheritance, and a useful
testing-overlay list customization through `config.sequences`.
3. Cover append/upsert/replace on an existing list using an appropriate authored
identity, such as publish output `source`. Whole-record upsert and empty-list
behavior follow ConfigKit. No application identity registration is required
merely to enable authored collection controls.
4. Demonstrate a split roster and local secondary imports using the sample
campaign, with companion resource paths following source-origin policy.
Explain that full-root and disjoint definitions remain accepted too.
5. Use ordinary `Load` for this authored-import workflow. Layered loading is
useful for actual lower-priority sources, not required for defaults alone.
If such a source is genuinely needed, use `LoadLayered` and `MainRelative`
directly with focused coverage instead of another merge/search mechanism.
Load examples through production definitions. Check dependent logical IDs,
concrete family members, testing differences, and standalone prepared inputs.
Keep examples synthetic and credential-free.
## Stage 11: Align canonical documentation and project-specific policy [x]
Read documentation/OSS/release policies, affected references, and the approved
roadmap. Finish current-behavior documentation for the assembled implementation.
1. Update config/CLI references, operations/bootstrap guidance, troubleshooting,
configuration internals, prepare internals, and party integration docs for the
new discovery, import, resource-base, materialization, and inspection behavior.
2. Replace obsolete import restrictions with an accurate native summary and
pinned upstream links. Explain optional imports, `required`, filters,
destinations, source-local root controls, list modes, and reserved-key scope.
Keep effective domain validation Narratio-owned.
3. Align project-specific architecture configuration/startup statements and the
current raw-party-copy description in its actual owner. Preserve accepted
ADR 0003 and generic adopted policies. Routine integration details do not
require new ADRs.
4. Write manual upgrade guidance for control names, native discovery filenames/
precedence, explicit custom filenames, secrets override, relative path changes,
required imports where desired, and list operations. Explain effective prepared
documents versus authored roots; downstream migrations remain separate.
5. Finish Go prerequisites, CI/runbook toolchain alignment, and attribution from
Stage 1. Record impact for eventual major-release preparation without inventing
a release version or changing published notes. Leave pending Audita/workspace
roadmaps and Proposed ADRs intact.
Run doccheck and maintained example validation. Each volatile contract has one
canonical owner. Retire completed roadmap files only when separately requested.
## Stage 12: Verify the assembled migration and close the sprint [x]
1. Extend a command-level temporary bundle with native discovery, explicit
secrets override, imported campaign/party/session resources, destination-based
artifacts, family expansion, and a testing overlay. Reuse preceding focused
coverage instead of repeating ConfigKit's structural matrix.
2. Exercise validate/show/sources/artifacts/diff through real composition. Check
selected root, resources, generated values, optional-import evidence, safe
credentials, and absence of runtime workspace creation by inspection. Exercise
plan or prepare with fake external seams to establish execution parity.
3. Trace remaining callers of old search, rebasing, raw-copy, and diff helpers.
Remove dead code/test seams. Review intentional test expectation changes
against the roadmap and preserve existing consequential domain/safety tests.
4. With Go 1.27.4 and dependencies prepared, run `go test ./...`,
`go test -race ./...`, `go vet ./...`, `go build ./...`,
`go mod tidy -diff`, and `git diff --check`. Use the existing release asset
builder with a temporary output directory and its existing `v0.0.0-test` label
to check the three targets; this invokes no publication or new release note.
5. Inspect `go list -m` for the exact release, absence of the old module path
and filesystem replacements, and expected notices. Review every roadmap
capability and all four approved decisions; native structural behavior must
not be narrowed by new Narratio acceptance gates.
6. Commit the finished unit and report checks, compatibility changes, revision,
and any genuine unfinished work. Leave a clean workspace. Actual release
publication and the next feature require separate authorization.
Complete when the native workflow works end to end, prepared inputs survive
loss of authoring companions, operator inspection covers final domain values,
and final validation passes.
## Open Questions
None required for this sprint. The four product choices are approved. This plan
selects concrete native snapshot, source-context, materialization, and toolchain
approaches without adding compatibility shims, structural restrictions, or
stronger runtime guarantees.

View File

@@ -1,459 +0,0 @@
# ConfigKit adoption review remediation
## Review scope
Reviewed the twelve completed stages in [implementation.md](implementation.md)
against [the approved target](configkit.md), the implementation diff from
`65d0e51` through `39a958675d68e940799f4cb8c8e6748830d1483a`, current code,
documentation, focused diagnostic reproductions, and repository checks.
An integration-focused follow-up reviewed the same implementation at
`bccf03a99f45c2b9b342412b85f70e356663df61`, tracing each stage into existing
command, preparation, publication, restore, and inspection workflows. Items
5–6 were additional findings from that pass; items 1–4 were outstanding then.
An algorithm-focused pass reviewed all twelve stages at
`583c5bdf1bc8f0af8ec2e257bcb03ecccbf283b1`. Items 7–10 are refactoring
opportunities for consideration. Complexity follows the code and the selected
ConfigKit v1.2.1 implementation; no operational timings or benchmarks were
collected. The pass preserves algorithms expressly chosen by the plan.
All ten items below are now complete. The original findings are retained with
completion annotations; final verification is recorded at the end of this file.
## 1. P2: Distinguish effective resource origins from superseded history
Status: Complete. Native pre-resolution Previous evidence supplies current
contributors; regression coverage includes overlay replacement and inheritance.
Stage: 9, with documentation coverage in Stage 11.
Location: `internal/config/effective_pipeline.go:274–280`,
`effectivePipelineOwners`.
For a resolution-origin field, the projection adds every main/overlay origin
from native history to the same current source-role records. This conflates
effective ownership with historical definitions. For example, a root containing
`audita: {config_path: original.yml}` and an overlay containing
`audita: {config_path: replacement.yml}` produce `audita.config_path` records
for the root, overlay, and resolution. The root's superseded value did not
supply the effective resource; its record is indistinguishable from an effective
contributor. The error is visible to operators using `config sources` to locate
the definition that controls an integration path.
Required change: use native effective/pre-normalization evidence for effective
input attribution. If historical definitions are also displayed, label them
separately. Preserve historical contributor information in the artifact catalog
where it is explicitly definition evidence. Keep the projection thin; another
provenance reconstruction service is unnecessary.
Verification: cover a normalized resource with a superseded root/import value,
an overlay replacement, and inheritance. The effective overlay and resolution
remain visible, while superseded evidence is either absent from current-owner
records or explicitly identified as history. Update the command reference for
the resulting roles.
## 2. P2: Remove the restore test's incidental startup deadline
Status: Complete. Cancellation is synchronized after discovery with the session
lock held. Ten focused race repetitions and the full repository race suite pass.
Stage: 12; the fragile timing assumption predates this migration.
Location: `internal/app/restore_execution_test.go:383–387`,
`TestExecuteRestoreLockConflictFailsAndWritesNothing`.
The test starts a 100 ms context deadline before calling the complete restore
command, then requires an error from session-lock acquisition. With the current
configuration workflow under the full race suite, the deadline can expire
during earlier work. The reproduced failure was:
```text
restore: remote check current commit pointer "dnd/campaigns/sample-campaign/sessions/2026-05-03/current/commit-pointer.json": context deadline exceeded, want lock failure
```
An isolated application-suite run passed, while a subsequent full race run
failed at this assertion. A green isolated run therefore does not establish
Stage 12's repeatable final validation. The test implicitly requires unrelated
startup work to finish within 100 ms, which is not an application requirement.
This is not evidence that production session locking is incorrect.
Required change: make the contention/cancellation test target the intended
lock boundary without timing all preceding configuration and remote discovery.
Use synchronization at an appropriate existing test boundary, or separate
lock-service contention coverage from command-level no-write coverage. A larger
arbitrary startup timeout alone leaves the underlying timing dependence.
Preserve the assertions that restore cannot write protected files while another
invocation owns the session lock.
Verification: repeat the focused test and rerun the complete race suite under
normal package concurrency. Report actual outcomes rather than treating an
isolated pass as full-suite success.
## 3. P3: Finish the planned field-level publish resolution cleanup
Status: Complete. Field-only defaults and destinations retain precise native
history. Publish and authored append/upsert/replace regression checks pass.
Stages: 1 and 3; the roadmap's integration scope requires field-level recording
for field-only changes.
Location: `internal/config/pipeline_configkit.go:202–215,232–235`,
`resolvePipelineConfig`.
Publish requiredness and derived destinations already use field-level native
helpers. The resolver additionally tracks `defaultedOutput`, records the entire
`publish.outputs` list for those field edits, and retains a comment claiming
that child records cannot cover the changes. ConfigKit v1.2.1 supports exactly
this field-level coverage. The ancestor record adds redundant history: querying
an untouched output source also returns a whole-list normalization operation
with publish-default/destination reasons.
Required change: remove the obsolete list-coverage tracking, redundant ancestor
records, and inaccurate comment for these field-only edits. Keep field helpers
and any whole-list records actually needed for structural membership/order
changes. This is completion of the selected simplification, not a new ban on
container-level records.
Verification: a temporary diagnostic overlay removing only the obsolete
workarounds successfully loaded defaulted/derived publish outputs and passed
the append/upsert/replace collection tests. Add suitable behavioral coverage
for effective values and precise field history, without asserting private
helper invocation counts.
## 4. P3: Pin Go attribution links to the selected source commit
Status: Complete. Go license, patent, and source links identify the verified
Go 1.27.4 source commit; documentation checks pass.
Stages: 1 and 11.
Location: `docs/oss/attribution.md:125–128`.
The updated Go entry uses `go1.27.4` in the LICENSE, PATENTS, and Source URLs.
The adopted OSS policy requires commit-specific upstream source and notice
links. The preceding entry used a source hash; this update replaced it with
tag-based references.
Required change: use the verified Go 1.27.4 source commit
`7b2881024c95a11ddb86663a616340aa3a88b67f` in those three URLs. The hash was
resolved through the official Go repository's release tag. Keep the existing
license texts, factual notice wording, and Go version identification.
Verification: confirm the links identify that source revision and run the
documentation checks. This requires no licensing change or new release gate.
## 5. P2: Give session initialization the correct authoring context
Status: Complete. Rendered templates load with native local context and imports;
local and remote products are standalone effective documents. Relative CLI audio,
import removal, placeholder/identity, and initialization race checks pass.
Stage: 4, interacting with materialization and command validation in Stages
6, 11, and 12.
Locations: `internal/app/operator_session_init.go:78–102` and
`internal/config/load.go`, `LoadSessionBytesWithOptions`.
The existing initializer validates its rendered document through
`LoadSessionBytesWithOptions`. That loader now deliberately describes a
contextless remote snapshot with imports disabled. A generated local session
is an authored file, so this caller no longer uses the same semantics as the
ordinary file loader.
Two focused reproductions show the mismatch:
- A local template with a required `shared.yml` import containing constant
metadata fails with `import.disabled`, even when the import exists beside
the generated destination and the resulting local session is otherwise valid.
- With `--audio-dir ./audio` and an output in another directory, initialization
validates the invocation-directory audio path, then writes the original
relative text. Loading that generated file selects the output-directory
audio path. The persisted resource differs from the validated resource and
the documented invocation-relative CLI interpretation.
Required change: distinguish authored initialization from contextless snapshot
loading. Use ConfigKit's supported source/context facilities for local
composition, and make persisted resource references reproduce the validated
configuration. Materialize effective values where moving or uploading the
document would leave authoring imports unresolved. Preserve the narrow template
substitution contract and the existing contextless remote-loading boundary.
Verification: initialize a local imported template, load its output through the
ordinary loader, and check the imported values. Cover a destination outside the
invocation directory and relative CLI audio input. Check that remote generation
emits a consumable standalone snapshot and that ordinary placeholder and identity
validation still applies. These cases exercise initialization through the real
configuration owner without adding remote companion retrieval.
## 6. P2: Reconcile remote session updates with committed-session selection
Status: Complete. Authored remote input takes precedence for later commands;
committed snapshots remain immutable fallback input and restore authority.
Publication/update regression coverage checks changed title and audio selection.
Stage: 7, interacting with the existing initializer and shared command loader
from Stages 2 and 4.
Locations: `internal/app/config_loader.go:194–223` and
`internal/app/operator_session_init.go:113–138`.
The new remote loader selects a committed `run:prepare/inputs/session.yml`
whenever the current commit declares one. The existing `session init --remote`
writer continues to update the session-root `session.yml`. After publication,
the writer and reader therefore use different authoritative objects.
A prepare → publish → `session init --remote --force --title "After publication"`
reproduction succeeds and writes the requested title to the root object. A
subsequent ordinary command load still returns the old title from the committed
prepared session. The same selection would ignore changed audio or previous-session
settings. The documented overwrite workflow silently stops affecting later runs
once the session has been published.
Required change: reconcile remote input authoring and selection so successful
operator updates can be used by subsequent runs. Preserve immutable published
snapshots and committed restore authority. Keep an effective remote editing or
override workflow; silently ignoring updates or introducing a blanket ban on
editing published sessions would leave the integration unresolved. Update the
canonical operator guidance for the resulting selection behavior.
Verification: publish a prepared session, update it through the supported remote
workflow, then load it through the ordinary run/plan configuration path and check
the updated values. Include audio or previous-session selection as well as title.
Verify that restore still uses its selected immutable commit and that published
objects are unchanged by the authoring update.
## 7. P3: Reuse enumerated source records and one inspection source index
Status: Complete. Source enumeration uses complete native records directly;
catalog projection shares one index and retained family sources. Nested empty
imports and existing composite/family inspection checks pass.
Stage: 9.
Locations: `internal/config/effective_pipeline.go:57–75`,
`internal/config/pipeline_configkit.go:354–371`, and
`internal/config/artifact_catalog.go`, `declarationDefinitionSources`.
`emptyImportSelections` enumerates `Result.Sources()` and then calls
`Result.Source(id)` for every enumerated record. `pipelineSourcePaths` does the
same lookup for every retained source. In ConfigKit v1.2.1, completed-result
source lookup scans the source slice. Enumerating S sources this way performs
Θ(S²) source-ID comparisons, although the enumerated records already contain
the required information, including empty-import evidence.
The artifact catalog additionally rebuilds this source map for each declaration.
For A entries this introduces O(A S²) source-index work; with one imported file
per artifact, that component can become cubic in A. This is separate from the
native history queries and necessary output construction.
Potential refactoring: consume the complete enumerated records directly. Build
the source-ID/path map once per inspection operation and share it with declaration
projection. This reduces empty-import enumeration to O(S + E), where E is the
returned selection evidence, and source-index work across the catalog to
expected O(S + A). Expanded members can also reuse the family-source lists
already retained in the family catalog. Keep native history and origin queries
as the provenance authority.
Verification: temporary functional checks confirmed that enumerated records and
individual lookups have equal contents, and that direct enumeration preserves
source paths and nested empty-import selections. Preserve operator source/catalog
output, including composite and inherited contributors, when refactoring.
## 8. P3: Reuse party metadata and per-character provenance within inspection
Status: Complete. Each source/catalog projection shares one native metadata
snapshot, one source index, and a per-character cache. Imported-party, composite
provenance, family, and command agreement tests pass under the race detector.
Stage: 9, consuming Stage 5's imported party result.
Locations: `internal/config/artifact_catalog.go:67–105` and
`internal/config/effective_pipeline.go`, `appendGeneratedArtifactSources`.
`partyCharacterSources` calls `result.Metadata()` separately for Lookup and
History. ConfigKit v1.2.1 returns detached metadata by copying the complete
effective-path and field maps, including their origin records. This helper is
called for each generated artifact in the catalog and for each generated
artifact field in source inspection. The same character's provenance is
therefore repeatedly recomputed across fields and families.
Let M be the copied party-metadata size and P the number of provenance requests.
The avoidable metadata-copy work and cumulative allocation are Θ(P M). Even
with one family and bounded-size character entries, M and P both grow with the
character count C, producing quadratic work. More families and fields multiply
the requests. This describes the provenance pass, not the complexity of the
whole command or unchanged party validation.
Potential refactoring: obtain one native Metadata snapshot and source index per
inspection operation. Resolve each requested character's source list once and
reuse it for that operation. The full-metadata copy term becomes O(M), with
native per-character query costs and required output emission remaining. Keep
the cache local to the projection; a new persistent provenance service is
unnecessary. Apply the same snapshot reuse to campaign provenance helpers where
convenient, although their fixed field count is not a quadratic case.
Verification: a temporary imported/inherited-roster check confirmed equivalent
source lists using shared native metadata and an enumerated source index.
Retain contributor accuracy, ordering, and secret-free inspection output.
## 9. P3: Read the selected remote session without enumerating run history
Status: Complete. Remote loading opens the selected object once, verifies its
returned identity, and streams into a command-owned temporary file. Remote and
publication round trips pass under race; a rejecting-list fake checks enumeration
is unnecessary, and cleanup remains idempotent.
Stage: 7.
Location: `internal/app/config_loader.go:194–228`, `findRemoteSessionConfig`.
After selecting the committed prepared-session key, this function lists the
entire session prefix and scans it for that exact key. If H objects are retained
across historical runs, this adds Θ(H) listing/scan work and Θ(H) object-record
memory to every remote session load. The production S3 adapter paginates and
accumulates the complete listing. The scan predates the migration and was
retained in the new committed-object selection path.
Potential refactoring: retrieve the exact selected key through the existing
`ObjectStore.Read` boundary, which returns metadata with the opened body. Use
that metadata for the existing size/generation checks and stream the body into
the command-owned temporary file while retaining checksum verification. The
object lookup/transfer then takes a constant number of requests independent of
H, plus the necessary O(B) work for B session bytes. Current-commit acquisition
and validation costs remain. Apply this after reconciling the selection policy
in item 6; no new backend interface is needed.
Verification: the production S3 Read implementation performs one exact-key
GetObject and returns its size and generation. Preserve fake-store coverage for
missing objects, selected identity, checksum/size/generation mismatches, and
temporary-file cleanup. Confirm that unrelated historical keys are not enumerated.
## 10. P3: Stream prepared-input archival copies
Status: Complete. Verified descriptors stream through the existing atomic-reader
copy helper, with unchanged-destination reuse. Archive bytes, checksums, modes,
and tamper rejection are covered; full ordinary and race suites pass.
Stage: 7.
Location: `internal/stage/publish_sources.go:29–64`,
`collectPublishPreparedInputs`.
After opening and verifying each prepared document, the new archival path reads
the whole file with `io.ReadAll` and passes the byte slice to
`writeBytesIfChanged`. For a document of B bytes this requires Θ(B) auxiliary
transfer memory. Expanded imported party documents can be larger than any one
authoring fragment. A full in-memory document is unnecessary for this file copy.
Potential refactoring: preserve the current confined source opening and checksum
checks, then stream the opened descriptor into the run archive. The existing
`fileops.WriteReaderAtomicWithChecksum` already supplies reader-based atomic
replacement and hashing through `io.Copy`. Preserve unchanged-destination reuse
using the known checksum. Transfer time remains Θ(B), while auxiliary copy
memory becomes O(1) with a fixed-size buffer. This does not alter the plan's
separate canonical YAML rendering algorithm or total in-memory configuration size.
Verification: preserve archived bytes/checksums, workspace file modes, source
validation, failure handling, and immutable publication behavior. Existing
reader-copy and prepared-session publication tests provide suitable behavioral
coverage; allocation or wall-clock thresholds are unnecessary.
## Stage-by-stage disposition at the initial reviews
| Stage | Review result |
| --- | --- |
| 1: Dependency/toolchain | ConfigKit v1.2.1 and Go 1.27.0/1.27.4 selected; remaining notice pinning in item 4. |
| 2: Discovery/bootstrap | Native discovery, exclusive explicit selection, independent bootstrap, and shared diff bootstrap are present. |
| 3: Resource bases | Native origin-based resolution and fallbacks are present; field-record cleanup remains in item 3. |
| 4: Campaign/session imports | File contexts, contextless remote snapshots, native optional discovery, and detached document values are present; initialization still uses the snapshot path in item 5. |
| 5: Party imports | Imported rosters, source-aware ordering, canonical rendering, players, and family resolution are present; the planned roster sort is retained. |
| 6: Prepared inputs/resume | Canonical document materialization and matching resume evidence are present. |
| 7: Publish/restore | Prepared documents flow through immutable committed run objects and restore; item 6 is a correctness gap and items 9–10 are scalability opportunities. |
| 8: Final-domain snapshots | Defaults-backed native snapshots, safe credential identities, rendering, and digest integration are present. |
| 9: Diff/provenance | Native diff and pointer paths are present; item 1 concerns attribution correctness and items 7–8 concern avoidable provenance work. |
| 10: Examples | Destination-based definitions, directory imports, list operations, split roster, and secondary imports are demonstrated. |
| 11: Documentation/policy | Current references cover the migration; finish attribution, source-role, and initialization/selection corrections above. |
| 12: Final verification | Existing integration checks pass; add coverage for items 5–6. Full race validation remains unreliable because of item 2. |
The approved discovery, bootstrap, relative-resource, and secondary-import
changes are deliberate contract changes. Items 5–6 describe implementation
inconsistencies in existing workflows. Item 2 concerns an incidental test-time
performance assumption, not a requested startup guarantee.
## Initial review validation
Using Go 1.27.4:
- `go test ./...` passed.
- `go test -race ./...` failed at item 2. The isolated application race suite
passed, as did five subsequent repetitions of the focused restore test.
- `go vet ./...`, `go build ./...`, and `go mod tidy -diff` passed.
- Release-asset cross-builds passed for Linux amd64, Linux arm64, and macOS
arm64 with the existing `v0.0.0-test` label and temporary output directory.
- Focused diagnostic reproductions confirmed items 1 and 3. Temporary Go
overlays kept those diagnostics and proposed-code experiments outside the
repository.
- Documentation checks and `git diff --check` passed.
Review execution made no application changes and invoked no live subprocess
services or release publication.
## Integration follow-up validation
Using Go 1.27.4 at the follow-up revision:
- `go test ./...` passed with local test-server sockets permitted. The sandboxed
attempt was blocked by `httptest` socket restrictions.
- Focused migration checks across config, stage, and app packages passed both
ordinarily and under `-race`. These exercised native discovery/inspection,
imported inputs, canonical materialization, published-session round trips,
initialization, and command/execution agreement.
- Temporary overlay diagnostics reproduced both aspects of item 5 and the
publication/update conflict in item 6, ordinarily and under `-race`.
- The full race suite was not repeated in this pass; item 2 retains the initial
review's observed failure. Application code is unchanged.
Only this remediation document changed. Diagnostic files remained outside the
repository, and object storage used an in-memory fake.
## Algorithm-focused review coverage and validation
The stage-by-stage pass covered dependency/API substitutions (1), native
discovery/bootstrap (2), resource-base queries (3), secondary loading and
materialization data flow (4), roster ordering/rendering (5), prepare/resume
serialization (6), archive/remote/restore paths (7), finalized snapshots (8),
diff/provenance projection (9), examples (10), documentation (11), and assembled
verification (12).
The chosen roster ordering, defaults-backed snapshot construction, native Diff,
and required content validation remain accepted. Canonical rendering is linear
in emitted data; the planned roster comparison sort is O(C log C) after deriving
ordering keys. Unchanged algorithms such as pairwise party-name validation and
prepare-input matching are not new findings from this migration-focused pass.
Cost analysis followed the released module's public API wrappers through native
source lookup and metadata-copy implementations. Temporary functional Go-overlay
checks validated source-enumeration and shared-metadata equivalence, ordinarily
and under `-race`. Focused configuration, file-reader copy, S3 opened-object
metadata, native inspection, and prepared-session publication checks passed on
Go 1.27.4. No performance data, timing assertions, benchmark gates, or application
changes were introduced.
## Completed remediation validation
Each issue was addressed in order and committed with its completion marker.
Using Go 1.27.4 on the completed implementation:
- `go test ./...` passed.
- `go test -race ./...` passed, including the corrected restore contention test.
- `go vet ./...`, `go build ./...`, and `go mod tidy -diff` passed.
- Focused regressions cover effective provenance, field-level publish history,
local/imported initialization, standalone remote input, remote authoring after
publication, enumeration-free loading, and verified streaming archives.
- Documentation checks and `git diff --check` passed.
Remote session authoring updates affect future command inputs. Committed
publication objects remain immutable and restore continues to validate its
pointer-selected remote state. Inspection caches are scoped to one operation;
ConfigKit remains the authority for configuration and provenance.