Remove completed ConfigKit planning documents
All checks were successful
ci/woodpecker/push/verify Pipeline was successful
All checks were successful
ci/woodpecker/push/verify Pipeline was successful
This commit is contained in:
@@ -1,243 +0,0 @@
|
|||||||
# Adopt ConfigKit v0.5.0
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
Upgrade Narratio from ConfigKit v0.4.2 to v0.5.0 and integrate its native
|
|
||||||
sibling inheritance with Narratio's configuration validation and inspection.
|
|
||||||
Keep the upgrade focused on the existing loading model and the adjustments
|
|
||||||
needed to consume the new release correctly. Rely on ConfigKit's assembly and
|
|
||||||
metadata features; validate Narratio's effective domain configuration without
|
|
||||||
adding acceptance rules about how that configuration was authored.
|
|
||||||
|
|
||||||
This adoption precedes [Audita adoption](audita.md) and the
|
|
||||||
[workspace reorganization](workspace.md). This document defines the target
|
|
||||||
and required changes; [the implementation plan](implementation.md) defines
|
|
||||||
the steps for reaching that target.
|
|
||||||
|
|
||||||
## Reviewed Upstream Release
|
|
||||||
|
|
||||||
ConfigKit v0.5.0 identifies commit
|
|
||||||
`dc09959cca4e105f49ec316c6d4ba585bd2340bd`. The
|
|
||||||
[release notes](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/releases/v0.5.0.md),
|
|
||||||
[configuration contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/config.md),
|
|
||||||
and [consumer guide](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.5.0/docs/consumers/README.md)
|
|
||||||
own the upstream behavior.
|
|
||||||
|
|
||||||
The release introduces native `extends` relationships between sibling mappings,
|
|
||||||
layered loading with ordered underlays, caller-owned filesystem and mapping
|
|
||||||
sources, and passive source-resource contexts. Inheritance preserves explicit
|
|
||||||
presence, provenance, and secret classification.
|
|
||||||
|
|
||||||
The existing loading API remains available. ConfigKit's minimum Go line, YAML
|
|
||||||
dependency, and LGPL-3.0-or-later license are unchanged. Narratio can retain its
|
|
||||||
current Go toolchain and ordinary loading calls.
|
|
||||||
|
|
||||||
## Current Baseline
|
|
||||||
|
|
||||||
The [configuration reference](../config.md) owns Narratio's current public
|
|
||||||
semantics; [configuration internals](../internal/configuration.md) describe
|
|
||||||
loading and inspection.
|
|
||||||
|
|
||||||
- Pipeline loading uses ConfigKit v0.5.0 with explicit additive imports and one
|
|
||||||
selected overlay, typed defaults, recorded resolution, and domain validation.
|
|
||||||
- Campaign, session, and canonical-party inputs use separate single-source
|
|
||||||
loads. Secret-directory bootstrap remains an explicit invocation operation.
|
|
||||||
- Fields with root-relative pipeline path semantics retain that base when their
|
|
||||||
declarations move into imports. For example, enabled Notarius configuration
|
|
||||||
and working-directory paths resolve from the root pipeline directory.
|
|
||||||
- Distinct fields of one artifact or artifact-family entry can compose across
|
|
||||||
base files under ConfigKit's structural rules. Narratio validates the resulting
|
|
||||||
effective entry, and the selected overlay may replace or remove those values.
|
|
||||||
- Source inspection and artifact catalogs consume retained ConfigKit metadata.
|
|
||||||
Effective show, diff, and digest projections describe the resolved domain
|
|
||||||
configuration rather than source layout.
|
|
||||||
|
|
||||||
The released v0.5.0 module is now selected directly, and its attribution points
|
|
||||||
to the reviewed commit. Cross-file artifact and family inheritance and disjoint
|
|
||||||
entry composition now pass through Narratio's effective configuration checks.
|
|
||||||
The remaining integration work verifies native provenance, explicit presence,
|
|
||||||
safe projections, and the assembled inspection workflow described below.
|
|
||||||
|
|
||||||
## Targeted End State
|
|
||||||
|
|
||||||
### Native inheritance
|
|
||||||
|
|
||||||
Use ConfigKit's built-in sibling inheritance wherever its input contract and
|
|
||||||
Narratio's domain schema permit it. ConfigKit resolves `extends` after source
|
|
||||||
composition and overlay precedence, before typed decoding and application
|
|
||||||
resolution. Narratio receives the resulting typed configuration.
|
|
||||||
|
|
||||||
Inheritance can reduce repetition between artifacts and artifact families,
|
|
||||||
including definitions in separate imported files. Distinct child keys retain
|
|
||||||
their identities. Each resulting entry undergoes ordinary Narratio validation,
|
|
||||||
family expansion, dependency resolution, and execution selection.
|
|
||||||
|
|
||||||
Parent entries remain ordinary domain entries. They must satisfy applicable
|
|
||||||
validation and participate in the effective configuration. This adoption does
|
|
||||||
not introduce abstract artifacts or a separate template registry. Operators
|
|
||||||
must override inherited fields where needed to keep the resulting configuration
|
|
||||||
valid, such as output destinations.
|
|
||||||
|
|
||||||
Inherited false, zero, empty, and null input retains ConfigKit's explicit
|
|
||||||
presence semantics. Narratio's absence-based defaults must not refill such
|
|
||||||
values. Normal domain validation still determines whether a resulting value
|
|
||||||
is acceptable.
|
|
||||||
|
|
||||||
### Effective configuration and native provenance
|
|
||||||
|
|
||||||
Remove Narratio's one-base-file-per-artifact/family restriction. If ConfigKit
|
|
||||||
accepts composition of one entry from several files, Narratio accepts that
|
|
||||||
entry when its effective domain values are valid. ConfigKit continues to reject
|
|
||||||
structural conflicts under its own contract. Narratio does not reconstruct
|
|
||||||
declaration history to add another conflict rule or reject domain values that
|
|
||||||
were removed or replaced before the final configuration handoff.
|
|
||||||
|
|
||||||
Domain validation concerns the effective configuration, including application
|
|
||||||
resolution and family expansion. Family publication requires an applicable
|
|
||||||
effective publish configuration, not separate evidence that the section was
|
|
||||||
explicitly authored rather than defaulted. Existing checks on destinations,
|
|
||||||
dependencies, identities, and effective publish rules remain.
|
|
||||||
|
|
||||||
Retain `config sources` and useful provenance in `config artifacts`. Consume
|
|
||||||
ConfigKit's public source, origin, presence, history, and explanation APIs as
|
|
||||||
intended. A thin formatting/projection layer may add Narratio-specific family,
|
|
||||||
character, and party context. It must not duplicate upstream source tracking,
|
|
||||||
inheritance analysis, or derivation bookkeeping.
|
|
||||||
|
|
||||||
An entry can have several contributors. Report the evidence ConfigKit provides
|
|
||||||
without inventing a unique defining-file guarantee or choosing an arbitrary
|
|
||||||
contributor as the owner. Keep logical artifact IDs independent of source files.
|
|
||||||
Small representation changes needed for truthful composite provenance belong
|
|
||||||
to this adoption; retiring inspection commands does not.
|
|
||||||
|
|
||||||
Effective configuration, diffs, and digests continue to reflect resolved
|
|
||||||
domain values. Equivalent inherited and explicitly authored configurations
|
|
||||||
should have equivalent effective projections, regardless of their provenance.
|
|
||||||
Stage reuse continues to use existing semantic evidence.
|
|
||||||
|
|
||||||
### Loading and ownership boundaries
|
|
||||||
|
|
||||||
Retain ordinary ConfigKit loading, centralized typed defaults, imports, selected
|
|
||||||
overlays, domain resolution, credential bootstrap, and exact-name credential
|
|
||||||
forwarding. Preserve current path bases and read-only inspection workflows.
|
|
||||||
|
|
||||||
ConfigKit owns parsing, source composition, structural inheritance, presence,
|
|
||||||
provenance, and secret resolution. Narratio owns its schema, domain defaults,
|
|
||||||
semantic normalization and validation, campaign/family resolution, and runtime
|
|
||||||
use. Supported defaults, resolver, and validator callbacks remain appropriate;
|
|
||||||
there is no requirement to move them after `Load` merely to establish this
|
|
||||||
ownership boundary. ConfigKit presence APIs remain suitable for absence-based
|
|
||||||
defaults, and native metadata remains suitable for operator inspection and
|
|
||||||
existing party ordering.
|
|
||||||
|
|
||||||
Layered loading, embedded configuration sources, programmatic mapping sources,
|
|
||||||
and source-relative resource rebasing are available upstream but are outside
|
|
||||||
this focused adoption. Do not relax empty/null default semantics or change
|
|
||||||
party ordering just to avoid using useful native APIs.
|
|
||||||
|
|
||||||
## Required Changes
|
|
||||||
|
|
||||||
### Dependency and attribution
|
|
||||||
|
|
||||||
Update `go.mod` and `go.sum` to ConfigKit v0.5.0. Update its entry in
|
|
||||||
[attribution](../oss/attribution.md) to the selected version and the reviewed
|
|
||||||
commit's source and notice links. Existing applicable license texts can remain.
|
|
||||||
|
|
||||||
### Configuration integration
|
|
||||||
|
|
||||||
Delete the duplicate-definition check and its obsolete declaration-accounting
|
|
||||||
tests. Remove the family-publish explicit-declaration gate and metadata retained
|
|
||||||
only for that gate. Use existing final-value domain checks instead.
|
|
||||||
|
|
||||||
Simplify source and artifact-definition projections around ConfigKit's public
|
|
||||||
metadata. Remove duplicated tracking or inferred ownership where native evidence
|
|
||||||
already provides the needed information. Retain application context and
|
|
||||||
formatting that actually serve operators. Use the completed load's evidence
|
|
||||||
rather than reparsing sources or performing a second base-only load that could
|
|
||||||
resolve superseded credential references.
|
|
||||||
|
|
||||||
Keep effective rendering and digest construction grounded in the existing safe
|
|
||||||
typed projection, with credential identities rather than resolved values.
|
|
||||||
|
|
||||||
Use ConfigKit's native operation directly. No application-owned inheritance
|
|
||||||
engine, collection registration, alternative keyword, or compatibility loader
|
|
||||||
is needed.
|
|
||||||
|
|
||||||
### Policy alignment
|
|
||||||
|
|
||||||
Revise the Configuration section of the
|
|
||||||
[architecture policy](../policy/architecture.md) to record this responsibility
|
|
||||||
boundary: native loading and metadata belong to ConfigKit; Narratio owns domain
|
|
||||||
meaning and fail-fast effective validation. Explicitly permit supported
|
|
||||||
callbacks and native provenance inspection. Remove wording inconsistent with
|
|
||||||
the established explicit-presence/defaulting semantics rather than introducing
|
|
||||||
a new empty-value policy. General agent, testing, documentation, and OSS
|
|
||||||
policies need no broad revisions for this feature.
|
|
||||||
|
|
||||||
### Compatibility and operator guidance
|
|
||||||
|
|
||||||
Document that `extends` is now reserved structural input throughout ConfigKit
|
|
||||||
loads, including dynamic mappings such as artifact IDs and prompt variables.
|
|
||||||
Operators using that key as ordinary data must rename it. Inheritance is active
|
|
||||||
without an opt-in switch; imports and overlay-file selection remain separately
|
|
||||||
controlled as before.
|
|
||||||
|
|
||||||
Explain native sibling lookup, inherited explicit presence, and the requirement
|
|
||||||
that parent and child entries satisfy Narratio's domain rules. Link to the
|
|
||||||
pinned upstream contract for detailed inheritance semantics. Clarify that a
|
|
||||||
selected overlay may contribute sibling inheritance directives; this does not
|
|
||||||
add inheritance or stacking between overlay files.
|
|
||||||
|
|
||||||
Replace documentation of the one-base-file restriction with ConfigKit's
|
|
||||||
composition contract and Narratio's effective-value validation boundary.
|
|
||||||
Describe any inspection representation changes without promising a unique
|
|
||||||
defining file for a composite entry.
|
|
||||||
|
|
||||||
Update the configuration reference, internal configuration guide, and a small
|
|
||||||
maintained example when implementation lands. Preserve each field's current
|
|
||||||
path-resolution rule when its value is inherited. Record the reserved-key
|
|
||||||
compatibility impact in the eventual release note; automatic configuration
|
|
||||||
conversion is outside this feature.
|
|
||||||
|
|
||||||
### Verification
|
|
||||||
|
|
||||||
Add focused Narratio coverage for cross-file artifact/family inheritance and
|
|
||||||
valid disjoint contributions to one entry. Verify that effective invalid
|
|
||||||
configuration still fails before execution and that structural conflicts still
|
|
||||||
surface as ConfigKit errors. Old source-history rejection tests are retired or
|
|
||||||
rewritten for the new contract, including overlay replacement/removal cases.
|
|
||||||
Verify inherited explicit values against Narratio's defaulting rules, credential
|
|
||||||
safety, native contributor reporting, and equivalent effective projections.
|
|
||||||
|
|
||||||
Concentrate tests on Narratio's integration behavior. ConfigKit's own tests own
|
|
||||||
its generic inheritance algorithm and source lifecycle. Narratio's default
|
|
||||||
tests remain offline, use temporary configuration bundles and controlled secret
|
|
||||||
values, and do not depend on the sibling ConfigKit checkout. Run the existing
|
|
||||||
suite against the selected released module before completing the upgrade.
|
|
||||||
|
|
||||||
## Completion Criteria
|
|
||||||
|
|
||||||
Narratio selects the released ConfigKit v0.5.0 module without a local checkout
|
|
||||||
replacement. Cross-file artifact and family inheritance works through the
|
|
||||||
ordinary loader; valid composition is not rejected by Narratio's declaration
|
|
||||||
history checks. Effective domain errors fail before execution. Native provenance
|
|
||||||
remains inspectable without unique-file assumptions; effective projections,
|
|
||||||
explicit-presence defaulting, and credential handling retain their semantics.
|
|
||||||
|
|
||||||
Maintained guidance and examples explain native inheritance and the reserved
|
|
||||||
`extends` key, and the architecture policy reflects the ConfigKit/Narratio
|
|
||||||
boundary. Attribution identifies the selected upstream revision. Focused
|
|
||||||
regression coverage and the existing suite pass without implementing Audita or
|
|
||||||
workspace changes as part of this upgrade.
|
|
||||||
|
|
||||||
## Relationship To Subsequent Work
|
|
||||||
|
|
||||||
ConfigKit adoption provides a verified configuration baseline for the later
|
|
||||||
Audita and workspace features. It makes no changes to subprocess destinations,
|
|
||||||
session layout, or cleanup behavior.
|
|
||||||
|
|
||||||
Audita adoption then migrates downstream profiles and adds adapter support for
|
|
||||||
explicit report/debug destinations. Audita owns its separate ConfigKit and
|
|
||||||
PromptKit dependencies. Shared latest-attempt links, final workspace paths, and
|
|
||||||
retention protection remain part of the workspace reorganization, so those
|
|
||||||
facilities are implemented once at their shared owner.
|
|
||||||
@@ -1,315 +0,0 @@
|
|||||||
# Adopt ConfigKit v1.2.1 and native configuration ergonomics
|
|
||||||
|
|
||||||
## Purpose
|
|
||||||
|
|
||||||
Adopt current ConfigKit as Narratio's configuration owner and make its native
|
|
||||||
authoring and inspection features useful to operators. Align configuration
|
|
||||||
semantics with other ConfigKit consumers. Existing Narratio assembly conventions
|
|
||||||
are migration inputs rather than compatibility requirements for this work.
|
|
||||||
|
|
||||||
This roadmap defines the target, user intent, and required changes. The
|
|
||||||
[implementation plan](implementation.md) defines the work needed to reach that
|
|
||||||
target. The earlier [v0.5.0 roadmap](config.md) records the completed adoption
|
|
||||||
that forms the current baseline.
|
|
||||||
|
|
||||||
The ownership boundary in [ADR 0003](../adr/0003-assign-configuration-ownership.md)
|
|
||||||
continues to apply: ConfigKit resolves configuration before its final handoff;
|
|
||||||
Narratio owns domain validation, application derivation, and runtime use.
|
|
||||||
Supported callbacks and native metadata provide the integration boundary.
|
|
||||||
|
|
||||||
## Reviewed baseline and releases
|
|
||||||
|
|
||||||
Narratio currently selects `gitea.maximumdirect.net/eric/ConfigKit v0.5.0`,
|
|
||||||
commit `dc09959cca4e105f49ec316c6d4ba585bd2340bd`, with Go 1.26.0 as its
|
|
||||||
consumer minimum. Release builds currently select Go 1.26.7.
|
|
||||||
|
|
||||||
The adoption target is `go.maximumdirect.net/configkit v1.2.1`, commit
|
|
||||||
`51ef3144158db1a09fabdbd1c7ae59908c84923b`. The published tag and the sibling
|
|
||||||
checkout agree on that revision. ConfigKit requires Go 1.27.0 and uses
|
|
||||||
`go.yaml.in/yaml/v4 v4.0.0`. Its code license remains LGPL-3.0-or-later.
|
|
||||||
|
|
||||||
The pinned [configuration contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/config.md)
|
|
||||||
and [consumer guide](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/consumers/README.md)
|
|
||||||
own upstream semantics. The intervening releases supply the following relevant
|
|
||||||
changes:
|
|
||||||
|
|
||||||
| Release | Adoption relevance |
|
|
||||||
| --- | --- |
|
|
||||||
| v0.6.0 | Loading and metadata optimizations; Go minimum moves to 1.26.0. |
|
|
||||||
| v0.7.0 | Native safe effective-configuration `Diff`. |
|
|
||||||
| v0.7.1 | Maintainer-tool update; runtime behavior unchanged. |
|
|
||||||
| v0.8.0 | Input/import destinations, sequence-item placement, shared source-version checks, and initial identified-sequence composition. |
|
|
||||||
| v0.9.0 | Directory and expanded import selection; optional-by-default imports; identified-list replace/append/upsert and inheritance; CLI assignments move after decoding and inheritance. Its final collection APIs replace the initial v0.8 registrations. |
|
|
||||||
| v0.10.0 | Native ordered configuration discovery through `SearchConfig`. |
|
|
||||||
| v0.10.1 | Loading consolidation and import bookkeeping improvements. |
|
|
||||||
| v0.10.2 | Correctness fixes for decoding, callback/resource lifetime, discovery, inspection budgets, and collection operations. |
|
|
||||||
| v1.0.0 | Vanity module path, source-root `config` controls, public API cleanup, source lookup, provenance and sensitivity fixes, and Go 1.27.0 minimum. |
|
|
||||||
| v1.1.0 | Field-level list resolution records and identity-aware inspection across list movement. |
|
|
||||||
| v1.2.0 | Main-relative companion inputs and overlay context for layered loading; overlay filesystem fixes. |
|
|
||||||
| v1.2.1 | Stable go-yaml v4.0.0 dependency; maintainer release-tool update. |
|
|
||||||
|
|
||||||
The corresponding `docs/releases/<version>.md` files at the pinned revision
|
|
||||||
provide the migration details. Select v1.2.1 directly; intermediate releases
|
|
||||||
provide context rather than required deployment stops.
|
|
||||||
|
|
||||||
## Current Narratio integration
|
|
||||||
|
|
||||||
The [configuration reference](../config.md) and
|
|
||||||
[configuration internals](../internal/configuration.md) describe current behavior.
|
|
||||||
|
|
||||||
- Commands select a pipeline path through Narratio's search loop, bootstrap
|
|
||||||
credentials from an explicit directory or a sibling `secrets/` directory,
|
|
||||||
and then call ConfigKit.
|
|
||||||
- Pipeline loading enables explicit path/glob imports and one selected overlay.
|
|
||||||
Campaign, session, and canonical-party documents use separate loads. Session
|
|
||||||
loading also supports command-scoped downloaded input snapshots.
|
|
||||||
- Typed defaults, recorded normalization, domain validation, and native
|
|
||||||
provenance are integrated. Additional path/dependency resolution and
|
|
||||||
campaign-owned family expansion occur after the pipeline load.
|
|
||||||
- Artifact files currently repeat enclosing `scriptorium.artifacts` mappings.
|
|
||||||
Logical IDs come from map keys. Families expand against canonical party data.
|
|
||||||
- Show, diff, and digest use a safe finalized-domain projection. Narratio owns
|
|
||||||
the current mapping-diff algorithm. Native load metadata supplies source and
|
|
||||||
artifact-catalog evidence.
|
|
||||||
|
|
||||||
Some public import descriptions still state restrictions from older ConfigKit
|
|
||||||
behavior. Replace these with current native semantics during adoption rather
|
|
||||||
than recreating them as Narratio checks.
|
|
||||||
|
|
||||||
## Targeted end state
|
|
||||||
|
|
||||||
### Native assembly and operator choice
|
|
||||||
|
|
||||||
ConfigKit owns source discovery, acquisition, composition, inheritance, presence,
|
|
||||||
typed secrets, provenance, and safe native inspection. Narratio supplies its
|
|
||||||
schema, domain defaults and validation, supported application resolution, and
|
|
||||||
operator-selected inputs.
|
|
||||||
|
|
||||||
Use ordinary `Load` for an ordinary configuration. Use public native facilities
|
|
||||||
where they serve the operator workflow. Layered loading is appropriate when
|
|
||||||
actual lower-priority inputs are needed; typed defaults remain appropriate for
|
|
||||||
simple defaults. Main-relative companion inputs allow such underlays to follow
|
|
||||||
a discovered main without Narratio searching for it again.
|
|
||||||
|
|
||||||
Expose the library's authored `config.imports` and `config.sequences` semantics
|
|
||||||
where applicable. Imports use ConfigKit defaults, including optional absence,
|
|
||||||
unless the operator explicitly requests other supported behavior. An invalid
|
|
||||||
selected source still fails. Operators mark essential imports `required: true`.
|
|
||||||
Directory imports can select YAML with an explicit `pattern`; unrestricted
|
|
||||||
filenames remain the library default.
|
|
||||||
|
|
||||||
Native destinations, nested imports, discovery filters, source-local relative
|
|
||||||
import paths, collection operations, and inheritance work under ConfigKit's
|
|
||||||
rules. Narratio validates the final effective domain entries. It adds no
|
|
||||||
fragment-count rule, mandatory fragment version, import-mode allowlist, or
|
|
||||||
parallel composition engine.
|
|
||||||
|
|
||||||
### Ergonomic artifact files and list customization
|
|
||||||
|
|
||||||
Support an artifact definition without repeated enclosing mappings through an
|
|
||||||
explicit import destination. For example:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
config:
|
|
||||||
imports:
|
|
||||||
- path: artifacts/session-recap.yml
|
|
||||||
destination: [scriptorium, artifacts, session_recap]
|
|
||||||
required: true
|
|
||||||
```
|
|
||||||
|
|
||||||
The imported file contains the artifact fields such as `prompt_id`, `inputs`,
|
|
||||||
and `output_path`. The literal destination defines the reusable key
|
|
||||||
`session_recap`, with logical source `narratio.artifact.session_recap`.
|
|
||||||
The filename is an organizational choice. Keyed files mounted at
|
|
||||||
`[scriptorium, artifacts]` support declared directory or glob imports when many
|
|
||||||
definitions should be discovered together. Apply the same pattern to families.
|
|
||||||
|
|
||||||
Keep keyed artifact/family mappings and campaign-owned character data. They
|
|
||||||
already fit native inheritance and stable identity. Full-root and disjoint
|
|
||||||
definitions remain valid native inputs; a one-artifact-per-file organization is
|
|
||||||
guidance rather than a validity requirement.
|
|
||||||
|
|
||||||
Allow authored native collection declarations for existing list fields where
|
|
||||||
their final shapes satisfy Narratio's schema. Operators can choose replacement,
|
|
||||||
append, or upsert and identified-record inheritance. Explain that upsert
|
|
||||||
replaces whole matching records and that an empty append/upsert contribution
|
|
||||||
does not clear a list. Register application-owned sequence identities only
|
|
||||||
where Narratio has a real domain identity requirement; generic library
|
|
||||||
capabilities do not establish new uniqueness requirements for every list.
|
|
||||||
|
|
||||||
### Discovery, credentials, and resource paths
|
|
||||||
|
|
||||||
Replace Narratio-owned local candidate probing with ConfigKit search descriptions
|
|
||||||
and `SearchConfig`. Use the native `WorkingDirectory`, `UserConfigDirectory`,
|
|
||||||
and `UnixSystemDirectories` presets in that order with application name
|
|
||||||
`narratio`. Use their documented filenames. Explicit `--config` selection
|
|
||||||
remains an exclusive `File` input. Report the source actually selected by the
|
|
||||||
library. This provides convenient development-local configuration and standard
|
|
||||||
ConfigKit naming. Existing `pipeline.yml` files remain explicitly selectable;
|
|
||||||
migration guidance explains the new discovery names and precedence.
|
|
||||||
|
|
||||||
Bootstrap uses `LoadSecretsIntoEnvironment` once per invocation before typed
|
|
||||||
credential resolution and concurrent work. Existing environment variables win,
|
|
||||||
including explicitly empty variables. Use the invocation-owned conventional
|
|
||||||
directory `/usr/local/etc/narratio/secrets`, with `--secrets-dir` selecting
|
|
||||||
another directory. A missing conventional directory remains optional; an
|
|
||||||
explicit selection follows the existing explicit-selection behavior. Retain
|
|
||||||
safe partial-report handling and exact-name forwarding to subprocesses.
|
|
||||||
|
|
||||||
The independent directory lets bootstrap precede ConfigKit's first and only
|
|
||||||
configuration selection. Service environments can provide credentials directly;
|
|
||||||
development invocations can choose a project-local secrets directory explicitly.
|
|
||||||
The implicit sibling `secrets/` convention retires with this cutover.
|
|
||||||
|
|
||||||
Resolve relative file-resource references from their effective defining source's
|
|
||||||
ConfigKit resource context. Inherited values follow their effective origin;
|
|
||||||
CLI-relative paths use the invocation directory. Application-derived paths
|
|
||||||
record their dependencies. Use the main source as a documented fallback for
|
|
||||||
origins without a distinct usable resource base. This makes imported definitions
|
|
||||||
portable across bundles.
|
|
||||||
|
|
||||||
Review actual path-bearing fields explicitly. Artifact destinations, command
|
|
||||||
names, session-relative products, and remote object keys retain their domain
|
|
||||||
meanings. This policy introduces no new target-existence or root-containment
|
|
||||||
checks. ConfigKit's consumer examples illustrate application policy; Narratio
|
|
||||||
owns the selected interpretation.
|
|
||||||
|
|
||||||
### Imports in secondary documents
|
|
||||||
|
|
||||||
Enable native explicit imports for locally sourced campaign, party, and session
|
|
||||||
documents as well as pipelines. This supports split rosters and shared metadata
|
|
||||||
with consistent library semantics. Campaign selection, party schema validation,
|
|
||||||
and session domain rules remain application-owned. Pipeline overlay selection
|
|
||||||
remains associated with the pipeline.
|
|
||||||
|
|
||||||
Materialize effective documents where raw root-file copying would leave
|
|
||||||
unresolved imports in downstream inputs or published session data. Remote
|
|
||||||
session snapshots retain their explicit source context; resolving imports
|
|
||||||
against an incidental download directory would give them an incorrect base.
|
|
||||||
Remote companion-file retrieval remains separate work.
|
|
||||||
|
|
||||||
### Final-domain inspection through native APIs
|
|
||||||
|
|
||||||
Keep validate, show, sources, artifact catalog, and diff useful for the operator's
|
|
||||||
fully resolved pipeline, including campaign-driven members and publish rules.
|
|
||||||
Use native effective rendering and `Diff` where they own the operation, with
|
|
||||||
Narratio formatting and generated-domain context at the presentation boundary.
|
|
||||||
|
|
||||||
A raw pipeline `Result` is a frozen load snapshot. Mutating its `Config` later
|
|
||||||
does not update rendering or diff. Integrate this distinction explicitly:
|
|
||||||
record domain resolution within a supported resolver when its inputs are
|
|
||||||
available there; where later campaign resolution remains appropriate, create
|
|
||||||
a completed native inspection result from the finalized safe domain projection
|
|
||||||
using ConfigKit's public loading APIs. That result describes application-derived
|
|
||||||
values; retained authoring results supply actual file provenance.
|
|
||||||
|
|
||||||
Use native comparison rather than maintaining a second general mapping-diff
|
|
||||||
engine. Preserve final-domain coverage and credential identities. Narratio's
|
|
||||||
digest and stage-owned reuse evidence remain application responsibilities.
|
|
||||||
Source-layout changes alone do not change equivalent domain values. Source
|
|
||||||
inspection can report composite origins, empty optional-import selections, and
|
|
||||||
application derivation without inventing a unique defining-file contract.
|
|
||||||
|
|
||||||
### Scope and compatibility
|
|
||||||
|
|
||||||
This is a clean native cutover. Update source-root `configkit:` controls to
|
|
||||||
`config:` and remove old-module/API references in maintained code and fixtures.
|
|
||||||
Nested `config` data remains ordinary data under ConfigKit's source-root rules.
|
|
||||||
The `configkit` package name and `configkit:"secret"` Go tag retain their
|
|
||||||
upstream spelling. Mapping and identified-list `extends` follow current native
|
|
||||||
rules.
|
|
||||||
|
|
||||||
This roadmap deliberately permits revised configuration semantics. Update
|
|
||||||
tests and documentation for selected changes rather than preserving historical
|
|
||||||
acceptance rules through compatibility loaders or keyword aliases.
|
|
||||||
|
|
||||||
Application identity, artifact dependency meaning, runtime output destinations,
|
|
||||||
campaign-owned party authority, remote session retrieval, and stage reuse remain
|
|
||||||
Narratio domain concerns. This scope does not redesign the runtime workspace,
|
|
||||||
change subprocess adapters, adopt ReleaseKit for Narratio, or add external
|
|
||||||
provider probes. Audita and workspace work retain their separate roadmaps.
|
|
||||||
|
|
||||||
## Required changes
|
|
||||||
|
|
||||||
### Dependency, toolchain, and notices
|
|
||||||
|
|
||||||
- Select the released `go.maximumdirect.net/configkit v1.2.1` module and migrate
|
|
||||||
all imports and exchanged named types together. Remove the old requirement;
|
|
||||||
use the released dependency rather than a filesystem replacement.
|
|
||||||
- Raise `go.mod` to Go 1.27.0 and select an available official Go 1.27 patch for
|
|
||||||
development/release validation and release builds. Align CI, runbook setup,
|
|
||||||
and incorporated Go attribution with that selection.
|
|
||||||
- Update ConfigKit source/license links to the reviewed commit and YAML v4
|
|
||||||
attribution to the selected stable release. Check the actual resulting module
|
|
||||||
graph. Maintainer tool dependencies are not automatically Narratio runtime
|
|
||||||
components. Narratio's separate direct YAML use need not be rewritten merely
|
|
||||||
because ConfigKit changed its internal parser dependency.
|
|
||||||
|
|
||||||
### Integration and simplification
|
|
||||||
|
|
||||||
- Apply the v1 API replacements actually used here, including
|
|
||||||
`LoadSecretsIntoEnvironment`, `LayerMain`, `RecordDerivedDefault`, and
|
|
||||||
`RecordNormalization`. Review other public names against the migration guide.
|
|
||||||
- Use keyed expanded options and definition literals. Keep metadata projections
|
|
||||||
compatible with current source evidence, native layer names, and safe errors.
|
|
||||||
- Replace broad list-record workarounds with supported field-level recording
|
|
||||||
when the application changes only fields. Record whole-list changes where
|
|
||||||
order or membership changes warrant them.
|
|
||||||
- Adopt native discovery, destination-based artifact authoring, authored
|
|
||||||
collection operations, and native final-domain rendering/comparison. Remove
|
|
||||||
replaced search, comparison, and redundant metadata machinery.
|
|
||||||
- Apply the agreed source-relative resource bases and secondary-document imports.
|
|
||||||
Share definitions and loading context between execution and inspection so an
|
|
||||||
operator validates the configuration the pipeline will actually consume.
|
|
||||||
- Review secondary-document input materialization and publication as well as
|
|
||||||
parsing. A copied root file can retain imports whose
|
|
||||||
companions are absent at its new location. Materialize the effective domain
|
|
||||||
document where a subprocess or restored session consumes it, retaining native
|
|
||||||
authoring evidence separately. Review placeholder checks against the effective
|
|
||||||
session input and preserve required external party/session formats.
|
|
||||||
|
|
||||||
### Documentation and maintained examples
|
|
||||||
|
|
||||||
Update the configuration reference, CLI reference, operations/bootstrap guidance,
|
|
||||||
configuration internals, contributor prerequisites, and release procedure for
|
|
||||||
the implemented semantics. Keep canonical facts in their respective owners.
|
|
||||||
Update project-specific architecture wording where selected path/bootstrap
|
|
||||||
decisions change it; preserve ADR 0003's ownership boundary.
|
|
||||||
|
|
||||||
Migrate the production/testing bundle to `config:`. Demonstrate destination-based
|
|
||||||
artifact and family files, a native directory import restricted to YAML, an
|
|
||||||
essential required import, native inheritance, and a useful selected-overlay
|
|
||||||
list customization. Keep illustrative optional choices labeled as choices.
|
|
||||||
Show how the declared artifact key is referenced by dependent artifacts.
|
|
||||||
|
|
||||||
Provide concise manual upgrade instructions for existing Narratio configurations:
|
|
||||||
control-key/API changes, optional import defaults, collection behavior, source
|
|
||||||
discovery, and selected path/bootstrap changes. Separately installed applications
|
|
||||||
own their configuration versions; Narratio's migration does not automatically
|
|
||||||
rewrite their files.
|
|
||||||
|
|
||||||
The eventual Narratio release requires a major version because existing valid
|
|
||||||
configuration control blocks change. The Go minimum increase alone is permitted
|
|
||||||
in a minor release under the adopted release policy. This roadmap does not
|
|
||||||
authorize tagging, publication, or retrospective release-note changes.
|
|
||||||
|
|
||||||
### Verification
|
|
||||||
|
|
||||||
Use temporary inputs and controlled synthetic credentials. Exercise native
|
|
||||||
discovery and the chosen bootstrap convention together, explicit file selection,
|
|
||||||
selected-source errors, optional and required imports, import destinations,
|
|
||||||
inherited artifact identities, native list customization, and final domain
|
|
||||||
validation. Add coverage at Narratio boundaries rather than duplicating
|
|
||||||
ConfigKit's complete structural test matrix.
|
|
||||||
|
|
||||||
Verify inherited presence/defaulting, party order, final campaign/family
|
|
||||||
expansion, resource paths, source attribution, and secret-safe inspection.
|
|
||||||
Diff/show must include derived members and publish rules; compare equal effective
|
|
||||||
configurations across different source arrangements. Credential rotation behind
|
|
||||||
the same reference name must remain absent from semantic diffs and digests.
|
|
||||||
|
|
||||||
Update tests that encode intentionally retired syntax or policies. Run focused
|
|
||||||
configuration/command/example/documentation checks and the established full
|
|
||||||
test, race, vet, build, and module checks on the adopted Go line. Preserve the
|
|
||||||
current supported build targets. No new live-service or CI-completion gate is
|
|
||||||
introduced by this migration.
|
|
||||||
@@ -1,431 +0,0 @@
|
|||||||
# ConfigKit v1.2.1 implementation plan
|
|
||||||
|
|
||||||
## Scope and authority
|
|
||||||
|
|
||||||
Implement [the approved ConfigKit roadmap](configkit.md). It owns the target,
|
|
||||||
native-semantics intent, and four approved product decisions. This plan replaces
|
|
||||||
the completed v0.5.0 plan. Its audience is a Luna coding agent implementing one
|
|
||||||
end-to-end sprint in the numbered order below.
|
|
||||||
|
|
||||||
Select `go.maximumdirect.net/configkit v1.2.1`, upstream commit
|
|
||||||
`51ef3144158db1a09fabdbd1c7ae59908c84923b`. The pinned
|
|
||||||
[consumer guide](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/consumers/README.md)
|
|
||||||
and [configuration contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/51ef3144158db1a09fabdbd1c7ae59908c84923b/docs/config.md)
|
|
||||||
own upstream semantics. Read the sections identified by each stage and public
|
|
||||||
declarations in the released module as needed. The sibling checkout is optional
|
|
||||||
reference material, not a build or test prerequisite.
|
|
||||||
|
|
||||||
## Working instructions
|
|
||||||
|
|
||||||
- Start with `docs/development.md` and `docs/policy/agents.md`. Architecture,
|
|
||||||
testing, documentation, OSS, and release policies apply; reuse unchanged
|
|
||||||
policy text already read. Prefer graph tools for code discovery.
|
|
||||||
- Implement the full native configuration workflow. Discovery, destinations,
|
|
||||||
collection controls, resource bases, secondary imports, materialization, and
|
|
||||||
native inspection are part of this sprint.
|
|
||||||
- Stages are work boundaries, not deployment releases. Intermediate stages may
|
|
||||||
fail to compile or have dependent tests awaiting later work. Record that state
|
|
||||||
accurately. Avoid temporary adapters or extra abstractions merely to make
|
|
||||||
every intermediate commit green. Final validation covers the assembled system.
|
|
||||||
- Commit completed nonempty units with short plain-English messages. Add focused
|
|
||||||
behavioral tests with their owner; run meaningful available checks. The final
|
|
||||||
stage runs repository-wide validation against the finished source.
|
|
||||||
- Use ConfigKit's public loading, presence, provenance, recording, rendering,
|
|
||||||
and comparison APIs. Narratio supplies domain meaning and runtime use.
|
|
||||||
Retire obsolete source-authoring checks and tests rather than reproducing them.
|
|
||||||
- Leave import defaults unspecified so ConfigKit supplies its defaults.
|
|
||||||
Explicit false/zero/empty/null and collection semantics follow the native
|
|
||||||
contract and actual Narratio domain rules.
|
|
||||||
- Use temporary inputs, real ConfigKit, and controlled synthetic credentials.
|
|
||||||
The default tests remain offline and independent of installed subprocesses.
|
|
||||||
- Write documentation in concise, affirmative, factual language. The roadmap
|
|
||||||
owns intent; this file owns sequencing and implementation choices.
|
|
||||||
|
|
||||||
Preserve stage ordering, artifact dependency meaning, the canonical party schema,
|
|
||||||
exact-name credential forwarding, publish commit semantics, and the existing
|
|
||||||
Linux amd64/arm64 plus best-effort macOS arm64 targets. This sprint does not
|
|
||||||
implement pending Audita/workspace features, migrate old sessions, rewrite
|
|
||||||
downstream configurations, add fragment version mandates or new path checks,
|
|
||||||
adopt ReleaseKit for Narratio, or publish a release.
|
|
||||||
|
|
||||||
## Concrete integration choices
|
|
||||||
|
|
||||||
- Consumer Go minimum: **1.27.0**. Development/release validation and build
|
|
||||||
toolchain: official **Go 1.27.4**, an available patch on that line. Selecting
|
|
||||||
it is not a new publication gate.
|
|
||||||
- Pipeline source when no explicit file is supplied:
|
|
||||||
`SearchConfig("narratio", Searches(WorkingDirectory(), UserConfigDirectory(),
|
|
||||||
UnixSystemDirectories()), SearchOptions{Required: true})`.
|
|
||||||
An explicit `--config` uses `File` exclusively.
|
|
||||||
- Bootstrap: optional `/usr/local/etc/narratio/secrets`, or explicit
|
|
||||||
`--secrets-dir`, through `LoadSecretsIntoEnvironment` before typed loading.
|
|
||||||
Existing environment values win. The implicit sibling directory retires.
|
|
||||||
- Local campaign/session/party loading enables imports. Only the pipeline gets
|
|
||||||
its selected overlay. Party version checking remains main-only; fragments
|
|
||||||
need no new version header. Remote session bytes are contextless snapshots.
|
|
||||||
- Existing campaign registry selection and local session search locations remain
|
|
||||||
domain conventions. Use native optional `SearchConfig` with `SearchPaths`
|
|
||||||
for local session candidates. Only actual absence permits remote fallback.
|
|
||||||
- Final-domain inspection uses a safe DTO and a defaults-backed
|
|
||||||
`Load(NoSource(), Definition.Defaults)` snapshot after domain resolution.
|
|
||||||
Credential fields contain reference names, not typed secrets that would
|
|
||||||
resolve again. Original results supply actual file provenance. Stages 8–9
|
|
||||||
explain this boundary.
|
|
||||||
|
|
||||||
## Stage 1: Establish the dependency and toolchain baseline [x]
|
|
||||||
|
|
||||||
Read `go.mod`, `go.sum`, `.woodpecker/*.yml`, release scripts/runbook, and
|
|
||||||
attribution. Read the upstream v1.0.0 migration table and v1.2.1 release note.
|
|
||||||
|
|
||||||
1. Select Go 1.27.0 in `go.mod` and ConfigKit v1.2.1 under its vanity path.
|
|
||||||
Migrate all production/test imports together and remove the old requirement.
|
|
||||||
Use the released module rather than a filesystem replacement.
|
|
||||||
2. Replace APIs actually used here: `LoadSecretsDir` →
|
|
||||||
`LoadSecretsIntoEnvironment`, `LayerBase` → `LayerMain`, and generic
|
|
||||||
`Resolution.Record` calls with `RecordDerivedDefault` or
|
|
||||||
`RecordNormalization`. Audit other removed public names and compared layer
|
|
||||||
or diagnostic strings. Use keyed expanded options/definition literals.
|
|
||||||
3. Rename genuine source-root `configkit:` controls to `config:` throughout
|
|
||||||
fixtures and examples. Keep the package name, `configkit:"secret"` Go tag,
|
|
||||||
and ordinary data keys distinct from source controls.
|
|
||||||
4. Align Go CI images and local/release instructions with Go 1.27.4. Preserve
|
|
||||||
platform targets and publication mechanics. Update incorporated Go notices.
|
|
||||||
5. Refresh ConfigKit and stable go-yaml v4.0.0 attribution from selected source
|
|
||||||
notices. Review actual module graph churn and tool-only dependencies. Keep
|
|
||||||
Narratio's separate YAML v3 use unless another scoped change needs it.
|
|
||||||
|
|
||||||
Check module identity and attribution. Attempt focused config/app tests; record
|
|
||||||
failures assigned to later stages. Complete when the dependency/API baseline is
|
|
||||||
established, without preserving old behavioral expectations as compatibility.
|
|
||||||
|
|
||||||
## Stage 2: Integrate native pipeline discovery and independent bootstrap [x]
|
|
||||||
|
|
||||||
Read `internal/app/pipeline_config_path.go`, `secrets_bootstrap.go`,
|
|
||||||
`command_dependencies.go`, `loadPipelineConfigWithDependencies` and callers,
|
|
||||||
inspection loaders, and `internal/config/pipeline_configkit.go`.
|
|
||||||
Read upstream discovery and secrets-bootstrap contracts.
|
|
||||||
|
|
||||||
1. Have the configuration owner accept an explicit file or the native required
|
|
||||||
discovery source above. Retain explicit-file package entry points where useful.
|
|
||||||
Commands load through one composition path instead of first probing with
|
|
||||||
`os.Stat` to obtain a default pathname.
|
|
||||||
2. Bootstrap once before typed loading or adapter creation from the independent
|
|
||||||
directory. Preserve explicit-path error behavior, safe partial reports, and
|
|
||||||
existing-empty environment precedence.
|
|
||||||
3. Obtain the actual selected root and import/overlay evidence from ConfigKit.
|
|
||||||
Carry that root into campaign selection, provenance, and command summaries.
|
|
||||||
Remove dependencies/search helpers used only for old pipeline preselection.
|
|
||||||
4. Apply this composition to run/range/alias commands, session init/plan/status/
|
|
||||||
restore, inspection, and remote-session workflows. A selected invalid source
|
|
||||||
fails rather than falling through to another pipeline candidate.
|
|
||||||
5. For overlay diff, load the left through native selection and the right from
|
|
||||||
that selected root with its own overlay. Share invocation bootstrap; neither
|
|
||||||
side performs a preliminary base-only load.
|
|
||||||
|
|
||||||
Test precedence and exclusive explicit selection with controlled working/user
|
|
||||||
directories and temporary native search descriptions. Test selected invalid
|
|
||||||
input, no source, bootstrap override/default behavior, and diff bootstrap once.
|
|
||||||
Keep tests isolated from the operator's system configuration and secrets.
|
|
||||||
|
|
||||||
## Stage 3: Resolve pipeline resources from native origin context [x]
|
|
||||||
|
|
||||||
Read `resolvePipelineConfig`, `resolveNotariusPaths`,
|
|
||||||
`finalizeLoadedPipeline`, `effective_digest.go`, and the upstream source-aware
|
|
||||||
path example. Resolve paths before normalization erases entry origins.
|
|
||||||
|
|
||||||
1. Introduce one small config-owned host-resource base helper using
|
|
||||||
`Resolution.EntryInfo`, `Resolution.Source`, and
|
|
||||||
`Resolution.SourceContext`. Use the value's effective origin rather than
|
|
||||||
its child's inheritance directive. A distinct usable source context supplies
|
|
||||||
its canonical base; missing or composite context falls back to the main source.
|
|
||||||
Explicit CLI-origin file references use the captured invocation directory.
|
|
||||||
If the main input is contextless too, use that invocation directory rather
|
|
||||||
than interpreting a diagnostic label as a filesystem base. This covers remote
|
|
||||||
byte snapshots without turning their temporary filenames into resource roots.
|
|
||||||
2. Normalize nonempty relative references with recorded helpers. Absolute values
|
|
||||||
stay absolute. Derive Notarius working directory from its already resolved
|
|
||||||
config path and record that dependency. Preserve domain validation of relevant
|
|
||||||
empty/disabled values.
|
|
||||||
3. Apply this classification:
|
|
||||||
|
|
||||||
| Fields | Interpretation |
|
|
||||||
| --- | --- |
|
|
||||||
| Audita/Scriptorium/Notarius `config_path`; Notarius `working_directory` | File/directory resource based on effective origin. |
|
|
||||||
| Relative `workspace.root`, `campaigns.root`, `spool.root`, `cache.root` | Host directory reference based on effective origin; absolute defaults stay unchanged. |
|
|
||||||
| Campaign template/stable-input paths; session stable-input/audio paths | Same resource policy, completed in Stage 4. |
|
|
||||||
| Artifact output paths, family output patterns, publish destinations, source IDs, prompt/profile IDs, schema identifiers, binary commands, Notarius reference bindings and chunk-cache modes | Existing domain interpretation, not file-resource rebasing. |
|
|
||||||
|
|
||||||
4. Remove duplicate pipeline-root rebasing and logical-Notarius path caches once
|
|
||||||
callers use resolved domain values. Show/digest describe those domain values,
|
|
||||||
rather than reconstructing an alternative authoring-path view.
|
|
||||||
5. Keep resource access at existing consumer boundaries; base selection adds no
|
|
||||||
target-existence check, root restriction, or history acceptance gate.
|
|
||||||
|
|
||||||
Test imported, inherited, overlay, absent-default, absolute, and ambiguous-origin
|
|
||||||
cases. Equal resolved resources have equal effective semantics across layouts;
|
|
||||||
relative text resolving to a different resource is a meaningful change.
|
|
||||||
|
|
||||||
## Stage 4: Load campaign/session imports with correct contexts [x]
|
|
||||||
|
|
||||||
Read `document_configkit.go`, `load.go`, `pathresolve.go`,
|
|
||||||
`party_resolution.go`, app session discovery/remote-session composition, and
|
|
||||||
session init. Read native import and resource-context contracts.
|
|
||||||
|
|
||||||
1. Use `File` with imports enabled for local campaign/session inputs. Share
|
|
||||||
definitions with byte loading and retain successful results/source contexts.
|
|
||||||
ConfigKit owns per-source and aggregate acquisition limits.
|
|
||||||
2. Replace local session candidate probing with native optional discovery over
|
|
||||||
the existing candidate paths. Permit absent input solely for remote fallback;
|
|
||||||
distinguish absence through native source evidence. Structure callbacks to
|
|
||||||
allow that discovery-only no-source result. Acquired invalid files still
|
|
||||||
receive ordinary validation and fail without remote fallback.
|
|
||||||
3. Downloaded session bytes receive no context inferred from their temporary
|
|
||||||
filename. Retain object provenance and current explicit session/previous-ID
|
|
||||||
checks. Use native contextless-import behavior; no remote import fetcher or
|
|
||||||
custom structural rejection layer is added.
|
|
||||||
4. Resolve campaign `session_template_file`, stable-input paths, and session
|
|
||||||
overrides/audio references with Stage 3's helper. Resolve each
|
|
||||||
`audio_files` element at its entry index before ordering. Keep campaign/
|
|
||||||
session stable-input precedence and campaign ownership of party input.
|
|
||||||
Provenance identifies the effective supplying configuration source.
|
|
||||||
5. Check unresolved template tokens in effective session string values, including
|
|
||||||
imported values. Session init still reads an authored template for intentional
|
|
||||||
substitution; runtime loading stays a configuration operation.
|
|
||||||
6. Retain detached campaign and session document values after their own loading
|
|
||||||
and resource resolution. Capture the session document before campaign-inherited
|
|
||||||
fields are copied into runtime session state. Stage 6 serializes these values.
|
|
||||||
|
|
||||||
Test local imports, resource bases, effective placeholder rejection, identity
|
|
||||||
mismatches, imported session-init template paths, absence versus selected-source
|
|
||||||
failure, and remote snapshots independent of temporary directories. Add no
|
|
||||||
secondary overlay flags.
|
|
||||||
|
|
||||||
## Stage 5: Resolve imported party definitions and canonical roster output [x]
|
|
||||||
|
|
||||||
Read `party.go`, `party_resolution.go`, `artifact_families.go`, the party
|
|
||||||
integration contract, and focused party/family tests.
|
|
||||||
|
|
||||||
1. Add a file-backed party entry point with imports enabled for campaign
|
|
||||||
resolution. Keep `ParseParty(bytes)` contextless for snapshots and tests.
|
|
||||||
Reuse the definition, main-only version rule, and domain validator.
|
|
||||||
2. Preserve `narratio.party.v1`, existing character/name/class/alias validation,
|
|
||||||
and campaign ownership. Native destinations allow split roster contributions;
|
|
||||||
no fragment version mandate or abstract/template record type is added.
|
|
||||||
3. Use native metadata for roster order and evidence, preserving authored order
|
|
||||||
within a source. Extend the current ordering helper using source acquisition
|
|
||||||
order, then line/column and ID as deterministic ties across sources. Prefer a
|
|
||||||
child's own declaration/inheritance location over its parent's location.
|
|
||||||
This is a presentation convention, not a new source acceptance rule.
|
|
||||||
4. Render canonical party YAML from the effective roster: schema version, keyed
|
|
||||||
characters, singular external `alias`, classes and optional levels. Preserve
|
|
||||||
meaningful roster/list order. A small typed DTO/YAML-node writer is suitable
|
|
||||||
for this Narratio-owned external format.
|
|
||||||
5. Keep players derivation and family expansion based on that same roster.
|
|
||||||
Replace output/checksum uses of `CanonicalParty.Raw` with canonical rendering;
|
|
||||||
retain raw data only where an actual diagnostic caller needs it.
|
|
||||||
|
|
||||||
Test split/inherited rosters, aliases/classes, effective duplicate-name rejection,
|
|
||||||
order, reloadable standalone canonical output, players, and family identities.
|
|
||||||
Use native evidence rather than a second source parse to reconstruct order.
|
|
||||||
|
|
||||||
## Stage 6: Materialize effective inputs and align prepare resume checks [x]
|
|
||||||
|
|
||||||
Read `internal/stage/prepare.go`, `prepare_party.go`, `prepare_resume.go`,
|
|
||||||
prepared-input descriptors, and prepare semantic evidence/tests.
|
|
||||||
|
|
||||||
1. Add configuration-owned deterministic campaign/session renderers from Stage
|
|
||||||
4's detached document values. Serialize public schemas and resolved file
|
|
||||||
references, excluding native directives and runtime/provenance state.
|
|
||||||
2. Preserve ownership: campaign-inherited stable inputs must not become explicit
|
|
||||||
session overrides. A published session that omitted `speakers_file` must
|
|
||||||
still inherit the receiving campaign's value. Explicit session overrides
|
|
||||||
remain explicit. Include actual session identity values where application
|
|
||||||
resolution supplies them, without freezing campaign defaults into the session.
|
|
||||||
3. Prepare writes these documents and Stage 5's party bytes through existing
|
|
||||||
artifact-store helpers. Keep filenames, input kinds, provenance fields, and
|
|
||||||
write/checksum behavior. Resume uses the same renderers instead of comparing
|
|
||||||
prepared effective output with a raw imported root.
|
|
||||||
4. Ensure imported effective changes reach existing stage-owned semantic/input
|
|
||||||
evidence. Equivalent domain values remain equivalent. Full pipeline digests,
|
|
||||||
source mtimes, or every authoring file are not new blanket invalidation keys.
|
|
||||||
5. Preserve ordinary stable-file copying/audio handling. Already resolved paths
|
|
||||||
must not be rebased again against the session root. Retire raw campaign/session
|
|
||||||
file requirements used solely for copying the old authoring documents.
|
|
||||||
|
|
||||||
Test standalone prepared documents, imported changes, equivalent layouts, and
|
|
||||||
remote-origin metadata. Canonical-byte format changes can cause a normal prepare
|
|
||||||
rerun; no session migration or new manifest schema is needed for serialization.
|
|
||||||
|
|
||||||
## Stage 7: Close publication, restore, and remote-session round trips [x]
|
|
||||||
|
|
||||||
Read publish input selection, remote-session loading, restore contracts, and
|
|
||||||
existing in-memory storage integration tests.
|
|
||||||
|
|
||||||
1. Trace the actual bytes published as remote `session.yml`. Publish the canonical
|
|
||||||
prepared session rather than the original root or command-scoped download.
|
|
||||||
Preserve immutable object/commit-pointer ordering and lock behavior.
|
|
||||||
2. Confirm campaign/party prepared products flow through existing selections
|
|
||||||
with their identities intact and without unresolved companion imports.
|
|
||||||
3. Verify a published canonical session loads through the existing contextless
|
|
||||||
path after authoring fragments and temporary files disappear. Restore uses
|
|
||||||
recorded contents and existing safety/locking. No remote import graph or
|
|
||||||
producer-host filesystem search is introduced.
|
|
||||||
4. Verify omitted session stable fields inherit the receiving campaign's values.
|
|
||||||
Keep remote object identity evidence separate from checksums of locally
|
|
||||||
materialized canonical bytes.
|
|
||||||
|
|
||||||
Use fake/in-memory storage for prepare → publish selection → remote load and
|
|
||||||
relevant restore coverage. Fix actual producers/consumers rather than adding a
|
|
||||||
parallel publication path. Existing atomic commit tests remain authoritative.
|
|
||||||
|
|
||||||
## Stage 8: Build native final-domain inspection snapshots [x]
|
|
||||||
|
|
||||||
Read `effective_projection.go`, `effective_digest.go`,
|
|
||||||
`effective_pipeline.go`, pipeline finalization/family expansion, and native
|
|
||||||
completion, defaults, rendering, and typed-secret contracts.
|
|
||||||
|
|
||||||
1. Keep one safe schema-specific final-domain DTO containing concrete expanded
|
|
||||||
artifacts and generated publish outputs. Omit unexpanded family declarations
|
|
||||||
and bookkeeping. Audit all credentials, including S3 fields and Audita/
|
|
||||||
Notarius/Scriptorium lists: use exact reference names, with no resolved
|
|
||||||
`Secret` fields in the inspection DTO.
|
|
||||||
2. Build a native DTO result with `Load(NoSource(), Options{Definition:
|
|
||||||
Definition{Defaults: ...}})`, supplying a fresh safe DTO and no external
|
|
||||||
credential lookup or application callback rerun. ConfigKit copies and freezes
|
|
||||||
these already resolved values. Using defaults avoids interpreting final
|
|
||||||
`config`/`extends` data as fresh directives or converting precise numbers
|
|
||||||
through a generic mapping-source reconstruction.
|
|
||||||
3. Build/refresh the snapshot at existing domain-finalization boundaries, after
|
|
||||||
applicable path/dependency/family resolution. Keep original authoring results
|
|
||||||
separately. Synthetic default origins describe this derived snapshot, not
|
|
||||||
original file provenance. No general snapshot service is needed.
|
|
||||||
4. Route effective show and `pipeline.resolved.yml` rendering through native
|
|
||||||
`WriteEffective`. Use the same DTO's deterministic JSON for Narratio's
|
|
||||||
semantic digest. Preserve exact integers, domain null/empty values, and
|
|
||||||
credential identities. Update the digest discriminator if its format changes.
|
|
||||||
5. Remove flattening/serialization machinery used only to duplicate native
|
|
||||||
show/diff. Keep narrow path enumeration and family/party context genuinely
|
|
||||||
needed by catalog/source presentation and stage-owned reuse evidence.
|
|
||||||
|
|
||||||
Test final family/publish expansion, inherited/expanded equivalence, large
|
|
||||||
integer variables, null/empty values, safe rendering, credential-reference
|
|
||||||
changes versus rotation, and no credential resolution during snapshot creation.
|
|
||||||
|
|
||||||
## Stage 9: Adopt native diff and truthful operator source inspection [x]
|
|
||||||
|
|
||||||
Read `internal/app/config_diff.go`, config command loaders/tests,
|
|
||||||
`artifact_catalog.go`, and native Diff/source/history contracts.
|
|
||||||
|
|
||||||
1. Fully resolve each overlay against the same selected campaign/party before
|
|
||||||
comparing Stage 8 snapshots with `configkit.Diff`. Remove
|
|
||||||
`diffEffectivePipelineValues`, its general map comparison, and consistency
|
|
||||||
checks predicated on the old flattened-leaf change model.
|
|
||||||
2. Present canonical JSON Pointer paths in native order. Derive added/removed/
|
|
||||||
changed from snapshot `Present`, not nil or redaction text. Use compact JSON
|
|
||||||
for safe values and expose `HiddenChanged` if present. Equal values produce
|
|
||||||
`no differences`. Document pointer notation and native changed-region/list
|
|
||||||
granularity instead of emulating the old engine.
|
|
||||||
3. Source/catalog evidence comes from authoring results through public source
|
|
||||||
lookup and metadata history/origins. Add clearly labeled empty optional-import
|
|
||||||
selection evidence to `config sources`, without guessed candidates or
|
|
||||||
reasons for absence.
|
|
||||||
4. Preserve artifact/family/character IDs and generated party context. Distinguish
|
|
||||||
normalization origins from retained definition evidence. Use native `main`
|
|
||||||
where an upstream layer is reported; root/import presentation roles may remain
|
|
||||||
where they express actual application context.
|
|
||||||
|
|
||||||
Test changed family members and publish rules, equivalent layouts, composite
|
|
||||||
contributors, optional empty imports, atomic lists, and pointer escaping in
|
|
||||||
allowed dynamic mapping keys. Inspection remains secret-safe and read-only,
|
|
||||||
without source reparsing or provenance-based acceptance gates.
|
|
||||||
|
|
||||||
## Stage 10: Exercise native authoring and maintain ergonomic examples [x]
|
|
||||||
|
|
||||||
Read the production/testing bundle, artifact/family tests, native destinations
|
|
||||||
and collection rules, and maintained example tests.
|
|
||||||
|
|
||||||
1. Convert artifact/family files to destination-based definitions. IDs remain
|
|
||||||
explicit destination keys or keyed payloads; filenames do not imply IDs.
|
|
||||||
2. Demonstrate a declared directory import with `pattern: "*.yml"`, essential
|
|
||||||
`required: true` input, an optional selection, inheritance, and a useful
|
|
||||||
testing-overlay list customization through `config.sequences`.
|
|
||||||
3. Cover append/upsert/replace on an existing list using an appropriate authored
|
|
||||||
identity, such as publish output `source`. Whole-record upsert and empty-list
|
|
||||||
behavior follow ConfigKit. No application identity registration is required
|
|
||||||
merely to enable authored collection controls.
|
|
||||||
4. Demonstrate a split roster and local secondary imports using the sample
|
|
||||||
campaign, with companion resource paths following source-origin policy.
|
|
||||||
Explain that full-root and disjoint definitions remain accepted too.
|
|
||||||
5. Use ordinary `Load` for this authored-import workflow. Layered loading is
|
|
||||||
useful for actual lower-priority sources, not required for defaults alone.
|
|
||||||
If such a source is genuinely needed, use `LoadLayered` and `MainRelative`
|
|
||||||
directly with focused coverage instead of another merge/search mechanism.
|
|
||||||
|
|
||||||
Load examples through production definitions. Check dependent logical IDs,
|
|
||||||
concrete family members, testing differences, and standalone prepared inputs.
|
|
||||||
Keep examples synthetic and credential-free.
|
|
||||||
|
|
||||||
## Stage 11: Align canonical documentation and project-specific policy [x]
|
|
||||||
|
|
||||||
Read documentation/OSS/release policies, affected references, and the approved
|
|
||||||
roadmap. Finish current-behavior documentation for the assembled implementation.
|
|
||||||
|
|
||||||
1. Update config/CLI references, operations/bootstrap guidance, troubleshooting,
|
|
||||||
configuration internals, prepare internals, and party integration docs for the
|
|
||||||
new discovery, import, resource-base, materialization, and inspection behavior.
|
|
||||||
2. Replace obsolete import restrictions with an accurate native summary and
|
|
||||||
pinned upstream links. Explain optional imports, `required`, filters,
|
|
||||||
destinations, source-local root controls, list modes, and reserved-key scope.
|
|
||||||
Keep effective domain validation Narratio-owned.
|
|
||||||
3. Align project-specific architecture configuration/startup statements and the
|
|
||||||
current raw-party-copy description in its actual owner. Preserve accepted
|
|
||||||
ADR 0003 and generic adopted policies. Routine integration details do not
|
|
||||||
require new ADRs.
|
|
||||||
4. Write manual upgrade guidance for control names, native discovery filenames/
|
|
||||||
precedence, explicit custom filenames, secrets override, relative path changes,
|
|
||||||
required imports where desired, and list operations. Explain effective prepared
|
|
||||||
documents versus authored roots; downstream migrations remain separate.
|
|
||||||
5. Finish Go prerequisites, CI/runbook toolchain alignment, and attribution from
|
|
||||||
Stage 1. Record impact for eventual major-release preparation without inventing
|
|
||||||
a release version or changing published notes. Leave pending Audita/workspace
|
|
||||||
roadmaps and Proposed ADRs intact.
|
|
||||||
|
|
||||||
Run doccheck and maintained example validation. Each volatile contract has one
|
|
||||||
canonical owner. Retire completed roadmap files only when separately requested.
|
|
||||||
|
|
||||||
## Stage 12: Verify the assembled migration and close the sprint [x]
|
|
||||||
|
|
||||||
1. Extend a command-level temporary bundle with native discovery, explicit
|
|
||||||
secrets override, imported campaign/party/session resources, destination-based
|
|
||||||
artifacts, family expansion, and a testing overlay. Reuse preceding focused
|
|
||||||
coverage instead of repeating ConfigKit's structural matrix.
|
|
||||||
2. Exercise validate/show/sources/artifacts/diff through real composition. Check
|
|
||||||
selected root, resources, generated values, optional-import evidence, safe
|
|
||||||
credentials, and absence of runtime workspace creation by inspection. Exercise
|
|
||||||
plan or prepare with fake external seams to establish execution parity.
|
|
||||||
3. Trace remaining callers of old search, rebasing, raw-copy, and diff helpers.
|
|
||||||
Remove dead code/test seams. Review intentional test expectation changes
|
|
||||||
against the roadmap and preserve existing consequential domain/safety tests.
|
|
||||||
4. With Go 1.27.4 and dependencies prepared, run `go test ./...`,
|
|
||||||
`go test -race ./...`, `go vet ./...`, `go build ./...`,
|
|
||||||
`go mod tidy -diff`, and `git diff --check`. Use the existing release asset
|
|
||||||
builder with a temporary output directory and its existing `v0.0.0-test` label
|
|
||||||
to check the three targets; this invokes no publication or new release note.
|
|
||||||
5. Inspect `go list -m` for the exact release, absence of the old module path
|
|
||||||
and filesystem replacements, and expected notices. Review every roadmap
|
|
||||||
capability and all four approved decisions; native structural behavior must
|
|
||||||
not be narrowed by new Narratio acceptance gates.
|
|
||||||
6. Commit the finished unit and report checks, compatibility changes, revision,
|
|
||||||
and any genuine unfinished work. Leave a clean workspace. Actual release
|
|
||||||
publication and the next feature require separate authorization.
|
|
||||||
|
|
||||||
Complete when the native workflow works end to end, prepared inputs survive
|
|
||||||
loss of authoring companions, operator inspection covers final domain values,
|
|
||||||
and final validation passes.
|
|
||||||
|
|
||||||
## Open Questions
|
|
||||||
|
|
||||||
None required for this sprint. The four product choices are approved. This plan
|
|
||||||
selects concrete native snapshot, source-context, materialization, and toolchain
|
|
||||||
approaches without adding compatibility shims, structural restrictions, or
|
|
||||||
stronger runtime guarantees.
|
|
||||||
@@ -1,459 +0,0 @@
|
|||||||
# ConfigKit adoption review remediation
|
|
||||||
|
|
||||||
## Review scope
|
|
||||||
|
|
||||||
Reviewed the twelve completed stages in [implementation.md](implementation.md)
|
|
||||||
against [the approved target](configkit.md), the implementation diff from
|
|
||||||
`65d0e51` through `39a958675d68e940799f4cb8c8e6748830d1483a`, current code,
|
|
||||||
documentation, focused diagnostic reproductions, and repository checks.
|
|
||||||
|
|
||||||
An integration-focused follow-up reviewed the same implementation at
|
|
||||||
`bccf03a99f45c2b9b342412b85f70e356663df61`, tracing each stage into existing
|
|
||||||
command, preparation, publication, restore, and inspection workflows. Items
|
|
||||||
5–6 were additional findings from that pass; items 1–4 were outstanding then.
|
|
||||||
|
|
||||||
An algorithm-focused pass reviewed all twelve stages at
|
|
||||||
`583c5bdf1bc8f0af8ec2e257bcb03ecccbf283b1`. Items 7–10 are refactoring
|
|
||||||
opportunities for consideration. Complexity follows the code and the selected
|
|
||||||
ConfigKit v1.2.1 implementation; no operational timings or benchmarks were
|
|
||||||
collected. The pass preserves algorithms expressly chosen by the plan.
|
|
||||||
|
|
||||||
All ten items below are now complete. The original findings are retained with
|
|
||||||
completion annotations; final verification is recorded at the end of this file.
|
|
||||||
|
|
||||||
## 1. P2: Distinguish effective resource origins from superseded history
|
|
||||||
|
|
||||||
Status: Complete. Native pre-resolution Previous evidence supplies current
|
|
||||||
contributors; regression coverage includes overlay replacement and inheritance.
|
|
||||||
|
|
||||||
Stage: 9, with documentation coverage in Stage 11.
|
|
||||||
|
|
||||||
Location: `internal/config/effective_pipeline.go:274–280`,
|
|
||||||
`effectivePipelineOwners`.
|
|
||||||
|
|
||||||
For a resolution-origin field, the projection adds every main/overlay origin
|
|
||||||
from native history to the same current source-role records. This conflates
|
|
||||||
effective ownership with historical definitions. For example, a root containing
|
|
||||||
`audita: {config_path: original.yml}` and an overlay containing
|
|
||||||
`audita: {config_path: replacement.yml}` produce `audita.config_path` records
|
|
||||||
for the root, overlay, and resolution. The root's superseded value did not
|
|
||||||
supply the effective resource; its record is indistinguishable from an effective
|
|
||||||
contributor. The error is visible to operators using `config sources` to locate
|
|
||||||
the definition that controls an integration path.
|
|
||||||
|
|
||||||
Required change: use native effective/pre-normalization evidence for effective
|
|
||||||
input attribution. If historical definitions are also displayed, label them
|
|
||||||
separately. Preserve historical contributor information in the artifact catalog
|
|
||||||
where it is explicitly definition evidence. Keep the projection thin; another
|
|
||||||
provenance reconstruction service is unnecessary.
|
|
||||||
|
|
||||||
Verification: cover a normalized resource with a superseded root/import value,
|
|
||||||
an overlay replacement, and inheritance. The effective overlay and resolution
|
|
||||||
remain visible, while superseded evidence is either absent from current-owner
|
|
||||||
records or explicitly identified as history. Update the command reference for
|
|
||||||
the resulting roles.
|
|
||||||
|
|
||||||
## 2. P2: Remove the restore test's incidental startup deadline
|
|
||||||
|
|
||||||
Status: Complete. Cancellation is synchronized after discovery with the session
|
|
||||||
lock held. Ten focused race repetitions and the full repository race suite pass.
|
|
||||||
|
|
||||||
Stage: 12; the fragile timing assumption predates this migration.
|
|
||||||
|
|
||||||
Location: `internal/app/restore_execution_test.go:383–387`,
|
|
||||||
`TestExecuteRestoreLockConflictFailsAndWritesNothing`.
|
|
||||||
|
|
||||||
The test starts a 100 ms context deadline before calling the complete restore
|
|
||||||
command, then requires an error from session-lock acquisition. With the current
|
|
||||||
configuration workflow under the full race suite, the deadline can expire
|
|
||||||
during earlier work. The reproduced failure was:
|
|
||||||
|
|
||||||
```text
|
|
||||||
restore: remote check current commit pointer "dnd/campaigns/sample-campaign/sessions/2026-05-03/current/commit-pointer.json": context deadline exceeded, want lock failure
|
|
||||||
```
|
|
||||||
|
|
||||||
An isolated application-suite run passed, while a subsequent full race run
|
|
||||||
failed at this assertion. A green isolated run therefore does not establish
|
|
||||||
Stage 12's repeatable final validation. The test implicitly requires unrelated
|
|
||||||
startup work to finish within 100 ms, which is not an application requirement.
|
|
||||||
This is not evidence that production session locking is incorrect.
|
|
||||||
|
|
||||||
Required change: make the contention/cancellation test target the intended
|
|
||||||
lock boundary without timing all preceding configuration and remote discovery.
|
|
||||||
Use synchronization at an appropriate existing test boundary, or separate
|
|
||||||
lock-service contention coverage from command-level no-write coverage. A larger
|
|
||||||
arbitrary startup timeout alone leaves the underlying timing dependence.
|
|
||||||
Preserve the assertions that restore cannot write protected files while another
|
|
||||||
invocation owns the session lock.
|
|
||||||
|
|
||||||
Verification: repeat the focused test and rerun the complete race suite under
|
|
||||||
normal package concurrency. Report actual outcomes rather than treating an
|
|
||||||
isolated pass as full-suite success.
|
|
||||||
|
|
||||||
## 3. P3: Finish the planned field-level publish resolution cleanup
|
|
||||||
|
|
||||||
Status: Complete. Field-only defaults and destinations retain precise native
|
|
||||||
history. Publish and authored append/upsert/replace regression checks pass.
|
|
||||||
|
|
||||||
Stages: 1 and 3; the roadmap's integration scope requires field-level recording
|
|
||||||
for field-only changes.
|
|
||||||
|
|
||||||
Location: `internal/config/pipeline_configkit.go:202–215,232–235`,
|
|
||||||
`resolvePipelineConfig`.
|
|
||||||
|
|
||||||
Publish requiredness and derived destinations already use field-level native
|
|
||||||
helpers. The resolver additionally tracks `defaultedOutput`, records the entire
|
|
||||||
`publish.outputs` list for those field edits, and retains a comment claiming
|
|
||||||
that child records cannot cover the changes. ConfigKit v1.2.1 supports exactly
|
|
||||||
this field-level coverage. The ancestor record adds redundant history: querying
|
|
||||||
an untouched output source also returns a whole-list normalization operation
|
|
||||||
with publish-default/destination reasons.
|
|
||||||
|
|
||||||
Required change: remove the obsolete list-coverage tracking, redundant ancestor
|
|
||||||
records, and inaccurate comment for these field-only edits. Keep field helpers
|
|
||||||
and any whole-list records actually needed for structural membership/order
|
|
||||||
changes. This is completion of the selected simplification, not a new ban on
|
|
||||||
container-level records.
|
|
||||||
|
|
||||||
Verification: a temporary diagnostic overlay removing only the obsolete
|
|
||||||
workarounds successfully loaded defaulted/derived publish outputs and passed
|
|
||||||
the append/upsert/replace collection tests. Add suitable behavioral coverage
|
|
||||||
for effective values and precise field history, without asserting private
|
|
||||||
helper invocation counts.
|
|
||||||
|
|
||||||
## 4. P3: Pin Go attribution links to the selected source commit
|
|
||||||
|
|
||||||
Status: Complete. Go license, patent, and source links identify the verified
|
|
||||||
Go 1.27.4 source commit; documentation checks pass.
|
|
||||||
|
|
||||||
Stages: 1 and 11.
|
|
||||||
|
|
||||||
Location: `docs/oss/attribution.md:125–128`.
|
|
||||||
|
|
||||||
The updated Go entry uses `go1.27.4` in the LICENSE, PATENTS, and Source URLs.
|
|
||||||
The adopted OSS policy requires commit-specific upstream source and notice
|
|
||||||
links. The preceding entry used a source hash; this update replaced it with
|
|
||||||
tag-based references.
|
|
||||||
|
|
||||||
Required change: use the verified Go 1.27.4 source commit
|
|
||||||
`7b2881024c95a11ddb86663a616340aa3a88b67f` in those three URLs. The hash was
|
|
||||||
resolved through the official Go repository's release tag. Keep the existing
|
|
||||||
license texts, factual notice wording, and Go version identification.
|
|
||||||
|
|
||||||
Verification: confirm the links identify that source revision and run the
|
|
||||||
documentation checks. This requires no licensing change or new release gate.
|
|
||||||
|
|
||||||
## 5. P2: Give session initialization the correct authoring context
|
|
||||||
|
|
||||||
Status: Complete. Rendered templates load with native local context and imports;
|
|
||||||
local and remote products are standalone effective documents. Relative CLI audio,
|
|
||||||
import removal, placeholder/identity, and initialization race checks pass.
|
|
||||||
|
|
||||||
Stage: 4, interacting with materialization and command validation in Stages
|
|
||||||
6, 11, and 12.
|
|
||||||
|
|
||||||
Locations: `internal/app/operator_session_init.go:78–102` and
|
|
||||||
`internal/config/load.go`, `LoadSessionBytesWithOptions`.
|
|
||||||
|
|
||||||
The existing initializer validates its rendered document through
|
|
||||||
`LoadSessionBytesWithOptions`. That loader now deliberately describes a
|
|
||||||
contextless remote snapshot with imports disabled. A generated local session
|
|
||||||
is an authored file, so this caller no longer uses the same semantics as the
|
|
||||||
ordinary file loader.
|
|
||||||
|
|
||||||
Two focused reproductions show the mismatch:
|
|
||||||
|
|
||||||
- A local template with a required `shared.yml` import containing constant
|
|
||||||
metadata fails with `import.disabled`, even when the import exists beside
|
|
||||||
the generated destination and the resulting local session is otherwise valid.
|
|
||||||
- With `--audio-dir ./audio` and an output in another directory, initialization
|
|
||||||
validates the invocation-directory audio path, then writes the original
|
|
||||||
relative text. Loading that generated file selects the output-directory
|
|
||||||
audio path. The persisted resource differs from the validated resource and
|
|
||||||
the documented invocation-relative CLI interpretation.
|
|
||||||
|
|
||||||
Required change: distinguish authored initialization from contextless snapshot
|
|
||||||
loading. Use ConfigKit's supported source/context facilities for local
|
|
||||||
composition, and make persisted resource references reproduce the validated
|
|
||||||
configuration. Materialize effective values where moving or uploading the
|
|
||||||
document would leave authoring imports unresolved. Preserve the narrow template
|
|
||||||
substitution contract and the existing contextless remote-loading boundary.
|
|
||||||
|
|
||||||
Verification: initialize a local imported template, load its output through the
|
|
||||||
ordinary loader, and check the imported values. Cover a destination outside the
|
|
||||||
invocation directory and relative CLI audio input. Check that remote generation
|
|
||||||
emits a consumable standalone snapshot and that ordinary placeholder and identity
|
|
||||||
validation still applies. These cases exercise initialization through the real
|
|
||||||
configuration owner without adding remote companion retrieval.
|
|
||||||
|
|
||||||
## 6. P2: Reconcile remote session updates with committed-session selection
|
|
||||||
|
|
||||||
Status: Complete. Authored remote input takes precedence for later commands;
|
|
||||||
committed snapshots remain immutable fallback input and restore authority.
|
|
||||||
Publication/update regression coverage checks changed title and audio selection.
|
|
||||||
|
|
||||||
Stage: 7, interacting with the existing initializer and shared command loader
|
|
||||||
from Stages 2 and 4.
|
|
||||||
|
|
||||||
Locations: `internal/app/config_loader.go:194–223` and
|
|
||||||
`internal/app/operator_session_init.go:113–138`.
|
|
||||||
|
|
||||||
The new remote loader selects a committed `run:prepare/inputs/session.yml`
|
|
||||||
whenever the current commit declares one. The existing `session init --remote`
|
|
||||||
writer continues to update the session-root `session.yml`. After publication,
|
|
||||||
the writer and reader therefore use different authoritative objects.
|
|
||||||
|
|
||||||
A prepare → publish → `session init --remote --force --title "After publication"`
|
|
||||||
reproduction succeeds and writes the requested title to the root object. A
|
|
||||||
subsequent ordinary command load still returns the old title from the committed
|
|
||||||
prepared session. The same selection would ignore changed audio or previous-session
|
|
||||||
settings. The documented overwrite workflow silently stops affecting later runs
|
|
||||||
once the session has been published.
|
|
||||||
|
|
||||||
Required change: reconcile remote input authoring and selection so successful
|
|
||||||
operator updates can be used by subsequent runs. Preserve immutable published
|
|
||||||
snapshots and committed restore authority. Keep an effective remote editing or
|
|
||||||
override workflow; silently ignoring updates or introducing a blanket ban on
|
|
||||||
editing published sessions would leave the integration unresolved. Update the
|
|
||||||
canonical operator guidance for the resulting selection behavior.
|
|
||||||
|
|
||||||
Verification: publish a prepared session, update it through the supported remote
|
|
||||||
workflow, then load it through the ordinary run/plan configuration path and check
|
|
||||||
the updated values. Include audio or previous-session selection as well as title.
|
|
||||||
Verify that restore still uses its selected immutable commit and that published
|
|
||||||
objects are unchanged by the authoring update.
|
|
||||||
|
|
||||||
## 7. P3: Reuse enumerated source records and one inspection source index
|
|
||||||
|
|
||||||
Status: Complete. Source enumeration uses complete native records directly;
|
|
||||||
catalog projection shares one index and retained family sources. Nested empty
|
|
||||||
imports and existing composite/family inspection checks pass.
|
|
||||||
|
|
||||||
Stage: 9.
|
|
||||||
|
|
||||||
Locations: `internal/config/effective_pipeline.go:57–75`,
|
|
||||||
`internal/config/pipeline_configkit.go:354–371`, and
|
|
||||||
`internal/config/artifact_catalog.go`, `declarationDefinitionSources`.
|
|
||||||
|
|
||||||
`emptyImportSelections` enumerates `Result.Sources()` and then calls
|
|
||||||
`Result.Source(id)` for every enumerated record. `pipelineSourcePaths` does the
|
|
||||||
same lookup for every retained source. In ConfigKit v1.2.1, completed-result
|
|
||||||
source lookup scans the source slice. Enumerating S sources this way performs
|
|
||||||
Θ(S²) source-ID comparisons, although the enumerated records already contain
|
|
||||||
the required information, including empty-import evidence.
|
|
||||||
|
|
||||||
The artifact catalog additionally rebuilds this source map for each declaration.
|
|
||||||
For A entries this introduces O(A S²) source-index work; with one imported file
|
|
||||||
per artifact, that component can become cubic in A. This is separate from the
|
|
||||||
native history queries and necessary output construction.
|
|
||||||
|
|
||||||
Potential refactoring: consume the complete enumerated records directly. Build
|
|
||||||
the source-ID/path map once per inspection operation and share it with declaration
|
|
||||||
projection. This reduces empty-import enumeration to O(S + E), where E is the
|
|
||||||
returned selection evidence, and source-index work across the catalog to
|
|
||||||
expected O(S + A). Expanded members can also reuse the family-source lists
|
|
||||||
already retained in the family catalog. Keep native history and origin queries
|
|
||||||
as the provenance authority.
|
|
||||||
|
|
||||||
Verification: temporary functional checks confirmed that enumerated records and
|
|
||||||
individual lookups have equal contents, and that direct enumeration preserves
|
|
||||||
source paths and nested empty-import selections. Preserve operator source/catalog
|
|
||||||
output, including composite and inherited contributors, when refactoring.
|
|
||||||
|
|
||||||
## 8. P3: Reuse party metadata and per-character provenance within inspection
|
|
||||||
|
|
||||||
Status: Complete. Each source/catalog projection shares one native metadata
|
|
||||||
snapshot, one source index, and a per-character cache. Imported-party, composite
|
|
||||||
provenance, family, and command agreement tests pass under the race detector.
|
|
||||||
|
|
||||||
Stage: 9, consuming Stage 5's imported party result.
|
|
||||||
|
|
||||||
Locations: `internal/config/artifact_catalog.go:67–105` and
|
|
||||||
`internal/config/effective_pipeline.go`, `appendGeneratedArtifactSources`.
|
|
||||||
|
|
||||||
`partyCharacterSources` calls `result.Metadata()` separately for Lookup and
|
|
||||||
History. ConfigKit v1.2.1 returns detached metadata by copying the complete
|
|
||||||
effective-path and field maps, including their origin records. This helper is
|
|
||||||
called for each generated artifact in the catalog and for each generated
|
|
||||||
artifact field in source inspection. The same character's provenance is
|
|
||||||
therefore repeatedly recomputed across fields and families.
|
|
||||||
|
|
||||||
Let M be the copied party-metadata size and P the number of provenance requests.
|
|
||||||
The avoidable metadata-copy work and cumulative allocation are Θ(P M). Even
|
|
||||||
with one family and bounded-size character entries, M and P both grow with the
|
|
||||||
character count C, producing quadratic work. More families and fields multiply
|
|
||||||
the requests. This describes the provenance pass, not the complexity of the
|
|
||||||
whole command or unchanged party validation.
|
|
||||||
|
|
||||||
Potential refactoring: obtain one native Metadata snapshot and source index per
|
|
||||||
inspection operation. Resolve each requested character's source list once and
|
|
||||||
reuse it for that operation. The full-metadata copy term becomes O(M), with
|
|
||||||
native per-character query costs and required output emission remaining. Keep
|
|
||||||
the cache local to the projection; a new persistent provenance service is
|
|
||||||
unnecessary. Apply the same snapshot reuse to campaign provenance helpers where
|
|
||||||
convenient, although their fixed field count is not a quadratic case.
|
|
||||||
|
|
||||||
Verification: a temporary imported/inherited-roster check confirmed equivalent
|
|
||||||
source lists using shared native metadata and an enumerated source index.
|
|
||||||
Retain contributor accuracy, ordering, and secret-free inspection output.
|
|
||||||
|
|
||||||
## 9. P3: Read the selected remote session without enumerating run history
|
|
||||||
|
|
||||||
Status: Complete. Remote loading opens the selected object once, verifies its
|
|
||||||
returned identity, and streams into a command-owned temporary file. Remote and
|
|
||||||
publication round trips pass under race; a rejecting-list fake checks enumeration
|
|
||||||
is unnecessary, and cleanup remains idempotent.
|
|
||||||
|
|
||||||
Stage: 7.
|
|
||||||
|
|
||||||
Location: `internal/app/config_loader.go:194–228`, `findRemoteSessionConfig`.
|
|
||||||
|
|
||||||
After selecting the committed prepared-session key, this function lists the
|
|
||||||
entire session prefix and scans it for that exact key. If H objects are retained
|
|
||||||
across historical runs, this adds Θ(H) listing/scan work and Θ(H) object-record
|
|
||||||
memory to every remote session load. The production S3 adapter paginates and
|
|
||||||
accumulates the complete listing. The scan predates the migration and was
|
|
||||||
retained in the new committed-object selection path.
|
|
||||||
|
|
||||||
Potential refactoring: retrieve the exact selected key through the existing
|
|
||||||
`ObjectStore.Read` boundary, which returns metadata with the opened body. Use
|
|
||||||
that metadata for the existing size/generation checks and stream the body into
|
|
||||||
the command-owned temporary file while retaining checksum verification. The
|
|
||||||
object lookup/transfer then takes a constant number of requests independent of
|
|
||||||
H, plus the necessary O(B) work for B session bytes. Current-commit acquisition
|
|
||||||
and validation costs remain. Apply this after reconciling the selection policy
|
|
||||||
in item 6; no new backend interface is needed.
|
|
||||||
|
|
||||||
Verification: the production S3 Read implementation performs one exact-key
|
|
||||||
GetObject and returns its size and generation. Preserve fake-store coverage for
|
|
||||||
missing objects, selected identity, checksum/size/generation mismatches, and
|
|
||||||
temporary-file cleanup. Confirm that unrelated historical keys are not enumerated.
|
|
||||||
|
|
||||||
## 10. P3: Stream prepared-input archival copies
|
|
||||||
|
|
||||||
Status: Complete. Verified descriptors stream through the existing atomic-reader
|
|
||||||
copy helper, with unchanged-destination reuse. Archive bytes, checksums, modes,
|
|
||||||
and tamper rejection are covered; full ordinary and race suites pass.
|
|
||||||
|
|
||||||
Stage: 7.
|
|
||||||
|
|
||||||
Location: `internal/stage/publish_sources.go:29–64`,
|
|
||||||
`collectPublishPreparedInputs`.
|
|
||||||
|
|
||||||
After opening and verifying each prepared document, the new archival path reads
|
|
||||||
the whole file with `io.ReadAll` and passes the byte slice to
|
|
||||||
`writeBytesIfChanged`. For a document of B bytes this requires Θ(B) auxiliary
|
|
||||||
transfer memory. Expanded imported party documents can be larger than any one
|
|
||||||
authoring fragment. A full in-memory document is unnecessary for this file copy.
|
|
||||||
|
|
||||||
Potential refactoring: preserve the current confined source opening and checksum
|
|
||||||
checks, then stream the opened descriptor into the run archive. The existing
|
|
||||||
`fileops.WriteReaderAtomicWithChecksum` already supplies reader-based atomic
|
|
||||||
replacement and hashing through `io.Copy`. Preserve unchanged-destination reuse
|
|
||||||
using the known checksum. Transfer time remains Θ(B), while auxiliary copy
|
|
||||||
memory becomes O(1) with a fixed-size buffer. This does not alter the plan's
|
|
||||||
separate canonical YAML rendering algorithm or total in-memory configuration size.
|
|
||||||
|
|
||||||
Verification: preserve archived bytes/checksums, workspace file modes, source
|
|
||||||
validation, failure handling, and immutable publication behavior. Existing
|
|
||||||
reader-copy and prepared-session publication tests provide suitable behavioral
|
|
||||||
coverage; allocation or wall-clock thresholds are unnecessary.
|
|
||||||
|
|
||||||
## Stage-by-stage disposition at the initial reviews
|
|
||||||
|
|
||||||
| Stage | Review result |
|
|
||||||
| --- | --- |
|
|
||||||
| 1: Dependency/toolchain | ConfigKit v1.2.1 and Go 1.27.0/1.27.4 selected; remaining notice pinning in item 4. |
|
|
||||||
| 2: Discovery/bootstrap | Native discovery, exclusive explicit selection, independent bootstrap, and shared diff bootstrap are present. |
|
|
||||||
| 3: Resource bases | Native origin-based resolution and fallbacks are present; field-record cleanup remains in item 3. |
|
|
||||||
| 4: Campaign/session imports | File contexts, contextless remote snapshots, native optional discovery, and detached document values are present; initialization still uses the snapshot path in item 5. |
|
|
||||||
| 5: Party imports | Imported rosters, source-aware ordering, canonical rendering, players, and family resolution are present; the planned roster sort is retained. |
|
|
||||||
| 6: Prepared inputs/resume | Canonical document materialization and matching resume evidence are present. |
|
|
||||||
| 7: Publish/restore | Prepared documents flow through immutable committed run objects and restore; item 6 is a correctness gap and items 9–10 are scalability opportunities. |
|
|
||||||
| 8: Final-domain snapshots | Defaults-backed native snapshots, safe credential identities, rendering, and digest integration are present. |
|
|
||||||
| 9: Diff/provenance | Native diff and pointer paths are present; item 1 concerns attribution correctness and items 7–8 concern avoidable provenance work. |
|
|
||||||
| 10: Examples | Destination-based definitions, directory imports, list operations, split roster, and secondary imports are demonstrated. |
|
|
||||||
| 11: Documentation/policy | Current references cover the migration; finish attribution, source-role, and initialization/selection corrections above. |
|
|
||||||
| 12: Final verification | Existing integration checks pass; add coverage for items 5–6. Full race validation remains unreliable because of item 2. |
|
|
||||||
|
|
||||||
The approved discovery, bootstrap, relative-resource, and secondary-import
|
|
||||||
changes are deliberate contract changes. Items 5–6 describe implementation
|
|
||||||
inconsistencies in existing workflows. Item 2 concerns an incidental test-time
|
|
||||||
performance assumption, not a requested startup guarantee.
|
|
||||||
|
|
||||||
## Initial review validation
|
|
||||||
|
|
||||||
Using Go 1.27.4:
|
|
||||||
|
|
||||||
- `go test ./...` passed.
|
|
||||||
- `go test -race ./...` failed at item 2. The isolated application race suite
|
|
||||||
passed, as did five subsequent repetitions of the focused restore test.
|
|
||||||
- `go vet ./...`, `go build ./...`, and `go mod tidy -diff` passed.
|
|
||||||
- Release-asset cross-builds passed for Linux amd64, Linux arm64, and macOS
|
|
||||||
arm64 with the existing `v0.0.0-test` label and temporary output directory.
|
|
||||||
- Focused diagnostic reproductions confirmed items 1 and 3. Temporary Go
|
|
||||||
overlays kept those diagnostics and proposed-code experiments outside the
|
|
||||||
repository.
|
|
||||||
- Documentation checks and `git diff --check` passed.
|
|
||||||
|
|
||||||
Review execution made no application changes and invoked no live subprocess
|
|
||||||
services or release publication.
|
|
||||||
|
|
||||||
## Integration follow-up validation
|
|
||||||
|
|
||||||
Using Go 1.27.4 at the follow-up revision:
|
|
||||||
|
|
||||||
- `go test ./...` passed with local test-server sockets permitted. The sandboxed
|
|
||||||
attempt was blocked by `httptest` socket restrictions.
|
|
||||||
- Focused migration checks across config, stage, and app packages passed both
|
|
||||||
ordinarily and under `-race`. These exercised native discovery/inspection,
|
|
||||||
imported inputs, canonical materialization, published-session round trips,
|
|
||||||
initialization, and command/execution agreement.
|
|
||||||
- Temporary overlay diagnostics reproduced both aspects of item 5 and the
|
|
||||||
publication/update conflict in item 6, ordinarily and under `-race`.
|
|
||||||
- The full race suite was not repeated in this pass; item 2 retains the initial
|
|
||||||
review's observed failure. Application code is unchanged.
|
|
||||||
|
|
||||||
Only this remediation document changed. Diagnostic files remained outside the
|
|
||||||
repository, and object storage used an in-memory fake.
|
|
||||||
|
|
||||||
## Algorithm-focused review coverage and validation
|
|
||||||
|
|
||||||
The stage-by-stage pass covered dependency/API substitutions (1), native
|
|
||||||
discovery/bootstrap (2), resource-base queries (3), secondary loading and
|
|
||||||
materialization data flow (4), roster ordering/rendering (5), prepare/resume
|
|
||||||
serialization (6), archive/remote/restore paths (7), finalized snapshots (8),
|
|
||||||
diff/provenance projection (9), examples (10), documentation (11), and assembled
|
|
||||||
verification (12).
|
|
||||||
|
|
||||||
The chosen roster ordering, defaults-backed snapshot construction, native Diff,
|
|
||||||
and required content validation remain accepted. Canonical rendering is linear
|
|
||||||
in emitted data; the planned roster comparison sort is O(C log C) after deriving
|
|
||||||
ordering keys. Unchanged algorithms such as pairwise party-name validation and
|
|
||||||
prepare-input matching are not new findings from this migration-focused pass.
|
|
||||||
|
|
||||||
Cost analysis followed the released module's public API wrappers through native
|
|
||||||
source lookup and metadata-copy implementations. Temporary functional Go-overlay
|
|
||||||
checks validated source-enumeration and shared-metadata equivalence, ordinarily
|
|
||||||
and under `-race`. Focused configuration, file-reader copy, S3 opened-object
|
|
||||||
metadata, native inspection, and prepared-session publication checks passed on
|
|
||||||
Go 1.27.4. No performance data, timing assertions, benchmark gates, or application
|
|
||||||
changes were introduced.
|
|
||||||
|
|
||||||
## Completed remediation validation
|
|
||||||
|
|
||||||
Each issue was addressed in order and committed with its completion marker.
|
|
||||||
Using Go 1.27.4 on the completed implementation:
|
|
||||||
|
|
||||||
- `go test ./...` passed.
|
|
||||||
- `go test -race ./...` passed, including the corrected restore contention test.
|
|
||||||
- `go vet ./...`, `go build ./...`, and `go mod tidy -diff` passed.
|
|
||||||
- Focused regressions cover effective provenance, field-level publish history,
|
|
||||||
local/imported initialization, standalone remote input, remote authoring after
|
|
||||||
publication, enumeration-free loading, and verified streaming archives.
|
|
||||||
- Documentation checks and `git diff --check` passed.
|
|
||||||
|
|
||||||
Remote session authoring updates affect future command inputs. Committed
|
|
||||||
publication objects remain immutable and restore continues to validate its
|
|
||||||
pointer-selected remote state. Inspection caches are scoped to one operation;
|
|
||||||
ConfigKit remains the authority for configuration and provenance.
|
|
||||||
Reference in New Issue
Block a user