From 8be65fd3a14dd854c87a9d7c96935481868b2a44 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Fri, 20 Nov 2026 12:54:56 +0000 Subject: [PATCH] Record CommandKit migration review findings --- docs/roadmap/remediation.md | 176 ++++++++++++++++++++++++++++++++++++ 1 file changed, 176 insertions(+) create mode 100644 docs/roadmap/remediation.md diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md new file mode 100644 index 0000000..dfd1886 --- /dev/null +++ b/docs/roadmap/remediation.md @@ -0,0 +1,176 @@ +# CommandKit migration review + +## Scope + +Reviewed all 21 stages of [implementation.md](implementation.md) against the +[approved roadmap](commandkit.md), the implementation from `86692d5` through +`0117f3f35d73b3025a0222d25b6924ef455592e1`, current source and documentation, +behavioral tests, and focused diagnostic reproductions. Completion annotations +were treated as claims to verify rather than proof of completion. + +The native command boundary, actual ConfigKit inspection results, separate domain +catalogs, independent selections, reporting, and cancellation architecture are +substantially implemented. The following three P2 findings require remediation. +This review makes no application changes. + +## 1. P2: Preserve campaign inheritance when materializing session documents + +Stages: 3, 6, and 13; preservation checks in Stage 21. + +Locations: `internal/config/load.go:254–255`, +`internal/config/document_render.go:52–62`, and +`internal/app/prepared_session_roundtrip_test.go:88`. + +`SessionDocument` now points to the completed session, which already contains +campaign defaults. `RenderSessionDocument` writes those inherited stable input +paths as explicit session fields. Reloading generated session YAML or the +published prepared-session fallback therefore replaces inheritance with fixed +paths from the producing campaign. The optional spell-catalog default is also +written whenever nonempty. + +Reproduction: load a session whose authored inputs contain only audio; render its +prepared session; change the selected campaign's `speakers_file`; reload the +prepared bytes with that campaign. The received session still selects the old +speaker file. Its native metadata now calls this a session override. The +round-trip test was changed to expect `session_config` ownership for the formerly +inherited speaker path, so it accepts the regression rather than protecting the +prior behavior. + +The [configuration contract](../config.md#pipeline-imports-and-overlays) says +omitted session fields remain eligible to inherit the receiving campaign's +values. Preserving materialization/authoring semantics was part of the plan. +Freezing inherited resource paths is an additional semantic commitment, not a +requirement of completing defaults before the ConfigKit handoff. It can select +obsolete resources after campaign updates and retain paths unavailable on a +receiving host. + +Required remediation: + +- Keep the actual completed ConfigKit result authoritative for inspection and + execution. Use native explicitness metadata when serializing session-owned + fields so inherited campaign input defaults remain omitted in materialized + session YAML. Retain explicit session overrides and explicit optional-input + clearing. Preserve import flattening and source-relative resolution. +- Apply the distinction to preparation, local/remote initialization, and + committed-session fallback through their shared renderer. Do not introduce a + replacement configuration result or reparse authoring files for inspection. +- Restore coverage that materializes an omitted default, changes the receiving + campaign, and observes the new default. Include an explicit override and an + explicit empty optional spell catalog to establish the difference. Update the + existing round-trip expectation rather than preserving the newly frozen + behavior as a contract. + +## 2. P2: Classify identifiable Narratio input errors consistently + +Stages: 1, 7–10, and 12; native outcome verification in Stage 21. + +Locations: `internal/app/configuration_commands.go:75–96`, +`internal/app/configuration_acquisition.go:294–297`, and +`internal/app/command_outcome.go:30–55`. + +Two integration paths return operational status 1 for identifiable input errors: + +1. Native configuration acquisition/comparison returns + `config.SessionIdentityError` for a supplied campaign/session identity that + differs from the selected file. The scope wrapper adds explanatory text but + passes the same error to ckconfig, whose classifier does not know Narratio's + type. For example, `config show --scope campaign --campaign-file FILE + --campaign WRONG` returns status 1. Session assertions and comparison-side + assertions have the same problem. The application outcome adapter recognizes + the type for domain commands, but native config commands do not use it. +2. Catalog derivation returns plain errors for invalid configuration relations. + A configured artifact with `depends_on: [missing]` reaches domain derivation, + then `artifacts list` and `run --stage analyze` return status 1. The error + unambiguously identifies invalid supplied configuration, not a service failure. + Such relational validation belongs in domain derivation; moving family + expansion into native app inspection is not the remedy. + +The roadmap and [CLI outcome contract](../cli.md#cancellation-and-exit-status) +reserve status 2 for invalid invocation or identifiable configuration input. +Current identity tests check only a nonzero status and explanatory text, leaving +this integration mismatch undetected. + +Required remediation: + +- Preserve the agreed status 2 classification for post-resolution identity + assertions in native config acquisition and comparison, including the right + side. Use genuine native resolution/validation or a supported classification + integration point while preserving the underlying typed cause. If the released + API lacks the necessary integration point, address that narrow capability + upstream rather than forking native commands or matching error strings. +- Give Narratio-owned invalid domain-configuration relations typed input + classification at their existing boundary. Keep cancellation, storage/access + failures, shutdown/persistence errors, and mixed operational causes at status 1. + Do not classify every acquisition or catalog error as user input indiscriminately. +- Add native invocation regressions asserting exact status classification for + campaign/session mismatch, comparison-side mismatch, and an unknown artifact + dependency, alongside an operational or joined-cause counterexample. + +## 3. P2: Correct the documented empty spell-catalog behavior + +Stage: 20. + +Location: `docs/config.md:706`, the `inputs.spell_catalog_file` session field row. + +The field table says an empty or omitted value inherits the campaign value. +The implemented native-default behavior distinguishes them: omission inherits; +an explicit empty string clears the optional input. This behavior is exercised +by `TestSessionDefaultsCompleteBeforeNativeHandoff` and is consistent with the +agreed explicit-presence rules. The table contradicts both implementation and +the reference's general defaults section. + +Required remediation: update the field row and any equivalent current guidance +to state the implemented distinction. Retain optional-input clearing behavior; +do not change it to satisfy the obsolete wording. This is a documentation-only +correction, separate from preserving inherited versus explicit fields during +materialization in finding 1. + +## Stage-by-stage coverage + +| Stage | Evidence reviewed and result | +| --- | --- | +| 1 | Released module selection, native application/process boundary, lazy construction, and outcome adapter; finding 2 covers classification gaps. | +| 2 | Retained pipeline/campaign results, lifecycle normalization/validation, native metadata, and absence of replacement inspection results. | +| 3 | Campaign defaults, explicit presence, session assertions, and resource bases; findings 1 and 2 cover materialization and native assertion outcomes. | +| 4 | Selected-party results/players, pure family expansion, collision/dependency checks, and separate publication rules. | +| 5 | Concrete-catalog consumers in planning, analysis, semantic fingerprints, adapter composition, and runtime registration. | +| 6 | Preparation, publication, restore round trips, and semantic evidence; finding 1 covers inherited input serialization. | +| 7 | Exact-file/local/remote acquisition, source labels, one-time bootstrap, retained errors, and resource cleanup; finding 2 covers typed input classification. | +| 8 | Four native scopes, path/history/findings capabilities, aggregate behavior, and lazy acquisition. | +| 9 | Independent comparison inputs/defaults, app-only overrides, native difference reports and status 3; finding 2 covers assertion failures. | +| 10 | Native bounded run/regeneration wiring, force and artifact selection, and shared execution; finding 2 covers invalid domain configuration. | +| 11 | Shared read-only planning, publication actions, and configured artifact catalog reporting. | +| 12 | Structured validation/status/artifact inspection, partial findings, local/remote evidence, and read-only behavior. | +| 13 | Initialization targets, authoring contexts, generated YAML, explicit lock commands, and conditional lock mutations; finding 1 covers inherited defaults. | +| 14 | Restore and cleanup declarations, dry-run behavior, exact action reports, confinement, conflicts, and partial failure handling. | +| 15 | Failed-attempt accounting, partial run results, checkpoint evidence, terminal persistence, cause retention, and lock release. | +| 16 | Interrupt/termination wiring, context propagation, worker settlement, detached terminal local persistence, and resumability. | +| 17 | Narratio-owned reporting seam, native visibility/channels, stage progress, and isolation from downstream debug capture. | +| 18 | Audio transfers/cache reuse, speaker/artifact durable completions, checkpoint reuse, and progress-only integration tests. | +| 19 | Native completion/static/path hints, lazy discovery tests, and retirement of handwritten routing/parsing/rendering. | +| 20 | Canonical references, workflows, migration table, attribution, supported platforms, and preserved release history; findings 1 and 3 cover remaining documentation divergence. | +| 21 | Full test/race suites, vet/build, assembled workflows, focused review reproductions, and remaining verification recorded below. | + +No other actionable requirement strengthening was identified. The removed CLI +aliases, native flag conventions, inspection-only overrides, domain/native-view +separation, and deferred workspace/downstream changes follow the agreed scope. + +## Review validation + +Using official Go 1.27.4 against the reviewed application source: + +- `go test ./...` passed. +- `go test -race ./...` passed. +- `go vet ./...` and `go build ./...` passed. +- Focused Go-overlay diagnostics reproduced both cases in finding 2 and the + campaign-default freezing in finding 1. Diagnostic files remained under + `/tmp`; application source and tests were unchanged. +- `go mod tidy -diff` passed. +- The nonpublishing asset builder passed with `v0.0.0-test`, producing Linux + amd64/arm64 and macOS arm64 binaries under `/tmp` and verifying the host + binary's embedded version. macOS evidence is compilation-only. +- Documentation-link checks and `git diff --check` passed with this review + document present. + +Only this remediation document changes the repository. No release, tag, push, +CI polling, or live downstream service invocation was performed.