Files
narratio/docs/roadmap/remediation.md

31 KiB
Raw Blame History

CommandKit migration review

Scope

Reviewed all 21 stages of implementation.md against the approved roadmap, the implementation from 86692d5 through 0117f3f35d73b3025a0222d25b6924ef455592e1, current source and documentation, behavioral tests, and focused diagnostic reproductions. Completion annotations were treated as claims to verify rather than proof of completion.

The native command boundary, actual ConfigKit inspection results, separate domain catalogs, independent selections, reporting, and cancellation architecture are substantially implemented. The initial pass identified findings 1–3. An integration-focused follow-up at 8be65fd adds finding 4; the application source is unchanged from the initial review. An algorithm-focused follow-up at 65d24fb adds findings 5–8: two P2 findings and two P3 optimization opportunities. Remediation status is recorded under each finding. The review passes made no application changes.

1. P2: Preserve campaign inheritance when materializing session documents

Status: Completed. The shared renderer uses one native explicitness snapshot to omit inherited stable inputs and retain authored overrides and optional clearing. Regression coverage checks changed receiving-campaign defaults, imported explicit overrides, inherited optional inputs, preparation/publication fallback, and local and remote initialization. Configuration, application, and stage suites pass.

Stages: 3, 6, and 13; preservation checks in Stage 21.

Locations: internal/config/load.go:254–255, internal/config/document_render.go:52–62, and internal/app/prepared_session_roundtrip_test.go:88.

SessionDocument now points to the completed session, which already contains campaign defaults. RenderSessionDocument writes those inherited stable input paths as explicit session fields. Reloading generated session YAML or the published prepared-session fallback therefore replaces inheritance with fixed paths from the producing campaign. The optional spell-catalog default is also written whenever nonempty.

Reproduction: load a session whose authored inputs contain only audio; render its prepared session; change the selected campaign's speakers_file; reload the prepared bytes with that campaign. The received session still selects the old speaker file. Its native metadata now calls this a session override. The round-trip test was changed to expect session_config ownership for the formerly inherited speaker path, so it accepts the regression rather than protecting the prior behavior.

The configuration contract says omitted session fields remain eligible to inherit the receiving campaign's values. Preserving materialization/authoring semantics was part of the plan. Freezing inherited resource paths is an additional semantic commitment, not a requirement of completing defaults before the ConfigKit handoff. It can select obsolete resources after campaign updates and retain paths unavailable on a receiving host.

Required remediation:

  • Keep the actual completed ConfigKit result authoritative for inspection and execution. Use native explicitness metadata when serializing session-owned fields so inherited campaign input defaults remain omitted in materialized session YAML. Retain explicit session overrides and explicit optional-input clearing. Preserve import flattening and source-relative resolution.
  • Apply the distinction to preparation, local/remote initialization, and committed-session fallback through their shared renderer. Do not introduce a replacement configuration result or reparse authoring files for inspection.
  • Restore coverage that materializes an omitted default, changes the receiving campaign, and observes the new default. Include an explicit override and an explicit empty optional spell catalog to establish the difference. Update the existing round-trip expectation rather than preserving the newly frozen behavior as a contract.

2. P2: Classify identifiable Narratio input errors consistently

Status: Domain-validation unit completed; native identity classification remains pending upstream support. Catalog derivation now identifies invalid relations with a typed input error, retaining its underlying cause. Native artifacts list and run --stage analyze regressions return status 2 for an unknown dependency; joined storage/cancellation errors retain status 1. ckconfig v0.9.0 and the current upstream checkout classify native ConfigKit load errors only and expose no consumer classification hook. An upstream extension and its adoption are still needed for campaign/session assertions in native acquisition and comparison; permission to modify that repository has been requested.

Stages: 1, 7–10, and 12; native outcome verification in Stage 21.

Locations: internal/app/configuration_commands.go:75–96, internal/app/configuration_acquisition.go:294–297, and internal/app/command_outcome.go:30–55.

Two integration paths return operational status 1 for identifiable input errors:

  1. Native configuration acquisition/comparison returns config.SessionIdentityError for a supplied campaign/session identity that differs from the selected file. The scope wrapper adds explanatory text but passes the same error to ckconfig, whose classifier does not know Narratio's type. For example, config show --scope campaign --campaign-file FILE --campaign WRONG returns status 1. Session assertions and comparison-side assertions have the same problem. The application outcome adapter recognizes the type for domain commands, but native config commands do not use it.
  2. Catalog derivation returns plain errors for invalid configuration relations. A configured artifact with depends_on: [missing] reaches domain derivation, then artifacts list and run --stage analyze return status 1. The error unambiguously identifies invalid supplied configuration, not a service failure. Such relational validation belongs in domain derivation; moving family expansion into native app inspection is not the remedy.

The roadmap and CLI outcome contract reserve status 2 for invalid invocation or identifiable configuration input. Current identity tests check only a nonzero status and explanatory text, leaving this integration mismatch undetected.

Required remediation:

  • Preserve the agreed status 2 classification for post-resolution identity assertions in native config acquisition and comparison, including the right side. Use genuine native resolution/validation or a supported classification integration point while preserving the underlying typed cause. If the released API lacks the necessary integration point, address that narrow capability upstream rather than forking native commands or matching error strings.
  • Give Narratio-owned invalid domain-configuration relations typed input classification at their existing boundary. Keep cancellation, storage/access failures, shutdown/persistence errors, and mixed operational causes at status 1. Do not classify every acquisition or catalog error as user input indiscriminately.
  • Add native invocation regressions asserting exact status classification for campaign/session mismatch, comparison-side mismatch, and an unknown artifact dependency, alongside an operational or joined-cause counterexample.

3. P2: Correct the documented empty spell-catalog behavior

Status: Completed. The session field reference distinguishes omission (inherit) from an explicit empty string (clear). Documentation checks pass; existing native-default and materialization tests cover both behaviors.

Stage: 20.

Location: docs/config.md:706, the inputs.spell_catalog_file session field row.

The field table says an empty or omitted value inherits the campaign value. The implemented native-default behavior distinguishes them: omission inherits; an explicit empty string clears the optional input. This behavior is exercised by TestSessionDefaultsCompleteBeforeNativeHandoff and is consistent with the agreed explicit-presence rules. The table contradicts both implementation and the reference's general defaults section.

Required remediation: update the field row and any equivalent current guidance to state the implemented distinction. Retain optional-input clearing behavior; do not change it to satisfy the obsolete wording. This is a documentation-only correction, separate from preserving inherited versus explicit fields during materialization in finding 1.

4. P2: Keep party-selection provenance out of prepare resume identity

Status: Completed. Prepare's versioned semantic projection excludes stable-input ownership labels; provenance remains available in source records. Native planning and execution regressions retain identical prepared checksums and downstream success for default/explicit selection of the same roster, and invalidate work for changed roster contents. Focused resume and semantic tests pass under race detection.

Stages: 4–7, 10–11, and 21.

Locations: internal/config/party_resolution.go:70–81, internal/stage/semantic_contracts_initial.go:151–171, internal/app/semantic_resume.go:44–45, and internal/app/run_control.go:78–83.

The new explicit party selector labels its source party_file; campaign-default selection labels the same roster campaign_config. Existing prepare semantic evidence hashes that source label in prepareStableInputSelection.Source. Selecting the very same file explicitly changes the semantic fingerprint even when the canonical roster, players projection, derived catalog, and all prepared file contents are unchanged.

An end-to-end native-invocation diagnostic reproduced this sequence:

  1. Run prepare successfully using the campaign-default roster and record a succeeded downstream transcribe stage.
  2. Plan the same session with --party-file pointing to that same roster. The planner reports result-affecting configuration changed; rerun the stage.
  3. Execute that bounded prepare request without force. Prepare executes again, and the previously succeeded transcription stage becomes stale. Every prepared input path/checksum pair remains identical.

The normal invalidation relation also applies to other succeeded downstream stages. A subsequent full run can therefore schedule unnecessary downstream processing solely because the operator made the source selection explicit. The selector and existing semantic-resume implementation work independently, but their integration turns provenance into result identity, contrary to the roadmap's result-affecting evidence boundary. This is separate from finding 1: the reproduction requires no serialization/reload or campaign-default change.

Required remediation:

  • Keep the selected source label in appropriate provenance and inspection records. Exclude or semantically normalize the provenance-only distinction in prepare's reuse fingerprint. Review equivalent stable-input source labels under the same result-affecting rule without weakening content validation.
  • Continue detecting actual roster/players/input changes through existing canonical payload checksums and stage/artifact evidence. Preserve the fixed downstream invalidation relation when inputs really change.
  • Add an integration regression covering native planning and execution with default versus explicit selection of the same roster, unchanged prepared bytes, and retained downstream success. Include changed roster contents as a counterexample that must invalidate affected work.

5. P2: Capture pipeline provenance once per catalog derivation

Status: Completed. One derivation-local resolver shares a native metadata snapshot and source-path index across family and authored-artifact lookups. Snapshot work is O(M) per derivation, plus existing native queries. The configuration suite and focused catalog, inheritance, overlay, and provenance tests pass; existing immutability and independent-roster coverage remains intact.

Stages: 2 and 4; catalog consumers in Stages 5, 7, and 11.

Locations: internal/config/artifact_catalog.go:49–65,127–143, internal/config/artifact_families.go:78, and internal/config/pipeline_configkit.go:291–307.

Every authored concrete-artifact provenance lookup calls cfg.ConfigKitResult().Metadata(). Family derivation calls the same accessor once per family. In the selected ConfigKit v1.2.1, this accessor calls Inspection.Metadata, which clones the complete effective-path and field-info maps. It is a detached snapshot operation, rather than a constant-time handle lookup. The immutable history model is shared; the effective metadata maps and their nested records are copied.

Let D be the number of authored artifact and family declarations, and M the size of the metadata copied by one snapshot. Snapshot acquisition alone costs O(D * M) work and cumulative allocation per catalog derivation, in addition to the actual history queries. With bounded-size declarations and M = Θ(D), this is quadratic work. Family lookups also rebuild the complete source-path index per family. The prior implementation reused its retained metadata snapshot; the repeated full-snapshot acquisition is introduced by this migration.

Refactoring opportunity:

  • Acquire one native metadata snapshot and one source-ID/path index inside a catalog derivation. Share them across authored-artifact and family provenance queries through a small local resolver or explicit helper parameters.
  • This reduces snapshot work to O(M) plus the existing per-declaration native queries and output processing. Build the source index once as well. The existing partyCharacterSourceLookup already demonstrates an invocation-local snapshot and lookup cache.
  • Retain native history, inheritance, and source semantics, with no global cache or replacement inspection result. Verify equivalent provenance for concrete definitions, families, imports, and overlays; preserve the completed ConfigKit values.

This opportunity concerns how Narratio calls the public API. It does not require changing ConfigKit's detached-snapshot contract or reviewing its selected configuration algorithms.

6. P3: Avoid cloning the complete family catalog for individual origins

Status: Completed. Resume reports, checkpoint records, and analysis diagnostics use a constant-time member lookup returning only scalar family/character identity. Origin lookup work is O(L) with no whole-catalog copies. Detached full records remain available through the existing catalog-copy API. Focused analysis, family, planning, and catalog tests pass.

Stages: 5 and 11; artifact evidence in Stage 15.

Locations: internal/stage/analyze_resume.go:88–103, internal/stage/analyze.go:179–185, internal/config/config.go:71–76, and internal/config/artifact_families.go:395–418.

exportAnalyzeResumeItems calls config.ArtifactFamilies(catalog) inside its per-item loop, then reads one member's family and character ID. That accessor deep-copies every family and member, including dependency slices and input maps. analyzeArtifactOrigin performs the same whole-catalog copy for individual artifact checkpoint and diagnostic records.

Let L be the number of origin lookups and K the total family/member catalog payload. These lookups cost O(L * K) work and cumulative allocation. When a plan reports a catalog-sized set of bounded-size members, L and K both grow with member count, producing quadratic copy work for a linear-sized report. This inefficiency predates the migration and remains in the rewired domain consumers; it is an optimization opportunity, rather than a new regression.

Refactoring opportunity:

  • Capture one detached family catalog outside a batch and reuse its member index across all three resume-summary lists. Copy work becomes O(K + L).
  • For execution's individual lookups, read the already-derived member index through a narrow accessor that returns the needed identity fields by value. If a caller needs mutable dependency/input records, copy that member only.
  • Preserve family identities, deterministic report ordering, and detached records where callers need them. A new defensive-copy framework or global cache is unnecessary.

7. P3: Build publication-report lookup tables once

Status: Completed. Publication reports build configured/extraction lookup tables and a first-declaration-per-source commit index once. Local lookup work is now O(A log A + X log X + K + R) plus copied entry payloads. Removed the unused destination-display helper. Status, remote artifact inspection, exact carried-key, partial-failure, and catalog-evidence tests pass.

Stage: 12.

Locations: internal/app/session_state_command.go:256–270, internal/app/artifact_inspection.go:79–131, and internal/artifacts/catalog.go:255–289.

For every publication rule, inspectPublishedOutputs calls helperPublishedOutputDest, which rebuilds the configured-output map and extraction-output set. Building each map first enumerates, sorts, and copies the entire corresponding runtime catalog. The same loop calls committedPublishedOutput, scanning all commit declarations to find one source.

Let R be publication-rule count, A configured-artifact count, X extraction-artifact count, and K commit-declaration count. With bounded-size catalog records, the repeated local work is O(R * (A log A + X log X + K)), plus copying any variable-sized entry payloads. When R, A, and K grow together, this includes quadratic-logarithmic sorting work and quadratic declaration scans. Required remote existence checks are separate from this CPU/allocation cost. These helpers and repeated lookups predate the migration and remain in the new structured-report collector.

Refactoring opportunity:

  • Build configured/extraction lookup tables once per report and pass them to destination resolution, or share the existing publication-action planner where its selection and failure behavior fit inspection.
  • Index published commit declarations once by source, preserving the existing declaration-selection behavior. Resolve each report row through that index.
  • Reusing the existing catalog enumeration APIs reduces this local lookup work to O(A log A + X log X + K + R), plus entry-payload copying. Keep configured report ordering, partial error reporting, pointer-selected authority, exact declared object keys, and the existing remote checks.

There is no need to add remote concurrency, caching across invocations, or a new publication protocol to obtain this improvement.

8. P2: Use canonical publication sources in session reports

Status: Completed. Inspection canonicalizes each source once for the report and all evidence lookups, preserving the authored configuration. A regression using real loading, validation, and publication planning failed for an accepted spaced source before the fix and now passes alongside its unspaced control. Carried-key, lock/outcome, partial-failure, and read-only inspection tests pass.

Stage: 12; assembled correctness verification in Stage 21.

Location: internal/app/session_state_command.go:256–270.

Publication validation and execution accept a source with surrounding whitespace and resolve it to its trimmed artifact identity. The new report collector uses the authored rule.Source unchanged for lock, outcome, and committed-declaration lookups, although its destination helper trims the same source.

A focused diagnostic loads and validates source: " narratio.transcript.final_trimmed " through the actual configuration loaders. The shared publication planner produces narratio.transcript.final_trimmed. Given a matching committed artifact and lock, inspection reports missing, unlocked, an empty outcome, and no declared key. The otherwise identical unspaced case reports published, locked, and carried_locked, with the declared object identity. The previous collector normalized the source before these lookups, making this a migration regression.

Required remediation:

  • Use the same canonical source identity as publication for every report lookup and the report's source field. Derive this identity locally or consume the shared publication-action representation; keep completed ConfigKit values unchanged.
  • Preserve accepted configuration, rather than adding a whitespace prohibition. Retain exact committed destination and origin-run reporting, including carried outputs.
  • Add a collector/native-report regression with an accepted spaced source, committed output, lock, and outcome, alongside an unspaced control.

Canonicalizing once per row is linear in the source's string length and adds no catalog-wide traversal. This correctness fix is independent of the repeated sorting/scanning optimization in finding 7.

Stage-by-stage coverage

Stage Evidence reviewed and result
1 Released module selection, native application/process boundary, lazy construction, and outcome adapter; finding 2 covers classification gaps.
2 Retained pipeline/campaign results, lifecycle normalization/validation, native metadata, and absence of replacement inspection results.
3 Campaign defaults, explicit presence, session assertions, and resource bases; findings 1 and 2 cover materialization and native assertion outcomes.
4 Selected-party results/players, pure family expansion, collision/dependency checks, and separate publication rules; finding 4 covers resume integration.
5 Concrete-catalog consumers in planning, analysis, semantic fingerprints, adapter composition, and runtime registration.
6 Preparation, publication, restore round trips, and semantic evidence; findings 1 and 4 cover serialization and provenance-sensitive reuse.
7 Exact-file/local/remote acquisition, source labels, one-time bootstrap, retained errors, and resource cleanup; finding 2 covers typed input classification.
8 Four native scopes, path/history/findings capabilities, aggregate behavior, and lazy acquisition.
9 Independent comparison inputs/defaults, app-only overrides, native difference reports and status 3; finding 2 covers assertion failures.
10 Native bounded run/regeneration wiring, force and artifact selection, and shared execution; finding 2 covers invalid domain configuration.
11 Shared read-only planning, publication actions, and configured artifact catalog reporting; finding 4 affects planned reuse.
12 Structured validation/status/artifact inspection, partial findings, local/remote evidence, and read-only behavior.
13 Initialization targets, authoring contexts, generated YAML, explicit lock commands, and conditional lock mutations; finding 1 covers inherited defaults.
14 Restore and cleanup declarations, dry-run behavior, exact action reports, confinement, conflicts, and partial failure handling.
15 Failed-attempt accounting, partial run results, checkpoint evidence, terminal persistence, cause retention, and lock release.
16 Interrupt/termination wiring, context propagation, worker settlement, detached terminal local persistence, and resumability.
17 Narratio-owned reporting seam, native visibility/channels, stage progress, and isolation from downstream debug capture.
18 Audio transfers/cache reuse, speaker/artifact durable completions, checkpoint reuse, and progress-only integration tests.
19 Native completion/static/path hints, lazy discovery tests, and retirement of handwritten routing/parsing/rendering.
20 Canonical references, workflows, migration table, attribution, supported platforms, and preserved release history; findings 1 and 3 cover remaining documentation divergence.
21 Full test/race suites, vet/build, assembled workflows, focused review reproductions, and remaining verification recorded below.

The initial pass identified no other actionable requirement strengthening. The removed CLI aliases, native flag conventions, inspection-only overrides, domain/native-view separation, and deferred workspace/downstream changes follow the agreed scope.

Review validation

Using official Go 1.27.4 against the reviewed application source:

  • go test ./... passed.
  • go test -race ./... passed.
  • go vet ./... and go build ./... passed.
  • Focused Go-overlay diagnostics reproduced both cases in finding 2 and the campaign-default freezing in finding 1. Diagnostic files remained under /tmp; application source and tests were unchanged.
  • go mod tidy -diff passed.
  • The nonpublishing asset builder passed with v0.0.0-test, producing Linux amd64/arm64 and macOS arm64 binaries under /tmp and verifying the host binary's embedded version. macOS evidence is compilation-only.
  • Documentation-link checks and git diff --check passed with this review document present.

Only this remediation document changes the repository. No release, tag, push, CI polling, or live downstream service invocation was performed.

Integration follow-up coverage and validation

The follow-up traversed the stages in order, emphasizing existing consumers:

  • Stages 1–3: process/CLI inputs into existing loaders, validation callers, authoring-context initialization, and materialized input consumers.
  • Stage 4: selected-party provenance and canonical roster/players derivation.
  • Stage 5: concrete catalog/dependency consumers in planning, execution, adapter composition, previous-session requirements, and runtime registration.
  • Stage 6: semantic fingerprints, preparation comparisons, publication actions, and restored input records. Finding 4 is an additional resume-integration gap.
  • Stage 7: explicit/default source selection into existing identity checks, local/remote acquisition, storage, and temporary-resource cleanup.
  • Stages 8–9: native scope acquisition, independent comparison state, and invocation-local overrides against the retained load results.
  • Stages 10–11: bounded run/regeneration and plan decisions against existing resume validation, analysis target closure, and dependent invalidation.
  • Stages 12–14: inspection, initialization, locks, restore, and cleanup against local manifests, pointer-selected remote declarations, conditional lock updates, confinement, and existing action-report persistence.
  • Stages 15–16: partial results and cancellation against speaker/artifact checkpoints, terminal persistence, adapter contexts, and local lock release.
  • Stages 17–19: progress/diagnostic delivery across existing concurrent workers and captured subprocess logs; native lazy discovery and retired routing.
  • Stages 20–21: current operator contracts and maintained workflow fixtures against the assembled boundaries above.

Application source remained unchanged. The initial full tests, race tests, vet, build, module, and supported-target checks therefore remain applicable. This pass additionally reran go test for internal/app, internal/config, internal/stage, internal/artifacts, internal/audio, and internal/manifest; all passed on Go 1.27.4. A new native-invocation Go-overlay diagnostic reproduced finding 4 under -race and verified unchanged prepared input checksums. Its files remained under /tmp. Documentation checks and whitespace validation were repeated for this document update.

Algorithmic follow-up coverage and validation

The algorithmic pass reviewed the completed stages in order, using the same application source and the exact pinned dependency implementations. Findings 5–8 add three scaling opportunities and one correctness regression. It preserves the algorithms and consequential behavior expressly selected in the plan.

Stage Algorithmic review
1 Fixed command declarations and native outcome adaptation; parser/reporting algorithms remain upstream-owned.
2 Native-result and source-metadata access; repeated catalog snapshots are covered by finding 5.
3 Presence-based defaults and fixed stable-input fields; no additional scaling finding.
4 Family expansion is output-sensitive in families × characters; maps support collision checks. Finding 5 covers avoidable metadata copies beyond that output work.
5 Concrete dependency/input consumers and family-origin access; finding 6 covers whole-catalog copies per lookup.
6 Safe materialization and semantic projections; retained checkpoint and publication algorithms were not reopened.
7 Invocation-local lazy acquisition reuses acquired products and keeps comparison sides separate.
8 Native scope dispatch and actual-result inspection; no replacement traversal algorithm is introduced.
9 Independent side acquisition and upstream comparison/override semantics; no additional scaling finding.
10 Artifact-name token processing uses a set and sorting; family selection expands the selected members, and stage selection operates on the fixed inventory.
11 Planning and catalog reporting use derived products; findings 5 and 6 apply before/during report construction.
12 Artifact/publication report collection; findings 7 and 8 cover repeated lookup construction and inconsistent source normalization.
13 Initialization and lock commands retain shared operations; no additional scaling finding.
14 Restore/cleanup action reports retain their existing planners and safety rules; no additional scaling finding in the conversion.
15 Durable summary collection traverses recorded outputs/checkpoints with per-stage sets and sorted keys; finding 6 also affects individual analysis records.
16 Cancellation propagation and worker settlement retain selected execution/checkpoint behavior.
17 Native diagnostic adaptation adds no growing history or extra durable progress collection.
18 Per-unit progress attaches to existing audio, speaker, and artifact completion/reuse events.
19 Static completion and legacy deletion add no campaign/artifact enumeration algorithm.
20 Documentation-only stage; no runtime algorithm to reassess.
21 Cross-checked assembled paths, selected dependency source, and focused correctness reproduction.

Complexity statements are derived from loops, call paths, and copied payloads; they are not benchmark results or predictions of measured operator latency. The pass did not reconsider upstream configuration algorithms, fixed pipeline ordering/invalidation, immediate durable checkpoints, or the selected publication and restore protocols.

The focused Go-overlay diagnostic used real configuration loading/validation and the shared publication planner with in-memory remote state. Its canonical-source control passed; its spaced-source case failed the expected-state assertions as described in finding 8. Diagnostic files remained under /tmp, with no repository test or application changes. Earlier full application verification remains applicable to the unchanged source. Documentation checks and git diff --check were repeated for this update. No live services or performance measurements were used.

Remediation verification

The completed Narratio units and domain-validation portion of finding 2 pass go test ./..., go test -race ./..., go vet ./..., and go build ./... with official Go 1.27.4. Documentation checks and git diff --check also pass. Native regression coverage verifies equivalent-party resume, real roster changes, source normalization, and typed domain-input outcomes. No upstream files, dependencies, releases, tags, or remote repositories were changed. Finding 2's native configuration identity classification remains outstanding as recorded above.