Files
narratio/docs/roadmap/remediation.md

482 lines
29 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# CommandKit migration review
## Scope
Reviewed all 21 stages of [implementation.md](implementation.md) against the
[approved roadmap](commandkit.md), the implementation from `86692d5` through
`0117f3f35d73b3025a0222d25b6924ef455592e1`, current source and documentation,
behavioral tests, and focused diagnostic reproductions. Completion annotations
were treated as claims to verify rather than proof of completion.
The native command boundary, actual ConfigKit inspection results, separate domain
catalogs, independent selections, reporting, and cancellation architecture are
substantially implemented. The initial pass identified findings 1–3. An
integration-focused follow-up at `8be65fd` adds finding 4; the application source
is unchanged from the initial review. An algorithm-focused follow-up at
`65d24fb` adds findings 5–8: two P2 findings and two P3 optimization
opportunities. Remediation status is recorded under each finding. The review
passes made no application changes.
## 1. P2: Preserve campaign inheritance when materializing session documents
Status: Completed. The shared renderer uses one native explicitness snapshot to
omit inherited stable inputs and retain authored overrides and optional clearing.
Regression coverage checks changed receiving-campaign defaults, imported explicit
overrides, inherited optional inputs, preparation/publication fallback, and local
and remote initialization. Configuration, application, and stage suites pass.
Stages: 3, 6, and 13; preservation checks in Stage 21.
Locations: `internal/config/load.go:254–255`,
`internal/config/document_render.go:52–62`, and
`internal/app/prepared_session_roundtrip_test.go:88`.
`SessionDocument` now points to the completed session, which already contains
campaign defaults. `RenderSessionDocument` writes those inherited stable input
paths as explicit session fields. Reloading generated session YAML or the
published prepared-session fallback therefore replaces inheritance with fixed
paths from the producing campaign. The optional spell-catalog default is also
written whenever nonempty.
Reproduction: load a session whose authored inputs contain only audio; render its
prepared session; change the selected campaign's `speakers_file`; reload the
prepared bytes with that campaign. The received session still selects the old
speaker file. Its native metadata now calls this a session override. The
round-trip test was changed to expect `session_config` ownership for the formerly
inherited speaker path, so it accepts the regression rather than protecting the
prior behavior.
The [configuration contract](../config.md#pipeline-imports-and-overlays) says
omitted session fields remain eligible to inherit the receiving campaign's
values. Preserving materialization/authoring semantics was part of the plan.
Freezing inherited resource paths is an additional semantic commitment, not a
requirement of completing defaults before the ConfigKit handoff. It can select
obsolete resources after campaign updates and retain paths unavailable on a
receiving host.
Required remediation:
- Keep the actual completed ConfigKit result authoritative for inspection and
execution. Use native explicitness metadata when serializing session-owned
fields so inherited campaign input defaults remain omitted in materialized
session YAML. Retain explicit session overrides and explicit optional-input
clearing. Preserve import flattening and source-relative resolution.
- Apply the distinction to preparation, local/remote initialization, and
committed-session fallback through their shared renderer. Do not introduce a
replacement configuration result or reparse authoring files for inspection.
- Restore coverage that materializes an omitted default, changes the receiving
campaign, and observes the new default. Include an explicit override and an
explicit empty optional spell catalog to establish the difference. Update the
existing round-trip expectation rather than preserving the newly frozen
behavior as a contract.
## 2. P2: Classify identifiable Narratio input errors consistently
Stages: 1, 7–10, and 12; native outcome verification in Stage 21.
Locations: `internal/app/configuration_commands.go:75–96`,
`internal/app/configuration_acquisition.go:294–297`, and
`internal/app/command_outcome.go:30–55`.
Two integration paths return operational status 1 for identifiable input errors:
1. Native configuration acquisition/comparison returns
`config.SessionIdentityError` for a supplied campaign/session identity that
differs from the selected file. The scope wrapper adds explanatory text but
passes the same error to ckconfig, whose classifier does not know Narratio's
type. For example, `config show --scope campaign --campaign-file FILE
--campaign WRONG` returns status 1. Session assertions and comparison-side
assertions have the same problem. The application outcome adapter recognizes
the type for domain commands, but native config commands do not use it.
2. Catalog derivation returns plain errors for invalid configuration relations.
A configured artifact with `depends_on: [missing]` reaches domain derivation,
then `artifacts list` and `run --stage analyze` return status 1. The error
unambiguously identifies invalid supplied configuration, not a service failure.
Such relational validation belongs in domain derivation; moving family
expansion into native app inspection is not the remedy.
The roadmap and [CLI outcome contract](../cli.md#cancellation-and-exit-status)
reserve status 2 for invalid invocation or identifiable configuration input.
Current identity tests check only a nonzero status and explanatory text, leaving
this integration mismatch undetected.
Required remediation:
- Preserve the agreed status 2 classification for post-resolution identity
assertions in native config acquisition and comparison, including the right
side. Use genuine native resolution/validation or a supported classification
integration point while preserving the underlying typed cause. If the released
API lacks the necessary integration point, address that narrow capability
upstream rather than forking native commands or matching error strings.
- Give Narratio-owned invalid domain-configuration relations typed input
classification at their existing boundary. Keep cancellation, storage/access
failures, shutdown/persistence errors, and mixed operational causes at status 1.
Do not classify every acquisition or catalog error as user input indiscriminately.
- Add native invocation regressions asserting exact status classification for
campaign/session mismatch, comparison-side mismatch, and an unknown artifact
dependency, alongside an operational or joined-cause counterexample.
## 3. P2: Correct the documented empty spell-catalog behavior
Status: Completed. The session field reference distinguishes omission (inherit)
from an explicit empty string (clear). Documentation checks pass; existing
native-default and materialization tests cover both behaviors.
Stage: 20.
Location: `docs/config.md:706`, the `inputs.spell_catalog_file` session field row.
The field table says an empty or omitted value inherits the campaign value.
The implemented native-default behavior distinguishes them: omission inherits;
an explicit empty string clears the optional input. This behavior is exercised
by `TestSessionDefaultsCompleteBeforeNativeHandoff` and is consistent with the
agreed explicit-presence rules. The table contradicts both implementation and
the reference's general defaults section.
Required remediation: update the field row and any equivalent current guidance
to state the implemented distinction. Retain optional-input clearing behavior;
do not change it to satisfy the obsolete wording. This is a documentation-only
correction, separate from preserving inherited versus explicit fields during
materialization in finding 1.
## 4. P2: Keep party-selection provenance out of prepare resume identity
Status: Completed. Prepare's versioned semantic projection excludes stable-input
ownership labels; provenance remains available in source records. Native planning
and execution regressions retain identical prepared checksums and downstream
success for default/explicit selection of the same roster, and invalidate work
for changed roster contents. Focused resume and semantic tests pass under race
detection.
Stages: 4–7, 10–11, and 21.
Locations: `internal/config/party_resolution.go:70–81`,
`internal/stage/semantic_contracts_initial.go:151–171`,
`internal/app/semantic_resume.go:44–45`, and
`internal/app/run_control.go:78–83`.
The new explicit party selector labels its source `party_file`; campaign-default
selection labels the same roster `campaign_config`. Existing prepare semantic
evidence hashes that source label in `prepareStableInputSelection.Source`.
Selecting the very same file explicitly changes the semantic fingerprint even
when the canonical roster, players projection, derived catalog, and all prepared
file contents are unchanged.
An end-to-end native-invocation diagnostic reproduced this sequence:
1. Run `prepare` successfully using the campaign-default roster and record a
succeeded downstream `transcribe` stage.
2. Plan the same session with `--party-file` pointing to that same roster. The
planner reports `result-affecting configuration changed; rerun the stage`.
3. Execute that bounded prepare request without force. Prepare executes again,
and the previously succeeded transcription stage becomes stale. Every
prepared input path/checksum pair remains identical.
The normal invalidation relation also applies to other succeeded downstream
stages. A subsequent full run can therefore schedule unnecessary downstream
processing solely because the operator made the source selection explicit.
The selector and existing semantic-resume implementation work independently,
but their integration turns provenance into result identity, contrary to the
roadmap's result-affecting evidence boundary. This is separate from finding 1:
the reproduction requires no serialization/reload or campaign-default change.
Required remediation:
- Keep the selected source label in appropriate provenance and inspection
records. Exclude or semantically normalize the provenance-only distinction in
prepare's reuse fingerprint. Review equivalent stable-input source labels
under the same result-affecting rule without weakening content validation.
- Continue detecting actual roster/players/input changes through existing
canonical payload checksums and stage/artifact evidence. Preserve the fixed
downstream invalidation relation when inputs really change.
- Add an integration regression covering native planning and execution with
default versus explicit selection of the same roster, unchanged prepared
bytes, and retained downstream success. Include changed roster contents as a
counterexample that must invalidate affected work.
## 5. P2: Capture pipeline provenance once per catalog derivation
Status: Completed. One derivation-local resolver shares a native metadata snapshot
and source-path index across family and authored-artifact lookups. Snapshot work
is `O(M)` per derivation, plus existing native queries. The configuration suite
and focused catalog, inheritance, overlay, and provenance tests pass; existing
immutability and independent-roster coverage remains intact.
Stages: 2 and 4; catalog consumers in Stages 5, 7, and 11.
Locations: `internal/config/artifact_catalog.go:49–65,127–143`,
`internal/config/artifact_families.go:78`, and
`internal/config/pipeline_configkit.go:291–307`.
Every authored concrete-artifact provenance lookup calls
`cfg.ConfigKitResult().Metadata()`. Family derivation calls the same accessor
once per family. In the selected ConfigKit v1.2.1, this accessor calls
`Inspection.Metadata`, which clones the complete effective-path and field-info
maps. It is a detached snapshot operation, rather than a constant-time handle
lookup. The immutable history model is shared; the effective metadata maps and
their nested records are copied.
Let `D` be the number of authored artifact and family declarations, and `M` the
size of the metadata copied by one snapshot. Snapshot acquisition alone costs
`O(D * M)` work and cumulative allocation per catalog derivation, in addition to
the actual history queries. With bounded-size declarations and `M = Θ(D)`, this
is quadratic work. Family lookups also rebuild the complete source-path index
per family. The prior implementation reused its retained metadata snapshot;
the repeated full-snapshot acquisition is introduced by this migration.
Refactoring opportunity:
- Acquire one native metadata snapshot and one source-ID/path index inside a
catalog derivation. Share them across authored-artifact and family provenance
queries through a small local resolver or explicit helper parameters.
- This reduces snapshot work to `O(M)` plus the existing per-declaration native
queries and output processing. Build the source index once as well. The
existing `partyCharacterSourceLookup` already demonstrates an invocation-local
snapshot and lookup cache.
- Retain native history, inheritance, and source semantics, with no global
cache or replacement inspection result. Verify equivalent provenance for
concrete definitions, families, imports, and overlays; preserve the completed
ConfigKit values.
This opportunity concerns how Narratio calls the public API. It does not require
changing ConfigKit's detached-snapshot contract or reviewing its selected
configuration algorithms.
## 6. P3: Avoid cloning the complete family catalog for individual origins
Stages: 5 and 11; artifact evidence in Stage 15.
Locations: `internal/stage/analyze_resume.go:88–103`,
`internal/stage/analyze.go:179–185`, `internal/config/config.go:71–76`, and
`internal/config/artifact_families.go:395–418`.
`exportAnalyzeResumeItems` calls `config.ArtifactFamilies(catalog)` inside its
per-item loop, then reads one member's family and character ID. That accessor
deep-copies every family and member, including dependency slices and input maps.
`analyzeArtifactOrigin` performs the same whole-catalog copy for individual
artifact checkpoint and diagnostic records.
Let `L` be the number of origin lookups and `K` the total family/member catalog
payload. These lookups cost `O(L * K)` work and cumulative allocation. When a
plan reports a catalog-sized set of bounded-size members, `L` and `K` both grow
with member count, producing quadratic copy work for a linear-sized report.
This inefficiency predates the migration and remains in the rewired domain
consumers; it is an optimization opportunity, rather than a new regression.
Refactoring opportunity:
- Capture one detached family catalog outside a batch and reuse its member
index across all three resume-summary lists. Copy work becomes `O(K + L)`.
- For execution's individual lookups, read the already-derived member index
through a narrow accessor that returns the needed identity fields by value.
If a caller needs mutable dependency/input records, copy that member only.
- Preserve family identities, deterministic report ordering, and detached
records where callers need them. A new defensive-copy framework or global
cache is unnecessary.
## 7. P3: Build publication-report lookup tables once
Stage: 12.
Locations: `internal/app/session_state_command.go:256–270`,
`internal/app/artifact_inspection.go:79–131`, and
`internal/artifacts/catalog.go:255–289`.
For every publication rule, `inspectPublishedOutputs` calls
`helperPublishedOutputDest`, which rebuilds the configured-output map and
extraction-output set. Building each map first enumerates, sorts, and copies
the entire corresponding runtime catalog. The same loop calls
`committedPublishedOutput`, scanning all commit declarations to find one source.
Let `R` be publication-rule count, `A` configured-artifact count, `X`
extraction-artifact count, and `K` commit-declaration count. With bounded-size
catalog records, the repeated local work is
`O(R * (A log A + X log X + K))`, plus copying any variable-sized entry payloads.
When `R`, `A`, and `K` grow together, this includes quadratic-logarithmic sorting
work and quadratic declaration scans. Required remote existence checks are
separate from this CPU/allocation cost. These helpers and repeated lookups
predate the migration and remain in the new structured-report collector.
Refactoring opportunity:
- Build configured/extraction lookup tables once per report and pass them to
destination resolution, or share the existing publication-action planner
where its selection and failure behavior fit inspection.
- Index published commit declarations once by source, preserving the existing
declaration-selection behavior. Resolve each report row through that index.
- Reusing the existing catalog enumeration APIs reduces this local lookup work
to `O(A log A + X log X + K + R)`, plus entry-payload copying. Keep configured
report ordering, partial error reporting, pointer-selected authority, exact
declared object keys, and the existing remote checks.
There is no need to add remote concurrency, caching across invocations, or a new
publication protocol to obtain this improvement.
## 8. P2: Use canonical publication sources in session reports
Stage: 12; assembled correctness verification in Stage 21.
Location: `internal/app/session_state_command.go:256–270`.
Publication validation and execution accept a source with surrounding whitespace
and resolve it to its trimmed artifact identity. The new report collector uses
the authored `rule.Source` unchanged for lock, outcome, and committed-declaration
lookups, although its destination helper trims the same source.
A focused diagnostic loads and validates `source:
" narratio.transcript.final_trimmed "` through the actual configuration loaders.
The shared publication planner produces `narratio.transcript.final_trimmed`.
Given a matching committed artifact and lock, inspection reports `missing`,
`unlocked`, an empty outcome, and no declared key. The otherwise identical
unspaced case reports `published`, `locked`, and `carried_locked`, with the
declared object identity. The previous collector normalized the source before
these lookups, making this a migration regression.
Required remediation:
- Use the same canonical source identity as publication for every report
lookup and the report's source field. Derive this identity locally or consume
the shared publication-action representation; keep completed ConfigKit values
unchanged.
- Preserve accepted configuration, rather than adding a whitespace prohibition.
Retain exact committed destination and origin-run reporting, including carried
outputs.
- Add a collector/native-report regression with an accepted spaced source,
committed output, lock, and outcome, alongside an unspaced control.
Canonicalizing once per row is linear in the source's string length and adds no
catalog-wide traversal. This correctness fix is independent of the repeated
sorting/scanning optimization in finding 7.
## Stage-by-stage coverage
| Stage | Evidence reviewed and result |
| --- | --- |
| 1 | Released module selection, native application/process boundary, lazy construction, and outcome adapter; finding 2 covers classification gaps. |
| 2 | Retained pipeline/campaign results, lifecycle normalization/validation, native metadata, and absence of replacement inspection results. |
| 3 | Campaign defaults, explicit presence, session assertions, and resource bases; findings 1 and 2 cover materialization and native assertion outcomes. |
| 4 | Selected-party results/players, pure family expansion, collision/dependency checks, and separate publication rules; finding 4 covers resume integration. |
| 5 | Concrete-catalog consumers in planning, analysis, semantic fingerprints, adapter composition, and runtime registration. |
| 6 | Preparation, publication, restore round trips, and semantic evidence; findings 1 and 4 cover serialization and provenance-sensitive reuse. |
| 7 | Exact-file/local/remote acquisition, source labels, one-time bootstrap, retained errors, and resource cleanup; finding 2 covers typed input classification. |
| 8 | Four native scopes, path/history/findings capabilities, aggregate behavior, and lazy acquisition. |
| 9 | Independent comparison inputs/defaults, app-only overrides, native difference reports and status 3; finding 2 covers assertion failures. |
| 10 | Native bounded run/regeneration wiring, force and artifact selection, and shared execution; finding 2 covers invalid domain configuration. |
| 11 | Shared read-only planning, publication actions, and configured artifact catalog reporting; finding 4 affects planned reuse. |
| 12 | Structured validation/status/artifact inspection, partial findings, local/remote evidence, and read-only behavior. |
| 13 | Initialization targets, authoring contexts, generated YAML, explicit lock commands, and conditional lock mutations; finding 1 covers inherited defaults. |
| 14 | Restore and cleanup declarations, dry-run behavior, exact action reports, confinement, conflicts, and partial failure handling. |
| 15 | Failed-attempt accounting, partial run results, checkpoint evidence, terminal persistence, cause retention, and lock release. |
| 16 | Interrupt/termination wiring, context propagation, worker settlement, detached terminal local persistence, and resumability. |
| 17 | Narratio-owned reporting seam, native visibility/channels, stage progress, and isolation from downstream debug capture. |
| 18 | Audio transfers/cache reuse, speaker/artifact durable completions, checkpoint reuse, and progress-only integration tests. |
| 19 | Native completion/static/path hints, lazy discovery tests, and retirement of handwritten routing/parsing/rendering. |
| 20 | Canonical references, workflows, migration table, attribution, supported platforms, and preserved release history; findings 1 and 3 cover remaining documentation divergence. |
| 21 | Full test/race suites, vet/build, assembled workflows, focused review reproductions, and remaining verification recorded below. |
The initial pass identified no other actionable requirement strengthening.
The removed CLI aliases, native flag conventions, inspection-only overrides, domain/native-view
separation, and deferred workspace/downstream changes follow the agreed scope.
## Review validation
Using official Go 1.27.4 against the reviewed application source:
- `go test ./...` passed.
- `go test -race ./...` passed.
- `go vet ./...` and `go build ./...` passed.
- Focused Go-overlay diagnostics reproduced both cases in finding 2 and the
campaign-default freezing in finding 1. Diagnostic files remained under
`/tmp`; application source and tests were unchanged.
- `go mod tidy -diff` passed.
- The nonpublishing asset builder passed with `v0.0.0-test`, producing Linux
amd64/arm64 and macOS arm64 binaries under `/tmp` and verifying the host
binary's embedded version. macOS evidence is compilation-only.
- Documentation-link checks and `git diff --check` passed with this review
document present.
Only this remediation document changes the repository. No release, tag, push,
CI polling, or live downstream service invocation was performed.
## Integration follow-up coverage and validation
The follow-up traversed the stages in order, emphasizing existing consumers:
- Stages 1–3: process/CLI inputs into existing loaders, validation callers,
authoring-context initialization, and materialized input consumers.
- Stage 4: selected-party provenance and canonical roster/players derivation.
- Stage 5: concrete catalog/dependency consumers in planning, execution,
adapter composition, previous-session requirements, and runtime registration.
- Stage 6: semantic fingerprints, preparation comparisons, publication actions,
and restored input records. Finding 4 is an additional resume-integration gap.
- Stage 7: explicit/default source selection into existing identity checks,
local/remote acquisition, storage, and temporary-resource cleanup.
- Stages 8–9: native scope acquisition, independent comparison state, and
invocation-local overrides against the retained load results.
- Stages 10–11: bounded run/regeneration and plan decisions against existing
resume validation, analysis target closure, and dependent invalidation.
- Stages 12–14: inspection, initialization, locks, restore, and cleanup against
local manifests, pointer-selected remote declarations, conditional lock
updates, confinement, and existing action-report persistence.
- Stages 15–16: partial results and cancellation against speaker/artifact
checkpoints, terminal persistence, adapter contexts, and local lock release.
- Stages 17–19: progress/diagnostic delivery across existing concurrent workers
and captured subprocess logs; native lazy discovery and retired routing.
- Stages 20–21: current operator contracts and maintained workflow fixtures
against the assembled boundaries above.
Application source remained unchanged. The initial full tests, race tests, vet,
build, module, and supported-target checks therefore remain applicable. This
pass additionally reran `go test` for `internal/app`, `internal/config`,
`internal/stage`, `internal/artifacts`, `internal/audio`, and `internal/manifest`;
all passed on Go 1.27.4. A new native-invocation Go-overlay diagnostic reproduced
finding 4 under `-race` and verified unchanged prepared input checksums. Its
files remained under `/tmp`. Documentation checks and whitespace validation
were repeated for this document update.
## Algorithmic follow-up coverage and validation
The algorithmic pass reviewed the completed stages in order, using the same
application source and the exact pinned dependency implementations. Findings 5–8
add three scaling opportunities and one correctness regression. It preserves the
algorithms and consequential behavior expressly selected in the plan.
| Stage | Algorithmic review |
| --- | --- |
| 1 | Fixed command declarations and native outcome adaptation; parser/reporting algorithms remain upstream-owned. |
| 2 | Native-result and source-metadata access; repeated catalog snapshots are covered by finding 5. |
| 3 | Presence-based defaults and fixed stable-input fields; no additional scaling finding. |
| 4 | Family expansion is output-sensitive in families × characters; maps support collision checks. Finding 5 covers avoidable metadata copies beyond that output work. |
| 5 | Concrete dependency/input consumers and family-origin access; finding 6 covers whole-catalog copies per lookup. |
| 6 | Safe materialization and semantic projections; retained checkpoint and publication algorithms were not reopened. |
| 7 | Invocation-local lazy acquisition reuses acquired products and keeps comparison sides separate. |
| 8 | Native scope dispatch and actual-result inspection; no replacement traversal algorithm is introduced. |
| 9 | Independent side acquisition and upstream comparison/override semantics; no additional scaling finding. |
| 10 | Artifact-name token processing uses a set and sorting; family selection expands the selected members, and stage selection operates on the fixed inventory. |
| 11 | Planning and catalog reporting use derived products; findings 5 and 6 apply before/during report construction. |
| 12 | Artifact/publication report collection; findings 7 and 8 cover repeated lookup construction and inconsistent source normalization. |
| 13 | Initialization and lock commands retain shared operations; no additional scaling finding. |
| 14 | Restore/cleanup action reports retain their existing planners and safety rules; no additional scaling finding in the conversion. |
| 15 | Durable summary collection traverses recorded outputs/checkpoints with per-stage sets and sorted keys; finding 6 also affects individual analysis records. |
| 16 | Cancellation propagation and worker settlement retain selected execution/checkpoint behavior. |
| 17 | Native diagnostic adaptation adds no growing history or extra durable progress collection. |
| 18 | Per-unit progress attaches to existing audio, speaker, and artifact completion/reuse events. |
| 19 | Static completion and legacy deletion add no campaign/artifact enumeration algorithm. |
| 20 | Documentation-only stage; no runtime algorithm to reassess. |
| 21 | Cross-checked assembled paths, selected dependency source, and focused correctness reproduction. |
Complexity statements are derived from loops, call paths, and copied payloads;
they are not benchmark results or predictions of measured operator latency.
The pass did not reconsider upstream configuration algorithms, fixed pipeline
ordering/invalidation, immediate durable checkpoints, or the selected publication
and restore protocols.
The focused Go-overlay diagnostic used real configuration loading/validation and
the shared publication planner with in-memory remote state. Its canonical-source
control passed; its spaced-source case failed the expected-state assertions as
described in finding 8. Diagnostic files remained under `/tmp`, with no repository
test or application changes. Earlier full application verification remains
applicable to the unchanged source. Documentation checks and `git diff --check`
were repeated for this update. No live services or performance measurements were
used.