Record algorithmic refactoring opportunities

This commit is contained in:
2026-11-18 01:10:44 +00:00
parent 583c5bdf1b
commit ba9079ee74

View File

@@ -12,9 +12,15 @@ An integration-focused follow-up reviewed the same implementation at
command, preparation, publication, restore, and inspection workflows. Items
5–6 are additional findings from that pass; items 1–4 remain outstanding.
An algorithm-focused pass reviewed all twelve stages at
`583c5bdf1bc8f0af8ec2e257bcb03ecccbf283b1`. Items 7–10 are refactoring
opportunities for consideration. Complexity follows the code and the selected
ConfigKit v1.2.1 implementation; no operational timings or benchmarks were
collected. The pass preserves algorithms expressly chosen by the plan.
The native configuration architecture is substantially implemented. The items
below are the remaining required corrections; this review does not implement
their fixes.
below record the six required corrections and four additional algorithmic
opportunities. This document implements none of their fixes.
## 1. P2: Distinguish effective resource origins from superseded history
@@ -198,6 +204,125 @@ the updated values. Include audio or previous-session selection as well as title
Verify that restore still uses its selected immutable commit and that published
objects are unchanged by the authoring update.
## 7. P3: Reuse enumerated source records and one inspection source index
Stage: 9.
Locations: `internal/config/effective_pipeline.go:57–75`,
`internal/config/pipeline_configkit.go:354–371`, and
`internal/config/artifact_catalog.go`, `declarationDefinitionSources`.
`emptyImportSelections` enumerates `Result.Sources()` and then calls
`Result.Source(id)` for every enumerated record. `pipelineSourcePaths` does the
same lookup for every retained source. In ConfigKit v1.2.1, completed-result
source lookup scans the source slice. Enumerating S sources this way performs
Θ(S²) source-ID comparisons, although the enumerated records already contain
the required information, including empty-import evidence.
The artifact catalog additionally rebuilds this source map for each declaration.
For A entries this introduces O(A S²) source-index work; with one imported file
per artifact, that component can become cubic in A. This is separate from the
native history queries and necessary output construction.
Potential refactoring: consume the complete enumerated records directly. Build
the source-ID/path map once per inspection operation and share it with declaration
projection. This reduces empty-import enumeration to O(S + E), where E is the
returned selection evidence, and source-index work across the catalog to
expected O(S + A). Expanded members can also reuse the family-source lists
already retained in the family catalog. Keep native history and origin queries
as the provenance authority.
Verification: temporary functional checks confirmed that enumerated records and
individual lookups have equal contents, and that direct enumeration preserves
source paths and nested empty-import selections. Preserve operator source/catalog
output, including composite and inherited contributors, when refactoring.
## 8. P3: Reuse party metadata and per-character provenance within inspection
Stage: 9, consuming Stage 5's imported party result.
Locations: `internal/config/artifact_catalog.go:67–105` and
`internal/config/effective_pipeline.go`, `appendGeneratedArtifactSources`.
`partyCharacterSources` calls `result.Metadata()` separately for Lookup and
History. ConfigKit v1.2.1 returns detached metadata by copying the complete
effective-path and field maps, including their origin records. This helper is
called for each generated artifact in the catalog and for each generated
artifact field in source inspection. The same character's provenance is
therefore repeatedly recomputed across fields and families.
Let M be the copied party-metadata size and P the number of provenance requests.
The avoidable metadata-copy work and cumulative allocation are Θ(P M). Even
with one family and bounded-size character entries, M and P both grow with the
character count C, producing quadratic work. More families and fields multiply
the requests. This describes the provenance pass, not the complexity of the
whole command or unchanged party validation.
Potential refactoring: obtain one native Metadata snapshot and source index per
inspection operation. Resolve each requested character's source list once and
reuse it for that operation. The full-metadata copy term becomes O(M), with
native per-character query costs and required output emission remaining. Keep
the cache local to the projection; a new persistent provenance service is
unnecessary. Apply the same snapshot reuse to campaign provenance helpers where
convenient, although their fixed field count is not a quadratic case.
Verification: a temporary imported/inherited-roster check confirmed equivalent
source lists using shared native metadata and an enumerated source index.
Retain contributor accuracy, ordering, and secret-free inspection output.
## 9. P3: Read the selected remote session without enumerating run history
Stage: 7.
Location: `internal/app/config_loader.go:194–228`, `findRemoteSessionConfig`.
After selecting the committed prepared-session key, this function lists the
entire session prefix and scans it for that exact key. If H objects are retained
across historical runs, this adds Θ(H) listing/scan work and Θ(H) object-record
memory to every remote session load. The production S3 adapter paginates and
accumulates the complete listing. The scan predates the migration and was
retained in the new committed-object selection path.
Potential refactoring: retrieve the exact selected key through the existing
`ObjectStore.Read` boundary, which returns metadata with the opened body. Use
that metadata for the existing size/generation checks and stream the body into
the command-owned temporary file while retaining checksum verification. The
object lookup/transfer then takes a constant number of requests independent of
H, plus the necessary O(B) work for B session bytes. Current-commit acquisition
and validation costs remain. Apply this after reconciling the selection policy
in item 6; no new backend interface is needed.
Verification: the production S3 Read implementation performs one exact-key
GetObject and returns its size and generation. Preserve fake-store coverage for
missing objects, selected identity, checksum/size/generation mismatches, and
temporary-file cleanup. Confirm that unrelated historical keys are not enumerated.
## 10. P3: Stream prepared-input archival copies
Stage: 7.
Location: `internal/stage/publish_sources.go:29–64`,
`collectPublishPreparedInputs`.
After opening and verifying each prepared document, the new archival path reads
the whole file with `io.ReadAll` and passes the byte slice to
`writeBytesIfChanged`. For a document of B bytes this requires Θ(B) auxiliary
transfer memory. Expanded imported party documents can be larger than any one
authoring fragment. A full in-memory document is unnecessary for this file copy.
Potential refactoring: preserve the current confined source opening and checksum
checks, then stream the opened descriptor into the run archive. The existing
`fileops.WriteReaderAtomicWithChecksum` already supplies reader-based atomic
replacement and hashing through `io.Copy`. Preserve unchanged-destination reuse
using the known checksum. Transfer time remains Θ(B), while auxiliary copy
memory becomes O(1) with a fixed-size buffer. This does not alter the plan's
separate canonical YAML rendering algorithm or total in-memory configuration size.
Verification: preserve archived bytes/checksums, workspace file modes, source
validation, failure handling, and immutable publication behavior. Existing
reader-copy and prepared-session publication tests provide suitable behavioral
coverage; allocation or wall-clock thresholds are unnecessary.
## Stage-by-stage disposition
| Stage | Review result |
@@ -206,11 +331,11 @@ objects are unchanged by the authoring update.
| 2: Discovery/bootstrap | Native discovery, exclusive explicit selection, independent bootstrap, and shared diff bootstrap are present. |
| 3: Resource bases | Native origin-based resolution and fallbacks are present; field-record cleanup remains in item 3. |
| 4: Campaign/session imports | File contexts, contextless remote snapshots, native optional discovery, and detached document values are present; initialization still uses the snapshot path in item 5. |
| 5: Party imports | Imported rosters, source-aware ordering, canonical rendering, players, and family resolution are present. |
| 5: Party imports | Imported rosters, source-aware ordering, canonical rendering, players, and family resolution are present; the planned roster sort is retained. |
| 6: Prepared inputs/resume | Canonical document materialization and matching resume evidence are present. |
| 7: Publish/restore | Prepared documents flow through immutable committed run objects and restore; remote initialization and committed selection conflict in item 6. |
| 7: Publish/restore | Prepared documents flow through immutable committed run objects and restore; item 6 is a correctness gap and items 9–10 are scalability opportunities. |
| 8: Final-domain snapshots | Defaults-backed native snapshots, safe credential identities, rendering, and digest integration are present. |
| 9: Diff/provenance | Native diff and pointer paths are present; effective/history attribution needs item 1. |
| 9: Diff/provenance | Native diff and pointer paths are present; item 1 concerns attribution correctness and items 7–8 concern avoidable provenance work. |
| 10: Examples | Destination-based definitions, directory imports, list operations, split roster, and secondary imports are demonstrated. |
| 11: Documentation/policy | Current references cover the migration; finish attribution, source-role, and initialization/selection corrections above. |
| 12: Final verification | Existing integration checks pass; add coverage for items 5–6. Full race validation remains unreliable because of item 2. |
@@ -255,3 +380,26 @@ Using Go 1.27.4 at the follow-up revision:
Only this remediation document changed. Diagnostic files remained outside the
repository, and object storage used an in-memory fake.
## Algorithm-focused review coverage and validation
The stage-by-stage pass covered dependency/API substitutions (1), native
discovery/bootstrap (2), resource-base queries (3), secondary loading and
materialization data flow (4), roster ordering/rendering (5), prepare/resume
serialization (6), archive/remote/restore paths (7), finalized snapshots (8),
diff/provenance projection (9), examples (10), documentation (11), and assembled
verification (12).
The chosen roster ordering, defaults-backed snapshot construction, native Diff,
and required content validation remain accepted. Canonical rendering is linear
in emitted data; the planned roster comparison sort is O(C log C) after deriving
ordering keys. Unchanged algorithms such as pairwise party-name validation and
prepare-input matching are not new findings from this migration-focused pass.
Cost analysis followed the released module's public API wrappers through native
source lookup and metadata-copy implementations. Temporary functional Go-overlay
checks validated source-enumeration and shared-metadata equivalence, ordinarily
and under `-race`. Focused configuration, file-reader copy, S3 opened-object
metadata, native inspection, and prepared-session publication checks passed on
Go 1.27.4. No performance data, timing assertions, benchmark gates, or application
changes were introduced.