Record algorithmic refactoring opportunities
This commit is contained in:
@@ -12,9 +12,15 @@ An integration-focused follow-up reviewed the same implementation at
|
||||
command, preparation, publication, restore, and inspection workflows. Items
|
||||
5–6 are additional findings from that pass; items 1–4 remain outstanding.
|
||||
|
||||
An algorithm-focused pass reviewed all twelve stages at
|
||||
`583c5bdf1bc8f0af8ec2e257bcb03ecccbf283b1`. Items 7–10 are refactoring
|
||||
opportunities for consideration. Complexity follows the code and the selected
|
||||
ConfigKit v1.2.1 implementation; no operational timings or benchmarks were
|
||||
collected. The pass preserves algorithms expressly chosen by the plan.
|
||||
|
||||
The native configuration architecture is substantially implemented. The items
|
||||
below are the remaining required corrections; this review does not implement
|
||||
their fixes.
|
||||
below record the six required corrections and four additional algorithmic
|
||||
opportunities. This document implements none of their fixes.
|
||||
|
||||
## 1. P2: Distinguish effective resource origins from superseded history
|
||||
|
||||
@@ -198,6 +204,125 @@ the updated values. Include audio or previous-session selection as well as title
|
||||
Verify that restore still uses its selected immutable commit and that published
|
||||
objects are unchanged by the authoring update.
|
||||
|
||||
## 7. P3: Reuse enumerated source records and one inspection source index
|
||||
|
||||
Stage: 9.
|
||||
|
||||
Locations: `internal/config/effective_pipeline.go:57–75`,
|
||||
`internal/config/pipeline_configkit.go:354–371`, and
|
||||
`internal/config/artifact_catalog.go`, `declarationDefinitionSources`.
|
||||
|
||||
`emptyImportSelections` enumerates `Result.Sources()` and then calls
|
||||
`Result.Source(id)` for every enumerated record. `pipelineSourcePaths` does the
|
||||
same lookup for every retained source. In ConfigKit v1.2.1, completed-result
|
||||
source lookup scans the source slice. Enumerating S sources this way performs
|
||||
Θ(S²) source-ID comparisons, although the enumerated records already contain
|
||||
the required information, including empty-import evidence.
|
||||
|
||||
The artifact catalog additionally rebuilds this source map for each declaration.
|
||||
For A entries this introduces O(A S²) source-index work; with one imported file
|
||||
per artifact, that component can become cubic in A. This is separate from the
|
||||
native history queries and necessary output construction.
|
||||
|
||||
Potential refactoring: consume the complete enumerated records directly. Build
|
||||
the source-ID/path map once per inspection operation and share it with declaration
|
||||
projection. This reduces empty-import enumeration to O(S + E), where E is the
|
||||
returned selection evidence, and source-index work across the catalog to
|
||||
expected O(S + A). Expanded members can also reuse the family-source lists
|
||||
already retained in the family catalog. Keep native history and origin queries
|
||||
as the provenance authority.
|
||||
|
||||
Verification: temporary functional checks confirmed that enumerated records and
|
||||
individual lookups have equal contents, and that direct enumeration preserves
|
||||
source paths and nested empty-import selections. Preserve operator source/catalog
|
||||
output, including composite and inherited contributors, when refactoring.
|
||||
|
||||
## 8. P3: Reuse party metadata and per-character provenance within inspection
|
||||
|
||||
Stage: 9, consuming Stage 5's imported party result.
|
||||
|
||||
Locations: `internal/config/artifact_catalog.go:67–105` and
|
||||
`internal/config/effective_pipeline.go`, `appendGeneratedArtifactSources`.
|
||||
|
||||
`partyCharacterSources` calls `result.Metadata()` separately for Lookup and
|
||||
History. ConfigKit v1.2.1 returns detached metadata by copying the complete
|
||||
effective-path and field maps, including their origin records. This helper is
|
||||
called for each generated artifact in the catalog and for each generated
|
||||
artifact field in source inspection. The same character's provenance is
|
||||
therefore repeatedly recomputed across fields and families.
|
||||
|
||||
Let M be the copied party-metadata size and P the number of provenance requests.
|
||||
The avoidable metadata-copy work and cumulative allocation are Θ(P M). Even
|
||||
with one family and bounded-size character entries, M and P both grow with the
|
||||
character count C, producing quadratic work. More families and fields multiply
|
||||
the requests. This describes the provenance pass, not the complexity of the
|
||||
whole command or unchanged party validation.
|
||||
|
||||
Potential refactoring: obtain one native Metadata snapshot and source index per
|
||||
inspection operation. Resolve each requested character's source list once and
|
||||
reuse it for that operation. The full-metadata copy term becomes O(M), with
|
||||
native per-character query costs and required output emission remaining. Keep
|
||||
the cache local to the projection; a new persistent provenance service is
|
||||
unnecessary. Apply the same snapshot reuse to campaign provenance helpers where
|
||||
convenient, although their fixed field count is not a quadratic case.
|
||||
|
||||
Verification: a temporary imported/inherited-roster check confirmed equivalent
|
||||
source lists using shared native metadata and an enumerated source index.
|
||||
Retain contributor accuracy, ordering, and secret-free inspection output.
|
||||
|
||||
## 9. P3: Read the selected remote session without enumerating run history
|
||||
|
||||
Stage: 7.
|
||||
|
||||
Location: `internal/app/config_loader.go:194–228`, `findRemoteSessionConfig`.
|
||||
|
||||
After selecting the committed prepared-session key, this function lists the
|
||||
entire session prefix and scans it for that exact key. If H objects are retained
|
||||
across historical runs, this adds Θ(H) listing/scan work and Θ(H) object-record
|
||||
memory to every remote session load. The production S3 adapter paginates and
|
||||
accumulates the complete listing. The scan predates the migration and was
|
||||
retained in the new committed-object selection path.
|
||||
|
||||
Potential refactoring: retrieve the exact selected key through the existing
|
||||
`ObjectStore.Read` boundary, which returns metadata with the opened body. Use
|
||||
that metadata for the existing size/generation checks and stream the body into
|
||||
the command-owned temporary file while retaining checksum verification. The
|
||||
object lookup/transfer then takes a constant number of requests independent of
|
||||
H, plus the necessary O(B) work for B session bytes. Current-commit acquisition
|
||||
and validation costs remain. Apply this after reconciling the selection policy
|
||||
in item 6; no new backend interface is needed.
|
||||
|
||||
Verification: the production S3 Read implementation performs one exact-key
|
||||
GetObject and returns its size and generation. Preserve fake-store coverage for
|
||||
missing objects, selected identity, checksum/size/generation mismatches, and
|
||||
temporary-file cleanup. Confirm that unrelated historical keys are not enumerated.
|
||||
|
||||
## 10. P3: Stream prepared-input archival copies
|
||||
|
||||
Stage: 7.
|
||||
|
||||
Location: `internal/stage/publish_sources.go:29–64`,
|
||||
`collectPublishPreparedInputs`.
|
||||
|
||||
After opening and verifying each prepared document, the new archival path reads
|
||||
the whole file with `io.ReadAll` and passes the byte slice to
|
||||
`writeBytesIfChanged`. For a document of B bytes this requires Θ(B) auxiliary
|
||||
transfer memory. Expanded imported party documents can be larger than any one
|
||||
authoring fragment. A full in-memory document is unnecessary for this file copy.
|
||||
|
||||
Potential refactoring: preserve the current confined source opening and checksum
|
||||
checks, then stream the opened descriptor into the run archive. The existing
|
||||
`fileops.WriteReaderAtomicWithChecksum` already supplies reader-based atomic
|
||||
replacement and hashing through `io.Copy`. Preserve unchanged-destination reuse
|
||||
using the known checksum. Transfer time remains Θ(B), while auxiliary copy
|
||||
memory becomes O(1) with a fixed-size buffer. This does not alter the plan's
|
||||
separate canonical YAML rendering algorithm or total in-memory configuration size.
|
||||
|
||||
Verification: preserve archived bytes/checksums, workspace file modes, source
|
||||
validation, failure handling, and immutable publication behavior. Existing
|
||||
reader-copy and prepared-session publication tests provide suitable behavioral
|
||||
coverage; allocation or wall-clock thresholds are unnecessary.
|
||||
|
||||
## Stage-by-stage disposition
|
||||
|
||||
| Stage | Review result |
|
||||
@@ -206,11 +331,11 @@ objects are unchanged by the authoring update.
|
||||
| 2: Discovery/bootstrap | Native discovery, exclusive explicit selection, independent bootstrap, and shared diff bootstrap are present. |
|
||||
| 3: Resource bases | Native origin-based resolution and fallbacks are present; field-record cleanup remains in item 3. |
|
||||
| 4: Campaign/session imports | File contexts, contextless remote snapshots, native optional discovery, and detached document values are present; initialization still uses the snapshot path in item 5. |
|
||||
| 5: Party imports | Imported rosters, source-aware ordering, canonical rendering, players, and family resolution are present. |
|
||||
| 5: Party imports | Imported rosters, source-aware ordering, canonical rendering, players, and family resolution are present; the planned roster sort is retained. |
|
||||
| 6: Prepared inputs/resume | Canonical document materialization and matching resume evidence are present. |
|
||||
| 7: Publish/restore | Prepared documents flow through immutable committed run objects and restore; remote initialization and committed selection conflict in item 6. |
|
||||
| 7: Publish/restore | Prepared documents flow through immutable committed run objects and restore; item 6 is a correctness gap and items 9–10 are scalability opportunities. |
|
||||
| 8: Final-domain snapshots | Defaults-backed native snapshots, safe credential identities, rendering, and digest integration are present. |
|
||||
| 9: Diff/provenance | Native diff and pointer paths are present; effective/history attribution needs item 1. |
|
||||
| 9: Diff/provenance | Native diff and pointer paths are present; item 1 concerns attribution correctness and items 7–8 concern avoidable provenance work. |
|
||||
| 10: Examples | Destination-based definitions, directory imports, list operations, split roster, and secondary imports are demonstrated. |
|
||||
| 11: Documentation/policy | Current references cover the migration; finish attribution, source-role, and initialization/selection corrections above. |
|
||||
| 12: Final verification | Existing integration checks pass; add coverage for items 5–6. Full race validation remains unreliable because of item 2. |
|
||||
@@ -255,3 +380,26 @@ Using Go 1.27.4 at the follow-up revision:
|
||||
|
||||
Only this remediation document changed. Diagnostic files remained outside the
|
||||
repository, and object storage used an in-memory fake.
|
||||
|
||||
## Algorithm-focused review coverage and validation
|
||||
|
||||
The stage-by-stage pass covered dependency/API substitutions (1), native
|
||||
discovery/bootstrap (2), resource-base queries (3), secondary loading and
|
||||
materialization data flow (4), roster ordering/rendering (5), prepare/resume
|
||||
serialization (6), archive/remote/restore paths (7), finalized snapshots (8),
|
||||
diff/provenance projection (9), examples (10), documentation (11), and assembled
|
||||
verification (12).
|
||||
|
||||
The chosen roster ordering, defaults-backed snapshot construction, native Diff,
|
||||
and required content validation remain accepted. Canonical rendering is linear
|
||||
in emitted data; the planned roster comparison sort is O(C log C) after deriving
|
||||
ordering keys. Unchanged algorithms such as pairwise party-name validation and
|
||||
prepare-input matching are not new findings from this migration-focused pass.
|
||||
|
||||
Cost analysis followed the released module's public API wrappers through native
|
||||
source lookup and metadata-copy implementations. Temporary functional Go-overlay
|
||||
checks validated source-enumeration and shared-metadata equivalence, ordinarily
|
||||
and under `-race`. Focused configuration, file-reader copy, S3 opened-object
|
||||
metadata, native inspection, and prepared-session publication checks passed on
|
||||
Go 1.27.4. No performance data, timing assertions, benchmark gates, or application
|
||||
changes were introduced.
|
||||
|
||||
Reference in New Issue
Block a user