Record algorithmic refactoring opportunities
This commit is contained in:
@@ -12,9 +12,15 @@ An integration-focused follow-up reviewed the same implementation at
|
|||||||
command, preparation, publication, restore, and inspection workflows. Items
|
command, preparation, publication, restore, and inspection workflows. Items
|
||||||
5–6 are additional findings from that pass; items 1–4 remain outstanding.
|
5–6 are additional findings from that pass; items 1–4 remain outstanding.
|
||||||
|
|
||||||
|
An algorithm-focused pass reviewed all twelve stages at
|
||||||
|
`583c5bdf1bc8f0af8ec2e257bcb03ecccbf283b1`. Items 7–10 are refactoring
|
||||||
|
opportunities for consideration. Complexity follows the code and the selected
|
||||||
|
ConfigKit v1.2.1 implementation; no operational timings or benchmarks were
|
||||||
|
collected. The pass preserves algorithms expressly chosen by the plan.
|
||||||
|
|
||||||
The native configuration architecture is substantially implemented. The items
|
The native configuration architecture is substantially implemented. The items
|
||||||
below are the remaining required corrections; this review does not implement
|
below record the six required corrections and four additional algorithmic
|
||||||
their fixes.
|
opportunities. This document implements none of their fixes.
|
||||||
|
|
||||||
## 1. P2: Distinguish effective resource origins from superseded history
|
## 1. P2: Distinguish effective resource origins from superseded history
|
||||||
|
|
||||||
@@ -198,6 +204,125 @@ the updated values. Include audio or previous-session selection as well as title
|
|||||||
Verify that restore still uses its selected immutable commit and that published
|
Verify that restore still uses its selected immutable commit and that published
|
||||||
objects are unchanged by the authoring update.
|
objects are unchanged by the authoring update.
|
||||||
|
|
||||||
|
## 7. P3: Reuse enumerated source records and one inspection source index
|
||||||
|
|
||||||
|
Stage: 9.
|
||||||
|
|
||||||
|
Locations: `internal/config/effective_pipeline.go:57–75`,
|
||||||
|
`internal/config/pipeline_configkit.go:354–371`, and
|
||||||
|
`internal/config/artifact_catalog.go`, `declarationDefinitionSources`.
|
||||||
|
|
||||||
|
`emptyImportSelections` enumerates `Result.Sources()` and then calls
|
||||||
|
`Result.Source(id)` for every enumerated record. `pipelineSourcePaths` does the
|
||||||
|
same lookup for every retained source. In ConfigKit v1.2.1, completed-result
|
||||||
|
source lookup scans the source slice. Enumerating S sources this way performs
|
||||||
|
Θ(S²) source-ID comparisons, although the enumerated records already contain
|
||||||
|
the required information, including empty-import evidence.
|
||||||
|
|
||||||
|
The artifact catalog additionally rebuilds this source map for each declaration.
|
||||||
|
For A entries this introduces O(A S²) source-index work; with one imported file
|
||||||
|
per artifact, that component can become cubic in A. This is separate from the
|
||||||
|
native history queries and necessary output construction.
|
||||||
|
|
||||||
|
Potential refactoring: consume the complete enumerated records directly. Build
|
||||||
|
the source-ID/path map once per inspection operation and share it with declaration
|
||||||
|
projection. This reduces empty-import enumeration to O(S + E), where E is the
|
||||||
|
returned selection evidence, and source-index work across the catalog to
|
||||||
|
expected O(S + A). Expanded members can also reuse the family-source lists
|
||||||
|
already retained in the family catalog. Keep native history and origin queries
|
||||||
|
as the provenance authority.
|
||||||
|
|
||||||
|
Verification: temporary functional checks confirmed that enumerated records and
|
||||||
|
individual lookups have equal contents, and that direct enumeration preserves
|
||||||
|
source paths and nested empty-import selections. Preserve operator source/catalog
|
||||||
|
output, including composite and inherited contributors, when refactoring.
|
||||||
|
|
||||||
|
## 8. P3: Reuse party metadata and per-character provenance within inspection
|
||||||
|
|
||||||
|
Stage: 9, consuming Stage 5's imported party result.
|
||||||
|
|
||||||
|
Locations: `internal/config/artifact_catalog.go:67–105` and
|
||||||
|
`internal/config/effective_pipeline.go`, `appendGeneratedArtifactSources`.
|
||||||
|
|
||||||
|
`partyCharacterSources` calls `result.Metadata()` separately for Lookup and
|
||||||
|
History. ConfigKit v1.2.1 returns detached metadata by copying the complete
|
||||||
|
effective-path and field maps, including their origin records. This helper is
|
||||||
|
called for each generated artifact in the catalog and for each generated
|
||||||
|
artifact field in source inspection. The same character's provenance is
|
||||||
|
therefore repeatedly recomputed across fields and families.
|
||||||
|
|
||||||
|
Let M be the copied party-metadata size and P the number of provenance requests.
|
||||||
|
The avoidable metadata-copy work and cumulative allocation are Θ(P M). Even
|
||||||
|
with one family and bounded-size character entries, M and P both grow with the
|
||||||
|
character count C, producing quadratic work. More families and fields multiply
|
||||||
|
the requests. This describes the provenance pass, not the complexity of the
|
||||||
|
whole command or unchanged party validation.
|
||||||
|
|
||||||
|
Potential refactoring: obtain one native Metadata snapshot and source index per
|
||||||
|
inspection operation. Resolve each requested character's source list once and
|
||||||
|
reuse it for that operation. The full-metadata copy term becomes O(M), with
|
||||||
|
native per-character query costs and required output emission remaining. Keep
|
||||||
|
the cache local to the projection; a new persistent provenance service is
|
||||||
|
unnecessary. Apply the same snapshot reuse to campaign provenance helpers where
|
||||||
|
convenient, although their fixed field count is not a quadratic case.
|
||||||
|
|
||||||
|
Verification: a temporary imported/inherited-roster check confirmed equivalent
|
||||||
|
source lists using shared native metadata and an enumerated source index.
|
||||||
|
Retain contributor accuracy, ordering, and secret-free inspection output.
|
||||||
|
|
||||||
|
## 9. P3: Read the selected remote session without enumerating run history
|
||||||
|
|
||||||
|
Stage: 7.
|
||||||
|
|
||||||
|
Location: `internal/app/config_loader.go:194–228`, `findRemoteSessionConfig`.
|
||||||
|
|
||||||
|
After selecting the committed prepared-session key, this function lists the
|
||||||
|
entire session prefix and scans it for that exact key. If H objects are retained
|
||||||
|
across historical runs, this adds Θ(H) listing/scan work and Θ(H) object-record
|
||||||
|
memory to every remote session load. The production S3 adapter paginates and
|
||||||
|
accumulates the complete listing. The scan predates the migration and was
|
||||||
|
retained in the new committed-object selection path.
|
||||||
|
|
||||||
|
Potential refactoring: retrieve the exact selected key through the existing
|
||||||
|
`ObjectStore.Read` boundary, which returns metadata with the opened body. Use
|
||||||
|
that metadata for the existing size/generation checks and stream the body into
|
||||||
|
the command-owned temporary file while retaining checksum verification. The
|
||||||
|
object lookup/transfer then takes a constant number of requests independent of
|
||||||
|
H, plus the necessary O(B) work for B session bytes. Current-commit acquisition
|
||||||
|
and validation costs remain. Apply this after reconciling the selection policy
|
||||||
|
in item 6; no new backend interface is needed.
|
||||||
|
|
||||||
|
Verification: the production S3 Read implementation performs one exact-key
|
||||||
|
GetObject and returns its size and generation. Preserve fake-store coverage for
|
||||||
|
missing objects, selected identity, checksum/size/generation mismatches, and
|
||||||
|
temporary-file cleanup. Confirm that unrelated historical keys are not enumerated.
|
||||||
|
|
||||||
|
## 10. P3: Stream prepared-input archival copies
|
||||||
|
|
||||||
|
Stage: 7.
|
||||||
|
|
||||||
|
Location: `internal/stage/publish_sources.go:29–64`,
|
||||||
|
`collectPublishPreparedInputs`.
|
||||||
|
|
||||||
|
After opening and verifying each prepared document, the new archival path reads
|
||||||
|
the whole file with `io.ReadAll` and passes the byte slice to
|
||||||
|
`writeBytesIfChanged`. For a document of B bytes this requires Θ(B) auxiliary
|
||||||
|
transfer memory. Expanded imported party documents can be larger than any one
|
||||||
|
authoring fragment. A full in-memory document is unnecessary for this file copy.
|
||||||
|
|
||||||
|
Potential refactoring: preserve the current confined source opening and checksum
|
||||||
|
checks, then stream the opened descriptor into the run archive. The existing
|
||||||
|
`fileops.WriteReaderAtomicWithChecksum` already supplies reader-based atomic
|
||||||
|
replacement and hashing through `io.Copy`. Preserve unchanged-destination reuse
|
||||||
|
using the known checksum. Transfer time remains Θ(B), while auxiliary copy
|
||||||
|
memory becomes O(1) with a fixed-size buffer. This does not alter the plan's
|
||||||
|
separate canonical YAML rendering algorithm or total in-memory configuration size.
|
||||||
|
|
||||||
|
Verification: preserve archived bytes/checksums, workspace file modes, source
|
||||||
|
validation, failure handling, and immutable publication behavior. Existing
|
||||||
|
reader-copy and prepared-session publication tests provide suitable behavioral
|
||||||
|
coverage; allocation or wall-clock thresholds are unnecessary.
|
||||||
|
|
||||||
## Stage-by-stage disposition
|
## Stage-by-stage disposition
|
||||||
|
|
||||||
| Stage | Review result |
|
| Stage | Review result |
|
||||||
@@ -206,11 +331,11 @@ objects are unchanged by the authoring update.
|
|||||||
| 2: Discovery/bootstrap | Native discovery, exclusive explicit selection, independent bootstrap, and shared diff bootstrap are present. |
|
| 2: Discovery/bootstrap | Native discovery, exclusive explicit selection, independent bootstrap, and shared diff bootstrap are present. |
|
||||||
| 3: Resource bases | Native origin-based resolution and fallbacks are present; field-record cleanup remains in item 3. |
|
| 3: Resource bases | Native origin-based resolution and fallbacks are present; field-record cleanup remains in item 3. |
|
||||||
| 4: Campaign/session imports | File contexts, contextless remote snapshots, native optional discovery, and detached document values are present; initialization still uses the snapshot path in item 5. |
|
| 4: Campaign/session imports | File contexts, contextless remote snapshots, native optional discovery, and detached document values are present; initialization still uses the snapshot path in item 5. |
|
||||||
| 5: Party imports | Imported rosters, source-aware ordering, canonical rendering, players, and family resolution are present. |
|
| 5: Party imports | Imported rosters, source-aware ordering, canonical rendering, players, and family resolution are present; the planned roster sort is retained. |
|
||||||
| 6: Prepared inputs/resume | Canonical document materialization and matching resume evidence are present. |
|
| 6: Prepared inputs/resume | Canonical document materialization and matching resume evidence are present. |
|
||||||
| 7: Publish/restore | Prepared documents flow through immutable committed run objects and restore; remote initialization and committed selection conflict in item 6. |
|
| 7: Publish/restore | Prepared documents flow through immutable committed run objects and restore; item 6 is a correctness gap and items 9–10 are scalability opportunities. |
|
||||||
| 8: Final-domain snapshots | Defaults-backed native snapshots, safe credential identities, rendering, and digest integration are present. |
|
| 8: Final-domain snapshots | Defaults-backed native snapshots, safe credential identities, rendering, and digest integration are present. |
|
||||||
| 9: Diff/provenance | Native diff and pointer paths are present; effective/history attribution needs item 1. |
|
| 9: Diff/provenance | Native diff and pointer paths are present; item 1 concerns attribution correctness and items 7–8 concern avoidable provenance work. |
|
||||||
| 10: Examples | Destination-based definitions, directory imports, list operations, split roster, and secondary imports are demonstrated. |
|
| 10: Examples | Destination-based definitions, directory imports, list operations, split roster, and secondary imports are demonstrated. |
|
||||||
| 11: Documentation/policy | Current references cover the migration; finish attribution, source-role, and initialization/selection corrections above. |
|
| 11: Documentation/policy | Current references cover the migration; finish attribution, source-role, and initialization/selection corrections above. |
|
||||||
| 12: Final verification | Existing integration checks pass; add coverage for items 5–6. Full race validation remains unreliable because of item 2. |
|
| 12: Final verification | Existing integration checks pass; add coverage for items 5–6. Full race validation remains unreliable because of item 2. |
|
||||||
@@ -255,3 +380,26 @@ Using Go 1.27.4 at the follow-up revision:
|
|||||||
|
|
||||||
Only this remediation document changed. Diagnostic files remained outside the
|
Only this remediation document changed. Diagnostic files remained outside the
|
||||||
repository, and object storage used an in-memory fake.
|
repository, and object storage used an in-memory fake.
|
||||||
|
|
||||||
|
## Algorithm-focused review coverage and validation
|
||||||
|
|
||||||
|
The stage-by-stage pass covered dependency/API substitutions (1), native
|
||||||
|
discovery/bootstrap (2), resource-base queries (3), secondary loading and
|
||||||
|
materialization data flow (4), roster ordering/rendering (5), prepare/resume
|
||||||
|
serialization (6), archive/remote/restore paths (7), finalized snapshots (8),
|
||||||
|
diff/provenance projection (9), examples (10), documentation (11), and assembled
|
||||||
|
verification (12).
|
||||||
|
|
||||||
|
The chosen roster ordering, defaults-backed snapshot construction, native Diff,
|
||||||
|
and required content validation remain accepted. Canonical rendering is linear
|
||||||
|
in emitted data; the planned roster comparison sort is O(C log C) after deriving
|
||||||
|
ordering keys. Unchanged algorithms such as pairwise party-name validation and
|
||||||
|
prepare-input matching are not new findings from this migration-focused pass.
|
||||||
|
|
||||||
|
Cost analysis followed the released module's public API wrappers through native
|
||||||
|
source lookup and metadata-copy implementations. Temporary functional Go-overlay
|
||||||
|
checks validated source-enumeration and shared-metadata equivalence, ordinarily
|
||||||
|
and under `-race`. Focused configuration, file-reader copy, S3 opened-object
|
||||||
|
metadata, native inspection, and prepared-session publication checks passed on
|
||||||
|
Go 1.27.4. No performance data, timing assertions, benchmark gates, or application
|
||||||
|
changes were introduced.
|
||||||
|
|||||||
Reference in New Issue
Block a user