Record ConfigKit source lookup scaling opportunity
This commit is contained in:
@@ -1,7 +1,7 @@
|
|||||||
# Configuration Upgrade Review Remediation
|
# Configuration Upgrade Review Remediation
|
||||||
|
|
||||||
The earlier PromptKit findings below retain their original scope and status.
|
The earlier PromptKit findings below retain their original scope and status.
|
||||||
The ConfigKit 1.2.0 review and subsequent integration audit appear below; they
|
The ConfigKit 1.2.0 review and subsequent focused audits appear below; they
|
||||||
do not reopen or attribute earlier issues to that upgrade.
|
do not reopen or attribute earlier issues to that upgrade.
|
||||||
|
|
||||||
## Scope And Overall Assessment
|
## Scope And Overall Assessment
|
||||||
@@ -405,3 +405,104 @@ is indicated by the current evidence.
|
|||||||
- The full-suite, vet, race, tidy, and build results from the preceding review
|
- The full-suite, vet, race, tidy, and build results from the preceding review
|
||||||
remain applicable to the unchanged implementation. No live provider call or
|
remain applicable to the unchanged implementation. No live provider call or
|
||||||
production-quality evaluation was performed.
|
production-quality evaluation was performed.
|
||||||
|
|
||||||
|
## ConfigKit Algorithmic Audit
|
||||||
|
|
||||||
|
Reviewed on 2026-11-18 against `b396581`. Production code remains identical to
|
||||||
|
`c9744ad`. This pass examines the 11 stages for algorithmic correctness and
|
||||||
|
avoidable scaling costs; it leaves the explicitly selected native composition,
|
||||||
|
inheritance, discovery, and ownership algorithms in place. CK-1 through CK-3
|
||||||
|
retain their earlier scope and status.
|
||||||
|
|
||||||
|
### Finding CK-4 — Memoize Source Directories During Path Normalization
|
||||||
|
|
||||||
|
Priority: P3. Category: efficiency/scalability opportunity. Status: open.
|
||||||
|
Affected stages: 2–3; exposed by adoption of richer ConfigKit source metadata.
|
||||||
|
|
||||||
|
`internal/core/config/document_paths.go:144–176`, `documentPathBase`, calls
|
||||||
|
`resolution.Source(sourceID)` for each file-authored relative path. Only the
|
||||||
|
source's `Path` is needed to calculate its directory. The same source is queried
|
||||||
|
again for every reference or destination it supplies.
|
||||||
|
|
||||||
|
In ConfigKit v1.2.0, `Source` returns a detached record that includes
|
||||||
|
`EmptyImportSelections`: one record for each completed optional import declaration
|
||||||
|
that selected no source. The pinned implementation clones these records and
|
||||||
|
their paths on each query:
|
||||||
|
|
||||||
|
- `lifecycle.go:287–290`: public resolution-source lookup and conversion;
|
||||||
|
- `internal/engine/load_metadata.go:47–52`: indexed lookup followed by a detached copy;
|
||||||
|
- `internal/engine/records.go:333–345`: copying of the empty-selection evidence;
|
||||||
|
- `conversion.go:45–66`: conversion of that evidence to the public record.
|
||||||
|
|
||||||
|
The older v0.5.0 source record did not contain this variable-sized evidence.
|
||||||
|
The per-field lookup pattern predates the upgrade, but its snapshot cost now
|
||||||
|
depends on metadata unrelated to the particular path being normalized. The
|
||||||
|
source-ID lookup itself is indexed; the repeated copying is the avoidable work.
|
||||||
|
|
||||||
|
Complexity, disregarding path-string lengths:
|
||||||
|
|
||||||
|
- Let `R` be the number of relative fields supplied by one source and `E` its
|
||||||
|
number of empty optional-import selections. Repeated source lookup contributes
|
||||||
|
`O(R * (1 + E))` time and cumulative allocation volume. With `R` and `E` growing
|
||||||
|
together, this component is quadratic in their combined count.
|
||||||
|
- A load-local cache of source ID to its directory/lookup result reduces this
|
||||||
|
component to `O(R + 1 + E)` for one source. Across visited sources, the bound is
|
||||||
|
`O(R + U + sum(E_i))`, with `O(U)` small cached entries for `U` source IDs.
|
||||||
|
- The cache should retain only the needed path-derived result, not every full
|
||||||
|
source snapshot. One-time snapshotting for inspection and main-source selection
|
||||||
|
can remain unchanged.
|
||||||
|
|
||||||
|
Recommended refactoring:
|
||||||
|
|
||||||
|
- Memoize the source-path/directory lookup within `documentLoadState`, scoped to
|
||||||
|
one load and its resolution callback. Keep per-field `EntryInfo` queries so
|
||||||
|
each surviving declaration still chooses its own source correctly.
|
||||||
|
- Preserve CLI-invocation-relative, default-main-relative, imported/inherited,
|
||||||
|
missing-source/non-file fallback, absolute-path, and empty-cache behavior.
|
||||||
|
Memoization should reuse the native immutable source evidence without changing
|
||||||
|
source acquisition or application path policy.
|
||||||
|
- Reuse the existing path-origin and ownership tests. Verify the lookup is shared
|
||||||
|
across paths from one source and remains distinct across sources; no elapsed-time
|
||||||
|
assertion, real-world benchmark, new input limit, or release gate is warranted.
|
||||||
|
|
||||||
|
Acceptance: paths from the same source no longer repeatedly clone its unrelated
|
||||||
|
import-selection evidence. Runtime paths, inspection results, and load isolation
|
||||||
|
remain unchanged. This is a bounded optimization opportunity, not a correctness
|
||||||
|
failure or a claim that ordinary inference runs will become materially faster.
|
||||||
|
For sources with no empty selections, the current lookup cost is already small.
|
||||||
|
|
||||||
|
### Stage-By-Stage Algorithmic Assessment
|
||||||
|
|
||||||
|
| Stage | Assessment |
|
||||||
|
| --- | --- |
|
||||||
|
| 1 — Dependencies/toolchain | The mixed released graph is an approved choice; no new Notarius runtime algorithm was introduced here. |
|
||||||
|
| 2 — API/handoff | LoadSource is a single delegation. MainSourcePath makes one source-metadata snapshot and scan, linear in that snapshot's size, rather than repeating discovery. CK-4 concerns repeated callback lookups, not this one-time accessor. |
|
||||||
|
| 3 — Normalization/origins | Local passes visit each contained pipeline, step, lane, validator, and reference a bounded number of times. Deterministic key sorting costs `O(k log k)` per map. CK-4 identifies a hidden multiplicative snapshot cost inside those traversals. |
|
||||||
|
| 4 — Collections | Identity registration delegates to the selected native algorithms. No application-side step matcher, concatenation loop, or quadratic merge was added. |
|
||||||
|
| 5 — Fixtures | Mechanical fixture migration introduces no production algorithm. Previously recorded correctness/coverage findings remain separate. |
|
||||||
|
| 6 — CLI discovery | Source selection has a fixed number of local branches and native preset descriptions; manual probes were removed. Selected-source reporting is performed once. |
|
||||||
|
| 7 — Discovery/inspection coverage | The new tests exercise bounded cases; reports delegate to retained inspection. No extra load or source search was added for provenance. |
|
||||||
|
| 8 — Examples | The extra fragment uses native dependency ordering and does not add application composition machinery. |
|
||||||
|
| 9 — Operator docs | No runtime algorithm introduced. CK-1 and CK-2 already cover the relevant prose inaccuracies. |
|
||||||
|
| 10 — Internals/notices | No runtime algorithm introduced or new dependency-graph translation added. |
|
||||||
|
| 11 — Final verification | No production algorithm introduced; existing tests support correctness but do not establish complexity bounds. |
|
||||||
|
|
||||||
|
### Algorithmic Audit Evidence And Limits
|
||||||
|
|
||||||
|
- Inspected the production diff and traced traversal, original-path remapping,
|
||||||
|
recording, source lookup, and CLI handoff helpers through the code graph.
|
||||||
|
Nested collection loops traverse separate children rather than repeatedly
|
||||||
|
scanning one global collection. Original-path remapping is bypassed during the
|
||||||
|
pre-move path pass; its later worker-presence lookup has a fixed schema path.
|
||||||
|
- Inspected the exact ConfigKit v1.2.0 source for snapshot ownership costs. Its
|
||||||
|
record aggregation uses indexed reason/dependency sets, so repeated ancestor
|
||||||
|
recording was not misclassified as a quadratic scan.
|
||||||
|
- An offline temporary Go-overlay test verified that a single declaring-source
|
||||||
|
snapshot contains four empty optional-import records while multiple relative
|
||||||
|
reference paths normalize correctly. The probe passed; its files remain outside
|
||||||
|
the repository. No production code or repository tests were modified.
|
||||||
|
- Complexity conclusions come from source/data-flow analysis. No production
|
||||||
|
inference, timing benchmark, or real-world performance measurement was used.
|
||||||
|
Existing validation results remain applicable to the unchanged implementation.
|
||||||
|
|
||||||
|
No additional ordinary correctness bug was identified in this pass.
|
||||||
|
|||||||
Reference in New Issue
Block a user