Record ConfigKit source lookup scaling opportunity

This commit is contained in:
2026-11-18 00:25:26 +00:00
parent b396581d21
commit 068a4b1b5a

View File

@@ -1,7 +1,7 @@
# Configuration Upgrade Review Remediation # Configuration Upgrade Review Remediation
The earlier PromptKit findings below retain their original scope and status. The earlier PromptKit findings below retain their original scope and status.
The ConfigKit 1.2.0 review and subsequent integration audit appear below; they The ConfigKit 1.2.0 review and subsequent focused audits appear below; they
do not reopen or attribute earlier issues to that upgrade. do not reopen or attribute earlier issues to that upgrade.
## Scope And Overall Assessment ## Scope And Overall Assessment
@@ -405,3 +405,104 @@ is indicated by the current evidence.
- The full-suite, vet, race, tidy, and build results from the preceding review - The full-suite, vet, race, tidy, and build results from the preceding review
remain applicable to the unchanged implementation. No live provider call or remain applicable to the unchanged implementation. No live provider call or
production-quality evaluation was performed. production-quality evaluation was performed.
## ConfigKit Algorithmic Audit
Reviewed on 2026-11-18 against `b396581`. Production code remains identical to
`c9744ad`. This pass examines the 11 stages for algorithmic correctness and
avoidable scaling costs; it leaves the explicitly selected native composition,
inheritance, discovery, and ownership algorithms in place. CK-1 through CK-3
retain their earlier scope and status.
### Finding CK-4 — Memoize Source Directories During Path Normalization
Priority: P3. Category: efficiency/scalability opportunity. Status: open.
Affected stages: 2–3; exposed by adoption of richer ConfigKit source metadata.
`internal/core/config/document_paths.go:144–176`, `documentPathBase`, calls
`resolution.Source(sourceID)` for each file-authored relative path. Only the
source's `Path` is needed to calculate its directory. The same source is queried
again for every reference or destination it supplies.
In ConfigKit v1.2.0, `Source` returns a detached record that includes
`EmptyImportSelections`: one record for each completed optional import declaration
that selected no source. The pinned implementation clones these records and
their paths on each query:
- `lifecycle.go:287–290`: public resolution-source lookup and conversion;
- `internal/engine/load_metadata.go:47–52`: indexed lookup followed by a detached copy;
- `internal/engine/records.go:333–345`: copying of the empty-selection evidence;
- `conversion.go:45–66`: conversion of that evidence to the public record.
The older v0.5.0 source record did not contain this variable-sized evidence.
The per-field lookup pattern predates the upgrade, but its snapshot cost now
depends on metadata unrelated to the particular path being normalized. The
source-ID lookup itself is indexed; the repeated copying is the avoidable work.
Complexity, disregarding path-string lengths:
- Let `R` be the number of relative fields supplied by one source and `E` its
number of empty optional-import selections. Repeated source lookup contributes
`O(R * (1 + E))` time and cumulative allocation volume. With `R` and `E` growing
together, this component is quadratic in their combined count.
- A load-local cache of source ID to its directory/lookup result reduces this
component to `O(R + 1 + E)` for one source. Across visited sources, the bound is
`O(R + U + sum(E_i))`, with `O(U)` small cached entries for `U` source IDs.
- The cache should retain only the needed path-derived result, not every full
source snapshot. One-time snapshotting for inspection and main-source selection
can remain unchanged.
Recommended refactoring:
- Memoize the source-path/directory lookup within `documentLoadState`, scoped to
one load and its resolution callback. Keep per-field `EntryInfo` queries so
each surviving declaration still chooses its own source correctly.
- Preserve CLI-invocation-relative, default-main-relative, imported/inherited,
missing-source/non-file fallback, absolute-path, and empty-cache behavior.
Memoization should reuse the native immutable source evidence without changing
source acquisition or application path policy.
- Reuse the existing path-origin and ownership tests. Verify the lookup is shared
across paths from one source and remains distinct across sources; no elapsed-time
assertion, real-world benchmark, new input limit, or release gate is warranted.
Acceptance: paths from the same source no longer repeatedly clone its unrelated
import-selection evidence. Runtime paths, inspection results, and load isolation
remain unchanged. This is a bounded optimization opportunity, not a correctness
failure or a claim that ordinary inference runs will become materially faster.
For sources with no empty selections, the current lookup cost is already small.
### Stage-By-Stage Algorithmic Assessment
| Stage | Assessment |
| --- | --- |
| 1 — Dependencies/toolchain | The mixed released graph is an approved choice; no new Notarius runtime algorithm was introduced here. |
| 2 — API/handoff | LoadSource is a single delegation. MainSourcePath makes one source-metadata snapshot and scan, linear in that snapshot's size, rather than repeating discovery. CK-4 concerns repeated callback lookups, not this one-time accessor. |
| 3 — Normalization/origins | Local passes visit each contained pipeline, step, lane, validator, and reference a bounded number of times. Deterministic key sorting costs `O(k log k)` per map. CK-4 identifies a hidden multiplicative snapshot cost inside those traversals. |
| 4 — Collections | Identity registration delegates to the selected native algorithms. No application-side step matcher, concatenation loop, or quadratic merge was added. |
| 5 — Fixtures | Mechanical fixture migration introduces no production algorithm. Previously recorded correctness/coverage findings remain separate. |
| 6 — CLI discovery | Source selection has a fixed number of local branches and native preset descriptions; manual probes were removed. Selected-source reporting is performed once. |
| 7 — Discovery/inspection coverage | The new tests exercise bounded cases; reports delegate to retained inspection. No extra load or source search was added for provenance. |
| 8 — Examples | The extra fragment uses native dependency ordering and does not add application composition machinery. |
| 9 — Operator docs | No runtime algorithm introduced. CK-1 and CK-2 already cover the relevant prose inaccuracies. |
| 10 — Internals/notices | No runtime algorithm introduced or new dependency-graph translation added. |
| 11 — Final verification | No production algorithm introduced; existing tests support correctness but do not establish complexity bounds. |
### Algorithmic Audit Evidence And Limits
- Inspected the production diff and traced traversal, original-path remapping,
recording, source lookup, and CLI handoff helpers through the code graph.
Nested collection loops traverse separate children rather than repeatedly
scanning one global collection. Original-path remapping is bypassed during the
pre-move path pass; its later worker-presence lookup has a fixed schema path.
- Inspected the exact ConfigKit v1.2.0 source for snapshot ownership costs. Its
record aggregation uses indexed reason/dependency sets, so repeated ancestor
recording was not misclassified as a quadratic scan.
- An offline temporary Go-overlay test verified that a single declaring-source
snapshot contains four empty optional-import records while multiple relative
reference paths normalize correctly. The probe passed; its files remain outside
the repository. No production code or repository tests were modified.
- Complexity conclusions come from source/data-flow analysis. No production
inference, timing benchmark, or real-world performance measurement was used.
Existing validation results remain applicable to the unchanged implementation.
No additional ordinary correctness bug was identified in this pass.