Complete the performance audit and update documentation

This commit is contained in:
2026-10-09 01:54:31 +00:00
parent 62b4a48e4f
commit 18ff0d6639
8 changed files with 157 additions and 37 deletions

View File

@@ -1,6 +1,6 @@
# Performance implementation plan
Status: implementation in progress; Stages 1–8 are complete, Stages 9–11 have not started.
Status: complete; Stages 1–11 are complete.
Date: 2026-10-09.
@@ -876,7 +876,7 @@ are observations, not thresholds.
## Stage 11: audit the completed target and reconcile documentation
**Status:** not started. **Prerequisite:** Stages 1–10.
**Status:** complete. **Prerequisite:** Stages 1–10.
**Files:** this plan, `performance.md`,
[internal overview](../internal/overview.md), developer/README navigation,
@@ -925,4 +925,103 @@ implemented with relevant behavioral and efficiency evidence; maintained docs
identify their current owners. No performance guarantee or additional release
gate has been introduced.
**Evidence:** pending.
**Evidence:** Audited PF-01–PF-06 against the current implementation and
behavioral coverage. PF-01 selects the PromptKit renderer before serialization;
PF-02 indexes backend commands to declarations and uses parent links for the
selected path; PF-03 caches immutable bridge metadata per `App` and uses one
mutable adapter session per `Execute`, including help and usage; PF-04 scans
only YAML framing lines but retains complete framed bytes; PF-05 validates and
clones pending registrations before atomically installing them; PF-06 prepares
owned ordered scope metadata, lookup, and discovery data at registration. The
existing focused tests cover refresh after registration, parser/help/usage
session reuse, YAML boundaries/directives, registration rollback, prepared
scope ownership, selector collisions, and callback timing. OD-1 and OD-2 remain
as selected: backend state is fresh per invocation, and YAML output retains a
full framed buffer.
The default sprint suite passed on source revision `62b4a48` with
`GOWORK=off GOTOOLCHAIN=local GOPROXY=off GONOPROXY=none GOSUMDB=off go test
-mod=readonly -count=1 ./...`. Packages `commandkit`, `internal/cobra`, and
`scripts` passed; both examples compiled. `go list` and `go doc -all .` confirmed
that Cobra/pflag remain confined to `internal/cobra`, the adapter does not
import the root package, and the root exported declarations retain their
existing signatures. Review of the sprint Go diff found no changed or added
root exported declarations.
Final representative measurements used Go 1.26.8, linux/amd64, Linux
6.17.0-41-generic, Intel Core i5-7260U CPU @ 2.20 GHz, source revision
`62b4a48`, and the unchanged fixtures from earlier stages. The five-sample
command passed:
`GOWORK=off GOTOOLCHAIN=local GOPROXY=off GONOPROXY=none GOSUMDB=off go test
-mod=readonly -run '^$' -bench '^BenchmarkPerformance' -benchmem
-benchtime=200ms -count=5 . ./internal/cobra`. Raw output is in
`/tmp/commandkit-performance-final.txt`. Dispatch, prepared-scope, and scope
registration benchmarks were then repeated with the same five-sample settings.
Shortly after the measurements, the host load average was 3.62 on four logical
CPUs; historical stage measurements did not record load. Cross-run timing
comparisons are therefore inconclusive, while bytes and allocation counts
remained stable.
- **PF-01:** For 128 prompt inputs, final text measured 159,091–172,388 ns/op,
126,782–126,787 B/op, and 1,143 allocations; YAML measured 1,240,053–1,534,668
ns/op, 1,262,298–1,262,330 B/op, and 5,079 allocations. Stage 2 recorded
145,524–145,531 B and 1,171 allocations for text and 1,393,508–1,393,568 B
and 6,039–6,040 allocations for YAML. Four-input YAML is now 69,677–69,678 B
and 480 allocations versus 74,730–74,733 B and 557 at baseline. Selected
rendering reduces report work and allocations; profile-text timings remain
noisy.
- **PF-02:** Final adapter Parse measured 242,960–242,969 B/op and 1,706
allocations for the shallow/wide fixture, and 119,560 B/op and 827
allocations for the deep fixture. Stage 4's isolated change reduced deep
Parse by 384 B and one allocation; later session work adds a small wrapper
cost. Tree construction remains in these Parse measurements; selected-path
lookup is recorded separately in Stage 4.
- **PF-03:** Final repeated success measured 53,796–61,372 ns/op,
29,580–29,581 B/op, and 269 allocations. Local and explicit help used
41.4–41.9 KB and 509/516 allocations; parser and handler usage used about
30 KB and 292/284 allocations. Stage 9 measured 38,671–44,179 ns/op for
repeated success and about 41.4–41.9 KB with the same help allocation counts.
A focused repeat remained slower in time while allocation results stayed
stable. The host load noted above and the absence of a prior load record
prevent attributing this timing difference to the source change. Cold
construction plus first execution measured 84,000–93,954 ns/op and
38,418–38,419 B for a small app, and 616,203–678,200 ns/op and about 305 KB
for a wide app. The cached neutral tree still requires construction once per
metadata generation, and mutable backend trees are still built per invocation.
- **PF-04:** Direct 4 KiB framing measured 1,193–1,256 ns/op, 4,864 B/op, and
one allocation; direct 1 MiB framing measured 188,617–561,502 ns/op,
1,056,768–1,056,770 B/op, and one allocation. The 1 MiB aggregate writer
measured 1,624,170–1,806,960 ns/op, about 4.245 MB, and 185 allocations,
versus the Stage 2 baseline of about 8.695 MB and 215–216 allocations. The
full framed copy remains part of the selected OD-2 behavior.
- **PF-05:** Final 128-entry individual registration measured 101,116–118,013
ns/op, 29,420–29,421 B/op, and 293 allocations, compared with the Stage 1
baseline of 414,618–492,261 ns/op, about 405 KB, and 784 allocations. A
128-entry batch measured 104,559–118,844 ns/op, about 35,992 B/op, and 296
allocations, retaining the documented extra pending-index and map-growth
cost compared with the smaller Stage 1 batch allocation. The focused run
confirms this remains an allocation/timing tradeoff, not a correctness issue.
- **PF-06:** Final two-scope discovery, named selection, and aggregate
validation measured 33,003–34,199, 38,275–45,600, and 44,677–65,937 ns/op,
respectively; allocation counts were 195, 207, and 208. At 32 scopes,
discovery used 199 allocations, named selection 207, and aggregate validation
240. The focused repeat also remained slower than Stage 10 measurements while
retaining the same allocation counts. Compared with Stage 2,
retained metadata reduces two-scope discovery from 21,161 B/223 allocations
to 18,559–18,560 B/195, and 32-scope aggregate validation from 29,781–29,782
B/269 to 22,705–22,706 B/240. The prepared suite retains ordered scopes,
lookup, and discovery data; registration cost grows accordingly.
The runbook remains the owner of release operations, the ReleaseKit roadmap
owns adoption status, and this performance roadmap and record own PF-01–PF-06.
Updated the internal overview, README, and contributor navigation; replaced
obsolete ReleaseKit-plan links in the releasekit, initial, and remediation
records with the completed historical adoption record or current owner. Local
Markdown links and fragments passed checks across all eight affected documents,
and `git diff --check` passed. The Stage 1–10 implementation commits are
`aed4c59`, `a431540`, `628dc78`, `ae656d8`, `3f8841f`, `5816dd3`, `3ad3559`,
`a852301`, `a7960e5`, and `62b4a48`; the final audit is recorded in this commit.
Remaining costs are full backend tree construction for each invocation and a
full framed YAML buffer for delivery. No public performance guarantee or new
release gate was added.