Add dispatch and registration benchmarks

This commit is contained in:
2026-10-09 01:19:03 +00:00
parent 1274ae71c0
commit aed4c5921c
3 changed files with 291 additions and 3 deletions

View File

@@ -1,6 +1,6 @@
# Performance implementation plan
Status: ready for implementation; Stages 1–11 have not started.
Status: implementation in progress; Stage 1 is complete, Stages 2–11 have not started.
Date: 2026-10-09.
@@ -150,7 +150,7 @@ benchmark-analysis dependency or permanent numeric pass threshold is selected.
## Stage 1: establish dispatch and registration measurements
**Status:** not started. **Prerequisite:** orientation and this plan.
**Status:** complete. **Prerequisite:** orientation and this plan.
**Files:** new `performance_dispatch_test.go`,
`internal/cobra/performance_test.go`, and evidence here. Production code stays
@@ -182,7 +182,49 @@ production instrumentation, or full dependency test matrix is needed.
**Completion:** reproducible baseline results distinguish registration, cold
execution, repeated execution, parsing, and help/usage costs.
**Evidence:** pending.
**Evidence:** Benchmark fixtures were measured from source revision `1274ae7`
with the new benchmark files in the working tree (committed with this evidence).
Environment: Go 1.26.8, linux/amd64, Linux 6.17.0-41-generic, Intel Core
i5-7260U CPU @ 2.20 GHz. The required compile-only command passed:
`GOWORK=off GOTOOLCHAIN=local GOPROXY=off GONOPROXY=none GOSUMDB=off go test -mod=readonly -run '^$' . ./internal/cobra`.
The five-sample measurement command also passed:
`GOWORK=off GOTOOLCHAIN=local GOPROXY=off GONOPROXY=none GOSUMDB=off go test -mod=readonly -run '^$' -bench '^BenchmarkPerformance' -benchmem -benchtime=200ms -count=5 . ./internal/cobra`.
Raw output is in `/tmp/commandkit-performance-baseline.txt`; focused repeat output
for noisy cases is in `/tmp/commandkit-performance-baseline-repeat.txt`.
Fixtures: root dispatch used 8 and 128 flat commands, a 24-level path with
siblings at each parent, and a leaf with 24 flags. Dispatch covered success,
local and explicit help, parser and handler usage, cold construction plus first
execution, and repeated execution. Registration used 8 and 128 prebuilt
declarations with fresh `New` in individual and batch paths. Adapter parsing
used 128 shallow siblings and the same depth-24 path. All repeated app and
adapter fixtures were built outside the timed loops; benchmark outcomes were
checked for their intended classes.
Representative five-sample ranges (ns/op; B/op; allocs/op): small repeated
success 31,066–41,041; 33,209–33,210; 287. Small local help 103,022–116,183;
78,176–78,177; 810. Parser usage 62,596–74,475; 66,363–66,364; 591. Wide
success 218,073–253,010; 269,063–269,070; 1,898. Deep success 119,958–139,625;
139,787–139,791; 1,074. Wide help 491,172–622,188; 549,923–549,940; 4,033.
Explicit help measured 101,998–111,895 ns/op, 78,152–78,153 B/op, and 815
allocs/op; handler usage measured 61,878–72,264 ns/op, 66,211–66,212 B/op, and
583 allocs/op. Cold construction plus first execution measured 38,152–50,235
ns/op, 38,275–38,276 B/op, and 339 allocs/op for the small app, and
251,945–315,276 ns/op, 291,822–291,827 B/op, and 2,184 allocs/op for the wide
app. The 24-flag leaf focused repeat was 43,662–56,492 ns/op, 40,027–40,028
B/op, and 401 allocs/op.
Deep help was noisy in the full run (429,873–1,634,363 ns/op); a focused repeat
was 300,155–326,094 ns/op, 294,882–294,896 B/op, and 2,408 allocs/op. The initial
flag-heavy timing also varied widely before the focused range reported above.
Registration at 8 declarations measured individual 7,141–7,645 ns/op,
5,497 B/op, 64 allocs/op versus batch 4,159–5,190 ns/op, 3,456 B/op,
48 allocs/op. At 128 declarations, individual measured 414,618–492,261 ns/op,
405,011–405,018 B/op, 784 allocs/op versus batch 33,985–36,207 ns/op,
23,013–23,014 B/op, 288 allocs/op. Adapter parse measured shallow/wide
150,918–171,598 ns/op, 242,936 B/op, 1,705 allocs/op, and deep
86,282–109,697 ns/op, 119,920 B/op, 827 allocs/op. These are baselines, not
speedup claims; the large initial deep-help and flag-heavy timing variation
shows why later comparisons should report run variation.
## Stage 2: establish report and scope measurements