Add dispatch and registration benchmarks
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Performance implementation plan
|
||||
|
||||
Status: ready for implementation; Stages 1–11 have not started.
|
||||
Status: implementation in progress; Stage 1 is complete, Stages 2–11 have not started.
|
||||
|
||||
Date: 2026-10-09.
|
||||
|
||||
@@ -150,7 +150,7 @@ benchmark-analysis dependency or permanent numeric pass threshold is selected.
|
||||
|
||||
## Stage 1: establish dispatch and registration measurements
|
||||
|
||||
**Status:** not started. **Prerequisite:** orientation and this plan.
|
||||
**Status:** complete. **Prerequisite:** orientation and this plan.
|
||||
|
||||
**Files:** new `performance_dispatch_test.go`,
|
||||
`internal/cobra/performance_test.go`, and evidence here. Production code stays
|
||||
@@ -182,7 +182,49 @@ production instrumentation, or full dependency test matrix is needed.
|
||||
**Completion:** reproducible baseline results distinguish registration, cold
|
||||
execution, repeated execution, parsing, and help/usage costs.
|
||||
|
||||
**Evidence:** pending.
|
||||
**Evidence:** Benchmark fixtures were measured from source revision `1274ae7`
|
||||
with the new benchmark files in the working tree (committed with this evidence).
|
||||
Environment: Go 1.26.8, linux/amd64, Linux 6.17.0-41-generic, Intel Core
|
||||
i5-7260U CPU @ 2.20 GHz. The required compile-only command passed:
|
||||
`GOWORK=off GOTOOLCHAIN=local GOPROXY=off GONOPROXY=none GOSUMDB=off go test -mod=readonly -run '^$' . ./internal/cobra`.
|
||||
The five-sample measurement command also passed:
|
||||
`GOWORK=off GOTOOLCHAIN=local GOPROXY=off GONOPROXY=none GOSUMDB=off go test -mod=readonly -run '^$' -bench '^BenchmarkPerformance' -benchmem -benchtime=200ms -count=5 . ./internal/cobra`.
|
||||
Raw output is in `/tmp/commandkit-performance-baseline.txt`; focused repeat output
|
||||
for noisy cases is in `/tmp/commandkit-performance-baseline-repeat.txt`.
|
||||
|
||||
Fixtures: root dispatch used 8 and 128 flat commands, a 24-level path with
|
||||
siblings at each parent, and a leaf with 24 flags. Dispatch covered success,
|
||||
local and explicit help, parser and handler usage, cold construction plus first
|
||||
execution, and repeated execution. Registration used 8 and 128 prebuilt
|
||||
declarations with fresh `New` in individual and batch paths. Adapter parsing
|
||||
used 128 shallow siblings and the same depth-24 path. All repeated app and
|
||||
adapter fixtures were built outside the timed loops; benchmark outcomes were
|
||||
checked for their intended classes.
|
||||
|
||||
Representative five-sample ranges (ns/op; B/op; allocs/op): small repeated
|
||||
success 31,066–41,041; 33,209–33,210; 287. Small local help 103,022–116,183;
|
||||
78,176–78,177; 810. Parser usage 62,596–74,475; 66,363–66,364; 591. Wide
|
||||
success 218,073–253,010; 269,063–269,070; 1,898. Deep success 119,958–139,625;
|
||||
139,787–139,791; 1,074. Wide help 491,172–622,188; 549,923–549,940; 4,033.
|
||||
Explicit help measured 101,998–111,895 ns/op, 78,152–78,153 B/op, and 815
|
||||
allocs/op; handler usage measured 61,878–72,264 ns/op, 66,211–66,212 B/op, and
|
||||
583 allocs/op. Cold construction plus first execution measured 38,152–50,235
|
||||
ns/op, 38,275–38,276 B/op, and 339 allocs/op for the small app, and
|
||||
251,945–315,276 ns/op, 291,822–291,827 B/op, and 2,184 allocs/op for the wide
|
||||
app. The 24-flag leaf focused repeat was 43,662–56,492 ns/op, 40,027–40,028
|
||||
B/op, and 401 allocs/op.
|
||||
Deep help was noisy in the full run (429,873–1,634,363 ns/op); a focused repeat
|
||||
was 300,155–326,094 ns/op, 294,882–294,896 B/op, and 2,408 allocs/op. The initial
|
||||
flag-heavy timing also varied widely before the focused range reported above.
|
||||
Registration at 8 declarations measured individual 7,141–7,645 ns/op,
|
||||
5,497 B/op, 64 allocs/op versus batch 4,159–5,190 ns/op, 3,456 B/op,
|
||||
48 allocs/op. At 128 declarations, individual measured 414,618–492,261 ns/op,
|
||||
405,011–405,018 B/op, 784 allocs/op versus batch 33,985–36,207 ns/op,
|
||||
23,013–23,014 B/op, 288 allocs/op. Adapter parse measured shallow/wide
|
||||
150,918–171,598 ns/op, 242,936 B/op, 1,705 allocs/op, and deep
|
||||
86,282–109,697 ns/op, 119,920 B/op, 827 allocs/op. These are baselines, not
|
||||
speedup claims; the large initial deep-help and flag-heavy timing variation
|
||||
shows why later comparisons should report run variation.
|
||||
|
||||
## Stage 2: establish report and scope measurements
|
||||
|
||||
|
||||
Reference in New Issue
Block a user