Complete the performance audit and update documentation
This commit is contained in:
@@ -1,6 +1,6 @@
|
||||
# Performance implementation plan
|
||||
|
||||
Status: implementation in progress; Stages 1–8 are complete, Stages 9–11 have not started.
|
||||
Status: complete; Stages 1–11 are complete.
|
||||
|
||||
Date: 2026-10-09.
|
||||
|
||||
@@ -876,7 +876,7 @@ are observations, not thresholds.
|
||||
|
||||
## Stage 11: audit the completed target and reconcile documentation
|
||||
|
||||
**Status:** not started. **Prerequisite:** Stages 1–10.
|
||||
**Status:** complete. **Prerequisite:** Stages 1–10.
|
||||
|
||||
**Files:** this plan, `performance.md`,
|
||||
[internal overview](../internal/overview.md), developer/README navigation,
|
||||
@@ -925,4 +925,103 @@ implemented with relevant behavioral and efficiency evidence; maintained docs
|
||||
identify their current owners. No performance guarantee or additional release
|
||||
gate has been introduced.
|
||||
|
||||
**Evidence:** pending.
|
||||
**Evidence:** Audited PF-01–PF-06 against the current implementation and
|
||||
behavioral coverage. PF-01 selects the PromptKit renderer before serialization;
|
||||
PF-02 indexes backend commands to declarations and uses parent links for the
|
||||
selected path; PF-03 caches immutable bridge metadata per `App` and uses one
|
||||
mutable adapter session per `Execute`, including help and usage; PF-04 scans
|
||||
only YAML framing lines but retains complete framed bytes; PF-05 validates and
|
||||
clones pending registrations before atomically installing them; PF-06 prepares
|
||||
owned ordered scope metadata, lookup, and discovery data at registration. The
|
||||
existing focused tests cover refresh after registration, parser/help/usage
|
||||
session reuse, YAML boundaries/directives, registration rollback, prepared
|
||||
scope ownership, selector collisions, and callback timing. OD-1 and OD-2 remain
|
||||
as selected: backend state is fresh per invocation, and YAML output retains a
|
||||
full framed buffer.
|
||||
|
||||
The default sprint suite passed on source revision `62b4a48` with
|
||||
`GOWORK=off GOTOOLCHAIN=local GOPROXY=off GONOPROXY=none GOSUMDB=off go test
|
||||
-mod=readonly -count=1 ./...`. Packages `commandkit`, `internal/cobra`, and
|
||||
`scripts` passed; both examples compiled. `go list` and `go doc -all .` confirmed
|
||||
that Cobra/pflag remain confined to `internal/cobra`, the adapter does not
|
||||
import the root package, and the root exported declarations retain their
|
||||
existing signatures. Review of the sprint Go diff found no changed or added
|
||||
root exported declarations.
|
||||
|
||||
Final representative measurements used Go 1.26.8, linux/amd64, Linux
|
||||
6.17.0-41-generic, Intel Core i5-7260U CPU @ 2.20 GHz, source revision
|
||||
`62b4a48`, and the unchanged fixtures from earlier stages. The five-sample
|
||||
command passed:
|
||||
`GOWORK=off GOTOOLCHAIN=local GOPROXY=off GONOPROXY=none GOSUMDB=off go test
|
||||
-mod=readonly -run '^$' -bench '^BenchmarkPerformance' -benchmem
|
||||
-benchtime=200ms -count=5 . ./internal/cobra`. Raw output is in
|
||||
`/tmp/commandkit-performance-final.txt`. Dispatch, prepared-scope, and scope
|
||||
registration benchmarks were then repeated with the same five-sample settings.
|
||||
Shortly after the measurements, the host load average was 3.62 on four logical
|
||||
CPUs; historical stage measurements did not record load. Cross-run timing
|
||||
comparisons are therefore inconclusive, while bytes and allocation counts
|
||||
remained stable.
|
||||
|
||||
- **PF-01:** For 128 prompt inputs, final text measured 159,091–172,388 ns/op,
|
||||
126,782–126,787 B/op, and 1,143 allocations; YAML measured 1,240,053–1,534,668
|
||||
ns/op, 1,262,298–1,262,330 B/op, and 5,079 allocations. Stage 2 recorded
|
||||
145,524–145,531 B and 1,171 allocations for text and 1,393,508–1,393,568 B
|
||||
and 6,039–6,040 allocations for YAML. Four-input YAML is now 69,677–69,678 B
|
||||
and 480 allocations versus 74,730–74,733 B and 557 at baseline. Selected
|
||||
rendering reduces report work and allocations; profile-text timings remain
|
||||
noisy.
|
||||
- **PF-02:** Final adapter Parse measured 242,960–242,969 B/op and 1,706
|
||||
allocations for the shallow/wide fixture, and 119,560 B/op and 827
|
||||
allocations for the deep fixture. Stage 4's isolated change reduced deep
|
||||
Parse by 384 B and one allocation; later session work adds a small wrapper
|
||||
cost. Tree construction remains in these Parse measurements; selected-path
|
||||
lookup is recorded separately in Stage 4.
|
||||
- **PF-03:** Final repeated success measured 53,796–61,372 ns/op,
|
||||
29,580–29,581 B/op, and 269 allocations. Local and explicit help used
|
||||
41.4–41.9 KB and 509/516 allocations; parser and handler usage used about
|
||||
30 KB and 292/284 allocations. Stage 9 measured 38,671–44,179 ns/op for
|
||||
repeated success and about 41.4–41.9 KB with the same help allocation counts.
|
||||
A focused repeat remained slower in time while allocation results stayed
|
||||
stable. The host load noted above and the absence of a prior load record
|
||||
prevent attributing this timing difference to the source change. Cold
|
||||
construction plus first execution measured 84,000–93,954 ns/op and
|
||||
38,418–38,419 B for a small app, and 616,203–678,200 ns/op and about 305 KB
|
||||
for a wide app. The cached neutral tree still requires construction once per
|
||||
metadata generation, and mutable backend trees are still built per invocation.
|
||||
- **PF-04:** Direct 4 KiB framing measured 1,193–1,256 ns/op, 4,864 B/op, and
|
||||
one allocation; direct 1 MiB framing measured 188,617–561,502 ns/op,
|
||||
1,056,768–1,056,770 B/op, and one allocation. The 1 MiB aggregate writer
|
||||
measured 1,624,170–1,806,960 ns/op, about 4.245 MB, and 185 allocations,
|
||||
versus the Stage 2 baseline of about 8.695 MB and 215–216 allocations. The
|
||||
full framed copy remains part of the selected OD-2 behavior.
|
||||
- **PF-05:** Final 128-entry individual registration measured 101,116–118,013
|
||||
ns/op, 29,420–29,421 B/op, and 293 allocations, compared with the Stage 1
|
||||
baseline of 414,618–492,261 ns/op, about 405 KB, and 784 allocations. A
|
||||
128-entry batch measured 104,559–118,844 ns/op, about 35,992 B/op, and 296
|
||||
allocations, retaining the documented extra pending-index and map-growth
|
||||
cost compared with the smaller Stage 1 batch allocation. The focused run
|
||||
confirms this remains an allocation/timing tradeoff, not a correctness issue.
|
||||
- **PF-06:** Final two-scope discovery, named selection, and aggregate
|
||||
validation measured 33,003–34,199, 38,275–45,600, and 44,677–65,937 ns/op,
|
||||
respectively; allocation counts were 195, 207, and 208. At 32 scopes,
|
||||
discovery used 199 allocations, named selection 207, and aggregate validation
|
||||
240. The focused repeat also remained slower than Stage 10 measurements while
|
||||
retaining the same allocation counts. Compared with Stage 2,
|
||||
retained metadata reduces two-scope discovery from 21,161 B/223 allocations
|
||||
to 18,559–18,560 B/195, and 32-scope aggregate validation from 29,781–29,782
|
||||
B/269 to 22,705–22,706 B/240. The prepared suite retains ordered scopes,
|
||||
lookup, and discovery data; registration cost grows accordingly.
|
||||
|
||||
The runbook remains the owner of release operations, the ReleaseKit roadmap
|
||||
owns adoption status, and this performance roadmap and record own PF-01–PF-06.
|
||||
Updated the internal overview, README, and contributor navigation; replaced
|
||||
obsolete ReleaseKit-plan links in the releasekit, initial, and remediation
|
||||
records with the completed historical adoption record or current owner. Local
|
||||
Markdown links and fragments passed checks across all eight affected documents,
|
||||
and `git diff --check` passed. The Stage 1–10 implementation commits are
|
||||
`aed4c59`, `a431540`, `628dc78`, `ae656d8`, `3f8841f`, `5816dd3`, `3ad3559`,
|
||||
`a852301`, `a7960e5`, and `62b4a48`; the final audit is recorded in this commit.
|
||||
|
||||
Remaining costs are full backend tree construction for each invocation and a
|
||||
full framed YAML buffer for delivery. No public performance guarantee or new
|
||||
release gate was added.
|
||||
|
||||
Reference in New Issue
Block a user