Plan staged performance implementation

docs/roadmap/performance.md

@@ -1,6 +1,6 @@
 # Computational efficiency roadmap
 
-Status: proposed improvements; OD-1 and OD-2 resolved; implementation has not begun.
+Status: ready for implementation; OD-1 and OD-2 resolved; implementation has not begun.
 
 Date: 2026-10-09.
 
@@ -11,8 +11,10 @@ registration, and inspection reports. Favor straightforward changes with clear
 benefits, and use measurements to decide whether more complex optimizations are
 worth maintaining.
 
-This document owns the proposed improvements, priorities, and completion
-evidence. The [architecture policy](../policy/architecture.md), accepted
+This document owns the target state, priorities, and selected implementation
+scope. The [implementation plan](implementation.md) owns ordered stages,
+measurement fixtures, validation, and execution evidence. The
+[architecture policy](../policy/architecture.md), accepted
 [ADRs](../adr/), [Go declarations](../../app.go), and
 [CLI reference](../cli.md) continue to govern behavior. Follow the
 [agent policy](../policy/agents.md) for proportionate implementation and the
@@ -26,14 +28,14 @@ findings come from source inspection; no benchmarks or allocation profiles were
 run. Complexity observations identify scaling costs, not measured bottlenecks
 or promised speedups. Recheck the affected implementation before starting work.
 
-| ID | Observed work | Proposed result | Priority |
+| ID | Observed work | Target result | Primary benefit |
 | --- | --- | --- | --- |
-| PF-01 | PromptKit text reports are built even for YAML requests. | Render only the requested format and avoid redundant report buffers. | First pass |
-| PF-02 | Selected-command lookup scans a map; each ancestor is rediscovered from the root. | Direct declaration lookup and ancestry traversal proportional to path depth. | First pass |
-| PF-03 | Every invocation translates and sorts metadata; help and usage construct another backend tree. | Reuse immutable metadata and the invocation's prepared tree. | Measure and improve after the first pass |
-| PF-04 | Aggregate YAML framing splits every line and copies the complete native document. | Inspect only the opening lines and retain a complete framed buffer for delivery. | First pass |
-| PF-05 | Each registration copies the existing top-level registry. | Prepare new entries separately and install a validated batch atomically. | Lower priority |
-| PF-06 | Scope ordering, lookup, and selection repeat; selector duplicate checks scan previous entries. | Prepare stable scope metadata once and use direct duplicate checks. | Lower priority |
+| PF-01 | PromptKit text reports are built even for YAML requests. | Render only the requested format and avoid redundant report buffers. | Inspection CPU and allocation reduction |
+| PF-02 | Selected-command lookup scans a map; each ancestor is rediscovered from the root. | Direct declaration lookup and ancestry traversal proportional to path depth. | Wide or deeply nested command trees |
+| PF-03 | Every invocation translates and sorts metadata; help and usage construct another backend tree. | Reuse immutable metadata and the invocation's prepared tree. | Repeated execution and help/usage paths |
+| PF-04 | Aggregate YAML framing splits every line and copies the complete native document. | Inspect only the opening lines and retain a complete framed buffer for delivery. | Large or newline-dense reports |
+| PF-05 | Each registration copies the existing top-level registry. | Prepare new entries separately and install a validated batch atomically. | App construction with many registrations |
+| PF-06 | Scope ordering, lookup, and selection repeat; selector duplicate checks scan previous entries. | Prepare stable scope metadata once and use direct duplicate checks. | Growing scope and selector sets |
 
 The implemented package inventory remains owned by the
 [internal overview](../internal/overview.md). The target is internal improvement
@@ -68,7 +70,7 @@ and scope preparation remain with the corresponding root handlers. Acquisition,
 configuration resolution, and provider lifetime remain with the existing
 integration contracts and consuming applications.
 
-## Proposed improvements
+## Target improvements
 
 ### PF-01: Render only the requested PromptKit format
 
@@ -79,15 +81,11 @@ profile text formatter serializes `ExtraParams` as nested YAML; a YAML request
 then serializes those parameters again as part of the structured report. Text
 delivery also copies an already completed report into another builder.
 
-Choose the requested formatter before rendering. Use a straightforward buffer or
-completed report representation that avoids the redundant text-buffer copy.
-Continue using the existing structured report representation and YAML encoder;
-the optimization concerns when rendering occurs and how its result is delivered.
-
-Completion evidence should show unchanged text and YAML report semantics,
-including `ExtraParams`, diagnostics, and delivery failures, with eliminated
-unused text construction for YAML requests. Measure prompt input collections and
-profile parameter maps large enough to expose the redundant work.
+The target selects the requested formatter before rendering and delivers the
+completed report without a redundant intermediate text buffer. It retains the
+existing structured report representation and YAML encoder, including
+`ExtraParams`, diagnostics, and delivery-failure semantics. YAML requests produce
+only the structured report; text requests produce only the text report.
 
 ### PF-02: Index command declarations and ancestry
 
@@ -97,14 +95,10 @@ declaration-to-backend map to find a selected backend command. `pathFor` calls
 depth D, repeated parent searches can take O(N × D) work; a chain can produce
 quadratic work.
 
-Record a backend-to-declaration mapping during construction. Build parent links
-alongside it, or traverse Cobra's existing parent relationships internally.
-Prefer an implementation that reconstructs only the selected path rather than
-storing a separate complete path for every node. Declaration lookup should be
-constant time, with path construction proportional to D after tree preparation.
-
-Exercise shallow, wide, and deeply nested declarations through dispatch, help,
-and usage. Preserve exact registered-path matching and error classification.
+The target provides direct backend-to-declaration lookup and reconstructs only
+the selected path from parent relationships inside the adapter. Declaration
+lookup is constant time, with path construction proportional to D after tree
+preparation. Exact registered-path matching and error classification are retained.
 This improvement removes redundant searches; constructing the complete backend
 tree still has a separate cost addressed by PF-03.
 
@@ -115,19 +109,16 @@ every invocation, including flag and operand declarations. The adapter then
 builds backend commands and flag sets for all nodes. Help and usage rendering
 build another complete backend tree after parsing has already built one.
 
-Pursue two improvements:
-
-1. Retain the invocation's prepared backend tree inside the adapter and use it
-   for parsing, help, and usage. The root coordinates these operations without
-   receiving Cobra/pflag objects. Keep the internal coordination API narrow.
-2. Prepare or lazily cache immutable bridge metadata and sorted child order per
-   `App`. Refresh that derived state after successful registration. Failed
-   registration must leave the previously registered commands usable, including
-   their help and dispatch behavior.
+The target retains the invocation's prepared backend tree inside the adapter for
+parsing, help, and usage. The root coordinates these operations through a narrow
+internal handle without receiving Cobra/pflag objects. Immutable bridge metadata
+and sorted child order are cached per `App` and refreshed when registration
+changes the declared tree. Failed registration leaves existing commands usable,
+including their help and dispatch behavior.
 
 Metadata caching principally benefits repeated `Execute` calls. Avoiding a
 second backend construction also benefits a single CLI help or usage request.
-Measure both cases separately, including the cost of app construction and
+Efficiency evidence distinguishes both cases, including app construction and
 registration for consumers that execute only once.
 
 Under the selected OD-1 approach, construct fresh mutable backend parse state for
@@ -145,10 +136,10 @@ opening content. These splits scan all bytes and allocate a slice entry for each
 line, even when the framing decision can be made immediately. Framing then
 allocates a complete copy alongside the native buffer.
 
-First, scan the preamble with byte offsets, stopping when the directive or marker
-decision is complete. Share the resulting framing analysis where useful so
-directive detection and scope-label placement do not independently scan the same
-preamble. Avoid a document-wide line index and unnecessary string conversions.
+The target analyzes the preamble with byte offsets and stops when the directive
+or marker decision is complete. Directive detection and scope-label placement
+share that analysis. Framing needs no document-wide line index or unnecessary
+string conversions.
 
 Under the selected OD-2 approach, retain the native render buffer and assemble a
 complete framed buffer for each document before delivery. This keeps delivery
@@ -162,9 +153,8 @@ contract. Keep single-scope native delivery direct. Aggregate acquisition remain
 complete before output; the current code buffers one rendered document at a time,
 not every scope's rendered bytes simultaneously.
 
-Measure documents with many short lines as well as large values. The initial
-target removes avoidable scans and allocations; complete rendering and delivery
-will still require work proportional to output size.
+This target removes avoidable scans and allocations; complete rendering and
+delivery still require work proportional to output size.
 
 ### PF-05: Avoid repeated registry copying
 
@@ -173,15 +163,14 @@ will still require work proportional to output size.
 O(R²) cumulative map-copy work, apart from validation and declaration cloning.
 This is a construction cost, not work performed by a registered handler.
 
-Validate declarations, check collisions against both existing and pending
-entries, and prepare owned new entries in temporary storage. Install them only
-after the complete batch succeeds. Preserve declaration ownership and atomic
-validation without copying every existing map entry.
+The target prepares owned pending declarations and checks collisions against
+existing and pending entries before changing the registry. Successful batch
+validation installs the pending entries without copying existing map entries,
+while preserving declaration ownership and atomic validation. Its completion
+point also invalidates derived metadata for PF-03 when the tree changes.
 
-Compare individual and batch registration at representative sizes. Batch
-registration is already available to consumers; adopting it is an optional
-consumer optimization, not a new requirement. Coordinate this work with PF-03's
-metadata refresh so registration has one clear completion point.
+Batch registration remains an optional consumer optimization. Both individual
+and batch registration are supported by this target.
 
 ### PF-06: Prepare stable configuration scope metadata
 
@@ -193,76 +182,41 @@ occurs during preflight and again in the handler. In
 selector with all previous selectors; PromptKit already uses name and short-name
 maps for the equivalent check.
 
-Prepare stable scope order and a name lookup during suite registration, keeping
-that derived data private. Use direct name and short-name checks for selector
-duplicates. Assess retaining the selected scopes in invocation-owned input after
-preflight if repeated selection remains significant; keep selector applicability
-and per-invocation contexts fresh.
+The target retains normalized declarations, stable scope order, direct name
+lookup, and discovery inventory as private registered-suite metadata. Selector
+duplicate validation uses direct name and short-name checks. Context projection
+and selection validation remain fresh for each invocation; both preflight and
+handlers can consult the same prepared metadata without repeated sorting or
+quadratic scope searches.
 
-Measure the normal small scope set before adopting additional indexes for
-selector applicability. Preserve scope order, declared operation availability,
-and scope-aware selector validation under
-[ADR 0009](../adr/0009-configuration-scopes.md). Do not invoke acquisition while
-preparing this metadata.
+Scope order, declared operation availability, and scope-aware selector validation
+remain governed by [ADR 0009](../adr/0009-configuration-scopes.md). Metadata
+preparation leaves acquisition and runtime context callbacks uninvoked.
 
-## Recommended sequence and measurement
+## Delivery scope and evidence
 
-1. Establish focused before-change measurements for PF-01, PF-02, and PF-04,
-   then implement conditional formatting, direct command lookup/ancestry, and
-   YAML prefix scanning as independently reviewable changes.
-2. Measure complete dispatch paths, then implement PF-03's reuse within an
-   invocation and immutable metadata caching. Compare the resulting runtime
-   gains with the coordination cost and retained memory.
-3. Evaluate PF-05 and PF-06 using the first-pass evidence and expected consumer
-   sizes. Record a deferral if a proposed change adds complexity without a
-   worthwhile benefit.
-4. Record remaining costs after the selected improvements. Mutable backend reuse
-   across invocations and segmented YAML delivery remain future possibilities
-   beyond this target; any reconsideration should use the measurements and a
-   deliberate change to the selected approach.
+PF-01 through PF-06 are included in the implementation target. Conditional
+formatting, indexed ancestry, and YAML preamble analysis address the clearest
+redundant work. Registry and scope preparation are lower-priority, bounded
+changes that remove observed scaling costs without adding consumer requirements.
+The [implementation plan](implementation.md) specifies the concrete mechanisms
+and divisions of work.
 
-Use Go benchmarks with controlled, offline fixtures when implementation work is
-authorized. Keep external loading and service latency out of isolated CommandKit
-measurements, and identify when an integration measurement includes provider
-work. Include:
+Efficiency evidence compares elapsed cost, bytes allocated, and allocations per
+operation for representative dispatch, reports, registration, and scope sizes.
+It distinguishes cold and repeated execution, isolated and complete command
+costs, provider work, and memory retained by caches. Source analysis alone does
+not establish a measured speedup. Measurements establish no fixed timing
+threshold, public allocation budget, or additional release gate.
 
-| Area | Representative comparisons |
-| --- | --- |
-| Dispatch | Small and larger registries; wide and deep trees; few and many flags; successful commands, help, and usage errors. |
-| Lifecycle | Construction, registration, and first execution versus repeated execution on an existing `App`. |
-| Reports | Prompt text/YAML; profile text/YAML with varied parameter maps; aggregate configuration documents with varied size and line count. |
-| Registration and scopes | Individual/batch registration; small and larger scope and selector sets. |
+Implementation preserves existing public API and CLI contracts. Internal
+documentation describes the changed mechanics when they land; detailed
+validation and before/after evidence belong in the implementation plan.
 
-Record elapsed cost, bytes allocated, and allocations per operation with the Go
-version, machine, fixture dimensions, and benchmark scope. Use repeated samples
-for comparisons; profile CPU or allocation behavior when it helps explain a
-remaining cost. Account for memory retained by caches as well as allocation
-reductions. Avoid combining every fixture dimension into a large benchmark
-matrix without a specific question.
-
-Choose behavioral validation according to each change's risks and existing
-coverage. Important cases include fresh presence and collections across runs,
-registration after earlier execution, failed registration, help without provider
-loading, YAML framing, cancellation, and partial output. Behavioral tests should
-protect the existing contracts; benchmark results assess efficiency without
-locking in private data structures or exact helper-call sequences.
-
-These measurements guide implementation choices. They do not establish fixed
-timing thresholds, allocation budgets, or additional release gates.
-
-## Completion and documentation
-
-For each PF item, record whether it was implemented, deferred, or narrowed,
-alongside its change reference, relevant behavioral validation, and measurement
-results. An unmeasured improvement must be described as such. Completion means
-the selected changes preserve the existing contracts and have evidence supporting
-their claimed benefit; it does not require implementing every speculative reuse
-option.
-
-Update [internal documentation](../internal/overview.md) when implemented
-mechanics change. Update public GoDoc or the CLI reference only when their owning
-contracts need clarification. Keep proposal status and performance evidence in
-this roadmap or a subsequent implementation plan.
+Selected-scope memoization and additional selector-membership indexes are
+deferred pending evidence that their bookkeeping is worthwhile. Mutable backend
+reuse across invocations and segmented YAML delivery are outside the selected
+target; future reconsideration needs a deliberate change to OD-1 or OD-2.
 
 Provider caching, consumer loader redesign, removing copies required for value
 ownership, and replacing Cobra are outside this target. They require their own