Files
commandkit/docs/roadmap/audit/catalog.md

501 lines
28 KiB
Markdown
Raw Normal View History

# Consolidated audit catalog
Status: **22 distinct open findings: 22 P3**, consolidated from 30
numbered entries in seven audit documents. Reviewed audit state:
`db732e3a1de55c844f77870e1c6db683c2685d5d`.
2026-11-07 14:11:19 +00:00
This catalog owns grouping, recommended review order, and subsequent remediation
decisions. The linked original entries retain code locations, evidence,
contract analysis, and detailed
remediation options. Consolidation alone does not resolve a finding or select a
proposed contract change. Explicit decisions recorded below supersede the
original entries' alternatives; findings remain open until remediated.
2026-11-07 14:11:19 +00:00
Original priorities are retained. Within those priorities, incorrect outcomes
and request isolation lead; blocked validation,
resource retention, and consumer contracts precede scaling and maintainability
work; editorial and historical navigation issues follow.
The [remediation analysis](remediation-analysis.md) records the completed design
review and its rationale. All choices identified for discussion have been
resolved; no further product or architectural decision is required before
implementation planning. Accepted conclusions are recorded in this catalog;
the remaining findings still require remediation and validation.
Three pairs describe the same underlying issue: correctness C2/concurrency N2,
correctness C4/concurrency N3, and complexity M6/responsibility R1. Other
related findings remain distinct where they concern different failure modes or
changes. Documentation cross-references and the responsibility review's closing
assessment are supporting discussion, not additional numbered findings.
The [internal overview](../../internal/overview.md) inventories the
architecture: core owns declarations, routing, typed input preparation,
dispatch, completion, and reporting; neutral private packages isolate the
Cobra/pflag adapter. ckconfig and ckprompt adapt native upstream capabilities
through separate optional modules. Applications retain loaders, builders,
runtime resources, and executable lifecycle; release launchers delegate to
ReleaseKit. The findings call for specific fixes and simplifications within
these boundaries.
## F01 — P2: Aggregate smoke testing can stop on cancellation and return success
**Complete.** The later cancellation checkpoint retains a cause when work is
omitted, without duplicating a cause already recorded. Focused aggregate and
cancellation tests pass in the aligned workspace; the new progress-delivery
regression failed against the original behavior for both successful and failed
provider calls, and also covers cancellation after the last completed check.
In ckprompt's `runSmokeAggregate`, cancellation triggered during progress
delivery can stop the remaining smoke kinds without entering the operation
cause. A partial aggregate can consequently return status 0; cancellation can
also be omitted when another error already exists.
Record cancellation at the checkpoint that actually omits remaining work,
preserving acquired reports and prior errors. This does not require treating
cancellation after all requested work has completed as a failure.
**Original:** [04-concurrency N1](04-concurrency.md#n1--p2-cancellation-can-stop-aggregate-smoke-testing-with-a-success-outcome).
Also noted in the [documentation cross-references](13-documentation.md#previously-recorded-discrepancies-affecting-documented-behavior).
## F02 — P2: CLI provider parameters share mutable state across requests
2026-11-07 15:38:06 +00:00
**Complete.** CLI JSON is copied at captured binding application and each
aggregate seed boundary. Both regressions failed before the fix and focused
aligned-workspace tests passed, including nested objects and arrays, nil and
empty values, and numeric types.
ckprompt assigns parsed `ExtraParams` maps directly to request destinations, and
smoke seeds share those nested maps. Mutation by one callback can change later
smoke requests or another destination receiving the same captured bindings.
Clone the CLI-owned JSON maps and arrays at binding application and seed
creation. Preserve presence and numeric semantics, and application builders'
authority; a general deep-copy framework is unnecessary.
**Original:** [01-correctness C1](01-correctness.md#c1--p2-cli-provider-parameter-objects-leak-mutations-between-requests).
Also noted in the [documentation cross-references](13-documentation.md#previously-recorded-discrepancies-affecting-documented-behavior).
## F03 — P2: Finalization can discard an accepted deferred message
**Complete.** The acceptance cutoff and batch detachment now share one locked
transition. A deterministic regression reproduces the former lost submission and
checks late empty/hidden submissions, accepted delivery, completion, and terminal
ordering. Focused reporter tests passed normally and under the race detector;
existing tests cover visibility and presentation failures.
Core reporter finalization detaches the deferred batch before closing
acceptance. A concurrent submission can return success while presentation is in
progress, then be discarded when finalization clears the queue. The returned
delivery evidence can still indicate success.
2026-11-07 14:11:19 +00:00
**Decision:** atomically close deferred-message acceptance when finalization
begins, deliver the already accepted batch under the existing presentation rules,
and return `ErrReporterClosed` for later deferred submissions. Preserve the
finalizer's own delivery path and document the acceptance cutoff. Repeatedly
draining newly arriving submissions is not selected; completion must not depend
on application producers eventually stopping. This does not add responsibility
for application goroutines or change ordinary message visibility.
The two source entries describe the same logical ordering defect, rather than a
data race.
**Originals:** [01-correctness C2](01-correctness.md#c2--p2-finalization-can-discard-a-successfully-queued-deferred-message);
[04-concurrency N2](04-concurrency.md#n2--p2-deferred-message-acceptance-remains-open-after-the-final-batch-detaches).
Also noted in the [documentation cross-references](13-documentation.md#previously-recorded-discrepancies-affecting-documented-behavior).
## F04 — P2: Prepared-request preview can succeed without a prepared result
2026-11-07 15:41:18 +00:00
**Complete.** The command rejects a missing prepared request after provider and
cancellation checks, preserving acquired evidence. Its public-command regression
failed against the original behavior; focused preparation tests pass with valid,
missing, provider-error and canceled acquisitions.
The ckprompt prepared-request handler accepts an empty `Acquired` value with a
nil error from `PrepareRequest`. It renders `null` and reports success despite
having no prepared request.
Reject the missing result as an operational failure in this handler while
preserving diagnostics and explanations. Keep core's general ability to render a
legitimate null report.
**Original:** [01-correctness C3](01-correctness.md#c3--p2-prepared-request-inspection-reports-success-without-a-result).
Also noted in the [documentation cross-references](13-documentation.md#previously-recorded-discrepancies-affecting-documented-behavior).
## F05 — P2: Configuration-show cancellation is also recorded as a rendering failure
**Complete.** A cancellation checkpoint stops buffered show preparation without
reporting cancellation as a render error. A regression failed before the fix;
focused tests now verify no partial output, cancellation identity, and retained
native rendering errors when both occur.
ckconfig checks cancellation during buffered show preparation inside the
`RenderOutput` callback. Returning that cancellation from the renderer records
both the correct operation cause and a false presentation failure, conflating
two established outcome dimensions.
Carry preparation cancellation separately from renderer errors while retaining
genuine rendering and writing failures and avoiding partial aggregate output.
Both audits identify the same classification defect.
**Originals:** [01-correctness C4](01-correctness.md#c4--p2-show-preparation-labels-ordinary-cancellation-as-a-rendering-error);
[04-concurrency N3](04-concurrency.md#n3--p2-buffered-configuration-show-cancellation-becomes-a-presentation-failure).
Also noted in the [documentation cross-references](13-documentation.md#previously-recorded-discrepancies-affecting-documented-behavior).
## F06 — P3: The optional interactive completion consumer fails before shell validation
The completion consumer fixture declares the reserved `format` flag, so App
construction fails before it can generate scripts or exercise interactive
completion. Default test suites skip this optional path.
Use core's format flag or rename the fixture's custom flag, update its
expectations, and rerun available shell checks. The fixture failure does not
establish a defect in generated completion scripts.
**Original:** [01-correctness C6](01-correctness.md#c6--p3-the-interactive-completion-consumer-cannot-construct-its-app).
Also noted in the [documentation cross-references](13-documentation.md#previously-recorded-discrepancies-affecting-documented-behavior).
## F07 — P3: Closed reporting handles retain both borrowed destinations
A retained closed `Reporter`, `Output`, or `DeferredReporter` keeps the shared
reporting state and both destination writers reachable. Applications retaining a
closed handle can therefore retain large buffers or resource-bearing writer
objects after invocation completion.
Clear direct borrowed-writer references after final presentation under the
reporting lock, preserving closed-handle behavior and returned error evidence.
Do not close or flush application-owned writers. This is conditional retention
through a reachable handle, not an unreachable heap leak.
**Original:** [05-resources RM1](05-resources.md#rm1--p3-closed-reporting-handles-retain-both-destination-objects).
## F08 — P3: The core example overwrites its optional argument default
The example's scenario setter replaces the factory's `"demo"` default with an
unsupplied empty value. The ordinary `report` invocation consequently fails with
a usage error.
Assign the optional argument only when its supplied state is set. Fix the
consumer without changing core's optional-binding semantics.
**Original:** [01-correctness C5](01-correctness.md#c5--p3-the-core-example-overwrites-its-optional-scenario-default).
## F09 — P3: The configuration provider contract omits its required YAML representation
`EffectiveWriter` GoDoc promises a native representation but omits that `config
show` converts one supported YAML document through `DecodeYAMLReport`. Custom
providers can supply output that appears compatible with the description yet
fails conversion; native formatting is not passed through.
Document that boundary with the public interface and conversion helper, and
update `View`'s report description. This clarification is independent of the
unused explanation-writer requirement in F11.
**Original:** [13-documentation D1](13-documentation.md#d1--p3-configuration-provider-godoc-omits-the-effective-output-contract).
## F10 — P3: Public flag metadata omits enforced naming and reserved-spelling rules
`FlagMeta` describes names without leading dashes and an optional short
spelling, while installation enforces additional name grammar and reserved
spellings. Consumers must discover those constraints through errors or private
implementation.
**Decision:** retain and document the existing declaration rules. Long names use
Unicode lowercase letters and digits separated by single hyphens; numeric names
and names beginning with digits remain allowed. Keep core-owned reserved names
and the optional one-byte short-spelling rules, excluding `-` and reserved `h`.
Do not add ASCII-only or leading-letter restrictions, or automatic rewriting of
invalid names. These rules concern flag names, not supplied values.
Document the rules once with `FlagMeta` and link related public helpers. This
remediation changes documentation, not runtime validation.
The [F10 analysis](remediation-analysis.md#f10--flag-declaration-naming) reviews
the naming convention's basis and records the accepted direction. Implementation
of the documentation correction remains pending.
**Original:** [13-documentation D2](13-documentation.md#d2--p3-public-flag-metadata-omits-enforced-declaration-constraints).
## F11 — P3: Explanation providers must implement an unused text writer
`ExplanationReader` still requires `WriteExplanation` even though production
show/explain handlers acquire evidence with `Explain` and core renders it. This
burdens custom providers and suggests a presentation responsibility that
standard commands no longer use.
2026-11-07 14:11:19 +00:00
**Decision:** remove `WriteExplanation` from `ExplanationReader`, leaving the
acquisition responsibility, and update GoDoc, writer-only stubs, and assertions.
Native provider objects may retain their own rendering methods. Do not add
compatibility scaffolding or restore provider-owned rendering to justify the
unused requirement. Both audits identify the same obsolete requirement.
The interface change removes the ability to invoke that method through
`View.Explanation`. CommandKit is still pre-release with no consumers, so no
consumer compatibility or migration work is needed.
**Originals:** [03-complexity M6](03-complexity.md#m6--p3-explanation-providers-must-implement-an-unused-rendering-capability);
[A1-responsibility R1](A1-responsibility.md#r1--p3-explanation-providers-still-owe-an-unused-text-writer).
## F12 — P3: Parser sessions carry unused execution resources
Core and the Cobra adapter duplicate stream normalization and retain context and
streams through parser options and backend nodes, although the adapter parses
and renders buffered help rather than executing callbacks or reading application
input. The execution object also retains an unread streams field.
Remove unused resource plumbing while preserving real invocation resources and
explicit internal discard destinations/local help buffers. This simplifies
ownership; it is separate from the closed-reporting-handle retention in F07.
**Original:** [A1-responsibility R2](A1-responsibility.md#r2--p3-parser-sessions-carry-application-execution-resources-they-do-not-use).
## F13 — P3: Flag-alias compilation repeatedly walks shared chain suffixes
For A chained effective flag mappings across C contexts, independent target
walks can perform Θ(CA²) target examinations, including inactive chains. Each
source repeats work already performed for another source's suffix.
Use context-local visitation states and memoized terminal results. Traversal can
become expected O(A) per context, with existing deterministic sorting retaining
O(A log A) total work for these steps. Preserve scoped precedence, canonical
endpoints, missing-target behavior, and cycle diagnostics.
2026-11-07 14:11:19 +00:00
**Decision for F13–F15:** the repeated-work and scaling analysis sufficiently
justifies remediation. Performance testing is not a prerequisite for selecting
or implementing these changes. Validate correctness and preservation of the
selected routing and discovery semantics.
**Original:** [02-efficiency E1](02-efficiency.md#e1--p3-flag-alias-compilation-repeatedly-resolves-shared-chain-suffixes).
## F14 — P3: Discovery-context preparation repeatedly rebuilds shared prefixes
R alias sources sharing a depth-D proper prefix cause Θ(RD²) copying and key
construction before result deduplication, with repeated classification as well.
The source declarations contain only Θ(RD) tokens.
Traverse distinct source prefixes once and classify each once. For the audited
2026-11-07 14:11:19 +00:00
shape, this phase can become O(RD + D²).
**Decision:** implement F14 and F15 together using the existing alias resolver's
source-path tree. Add narrow internal read-only operations to enumerate distinct
nonempty proper prefixes for F14 and find strictly descendant alias sources for
F15. Retain the existing discovery classification and exclusions, applying them
once per candidate prefix.
Reuse the tree's routing-snapshot lifetime; do not introduce a parallel index or
a cross-invocation result cache. The repeated preparation and help-time scan
remain distinct findings. The [joint analysis](remediation-analysis.md#f14f15--joint-source-prefix-design)
records the accepted design. The performance-testing decision under F13 applies
to both; implementation remains pending.
**Original:** [02-efficiency E3](02-efficiency.md#e3--p3-discovery-context-preparation-rebuilds-shared-prefixes-repeatedly).
## F15 — P3: Alias help scans the entire source catalog for each intermediate route
Help discovery examines all A alias sources at each of P rewrite paths, doing PA
examinations and repeated copies even when there are no descendants. A valid
single-token alias chain produces Θ(A²) work to discover an empty alias section
on every such help request.
**Decision:** query the existing resolver tree for strict descendants through
the internal operation shared with F14's design. Continue visiting intermediate
resolution paths, resolving composed entered paths through the existing
resolver, and retaining canonical-target checks, suffix deduplication, and
output sorting. Root help may retain the complete catalog when all entries are
needed. No public API or CLI behavior change is selected.
The identified discovery work becomes O(A) for that example; general cases still pay for
2026-11-07 14:11:19 +00:00
matching candidates, resolution, and output sorting. Design this together with
F14 under the decisions recorded there and in F13.
**Original:** [02-efficiency E2](02-efficiency.md#e2--p3-alias-help-scans-every-source-for-every-intermediate-route).
## F16 — P3: Parsing and completion duplicate flag-consumption grammar
Execution parsing and completion separately implement short clusters,
attached/next values, boolean assignments, terminators, and unknown spellings.
Shared classification helpers cover only part of the grammar, leaving changes to
cross the core/backend boundary.
**Decision:** share a small flag-consumption operation through the existing
neutral internal helpers. Share facts about flag identity, attached versus
missing values, consumption of the next argument, and short-cluster position.
Keep execution normalization, failure provenance, and partial-word completion
policies in their existing owners. Preserve explicitly empty values and current
CLI behavior; no public parser or replacement backend is selected.
Retain caller-specific branches where combining them would require extensive
modes or special cases. Repair F06's completion fixture before validating this
refactor. The [F16 analysis](remediation-analysis.md#f16--shared-flag-consumption)
records the accepted boundary; implementation remains pending.
**Original:** [03-complexity M3](03-complexity.md#m3--p3-parsing-and-completion-maintain-parallel-flag-consumption-logic).
## F17 — P3: Configuration scope capture combines several compilation phases
ckconfig's `captureSuite` interleaves scope and selector capture, validation,
comparison-binding renaming, binding attachment, operation unions, and discovery
inventory. Several partially populated collections must be understood together.
**Decision:** extract cohesive scope capture/validation and selector attachment
responsibilities, and make operation bindings and discovery inventory
construction explicit. Use existing prepared types and keep related loops
together. Preserve retained error order, declaration ownership, scope and binding
identity, and metadata-only construction. Do not create new intermediate records
or a helper for each loop merely to shorten the function.
**Original:** [03-complexity M1](03-complexity.md#m1--p3-scope-capture-combines-several-compilation-phases).
## F18 — P3: Configuration preflight interleaves shared and operation-specific rules
The configuration command builder and `prepareConfigOperation` mix flag
combinations, help selection, capabilities, pointer parsing, plan assembly,
ordinary inputs, and independent comparison inputs. Understanding one operation
requires reasoning through irrelevant submodes.
**Decision:** separate operation-specific option checks, shared scope/capability
selection, and comparison input preparation with small private functions. Keep
the overall sequence explicit, including explain's help path, aggregate checks
before input construction, independent comparison sides, and input checks before
shared state creation and acquisition. Extract flag groups where this clarifies
their purpose; no generic operation engine or rule registry is selected.
**Original:** [03-complexity M2](03-complexity.md#m2--p3-configuration-preflight-mixes-shared-and-operation-specific-rules).
## F19 — P3: Typed input preparation mixes conversion, provenance, and replay assembly
`prepareInputWithOccurrences` handles shape validation, conversion-error
spelling, occurrence identities, application order, required flags, operands,
and prepared callbacks. Captured binding replay depends on those shared
identities across another code path.
**Decision:** make small extractions for conversion-error attribution and operand
preparation. Keep flag preparation, occurrence identity, and order reconstruction
together; that block may remain in the orchestration function if extraction
requires excessive plumbing. Retain dense reconstruction, the captured-replay
heap merge, binding identity, supplied-state distinctions, conversion before
input factories, and error identity. No new state containers or replay framework
are selected.
**Original:** [03-complexity M4](03-complexity.md#m4--p3-typed-preparation-combines-conversion-provenance-and-replay-assembly).
## F20 — P3: Reporter message paths duplicate detail-rendering bookkeeping
Terminal and ordinary message paths repeat callback invocation, managed-writer
setup, render-versus-write error classification, and first-error retention.
Maintaining the same presentation policy in two branches increases the chance of
drift.
**Decision:** extract one private detail-rendering helper under the existing
reporting lock, returning rendering and writing errors separately. Callers retain
visibility/discard selection, headings, delivery observations, and terminal
timing. Preserve callback order, hidden-detail execution, sticky writer failures,
and non-reentrancy; add no locks or interchangeable rendering machinery.
Keep this refactor separately reviewable from F03 and F07; it does not itself
fix deferred-message loss or writer retention. After correctness repairs, favor
F20, then F18 and F17, while keeping F19 modest. All four use private functions
and existing types, with no intended public API or CLI behavior change. The
[F17–F20 analysis](remediation-analysis.md#f17f20--proportionate-private-extractions)
records the accepted scope; implementation remains pending.
**Original:** [03-complexity M7](03-complexity.md#m7--p3-terminal-and-ordinary-messages-duplicate-detail-rendering-mechanics).
## F21 — P3: Both integrations maintain the same ConfigKit diagnostic adaptation
ckconfig and ckprompt independently snapshot ConfigKit diagnostic evidence,
partition severities, construct detail callbacks, label sources, and delegate
native text formatting. Related Reporter scheduling switches are duplicated too.
**Decision:** expose one narrow public ckconfig diagnostic adaptation helper
and use it in both integrations. Reuse the existing private helper's shape:
accept a caller-supplied heading, native sources, and native diagnostic records;
return existing core diagnostics and an explanation ready for CLI reporting.
Preserve evidence snapshots, severity treatment, ordering within each group,
source labels, and delegation to ConfigKit's native formatter.
Callers retain operation-specific context, outcomes, error causes, and message
scheduling. Core retains visibility and delivery. Leave the smaller scheduling
switches with their callers; no additional public scheduling API or general
diagnostics framework is selected. ckprompt already depends on ckconfig, so no
new module dependency is needed.
CommandKit is pre-release with no consumers; compatibility and migration are
not constraints. The [F21 analysis](remediation-analysis.md#f21--shared-configkit-diagnostic-adaptation)
records the accepted scope and illustrative signature. Implementation remains
pending.
2026-11-07 14:11:19 +00:00
**Original:** [03-complexity M5](03-complexity.md#m5--p3-configuration-diagnostic-adaptation-is-maintained-in-both-integrations).
Also discussed in the [responsibility assessment](A1-responsibility.md#machinery-worth-retaining-under-the-selected-contracts).
## F22 — P3: Successful declaration validation eagerly formats every full path
`collectDeclarationIssues` copies and formats each ancestral path before knowing
whether a diagnostic is needed. A valid depth-N declaration chain incurs Θ(N²)
cumulative path work and allocation while producing no errors.
Use an append/truncate scratch path and format only emitted diagnostics. The
successful validation walk can become O(N) with O(D) path storage. Other
installation phases retain their own costs; this is not a claim that all App
construction becomes linear.
**Original:** [02-efficiency E4](02-efficiency.md#e4--p3-successful-declaration-validation-eagerly-constructs-every-full-path).
## F23 — P3: Concise diff rendering validates already-valid native paths
ckconfig's private pointer decoder validates arbitrary strings even though its
sole production caller supplies the canonical string of an immutable ConfigKit
`Path`. Tests sustain malformed-input branches that this boundary cannot reach.
Accept the native path directly and retain only the key-token decoding needed by
the selected concise report. Decode escapes once so `~01` remains `~1`; preserve
snapshot errors and operator-input validation.
**Original:** [A1-responsibility R3](A1-responsibility.md#r3--p3-concise-diff-rendering-revalidates-native-configkit-paths).
## F24 — P3: Obsolete private production entry points remain for tests alone
Legacy reporter setters/close, `App.parse`, and typed-preparation wrappers have
only test callers, while production uses current selection, finalization,
routed-session, and occurrence-aware paths. Some tests therefore exercise
superseded lifecycle entry points.
Move tests to the real private paths and remove obsolete production wrappers.
Keep any concise setup helpers in `_test.go` files, retaining useful behavioral
assertions without new exported test hooks.
**Original:** [A1-responsibility R4](A1-responsibility.md#r4--p3-private-production-entry-points-exist-only-to-sustain-older-tests).
## F25 — P3: The reporting ADR mixes implemented status with stale planning language
ADR 0006 marks shared reporting implemented but still directs implementation
status and format choices to a retired roadmap and describes the extension as
planned. Readers can mistake completed work for a pending prerequisite.
Clarify historical wording and link current behavior to GoDoc and the CLI
reference. Preserve the accepted architectural decision; an unchanged decision
does not need a replacement ADR.
**Original:** [13-documentation D3](13-documentation.md#d3--p3-reporting-adr-still-directs-readers-to-a-planned-implementation).
## F26 — P3: The README duplicates volatile module compatibility guidance
The root README repeats exact version combinations and candidate dependency
details already maintained by the consumer guide. The facts currently agree, but
two owners create policy and maintenance drift risk.
Keep a brief status/module overview in README and link the consumer guide for
exact compatibility and installation choices. This is documentation
consolidation, not a dependency change.
**Original:** [13-documentation D5](13-documentation.md#d5--p3-the-root-readme-duplicates-the-compatibility-matrix).
## F27 — P3: A historical release note points to a nonexistent section
The ckprompt v0.2.0 release note uses the wrong consumer-guide fragment at its
immutable tag. The file exists, but the requested section does not.
Provide the correct pinned section link in a useful forward-only correction or
erratum. Preserve the published tag, historical note, and hosted release
contents; republishing is unnecessary.
**Original:** [13-documentation D4](13-documentation.md#d4--p3-a-historical-release-note-links-to-a-nonexistent-section).