base-layer/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Consolidate audit findings and record open decisions

Browse Source
This commit is contained in:
Eric Rakestraw
2026-11-04 18:05:43 +00:00
parent 69203dcb12
commit 668d09997a
3 changed files with 597 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

215
docs/roadmap/audit/05-resources.md Normal file
View File

@@ -0,0 +1,215 @@
# Resource Management Audit
Reviewed 2026-11-04 against commit `fb87fab`.
## Scope And Evidence
Reviewed file and response-body ownership, bounded reads, HTTP client/transport
lifetimes, timeout contexts and retry timers, backend admission and permits,
prepared-handle cleanup, retained snapshots, JSON conversion, rendering buffers,
schema resources, and repair feedback. The architecture uses immutable
engine-lifetime configuration, invocation-local execution/validation resources,
and caller-owned public results. Production code launches no PromptKit-owned
goroutines or background workers requiring a shutdown protocol.
Evidence consists of source/call-graph review, relevant existing tests, and a
small temporary offline diagnostic outside the repository. The diagnostic used
a controlled HTTP transport, counted body reads/closure, inspected the existing
prepared-handle owner reference, and compared string backing addresses. No
benchmark, heap-size measurement, deliberate exhaustion, live network call,
production change, or new repository test was needed.
Focused existing tests passed for content-file closure, HTTP error-body limits
and closure, independent attempt contexts/body closure, read cancellation,
prepared discard/ownership, diagnostic formatting, synchronous schema
cancellation, and repeated-value JSON work bounds. These tests establish the
existing protected behavior; they do not cover all findings below.
## 1. Authentication Diagnostics Read An Unbounded Upstream Body
**Priority:** P2. Unbounded remote-response consumption; a finite limit would be
a new policy decision, not enforcement of an existing documented byte limit.
**Code:** [runOpenRouterKey](../../../internal/usecase/smoke_authentication.go:66),
[recognizedOpenRouterKeyResponse](../../../internal/usecase/smoke_authentication.go:139),
and [diagnostic HTTP execution](../../../internal/llm/http_evidence.go:129).
**Circumstances:** An authentication endpoint returns HTTP 200 with a large or
continuously growing body. Recognition calls unrestricted `io.ReadAll` and then
unmarshals the whole document into raw-message maps. Neither declared length nor
streamed bytes are bounded on this path. Generation's 16 MiB success-body limit
and 64 KiB error-body limit do not apply to diagnostic GETs.
**Consequences / evidence:** The offline diagnostic supplied a valid small
identifying `data` object plus 4 MiB of irrelevant padding. The probe read all
4,194,375 bytes, closed the body, and reported credential acceptance. Increasing
the padding has no corresponding byte-budget check. A fast misconfigured or
unexpected endpoint can consume substantial memory before its timeout; concurrent
authentication probes multiply consumption because HTTP-only diagnostics bypass
generation admission. A timeout bounds elapsed time, not bytes allocated within
that time. There is no demonstrated descriptor leak: the body is closed.
**Potential remediation:** Select a finite body budget appropriate to this small
diagnostic protocol, enforce it for both declared and streamed/underreported
lengths, and close without unbounded draining on overflow. Preserve single-JSON
framing and the distinction between unknown acceptance and observed rejection;
an oversized response must not be reported as credential rejection. Decide the
budget and overflow report classification explicitly, then update the owning
contract and tests. This need not impose new limits on application prompts,
artifacts, or generated output.
## 2. Convenience Source Conversion Expands Shared Graphs Before Bounded Loading
**Priority:** P2. Potentially exponential allocation before upstream resource
checks can run.
**Code:** [providerMapForMapping](../../../provider_values.go:19),
[providerValueForMapping](../../../provider_values.go:31),
[ProfileSource](../../../profiles.go:33), and
[BackendSource](../../../backends.go:100).
**Circumstances:** A caller builds a small acyclic Go value graph containing
repeated map/slice references. Each level can point twice to the same child:
```go
value := map[string]any{"value": true}
for range depth {
value = map[string]any{"left": value, "right": value}
}
```
The source converter tracks only active ancestors to identify cycles. Once one
branch finishes, its visit is removed, so the sibling independently converts
the same entire child. It allocates a fresh expanded tree before calling
`configkit.Mapping`. For d levels, O(d) distinct input maps become
Theta(2^d) output maps. The depth guard does not constrain width or total work;
this expansion can become impractical well below depth 100.
**Consequences:** Programmatic source creation can exhaust memory before
`NewEngine` or ConfigKit's deferred diagnostics are reached. This is not a claim
that ConfigKit lacks bounds: the selected ConfigKit v1.1.0's mapping builder
checks node, byte, and depth budgets during its own snapshot traversal
(`internal/engine/value.go`, `valueBuilder.build`). It is called only after
PromptKit has already expanded the graph. PromptKit's later `jsonvalue` node
budget is also too late to protect this conversion.
**Potential remediation:** Memoize completed structural conversions so repeated
input containers reuse already-converted containers until ConfigKit snapshots
them with its existing bounds. Keep active-ancestor tracking distinct from the
completed memo so cycles still become deferred source errors. Account correctly
for container identity, including distinct slice views. Alternatively, seek
upstream support for the additional Go value forms so this conversion can be
removed. Preserve immediate source ownership and deferred failure behavior;
do not introduce a PromptKit raw-source validation pass or duplicate ConfigKit's
configuration limits to fix this pre-handoff allocation problem.
## 3. Inert Prepared Handles Retain Their Entire Execution Owner
**Priority:** P3. Avoidable retained-memory dependency after use/discard.
**Code:** [PreparedExecution.owner](../../../internal/usecase/prepared_execution.go:24),
[Discard](../../../internal/usecase/prepared_execution.go:96),
[claim](../../../internal/usecase/prepared_execution.go:172), and
[payload clearing](../../../internal/usecase/prepared_execution.go:188).
**Circumstances:** A consumer retains a completed or discarded handle for its
supported `Details` access, while replacing or dropping the originating engine.
Payload clearing releases the per-execution plan, credential, observer, and
execution snapshot, but the handle's `owner *Runner` remains unchanged.
**Consequences / evidence:** The reference path is
`handle -> Runner -> captured repositories/clients/validator/capacity manager`.
Thus a small inert handle keeps the originating engine's complete runtime
snapshot and collaborators reachable, not just its deliberately retained details.
The offline diagnostic confirmed the same nonnil runner pointer after both
discard and successful execution. Retaining handles from successive engine
reconstructions can retain successive large snapshots. Keeping details itself
is required behavior, and this has no additional owner-retention cost while the
application still retains the same engine anyway.
**Potential remediation:** Clear the owner reference when the handle becomes
inert, checking ownership under the same mutex to avoid racing the current
unlocked owner read. Another option is a small standalone identity token shared
by a runner and its handles, which establishes ownership without retaining the
runner graph. Such a token must have reliable distinct identity; do not rely on
distinct zero-sized allocations comparing unequal. Preserve related namespace
views, foreign-owner rejection, single claim, discard/run races, and details
availability. No engine `Close` method or automatic closing of caller-owned
collaborators is implied.
## 4. Repair Feedback's Output Bound Does Not Bound Its Temporary Serialization
**Priority:** P3. Avoidable memory amplification on validation failures.
**Code:** [formatRepairDiagnostics](../../../internal/usecase/repairer.go:119).
**Circumstances:** A failed schema validation produces a large diagnostic or many
diagnostics and repair is enabled. The formatter normalizes every string, marshals
the complete array, then discovers that it exceeds 64 KiB. It subsequently
marshals individual full entries again before truncating one of them.
**Consequences:** Although the returned feedback is bounded correctly, temporary
storage scales with the complete diagnostic text, including text omitted from
the request. JSON escaping can amplify its size, and normalization adds another
copy for invalid UTF-8. Large validation messages and concurrent repairs can
therefore create large additional buffers despite producing tiny feedback.
The complete diagnostics already exist for the validation result; these extra
full encodings do not help preserve them. Existing formatting tests check output
bounds/data preservation, not scratch-space use.
**Potential remediation:** Encode feedback incrementally with bounded buffering
and lookahead sufficient to distinguish a complete array from an omitted suffix.
Stop encoding omitted text rather than allocating its full JSON representation.
Keep the existing 64 KiB result bound, ordered entries, rune-safe truncation,
valid UTF-8, JSON escaping, and in-bound omission notice. Leave the original
validation diagnostics and full previous candidate unchanged. This is an
implementation opportunity, not a claim that the existing contract promises a
64 KiB peak heap footprint or permission to add a diagnostic-input limit.
## 5. Short Normalized Session IDs Can Pin Large Input Strings
**Priority:** P3. Avoidable backing-buffer retention in retained metadata.
**Code:** [NormalizeSessionID](../../../internal/domain/session.go:10),
[preparation](../../../internal/usecase/runner.go:312), and
[prepared metadata construction](../../../internal/usecase/runner.go:491).
**Circumstances:** A direct or rendered session ID contains substantial leading
or trailing whitespace around a short valid identifier. `strings.TrimSpace`
returns a substring; normalization validates that substring and returns it
without detaching its backing storage. Public conversion preserves the string.
**Consequences / evidence:** An offline public-API probe prepared a request with
1 MiB of spaces followed by `s`. The returned session ID had length one but its
backing address still pointed inside the large original allocation. Retaining a
prepared value, handle details, or run result can therefore retain that buffer
after the original request/rendering buffer would otherwise be collectible.
The 256-code-point semantic limit does not prevent this retention. This does not
require changing which padded identifiers are accepted.
**Potential remediation:** Detach accepted normalized session metadata with a
small owned string, for example `strings.Clone` after validation. Preserve empty
omission, Unicode trimming/counting, hashes, and wire content. Review other
short canonical metadata returned by trimming for the same pattern when making
the change; do not indiscriminately clone large prompt/message payloads or add a
new raw-session-size restriction.
## Cleanup Checks And Non-Findings
No additional descriptor, timer, connection-ownership, or goroutine leak was
established. Artifact/schema handles use deferred closure; prompt content handles
close after reading; generation response bodies close on success and decoding,
status, and size failures. Diagnostic bodies close, and their timeout cancellation
is attached to body closure. Retry timers stop on exit. Admission/active permits
release on errors, cancellation, and panic unwinding. Prepared payloads otherwise
clear as intended.
Caller-selected artifact/schema/prompt sizes, intentionally unlimited backends,
and caller-retained ready handles are not classified as bugs merely because
their total memory can grow. Synchronous dependency calls may delay cancellation
by design, rather than being detached into leaking goroutines. Borrowed HTTP
transports/filesystems/clients remain subject to their ownership contracts; this
audit proposes no blanket shutdown or disposal API.
Previously recorded repeated-copying issues remain in
[the efficiency audit](02-efficiency.md); they are not duplicated here.

197
docs/roadmap/audit/06-error-handling.md Normal file
View File

@@ -0,0 +1,197 @@
# Error-Handling Audit
Reviewed 2026-11-04 against commit `6b1323d`.
## Scope And Evidence
Reviewed error detection, wrapping, public translation, diagnostic reporting,
cancellation and timeout handling, retries, validation/repair failures, and
cleanup across construction, configuration inspection, prompt/profile selection,
artifact reading, rendering, ordinary and prepared execution, generation, and
smoke diagnostics, plus the small release-entrypoint wrappers. Contracts were
checked against GoDoc, integration references, the architecture policy, ADR 0001,
and existing tests.
Configuration composition belongs to ConfigKit; PromptKit validates and
interprets its final handoff and preserves upstream load errors. The runner adds
operation context, generation coordinates attempts and status retries, the HTTP
adapter classifies/redacts provider failures, and the public facade translates
internal categories. Content-validation failures and concluded diagnostic
failures are generally data rather than Go errors under their respective
contracts. Prepared handles permit one execution claim; failure after a
successful claim does not make the handle reusable.
Evidence includes source/call-graph review and a temporary public-API diagnostic
outside the repository. It used injected collaborators and a controlled HTTP
transport; it made no live network calls. No production code or repository
tests were changed.
Existing error, failure, cancellation, timeout, panic-cleanup, and retry tests
passed using:
```sh
go test -count=1 -run 'Error|Failure|Fail|Cancel|Timeout|Panic|Release|Retry|Retries' \
. ./internal/llm ./internal/generation ./internal/usecase \
./internal/capacity ./internal/validate ./internal/artifact \
./internal/configuration
```
These tests establish protected behavior but do not cover the three reproductions
below. Findings from the earlier correctness and resource-management audits are
not repeated here.
## 1. Nested Collaborator Errors Can Hide The Owning Operation's Public Category
**Priority:** P2. Confirmed public error-classification defect.
**Code:** [publicErrorFor](../../../errors.go:63),
[mapPublicError](../../../errors.go:16), and
[wrapGenerationError](../../../internal/usecase/runner.go:271).
**Failure condition:** An injected `LLMClient` or `ArtifactReader` returns an
error obtained from another PromptKit operation. Ordinary mapped errors retain
their internal chain. `publicErrorFor` searches that complete chain with
`errors.Is` and chooses the first matching category in a fixed priority order,
regardless of which operation the outer runner classified.
**Expected:** A model-client failure matches `ErrLLMGenerate`; an artifact-reader
failure matches `ErrArtifactLoad`. The injected cause remains available through
`errors.Is`, as their [public GoDoc](../../../engine.go:49) specifies. Preserving
the nested cause may also preserve its public categories; it must not replace
the category for the operation that just failed.
**Actual / reproduction:** A child engine's `Run` returns a missing-prompt error.
A parent engine selects a valid prompt, then its injected client returns that
child error. The outer error contains the text `failed to generate output`, but:
```text
errors.Is(err, ErrLLMGenerate) = false
errors.Is(err, ErrPromptNotFound) = true
errors.Is(err, childError) = true
```
Returning the same child error from an injected artifact reader similarly
produces `errors.Is(err, ErrArtifactLoad) == false`. This is reproducible using
only public APIs, without importing internal sentinels. Consumers can therefore
miss the documented failure category when composing PromptKit-backed adapters.
**Potential remediation:** Translate the owning operation's classification
independently of an arbitrary collaborator's nested chain, while retaining the
original injected cause. Explicit boundary/stage information or mapping at the
operation boundary can do this. Merely reordering the global switch risks
breaking intentional direct-selection behavior: a genuinely missing prompt or
profile must retain its documented distinction from other load failures.
Protect nested-client and nested-reader cases through public-boundary regression
coverage; preserve existing capacity, retry-exhaustion, provider-error redaction,
and context identities.
## 2. Successful-Status Body Read Failures Are Misreported As Malformed JSON
**Priority:** P2. Confirmed loss of failure provenance and cause.
**Code:** [decodeOpenAIChatResponse](../../../internal/llm/openai_compatible_client.go:373).
**Failure condition:** A 2xx response body returns a non-EOF I/O error other than
`context.Canceled` or `context.DeadlineExceeded`, either during the first decode
or while checking the required final EOF. For example, a connection can fail
while reading the response stream after headers were received.
**Expected:** Distinguish an underlying read/transport failure from evidence of
invalid JSON or trailing data. Retain the read cause through a redacting error
so callers can inspect it with `errors.Is`/`errors.As`, consistent with the
[transport category](../../../engine.go:65). Do not report trailing data when the
reader failed without supplying any. This does not change the established
classification of invalid/truncated JSON or introduce transport retries.
**Actual / reproduction:** The controlled transport returned HTTP 200 and a body
that supplied either a partial JSON prefix or a complete valid chat envelope,
then returned a distinct read-error sentinel. Both executions returned
`ErrLLMGenerate` plus `ErrMalformedResponse`, did not match `ErrTransport`, and
discarded the sentinel. For the complete envelope, the error said `response
contains trailing data`, although no trailing bytes were supplied. Inference
smoke diagnostics likewise reported `malformed_response` with nil Go error
rather than a transport failure. Thus the issue affects both public execution
errors and factual diagnostic classification.
The integration contract explicitly guarantees exact causes for
[`http.Client.Do` failures](../../integrations/openai-compatible-chat.md#response-handling),
not every decoder error. The finding concerns the separate loss and
misclassification of an actual body-read failure, rather than asserting a new
requirement to expose JSON parser errors or provider content.
**Potential remediation:** Track underlying reader failures separately from
decoder/framing failures, or reliably distinguish those classes before replacing
the decode error. Reuse the redacting transport-error path for genuine read
failures. Keep syntax/type errors, EOF-truncated JSON, extra JSON values, and
size overflow in the existing malformed-response category; preserve cancellation
precedence and bounded reads/closure. Add focused coverage for interrupted
initial decoding and interrupted EOF verification, including diagnostic failure
classification. Default formatting must remain free of raw response content and
private transport details.
## 3. Inference Diagnostics Swallow Some Invalid Request-Construction Errors
**Priority:** P2. Confirmed mismatch with the diagnostic error contract.
**Code:** [runInference](../../../internal/usecase/smoke_tester.go:207),
[generation preparation](../../../internal/generation/coordinator.go:96), and
[request payload validation](../../../internal/llm/openai_compatible_client.go:511).
**Failure condition:** An inference diagnostic's effective settings are invalid
in a way detected by built-in request construction rather than target selection.
A runtime `Execution.ExtraParams` override containing the reserved key `model`
is one public example. The coordinator returns `llm.ErrInvalidRequest` before
dispatch with no attempt evidence. `runInference` treats every generation error
as a concluded diagnostic failure.
**Expected:** The [SmokeTest contract](../../../smoke_diagnostics.go:238) says invalid
kinds, settings, and selections return an ordinary Go error before diagnostic
work. A locally invalid request should follow that path and expose the public
invalid-request category. This is distinct from a provider rejecting a validly
constructed request or a diagnostic concluding with an observed failure.
**Actual / reproduction:** Calling text-inference `SmokeTest` with
`Execution.ExtraParams: map[string]any{"model": "override"}` returned nil Go
error and a report with `Completed == true`, `Failure == invalid_request`, no
attempts, and zero transport calls. The request-construction error and its
explanation were discarded. The caller cannot use the documented ordinary-error
path to identify this configuration mistake.
**Potential remediation:** Preserve the distinction between local built-in
request-preparation failures and concluded execution failures. Return preparation
errors through the use-case/public invalid-request mapping rather than marking
the diagnostic complete; preparation can remain in the generation adapter, with
enough provenance returned to the caller to make that distinction. Do not infer
that every zero-attempt diagnostic is an error: missing required credentials,
capacity rejection, unsupported operations, and inapplicability have intentional
report semantics. Preserve concluded provider failures and the separate
caller-cancellation behavior. Add one public-boundary regression for the
reserved-field override, retaining existing tests for those intentional reports.
## Reviewed Behavior Not Raised As Defects
- ConfigKit owns raw-source composition and errors before the accepted handoff;
no additional PromptKit raw-source validation is proposed.
- Non-2xx provider detail parsing is deliberately best-effort. Unreadable,
oversized, or unrecognized error bodies retain the received status without
exposing the raw body. That differs intentionally from decoding a 2xx body.
- Retries are limited to selected HTTP statuses. Transport failures, malformed
responses, and arbitrary injected-client failures are not automatically
retried; no expansion is needed for these findings.
- Completed output-validation failures are returned as validation results;
operational schema failures are errors. Repair failure propagation and
exhaustion retain that distinction.
- Caller cancellation, configured request timeouts, cancellation during capacity
acquisition/backoff, synchronous schema operations, and partial diagnostic
evidence have explicit, tested handling. Callback panics propagate while
deferred permit/admission cleanup still runs.
- Ignored SHA-256 writer errors cannot occur with the selected implementation.
Best-effort closure of read-only files and HTTP bodies does not imply a
persistence guarantee or require converting a completed read into a failure.
- Release wrappers use fail-fast shell execution and `exec` to propagate the
selected tool's status. Publication and recovery implementation belongs to
ReleaseKit; no release or publication operation was run for this audit.
No additional state-recovery or cancellation defect was substantiated in this
pass. The findings do not call for a new recovery subsystem, automatic replay of
possibly completed provider work, or changes to source ownership.

570
docs/roadmap/audit/catalog.md Normal file
View File

@@ -0,0 +1,570 @@
# Consolidated Pre-1.0 Audit Catalog
Consolidated on 2026-11-04 from the 11 audit documents in this directory.
Their 41 numbered findings are represented by 34 entries below. Entries that
describe the same underlying issue share one remediation scope; missing
regression coverage for an identified defect is included with that defect.
Some broad testing findings consequently support more than one entry.
The order reflects consequence, exposure, and the value of clarifying public
contracts before v1.0. Confirmed defects and conditional security exposure come
first, followed by contract/test gaps, resource efficiency, and maintainability.
This ordering does not upgrade the original severities or turn every cleanup
into a release gate. The source audits retain their evidence, qualifications,
and validation results. None of these entries is marked remediated by this
documentation-only consolidation.
## 1. Update The Audited Maintainer Toolchain For Applicable Security Fixes
**Conditional security exposure.** The audited Go 1.27.0 toolchain predates
networking fixes applicable to HTTP clients, including HTTP/2 CPU exhaustion
and conditional certificate-trust exposure. Use a patched maintainer toolchain
and repeat the focused security assessment against it. Actual downstream
exposure depends on the toolchain that builds the consumer and its transport
and trust configuration; raising PromptKit's minimum Go requirement is a
separate compatibility decision, not an implied remedy.
Sources: [Security 1](07-security.md#1-the-audited-go-toolchain-predates-applicable-networking-security-fixes).
## 2. Bound Authentication-Probe Response Consumption
**Confirmed resource/security defect.** Authentication diagnostics read a
successful upstream body with unbounded `io.ReadAll`. A deadline does not bound
the bytes a fast server can supply. Introduce finite response consumption and
clean closure, with an explicit choice of byte budget and overflow reporting.
Overflow should not be interpreted as proof of rejected authentication.
Relocating the provider adapter in entry 26 would not itself fix this defect.
Sources: [Resources 1](05-resources.md#1-authentication-diagnostics-read-an-unbounded-upstream-body),
[Security 2](07-security.md#2-authentication-probes-permit-unbounded-remote-body-memory-consumption).
## 3. Avoid Exponential Expansion During Convenience Source Conversion
**Confirmed resource defect.** `ProfileSource` and `BackendSource` conversion
expand repeated references before ConfigKit performs its bounded snapshot.
A small shared acyclic graph can therefore cause exponential work and
allocation. Preserve shared references during conversion, with suitable cycle
and slice-view handling, rather than duplicating ConfigKit's validation or
introducing PromptKit-side raw-source limits. Add focused shared-graph and
deferred-failure regressions for both convenience helpers.
Sources: [Resources 2](05-resources.md#2-convenience-source-conversion-expands-shared-graphs-before-bounded-loading),
[Testing 3](08-testing.md#3-programmatic-provider-value-conversion-lacks-representative-value-cases).
## 4. Resolve The ArtifactReader Buffer-Ownership Contract
**Confirmed race under the documented collaborator contract.** The public
reader contract permits buffer reuse after `Read` returns, before PromptKit
can finish copying it. Concurrent preparation with a compliant custom reader
can race. Clarify ownership/lifetime per invocation and align adapters, GoDoc,
and concurrency tests. This needs a deliberate public-contract decision before
v1.0: an immediate copy alone cannot guarantee safety if the reader is allowed
to mutate the buffer immediately. No default-reader defect was established.
Sources: [Interfaces 1](09-interfaces.md#1-artifactreader-permits-buffer-reuse-before-promptkit-can-copy-it).
## 5. Preserve Exact Numbers In Programmatic Provider Sources
**Confirmed data-integrity defect.** Convenience source conversion falls back
from integer parsing to `float64` without checking precision, silently changing
some accepted `json.Number` values. Preserve an exact ConfigKit-compatible
representation, or defer an unsupported-value error to ConfigKit instead of
rounding or converting the number into a string. Regression coverage should
include exact-number boundaries, nested typed collections, mutation isolation,
and deferred unsupported/cyclic-value errors for both source helpers.
Sources: [Correctness 1](01-correctness.md#1-provider-source-helpers-silently-change-exact-json-numbers),
[Testing 3](08-testing.md#3-programmatic-provider-value-conversion-lacks-representative-value-cases).
## 6. Make Provider-Native Schema Constraints Self-Contained
**Confirmed implementation gap; packaging policy needs selection.** Local
validation freezes referenced schema resources, but the provider receives only
the root document and cannot resolve its local external references. Select
correct resource packaging, or an explicit local-only fallback policy, without
naively inlining recursive schemas or silently dropping native constraints.
Add a wire-level regression demonstrating a usable provider constraint, not
merely successful local compilation. Entry 21 is a separate compilation
refactor and does not solve this packaging problem.
Sources: [Correctness 2](01-correctness.md#2-external-schema-resources-do-not-reach-the-provider-constraint),
[Testing 5](08-testing.md#5-multi-resource-schema-tests-do-not-establish-a-usable-provider-constraint).
## 7. Preserve The Owning Operation's Public Error Category
**Confirmed classification defect.** Public error conversion searches nested
causes in a global priority order, allowing a collaborator's sentinel to hide
the enclosing operation's category. Map the owning boundary independently
while retaining the original cause for `errors.Is`/`errors.As`. Add composed
cause regressions; simply reordering global checks can break legitimate direct
not-found versus load-failure distinctions.
Sources: [Error handling 1](06-error-handling.md#1-nested-collaborator-errors-can-hide-the-owning-operations-public-category),
[Testing 4](08-testing.md#4-error-tests-miss-composed-causes-and-pre-dispatch-diagnostic-failures).
## 8. Distinguish Response Stream Failures From Malformed JSON
**Confirmed error-handling defect.** Non-context I/O failures while decoding a
successful HTTP response or checking its end are reported as malformed JSON,
losing the transport cause; complete JSON followed by a read failure can also
be misreported as trailing data. Retain and classify reader failures separately
using the existing redacted transport boundary. Preserve cancellation
precedence and genuine malformed/truncated/oversized-content handling. Cover
both read locations without adding transport retries.
Sources: [Error handling 2](06-error-handling.md#2-successful-status-body-read-failures-are-misreported-as-malformed-json),
[Testing 4](08-testing.md#4-error-tests-miss-composed-causes-and-pre-dispatch-diagnostic-failures).
## 9. Return Invalid Diagnostic Request Construction As A Go Error
**Confirmed diagnostic-contract defect.** Some invalid inference settings fail
before dispatch yet produce a completed report with a nil Go error. Propagate
local request-construction failures through `ErrInvalidRequest` and add a
no-dispatch regression. Preserve intentionally completed diagnostic outcomes
for missing credentials, capacity outcomes, and unsupported or inapplicable
operations; not every no-dispatch outcome is invalid configuration.
Sources: [Error handling 3](06-error-handling.md#3-inference-diagnostics-swallow-some-invalid-request-construction-errors),
[Testing 4](08-testing.md#4-error-tests-miss-composed-causes-and-pre-dispatch-diagnostic-failures).
## 10. Remove Quadratic Requirement-Evidence Union Work
**Confirmed algorithmic inefficiency.** A left-heavy recognized condition can
repeatedly copy growing missing-key evidence sets, producing quadratic work.
Use operation-local accumulation or ownership-aware merging while preserving
classification and conservative inspection coverage. Do not address this by
adding expression limits or expanding the recognized template language. Entry
23's traversal cleanup is related but is not a substitute for fixing this
specific accumulation algorithm.
Sources: [Efficiency 1](02-efficiency.md#1-condition-evidence-unions-can-require-quadratic-work).
## 11. Exercise RunRequest's Privacy Formatting Boundary
**Missing protection, not a demonstrated leak.** Request formatting methods
have no execution coverage despite owning a privacy-sensitive public boundary.
Add focused tests with distinct synthetic private markers for the relevant
value/pointer formatting paths and JSON omission behavior. Assert privacy and
the documented boundary, not an incidental complete diagnostic or JSON shape.
Sources: [Testing 1](08-testing.md#1-runrequests-privacy-formatting-contract-is-not-exercised).
## 12. Correct Profile Inspection Credential Documentation
**Public-contract documentation defect.** Inspection GoDoc incorrectly makes
required credentials and an environment variable mutually exclusive. Actual
behavior permits an explicit profile environment variable while suppressing
only the backend-derived default when required credentials are selected.
Correct the field/method GoDoc and consumer example, keeping the established
configuration behavior and DTO shape.
Sources: [Interfaces 2](09-interfaces.md#2-profile-inspection-misstates-required-credential-semantics).
## 13. Correct The Configured Provider-Map Wire Documentation
**Integration-contract documentation defect.** The wire document says backend
and profile provider maps replace one another, but configuration recursively
overlays them; runtime replacement is a separate rule. Align the integration
summary with the canonical format reference rather than maintaining a second
conflicting precedence definition. Entry 19 concerns the efficiency of the
existing overlay, not a change to its semantics.
Sources: [Interfaces 3](09-interfaces.md#3-the-wire-contract-incorrectly-says-configured-provider-maps-replace).
## 14. Remove Stale Identity And Version GoDoc
**Public-contract documentation defect.** Inspection comments still describe
trimmed profile IDs, and listing comments describe a removed prompt-version
field. Update them to the implemented literal identity and current collection
version model. Do not normalize meaningful whitespace or reintroduce a field
to match obsolete comments.
Sources: [Interfaces 4](09-interfaces.md#4-identity-godoc-still-describes-removed-normalization-and-version-fields).
## 15. Cover The Public Single-Schema-File Option
**Public wiring coverage gap.** Direct filesystem-validator tests do not
exercise `WithSchemaFile` through engine construction. Add offline temporary
file cases through public preparation/validation, including option precedence
and relevant file errors. This is a supported feature, not dead code; entry
32 removes only an unused private helper result.
Sources: [Testing 2](08-testing.md#2-the-public-single-schema-file-option-has-no-execution-coverage).
## 16. Detach Inert Prepared Handles From The Execution Owner
**Avoidable retention.** Used or discarded prepared handles retain their
runner, and therefore the engine's captured configuration and collaborators.
Use a narrower ownership identity or clear the unnecessary owner reference
safely. Preserve foreign-owner rejection, one-time claim, namespace-view
behavior, and available details. This is unnecessary retention when a handle
outlives its engine, not evidence of an ever-growing global leak.
Sources: [Resources 3](05-resources.md#3-inert-prepared-handles-retain-their-entire-execution-owner).
## 17. Bound Repair Feedback's Temporary Serialization
**Avoidable peak allocation.** Feedback formatting serializes potentially
large diagnostic text before truncating the final message, so its existing
output bound does not bound scratch memory. Produce bounded encoded feedback
incrementally, preserving the existing output semantics and complete stored
validation diagnostics and previous candidate. A new raw-diagnostic input
limit is not implied.
Sources: [Resources 4](05-resources.md#4-repair-feedbacks-output-bound-does-not-bound-its-temporary-serialization).
## 18. Detach Normalized Session IDs From Large Input Strings
**Avoidable retention.** A short trimmed session ID can retain the allocation
of a much larger padded caller string. Clone the accepted normalized value at
its ownership boundary. Preserve trimming, accepted content, code-point
validation, and resulting wire/hash content rather than adding a raw-input
size restriction.
Sources: [Resources 5](05-resources.md#5-short-normalized-session-ids-can-pin-large-input-strings).
## 19. Avoid Repeated Descendant Cloning During Provider Overlay
**Construction-time inefficiency.** Recursive overlays repeatedly clone and
sort overlapping descendant maps, giving work proportional to tree size times
depth. Existing depth bounds mean this is not unbounded quadratic growth.
Clone ownership once or use a single traversal while retaining recursive-map,
null, and array semantics. Coordinate with entry 24 without assuming all JSON
walkers have interchangeable policies.
Sources: [Efficiency 2](02-efficiency.md#2-recursive-provider-overlay-repeatedly-clones-descendant-subtrees).
## 20. Index Filtered Listings By Namespace
**Repeated inspection overhead.** Filtered listing currently copies and scans
the complete inventory even when few or no entries match. Capture namespace
buckets or equivalent ranges so queries operate on the relevant subset.
Preserve ordering, exact identity matching, cancellation, and caller-owned
results. A pagination API or runtime cache is not needed for this improvement.
Sources: [Efficiency 3](02-efficiency.md#3-filtered-listings-copy-and-scan-the-complete-inventory).
## 21. Share Schema Compilation And Declare The Runner's Real Dependency
**Maintainability and interface clarity.** Four host-file/`fs.FS`,
direct/prepared paths repeat the schema compilation lifecycle. The runner also
declares a validator interface while requiring preparation through a runtime
assertion. Share the lifecycle behind source-specific adapters and make that
dependency explicit. Preserve malformed-output short-circuiting before schema
I/O, source containment, exact numbers, cancellation, and frozen plans. Retain
independent cross-source tests; do not force whole entry points to be identical.
Sources: [Complexity 1](03-complexity.md#1-consolidate-schema-compilation-and-express-the-runners-actual-interface),
[Equivalence 1](10-equivalence.md#1-share-the-schema-compilation-lifecycle-across-source-and-execution-paths).
## 22. Decompose Configuration Validation And Materialization
**Maintainability.** Large configuration operations mix record-specific
validation, resource capture, and effective-profile construction. Extract
focused helpers with visible inputs and ownership, rather than another generic
validation framework. Preserve final-handoff validation, presence/null rules,
origins, and meaningful fail-fast/error-precedence behavior. Do not turn
incidental ordering of independent validation failures into a new guarantee. The configuration-boundary ADR remains
authoritative; this is not a proposal to inspect raw ConfigKit sources.
Sources: [Complexity 2](03-complexity.md#2-decompose-configuration-validation-and-resource-materialization).
## 23. Clarify Requirement Traversal And Share Branch Orchestration
**Maintainability.** Separate recognition/evaluation from reachability and
evidence bookkeeping, particularly the repeated `if`/`with` branch protocol.
Keep their scope differences explicit and retain independent comparisons with
the actual Go renderer. Preserve conservative partial coverage and inactive
computed-name behavior. This does not justify a general template interpreter
or stronger execution-readiness promises from requirements inspection.
Sources: [Complexity 3](03-complexity.md#3-untangle-requirement-traversal-expression-evaluation-and-evidence),
[Equivalence 3](10-equivalence.md#3-share-ifwith-requirement-branch-orchestration-not-template-execution).
## 24. Centralize Compatible JSON Copying Without Conflating Policies
**Maintainability and ownership clarity.** The facade maintains a separate
reflection traversal for already-normalized values despite an existing
`internal/jsonvalue` owner. Consolidate genuinely compatible structural
copying while retaining DTO projection at the facade and distinct source
encoding/runtime validation policies. Preserve concrete types, nils, exact
numbers, schema keys, and ownership. Infallible projection must not acquire
new errors, panics, or silent zero-value fallbacks. Entries 3 and 5 remain
separate source-conversion defects, not reasons to reject pre-handoff sources.
Sources: [Complexity 4](03-complexity.md#4-clarify-ownership-and-policies-across-the-json-value-walkers),
[Equivalence 2](10-equivalence.md#2-centralize-structural-copying-of-already-normalized-json-values).
## 25. Separate Attempt Accounting From Generation Control Flow
**Maintainability.** Generation coordination interleaves dispatch/retry policy
with parallel return values and repeated evidence/observation projection.
Introduce a small attempt result and focused accounting helpers. Preserve
built-in versus injected-client differences, permit release before callbacks
or backoff, frozen requests, existing retry classification, private evidence,
and known/unknown usage distinctions. Do not collapse legitimately different
dispatch strategies merely to reduce lines.
Sources: [Complexity 5](03-complexity.md#5-extract-attempt-accounting-from-generation-control-flow).
## 26. Move OpenRouter Authentication Protocol Details Into Its Adapter
**Architectural ownership.** Use-case code currently interprets OpenRouter's
account-response fields and provider-relative endpoint while the outbound
adapter owns HTTP execution. Place protocol construction/recognition with the
outbound integration; keep diagnostic selection, applicability, cancellation,
completion, and neutral reporting in the use case. Preserve credential/body
privacy and anonymous probing. No plugin framework, reverse dependency, or
blanket prohibition on HTTP facts is needed. Coordinate the body-bound fix
in entry 2, which is a distinct behavioral change.
Sources: [Boundaries 2](11-boundaries.md#2-put-openrouter-authentication-protocol-details-behind-the-outbound-adapter).
## 27. Narrow Smoke-Value Copy Helpers To Real Consumers
**Ownership cleanup.** A request cloner has only its self-test as a caller;
the full-report cloner is used to detach only a target. Express that actual
copy operation directly and retire unused broader cloning after checking
callers. Keep facade request/report projection, active backend cloning,
provider-map errors, ownership tests, and redaction. Do not introduce shared
public/internal DTOs or an additional full-report copy.
Sources: [Boundaries 3](11-boundaries.md#3-narrow-smoke-value-copy-helpers-to-their-actual-ownership-boundaries).
## 28. Give Bundled Catalog Expectations One Test Owner
**Test duplication.** Complete bundled profile/model/provider expectations are
repeated across several test layers. Keep one independently specified asset
oracle, with smaller tests for distinct loading, wire, and diagnostic behavior.
Use controlled fixtures where bundled churn is irrelevant and structural
assertions rather than formatted-map strings. Retain meaningful bundled
coverage instead of deriving every expected value from the loader under test.
Sources: [Testing 7](08-testing.md#7-bundled-catalog-expectations-have-multiple-test-owners).
## 29. Stop Pinning The Opaque Artifact Hash Algorithm In Tests
**Over-specified tests.** Artifact tests and a benchmark assert a literal hash
algorithm despite the public value being opaque. Assert the documented
equality/ownership relationships and actual byte/size behavior instead; keep
benchmark setup outside its timed work. Coordinate with entry 31, but do not
turn today's hash representation into a new compatibility guarantee.
Sources: [Testing 6](08-testing.md#6-artifact-tests-and-benchmarks-pin-an-explicitly-opaque-hash-algorithm).
## 30. Remove Unconsumed Backend Presence Bookkeeping
**Low-priority state cleanup.** Backend field-state maps and
`QueueCapacitySet` have no runtime reader; effective values and ConfigKit
already own queue defaulting and provenance. Remove only these unused records
and adjust tests to assert effective behavior or operator inspection. Keep
profile/input presence tracking, explicit zero capacity, and derived defaults.
Sources: [Dead code 1](12-deadcode.md#1-remove-backend-presence-bookkeeping-with-no-runtime-consumer).
## 31. Retire Duplicate Fields In Captured Prompt Definitions
**Low-priority representation cleanup.** Captured definitions duplicate the
output format and carry an unused runtime message-file field after content
materialization. Use the captured output contract and text as their respective
sources of truth. Preserve real YAML `content_file` loading, origins, and the
request-override default rule. Removing serialized internal fields changes
current definition hashes: retain documented opaque equality relationships,
not an invented bit-identical encoding promise.
Sources: [Dead code 2](12-deadcode.md#2-retire-legacy-fields-in-the-captured-prompt-representation).
## 32. Remove Unused Private Diagnostic State And Helper Results
**Low-priority cleanup.** A private reachability outcome echoes an endpoint
that report assembly never reads; the schema-file helper returns a directory
that its only caller discards. Remove the unused state/result while preserving
actual sanitized request/report evidence, directory calculation, file checks,
and public single-file schema support. Keep privacy tests against the values
that real execution consumes.
Sources: [Dead code 3](12-deadcode.md#3-remove-unused-diagnostic-state-and-a-discarded-helper-result).
## 33. Remove Orphan Internal Helpers And Their Self-Tests
**Low-priority cleanup.** `filecatalog.RelativePath` and
`resourceNamespaceFromID` have only their own tests as callers, and the runner
test's `stringPtr` helper is entirely unused. Remove those narrow surfaces
without replacement, preserving real discovery, identity qualification, and
namespace inventory/filtering coverage. Reachability from examples alone is
not a reason to remove optional public APIs.
Sources: [Dead code 4](12-deadcode.md#4-remove-orphan-helpers-whose-only-callers-are-their-own-tests).
## 34. Decide Whether To Retain The Legacy Capacity Client Wrapper
**Decision-dependent architectural cleanup.** The old capacity client wrapper
is used by tests, whereas production dispatch uses the generation coordinator.
It is also explicitly retained in current internal documentation; the dead-code
audit does not classify it as obsolete functionality. If that retention decision
is revised, exercise the production coordinator in runner tests and move any
needed one-shot capacity scaffolding into tests before retiring the wrapper.
Preserve FIFO, cancellation, independent-pool, and panic-release coverage.
This is not a second scheduling algorithm or a confirmed concurrency defect.
Sources: [Boundaries 1](11-boundaries.md#1-retire-the-test-only-capacity-client-as-a-production-scheduling-surface),
[Dead-code retention qualification](12-deadcode.md#intentional-retention-and-related-findings).
## Open Decisions
Work through the decisions below in this order. These are unresolved choices,
not additional findings or approved changes to public contracts. A decision
blocks its dependent remediation, not unrelated fixes. In particular, updating
the maintainer toolchain need not wait for the consumer-minimum decision.
Implementation details and optional cleanup do not become release gates merely
by appearing in this catalog.
### Decision 1: ArtifactReader Result Ownership And Lifetime
**Related finding:** [4](#4-resolve-the-artifactreader-buffer-ownership-contract).
**Question:** What ownership does a successful `Read` transfer, and when may
an injected reader mutate or reuse the returned storage?
**Recommendation:** Transfer an independent result to PromptKit on successful
return. The reader must stop mutating the returned artifact and its body,
including through retained aliases. Fresh per-invocation storage is the simplest
implementation; immutable shared data can remain shared if no collaborator
mutates it. This removes the impossible immediate-reuse permission without
adding synchronization or a release protocol to the interface. Explain the
consumer impact as a deliberate pre-1.0 contract correction.
**Alternative:** Permit borrowing until the enclosing preparation/execution
operation returns. This can support application-managed reuse, but a reader
cannot generally observe that lifetime and concurrent calls need independent
storage or caller coordination. If reuse immediately after `Read` returns is
required, an explicit acknowledgement/release API is another viable design;
it enables pooling at the cost of a larger interface and lifecycle change.
**Resolve before implementation:** Select the ownership/lifetime rule and update
GoDoc, adapter assumptions, and the regression scenario together. Another
unsynchronized copy is not a remedy for the current permission.
### Decision 2: Provider Submission Of Multi-Resource Schemas
**Related findings:** [6](#6-make-provider-native-schema-constraints-self-contained)
and [21](#21-share-schema-compilation-and-declare-the-runners-real-dependency).
**Question:** How should native structured output represent a schema whose
local validation depends on external resources?
**Recommendation:** First investigate packaging the operation's captured
resource graph into a self-contained provider constraint. Check identifiers,
relative references, anchors, recursion, and supported dialects before choosing
an encoding. Prefer correct packaging when it fits a proportionate scope,
while keeping resource completeness distinct from any provider's supported
schema subset. A standalone offline validation of the actual emitted constraint
should establish the selected packaging behavior.
**Alternative:** Offer an explicit local-only validation choice when native
submission cannot represent the resource graph. This preserves useful local
validation and avoids building a general schema transformation system, but
forgoes native generation constraints for that selected mode and needs a clear
consumer-facing selection mechanism. Silently dropping native constraints is
not equivalent to an explicit choice.
**Resolve before implementation:** Establish packaging feasibility and select
the submission/fallback policy, including behavior for unsupported cases.
Coordinate the compilation refactor with that result; it does not itself fix
provider packaging or establish universal provider acceptance.
### Decision 3: Authentication Response Budget And Overflow Outcome
**Related findings:** [2](#2-bound-authentication-probe-response-consumption)
and [26](#26-move-openrouter-authentication-protocol-details-into-its-adapter).
**Question:** What finite response budget should the authentication adapter
apply, and how should a probe report exceeding it?
**Recommendation:** Select a conservative fixed budget for this small protocol,
check actual streamed bytes regardless of declared length, and close on overflow
without unbounded draining. Report the diagnostic failure with acceptance
unknown, retaining any independently observed HTTP facts. Agree the exact byte
count and its representation in the existing diagnostic report before coding;
the catalog does not select either implicitly.
**Alternative:** Expose a caller-configurable budget with a finite default.
This accommodates unusual legitimate responses but expands configuration and
validation surface. It is worthwhile if a concrete consumer requirement cannot
be served by an appropriately chosen fixed budget; it is not necessary merely
to address the demonstrated unbounded read.
**Resolve before implementation:** Choose the size, overflow classification,
and whether configurability is warranted. Keep this fix independent of adapter
relocation so architectural cleanup does not delay the resource remedy.
### Decision 4: Exact Numbers And The Source-Conversion Boundary
**Related findings:** [3](#3-avoid-exponential-expansion-during-convenience-source-conversion),
[5](#5-preserve-exact-numbers-in-programmatic-provider-sources), and
[24](#24-centralize-compatible-json-copying-without-conflating-policies).
**Question:** Which exact numeric values can the convenience helpers preserve
through the current ConfigKit mapping boundary, and what happens to others?
**Recommendation:** Preserve supported values through the complete conversion,
load, and outbound-encoding path; defer an unsupported-value error when exact
preservation is unavailable. Compare numeric meaning rather than spelling or
successful parsing alone. Address shared-graph expansion in the same adapter
with completed-conversion memoization, distinct cycle tracking, and correct
slice-view identity, preserving source capture and deferred failures.
**Alternative:** Request upstream support for additional exact numeric/value
forms, then simplify or remove the relevant PromptKit conversion. This may
provide a cleaner long-term boundary and broader preservation, but introduces
an upstream dependency and should not postpone fixing silent rounding or
exponential expansion. Runtime JSON copying remains a separate policy.
**Resolve before implementation:** Establish the supported numeric round-trip
cases and whether broader upstream support is needed now. Do not substitute
numeric-looking strings or introduce a second raw-source validation system.
### Decision 5: Consumer Go Minimum After The Toolchain Update
**Related finding:** [1](#1-update-the-audited-maintainer-toolchain-for-applicable-security-fixes).
**Question:** Should the security fixes also change PromptKit's declared minimum
consumer Go version?
**Recommendation:** Update the maintainer toolchain and repeat the focused
assessment promptly; initially keep the consumer minimum decision separate.
Advise affected consumers to rebuild with a patched toolchain. This recognizes
that PromptKit publishes source and that downstream build/transport choices
control the deployed exposure.
**Alternative:** Raise the declared minimum to the selected patched release.
This makes the supported floor clearer and steers module consumers toward the
fixed toolchain, but changes compatibility and does not repair already-built
applications. Assess it under the existing release policy rather than treating
it as an automatic consequence of the scanner output.
**Resolve before changing the minimum:** Confirm current advisory applicability,
select the patched version, and decide whether changing the supported consumer
floor is justified. No new recurring release gate is implied.
### Decision 6: Retention Of The Legacy Capacity Client Wrapper
**Related finding:** [34](#34-decide-whether-to-retain-the-legacy-capacity-client-wrapper).
**Question:** Does the wrapper still justify a production surface for its
currently documented test-focused purpose?
**Recommendation:** Retire the production wrapper after moving the runner's
integration test to the production generation coordinator. Keep direct manager
coverage and any useful one-shot scaffolding in tests. This reduces maintenance
and makes assembled scheduling tests reflect production while preserving the
actual FIFO, cancellation, independent-pool, and panic-release scenarios.
**Alternative:** Retain it if a concrete internal use justifies the small
adapter. This avoids test migration and preserves that convenient composition,
but keeps a second assembly surface to understand and maintain. Existing
self-tests alone do not establish a future production need.
**Resolve before removal:** Confirm the intended retention decision and test
ownership. This cleanup can be deferred without blocking the behavioral fixes;
it is not evidence of a second scheduler or a concurrency defect.