Consolidate audit findings and record open decisions
This commit is contained in:
@@ -30,9 +30,9 @@ existing protected behavior; they do not cover all findings below.
|
||||
**Priority:** P2. Unbounded remote-response consumption; a finite limit would be
|
||||
a new policy decision, not enforcement of an existing documented byte limit.
|
||||
|
||||
**Code:** [runOpenRouterKey](../../internal/usecase/smoke_authentication.go:66),
|
||||
[recognizedOpenRouterKeyResponse](../../internal/usecase/smoke_authentication.go:139),
|
||||
and [diagnostic HTTP execution](../../internal/llm/http_evidence.go:129).
|
||||
**Code:** [runOpenRouterKey](../../../internal/usecase/smoke_authentication.go:66),
|
||||
[recognizedOpenRouterKeyResponse](../../../internal/usecase/smoke_authentication.go:139),
|
||||
and [diagnostic HTTP execution](../../../internal/llm/http_evidence.go:129).
|
||||
|
||||
**Circumstances:** An authentication endpoint returns HTTP 200 with a large or
|
||||
continuously growing body. Recognition calls unrestricted `io.ReadAll` and then
|
||||
@@ -63,10 +63,10 @@ artifacts, or generated output.
|
||||
**Priority:** P2. Potentially exponential allocation before upstream resource
|
||||
checks can run.
|
||||
|
||||
**Code:** [providerMapForMapping](../../provider_values.go:19),
|
||||
[providerValueForMapping](../../provider_values.go:31),
|
||||
[ProfileSource](../../profiles.go:33), and
|
||||
[BackendSource](../../backends.go:100).
|
||||
**Code:** [providerMapForMapping](../../../provider_values.go:19),
|
||||
[providerValueForMapping](../../../provider_values.go:31),
|
||||
[ProfileSource](../../../profiles.go:33), and
|
||||
[BackendSource](../../../backends.go:100).
|
||||
|
||||
**Circumstances:** A caller builds a small acyclic Go value graph containing
|
||||
repeated map/slice references. Each level can point twice to the same child:
|
||||
@@ -107,10 +107,10 @@ configuration limits to fix this pre-handoff allocation problem.
|
||||
|
||||
**Priority:** P3. Avoidable retained-memory dependency after use/discard.
|
||||
|
||||
**Code:** [PreparedExecution.owner](../../internal/usecase/prepared_execution.go:24),
|
||||
[Discard](../../internal/usecase/prepared_execution.go:96),
|
||||
[claim](../../internal/usecase/prepared_execution.go:172), and
|
||||
[payload clearing](../../internal/usecase/prepared_execution.go:188).
|
||||
**Code:** [PreparedExecution.owner](../../../internal/usecase/prepared_execution.go:24),
|
||||
[Discard](../../../internal/usecase/prepared_execution.go:96),
|
||||
[claim](../../../internal/usecase/prepared_execution.go:172), and
|
||||
[payload clearing](../../../internal/usecase/prepared_execution.go:188).
|
||||
|
||||
**Circumstances:** A consumer retains a completed or discarded handle for its
|
||||
supported `Details` access, while replacing or dropping the originating engine.
|
||||
@@ -141,7 +141,7 @@ collaborators is implied.
|
||||
|
||||
**Priority:** P3. Avoidable memory amplification on validation failures.
|
||||
|
||||
**Code:** [formatRepairDiagnostics](../../internal/usecase/repairer.go:119).
|
||||
**Code:** [formatRepairDiagnostics](../../../internal/usecase/repairer.go:119).
|
||||
|
||||
**Circumstances:** A failed schema validation produces a large diagnostic or many
|
||||
diagnostics and repair is enabled. The formatter normalizes every string, marshals
|
||||
@@ -170,9 +170,9 @@ implementation opportunity, not a claim that the existing contract promises a
|
||||
|
||||
**Priority:** P3. Avoidable backing-buffer retention in retained metadata.
|
||||
|
||||
**Code:** [NormalizeSessionID](../../internal/domain/session.go:10),
|
||||
[preparation](../../internal/usecase/runner.go:312), and
|
||||
[prepared metadata construction](../../internal/usecase/runner.go:491).
|
||||
**Code:** [NormalizeSessionID](../../../internal/domain/session.go:10),
|
||||
[preparation](../../../internal/usecase/runner.go:312), and
|
||||
[prepared metadata construction](../../../internal/usecase/runner.go:491).
|
||||
|
||||
**Circumstances:** A direct or rendered session ID contains substantial leading
|
||||
or trailing whitespace around a short valid identifier. `strings.TrimSpace`
|
||||
@@ -212,4 +212,4 @@ transports/filesystems/clients remain subject to their ownership contracts; this
|
||||
audit proposes no blanket shutdown or disposal API.
|
||||
|
||||
Previously recorded repeated-copying issues remain in
|
||||
[the efficiency audit](audit/02-efficiency.md); they are not duplicated here.
|
||||
[the efficiency audit](02-efficiency.md); they are not duplicated here.
|
||||
@@ -44,9 +44,9 @@ not repeated here.
|
||||
|
||||
**Priority:** P2. Confirmed public error-classification defect.
|
||||
|
||||
**Code:** [publicErrorFor](../../errors.go:63),
|
||||
[mapPublicError](../../errors.go:16), and
|
||||
[wrapGenerationError](../../internal/usecase/runner.go:271).
|
||||
**Code:** [publicErrorFor](../../../errors.go:63),
|
||||
[mapPublicError](../../../errors.go:16), and
|
||||
[wrapGenerationError](../../../internal/usecase/runner.go:271).
|
||||
|
||||
**Failure condition:** An injected `LLMClient` or `ArtifactReader` returns an
|
||||
error obtained from another PromptKit operation. Ordinary mapped errors retain
|
||||
@@ -56,7 +56,7 @@ regardless of which operation the outer runner classified.
|
||||
|
||||
**Expected:** A model-client failure matches `ErrLLMGenerate`; an artifact-reader
|
||||
failure matches `ErrArtifactLoad`. The injected cause remains available through
|
||||
`errors.Is`, as their [public GoDoc](../../engine.go:49) specifies. Preserving
|
||||
`errors.Is`, as their [public GoDoc](../../../engine.go:49) specifies. Preserving
|
||||
the nested cause may also preserve its public categories; it must not replace
|
||||
the category for the operation that just failed.
|
||||
|
||||
@@ -89,7 +89,7 @@ and context identities.
|
||||
|
||||
**Priority:** P2. Confirmed loss of failure provenance and cause.
|
||||
|
||||
**Code:** [decodeOpenAIChatResponse](../../internal/llm/openai_compatible_client.go:373).
|
||||
**Code:** [decodeOpenAIChatResponse](../../../internal/llm/openai_compatible_client.go:373).
|
||||
|
||||
**Failure condition:** A 2xx response body returns a non-EOF I/O error other than
|
||||
`context.Canceled` or `context.DeadlineExceeded`, either during the first decode
|
||||
@@ -99,7 +99,7 @@ while reading the response stream after headers were received.
|
||||
**Expected:** Distinguish an underlying read/transport failure from evidence of
|
||||
invalid JSON or trailing data. Retain the read cause through a redacting error
|
||||
so callers can inspect it with `errors.Is`/`errors.As`, consistent with the
|
||||
[transport category](../../engine.go:65). Do not report trailing data when the
|
||||
[transport category](../../../engine.go:65). Do not report trailing data when the
|
||||
reader failed without supplying any. This does not change the established
|
||||
classification of invalid/truncated JSON or introduce transport retries.
|
||||
|
||||
@@ -114,7 +114,7 @@ rather than a transport failure. Thus the issue affects both public execution
|
||||
errors and factual diagnostic classification.
|
||||
|
||||
The integration contract explicitly guarantees exact causes for
|
||||
[`http.Client.Do` failures](../integrations/openai-compatible-chat.md#response-handling),
|
||||
[`http.Client.Do` failures](../../integrations/openai-compatible-chat.md#response-handling),
|
||||
not every decoder error. The finding concerns the separate loss and
|
||||
misclassification of an actual body-read failure, rather than asserting a new
|
||||
requirement to expose JSON parser errors or provider content.
|
||||
@@ -133,9 +133,9 @@ private transport details.
|
||||
|
||||
**Priority:** P2. Confirmed mismatch with the diagnostic error contract.
|
||||
|
||||
**Code:** [runInference](../../internal/usecase/smoke_tester.go:207),
|
||||
[generation preparation](../../internal/generation/coordinator.go:96), and
|
||||
[request payload validation](../../internal/llm/openai_compatible_client.go:511).
|
||||
**Code:** [runInference](../../../internal/usecase/smoke_tester.go:207),
|
||||
[generation preparation](../../../internal/generation/coordinator.go:96), and
|
||||
[request payload validation](../../../internal/llm/openai_compatible_client.go:511).
|
||||
|
||||
**Failure condition:** An inference diagnostic's effective settings are invalid
|
||||
in a way detected by built-in request construction rather than target selection.
|
||||
@@ -144,7 +144,7 @@ is one public example. The coordinator returns `llm.ErrInvalidRequest` before
|
||||
dispatch with no attempt evidence. `runInference` treats every generation error
|
||||
as a concluded diagnostic failure.
|
||||
|
||||
**Expected:** The [SmokeTest contract](../../smoke_diagnostics.go:238) says invalid
|
||||
**Expected:** The [SmokeTest contract](../../../smoke_diagnostics.go:238) says invalid
|
||||
kinds, settings, and selections return an ordinary Go error before diagnostic
|
||||
work. A locally invalid request should follow that path and expose the public
|
||||
invalid-request category. This is distinct from a provider rejecting a validly
|
||||
570
docs/roadmap/audit/catalog.md
Normal file
570
docs/roadmap/audit/catalog.md
Normal file
@@ -0,0 +1,570 @@
|
||||
# Consolidated Pre-1.0 Audit Catalog
|
||||
|
||||
Consolidated on 2026-11-04 from the 11 audit documents in this directory.
|
||||
Their 41 numbered findings are represented by 34 entries below. Entries that
|
||||
describe the same underlying issue share one remediation scope; missing
|
||||
regression coverage for an identified defect is included with that defect.
|
||||
Some broad testing findings consequently support more than one entry.
|
||||
|
||||
The order reflects consequence, exposure, and the value of clarifying public
|
||||
contracts before v1.0. Confirmed defects and conditional security exposure come
|
||||
first, followed by contract/test gaps, resource efficiency, and maintainability.
|
||||
This ordering does not upgrade the original severities or turn every cleanup
|
||||
into a release gate. The source audits retain their evidence, qualifications,
|
||||
and validation results. None of these entries is marked remediated by this
|
||||
documentation-only consolidation.
|
||||
|
||||
## 1. Update The Audited Maintainer Toolchain For Applicable Security Fixes
|
||||
|
||||
**Conditional security exposure.** The audited Go 1.27.0 toolchain predates
|
||||
networking fixes applicable to HTTP clients, including HTTP/2 CPU exhaustion
|
||||
and conditional certificate-trust exposure. Use a patched maintainer toolchain
|
||||
and repeat the focused security assessment against it. Actual downstream
|
||||
exposure depends on the toolchain that builds the consumer and its transport
|
||||
and trust configuration; raising PromptKit's minimum Go requirement is a
|
||||
separate compatibility decision, not an implied remedy.
|
||||
|
||||
Sources: [Security 1](07-security.md#1-the-audited-go-toolchain-predates-applicable-networking-security-fixes).
|
||||
|
||||
## 2. Bound Authentication-Probe Response Consumption
|
||||
|
||||
**Confirmed resource/security defect.** Authentication diagnostics read a
|
||||
successful upstream body with unbounded `io.ReadAll`. A deadline does not bound
|
||||
the bytes a fast server can supply. Introduce finite response consumption and
|
||||
clean closure, with an explicit choice of byte budget and overflow reporting.
|
||||
Overflow should not be interpreted as proof of rejected authentication.
|
||||
Relocating the provider adapter in entry 26 would not itself fix this defect.
|
||||
|
||||
Sources: [Resources 1](05-resources.md#1-authentication-diagnostics-read-an-unbounded-upstream-body),
|
||||
[Security 2](07-security.md#2-authentication-probes-permit-unbounded-remote-body-memory-consumption).
|
||||
|
||||
## 3. Avoid Exponential Expansion During Convenience Source Conversion
|
||||
|
||||
**Confirmed resource defect.** `ProfileSource` and `BackendSource` conversion
|
||||
expand repeated references before ConfigKit performs its bounded snapshot.
|
||||
A small shared acyclic graph can therefore cause exponential work and
|
||||
allocation. Preserve shared references during conversion, with suitable cycle
|
||||
and slice-view handling, rather than duplicating ConfigKit's validation or
|
||||
introducing PromptKit-side raw-source limits. Add focused shared-graph and
|
||||
deferred-failure regressions for both convenience helpers.
|
||||
|
||||
Sources: [Resources 2](05-resources.md#2-convenience-source-conversion-expands-shared-graphs-before-bounded-loading),
|
||||
[Testing 3](08-testing.md#3-programmatic-provider-value-conversion-lacks-representative-value-cases).
|
||||
|
||||
## 4. Resolve The ArtifactReader Buffer-Ownership Contract
|
||||
|
||||
**Confirmed race under the documented collaborator contract.** The public
|
||||
reader contract permits buffer reuse after `Read` returns, before PromptKit
|
||||
can finish copying it. Concurrent preparation with a compliant custom reader
|
||||
can race. Clarify ownership/lifetime per invocation and align adapters, GoDoc,
|
||||
and concurrency tests. This needs a deliberate public-contract decision before
|
||||
v1.0: an immediate copy alone cannot guarantee safety if the reader is allowed
|
||||
to mutate the buffer immediately. No default-reader defect was established.
|
||||
|
||||
Sources: [Interfaces 1](09-interfaces.md#1-artifactreader-permits-buffer-reuse-before-promptkit-can-copy-it).
|
||||
|
||||
## 5. Preserve Exact Numbers In Programmatic Provider Sources
|
||||
|
||||
**Confirmed data-integrity defect.** Convenience source conversion falls back
|
||||
from integer parsing to `float64` without checking precision, silently changing
|
||||
some accepted `json.Number` values. Preserve an exact ConfigKit-compatible
|
||||
representation, or defer an unsupported-value error to ConfigKit instead of
|
||||
rounding or converting the number into a string. Regression coverage should
|
||||
include exact-number boundaries, nested typed collections, mutation isolation,
|
||||
and deferred unsupported/cyclic-value errors for both source helpers.
|
||||
|
||||
Sources: [Correctness 1](01-correctness.md#1-provider-source-helpers-silently-change-exact-json-numbers),
|
||||
[Testing 3](08-testing.md#3-programmatic-provider-value-conversion-lacks-representative-value-cases).
|
||||
|
||||
## 6. Make Provider-Native Schema Constraints Self-Contained
|
||||
|
||||
**Confirmed implementation gap; packaging policy needs selection.** Local
|
||||
validation freezes referenced schema resources, but the provider receives only
|
||||
the root document and cannot resolve its local external references. Select
|
||||
correct resource packaging, or an explicit local-only fallback policy, without
|
||||
naively inlining recursive schemas or silently dropping native constraints.
|
||||
Add a wire-level regression demonstrating a usable provider constraint, not
|
||||
merely successful local compilation. Entry 21 is a separate compilation
|
||||
refactor and does not solve this packaging problem.
|
||||
|
||||
Sources: [Correctness 2](01-correctness.md#2-external-schema-resources-do-not-reach-the-provider-constraint),
|
||||
[Testing 5](08-testing.md#5-multi-resource-schema-tests-do-not-establish-a-usable-provider-constraint).
|
||||
|
||||
## 7. Preserve The Owning Operation's Public Error Category
|
||||
|
||||
**Confirmed classification defect.** Public error conversion searches nested
|
||||
causes in a global priority order, allowing a collaborator's sentinel to hide
|
||||
the enclosing operation's category. Map the owning boundary independently
|
||||
while retaining the original cause for `errors.Is`/`errors.As`. Add composed
|
||||
cause regressions; simply reordering global checks can break legitimate direct
|
||||
not-found versus load-failure distinctions.
|
||||
|
||||
Sources: [Error handling 1](06-error-handling.md#1-nested-collaborator-errors-can-hide-the-owning-operations-public-category),
|
||||
[Testing 4](08-testing.md#4-error-tests-miss-composed-causes-and-pre-dispatch-diagnostic-failures).
|
||||
|
||||
## 8. Distinguish Response Stream Failures From Malformed JSON
|
||||
|
||||
**Confirmed error-handling defect.** Non-context I/O failures while decoding a
|
||||
successful HTTP response or checking its end are reported as malformed JSON,
|
||||
losing the transport cause; complete JSON followed by a read failure can also
|
||||
be misreported as trailing data. Retain and classify reader failures separately
|
||||
using the existing redacted transport boundary. Preserve cancellation
|
||||
precedence and genuine malformed/truncated/oversized-content handling. Cover
|
||||
both read locations without adding transport retries.
|
||||
|
||||
Sources: [Error handling 2](06-error-handling.md#2-successful-status-body-read-failures-are-misreported-as-malformed-json),
|
||||
[Testing 4](08-testing.md#4-error-tests-miss-composed-causes-and-pre-dispatch-diagnostic-failures).
|
||||
|
||||
## 9. Return Invalid Diagnostic Request Construction As A Go Error
|
||||
|
||||
**Confirmed diagnostic-contract defect.** Some invalid inference settings fail
|
||||
before dispatch yet produce a completed report with a nil Go error. Propagate
|
||||
local request-construction failures through `ErrInvalidRequest` and add a
|
||||
no-dispatch regression. Preserve intentionally completed diagnostic outcomes
|
||||
for missing credentials, capacity outcomes, and unsupported or inapplicable
|
||||
operations; not every no-dispatch outcome is invalid configuration.
|
||||
|
||||
Sources: [Error handling 3](06-error-handling.md#3-inference-diagnostics-swallow-some-invalid-request-construction-errors),
|
||||
[Testing 4](08-testing.md#4-error-tests-miss-composed-causes-and-pre-dispatch-diagnostic-failures).
|
||||
|
||||
## 10. Remove Quadratic Requirement-Evidence Union Work
|
||||
|
||||
**Confirmed algorithmic inefficiency.** A left-heavy recognized condition can
|
||||
repeatedly copy growing missing-key evidence sets, producing quadratic work.
|
||||
Use operation-local accumulation or ownership-aware merging while preserving
|
||||
classification and conservative inspection coverage. Do not address this by
|
||||
adding expression limits or expanding the recognized template language. Entry
|
||||
23's traversal cleanup is related but is not a substitute for fixing this
|
||||
specific accumulation algorithm.
|
||||
|
||||
Sources: [Efficiency 1](02-efficiency.md#1-condition-evidence-unions-can-require-quadratic-work).
|
||||
|
||||
## 11. Exercise RunRequest's Privacy Formatting Boundary
|
||||
|
||||
**Missing protection, not a demonstrated leak.** Request formatting methods
|
||||
have no execution coverage despite owning a privacy-sensitive public boundary.
|
||||
Add focused tests with distinct synthetic private markers for the relevant
|
||||
value/pointer formatting paths and JSON omission behavior. Assert privacy and
|
||||
the documented boundary, not an incidental complete diagnostic or JSON shape.
|
||||
|
||||
Sources: [Testing 1](08-testing.md#1-runrequests-privacy-formatting-contract-is-not-exercised).
|
||||
|
||||
## 12. Correct Profile Inspection Credential Documentation
|
||||
|
||||
**Public-contract documentation defect.** Inspection GoDoc incorrectly makes
|
||||
required credentials and an environment variable mutually exclusive. Actual
|
||||
behavior permits an explicit profile environment variable while suppressing
|
||||
only the backend-derived default when required credentials are selected.
|
||||
Correct the field/method GoDoc and consumer example, keeping the established
|
||||
configuration behavior and DTO shape.
|
||||
|
||||
Sources: [Interfaces 2](09-interfaces.md#2-profile-inspection-misstates-required-credential-semantics).
|
||||
|
||||
## 13. Correct The Configured Provider-Map Wire Documentation
|
||||
|
||||
**Integration-contract documentation defect.** The wire document says backend
|
||||
and profile provider maps replace one another, but configuration recursively
|
||||
overlays them; runtime replacement is a separate rule. Align the integration
|
||||
summary with the canonical format reference rather than maintaining a second
|
||||
conflicting precedence definition. Entry 19 concerns the efficiency of the
|
||||
existing overlay, not a change to its semantics.
|
||||
|
||||
Sources: [Interfaces 3](09-interfaces.md#3-the-wire-contract-incorrectly-says-configured-provider-maps-replace).
|
||||
|
||||
## 14. Remove Stale Identity And Version GoDoc
|
||||
|
||||
**Public-contract documentation defect.** Inspection comments still describe
|
||||
trimmed profile IDs, and listing comments describe a removed prompt-version
|
||||
field. Update them to the implemented literal identity and current collection
|
||||
version model. Do not normalize meaningful whitespace or reintroduce a field
|
||||
to match obsolete comments.
|
||||
|
||||
Sources: [Interfaces 4](09-interfaces.md#4-identity-godoc-still-describes-removed-normalization-and-version-fields).
|
||||
|
||||
## 15. Cover The Public Single-Schema-File Option
|
||||
|
||||
**Public wiring coverage gap.** Direct filesystem-validator tests do not
|
||||
exercise `WithSchemaFile` through engine construction. Add offline temporary
|
||||
file cases through public preparation/validation, including option precedence
|
||||
and relevant file errors. This is a supported feature, not dead code; entry
|
||||
32 removes only an unused private helper result.
|
||||
|
||||
Sources: [Testing 2](08-testing.md#2-the-public-single-schema-file-option-has-no-execution-coverage).
|
||||
|
||||
## 16. Detach Inert Prepared Handles From The Execution Owner
|
||||
|
||||
**Avoidable retention.** Used or discarded prepared handles retain their
|
||||
runner, and therefore the engine's captured configuration and collaborators.
|
||||
Use a narrower ownership identity or clear the unnecessary owner reference
|
||||
safely. Preserve foreign-owner rejection, one-time claim, namespace-view
|
||||
behavior, and available details. This is unnecessary retention when a handle
|
||||
outlives its engine, not evidence of an ever-growing global leak.
|
||||
|
||||
Sources: [Resources 3](05-resources.md#3-inert-prepared-handles-retain-their-entire-execution-owner).
|
||||
|
||||
## 17. Bound Repair Feedback's Temporary Serialization
|
||||
|
||||
**Avoidable peak allocation.** Feedback formatting serializes potentially
|
||||
large diagnostic text before truncating the final message, so its existing
|
||||
output bound does not bound scratch memory. Produce bounded encoded feedback
|
||||
incrementally, preserving the existing output semantics and complete stored
|
||||
validation diagnostics and previous candidate. A new raw-diagnostic input
|
||||
limit is not implied.
|
||||
|
||||
Sources: [Resources 4](05-resources.md#4-repair-feedbacks-output-bound-does-not-bound-its-temporary-serialization).
|
||||
|
||||
## 18. Detach Normalized Session IDs From Large Input Strings
|
||||
|
||||
**Avoidable retention.** A short trimmed session ID can retain the allocation
|
||||
of a much larger padded caller string. Clone the accepted normalized value at
|
||||
its ownership boundary. Preserve trimming, accepted content, code-point
|
||||
validation, and resulting wire/hash content rather than adding a raw-input
|
||||
size restriction.
|
||||
|
||||
Sources: [Resources 5](05-resources.md#5-short-normalized-session-ids-can-pin-large-input-strings).
|
||||
|
||||
## 19. Avoid Repeated Descendant Cloning During Provider Overlay
|
||||
|
||||
**Construction-time inefficiency.** Recursive overlays repeatedly clone and
|
||||
sort overlapping descendant maps, giving work proportional to tree size times
|
||||
depth. Existing depth bounds mean this is not unbounded quadratic growth.
|
||||
Clone ownership once or use a single traversal while retaining recursive-map,
|
||||
null, and array semantics. Coordinate with entry 24 without assuming all JSON
|
||||
walkers have interchangeable policies.
|
||||
|
||||
Sources: [Efficiency 2](02-efficiency.md#2-recursive-provider-overlay-repeatedly-clones-descendant-subtrees).
|
||||
|
||||
## 20. Index Filtered Listings By Namespace
|
||||
|
||||
**Repeated inspection overhead.** Filtered listing currently copies and scans
|
||||
the complete inventory even when few or no entries match. Capture namespace
|
||||
buckets or equivalent ranges so queries operate on the relevant subset.
|
||||
Preserve ordering, exact identity matching, cancellation, and caller-owned
|
||||
results. A pagination API or runtime cache is not needed for this improvement.
|
||||
|
||||
Sources: [Efficiency 3](02-efficiency.md#3-filtered-listings-copy-and-scan-the-complete-inventory).
|
||||
|
||||
## 21. Share Schema Compilation And Declare The Runner's Real Dependency
|
||||
|
||||
**Maintainability and interface clarity.** Four host-file/`fs.FS`,
|
||||
direct/prepared paths repeat the schema compilation lifecycle. The runner also
|
||||
declares a validator interface while requiring preparation through a runtime
|
||||
assertion. Share the lifecycle behind source-specific adapters and make that
|
||||
dependency explicit. Preserve malformed-output short-circuiting before schema
|
||||
I/O, source containment, exact numbers, cancellation, and frozen plans. Retain
|
||||
independent cross-source tests; do not force whole entry points to be identical.
|
||||
|
||||
Sources: [Complexity 1](03-complexity.md#1-consolidate-schema-compilation-and-express-the-runners-actual-interface),
|
||||
[Equivalence 1](10-equivalence.md#1-share-the-schema-compilation-lifecycle-across-source-and-execution-paths).
|
||||
|
||||
## 22. Decompose Configuration Validation And Materialization
|
||||
|
||||
**Maintainability.** Large configuration operations mix record-specific
|
||||
validation, resource capture, and effective-profile construction. Extract
|
||||
focused helpers with visible inputs and ownership, rather than another generic
|
||||
validation framework. Preserve final-handoff validation, presence/null rules,
|
||||
origins, and meaningful fail-fast/error-precedence behavior. Do not turn
|
||||
incidental ordering of independent validation failures into a new guarantee. The configuration-boundary ADR remains
|
||||
authoritative; this is not a proposal to inspect raw ConfigKit sources.
|
||||
|
||||
Sources: [Complexity 2](03-complexity.md#2-decompose-configuration-validation-and-resource-materialization).
|
||||
|
||||
## 23. Clarify Requirement Traversal And Share Branch Orchestration
|
||||
|
||||
**Maintainability.** Separate recognition/evaluation from reachability and
|
||||
evidence bookkeeping, particularly the repeated `if`/`with` branch protocol.
|
||||
Keep their scope differences explicit and retain independent comparisons with
|
||||
the actual Go renderer. Preserve conservative partial coverage and inactive
|
||||
computed-name behavior. This does not justify a general template interpreter
|
||||
or stronger execution-readiness promises from requirements inspection.
|
||||
|
||||
Sources: [Complexity 3](03-complexity.md#3-untangle-requirement-traversal-expression-evaluation-and-evidence),
|
||||
[Equivalence 3](10-equivalence.md#3-share-ifwith-requirement-branch-orchestration-not-template-execution).
|
||||
|
||||
## 24. Centralize Compatible JSON Copying Without Conflating Policies
|
||||
|
||||
**Maintainability and ownership clarity.** The facade maintains a separate
|
||||
reflection traversal for already-normalized values despite an existing
|
||||
`internal/jsonvalue` owner. Consolidate genuinely compatible structural
|
||||
copying while retaining DTO projection at the facade and distinct source
|
||||
encoding/runtime validation policies. Preserve concrete types, nils, exact
|
||||
numbers, schema keys, and ownership. Infallible projection must not acquire
|
||||
new errors, panics, or silent zero-value fallbacks. Entries 3 and 5 remain
|
||||
separate source-conversion defects, not reasons to reject pre-handoff sources.
|
||||
|
||||
Sources: [Complexity 4](03-complexity.md#4-clarify-ownership-and-policies-across-the-json-value-walkers),
|
||||
[Equivalence 2](10-equivalence.md#2-centralize-structural-copying-of-already-normalized-json-values).
|
||||
|
||||
## 25. Separate Attempt Accounting From Generation Control Flow
|
||||
|
||||
**Maintainability.** Generation coordination interleaves dispatch/retry policy
|
||||
with parallel return values and repeated evidence/observation projection.
|
||||
Introduce a small attempt result and focused accounting helpers. Preserve
|
||||
built-in versus injected-client differences, permit release before callbacks
|
||||
or backoff, frozen requests, existing retry classification, private evidence,
|
||||
and known/unknown usage distinctions. Do not collapse legitimately different
|
||||
dispatch strategies merely to reduce lines.
|
||||
|
||||
Sources: [Complexity 5](03-complexity.md#5-extract-attempt-accounting-from-generation-control-flow).
|
||||
|
||||
## 26. Move OpenRouter Authentication Protocol Details Into Its Adapter
|
||||
|
||||
**Architectural ownership.** Use-case code currently interprets OpenRouter's
|
||||
account-response fields and provider-relative endpoint while the outbound
|
||||
adapter owns HTTP execution. Place protocol construction/recognition with the
|
||||
outbound integration; keep diagnostic selection, applicability, cancellation,
|
||||
completion, and neutral reporting in the use case. Preserve credential/body
|
||||
privacy and anonymous probing. No plugin framework, reverse dependency, or
|
||||
blanket prohibition on HTTP facts is needed. Coordinate the body-bound fix
|
||||
in entry 2, which is a distinct behavioral change.
|
||||
|
||||
Sources: [Boundaries 2](11-boundaries.md#2-put-openrouter-authentication-protocol-details-behind-the-outbound-adapter).
|
||||
|
||||
## 27. Narrow Smoke-Value Copy Helpers To Real Consumers
|
||||
|
||||
**Ownership cleanup.** A request cloner has only its self-test as a caller;
|
||||
the full-report cloner is used to detach only a target. Express that actual
|
||||
copy operation directly and retire unused broader cloning after checking
|
||||
callers. Keep facade request/report projection, active backend cloning,
|
||||
provider-map errors, ownership tests, and redaction. Do not introduce shared
|
||||
public/internal DTOs or an additional full-report copy.
|
||||
|
||||
Sources: [Boundaries 3](11-boundaries.md#3-narrow-smoke-value-copy-helpers-to-their-actual-ownership-boundaries).
|
||||
|
||||
## 28. Give Bundled Catalog Expectations One Test Owner
|
||||
|
||||
**Test duplication.** Complete bundled profile/model/provider expectations are
|
||||
repeated across several test layers. Keep one independently specified asset
|
||||
oracle, with smaller tests for distinct loading, wire, and diagnostic behavior.
|
||||
Use controlled fixtures where bundled churn is irrelevant and structural
|
||||
assertions rather than formatted-map strings. Retain meaningful bundled
|
||||
coverage instead of deriving every expected value from the loader under test.
|
||||
|
||||
Sources: [Testing 7](08-testing.md#7-bundled-catalog-expectations-have-multiple-test-owners).
|
||||
|
||||
## 29. Stop Pinning The Opaque Artifact Hash Algorithm In Tests
|
||||
|
||||
**Over-specified tests.** Artifact tests and a benchmark assert a literal hash
|
||||
algorithm despite the public value being opaque. Assert the documented
|
||||
equality/ownership relationships and actual byte/size behavior instead; keep
|
||||
benchmark setup outside its timed work. Coordinate with entry 31, but do not
|
||||
turn today's hash representation into a new compatibility guarantee.
|
||||
|
||||
Sources: [Testing 6](08-testing.md#6-artifact-tests-and-benchmarks-pin-an-explicitly-opaque-hash-algorithm).
|
||||
|
||||
## 30. Remove Unconsumed Backend Presence Bookkeeping
|
||||
|
||||
**Low-priority state cleanup.** Backend field-state maps and
|
||||
`QueueCapacitySet` have no runtime reader; effective values and ConfigKit
|
||||
already own queue defaulting and provenance. Remove only these unused records
|
||||
and adjust tests to assert effective behavior or operator inspection. Keep
|
||||
profile/input presence tracking, explicit zero capacity, and derived defaults.
|
||||
|
||||
Sources: [Dead code 1](12-deadcode.md#1-remove-backend-presence-bookkeeping-with-no-runtime-consumer).
|
||||
|
||||
## 31. Retire Duplicate Fields In Captured Prompt Definitions
|
||||
|
||||
**Low-priority representation cleanup.** Captured definitions duplicate the
|
||||
output format and carry an unused runtime message-file field after content
|
||||
materialization. Use the captured output contract and text as their respective
|
||||
sources of truth. Preserve real YAML `content_file` loading, origins, and the
|
||||
request-override default rule. Removing serialized internal fields changes
|
||||
current definition hashes: retain documented opaque equality relationships,
|
||||
not an invented bit-identical encoding promise.
|
||||
|
||||
Sources: [Dead code 2](12-deadcode.md#2-retire-legacy-fields-in-the-captured-prompt-representation).
|
||||
|
||||
## 32. Remove Unused Private Diagnostic State And Helper Results
|
||||
|
||||
**Low-priority cleanup.** A private reachability outcome echoes an endpoint
|
||||
that report assembly never reads; the schema-file helper returns a directory
|
||||
that its only caller discards. Remove the unused state/result while preserving
|
||||
actual sanitized request/report evidence, directory calculation, file checks,
|
||||
and public single-file schema support. Keep privacy tests against the values
|
||||
that real execution consumes.
|
||||
|
||||
Sources: [Dead code 3](12-deadcode.md#3-remove-unused-diagnostic-state-and-a-discarded-helper-result).
|
||||
|
||||
## 33. Remove Orphan Internal Helpers And Their Self-Tests
|
||||
|
||||
**Low-priority cleanup.** `filecatalog.RelativePath` and
|
||||
`resourceNamespaceFromID` have only their own tests as callers, and the runner
|
||||
test's `stringPtr` helper is entirely unused. Remove those narrow surfaces
|
||||
without replacement, preserving real discovery, identity qualification, and
|
||||
namespace inventory/filtering coverage. Reachability from examples alone is
|
||||
not a reason to remove optional public APIs.
|
||||
|
||||
Sources: [Dead code 4](12-deadcode.md#4-remove-orphan-helpers-whose-only-callers-are-their-own-tests).
|
||||
|
||||
## 34. Decide Whether To Retain The Legacy Capacity Client Wrapper
|
||||
|
||||
**Decision-dependent architectural cleanup.** The old capacity client wrapper
|
||||
is used by tests, whereas production dispatch uses the generation coordinator.
|
||||
It is also explicitly retained in current internal documentation; the dead-code
|
||||
audit does not classify it as obsolete functionality. If that retention decision
|
||||
is revised, exercise the production coordinator in runner tests and move any
|
||||
needed one-shot capacity scaffolding into tests before retiring the wrapper.
|
||||
Preserve FIFO, cancellation, independent-pool, and panic-release coverage.
|
||||
This is not a second scheduling algorithm or a confirmed concurrency defect.
|
||||
|
||||
Sources: [Boundaries 1](11-boundaries.md#1-retire-the-test-only-capacity-client-as-a-production-scheduling-surface),
|
||||
[Dead-code retention qualification](12-deadcode.md#intentional-retention-and-related-findings).
|
||||
|
||||
## Open Decisions
|
||||
|
||||
Work through the decisions below in this order. These are unresolved choices,
|
||||
not additional findings or approved changes to public contracts. A decision
|
||||
blocks its dependent remediation, not unrelated fixes. In particular, updating
|
||||
the maintainer toolchain need not wait for the consumer-minimum decision.
|
||||
Implementation details and optional cleanup do not become release gates merely
|
||||
by appearing in this catalog.
|
||||
|
||||
### Decision 1: ArtifactReader Result Ownership And Lifetime
|
||||
|
||||
**Related finding:** [4](#4-resolve-the-artifactreader-buffer-ownership-contract).
|
||||
|
||||
**Question:** What ownership does a successful `Read` transfer, and when may
|
||||
an injected reader mutate or reuse the returned storage?
|
||||
|
||||
**Recommendation:** Transfer an independent result to PromptKit on successful
|
||||
return. The reader must stop mutating the returned artifact and its body,
|
||||
including through retained aliases. Fresh per-invocation storage is the simplest
|
||||
implementation; immutable shared data can remain shared if no collaborator
|
||||
mutates it. This removes the impossible immediate-reuse permission without
|
||||
adding synchronization or a release protocol to the interface. Explain the
|
||||
consumer impact as a deliberate pre-1.0 contract correction.
|
||||
|
||||
**Alternative:** Permit borrowing until the enclosing preparation/execution
|
||||
operation returns. This can support application-managed reuse, but a reader
|
||||
cannot generally observe that lifetime and concurrent calls need independent
|
||||
storage or caller coordination. If reuse immediately after `Read` returns is
|
||||
required, an explicit acknowledgement/release API is another viable design;
|
||||
it enables pooling at the cost of a larger interface and lifecycle change.
|
||||
|
||||
**Resolve before implementation:** Select the ownership/lifetime rule and update
|
||||
GoDoc, adapter assumptions, and the regression scenario together. Another
|
||||
unsynchronized copy is not a remedy for the current permission.
|
||||
|
||||
### Decision 2: Provider Submission Of Multi-Resource Schemas
|
||||
|
||||
**Related findings:** [6](#6-make-provider-native-schema-constraints-self-contained)
|
||||
and [21](#21-share-schema-compilation-and-declare-the-runners-real-dependency).
|
||||
|
||||
**Question:** How should native structured output represent a schema whose
|
||||
local validation depends on external resources?
|
||||
|
||||
**Recommendation:** First investigate packaging the operation's captured
|
||||
resource graph into a self-contained provider constraint. Check identifiers,
|
||||
relative references, anchors, recursion, and supported dialects before choosing
|
||||
an encoding. Prefer correct packaging when it fits a proportionate scope,
|
||||
while keeping resource completeness distinct from any provider's supported
|
||||
schema subset. A standalone offline validation of the actual emitted constraint
|
||||
should establish the selected packaging behavior.
|
||||
|
||||
**Alternative:** Offer an explicit local-only validation choice when native
|
||||
submission cannot represent the resource graph. This preserves useful local
|
||||
validation and avoids building a general schema transformation system, but
|
||||
forgoes native generation constraints for that selected mode and needs a clear
|
||||
consumer-facing selection mechanism. Silently dropping native constraints is
|
||||
not equivalent to an explicit choice.
|
||||
|
||||
**Resolve before implementation:** Establish packaging feasibility and select
|
||||
the submission/fallback policy, including behavior for unsupported cases.
|
||||
Coordinate the compilation refactor with that result; it does not itself fix
|
||||
provider packaging or establish universal provider acceptance.
|
||||
|
||||
### Decision 3: Authentication Response Budget And Overflow Outcome
|
||||
|
||||
**Related findings:** [2](#2-bound-authentication-probe-response-consumption)
|
||||
and [26](#26-move-openrouter-authentication-protocol-details-into-its-adapter).
|
||||
|
||||
**Question:** What finite response budget should the authentication adapter
|
||||
apply, and how should a probe report exceeding it?
|
||||
|
||||
**Recommendation:** Select a conservative fixed budget for this small protocol,
|
||||
check actual streamed bytes regardless of declared length, and close on overflow
|
||||
without unbounded draining. Report the diagnostic failure with acceptance
|
||||
unknown, retaining any independently observed HTTP facts. Agree the exact byte
|
||||
count and its representation in the existing diagnostic report before coding;
|
||||
the catalog does not select either implicitly.
|
||||
|
||||
**Alternative:** Expose a caller-configurable budget with a finite default.
|
||||
This accommodates unusual legitimate responses but expands configuration and
|
||||
validation surface. It is worthwhile if a concrete consumer requirement cannot
|
||||
be served by an appropriately chosen fixed budget; it is not necessary merely
|
||||
to address the demonstrated unbounded read.
|
||||
|
||||
**Resolve before implementation:** Choose the size, overflow classification,
|
||||
and whether configurability is warranted. Keep this fix independent of adapter
|
||||
relocation so architectural cleanup does not delay the resource remedy.
|
||||
|
||||
### Decision 4: Exact Numbers And The Source-Conversion Boundary
|
||||
|
||||
**Related findings:** [3](#3-avoid-exponential-expansion-during-convenience-source-conversion),
|
||||
[5](#5-preserve-exact-numbers-in-programmatic-provider-sources), and
|
||||
[24](#24-centralize-compatible-json-copying-without-conflating-policies).
|
||||
|
||||
**Question:** Which exact numeric values can the convenience helpers preserve
|
||||
through the current ConfigKit mapping boundary, and what happens to others?
|
||||
|
||||
**Recommendation:** Preserve supported values through the complete conversion,
|
||||
load, and outbound-encoding path; defer an unsupported-value error when exact
|
||||
preservation is unavailable. Compare numeric meaning rather than spelling or
|
||||
successful parsing alone. Address shared-graph expansion in the same adapter
|
||||
with completed-conversion memoization, distinct cycle tracking, and correct
|
||||
slice-view identity, preserving source capture and deferred failures.
|
||||
|
||||
**Alternative:** Request upstream support for additional exact numeric/value
|
||||
forms, then simplify or remove the relevant PromptKit conversion. This may
|
||||
provide a cleaner long-term boundary and broader preservation, but introduces
|
||||
an upstream dependency and should not postpone fixing silent rounding or
|
||||
exponential expansion. Runtime JSON copying remains a separate policy.
|
||||
|
||||
**Resolve before implementation:** Establish the supported numeric round-trip
|
||||
cases and whether broader upstream support is needed now. Do not substitute
|
||||
numeric-looking strings or introduce a second raw-source validation system.
|
||||
|
||||
### Decision 5: Consumer Go Minimum After The Toolchain Update
|
||||
|
||||
**Related finding:** [1](#1-update-the-audited-maintainer-toolchain-for-applicable-security-fixes).
|
||||
|
||||
**Question:** Should the security fixes also change PromptKit's declared minimum
|
||||
consumer Go version?
|
||||
|
||||
**Recommendation:** Update the maintainer toolchain and repeat the focused
|
||||
assessment promptly; initially keep the consumer minimum decision separate.
|
||||
Advise affected consumers to rebuild with a patched toolchain. This recognizes
|
||||
that PromptKit publishes source and that downstream build/transport choices
|
||||
control the deployed exposure.
|
||||
|
||||
**Alternative:** Raise the declared minimum to the selected patched release.
|
||||
This makes the supported floor clearer and steers module consumers toward the
|
||||
fixed toolchain, but changes compatibility and does not repair already-built
|
||||
applications. Assess it under the existing release policy rather than treating
|
||||
it as an automatic consequence of the scanner output.
|
||||
|
||||
**Resolve before changing the minimum:** Confirm current advisory applicability,
|
||||
select the patched version, and decide whether changing the supported consumer
|
||||
floor is justified. No new recurring release gate is implied.
|
||||
|
||||
### Decision 6: Retention Of The Legacy Capacity Client Wrapper
|
||||
|
||||
**Related finding:** [34](#34-decide-whether-to-retain-the-legacy-capacity-client-wrapper).
|
||||
|
||||
**Question:** Does the wrapper still justify a production surface for its
|
||||
currently documented test-focused purpose?
|
||||
|
||||
**Recommendation:** Retire the production wrapper after moving the runner's
|
||||
integration test to the production generation coordinator. Keep direct manager
|
||||
coverage and any useful one-shot scaffolding in tests. This reduces maintenance
|
||||
and makes assembled scheduling tests reflect production while preserving the
|
||||
actual FIFO, cancellation, independent-pool, and panic-release scenarios.
|
||||
|
||||
**Alternative:** Retain it if a concrete internal use justifies the small
|
||||
adapter. This avoids test migration and preserves that convenient composition,
|
||||
but keeps a second assembly surface to understand and maintain. Existing
|
||||
self-tests alone do not establish a future production need.
|
||||
|
||||
**Resolve before removal:** Confirm the intended retention decision and test
|
||||
ownership. This cleanup can be deferred without blocking the behavioral fixes;
|
||||
it is not evidence of a second scheduler or a concurrency defect.
|
||||
Reference in New Issue
Block a user