promptkit/docs/roadmap/planning-assessment.md

Audit Remediation Planning Assessment

Purpose And Authority

Work through the unresolved design and evidence-dependent choices in the audit remediation roadmap before writing its staged implementation plan. Each numbered topic below is an assessment unit: gather the stated evidence, compare viable approaches, and record a recommendation before proceeding to the next topic.

The catalog's six accepted decisions remain authoritative. This assessment selects ways to implement those decisions; it does not reopen them or repeat the repository-wide audits. Catalog numbers below identify the relevant findings, not additional implementation stages. Assessment started on 2026-11-05 at clean commit 21fea46fb3b0ce9483db2399c9587f4a2323d687. Assessment records below distinguish completed analysis from still-pending implementation.

Assessment Method And Outputs

Read the contributor guide and applicable policies before each topic. Reuse existing audit evidence, source inspection, and successful checks where sufficient. Prefer graph discovery for callers and dependencies, then inspect the relevant code, contracts, and tests. Follow the configuration-boundary ADR.

This is analysis, not remediation. Do not change production code, dependencies, installed toolchains, public contracts, or repository tests during the assessment. No prototype or alternative implementation is needed. Use existing offline benchmarks and tools where helpful; temporary measurement harnesses outside the repository are appropriate only when needed to answer a specific question. Record their setup sufficiently to interpret or repeat the results. Do not use live inference, credentials, or private production data.

Append an assessment record beneath each completed topic containing:

• Evidence: repository commit, concrete declarations/callers/tests, relevant dependency version, and measurements or commands actually used.
• Alternatives: plausible approaches and their semantic, maintenance, and resource tradeoffs. Do not manufacture alternatives to an obvious correction.
• Recommendation: the selected design or a justified proposed deferral, affected boundaries, and reasoning. Distinguish observed results from inference.
• Plan handoff: consequential signatures, fields, algorithms, or ownership transitions to specify; dependencies; migration/documentation impact; and implementation-time validation needed.
• Status: assessed, or a specific remaining uncertainty and the next bounded investigation needed. Assessment completion does not mean remediation complete.

For measurements, record the selected Go version, workload size/shape, and the relevant time, allocations, bytes, or retention evidence. Use small ordinary cases and larger valid cases to distinguish fixed overhead from scaling. Repeat noisy measurements only as needed; do not introduce benchmark thresholds, a generalized benchmark project, or a recurring release gate. Code-based work counts can establish avoidable traversal without timing every helper. Do not present estimated improvements as measured results from unimplemented code.

Recommendations should settle consequential implementation choices, not private helper names or equivalent coding details. Proceed between topics without new human-review or publication gates. If evidence requires changing an accepted decision or broadening scope, identify that question for the maintainer rather than silently selecting a new contract.

1. Establish The Evidence And Toolchain Baseline

Catalog: 1; applies to subsequent measurements.

Collect the current commit and worktree state, dependency versions, available Go installations, and the effective toolchain used by existing development and release commands. Compare the selected local toolchain against the specific networking fixes identified in the security audit, using authoritative upstream information where needed. Distinguish library/client applicability from server or platform-only findings and uncertainty about downstream builds.

Recommend the toolchain to use and any environment update needed before remediation validation. Record an unavailable patched toolchain as environment work to schedule, not permission to install one during assessment or to stop independent source analysis. Retain the accepted separation between maintainer selection and consumer minimum; do not add consumer advisories or scan gates.

Assessment Record

Status: Assessed; no environment installation is needed.

Evidence: Default /snap/bin/go reports Go 1.27.0, but the existing module cache contains executable Go 1.27.3 at /home/eric/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.27.3.linux-amd64/bin/go; invoking it reports go1.27.3 linux/amd64. Other inspected SDK/cache installs are older. go.mod declares 1.27.0, ConfigKit 1.1.0, jsonschema/v6 6.0.2, and ReleaseKit 0.9.0. scripts/releasekit.sh sets GOTOOLCHAIN=local, so its toolchain is selected by PATH, not automatically upgraded.

Recommendation and alternatives: Put the existing 1.27.3 binary directory first on PATH, with GOTOOLCHAIN=local and GOWORK=off, for subsequent assessment commands and implementation validation. The authoritative affected ranges for GO-2026-6611 and GO-2026-6809 exclude 1.27.3: the two specific audited exposures are addressed by that selection. This is a focused version/applicability reassessment, not a new full vulnerability scan or a claim about consumer binaries. Continuing with default 1.27.0 retains the identified exposure and is not recommended; installation is unnecessary.

Plan handoff: Clarify ordinary development toolchain selection in docs/development.md; leave go 1.27.0 unchanged. Record the actually selected patch when validating remediation. No consumer advisory or recurring scan gate.

2. Specify Direct ConfigKit Source Handoff

Catalog: 3, 5; informs 24.

Inspect the pinned ConfigKit Mapping contract and implementation at the snapshot/error boundary, plus both PromptKit convenience helpers and their callers. Establish which ordinary field projections remain, when values are captured, and how unsupported values, cycles, and shared graphs reach deferred diagnostics. Inventory existing fixtures using formerly converted types.

Recommend the exact removal boundary for the compatibility converter, supported handoff cases, and migration examples. Identify which current tests remain meaningful and which assert intentionally removed accommodation. Do not design a replacement converter, pre-validation pass, or upstream feature. Existing upstream ownership evidence is sufficient; exhaustive ConfigKit limit testing is not an assessment deliverable.

Assessment Record

Status: Assessed; remove the compatibility adapter without replacement.

Evidence: ProfileSource (profiles.go:29) and BackendSource (backends.go:82) already project optional typed fields into built-in values and wrap definitions in []any. Their only recursive compatibility dependency is providerMapForMapping(config.ExtraParams). The pinned ConfigKit 1.1.0 source.go:Mapping promises immediate capture and deferred unsupported/cyclic diagnostics. Its internal/engine/source.go:Mapping builds an owned document and retains mapIssue; readSource reports it when consumed. PromptKit's provider_values.go instead expands maps/slices before that bounded handoff and parses json.Number through a lossy float fallback. Existing sources_contract_test.go mainly protects flat-map ownership, not this broader input boundary.

Recommendation and alternatives: Assign config.ExtraParams directly into the ordinary fields map in both helpers, then call configkit.Mapping as now. Delete provider_values.go after its two callers are replaced. A shallow map copy adds no capture protection beyond Mapping; another recursive converter or raw validation would retain the responsibility being removed. Runtime overrides continue using their separate JSON-value boundary.

Plan handoff: Cover both helpers with nested built-in maps/[]any, mutation after source creation, and deferred failures for representative json.Number, named scalar/typed container, pointer, and cyclic inputs. Keep construction order/optional-field coverage. Explain caller conversion to supported forms: for example, []string becomes []any of strings; callers choose and check numeric conversion themselves. Do not assert preserved numeric spelling or duplicate ConfigKit's entire resource-limit suite. Source-helper GoDoc and migration notes own the changed accepted-value boundary.

3. Specify Self-Contained Schema Resolution And Compilation

Catalog: 6, 15, 21; coordinates with 32.

Inspect the pinned schema compiler's resource registration/loading facilities, the four direct/prepared and host-file/fs.FS paths, native-schema projection, and runner validation interfaces. Establish where root loading, reference resolution, compilation, cancellation, and malformed-output short-circuiting occur.

Use a compact semantic case inventory to distinguish references resolved inside the supplied document from references needing another document. Include local fragments, anchors, recursion, and supported identifiers/base-URI behavior; preserve existing dialect support rather than adding it. Do not equate an external-looking reference string with external resource loading without checking resolution semantics.

Recommend the shared compilation boundary, needed source adapters and internal interface signatures, and external-resource rejection/error path. Identify obsolete loaders/tests and retained containment, single-file, freezing, and direct-validation cases. Specify how the provider receives the self-contained document unchanged in meaning. No bundler, schema rewriting prototype, or live provider compatibility experiment is needed.

Assessment Record

Status: Assessed; use explicit root registration and a no-external-resource loader, with coverage for unused schema definitions.

Evidence: internal/validate/standard_validator.go has four compilation paths; its root loaders already preserve JSON numbers, validate the 2020-12 dialect, and check source containment/context. Runner.prepareValidation uses only ValidationPreparer, despite storing Validator. In pinned jsonschema 6.0.2, Compiler.AddResource registers the decoded document, roots.go collects embedded IDs/anchors, and loader.go consults registered resources and bundled metaschemas before the custom loader.

An offline dependency-contract probe in /tmp/promptkit-assessment-aqn3B8/schema.go, run with Go 1.27.3 using go run -mod=readonly and GOPROXY=off, registered one document and installed a loader that only counts/rejects external loads. Fragment, anchor, recursion, absolute self-ID, embedded child-ID, and standard metaschema cases compiled with zero external loads; a real external reference failed with one load. Crucially, an unused $defs entry with an external reference did not trigger a load until compiled explicitly. An example payload containing a literal $ref did not trigger loading. A denying loader alone therefore does not establish that every declared schema is self-contained.

Recommendation and alternatives: Retain source-specific root loading, then share compileSchemaDocument(ctx, resourceURL, document) returning the compiled executor. Register the root, retain Draft2020, and install a context-aware loader returning an actionable external-resource error without filesystem/network I/O. Keep compiler resolution authoritative for IDs and anchors. To cover otherwise unused declarations, enumerate schema locations using a small, explicit table of the pinned 2020-12 schema-bearing keyword positions (including its supported legacy positions) and compile those locations with the same compiler/cache. Do not traverse annotation/example objects as schemas. This bounded policy walk is not a URI resolver, bundler, or general schema framework. A lexical ban on non-fragment references rejects embedded IDs; root-only compilation misses the demonstrated unused-definition case. Neither is recommended.

Plan handoff: Specify escaped JSON-pointer location enumeration and retained embedded-ID/dynamic-reference semantics in focused cases; preserve dialect and number handling. Both direct schema callbacks call the shared compilation only after instance decoding, while prepared paths call it eagerly as before. Remove external loaders, not root path containment/read helpers. Change runner fields and constructor parameters to validate.ValidationPreparer, retaining the nil no-op and nil-plan error behavior. Validator constructors can return their concrete pointer types so existing direct Validate callers and preparation both compile; no public facade signature changes are needed. Replace external resource success fixtures with early-error cases, retain independent root-source tests, and add public WithSchemaFile option/replacement/error coverage.

4. Specify Ownership And Retention Remedies

Catalog: 4, 16, 18.

Trace public artifact adaptation and prepared-handle creation, claim, discard, details, and namespace-view operations. Inspect which references keep an engine alive and which synchronization owns state changes. Trace session normalization to its retained copies. Use reference/lifetime analysis first; use an existing allocation profile only if the retained graph is unclear.

Recommend the artifact contract/documentation changes under accepted ownership transfer, retaining the defensive copy initially. Specify the prepared-handle ownership identity and state transitions without relying on distinct zero-sized allocations comparing unequal. Identify the session-string detachment point. List relevant concurrent and mutation regressions for implementation, without building a new ownership framework or changing lifetimes during assessment.

Assessment Record

Status: Assessed; narrow references rather than adding lifecycle machinery.

Evidence: artifact_reader.go:publicArtifactReaderAdapter.Read copies the returned body only after the consumer call returns. PreparedExecution in internal/usecase/prepared_execution.go retains owner *Runner even after its mutex-protected claim/discard drops the payload; Details uses separately captured data. Runner owns repositories, clients, and admission. Namespace views delegate to the same internal engine/runner. NormalizeSessionID in internal/domain/session.go returns strings.TrimSpace(raw) directly.

Recommendation and alternatives: Document successful artifact return as ownership transfer, as accepted, and keep adapter copying. Replace the prepared handle's runner pointer with a pointer to a non-zero-sized private identity object allocated once by each runner constructor and copied into its handles. Keep that immutable token after claim/discard; it retains no runner graph and preserves foreign-owner validation order without a new owner-field race. Clearing a mutable runner pointer is viable but complicates synchronized checks and error ordering unnecessarily. Clone the validated nonempty normalized session string with strings.Clone before returning it; this detaches padded input without imposing a new limit.

Plan handoff: Retain ready-to-claimed/discarded transitions under the current mutex, private payload clearing, one-shot execution, and detached details. Protect cross-engine rejection, same-engine namespace views, concurrent claim/discard, and details after discard/use. Artifact regressions must use independently returned buffers and avoid claiming safety for contract-violating readers. Existing mutation, trimming, Unicode, hash, and wire assertions remain the semantic oracle; a GC timing assertion is unnecessary.

5. Specify Error Ownership And Preparation-Stage Information

Catalog: 7, 8, 9; informs 25.

Trace representative collaborator failures through internal wrapping and public translation, including direct lookup failures and nested public sentinels. Inspect body-reader errors at initial decoding and end verification, current redaction, context precedence, and retry classification. Trace diagnostic preparation versus dispatch, including legitimate completed no-dispatch cases.

Recommend how owning-operation information and preparation-stage failures are represented and propagated. Specify the distinction between stream I/O failure, malformed/truncated content, size overflow, and cancellation using existing boundaries where possible. Give a small expected-outcome matrix of public category, retained cause, diagnostic completion, and dispatch behavior. Retained causes may match multiple sentinels; category exclusivity is not the target.

Assessment Record

Status: Assessed; carry ownership and preparation provenance explicitly.

Evidence: errors.go:publicErrorFor uses whole-chain errors.Is in a fixed order; mapPublicError also searches nested capacity/exhaustion/provider types before normal mapping. wrapGenerationError and artifact loading add operation sentinels but no separately readable owning category. In internal/llm/openai_compatible_client.go:decodeOpenAIChatResponse, both decoder calls replace non-context errors with malformed-response messages. In generation.Coordinator.generate, PrepareGeneration fails before any capacity attempt, while SmokeTester.runInference concludes every generation error as report data. GenerationEvidence currently lacks preparation-stage information.

Recommendation and alternatives: Add a small internal use-case operation error carrying category and cause, with Unwrap/Is behavior. Apply it at owning boundaries and let facade mapping select that category independently of nested causes. Preserve legitimate direct not-found mappings when assigning the category. Keep specialized public capacity/provider/exhaustion projections, but do not let their presence inside a collaborator cause erase the outer owning category. A globally reordered sentinel switch cannot express this distinction.

Add PreparationFailed bool to internal GenerationEvidence, set only at the built-in PrepareGeneration failure return. Diagnostic orchestration checks it after caller cancellation and returns ErrInvalidRequest with the cause. Do not infer this stage from error identity or attempt count; injected failures and intentional missing-key/capacity reports keep their established semantics. This needs no new public field or retry policy.

Wrap the successful-response reader with a small read-error recorder beneath the existing limited reader. At decoding/EOF verification failure, prefer an observed context error; preserve size overflow as malformed; otherwise propagate an actual non-EOF reader failure through requestFailedError. Parser-generated syntax/type/truncation failures, EOF-truncated JSON, and actual extra values remain malformed. Distinguish errors returned by the reader from those created by encoding/json, rather than categorizing every io.ErrUnexpectedEOF alike. Retain redacted default formatting and exact underlying read causes.

Plan handoff: Protect these expected outcomes at the public boundary:

Case	Owning category / result	Retained evidence
Client returns child missing-prompt error	ErrLLMGenerate; error	Child cause remains matchable, including its category.
Reader returns the same child error	ErrArtifactLoad; error	Child cause remains matchable.
Direct missing prompt/profile	Existing not-found category; error	No conversion into a generic load failure.
2xx body read fails during decode or final EOF check	Generation plus transport; diagnostic transport failure	Original reader cause, no raw body/detail formatting.
Invalid/truncated JSON, extra value, size overflow	Existing malformed-response behavior	No new retry or parser-detail disclosure.
Built-in diagnostic preparation fails	ErrInvalidRequest; not a completed diagnostic	Zero dispatch and original preparation cause.
Missing required credential/capacity report	Existing completed diagnostic outcome	No inferred construction failure.
Caller cancellation	Existing cancellation/error handling	Cancellation takes precedence over publishing success.

6. Specify Authentication Overflow Reporting And Adapter Ownership

Catalog: 2, 26, 27; uses topic 5's error/stage conclusions.

Inspect public/domain diagnostic DTOs, outcome labels, body ownership, HTTP evidence, authentication recognition, and facade projection. Determine whether the selected overflow outcome fits existing fields or needs an explicit additive representation. Inventory actual smoke-copy consumers rather than assuming full reports need cloning.

Recommend concrete field/type or outcome changes and their public/internal mapping. State the facts reported for overflow: acceptance unknown, explicit limit evidence, and independently available HTTP status/timing. Explain the interaction with non-success statuses and cancellation without discarding established independently known evidence. Preserve neutral reporting and privacy.

Specify the bounded read/overflow detection operation around the accepted 1 MiB allowance, including exact-boundary, missing/inaccurate length, and close behavior. Recommend the outbound protocol-adapter seam and narrow target-copy operation. Keep the body-bound fix independently implementable before adapter relocation; do not add a public budget option or general diagnostic plugin framework.

Assessment Record

Status: Assessed; one reason constant suffices, without a new report field.

Evidence: Public/domain smoke reports already have Reason, Failure, nullable CredentialsAccepted, HTTP status on attempts, and timing. Current smoke_authentication.go reads a body only to recognize HTTP 200; 401 establishes rejection without reading it. Other statuses retain their existing outcome. smoke_tester.go copies these facts into attempts and finishHTTP handles caller cancellation. The outbound client already owns authenticated GET and HTTP evidence. The broad domain request/report cloners are used only by their self-tests and a target-only copy respectively, as recorded in audit 11.

Recommendation and alternatives: Add matching public/domain SmokeTestReasonAuthenticationResponseLimitExceeded constants with value authentication_response_limit_exceeded. On a 200 recognition overflow set that reason, existing ExecutionFailureOther, acceptance nil, and retain HTTP/timing facts; conclude with a nil Go error unless caller cancellation applies. A new general failure enum or report Boolean is unnecessary: the existing reason already provides an explicit machine-readable diagnostic. Keep 401 rejection and non-200 behavior independent of unused response bodies.

Read at most allowance plus one byte with io.LimitReader/io.ReadAll, detect length greater than 1 MiB before JSON recognition, and close without draining. Do not trust ContentLength. Existing bounded bytes plus one-byte lookahead are sufficient; streaming the small account parser adds complexity without need. Cancellation takes its existing precedence; overflow is not malformed content.

Move account-response recognition and bounded response ownership into an internal/llm OpenRouter authentication operation, returning HTTP evidence, optional acceptance, overflow/unrecognized indicators, and operational error. Move default endpoint derivation into a small llm adapter helper used by selection; explicit overrides remain unchanged. Use-case orchestration maps those factual results to report reasons/completion and consumes a narrow authentication capability alongside reachability HTTP capability. No reverse dependency or plugin registry is needed. Replace target-only full-report cloning with direct target/provider-map detachment; retain facade projection and active backend cloners.

Plan handoff: Fix and test the byte bound before relocation. Include exact limit versus limit-plus-one, escaping/invalid bodies, inaccurate declared length, closure, 401 without body consumption, context interruption, redaction, and detached report values. Update reason GoDoc/public conversion and focused diagnostic documentation in the same implementation unit. No secret-bearing body, header, or provider detail is added to report data.

7. Assess JSON Copying Compatibility And Provider Overlay Cost

Catalog: 19, 24; uses topic 2's converter-removal boundary.

Inventory callers of facade copying, runtime JSON validation/copying, prepared snapshots, and configuration overlay. Build a compact compatibility matrix for accepted concrete types, nil/indirection handling, exact numbers, empty keys, error behavior, bounds, and mutable ownership. Separate trusted-value copying from fallible input acceptance and source encoding.

Collect existing overlay benchmark/allocation evidence, supplemented only where needed with representative depth, breadth, overlapping/disjoint maps, and array replacement cases within current limits. Identify repeated visits/copies and their construction-time cost; respect the existing bounded-depth qualification.

Recommend which traversal can actually be removed, the shared operation's contract, and the overlay algorithm/ownership model. Prefer retaining a small distinct walker over a configurable framework when policies differ. Recommend implementing or explicitly deferring the optional overlay optimization based on expected benefit and complexity; identify estimates as such. Do not add errors, panics, or silent fallbacks to an infallible facade projection.

8. Assess Requirement Evidence Accumulation And Traversal

Catalog: 10, 23.

Trace ownership and merging of condition-evidence sets and the shared versus node-specific responsibilities of if/with. Inspect real-renderer comparison tests and scope/reachability cases. Collect operation counts or targeted baseline measurements across increasing left-heavy recognized conditions; include a small ordinary case so pathological scaling is not the only input.

Recommend an accumulation/merge algorithm and a narrow branch-orchestration boundary, explaining expected work without promising a new universal complexity contract. Keep the algorithmic fix distinct from traversal cleanup. Specify preserved dot/root scopes, inactive computed names, and conservative incomplete analysis. No interpreter, expanded grammar, or expression limit is proposed.

9. Assess Bounded Repair-Feedback Encoding

Catalog: 17.

Inspect feedback framing, JSON escaping, truncation, and the locations retaining full diagnostics and prior output. Measure current formatting with ordinary, large, and heavily escaped synthetic diagnostic strings, separating final output size from temporary allocations. Identify which complete serializations account for the avoidable memory cost.

Compare a bounded incremental approach with the current implementation using source-level algorithm and complexity reasoning; no alternative encoder needs to be built. Recommend an encoding/truncation approach and its preservation cases, or a justified proposed deferral if benefit does not warrant complexity. Any deferral must acknowledge the outstanding allocation issue. Do not cap raw diagnostics, discard stored errors, or assert an exact peak-heap guarantee.

10. Assess Namespace Listing Index Tradeoffs

Catalog: 20.

Inspect inventory capture, namespace identity, ordering, filtered retrieval, cancellation, and result detachment. Gather baseline time/allocation evidence for unfiltered listings and sparse, dense, and absent namespace matches at ordinary and larger valid inventory sizes. Consider both a single request and repeated namespace inspection, without assuming every consumer repeatedly lists entries.

Compare captured buckets/ranges with the existing scan. Account for construction work, retained index memory, and query output copying; do not optimize query time while ignoring its construction/retention tradeoff. Recommend a specific layout and query algorithm, or a justified proposed deferral. No pagination, runtime cache, or new ordering/cancellation guarantee is needed.

11. Select Remaining Refactoring, Test, And Cleanup Boundaries

Catalog: 11–14, 22, 25, 28–34; consolidate related conclusions above.

Reuse existing audit evidence and perform targeted caller checks rather than reauditing these findings. Identify the configuration helper boundaries, generation attempt-result fields, and accounting seam consistent with topics 2–8. Locate the real coordinator assembly to use in scheduling tests and the behavioral coverage that must survive wrapper retirement.

Select one independent bundled-catalog test oracle and the smaller distinct risks owned elsewhere. Identify privacy-formatting cases, public schema-file coverage from topic 3, and replacement assertions for opaque hashes. Map obsolete fields/helpers to actual callers and tests, including serialized definition-hash effects. Record the owning documentation for each correction/migration.

Recommend a concrete removal/extraction inventory, preservation tests, and ordering dependencies. Do not infer that self-tests justify production helpers, or that lack of example coverage makes supported APIs obsolete. This is not a new deletion sweep, exact helper-design exercise, or complexity-metric target.

12. Consolidate Recommendations For Implementation Planning

Review the assessment records in order and reconcile cross-topic choices. Produce a compact handoff covering all 34 catalog entries: selected remedy or explicit proposed deferral, supporting assessment topic, consequential design choice, dependencies, and implementation-time validation. Findings whose remedy was already settled need only a brief mapping, not another investigation.

Ensure source conversion disappears rather than migrating into shared copying; schema simplification and compilation agree; errors/stages precede accounting refactoring; the authentication fix can precede relocation; and actual coordinator tests precede wrapper removal. Separate optional cleanup from fixes so a deferral does not obstruct unrelated remediation.

Record any scope/contract question that still needs maintainer direction with a recommendation and viable alternative. A deferral is a recommendation, not silent authorization to drop accepted work. Unavailable measurements or tooling must be identified, together with whether they could change the recommendation; do not present a conditional choice as a settled implementation instruction.

Once consequential choices are settled, the subsequent implementation plan can define focused ordered stages, selected designs, migration/documentation work, and proportionate validation. Implementation correctness, race regressions, post-change measurements, and release checks remain work for implementation and release preparation. No claim of remediation completion follows from this assessment, and no new recurring release or human-review gate is introduced.