14 KiB
Implementation Review Remediation
PromptKit v0.12.0 Review
Reviewed on 2026-10-04 at
58cb02f9732b09740ff7e22096d2a4072ddfddca, covering the thirteen implementation
commits after b1bb55a. The runtime architecture matches the accepted
feature roadmap and implementation plan.
No runtime regression, missing framework capability, or newly imposed runtime
restriction was identified. Three low-priority documentation/help gaps were
identified; their status and resolutions are recorded below. The first retained
a loading guarantee that was incompatible with
the deliberately selected full-source engine construction.
R3: Correct The Definition-Inspection Loading Contract
Priority: P3 (low). Status: Complete.
Locations: docs/cli.md:208–224; related orientation in
docs/internal/adapters.md:49–56. The CLI wording was retained in b913ef0
despite the engine-wiring change in 0df2aad.
The profile-inspection reference still says it never loads a prompt. Both
definition-inspection commands now construct the same complete selected engine
as execution. PromptKit resolves and validates all final definitions and
captures their message content before the selected inspection runs. Consequently,
an unused prompt with invalid syntax or an unreadable content_file can prevent
inspection of an otherwise valid profile. Prompt inspection's unqualified
claim that it does not resolve a profile also needs to distinguish engine
assembly from request-specific target resolution.
Evidence: inspecting a valid profile in a selected combined file with an
unused prompt referring to a missing message file exits 1 before emitting a
profile report. The existing
TestProfileInspectionUsesCombinedSourcesAndValidatesUnusedPrompts also
protects this intentional eager-validation boundary.
Required change: explain that both commands construct and validate the full selected configuration, including unused definitions and message resources, before inspecting the chosen definition. Preserve the distinction that they do not prepare an execution request, load request-time schemas/artifacts, resolve credential values, reserve capacity, or call a provider. Clarify that no prompt collection is mandatory, but selected prompts are not ignored. Align the internal summary with the CLI contract rather than restating a stronger independence guarantee. Do not restore the empty-prompt-filesystem workaround or weaken eager validation to satisfy the stale wording.
Acceptance: the owning contract and internal summary agree with the roadmap, shared engine construction, and existing tests. Review the documentation diff and links; no production-code change or new runtime guarantee is needed.
Resolution: the CLI contract now distinguishes full-source engine construction from inspecting a selected definition, explains eager validation/content capture and unused-definition failures, and retains the request-time exclusions. It states that no prompt collection is mandatory but selected prompts are not ignored. The internal summary links to that contract without promising a separate no-prompt-loading path. Existing definition-inspection tests and documentation link/diff checks passed; no production code changed.
R4: Update The Composed-Example Description
Priority: P3 (low). Status: Open.
Location: docs/config.md:95–96, after the example migration in 2027b80.
The reference says examples/config.composed.yml imports two application
fragments. It now imports only config-fragments/sources.yml; the former
backend fragment became examples/framework/backends.yml, selected through
promptkit_sources rather than imported into the application document.
The example and code are correct, but the explanation confuses the two source
boundaries introduced by this migration.
Required change: describe the actual source-selection fragment and separately selected framework backend file, with links where useful. Do not reintroduce an application backend mapping or add an unnecessary fragment merely to match the old prose. The old backend fragment had no application defaults to preserve.
Acceptance: the description matches the maintained files and the existing composed-overlay render command still succeeds offline.
R5: Include Framework Configuration Inspection In Built-In Usage
Priority: P3 (low). Status: Open.
Location: internal/adapter/cli/run.go:742–749 (printUsage), following the
new command-family implementation in 2ba2ce1 and 411a67b.
The top-level usage lists inspect prompt, inspect profile, and application
config, but omits the implemented inspect configuration family entirely.
Invoking the binary without arguments reproduces this omission, so users
relying on its built-in command summary cannot discover the new framework
validation/provenance interface or distinguish it from application inspection.
Required change: add a concise usage line identifying all four operations,
the supported selection flags, and the --path requirement for explain.
Preserve existing exit behavior and command parsing; this does not require a
new help subsystem or aliases. Protect discoverability with a small CLI-entry
assertion for the family/operations, not a snapshot of the entire usage text
or its incidental ordering.
Acceptance: built-in usage advertises the actual family and remains consistent
with docs/cli.md; the focused CLI check and development baseline pass.
Review Verification
go test -count=1 ./...,go test -race -count=1 ./..., andgo vet ./...passed. A native binary built outside the repository.- Changed Go files are formatted; diff whitespace checks passed.
- All four maintained application configurations passed both application and framework validation with provider credentials unset. Both render scripts, composed-overlay render, and inheritance render passed offline.
- All four captured framework inspection operations passed against the complete example. Focused tests cover native layering/imports/resource capture, literal IDs, presence/null inheritance, partial-profile completion, scoped effective/history redaction, and unchanged existing inspection behavior.
- The selected graph is PromptKit v0.12.0, ConfigKit v0.5.0, and the two
/v3catalogs at v3.20260930.0, without a local replacement or workspace. Selected PromptKit contract and notice links resolved. - The broader redaction in the new framework-report family, removed legacy source fields/flags, and eager definition capture are explicitly approved changes, not accidental strengthening. Shared server capacity, request independence, HTTP wire behavior, and lexical artifact containment remain intact. No implementation fixes were made during this review.
ConfigKit v0.5.0 Review
Reviewed on 2026-10-03 at bcf2ddf, covering the five implementation
commits after e1ce526. The ConfigKit adoption and runtime ownership
boundaries match the accepted roadmap. No runtime defect, missing application
feature, or unrequested strengthening of contracts was identified. One
documentation handoff issue was identified; its current status is recorded below.
R2: Correct The Completed Plan's Introductory Status
Priority: P3 (low). Status: Superseded; no action against the current plan.
Location: docs/roadmap/implementation.md:5–9.
The introduction still says "All five stages are pending" and directs the reader to implement Stage 1 through Stage 5. Each stage is actually marked Complete with completion evidence, and the feature roadmap identifies the adoption as delivered. These contradictory directions leave the overall handoff inaccurate and could lead a later coding agent to repeat completed work.
This is a completion-documentation issue, not an unfinished runtime feature. The documentation policy assigns implementation status to roadmaps; the overall status should agree with the stage evidence.
Required change: update the introduction to state that all five stages are complete. Describe the retained steps as implementation history and acceptance evidence rather than instructions to start the work again. Preserve the original planning baseline, completed stage records, deferred scope, and separate release handoff. Do not change application code or rerun the feature implementation to repair the wording.
Acceptance: the overall status, individual stages, and feature roadmap agree
that implementation is complete, while release preparation remains separate.
Review the documentation diff and run git diff --check; this correction
does not require new tests.
The ConfigKit plan was replaced by the PromptKit plan in b1bb55a. Its
historical snapshot remains in Git, while the current introduction correctly
states that all thirteen PromptKit stages are complete. This old finding does
not direct changes to the new plan or repetition of either implementation.
Review Verification
go test -race -count=1 ./...passed, including the new configuration, inspection, and engine-adoption tests and existing shared-capacity coverage.go vet ./..., a native build outside the repository, and diff checks passed.- All five maintained inheritance example commands succeeded with
LOCAL_GPU_API_KEYabsent: validate, show, sources, explain, and render. The child selected its own backend ID with the inherited endpoint, zero queue capacity and false parameter remained explicit, and explanation retained the parent/destination/directive relationship. - Changed upstream documentation and notice links, affected local paths, and the historical policy-plan reference resolved.
- The work set changes dependency pins, tests, examples, and documentation; no production Go implementation was changed. ConfigKit v0.5.0 owns inheritance through the existing single-load integration; Promptkit remains v0.11.2.
- The
extendsreservation is the explicitly accepted upstream compatibility change, not a new Scriptorium restriction. Parameter and credential-reference visibility, working-directory-relative paths, ordinary parent registration, and existing shared server capacity remain as selected.
Earlier Promptkit Upgrade Review (Resolved)
Scope And Status
Review baseline: e177d7c, covering implementation commits after 5c3b0cf.
This document records a resolved test-contract issue; it does not authorize
unrelated implementation changes. No runtime defect or missing planned feature
was identified in this review. Release preparation remains separate work.
R1: Remove The Unrequested Context-Identity Invariant
Priority: P3 (low). Status: Complete.
Original location: internal/adapter/http/handler_test.go:334–349,
TestHandlerRunTimeoutValidationAndContextMapping, introduced in 7de2b02.
The test's child field and (ctx != parent) != tc.child assertion required
the exact parent context object for omitted/null/zero timeouts and a distinct
object for a positive timeout. The requested contract concerns effective
deadlines and cancellation propagation, not Go interface identity. In
particular, a harmless derived context that preserves the parent's deadline,
values, and cancellation would fail the omission cases despite satisfying the
application contract.
This elevated an incidental implementation choice into a test-enforced invariant, contrary to the agent policy and testing policy. The current handler's context behavior is correct; this is a maintainability issue, not a runtime cancellation bug.
Required Change
Remove the identity comparison and the identity-only table field. Retain behavioral assertions that an earlier parent deadline is not extended and parent cancellation reaches execution. For omitted/null/zero values, verify that no deadline is introduced when the parent has none. Retain coverage of positive deadlines and the distinction between a canceled execution context and a provider-attempt timeout while the execution context remains live.
Use the existing runner seam and focused cases; do not introduce a context wrapper framework, timer abstraction, new public guarantee, or production refactor solely to change this test.
Completion Criteria
- Tests protect effective deadline/cancellation behavior without requiring context object identity.
- A semantically equivalent derived context does not fail merely because it is a different object.
- Existing timeout validation and HTTP outcome coverage remains intact.
Resolution And Validation
Removed the identity-only table field and context comparisons. The test now checks effective deadlines, including omitted/null/zero requests with no parent deadline, and explicitly checks that parent cancellation reaches the runner. Existing positive-deadline integration tests and live-context provider-timeout mapping remain intact. No production code changed.
Focused HTTP timeout tests, go test ./..., go test -race ./...,
go vet ./..., a native build outside the repository, and git diff --check
all passed after remediation.
Original Review Verification
go test -race -count=1 ./...passed, including retry, shared-capacity, deadline, report, definition-conformance, and architecture-boundary tests.go vet ./...passed.- A native
cmd/scriptoriumbuild passed, with output outside the repository. - The selected dependency is Promptkit v0.11.0 with its intended v2 catalogs.
- Application code, tests, release automation, and historical releases were not changed during this review.
The runtime ownership boundaries remain as intended: Promptkit owns framework formats, execution, retries, and backend admission; Scriptorium owns executable adapters, application settings, HTTP artifact policy, deadlines, and terminal report presentation. No additional runtime restrictions, durable workflow state, or extra release compliance gates were identified in the reviewed diff.