Files
scriptorium/docs/roadmap/remediation.md

13 KiB
Raw Blame History

Implementation Review Remediation

PromptKit v0.12.0 Review

Reviewed on 2026-10-04 at 58cb02f9732b09740ff7e22096d2a4072ddfddca, covering the thirteen implementation commits after b1bb55a. The runtime architecture matches the accepted feature roadmap and implementation plan. No runtime regression, missing framework capability, or newly imposed runtime restriction was identified. Three low-priority documentation/help gaps need correction. The first retains a loading guarantee that is incompatible with the deliberately selected full-source engine construction.

R3: Correct The Definition-Inspection Loading Contract

Priority: P3 (low). Status: Open.

Locations: docs/cli.md:208–224; related orientation in docs/internal/adapters.md:49–56. The CLI wording was retained in b913ef0 despite the engine-wiring change in 0df2aad.

The profile-inspection reference still says it never loads a prompt. Both definition-inspection commands now construct the same complete selected engine as execution. PromptKit resolves and validates all final definitions and captures their message content before the selected inspection runs. Consequently, an unused prompt with invalid syntax or an unreadable content_file can prevent inspection of an otherwise valid profile. Prompt inspection's unqualified claim that it does not resolve a profile also needs to distinguish engine assembly from request-specific target resolution.

Evidence: inspecting a valid profile in a selected combined file with an unused prompt referring to a missing message file exits 1 before emitting a profile report. The existing TestProfileInspectionUsesCombinedSourcesAndValidatesUnusedPrompts also protects this intentional eager-validation boundary.

Required change: explain that both commands construct and validate the full selected configuration, including unused definitions and message resources, before inspecting the chosen definition. Preserve the distinction that they do not prepare an execution request, load request-time schemas/artifacts, resolve credential values, reserve capacity, or call a provider. Clarify that no prompt collection is mandatory, but selected prompts are not ignored. Align the internal summary with the CLI contract rather than restating a stronger independence guarantee. Do not restore the empty-prompt-filesystem workaround or weaken eager validation to satisfy the stale wording.

Acceptance: the owning contract and internal summary agree with the roadmap, shared engine construction, and existing tests. Review the documentation diff and links; no production-code change or new runtime guarantee is needed.

R4: Update The Composed-Example Description

Priority: P3 (low). Status: Open.

Location: docs/config.md:95–96, after the example migration in 2027b80.

The reference says examples/config.composed.yml imports two application fragments. It now imports only config-fragments/sources.yml; the former backend fragment became examples/framework/backends.yml, selected through promptkit_sources rather than imported into the application document. The example and code are correct, but the explanation confuses the two source boundaries introduced by this migration.

Required change: describe the actual source-selection fragment and separately selected framework backend file, with links where useful. Do not reintroduce an application backend mapping or add an unnecessary fragment merely to match the old prose. The old backend fragment had no application defaults to preserve.

Acceptance: the description matches the maintained files and the existing composed-overlay render command still succeeds offline.

R5: Include Framework Configuration Inspection In Built-In Usage

Priority: P3 (low). Status: Open.

Location: internal/adapter/cli/run.go:742–749 (printUsage), following the new command-family implementation in 2ba2ce1 and 411a67b.

The top-level usage lists inspect prompt, inspect profile, and application config, but omits the implemented inspect configuration family entirely. Invoking the binary without arguments reproduces this omission, so users relying on its built-in command summary cannot discover the new framework validation/provenance interface or distinguish it from application inspection.

Required change: add a concise usage line identifying all four operations, the supported selection flags, and the --path requirement for explain. Preserve existing exit behavior and command parsing; this does not require a new help subsystem or aliases. Protect discoverability with a small CLI-entry assertion for the family/operations, not a snapshot of the entire usage text or its incidental ordering.

Acceptance: built-in usage advertises the actual family and remains consistent with docs/cli.md; the focused CLI check and development baseline pass.

Review Verification

  • go test -count=1 ./..., go test -race -count=1 ./..., and go vet ./... passed. A native binary built outside the repository.
  • Changed Go files are formatted; diff whitespace checks passed.
  • All four maintained application configurations passed both application and framework validation with provider credentials unset. Both render scripts, composed-overlay render, and inheritance render passed offline.
  • All four captured framework inspection operations passed against the complete example. Focused tests cover native layering/imports/resource capture, literal IDs, presence/null inheritance, partial-profile completion, scoped effective/history redaction, and unchanged existing inspection behavior.
  • The selected graph is PromptKit v0.12.0, ConfigKit v0.5.0, and the two /v3 catalogs at v3.20260930.0, without a local replacement or workspace. Selected PromptKit contract and notice links resolved.
  • The broader redaction in the new framework-report family, removed legacy source fields/flags, and eager definition capture are explicitly approved changes, not accidental strengthening. Shared server capacity, request independence, HTTP wire behavior, and lexical artifact containment remain intact. No implementation fixes were made during this review.

ConfigKit v0.5.0 Review

Reviewed on 2026-10-03 at bcf2ddf, covering the five implementation commits after e1ce526. The ConfigKit adoption and runtime ownership boundaries match the accepted roadmap. No runtime defect, missing application feature, or unrequested strengthening of contracts was identified. One documentation handoff issue was identified; its current status is recorded below.

R2: Correct The Completed Plan's Introductory Status

Priority: P3 (low). Status: Superseded; no action against the current plan.

Location: docs/roadmap/implementation.md:5–9.

The introduction still says "All five stages are pending" and directs the reader to implement Stage 1 through Stage 5. Each stage is actually marked Complete with completion evidence, and the feature roadmap identifies the adoption as delivered. These contradictory directions leave the overall handoff inaccurate and could lead a later coding agent to repeat completed work.

This is a completion-documentation issue, not an unfinished runtime feature. The documentation policy assigns implementation status to roadmaps; the overall status should agree with the stage evidence.

Required change: update the introduction to state that all five stages are complete. Describe the retained steps as implementation history and acceptance evidence rather than instructions to start the work again. Preserve the original planning baseline, completed stage records, deferred scope, and separate release handoff. Do not change application code or rerun the feature implementation to repair the wording.

Acceptance: the overall status, individual stages, and feature roadmap agree that implementation is complete, while release preparation remains separate. Review the documentation diff and run git diff --check; this correction does not require new tests.

The ConfigKit plan was replaced by the PromptKit plan in b1bb55a. Its historical snapshot remains in Git, while the current introduction correctly states that all thirteen PromptKit stages are complete. This old finding does not direct changes to the new plan or repetition of either implementation.

Review Verification

  • go test -race -count=1 ./... passed, including the new configuration, inspection, and engine-adoption tests and existing shared-capacity coverage.
  • go vet ./..., a native build outside the repository, and diff checks passed.
  • All five maintained inheritance example commands succeeded with LOCAL_GPU_API_KEY absent: validate, show, sources, explain, and render. The child selected its own backend ID with the inherited endpoint, zero queue capacity and false parameter remained explicit, and explanation retained the parent/destination/directive relationship.
  • Changed upstream documentation and notice links, affected local paths, and the historical policy-plan reference resolved.
  • The work set changes dependency pins, tests, examples, and documentation; no production Go implementation was changed. ConfigKit v0.5.0 owns inheritance through the existing single-load integration; Promptkit remains v0.11.2.
  • The extends reservation is the explicitly accepted upstream compatibility change, not a new Scriptorium restriction. Parameter and credential-reference visibility, working-directory-relative paths, ordinary parent registration, and existing shared server capacity remain as selected.

Earlier Promptkit Upgrade Review (Resolved)

Scope And Status

Review baseline: e177d7c, covering implementation commits after 5c3b0cf. This document records a resolved test-contract issue; it does not authorize unrelated implementation changes. No runtime defect or missing planned feature was identified in this review. Release preparation remains separate work.

R1: Remove The Unrequested Context-Identity Invariant

Priority: P3 (low). Status: Complete.

Original location: internal/adapter/http/handler_test.go:334–349, TestHandlerRunTimeoutValidationAndContextMapping, introduced in 7de2b02.

The test's child field and (ctx != parent) != tc.child assertion required the exact parent context object for omitted/null/zero timeouts and a distinct object for a positive timeout. The requested contract concerns effective deadlines and cancellation propagation, not Go interface identity. In particular, a harmless derived context that preserves the parent's deadline, values, and cancellation would fail the omission cases despite satisfying the application contract.

This elevated an incidental implementation choice into a test-enforced invariant, contrary to the agent policy and testing policy. The current handler's context behavior is correct; this is a maintainability issue, not a runtime cancellation bug.

Required Change

Remove the identity comparison and the identity-only table field. Retain behavioral assertions that an earlier parent deadline is not extended and parent cancellation reaches execution. For omitted/null/zero values, verify that no deadline is introduced when the parent has none. Retain coverage of positive deadlines and the distinction between a canceled execution context and a provider-attempt timeout while the execution context remains live.

Use the existing runner seam and focused cases; do not introduce a context wrapper framework, timer abstraction, new public guarantee, or production refactor solely to change this test.

Completion Criteria

  • Tests protect effective deadline/cancellation behavior without requiring context object identity.
  • A semantically equivalent derived context does not fail merely because it is a different object.
  • Existing timeout validation and HTTP outcome coverage remains intact.

Resolution And Validation

Removed the identity-only table field and context comparisons. The test now checks effective deadlines, including omitted/null/zero requests with no parent deadline, and explicitly checks that parent cancellation reaches the runner. Existing positive-deadline integration tests and live-context provider-timeout mapping remain intact. No production code changed.

Focused HTTP timeout tests, go test ./..., go test -race ./..., go vet ./..., a native build outside the repository, and git diff --check all passed after remediation.

Original Review Verification

  • go test -race -count=1 ./... passed, including retry, shared-capacity, deadline, report, definition-conformance, and architecture-boundary tests.
  • go vet ./... passed.
  • A native cmd/scriptorium build passed, with output outside the repository.
  • The selected dependency is Promptkit v0.11.0 with its intended v2 catalogs.
  • Application code, tests, release automation, and historical releases were not changed during this review.

The runtime ownership boundaries remain as intended: Promptkit owns framework formats, execution, retries, and backend admission; Scriptorium owns executable adapters, application settings, HTTP artifact policy, deadlines, and terminal report presentation. No additional runtime restrictions, durable workflow state, or extra release compliance gates were identified in the reviewed diff.