11 KiB
PromptKit v0.23.0 Implementation Plan
Status: Ready for implementation; all tasks below are pending.
The adoption roadmap owns the target behavior and boundaries. This plan sequences the direct v0.21.0-to-v0.23.0 migration and the evidence needed to establish full adoption. It does not authorize application code changes during the current documentation pass.
1. Upgrade and unify execution
- Update
go.modandgo.sumdirectly togo.maximumdirect.net/promptkit v0.23.0. Use released module resolution; remove temporary replacements/workspace files. Check the selected module graph and compatibility with the existing CommandKit/ckprompt integration. - Refactor the LLM adapter around
shared request construction, response interpretation, and error mapping that
can serve both ordinary calls and coordinator-owned turns. Keep those helpers
narrow; do not wrap
Coordinatein another turn loop. - Move whole-operation timing to
RunRequest.OperationTimeout. Use ordinaryRunfor calls whose prepare/run split exists only to obtain diagnostics. Avoid retaining a redundant child timer around the same ordinary operation. - Supply
OnPreparedto collect effective metadata and selected credential source names, and retain prepared content for debug when enabled. Use the delivered snapshot without a separatePreparecall. Clear per-call state so a failure before notification cannot reuse a prior turn's evidence. - Preserve per-role profile/schema/variable selection, transient and
structural-repair budgets, ordinary observers, output decoding, and error
disposition. Map
ErrOperationTimeoutexplicitly, including timeouts during preparation. Distinguish parent cancellation and provider-local deadlines. Retain the public processing reason codes where their meanings still apply. - Ensure call finalization runs on both results and errors; coordinator execution errors do not arrive as ordinary response callbacks. Preserve the last attempted operation's diagnostic and failure context for final reporting.
Verification: focused adapter tests for exact prepared evidence after a provider failure, absent evidence before notification, independent operation budgets, timeout classification, metadata redaction, and ordinary observer composition. Use controlled model clients and cancellation. Retain explicit prepared execution only where an identified consumer actually requires it.
2. Replace history and repair-loop mechanics
- Replace the local
Exchange/Replaymodel in structured completion contracts with a minimal opaque bridge to native captured exchanges. The adapter owns PromptKit execution; modules and validators must not reconstruct messages or depend on provider-specific state. - Capture the original proposal run through
CaptureExchange. Select the repair prompt directly for a continuation; remove original-prompt rerendering, manual assistant/suffix assembly, prefix comparison, andRepairInstructionhistory plumbing. Preserve prepared snapshots used only for diagnostic output. - Replace the attempt loop in
repair coordination with one
adapter-backed
Engine.Coordinateinvocation per actionable rejected proposal. Keep scheduling across proposals in Audita. Pass the resolved correction repair limit asAdditionalTurns, and bypass coordination when repair is disabled. - Keep domain branch state in the callback: original proposal identity, current revision, local segment mapping, current before/after preview, safe rejection findings, and domain outcome. The initial callback uses existing findings without rerunning the initial batch validation.
- Build each follow-up request with the repair definition, role profile,
repair schema, variables, operation timeout, prepared notification, and selected
observers. Let the coordinator attach
ContinueFromand manage history. - Process
ConversationTurn.Resultonly when present. Check structural validation before schema decoding, interpret withdrawal, and revalidate every revision from the beginning using the parent callback context. Validator calls select their own profiles, timeouts, and observers through the shared adapter. - Return a nil next request for accepted, withdrawn, or unavailable domain
outcomes. If rejection remains actionable, request a further turn even at the
budget boundary so PromptKit reports
CoordinationExhausted; do not recreate an independent exhaustion loop. The final permitted revision is still assessed. - Keep diagnostic attempt coordinates and operation records, but allocate execution evidence only for runs actually started. The callback may construct a request that is never executed because the turn budget is exhausted.
- Map execution errors, callback errors, stopping, exhaustion, and
Undeliveredseparately. Unassessed or structurally invalid content never reaches the approved set. Preserve accepted sibling proposals and apply-time checks; fatal/canceled work still cancels and joins peers.
Verification: a controlled workflow with one initial approval and two rejected proposals demonstrates independent histories, withdrawal, revised acceptance, full revalidation, and deterministic application. Cover final-turn acceptance, exhaustion, zero repair, structural invalidity, recoverable failures, parent cancellation before delivery, and capacity-one validator calls. Assertions should exercise Audita outcomes through the real upstream coordinator with a fake model boundary, rather than test a fake coordinator's loop behavior.
3. Adopt native execution accounting
- Introduce an invocation-scoped
ExecutionGroupand compose itsObservecallback with existing diagnostic observers usingComposeObservers. Ensure observation remains enabled wherever required accounting is consumed, independently of whether debug artifacts are enabled. - Explicitly enroll proposal, repair-generation, and validator runs. Use
CoordinateRequest.Observerfor continuation membership and select membership separately for nested validation requests. Keep conversation parent identity distinct from Audita's stage/section/proposal/validator labels. - Replace the terminal-map deduplication and usage summation in
observation summaries with
ExecutionGroup.Snapshot(). Convert upstream summary fields only at presentation boundaries. Retain logical call metadata, safe event evidence, statuses, and deterministic ordering that the upstream group does not supply. - Carry terminal coverage separately from usage known/completeness into the internal summary and its diagnostic representation. Do not present missing terminals as complete zero accounting. Keep the established ordinary report scope; adding token payloads to the ordinary errors/recoveries report is not part of this upgrade.
- Count shared initial work once in the invocation group; count each new continuation and validator run separately. Do not sum overlapping subgroup totals or attempt events on top of their terminal totals. Sort any presented member lists deterministically rather than relying on native snapshot ordering.
- Snapshot final totals after all selected work has joined. Groups add no separate close/wait lifecycle. Audit and delete local aggregation that became redundant; document the purpose of any remaining application-specific counters.
Verification: integration tests demonstrate composition with existing observers, explicit validator membership, shared initial work, failure terminal facts, unknown versus zero usage, missing terminal coverage, and concurrent calls. Reuse upstream group semantics; test Audita's enrollment and presentation instead of duplicating the library's unit suite.
4. Cleanup, documentation, and acceptance
- Audit references to the old exchange/replay types, prefix checks,
RepairInstruction, local repair attempt loop, redundant timeout wrapper, and terminal aggregation. Delete dead production helpers and replace tests tied to those implementations with focused behavior coverage. A thin local adapter is justified; a parallel conversation/coordinator/accounting implementation is not. - Update the canonical LLM runtime, pipeline, diagnostics/reporting internals, and LLM integration contract as behavior lands. Update pinned upstream references and dependency attribution as needed. Document the selected notification boundary and timeout mapping without claiming that preparation or admission failures always have a prepared snapshot.
- Update configuration/operations references only for actual externally visible changes. No prompt rewrite, validator tuning, new configuration format, compatibility shim, new scheduler, or unavailable-decision regeneration is required by this migration.
- Follow the testing policy: keep tests that protect domain and integration behavior, remove obsolete implementation-coupled tests, and avoid prompt word/phrase change detectors. Review final tests for redundancy.
- Run the focused migration tests, the complete Audita suite, and a race-enabled check of changed concurrent adapter/runner/accounting paths. Build the CLI and exercise existing offline processing and inspection coverage. No paid provider calls are required for acceptance.
- Review the final diff, verify local Markdown links and
git diff --check, and confirm no local dependency replacements or workspace artifacts remain. Verify the selected PromptKit version is v0.23.0. - Confirm all three upstream capability families are used in production processing and the agreed local machinery is retired. Record completion only after this end-state audit; move useful current contracts into their canonical owners and retire these temporary planning documents under repository policy.
Suggested final checks, using the locally selected Go 1.27.0-or-newer toolchain:
go list -m go.maximumdirect.net/promptkit
go test ./...
go test -race ./internal/framework/llm ./internal/framework/runner ./internal/framework/processreport
go build -o ./bin/audita ./cmd/audita
git diff --check
Adjust focused package checks if code ownership moves. A green dependency-only suite is not evidence that the new workflow has been adopted; acceptance also requires the coordinator, conversation, and group integration scenarios above.