Plan full PromptKit v0.23.0 adoption

This commit is contained in:
2026-11-15 01:11:58 +00:00
parent f42573f8b0
commit d28b560265
2 changed files with 304 additions and 0 deletions

View File

@@ -0,0 +1,168 @@
# PromptKit v0.23.0 Implementation Plan
**Status:** Ready for implementation; all tasks below are pending.
The [adoption roadmap](promptkit.md) owns the target behavior and boundaries.
This plan sequences the direct v0.21.0-to-v0.23.0 migration and the evidence needed
to establish full adoption. It does not authorize application code changes during
the current documentation pass.
## 1. Upgrade and unify execution
- [ ] Update `go.mod` and `go.sum` directly to
`go.maximumdirect.net/promptkit v0.23.0`. Use released module resolution; remove
temporary replacements/workspace files. Check the selected module graph and
compatibility with the existing CommandKit/ckprompt integration.
- [ ] Refactor [the LLM adapter](../../internal/framework/llm/adapter.go) around
shared request construction, response interpretation, and error mapping that
can serve both ordinary calls and coordinator-owned turns. Keep those helpers
narrow; do not wrap `Coordinate` in another turn loop.
- [ ] Move whole-operation timing to `RunRequest.OperationTimeout`. Use ordinary
`Run` for calls whose prepare/run split exists only to obtain diagnostics.
Avoid retaining a redundant child timer around the same ordinary operation.
- [ ] Supply `OnPrepared` to collect effective metadata and selected credential
source names, and retain prepared content for debug when enabled. Use the
delivered snapshot without a separate `Prepare` call. Clear per-call state
so a failure before notification cannot reuse a prior turn's evidence.
- [ ] Preserve per-role profile/schema/variable selection, transient and
structural-repair budgets, ordinary observers, output decoding, and error
disposition. Map `ErrOperationTimeout` explicitly, including timeouts during
preparation. Distinguish parent cancellation and provider-local deadlines.
Retain the public processing reason codes where their meanings still apply.
- [ ] Ensure call finalization runs on both results and errors; coordinator
execution errors do not arrive as ordinary response callbacks. Preserve the
last attempted operation's diagnostic and failure context for final reporting.
**Verification:** focused adapter tests for exact prepared evidence after a
provider failure, absent evidence before notification, independent operation
budgets, timeout classification, metadata redaction, and ordinary observer
composition. Use controlled model clients and cancellation. Retain explicit
prepared execution only where an identified consumer actually requires it.
## 2. Replace history and repair-loop mechanics
- [ ] Replace the local `Exchange`/`Replay` model in
[structured completion contracts](../../internal/framework/structuredoutput/call.go)
with a minimal opaque bridge to native captured exchanges. The adapter owns
PromptKit execution; modules and validators must not reconstruct messages or
depend on provider-specific state.
- [ ] Capture the original proposal run through `CaptureExchange`. Select the
repair prompt directly for a continuation; remove original-prompt rerendering,
manual assistant/suffix assembly, prefix comparison, and `RepairInstruction`
history plumbing. Preserve prepared snapshots used only for diagnostic output.
- [ ] Replace the attempt loop in
[repair coordination](../../internal/framework/runner/repair.go) with one
adapter-backed `Engine.Coordinate` invocation per actionable rejected proposal.
Keep scheduling across proposals in Audita. Pass the resolved correction repair
limit as `AdditionalTurns`, and bypass coordination when repair is disabled.
- [ ] Keep domain branch state in the callback: original proposal identity,
current revision, local segment mapping, current before/after preview, safe
rejection findings, and domain outcome. The initial callback uses existing
findings without rerunning the initial batch validation.
- [ ] Build each follow-up request with the repair definition, role profile,
repair schema, variables, operation timeout, prepared notification, and selected
observers. Let the coordinator attach `ContinueFrom` and manage history.
- [ ] Process `ConversationTurn.Result` only when present. Check structural
validation before schema decoding, interpret withdrawal, and revalidate every
revision from the beginning using the parent callback context. Validator calls
select their own profiles, timeouts, and observers through the shared adapter.
- [ ] Return a nil next request for accepted, withdrawn, or unavailable domain
outcomes. If rejection remains actionable, request a further turn even at the
budget boundary so PromptKit reports `CoordinationExhausted`; do not recreate
an independent exhaustion loop. The final permitted revision is still assessed.
- [ ] Keep diagnostic attempt coordinates and operation records, but allocate
execution evidence only for runs actually started. The callback may construct
a request that is never executed because the turn budget is exhausted.
- [ ] Map execution errors, callback errors, stopping, exhaustion, and
`Undelivered` separately. Unassessed or structurally invalid content never
reaches the approved set. Preserve accepted sibling proposals and apply-time
checks; fatal/canceled work still cancels and joins peers.
**Verification:** a controlled workflow with one initial approval and two rejected
proposals demonstrates independent histories, withdrawal, revised acceptance,
full revalidation, and deterministic application. Cover final-turn acceptance,
exhaustion, zero repair, structural invalidity, recoverable failures, parent
cancellation before delivery, and capacity-one validator calls. Assertions should
exercise Audita outcomes through the real upstream coordinator with a fake model
boundary, rather than test a fake coordinator's loop behavior.
## 3. Adopt native execution accounting
- [ ] Introduce an invocation-scoped `ExecutionGroup` and compose its `Observe`
callback with existing diagnostic observers using `ComposeObservers`. Ensure
observation remains enabled wherever required accounting is consumed,
independently of whether debug artifacts are enabled.
- [ ] Explicitly enroll proposal, repair-generation, and validator runs. Use
`CoordinateRequest.Observer` for continuation membership and select membership
separately for nested validation requests. Keep conversation parent identity
distinct from Audita's stage/section/proposal/validator labels.
- [ ] Replace the terminal-map deduplication and usage summation in
[observation summaries](../../internal/framework/llm/observations.go) with
`ExecutionGroup.Snapshot()`. Convert upstream summary fields only at presentation
boundaries. Retain logical call metadata, safe event evidence, statuses, and
deterministic ordering that the upstream group does not supply.
- [ ] Carry terminal coverage separately from usage known/completeness into the
internal summary and its diagnostic representation. Do not present missing
terminals as complete zero accounting. Keep the established ordinary report
scope; adding token payloads to the ordinary errors/recoveries report is not
part of this upgrade.
- [ ] Count shared initial work once in the invocation group; count each new
continuation and validator run separately. Do not sum overlapping subgroup
totals or attempt events on top of their terminal totals. Sort any presented
member lists deterministically rather than relying on native snapshot ordering.
- [ ] Snapshot final totals after all selected work has joined. Groups add no
separate close/wait lifecycle. Audit and delete local aggregation that became
redundant; document the purpose of any remaining application-specific counters.
**Verification:** integration tests demonstrate composition with existing
observers, explicit validator membership, shared initial work, failure terminal
facts, unknown versus zero usage, missing terminal coverage, and concurrent calls.
Reuse upstream group semantics; test Audita's enrollment and presentation instead
of duplicating the library's unit suite.
## 4. Cleanup, documentation, and acceptance
- [ ] Audit references to the old exchange/replay types, prefix checks,
`RepairInstruction`, local repair attempt loop, redundant timeout wrapper, and
terminal aggregation. Delete dead production helpers and replace tests tied to
those implementations with focused behavior coverage. A thin local adapter is
justified; a parallel conversation/coordinator/accounting implementation is not.
- [ ] Update the canonical [LLM runtime](../internal/llm-runtime.md),
[pipeline](../internal/pipeline.md),
[diagnostics/reporting internals](../internal/diagnostics-reporting.md), and
[LLM integration contract](../integrations/openai-compatible-llm.md) as behavior
lands. Update pinned upstream references and dependency attribution as needed.
Document the selected notification boundary and timeout mapping without claiming
that preparation or admission failures always have a prepared snapshot.
- [ ] Update configuration/operations references only for actual externally
visible changes. No prompt rewrite, validator tuning, new configuration format,
compatibility shim, new scheduler, or unavailable-decision regeneration is
required by this migration.
- [ ] Follow the [testing policy](../policy/testing.md): keep tests that protect
domain and integration behavior, remove obsolete implementation-coupled tests,
and avoid prompt word/phrase change detectors. Review final tests for redundancy.
- [ ] Run the focused migration tests, the complete Audita suite, and a race-enabled
check of changed concurrent adapter/runner/accounting paths. Build the CLI and
exercise existing offline processing and inspection coverage. No paid provider
calls are required for acceptance.
- [ ] Review the final diff, verify local Markdown links and `git diff --check`,
and confirm no local dependency replacements or workspace artifacts remain.
Verify the selected PromptKit version is v0.23.0.
- [ ] Confirm all three upstream capability families are used in production
processing and the agreed local machinery is retired. Record completion only
after this end-state audit; move useful current contracts into their canonical
owners and retire these temporary planning documents under repository policy.
Suggested final checks, using the locally selected Go 1.27.0-or-newer toolchain:
```sh
go list -m go.maximumdirect.net/promptkit
go test ./...
go test -race ./internal/framework/llm ./internal/framework/runner ./internal/framework/processreport
go build -o ./bin/audita ./cmd/audita
git diff --check
```
Adjust focused package checks if code ownership moves. A green dependency-only
suite is not evidence that the new workflow has been adopted; acceptance also
requires the coordinator, conversation, and group integration scenarios above.

136
docs/roadmap/promptkit.md Normal file
View File

@@ -0,0 +1,136 @@
# PromptKit v0.23.0 Adoption Roadmap
**Status:** Accepted target; implementation pending. This pass changes planning
documents only. The [implementation plan](promptkit-implementation.md) owns the
ordered tasks and verification.
## Objective and Baseline
Upgrade Audita directly from PromptKit v0.21.0 to v0.23.0 and fully adopt its
conversation, coordinator, and execution-group APIs. Removing redundant local
machinery is an explicit completion requirement. A dependency-only upgrade or
continuation adoption with the old repair loop retained is an intermediate state.
The current [dependency manifest](../../go.mod) pins v0.21.0. Audita captures its
own exchange and replay values, reconstructs and compares conversation prefixes,
appends repair history, manages the revision loop, and aggregates terminal
observations locally. Each role client wraps explicit preparation and execution
in a local deadline. See the [LLM runtime](../internal/llm-runtime.md),
[pipeline](../internal/pipeline.md), and
[diagnostic implementation](../internal/diagnostics-reporting.md).
[PromptKit v0.23.0](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.0/docs/releases/v0.23.0.md)
includes the v0.22.0 conversation and accounting capabilities plus the execution
controls needed for full coordinator adoption. The former upstream timeout and
prepared-request gaps are closed; no intermediate v0.22.0 rollout or further
upstream feature gate is required. The Go minimum and dependency versions are
unchanged by these upstream releases.
The release review verified Audita's existing suite against the tagged v0.23.0
source through a temporary module override and selected upstream packages with
race detection. That is compatibility evidence, not verification of the migration
described here. Implementation must use the released dependency, without a local
replacement or committed workspace file.
## Target Ownership
| Responsibility | Target owner |
| --- | --- |
| Immutable captured history, continuation assembly, and parent run identity | PromptKit `Exchange`, `CaptureExchange`, and `ContinueFrom` |
| Sequential turns, additional-turn budget, callback delivery, and coordination termination | PromptKit `Engine.Coordinate` |
| Whole-operation deadlines and timeout attribution | PromptKit `OperationTimeout` and `ErrOperationTimeout` |
| Exact prepared-request notification before generation | PromptKit `OnPrepared` |
| Selected run membership, deduplication, and terminal count/usage aggregation | PromptKit `ExecutionGroup` and `ComposeObservers` |
| Candidate identity, actionable rejection, feedback, revision decoding, withdrawal, and full revalidation | Audita |
| Branch scheduling, accepted sibling proposals, application order, and transcript mutation | Audita |
| Domain operation labels, failure disposition, reports, redaction, and debug artifact storage | Audita |
Use the supported public API through Audita's framework adapter boundaries.
Keep module and validator contracts domain-oriented. Any bridge needed by the
runner must delegate to the real coordinator and retain opaque conversation
state, rather than implementing another conversation engine or general workflow
framework.
## Selected Behavior
### Execution and diagnostics
Use ordinary `Engine.Run` with per-request `OperationTimeout` and `OnPrepared`
where explicit preparation currently serves only to inspect and execute the same
request. Map the resolved Audita operation timeout onto each proposal, validation,
and repair operation. Keep provider, transport-retry, structural-repair, and
correction-revision budgets distinct.
Use `ErrOperationTimeout` to identify the selected local budget, while retaining
Audita's parent-cancellation and recoverable-provider-failure handling. Do not
classify every wrapped preparation error as fatal before considering its timeout
identity. Ordinary configuration failures remain fatal. If split preparation is
still needed for a concrete consumer, one caller context must cover both calls
when a total preparation-plus-execution deadline is required; the upstream
prepared-handle duration starts only at execution.
`OnPrepared` receives the actual detached snapshot. Retain the metadata needed
for attribution and credential-source redaction, and capture content for optional
debug evidence. Keep the callback short; application validation belongs in the
next-turn callback after admission is released. Notification may be absent on
preparation/admission failure or cancellation before delivery. Reports must not
invent missing snapshots. Optional debug write failures remain nonfatal.
### Conversations and correction repair
Capture the proposal exchange when repair needs it. Every rejected candidate's
branch starts from that same immutable exchange; subsequent turns remain private
to that branch. Follow-ups select the module's repair prompt, role profile,
variables, and repair output contract through ordinary PromptKit requests.
Retain native operator prompt overrides and the existing domain prompt content.
Use one coordinator per actionable correction. Its initial callback uses existing
rejection findings. Each newly delivered response must pass structural validation
before decoding; captured content alone does not imply validity. A decoded
revision is fully revalidated before acceptance. Withdrawal, unavailable results,
and accepted revisions remain Audita outcomes; a normal coordinator stop is not
itself an approval.
The last permitted response follows the same assessment path. Further requested
work beyond the budget maps to exhaustion. Skip coordination when Audita's
correction-repair setting disables it: upstream's zero budget still invokes an
initial callback. Cancellation's `Undelivered` response is diagnostic evidence,
not an approved correction, and must not trigger model work after cancellation.
Recoverable branch failures preserve independent approvals; fatal failures and
caller cancellation retain Audita's cancel-and-join behavior.
### Execution groups
Use native groups as the source of aggregate provider-attempt, retry, repair,
and available token totals. Explicitly select membership for proposal calls,
coordinated continuations, and validator calls. Coordinator observers cover its
own turns; nested validators require their own observer selection.
Maintain an invocation group for combined execution totals. Add narrower groups
only for accounting views Audita consumes. A shared initial proposal run belongs
once to the invocation total; replaying its history does not enroll it again.
Each new call's reported history tokens still count. Overlapping group totals
must not be summed as disjoint work.
Preserve upstream usage presence/completeness and terminal coverage separately.
Keep domain logical-call records and deterministic presentation where needed;
their count need not equal observed run membership. Group summaries do not replace
Audita's errors/recoveries, validation decisions, or per-operation failure facts.
## Delivery and Completion
Implement in ordered, reviewable stages: dependency and execution adapter;
captured conversations and coordinator; execution groups; cleanup, integration
verification, and canonical documentation. All stages belong to this adoption.
Completion requires removal of local exchange/replay history, prefix checks,
repair turn-budget loops, and duplicate terminal aggregation where replaced by
PromptKit. Retain local behavior only where it has a concrete Audita domain or
presentation responsibility. Preserve transcript correctness, repair prompts,
validation semantics, deterministic reports, standard inspection commands, and
operator configuration unless a deliberate change is documented and tested.
No product-level open decisions remain. Exact internal bridge signatures are
implementation choices governed by this ownership split. Future validator
decision recovery and new scheduling behavior remain in the
[deferred work catalog](future.md).