base-layer/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 7 Wiki Activity

Prepare v0.23.0 release notes and retire completed roadmaps

Browse Source
This commit is contained in:
Eric Rakestraw
2026-11-14 19:46:54 +00:00
parent d3bc439bb8
commit 8d697016c6
6 changed files with 66 additions and 959 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -45,6 +45,10 @@ boundary and constraints that framework work must preserve.
## Release Guidance
The [v0.23.0 release notes](docs/releases/v0.23.0.md) cover per-operation
timeouts and opt-in prepared-request notifications for ordinary and coordinated
execution.
The [v0.22.0 release notes](docs/releases/v0.22.0.md) cover captured conversation
exchanges, application-directed turn coordination, and explicit execution groups.

62
docs/releases/v0.23.0.md Normal file
View File

@@ -0,0 +1,62 @@
# Promptkit v0.23.0
This minor release adds optional whole-operation timeouts and prepared-request
notifications to ordinary and coordinated execution. Applications can bound
each continuation independently and retain the request selected for execution
without replacing Promptkit's turn loop or preparing the same request twice.
## Bound Individual Operations
`RunRequest.OperationTimeout` selects a fresh cooperative budget covering
preparation and execution, including capacity waiting, retries, validation,
structural repair, and prepared-request notification. It is independent of the
provider generation timeout. Zero or negative values select no additional
timeout; existing parent cancellation and earlier deadlines remain effective.
For split execution, `PrepareExecution` retains the duration without starting
a timer. `RunPrepared` starts that budget after claiming the handle, so prior
preparation and idle time do not consume it. Callers can still supply one shared
deadline context to bound both calls together.
Local budget failures match both `ErrOperationTimeout` and
`context.DeadlineExceeded`. Each coordinated turn uses its selected budget;
expiry stops that coordination through the existing operational-error path
without canceling its parent or sibling work. Completed outcomes are fixed
before terminal observation and are not changed by a slow terminal observer.
Cancellation does not forcibly interrupt callbacks or collaborators that ignore
their context.
See the [execution-controls guide](../consumers/pkg-promptkit.md#bound-execution-and-retain-prepared-details)
and the [public request contract](../../types.go) and
[timeout error contract](../../engine.go) for exact behavior.
## Retain Prepared Details Before Generation
`RunRequest.OnPrepared` optionally receives a detached, credential-redacted
`PreparedRun` and the operation context before first generation. It uses the
actual prepared snapshot, including continuation history, without rereading
inputs or rerendering. Applications may retain these details even if generation
later fails without returning a result.
Notification is synchronous, at most once per logical execution, and runs while
logical admission is held. It is not repeated for retries or structural repair.
Preparation or admission failure can prevent delivery. The application owns
diagnostic storage and disclosure decisions; ordinary observers remain
metadata-only. See the [OnPrepared GoDoc](../../types.go) and updated
[offline observation example](../../examples/go-library/observe/main.go).
## Compatibility And Upgrade
Both controls are opt-in. Existing keyed requests and configuration files need
no migration from v0.22.0. `Prepare` and `ValidateRequest` continue using only
their supplied contexts and do not invoke `OnPrepared`. `RunRequest` gained
fields; consumers using positional struct literals must switch to keyed
literals. Prepared/result JSON representations are unchanged.
The Go consumer minimum remains 1.27.0, and dependency versions are unchanged.
```sh
go get go.maximumdirect.net/promptkit@v0.23.0
```
[COPYRIGHT](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.0/COPYRIGHT) | [LICENSE](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.0/LICENSE) | [ATTRIBUTION](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.0/docs/oss/attribution.md)

139
docs/roadmap/audita-request.md
View File

@@ -1,139 +0,0 @@
# Audita request: coordinator execution controls
Status: proposed follow-up to PromptKit v0.22.0. This document describes
consumer requirements; public API design and implementation remain upstream
decisions.
## Purpose
PromptKit v0.22.0 supplies the conversation capture, continuation, and execution
groups Audita requested. Audita can adopt those capabilities through its existing
execution adapter. To also adopt `Engine.Coordinate` and remove its local turn
loop, Audita needs two capabilities during coordinator-owned execution:
1. A fresh, application-selected timeout for each continuation operation.
2. Access to the exact prepared request before generation, including for turns
that subsequently fail without returning a result.
Both capabilities apply to applications that run bounded model operations and
retain optional diagnostic evidence. Neither requires PromptKit to understand
transcripts, correction proposals, or domain validation.
## Current workflow and integration gap
Audita's role-bound adapter currently derives a timeout context from its caller,
calls `PrepareExecution`, retains `PreparedExecution.Details()`, and executes
that same handle with `RunPrepared`. The operation deadline covers preparation,
capacity waiting, transient retries, generation, and structural repair. Each
model operation receives its own budget; application validation may make
additional, separately bounded model calls.
When debug evidence is enabled, Audita retains the prepared messages and
effective settings alongside the available response or failure information.
It owns artifact destinations, application-specific redaction, and handling of
optional diagnostic write failures. The same adapter supplies ordinary execution
observers and classifies operation errors for the application.
In v0.22.0, the [coordinator](../../coordinate.go) accepts a callback returning a
`RunRequest`, then directly calls `Engine.Run` with the coordinator's context.
The callback cannot supply a fresh execution context or a prepared handle.
There is no callback exposing the actual prepared request before that run.
A timeout around the entire coordinator bounds the conversation and application
callbacks together; it does not preserve a separate timeout for each operation.
Provider generation timeouts do not cover the whole operation either.
Separately calling `Prepare` before returning a request duplicates preparation
and does not bind the inspected snapshot to the execution. Successful exchange
inspection cannot supply a failed turn's prepared request when no new exchange
is returned. Existing metadata-only observations do not contain that content.
## Requested behavior
### Per-operation deadlines for coordinated turns
Allow the application to select a timeout for each requested continuation, with
these semantics:
- Establish a fresh deadline when that continuation starts, before its
preparation. Cover preparation, admission waiting, retries, generation, and
structural repair under the same operation budget.
- Respect caller cancellation and any earlier parent deadline. Expiry of a
local operation budget must not cancel the parent context or independent
branches.
- End the operation's timeout scope before invoking the application's next-turn
callback. Assessment calls made there can retain their own operation budgets.
- Preserve useful error identity so Audita can distinguish a local deadline
from cancellation of the caller. An execution timeout can end coordination
under the existing operational-error rule; no automatic turn replay or new
recovery policy is requested.
The existing additional-turn budget, transport retry budget, structural-repair
budget, and provider generation timeout retain their separate meanings.
### Exact prepared-request access
Provide an opt-in way to obtain caller-owned details of the actual prepared
continuation before its first generation attempt:
- Include the assembled captured history, follow-up definition messages,
appended messages, and ordinary resolved execution metadata, using the
existing [prepared-details contract](../../prepared_execution.go) where useful.
- Expose the snapshot that execution will use, without a second rendering or
input-reading pass. Access to details must not mutate that execution.
- Let the application retain those details even if generation subsequently
fails or is canceled and returns no ordinary result or captured exchange.
If preparation itself fails, a complete prepared snapshot is not required.
- Preserve existing credential-redaction semantics. Keep content-bearing access
explicitly selected and separate from ordinary metadata-only observations.
Audita needs the logical prepared request, not raw HTTP envelopes or an archive
of intermediate structural-repair requests. It can handle optional debug writes
itself without requiring PromptKit to own artifact storage or reporting policy.
## API direction and ownership
Prefer small execution-level controls that the coordinator honors through its
ordinary execution path. An optional operation timeout and a prepared-details
callback are one possible shape, not proposed exported names or signatures.
Their applicability to ordinary and prepared execution should follow a coherent
lifecycle rather than introducing a second coordinator-specific executor.
An alternative is allowing a coordinator turn to delegate execution through a
caller-controlled prepared-execution path. That could reuse Audita's adapter,
but exposes more execution responsibility and requires clear ownership of
capture, observers, admission release, and prepared-handle cleanup. Evaluate
that option only if it is simpler than the narrow controls above.
PromptKit should continue owning history assembly, turn progression, execution,
and observations. Audita retains response interpretation, validator calls,
branch policy, diagnostic storage, and application error classification. Existing
request observers, coordinator observers, and execution groups must remain
usable together. This request does not reopen the accepted conversation history,
final-turn delivery, or grouping contracts.
## Acceptance scenarios
1. Two successful continuation operations each finish within their own timeout,
while their combined duration exceeds one operation's budget. Both succeed
under an active parent context.
2. Preparation or admission waiting consumes the same budget as generation and
structural repair. Local expiry returns a recognizable deadline error and
stops that coordinator without canceling its parent or a sibling branch.
3. An earlier parent deadline remains effective. A validator called from the
next-turn callback runs after admission is released, including with backend
capacity one, under its own application-selected deadline.
4. The application receives the prepared continuation details before generation.
A later provider failure returns no result, but the retained details still
describe that failed operation. Mutating a details copy does not change the
messages executed.
5. Diagnostic access uses the same prepared snapshot that executes, including
when source inputs change afterward. It does not trigger another preparation
or provider call.
6. Existing observers and selected execution groups continue receiving ordinary
execution facts once per their existing contracts, including failure paths.
Applications that select neither new capability retain existing behavior.
The consumer implementation motivating this request is Audita's
`internal/framework/llm/adapter.go`, especially `RoleClient.CompleteStructured`.
This gap affects full coordinator adoption; it does not block adopting v0.22.0
capture, continuation, or execution groups independently.

291
docs/roadmap/execution-controls.md
View File

@@ -1,291 +0,0 @@
# Execution Controls For Coordinated Turns
Status: all four design decisions accepted. This document defines the target
state from v0.22.0; [implementation.md](implementation.md) selects the concrete
API and staged implementation work. The feature is not yet implemented.
## Purpose
Enable applications using `Engine.Coordinate` to select a fresh timeout for each
continuation and retain the exact prepared request before generation. These are
execution capabilities useful for bounded operations and optional diagnostics,
without assumptions about prompt content or application workflows.
[Audita's request](audita-request.md) supplies the motivating requirements.
Audita already uses one child context across explicit preparation and execution,
and retains prepared details for diagnostics. The goal is to make those controls
available during coordinator-owned execution without requiring a downstream
replacement for PromptKit's turn loop.
PromptKit owns history assembly, execution, cancellation propagation, and
observations. Applications select budgets, interpret results, run validators,
schedule branches, and own diagnostic destinations, disclosure policy, and write
failure handling. This work adds no artifact store or reporting framework.
## Current Baseline
[PromptKit v0.22.0](../releases/v0.22.0.md) supplies captured exchanges,
continuation, sequential coordination, and explicit execution groups.
- The [coordinator](../../coordinate.go) attaches the current exchange and composes
request/coordinator observers, then calls `Engine.Run` with the parent context.
Its callback returns a request, not an execution context or prepared handle.
- Ordinary [run execution](../../internal/usecase/runner.go) resolves the target,
obtains logical admission, completes preparation, and generates and validates
the response. Admission is released before the next-turn callback.
- Explicit [prepared execution](../../prepared_execution.go) already supplies
detached `Details()` and executes the retained snapshot. Preparation and
execution are separate calls; preparation does not reserve backend capacity.
- Ordinary [observations](../internal/observation.md) are metadata-only. Ordinary
runs observe preparation failures; split preparation itself emits no execution
events. A claimed prepared handle starts its observation lifecycle at execution.
A timeout around the entire coordinator includes all turns and application
callbacks. Provider timeouts cover a different scope. Neither provides a fresh
whole-operation budget for each selected turn. Separately calling `Prepare`
would duplicate preparation and would not bind the inspected details to the
request actually executed. An exchange cannot supply a failed turn's prepared
request when execution returns no result.
## Target End State
The selected approach is optional per-request controls honored by ordinary `Run`
and therefore by `Coordinate`. The accepted decisions below specify their
lifecycle; the coordinator does not delegate execution to an application executor.
### Optional Operation Timeout
An application can select a duration for each requested continuation. When that
continuation starts, establish a fresh child deadline before preparation and use
it for the entire operation: preparation, admission waiting, retries, generation,
structural validation, and structural repair, including the prepared-details
notification. Do not restart the budget between phases or attempts.
The selected timeout covers work according to the entry point:
| Entry point | Selected timeout behavior |
| --- | --- |
| `Run`, including coordinated turns | Starts before preparation and covers preparation and execution. |
| `PrepareExecution` | Retains the duration for later execution; preparation uses its supplied context. |
| `RunPrepared` | Starts the retained duration when invoked, covering the remaining execution. |
Idle time between split preparation and execution does not consume the retained
execution budget. Applications wanting one budget across both calls can pass the
same caller-created deadline context through them. `Prepare` and `ValidateRequest`
continue using their supplied contexts without starting the execution timeout.
Respect parent cancellation and earlier parent deadlines. A local timeout ends
the affected operation and coordination under the existing operational-error
rule; it does not cancel the parent or siblings. Release child-context resources
before returning to the application's next-turn callback. Assessment calls from
that callback retain their own application-selected contexts and budgets.
Determine the execution result and any local-timeout identity before publishing
the terminal observation, so the returned error and observed outcome agree.
A slow terminal observer or subsequent child-context cleanup must not
retroactively turn a completed operation into a timeout. Preserve the existing
coordinator behavior for completed responses, parent cancellation before
next-turn callback delivery, and the single undelivered response. No automatic
replay or new recovery policy is added.
No selected timeout means existing context behavior. This adds no framework-wide
recommended duration or mandatory cap. Additional-turn, transport-retry,
structural-repair, and provider-generation budgets keep their existing meanings.
Cancellation remains cooperative; a deadline does not forcibly terminate
application callbacks or injected collaborators that ignore their context, or
guarantee that the public call returns by that deadline.
### Opt-In Prepared Details Before Generation
Provide explicitly selected access to caller-owned details of the logical request
selected for execution, after successful preparation and before its first
generation attempt. Reuse the existing prepared-details representation and
credential redaction where practical. Include assembled captured history, rendered
follow-up messages, caller-appended messages, and ordinary resolved execution
metadata.
Use the execution's prepared snapshot rather than rendering or reading inputs
again. Retaining or modifying delivered details must not mutate the execution.
Once delivered, details remain available to the application after provider failure
or cancellation even when no ordinary result or exchange is returned. Diagnostic
access is available only when execution reaches the prepared-notification point,
not for every failed operation. Preparation failure requires no complete snapshot;
ordinary admission failure or timeout may occur before preparation completes.
Do not add rendering or rearrange admission solely to manufacture diagnostics for
those failures. Cancellation before notification does not promise delivery;
follow the context checks in [Decision 3](#3-what-is-the-prepared-details-callback-contract).
This access exposes logical prepared content, not raw HTTP envelopes or internal
structural-repair requests. Keep it separate from metadata-only observers and
execution-group summaries. It is not another next-turn callback: ordinary
execution may already hold backend admission at the notification point.
### Compatibility And Composition
Applications selecting neither capability retain existing behavior. Preserve
conversation assembly, capture, final-turn delivery, branching, and grouping
contracts. Request observers, coordinator observers, and explicitly selected
groups continue receiving ordinary execution facts without duplicate lifecycle
events or counting attempts twice.
Preserve ordinary admission, preparation-failure observation, and ownership
behavior. A simple replacement of ordinary `Run` with the public
`PrepareExecution`/`RunPrepared` sequence would change those behaviors. This does
not freeze the internal call structure: consolidation or refactoring is welcome
where it preserves the selected lifecycle.
Document which controls each entry point applies or retains. Execution-only
controls need not take effect during `Prepare` or `ValidateRequest`; accepting
request-shaped input there does not itself justify rejecting such controls or
starting a timer. Follow the accepted split-preparation and callback semantics.
## Affected Areas
The target spans request controls, execution-context lifetime, detached prepared
details, public error projection, and ordinary coordinator integration. It is an
additive runtime feature; no collection-format or ConfigKit changes are needed.
Public GoDoc will own exact API contracts, the
[consumer guide](../consumers/pkg-promptkit.md) will explain use, and internal
lifecycle documentation will describe the implementation when it lands.
The implementation plan owns API selections, task sequencing, concrete changes,
and validation steps. Keep that work proportional to the two requested controls.
## Validation Targets
Use offline public-boundary tests and existing execution collaborators, with
controlled cancellation and synchronization rather than fragile timing margins.
Cover the consumer request's [acceptance scenarios](audita-request.md#acceptance-scenarios)
and the accepted lifecycle choices:
- Successive turns receive fresh budgets: their combined duration may exceed one
operation budget without exhausting either individual operation.
- For ordinary and coordinated runs, preparation, admission waiting, retries,
validation, and repair share the same operation deadline. Local expiry stops
coordination without canceling parent or sibling work; earlier parent
cancellation/deadlines remain effective.
- Next-turn assessment uses the parent context after release, including another
call to a backend with capacity one. Timeout identity survives public wrapping.
- A slow terminal observer or child-context cleanup does not change a completed
outcome into a local timeout; the returned error and terminal observation agree.
- Prepared details precede generation, contain the assembled continuation, and
remain available after later failure. Mutation of the delivered copy or source
changes after preparation do not change the executed snapshot.
- Diagnostic access introduces no additional preparation or inference calls,
notification per internal repair, or duplicate observer lifecycle. Ordinary
retries and structural repair remain allowed. Credential redaction remains
intact.
- Admission or preparation failure before the notification point does not require
diagnostic delivery or additional rendering to produce a snapshot.
- Preparation failure, cancellation around notification, and callback panic follow
the selected delivery rules and do not strand admission or retained resources.
- Split preparation uses its supplied context and retains the selected duration
for `RunPrepared`; idle time does not consume that execution budget. A shared
caller deadline can still bound both calls. Neither discarded nor unsuccessfully
claimed handles start execution notifications.
- The prepared notification receives detached details and the operation context,
has no error return, and runs synchronously within the operation budget.
- Local-budget expiry has a narrow inspectable identity while preserving standard
context matching; parent and provider deadlines are not mislabeled as local.
- Observers/groups retain ordinary facts on success and failure. No-option use,
completed-result delivery, and final-turn behavior remain compatible.
## Accepted Decisions
All four decisions below are accepted. No significant product decisions remain
open. The implementation plan selects public names, signatures, timeout-value
conventions, and the Go representation of timeout identity.
### 1. Narrow Execution Controls Or Caller-Delegated Execution?
**Accepted approach:** add optional per-request execution controls honored by
ordinary `Run` and thus by `Coordinate`, including a timeout and
prepared-details callback. This keeps history, capture, admission, errors, and
observers in their existing owners and makes the capability useful outside
coordination. A caller can select different controls on each returned request;
an additional coordinator-wide configuration layer is not needed for the
demonstrated use case.
**Alternative considered, not selected:** let the coordinator delegate a turn
to a caller-provided executor or prepared-execution adapter. This would reuse
Audita's existing adapter and accommodate other execution customizations. It
also requires a larger contract for exchange assembly, capture, observer
composition, handle cleanup, and admission release. The narrow controls meet
the demonstrated need with fewer ownership questions; caller-delegated
execution is not part of this feature.
### 2. What Does A Timeout Mean For Split Preparation And Execution?
**Accepted approach:** bound each `Run` invocation from before preparation. For
`PrepareExecution`, retain the selected duration as an execution control and
apply it when `RunPrepared` starts; preparation itself continues using its
supplied context. Do not retain a live timer across an idle prepared handle.
Explicitly explain that this bounds the remaining execution, not preparation
already done. Callers needing one total budget across both calls can continue
passing the same caller-created deadline context, as Audita does today.
This preserves the distinction between executing now and preparing for later,
while letting a prepared handle carry the same selected execution controls. It
requires clear documentation: the duration alone does not retroactively cover
split preparation. `Prepare` and `ValidateRequest` likewise do not become timed
executions merely because they accept request-shaped input.
**Alternative considered, not selected:** start a budget during
`PrepareExecution` and carry its remaining deadline into `RunPrepared`. This
directly bounds preparation and execution together without caller context
plumbing. It also counts idle time, requires timer/context ownership and
discard semantics, and changes what a prepared handle retains. This is useful
if a consumer needs a whole prepared operation lifetime, but exceeds the
current coordinated-turn requirement.
### 3. What Is The Prepared-Details Callback Contract?
**Accepted approach:** a synchronous notification receiving the operation
context and a detached prepared value, with no error return. Invoke it once
after the snapshot is ready and before generation, only when execution proceeds
to that point. For prepared handles, retain the callback until execution and
clear it on claim/discard like the existing observer; inspection remains
available earlier through `Details()` without triggering the notification.
Check context before notification and again before generation. Include callback
time in the operation budget and retain ordinary cooperative cancellation and
panic propagation, with resource cleanup on unwind. Do not add background delivery
or stronger terminal-event guarantees. The application can retain details and
perform optional writes under its own policy; it need not turn a diagnostic write
failure into an inference failure.
Under the current ordinary run path this notification executes while logical
admission is held. Document that fact rather than promise the next-turn callback's
capacity-one nested-execution behavior. Do not prohibit arbitrary callback work;
applications must account for the held capacity and operation budget. Releasing
and reacquiring admission solely for notification is not necessary for this scope.
**Alternative considered, not selected:** let the callback return an error to
veto generation. This supports mandatory audit persistence or an application
pre-generation check, with ordinary cleanup and error propagation. It also
makes the callback an execution gate and requires failure classification
semantics absent from the optional-diagnostics request. The selected
notification satisfies that request; a generation veto would require a separate
demonstrated need.
### 4. How Precisely Should Local Timeout Errors Be Identified?
**Accepted approach:** preserve standard context matching and provide a narrow,
inspectable operation-timeout identity when this newly selected budget causes
termination. Keep parent cancellation/deadlines and provider-local failures
under their existing identities. The marker must survive public wrapping and
must not be inferred from any arbitrary `DeadlineExceeded` returned by a
collaborator. Its Go representation is selected in the implementation plan;
no new error-report hierarchy or terminal-event category is implied.
This lets callers distinguish the requested local budget from an earlier parent
or provider deadline without retaining the internal child context. Define cause
selection around races using the child context's recorded cause; do not promise
perfect ordering of independently simultaneous cancellations.
**Alternative considered, not selected:** return only existing context errors
and let callers inspect their parent context. This is smaller and often
sufficient when the parent remains active. However, `DeadlineExceeded` alone
may also originate in a provider timeout, and a parent may become canceled
before the caller inspects it. That weaker attribution does not meet the
selected distinction as directly as a narrow operation-timeout identity.

457
docs/roadmap/implementation.md
View File

@@ -1,457 +0,0 @@
# Execution Controls Implementation Plan
## Objective And Sprint Scope
Implement the accepted target in [execution-controls.md](execution-controls.md)
from the v0.22.0 execution architecture. The feature roadmap owns intent and the
four accepted lifecycle decisions; this document selects the API and the work
needed to implement them. [Audita's request](audita-request.md) supplies the
consumer scenarios.
Work through Stage 1 through Stage 8 in order. Each stage is scoped for one Luna
coding-agent prompt. This is one end-to-end sprint: intermediate stages may
temporarily fail to compile or leave wiring incomplete. Do not build temporary
compatibility layers merely to make intermediate commits green. There are no
intermediate human-review or external-publication gates. Completion requires
the assembled feature and its validation, not just completion of individual
stages.
Read [development.md](../development.md), the [agent policy](../policy/agents.md),
and the applicable architecture, testing, and documentation policies before
work. Preserve unrelated changes. Follow the existing commit policy for finished
units; report intermediate validation honestly. This plan authorizes no release,
push, dependency update, configuration-format change, or new consumer workflow
beyond the execution controls described here.
## Selected API And Semantics
Add these fields to the existing public `RunRequest`, without introducing an
execution-options object, coordinator executor, or general hook framework:
```go
OperationTimeout time.Duration `json:"-" yaml:"-"`
OnPrepared func(context.Context, PreparedRun) `json:"-" yaml:"-"`
```
Add one public sentinel, alongside the current execution errors:
```go
var ErrOperationTimeout = errors.New("operation timeout exceeded")
```
The spelling of an error message is not a contract. No exported timeout error
type, new observer event, or new failure-category enum is needed.
### Entry-Point Contract
| Entry point | OperationTimeout | OnPrepared |
| --- | --- | --- |
| `Run` | A positive value starts a fresh child budget before preparation, covering preparation and execution. | Notify once if execution reaches the prepared point before first generation. |
| `Coordinate` | Each returned request receives ordinary `Run` semantics with a fresh scope. | Ordinary per-request notification; no coordinator-specific copy of the option. |
| `PrepareExecution` | Retain the value privately, without starting a timer. Preparation uses its supplied context. | Retain the selected function privately without invoking it. |
| `RunPrepared` | Start the retained positive duration after the successful claim, before execution callbacks, credential revalidation, and admission. | Notify once when the claimed execution reaches the prepared point. |
| `Prepare`, `ValidateRequest` | Do not start a timer; use the supplied context as today. | Do not invoke or retain the notification. |
| `PreparedExecution.Details`, `Discard` | Do not start a timer. | Do not invoke the notification. Discard drops its retained reference. |
- A positive duration opts in. Zero and negative durations select no additional
timeout. This follows a nonpositive-means-unselected convention, avoids a new
request rejection rule, and leaves the parent deadline effective. There is no
default operation duration, upper bound, profile setting, or engine setting.
- The provider's existing generation timeout remains independent. Do not copy
`OperationTimeout` into `ExecutionTarget`, provider JSON, or prepared/result
metadata. Neither control changes prompt hashes or configuration inspection.
- The budget is continuous across preparation, capacity waiting, notification,
generation, transient retries/backoff, validation, and structural repair. It
is not restarted between phases, attempts, or repair generations.
- Split preparation retains a duration, not a context, deadline, timer, or
cancel function. Idle time does not count. A shared caller-created deadline
remains the way to bound preparation and later execution together.
- A nil notification selects no diagnostic copying. A selected function receives
the operation context and a detached, credential-redacted `PreparedRun` value
representing the logical request. Its nested mutable values belong to the
application. Reuse the existing projection used by `Details()`.
- Notification is synchronous, has no error return, and runs after successful
preparation and admission, before the first generation. Logical admission is
still held. Its time consumes the selected operation budget. It may retain
details for later diagnostics; PromptKit does not own writes or write errors.
- Check context before notification and again before generation after it
returns. A callback may block or ignore context; do not introduce a goroutine
to interrupt it. Callback panic propagates after deferred resource cleanup,
without a new terminal-event guarantee.
- Callback selection is copied as a function value, not as a deep copy of its
closure. Shared callback state is application-synchronized; separate runs can
invoke a common callback concurrently. Later reassignment of the original
request field cannot replace a prepared handle's retained function.
- Notification is conditional on reaching that execution point. Admission or
preparation failure can prevent it. Do not render again, reorder admission,
or promise a diagnostic snapshot for every failed operation. Split callers
can still inspect a successfully prepared handle through `Details()`.
### Timeout Attribution And Completion
Use the standard library's `context.WithTimeoutCause` with a distinct cause
value for each selected scope. Record the scope's own cause identity so an
inherited timeout from an enclosing operation cannot be mistaken for this
invocation's timer. Do not classify by error text or by `DeadlineExceeded`
alone. No strict ordering of independently racing cancellations is promised.
When an execution fails through its context deadline and the child context's
recorded cause is this scope's own timer cause, return an error matching both
`ErrOperationTimeout` and `context.DeadlineExceeded`. Preserve existing operation
and collaborator error identities. An earlier parent deadline/cancellation or
a provider deadline must not acquire the local marker merely because it is a
deadline error. An unrelated failure must not acquire it merely because the
timer has also expired. Parent-provided error causes, if any, retain their
existing behavior; this feature need not reinterpret them.
Finalize attribution from the operation's returned error before canceling its
child scope and before terminal observer delivery. A nil execution error remains
a completed result, including a completed structural rejection. Do not add a
post-terminal `ctx.Err()` check that converts it into an error. Slow terminal
observers and deferred context cleanup cannot change the chosen outcome. This
does not strengthen cancellation into forced interruption or a guarantee that
every context-ignoring collaborator will return by the deadline.
The terminal event continues using the existing deadline failure category for
a timeout error. Groups continue aggregating those ordinary terminal facts.
The child scope is cleaned up before `Run`/`RunPrepared` returns to the next-turn
callback; that callback receives the coordinator parent context, not the child.
Existing parent cancellation and undelivered-result rules remain unchanged.
## Implementation Map
Use existing owners rather than adding a new public package:
- Public declarations and contract: [types.go](../../types.go),
[engine.go](../../engine.go), and [prepared_execution.go](../../prepared_execution.go).
- Public conversion and error projection: [convert.go](../../convert.go),
[errors.go](../../errors.go), and [observations.go](../../observations.go).
- Internal request values: [domain.go](../../internal/domain/domain.go).
- Timeout lifecycle and notification orchestration: `internal/usecase`, chiefly
[runner.go](../../internal/usecase/runner.go) and
[prepared_execution.go](../../internal/usecase/prepared_execution.go).
- Existing private operation-error representation:
[operation_error.go](../../internal/usecase/operation_error.go).
- Ordinary coordinator integration: [coordinate.go](../../coordinate.go) and
[internal coordinator](../../internal/usecase/coordinate.go).
The root remains a facade. Keep timer construction and cause-attribution logic
in a small internal helper; the facade may establish that scope and project its
errors while managing its existing observation lifecycle. Internal packages must
not import the root package or depend on the public `PreparedRun` representation.
Private helper names and equivalent internal organization may change; the
selected public behavior and ownership boundaries are what matter.
## Stage 1: Add The Request Contract And Conversion
Status: complete.
Outcome: the selected API and internal request plumbing exist, ready for the
execution stages to consume.
1. Add `OperationTimeout`, `OnPrepared`, and `ErrOperationTimeout` to the public
declarations above. Write their GoDoc from the entry-point and ownership
contract in this plan, explicitly separating operation and provider timeouts.
2. Extend `domain.RunRequest` with the duration and an internal callback of shape
`func(context.Context, *domain.PreparedRun)`. Document the callback argument
as borrowed read-only internal state. It is not a public mutable snapshot.
3. Extend `toDomainRunRequest` to copy the duration and capture the selected
public callback function in a local variable. Only when it is non-nil, adapt
it by projecting the supplied prepared state through `fromDomainPreparedRun`
and passing the resulting value. This projection is the public ownership
boundary; do not deep-copy the domain snapshot first as well.
4. Keep these fields out of public prepared/result JSON and opaque handle
details. Request conversion does not start a timer, invoke a callback, or
reject an execution-only control on preparation-only entry points.
5. Add the private timeout error identity in `internal/usecase`, using the same
existing separation between internal and public sentinels. Add its public
projection to `errors.go` when introducing the attribution wrapper in Stage 2.
Validation: focused conversion tests should establish option forwarding and
conditional adaptation. Reuse existing nested-value projection tests instead of
retesting every existing `PreparedRun` field. Compilation may remain incomplete
until the execution wiring lands; do not claim the feature works at this stage.
## Stage 2: Bound Ordinary Execution And Preserve Its Error Outcome
Status: complete.
Outcome: `Engine.Run` owns one continuous optional budget and returns the same
failure classification it publishes to observers.
1. Add a small internal operation-scope helper using `WithTimeoutCause`. For
nonpositive durations, reuse the parent context without a timer. For positive
durations, retain a fresh per-invocation cause token and a cancel function.
Use a real distinct identity, not a shared sentinel or zero-sized token whose
pointer identity cannot distinguish scopes.
2. Give the helper an error-finalization operation: for a non-nil returned error,
require both a match to the operation context's deadline error and equality
of its recorded cause with this scope's token before adding the private
timeout identity. Leave nil errors and unrelated errors unchanged. Preserve
the original error as a cause.
3. Reuse `usecase.OperationError` with the private timeout category to carry the
marker around the existing error, rather than creating a public error
hierarchy. Project that category to `ErrOperationTimeout` through the
existing `publicOperationError` machinery. Keep the original phase category
and context cause inspectable through the nested chain.
4. Establish the ordinary scope in the facade's `Run` entry, after masking
inherited execution-observation state and before start-event delivery and
request preparation. Both observed and unobserved runs use it. Pass its
context through request conversion, internal engine delegation, admission,
preparation, and all generation/validation work. Do not also start a second
timer inside `Runner.Run`.
5. Adjust the private facade `run` path as needed to finalize the execution
error before applying public error projection once. On normal return, freeze
attribution and the returned result/error, clean up the child, then classify
and deliver the existing terminal event. Preserve normal observation of
request conversion and preparation failures. Handle all early returns; install
cleanup before any user callback can panic.
6. Do not change provider timeouts, retry eligibility, repair limits, scheduling,
or existing result timing conventions to implement this feature. A timer
expiring while terminal observation runs is not another execution failure.
Validation: narrow helper tests cover disabled scopes, local expiry, parent
deadline/cancellation, inherited operation causes, unrelated errors, and nil
completion. Add public `Run` tests proving the marker survives phase wrapping
and agrees with an ordinary deadline terminal event. Provider-only deadline
errors while the operation child is active must not gain the marker.
## Stage 3: Retain Controls Across Split Preparation
Status: complete.
Outcome: prepared execution carries the selected duration and notification,
without keeping a live operation scope while idle.
1. Extend `preparedExecutionPayload` with the duration and internal notification.
`PrepareExecution` copies them from the converted request only after its
ordinary successful preparation. Preparation itself uses its supplied
context, without starting the selected timer or calling the notification.
2. Extend payload clearing for both fields. Preserve the existing atomic claim,
owner checks, foreign/reused-handle behavior, and durable redacted details.
A discard or completed/unwound claim drops PromptKit's callback reference.
3. After a successful `Runner.RunPrepared` claim, establish the scope with the
retained duration, before the retained observer's start delivery, credential
revalidation, and admission. Defer payload and scope cleanup before callback
work. A failed claim neither starts notification nor changes existing silent
invalid-handle behavior.
4. Pass the child through execution and finalize the returned error with the
same helper before cleanup and facade terminal classification. Keep this
lifecycle in the internal prepared runner rather than borrowing duration or
callback state from an unclaimed handle in the facade.
5. `RunPrepared`'s facade projects the resulting error and emits its one existing
terminal event after resource release. Do not reapply the timer at the facade
or retain the preparation context. Namespace views keep their shared-owner
behavior and use the handle's original selections.
Validation: extend prepared-execution coverage to show that preparation and an
idle delay longer than the selected duration do not consume its execution
budget; later execution does. Reassignment of the original request does not
change the retained duration/callback. Cover earlier caller deadlines and a
shared caller deadline across preparation and execution. Existing claim-race,
discard, foreign-owner, credential, and error-identity tests remain applicable.
Notification invocation itself is completed in Stage 4.
## Stage 4: Notify From The Actual Shared Prepared Snapshot
Status: complete.
Outcome: ordinary and prepared execution expose one detached logical request
before first generation, without repeating preparation.
1. Thread the internal notification from `Runner.Run` and the claimed payload
into the existing shared `executePreparedRun` entry. At that entry, preparation
is complete and logical admission is held on both paths.
2. When selected, check the operation context, invoke the internal adapter with
the actual retained prepared state, and check context again before entering
generation. A context failure at this boundary follows the existing
generation operation-error wrapping, preserving context identity and gaining
local-timeout identity only through the scope finalizer.
3. Invoke once per executing logical request, not per transport retry or repair.
Do not call `Prepare`, `Details`, artifact readers, renderers, or schema
preparation again to produce the notification. With nil `OnPrepared`, avoid
constructing a public details copy.
4. Keep the prepared state's API key redacted. Generation still receives its
private direct credential through the existing separate path. Preserve the
complete assembled messages, schema, settings, input hashes, session,
ancestry, and preparation metadata in the delivered public value.
5. Leave logical admission held; do not release/reacquire it around notification.
Invoke application code without holding the handle mutex or a capacity
manager lock. Preserve panic propagation while defers release the lease,
timer, and claimed payload. A panic does not make a handle reusable.
Validation: one ordinary and one split public test establish ordering before
generation and exact callback selection. Add focused context-before/after
notification tests and an internal cleanup regression if the public capacity
probe alone cannot demonstrate payload cleanup. More extensive ownership and
failure assertions belong to Stage 6, not duplicate tests here.
## Stage 5: Verify Continuous Budgets Across Execution Phases
Status: complete.
Outcome: tests demonstrate that one scope reaches all relevant collaborators,
with correct attribution and observation on failure.
1. Add focused tests for preparation expiry through a context-aware injected
artifact reader, generation-capacity waiting on a real limited backend,
interrupted built-in retry backoff, and structural repair expiry. Verify
standard context matching, local marker, and retained phase identity where
that phase already supplies one. Cover ordinary and split paths without
mechanically duplicating every case on both.
2. Use a narrow internal runner test for validation-phase expiry, since validation
collaborators are not a public injection API. Check the same deadline reaches
preparation, validation, and generation; do not add a new public testing seam.
3. Test that notification time consumes the same budget and that expiry there
prevents generation. Cover an earlier parent deadline and parent cancellation
separately from a provider's own deadline. Preserve original injected error
identities and do not relabel a non-context error after timer expiry.
4. On both ordinary and split public execution, exercise a terminal observer
that outlasts the operation budget after successful completion. The result
stays successful and the terminal remains successful. Include the timeout
failure case to ensure public error mapping happens before terminal delivery.
5. After expiry, prove capacity remains usable and the caller parent stays
active. Respect the current capacity model: logical admission may reject
immediately, while admitted work can wait for an active generation permit.
Do not introduce a new blocking admission queue to satisfy a test label.
Use channels for ordering and `testing/synctest` for controlled elapsed-time
scenarios where its bubble rules apply. This repository's Go minimum supports
it. Prefer in-memory fixtures and controlled transports; avoid real short sleeps
or close-to-deadline assertions. Reuse the existing generation tests for retry
policy rather than building a second retry matrix. There is no production clock
framework, live provider probe, allocation threshold, or performance gate here.
Validation: run focused use-case/generation tests and the new public execution
control tests under the race detector. The new cases should protect deadline
propagation and classification, not private helper call sequences.
## Stage 6: Verify Notification Ownership And Failure Lifecycles
Status: complete.
Outcome: applications can safely retain diagnostic details without altering
execution or acquiring new lifecycle guarantees.
1. Add a representative continuation request containing history, follow-up
messages, appended cache-control metadata, input hashes, session, settings
with nested provider values, and a structured schema. Inside `OnPrepared`,
retain and mutate the delivered mutable values; the model request, captured
exchange, and later prepared-handle `Details()` remain unchanged. Use
synthetic direct/environment credentials and verify their values are absent
from diagnostic details while the existing client credential path still works.
2. Demonstrate snapshot fidelity when mutable input/schema sources change after
preparation and before execution. Reuse existing frozen-preparation fixtures;
notification must describe the executed snapshot, not reread the changed
sources. A later provider failure or cancellation may return no result, while
the application still retains the delivered details.
3. Verify absence of notification for preparation failure and ordinary admission
rejection, without requiring extra reads to produce diagnostics. For split
execution cover invalid/foreign/discarded/reused handles and pre-notification
cancellation. `Details()`, `Prepare`, `ValidateRequest`, and preparation of an
executable handle never notify or start this execution timer.
4. Verify that retries and structural repair do not repeat notification. Reuse
an existing retry-plus-repair integration scenario with an added notification
assertion rather than duplicating its full event/usage test elsewhere.
5. Recover a notification panic only in test code, then verify ordinary capacity
is reusable and a claimed handle stays consumed. Do not require a terminal
event after panic or impose a prohibition on callback reentrancy. A capacity
one probe documents held admission rather than promising nested success.
Validation: run affected public contract and private payload-cleanup tests under
the race detector. Mutable callback state in concurrent tests must be protected
by the test; do not make PromptKit serialize application callbacks globally.
## Stage 7: Verify Coordinated Turns And Observer Composition
Status: complete.
Outcome: coordination obtains the controls through ordinary `Run`, retaining
its existing branching and delivery semantics.
1. Check the coordinator's existing request-value copy preserves both new
controls while it replaces `ContinueFrom` and composes observers. No separate
coordinator timeout, executor, or public prepare/run sequence is needed.
2. Add a controlled-time, two-continuation success case. Each turn finishes
within its own timeout while their combined execution exceeds one turn's
duration. Callback details contain each turn's assembled history; no timer
from an earlier turn controls a later turn.
3. In the next-turn callback, verify the parent context is active and the prior
child context has been cleaned up. Run an explicitly selected assessment on
the same backend with concurrency one and queue capacity zero, using its own
timeout. This differs from `OnPrepared`, where admission is still held.
4. Exercise a locally timed-out continuation: coordination returns its ordinary
operational error with the marker, does not fabricate an outcome or completed
exchange, and leaves the parent and a sibling continuation usable. Retained
details are available if notification occurred before failure. Do not promise
them if execution stopped earlier.
5. Compose request and coordinator observers with an explicit execution group.
Check one ordinary lifecycle per selected run, normal terminal usage/attempt
totals, and no implicit membership for assessment calls. Reuse existing
final-turn and parent-cancellation/undelivered-result cases to verify that
cleanup and slow terminal observation have not changed delivery semantics.
Validation: run the coordinator, observation, group, and new execution-control
contract cases together under the race detector. Keep tests at the public
boundary for the assembled behavior; the internal coordinator loop need not
be redesigned solely because its ordinary run gained options.
## Stage 8: Document, Demonstrate, And Validate The Assembled Feature
Status: complete.
Outcome: the implementation and its supported consumer workflow are complete,
with documentation in the canonical owners.
1. Finish GoDoc on `RunRequest`, `ErrOperationTimeout`, `Run`, `Prepare`,
`ValidateRequest`, `PrepareExecution`, `RunPrepared`, and `PreparedExecution`.
Cross-reference the owning field/error contract rather than duplicating its
entire definition. Explain conditional notification, held admission,
cooperative cancellation, split preparation, callback ownership, and outcome
finalization before terminal observation.
2. Update the [consumer guide](../consumers/pkg-promptkit.md) with small ordinary,
coordinated, and split usage sketches linked to GoDoc. Show a caller-owned
retained `PreparedRun` on failure, `errors.Is` checks for both timeout
identities, and a separate budget for callback-owned assessment. Do not add
persistence, log destinations, redaction policy, or a mandatory diagnostic
write gate.
3. Extend the maintained [observation example](../../examples/go-library/observe/main.go)
and its semantic test with a positive operation budget on continuation,
selected prepared-details retention, and a separately budgeted next-turn
assessment. Keep it offline, synthetic, and successful; Stage 5/6 tests own
failure demonstrations. Do not add another standalone example framework.
4. Update [runner documentation](../internal/runner.md),
[observation documentation](../internal/observation.md), and the
[component inventory](../internal/overview.md) only where responsibilities or
lifecycle descriptions changed. No ConfigKit, collection-format, provider
payload, or architectural policy change is required by this work.
5. Review Audita's six acceptance scenarios and the feature roadmap's validation
targets against the completed tests. Consolidate duplicate tests; do not
expand this into a full cross-product of all entry points, phases, observers,
and callback behaviors. Retain existing no-option regression coverage.
6. Select the newest installed Go toolchain per the development guide. Format
changed Go files, run `go test -race ./...` and `go vet ./...` with
`GOTOOLCHAIN=local` and `GOWORK=off`, and run `git diff --check`. The full Go
suite includes the maintained examples; no second example execution gate is
needed. Use writable caches as required by the development environment.
Validate local links and inspect the full diff for unintended files/secrets.
Do not edit published v0.22.0 release notes, create a new release/tag, or require
CI or external publication to finish the sprint. Release preparation and
retirement of these roadmaps are subsequent work.
## Completion Criteria
- Ordinary and coordinated runs apply one optional whole-operation budget;
split handles retain duration and callback selection without live idle timers.
- Deadline attribution survives public wrapping, distinguishes this scope's
recorded timer cause, and agrees with terminal observation before cleanup.
- Notification supplies a detached redacted view of the actual logical prepared
request, once before generation when reached, without repeated preparation.
- Failure, cancellation, and panic paths preserve existing resource, handle,
error, observation, grouping, and continuation behavior.
- The maintained example, canonical documentation, focused tests, full race
suite, and vet are complete. No-option use stays compatible.
No significant open questions remain. The API and nonpositive-duration choices
above complete the planning decisions left by the accepted feature roadmap.

72
docs/roadmap/remediation.md
View File

@@ -1,72 +0,0 @@
# Execution Controls Review
Reviewed implementation stages 1–8 against commit `57b62d3`, the
[implementation plan](implementation.md), the accepted
[feature roadmap](execution-controls.md), and the applicable architecture,
testing, documentation, and agent policies.
## 1. Complete The Public Prepared-Notification Contract
Priority: P3. Status: complete. Owning stage: Stage 8.
Location: [RunRequest.OnPrepared GoDoc](../../types.go#L97).
The public GoDoc describes synchronous delivery, detached ownership, held
admission, and split preparation, but omits two explicitly selected lifecycle
facts:
- Notification occurs at most once per logical execution, when execution
reaches that point before the first generation. Transient retries and
structural repair do not repeat it.
- The callback receives the operation context, including the selected operation
deadline when enabled and inherited parent cancellation. In coordination this
is distinct from the parent context delivered to the next-turn callback.
These facts are already implemented by the
[shared execution boundary](../../internal/usecase/runner.go#L186) and covered
by the notification, retry/repair, and coordinator contract tests. They are also
expressly selected in the roadmap's
[accepted callback decision](execution-controls.md#3-what-is-the-prepared-details-callback-contract).
However, Go declarations and GoDoc own the exact public API contract; consumers
should not need a temporary roadmap or implementation inspection to discover
invocation cardinality or the scope of their callback's context.
Required remediation: add these two facts to the field's GoDoc. Preserve
conditional delivery and cooperative cancellation; do not promise notification
for every failed request, forced callback interruption, or terminal observation
after panic. This is a documentation completion, not a runtime change. Existing
behavioral tests already cover these semantics; duplicate tests are unnecessary.
Completion: expanded the field's GoDoc to state invocation cardinality,
non-repetition during retry/repair, and operation-context scope, including the
distinction from the coordinator's next-turn callback context. Runtime behavior
is unchanged. GoDoc rendering and the existing focused race-enabled notification,
retry/repair, and coordinated-budget tests passed.
## Stage Review Summary
| Stage | Result |
| --- | --- |
| 1 — API and conversion | Selected fields and sentinel are present; callback conversion captures the chosen function and detaches details only when selected. |
| 2 — Ordinary timeout | One internal scope covers ordinary execution; per-invocation cause attribution and public error projection precede terminal observation. |
| 3 — Split preparation | Handles retain duration and callback without an idle timer; claim/discard and cleanup behavior are preserved. |
| 4 — Prepared notification | Both execution paths use the actual shared snapshot, with context checks and held admission; retries and repair do not notify again. |
| 5 — Phase budgets | Tests cover preparation, capacity waiting, retry backoff, generation/repair, validation, notification time, and stable terminal outcomes. |
| 6 — Ownership and failures | Public tests cover detached messages, cache metadata, nested parameters, schema, credentials, frozen sources, retained diagnostics, and panic cleanup. |
| 7 — Coordination | Fresh turn budgets, parent-context assessment, released capacity, sibling use, observer composition, and explicit grouping are covered. |
| 8 — Documentation and example | Maintained example, consumer/internal documentation, and the completed canonical GoDoc reflect the feature; finding 1 is resolved. |
No runtime correctness defect, additional architectural gap, or inadvertently
strengthened caller restriction or guarantee was identified in this review.
## Validation
- The complete package suite passed under `go test -race ./...` when combining
the initial run with successful reruns of `internal/configuration` and
`internal/llm`. Those two packages initially failed solely because the sandbox
disallowed their local `httptest` listeners; their reruns permitted listeners.
- `go vet ./...` passed using Go 1.27.3 with `GOTOOLCHAIN=local`, `GOWORK=off`,
and offline module resolution.
- Changed Go files are formatted. `git diff --check` and local document-target
checks passed. No implementation, dependency, policy, or release files were
changed during the review.