Define the execution controls implementation plan
This commit is contained in:
139
docs/roadmap/audita-request.md
Normal file
139
docs/roadmap/audita-request.md
Normal file
@@ -0,0 +1,139 @@
|
|||||||
|
# Audita request: coordinator execution controls
|
||||||
|
|
||||||
|
Status: proposed follow-up to PromptKit v0.22.0. This document describes
|
||||||
|
consumer requirements; public API design and implementation remain upstream
|
||||||
|
decisions.
|
||||||
|
|
||||||
|
## Purpose
|
||||||
|
|
||||||
|
PromptKit v0.22.0 supplies the conversation capture, continuation, and execution
|
||||||
|
groups Audita requested. Audita can adopt those capabilities through its existing
|
||||||
|
execution adapter. To also adopt `Engine.Coordinate` and remove its local turn
|
||||||
|
loop, Audita needs two capabilities during coordinator-owned execution:
|
||||||
|
|
||||||
|
1. A fresh, application-selected timeout for each continuation operation.
|
||||||
|
2. Access to the exact prepared request before generation, including for turns
|
||||||
|
that subsequently fail without returning a result.
|
||||||
|
|
||||||
|
Both capabilities apply to applications that run bounded model operations and
|
||||||
|
retain optional diagnostic evidence. Neither requires PromptKit to understand
|
||||||
|
transcripts, correction proposals, or domain validation.
|
||||||
|
|
||||||
|
## Current workflow and integration gap
|
||||||
|
|
||||||
|
Audita's role-bound adapter currently derives a timeout context from its caller,
|
||||||
|
calls `PrepareExecution`, retains `PreparedExecution.Details()`, and executes
|
||||||
|
that same handle with `RunPrepared`. The operation deadline covers preparation,
|
||||||
|
capacity waiting, transient retries, generation, and structural repair. Each
|
||||||
|
model operation receives its own budget; application validation may make
|
||||||
|
additional, separately bounded model calls.
|
||||||
|
|
||||||
|
When debug evidence is enabled, Audita retains the prepared messages and
|
||||||
|
effective settings alongside the available response or failure information.
|
||||||
|
It owns artifact destinations, application-specific redaction, and handling of
|
||||||
|
optional diagnostic write failures. The same adapter supplies ordinary execution
|
||||||
|
observers and classifies operation errors for the application.
|
||||||
|
|
||||||
|
In v0.22.0, the [coordinator](../../coordinate.go) accepts a callback returning a
|
||||||
|
`RunRequest`, then directly calls `Engine.Run` with the coordinator's context.
|
||||||
|
The callback cannot supply a fresh execution context or a prepared handle.
|
||||||
|
There is no callback exposing the actual prepared request before that run.
|
||||||
|
|
||||||
|
A timeout around the entire coordinator bounds the conversation and application
|
||||||
|
callbacks together; it does not preserve a separate timeout for each operation.
|
||||||
|
Provider generation timeouts do not cover the whole operation either.
|
||||||
|
Separately calling `Prepare` before returning a request duplicates preparation
|
||||||
|
and does not bind the inspected snapshot to the execution. Successful exchange
|
||||||
|
inspection cannot supply a failed turn's prepared request when no new exchange
|
||||||
|
is returned. Existing metadata-only observations do not contain that content.
|
||||||
|
|
||||||
|
## Requested behavior
|
||||||
|
|
||||||
|
### Per-operation deadlines for coordinated turns
|
||||||
|
|
||||||
|
Allow the application to select a timeout for each requested continuation, with
|
||||||
|
these semantics:
|
||||||
|
|
||||||
|
- Establish a fresh deadline when that continuation starts, before its
|
||||||
|
preparation. Cover preparation, admission waiting, retries, generation, and
|
||||||
|
structural repair under the same operation budget.
|
||||||
|
- Respect caller cancellation and any earlier parent deadline. Expiry of a
|
||||||
|
local operation budget must not cancel the parent context or independent
|
||||||
|
branches.
|
||||||
|
- End the operation's timeout scope before invoking the application's next-turn
|
||||||
|
callback. Assessment calls made there can retain their own operation budgets.
|
||||||
|
- Preserve useful error identity so Audita can distinguish a local deadline
|
||||||
|
from cancellation of the caller. An execution timeout can end coordination
|
||||||
|
under the existing operational-error rule; no automatic turn replay or new
|
||||||
|
recovery policy is requested.
|
||||||
|
|
||||||
|
The existing additional-turn budget, transport retry budget, structural-repair
|
||||||
|
budget, and provider generation timeout retain their separate meanings.
|
||||||
|
|
||||||
|
### Exact prepared-request access
|
||||||
|
|
||||||
|
Provide an opt-in way to obtain caller-owned details of the actual prepared
|
||||||
|
continuation before its first generation attempt:
|
||||||
|
|
||||||
|
- Include the assembled captured history, follow-up definition messages,
|
||||||
|
appended messages, and ordinary resolved execution metadata, using the
|
||||||
|
existing [prepared-details contract](../../prepared_execution.go) where useful.
|
||||||
|
- Expose the snapshot that execution will use, without a second rendering or
|
||||||
|
input-reading pass. Access to details must not mutate that execution.
|
||||||
|
- Let the application retain those details even if generation subsequently
|
||||||
|
fails or is canceled and returns no ordinary result or captured exchange.
|
||||||
|
If preparation itself fails, a complete prepared snapshot is not required.
|
||||||
|
- Preserve existing credential-redaction semantics. Keep content-bearing access
|
||||||
|
explicitly selected and separate from ordinary metadata-only observations.
|
||||||
|
|
||||||
|
Audita needs the logical prepared request, not raw HTTP envelopes or an archive
|
||||||
|
of intermediate structural-repair requests. It can handle optional debug writes
|
||||||
|
itself without requiring PromptKit to own artifact storage or reporting policy.
|
||||||
|
|
||||||
|
## API direction and ownership
|
||||||
|
|
||||||
|
Prefer small execution-level controls that the coordinator honors through its
|
||||||
|
ordinary execution path. An optional operation timeout and a prepared-details
|
||||||
|
callback are one possible shape, not proposed exported names or signatures.
|
||||||
|
Their applicability to ordinary and prepared execution should follow a coherent
|
||||||
|
lifecycle rather than introducing a second coordinator-specific executor.
|
||||||
|
|
||||||
|
An alternative is allowing a coordinator turn to delegate execution through a
|
||||||
|
caller-controlled prepared-execution path. That could reuse Audita's adapter,
|
||||||
|
but exposes more execution responsibility and requires clear ownership of
|
||||||
|
capture, observers, admission release, and prepared-handle cleanup. Evaluate
|
||||||
|
that option only if it is simpler than the narrow controls above.
|
||||||
|
|
||||||
|
PromptKit should continue owning history assembly, turn progression, execution,
|
||||||
|
and observations. Audita retains response interpretation, validator calls,
|
||||||
|
branch policy, diagnostic storage, and application error classification. Existing
|
||||||
|
request observers, coordinator observers, and execution groups must remain
|
||||||
|
usable together. This request does not reopen the accepted conversation history,
|
||||||
|
final-turn delivery, or grouping contracts.
|
||||||
|
|
||||||
|
## Acceptance scenarios
|
||||||
|
|
||||||
|
1. Two successful continuation operations each finish within their own timeout,
|
||||||
|
while their combined duration exceeds one operation's budget. Both succeed
|
||||||
|
under an active parent context.
|
||||||
|
2. Preparation or admission waiting consumes the same budget as generation and
|
||||||
|
structural repair. Local expiry returns a recognizable deadline error and
|
||||||
|
stops that coordinator without canceling its parent or a sibling branch.
|
||||||
|
3. An earlier parent deadline remains effective. A validator called from the
|
||||||
|
next-turn callback runs after admission is released, including with backend
|
||||||
|
capacity one, under its own application-selected deadline.
|
||||||
|
4. The application receives the prepared continuation details before generation.
|
||||||
|
A later provider failure returns no result, but the retained details still
|
||||||
|
describe that failed operation. Mutating a details copy does not change the
|
||||||
|
messages executed.
|
||||||
|
5. Diagnostic access uses the same prepared snapshot that executes, including
|
||||||
|
when source inputs change afterward. It does not trigger another preparation
|
||||||
|
or provider call.
|
||||||
|
6. Existing observers and selected execution groups continue receiving ordinary
|
||||||
|
execution facts once per their existing contracts, including failure paths.
|
||||||
|
Applications that select neither new capability retain existing behavior.
|
||||||
|
|
||||||
|
The consumer implementation motivating this request is Audita's
|
||||||
|
`internal/framework/llm/adapter.go`, especially `RoleClient.CompleteStructured`.
|
||||||
|
This gap affects full coordinator adoption; it does not block adopting v0.22.0
|
||||||
|
capture, continuation, or execution groups independently.
|
||||||
@@ -1,8 +1,8 @@
|
|||||||
# Execution Controls For Coordinated Turns
|
# Execution Controls For Coordinated Turns
|
||||||
|
|
||||||
Status: all four design decisions accepted; ready for API and staged
|
Status: all four design decisions accepted. This document defines the target
|
||||||
implementation planning from v0.22.0. No code changes are authorized by this
|
state from v0.22.0; [implementation.md](implementation.md) selects the concrete
|
||||||
roadmap. Public names and signatures are not yet selected.
|
API and staged implementation work. The feature is not yet implemented.
|
||||||
|
|
||||||
## Purpose
|
## Purpose
|
||||||
|
|
||||||
@@ -139,37 +139,17 @@ controls need not take effect during `Prepare` or `ValidateRequest`; accepting
|
|||||||
request-shaped input there does not itself justify rejecting such controls or
|
request-shaped input there does not itself justify rejecting such controls or
|
||||||
starting a timer. Follow the accepted split-preparation and callback semantics.
|
starting a timer. Follow the accepted split-preparation and callback semantics.
|
||||||
|
|
||||||
## Work From The Baseline
|
## Affected Areas
|
||||||
|
|
||||||
1. **Design the public API.** Write small ordinary-run, coordinated-turn, and
|
The target spans request controls, execution-context lifetime, detached prepared
|
||||||
split-preparation usage sketches under the accepted lifecycle below. Specify
|
details, public error projection, and ordinary coordinator integration. It is an
|
||||||
omitted/zero/negative timeout handling using ordinary duration conventions,
|
additive runtime feature; no collection-format or ConfigKit changes are needed.
|
||||||
and express the selected callback timing and cleanup in the public contract.
|
Public GoDoc will own exact API contracts, the
|
||||||
Document the entry-point distinctions and conditional diagnostic availability.
|
[consumer guide](../consumers/pkg-promptkit.md) will explain use, and internal
|
||||||
Choose names and signatures without adding a broader lifecycle-hook framework.
|
lifecycle documentation will describe the implementation when it lands.
|
||||||
2. **Add operation-budget handling.** Establish and clean up the child scope at
|
|
||||||
the selected execution boundary. Preserve earlier parent deadlines and public
|
|
||||||
error identity through facade mapping and terminal observation classification.
|
|
||||||
Determine the outcome before terminal delivery and prevent observer latency
|
|
||||||
or child-context cleanup from changing it afterward.
|
|
||||||
Ensure cleanup does not affect the coordinator parent or subsequent callbacks.
|
|
||||||
3. **Expose the prepared snapshot.** Deliver detached public details from the
|
|
||||||
existing prepared state before generation. Cover both ordinary execution and
|
|
||||||
the selected prepared-execution lifecycle without a second preparation pass.
|
|
||||||
Preserve credential handling and release resources on cancellation or panic.
|
|
||||||
Avoid snapshot-copy work when the capability is not selected.
|
|
||||||
4. **Integrate and verify coordination.** Have ordinary execution honor the
|
|
||||||
selected controls on the callback-returned request under the accepted approach.
|
|
||||||
Preserve capture and observer composition; verify the next-turn callback runs
|
|
||||||
with the parent context after admission and child-budget cleanup.
|
|
||||||
5. **Document and demonstrate.** Update public GoDoc, the
|
|
||||||
[consumer guide](../consumers/pkg-promptkit.md), relevant internal lifecycle
|
|
||||||
documentation, and a focused offline example when implementation lands.
|
|
||||||
Show how applications retain diagnostics on failure and bound independent
|
|
||||||
assessment calls. Reconcile Audita's scenarios against the final API.
|
|
||||||
|
|
||||||
This is an additive runtime feature; no collection-format or ConfigKit changes
|
The implementation plan owns API selections, task sequencing, concrete changes,
|
||||||
are needed. Keep implementation proportional to the two requested controls.
|
and validation steps. Keep that work proportional to the two requested controls.
|
||||||
|
|
||||||
## Validation Targets
|
## Validation Targets
|
||||||
|
|
||||||
@@ -213,8 +193,8 @@ and the accepted lifecycle choices:
|
|||||||
## Accepted Decisions
|
## Accepted Decisions
|
||||||
|
|
||||||
All four decisions below are accepted. No significant product decisions remain
|
All four decisions below are accepted. No significant product decisions remain
|
||||||
open. Exact public names, signatures, timeout-value conventions, and the Go
|
open. The implementation plan selects public names, signatures, timeout-value
|
||||||
representation of timeout identity remain for API and implementation planning.
|
conventions, and the Go representation of timeout identity.
|
||||||
|
|
||||||
### 1. Narrow Execution Controls Or Caller-Delegated Execution?
|
### 1. Narrow Execution Controls Or Caller-Delegated Execution?
|
||||||
|
|
||||||
@@ -295,8 +275,8 @@ inspectable operation-timeout identity when this newly selected budget causes
|
|||||||
termination. Keep parent cancellation/deadlines and provider-local failures
|
termination. Keep parent cancellation/deadlines and provider-local failures
|
||||||
under their existing identities. The marker must survive public wrapping and
|
under their existing identities. The marker must survive public wrapping and
|
||||||
must not be inferred from any arbitrary `DeadlineExceeded` returned by a
|
must not be inferred from any arbitrary `DeadlineExceeded` returned by a
|
||||||
collaborator. Choose its Go representation during API design; no new
|
collaborator. Its Go representation is selected in the implementation plan;
|
||||||
error-report hierarchy or terminal-event category is implied.
|
no new error-report hierarchy or terminal-event category is implied.
|
||||||
|
|
||||||
This lets callers distinguish the requested local budget from an earlier parent
|
This lets callers distinguish the requested local budget from an earlier parent
|
||||||
or provider deadline without retaining the internal child context. Define cause
|
or provider deadline without retaining the internal child context. Define cause
|
||||||
|
|||||||
441
docs/roadmap/implementation.md
Normal file
441
docs/roadmap/implementation.md
Normal file
@@ -0,0 +1,441 @@
|
|||||||
|
# Execution Controls Implementation Plan
|
||||||
|
|
||||||
|
## Objective And Sprint Scope
|
||||||
|
|
||||||
|
Implement the accepted target in [execution-controls.md](execution-controls.md)
|
||||||
|
from the v0.22.0 execution architecture. The feature roadmap owns intent and the
|
||||||
|
four accepted lifecycle decisions; this document selects the API and the work
|
||||||
|
needed to implement them. [Audita's request](audita-request.md) supplies the
|
||||||
|
consumer scenarios.
|
||||||
|
|
||||||
|
Work through Stage 1 through Stage 8 in order. Each stage is scoped for one Luna
|
||||||
|
coding-agent prompt. This is one end-to-end sprint: intermediate stages may
|
||||||
|
temporarily fail to compile or leave wiring incomplete. Do not build temporary
|
||||||
|
compatibility layers merely to make intermediate commits green. There are no
|
||||||
|
intermediate human-review or external-publication gates. Completion requires
|
||||||
|
the assembled feature and its validation, not just completion of individual
|
||||||
|
stages.
|
||||||
|
|
||||||
|
Read [development.md](../development.md), the [agent policy](../policy/agents.md),
|
||||||
|
and the applicable architecture, testing, and documentation policies before
|
||||||
|
work. Preserve unrelated changes. Follow the existing commit policy for finished
|
||||||
|
units; report intermediate validation honestly. This plan authorizes no release,
|
||||||
|
push, dependency update, configuration-format change, or new consumer workflow
|
||||||
|
beyond the execution controls described here.
|
||||||
|
|
||||||
|
## Selected API And Semantics
|
||||||
|
|
||||||
|
Add these fields to the existing public `RunRequest`, without introducing an
|
||||||
|
execution-options object, coordinator executor, or general hook framework:
|
||||||
|
|
||||||
|
```go
|
||||||
|
OperationTimeout time.Duration `json:"-" yaml:"-"`
|
||||||
|
OnPrepared func(context.Context, PreparedRun) `json:"-" yaml:"-"`
|
||||||
|
```
|
||||||
|
|
||||||
|
Add one public sentinel, alongside the current execution errors:
|
||||||
|
|
||||||
|
```go
|
||||||
|
var ErrOperationTimeout = errors.New("operation timeout exceeded")
|
||||||
|
```
|
||||||
|
|
||||||
|
The spelling of an error message is not a contract. No exported timeout error
|
||||||
|
type, new observer event, or new failure-category enum is needed.
|
||||||
|
|
||||||
|
### Entry-Point Contract
|
||||||
|
|
||||||
|
| Entry point | OperationTimeout | OnPrepared |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `Run` | A positive value starts a fresh child budget before preparation, covering preparation and execution. | Notify once if execution reaches the prepared point before first generation. |
|
||||||
|
| `Coordinate` | Each returned request receives ordinary `Run` semantics with a fresh scope. | Ordinary per-request notification; no coordinator-specific copy of the option. |
|
||||||
|
| `PrepareExecution` | Retain the value privately, without starting a timer. Preparation uses its supplied context. | Retain the selected function privately without invoking it. |
|
||||||
|
| `RunPrepared` | Start the retained positive duration after the successful claim, before execution callbacks, credential revalidation, and admission. | Notify once when the claimed execution reaches the prepared point. |
|
||||||
|
| `Prepare`, `ValidateRequest` | Do not start a timer; use the supplied context as today. | Do not invoke or retain the notification. |
|
||||||
|
| `PreparedExecution.Details`, `Discard` | Do not start a timer. | Do not invoke the notification. Discard drops its retained reference. |
|
||||||
|
|
||||||
|
- A positive duration opts in. Zero and negative durations select no additional
|
||||||
|
timeout. This follows a nonpositive-means-unselected convention, avoids a new
|
||||||
|
request rejection rule, and leaves the parent deadline effective. There is no
|
||||||
|
default operation duration, upper bound, profile setting, or engine setting.
|
||||||
|
- The provider's existing generation timeout remains independent. Do not copy
|
||||||
|
`OperationTimeout` into `ExecutionTarget`, provider JSON, or prepared/result
|
||||||
|
metadata. Neither control changes prompt hashes or configuration inspection.
|
||||||
|
- The budget is continuous across preparation, capacity waiting, notification,
|
||||||
|
generation, transient retries/backoff, validation, and structural repair. It
|
||||||
|
is not restarted between phases, attempts, or repair generations.
|
||||||
|
- Split preparation retains a duration, not a context, deadline, timer, or
|
||||||
|
cancel function. Idle time does not count. A shared caller-created deadline
|
||||||
|
remains the way to bound preparation and later execution together.
|
||||||
|
- A nil notification selects no diagnostic copying. A selected function receives
|
||||||
|
the operation context and a detached, credential-redacted `PreparedRun` value
|
||||||
|
representing the logical request. Its nested mutable values belong to the
|
||||||
|
application. Reuse the existing projection used by `Details()`.
|
||||||
|
- Notification is synchronous, has no error return, and runs after successful
|
||||||
|
preparation and admission, before the first generation. Logical admission is
|
||||||
|
still held. Its time consumes the selected operation budget. It may retain
|
||||||
|
details for later diagnostics; PromptKit does not own writes or write errors.
|
||||||
|
- Check context before notification and again before generation after it
|
||||||
|
returns. A callback may block or ignore context; do not introduce a goroutine
|
||||||
|
to interrupt it. Callback panic propagates after deferred resource cleanup,
|
||||||
|
without a new terminal-event guarantee.
|
||||||
|
- Callback selection is copied as a function value, not as a deep copy of its
|
||||||
|
closure. Shared callback state is application-synchronized; separate runs can
|
||||||
|
invoke a common callback concurrently. Later reassignment of the original
|
||||||
|
request field cannot replace a prepared handle's retained function.
|
||||||
|
- Notification is conditional on reaching that execution point. Admission or
|
||||||
|
preparation failure can prevent it. Do not render again, reorder admission,
|
||||||
|
or promise a diagnostic snapshot for every failed operation. Split callers
|
||||||
|
can still inspect a successfully prepared handle through `Details()`.
|
||||||
|
|
||||||
|
### Timeout Attribution And Completion
|
||||||
|
|
||||||
|
Use the standard library's `context.WithTimeoutCause` with a distinct cause
|
||||||
|
value for each selected scope. Record the scope's own cause identity so an
|
||||||
|
inherited timeout from an enclosing operation cannot be mistaken for this
|
||||||
|
invocation's timer. Do not classify by error text or by `DeadlineExceeded`
|
||||||
|
alone. No strict ordering of independently racing cancellations is promised.
|
||||||
|
|
||||||
|
When an execution fails through its context deadline and the child context's
|
||||||
|
recorded cause is this scope's own timer cause, return an error matching both
|
||||||
|
`ErrOperationTimeout` and `context.DeadlineExceeded`. Preserve existing operation
|
||||||
|
and collaborator error identities. An earlier parent deadline/cancellation or
|
||||||
|
a provider deadline must not acquire the local marker merely because it is a
|
||||||
|
deadline error. An unrelated failure must not acquire it merely because the
|
||||||
|
timer has also expired. Parent-provided error causes, if any, retain their
|
||||||
|
existing behavior; this feature need not reinterpret them.
|
||||||
|
|
||||||
|
Finalize attribution from the operation's returned error before canceling its
|
||||||
|
child scope and before terminal observer delivery. A nil execution error remains
|
||||||
|
a completed result, including a completed structural rejection. Do not add a
|
||||||
|
post-terminal `ctx.Err()` check that converts it into an error. Slow terminal
|
||||||
|
observers and deferred context cleanup cannot change the chosen outcome. This
|
||||||
|
does not strengthen cancellation into forced interruption or a guarantee that
|
||||||
|
every context-ignoring collaborator will return by the deadline.
|
||||||
|
|
||||||
|
The terminal event continues using the existing deadline failure category for
|
||||||
|
a timeout error. Groups continue aggregating those ordinary terminal facts.
|
||||||
|
The child scope is cleaned up before `Run`/`RunPrepared` returns to the next-turn
|
||||||
|
callback; that callback receives the coordinator parent context, not the child.
|
||||||
|
Existing parent cancellation and undelivered-result rules remain unchanged.
|
||||||
|
|
||||||
|
## Implementation Map
|
||||||
|
|
||||||
|
Use existing owners rather than adding a new public package:
|
||||||
|
|
||||||
|
- Public declarations and contract: [types.go](../../types.go),
|
||||||
|
[engine.go](../../engine.go), and [prepared_execution.go](../../prepared_execution.go).
|
||||||
|
- Public conversion and error projection: [convert.go](../../convert.go),
|
||||||
|
[errors.go](../../errors.go), and [observations.go](../../observations.go).
|
||||||
|
- Internal request values: [domain.go](../../internal/domain/domain.go).
|
||||||
|
- Timeout lifecycle and notification orchestration: `internal/usecase`, chiefly
|
||||||
|
[runner.go](../../internal/usecase/runner.go) and
|
||||||
|
[prepared_execution.go](../../internal/usecase/prepared_execution.go).
|
||||||
|
- Existing private operation-error representation:
|
||||||
|
[operation_error.go](../../internal/usecase/operation_error.go).
|
||||||
|
- Ordinary coordinator integration: [coordinate.go](../../coordinate.go) and
|
||||||
|
[internal coordinator](../../internal/usecase/coordinate.go).
|
||||||
|
|
||||||
|
The root remains a facade. Keep timer construction and cause-attribution logic
|
||||||
|
in a small internal helper; the facade may establish that scope and project its
|
||||||
|
errors while managing its existing observation lifecycle. Internal packages must
|
||||||
|
not import the root package or depend on the public `PreparedRun` representation.
|
||||||
|
Private helper names and equivalent internal organization may change; the
|
||||||
|
selected public behavior and ownership boundaries are what matter.
|
||||||
|
|
||||||
|
## Stage 1: Add The Request Contract And Conversion
|
||||||
|
|
||||||
|
Outcome: the selected API and internal request plumbing exist, ready for the
|
||||||
|
execution stages to consume.
|
||||||
|
|
||||||
|
1. Add `OperationTimeout`, `OnPrepared`, and `ErrOperationTimeout` to the public
|
||||||
|
declarations above. Write their GoDoc from the entry-point and ownership
|
||||||
|
contract in this plan, explicitly separating operation and provider timeouts.
|
||||||
|
2. Extend `domain.RunRequest` with the duration and an internal callback of shape
|
||||||
|
`func(context.Context, *domain.PreparedRun)`. Document the callback argument
|
||||||
|
as borrowed read-only internal state. It is not a public mutable snapshot.
|
||||||
|
3. Extend `toDomainRunRequest` to copy the duration and capture the selected
|
||||||
|
public callback function in a local variable. Only when it is non-nil, adapt
|
||||||
|
it by projecting the supplied prepared state through `fromDomainPreparedRun`
|
||||||
|
and passing the resulting value. This projection is the public ownership
|
||||||
|
boundary; do not deep-copy the domain snapshot first as well.
|
||||||
|
4. Keep these fields out of public prepared/result JSON and opaque handle
|
||||||
|
details. Request conversion does not start a timer, invoke a callback, or
|
||||||
|
reject an execution-only control on preparation-only entry points.
|
||||||
|
5. Add the private timeout error identity in `internal/usecase`, using the same
|
||||||
|
existing separation between internal and public sentinels. Add its public
|
||||||
|
projection to `errors.go` when introducing the attribution wrapper in Stage 2.
|
||||||
|
|
||||||
|
Validation: focused conversion tests should establish option forwarding and
|
||||||
|
conditional adaptation. Reuse existing nested-value projection tests instead of
|
||||||
|
retesting every existing `PreparedRun` field. Compilation may remain incomplete
|
||||||
|
until the execution wiring lands; do not claim the feature works at this stage.
|
||||||
|
|
||||||
|
## Stage 2: Bound Ordinary Execution And Preserve Its Error Outcome
|
||||||
|
|
||||||
|
Outcome: `Engine.Run` owns one continuous optional budget and returns the same
|
||||||
|
failure classification it publishes to observers.
|
||||||
|
|
||||||
|
1. Add a small internal operation-scope helper using `WithTimeoutCause`. For
|
||||||
|
nonpositive durations, reuse the parent context without a timer. For positive
|
||||||
|
durations, retain a fresh per-invocation cause token and a cancel function.
|
||||||
|
Use a real distinct identity, not a shared sentinel or zero-sized token whose
|
||||||
|
pointer identity cannot distinguish scopes.
|
||||||
|
2. Give the helper an error-finalization operation: for a non-nil returned error,
|
||||||
|
require both a match to the operation context's deadline error and equality
|
||||||
|
of its recorded cause with this scope's token before adding the private
|
||||||
|
timeout identity. Leave nil errors and unrelated errors unchanged. Preserve
|
||||||
|
the original error as a cause.
|
||||||
|
3. Reuse `usecase.OperationError` with the private timeout category to carry the
|
||||||
|
marker around the existing error, rather than creating a public error
|
||||||
|
hierarchy. Project that category to `ErrOperationTimeout` through the
|
||||||
|
existing `publicOperationError` machinery. Keep the original phase category
|
||||||
|
and context cause inspectable through the nested chain.
|
||||||
|
4. Establish the ordinary scope in the facade's `Run` entry, after masking
|
||||||
|
inherited execution-observation state and before start-event delivery and
|
||||||
|
request preparation. Both observed and unobserved runs use it. Pass its
|
||||||
|
context through request conversion, internal engine delegation, admission,
|
||||||
|
preparation, and all generation/validation work. Do not also start a second
|
||||||
|
timer inside `Runner.Run`.
|
||||||
|
5. Adjust the private facade `run` path as needed to finalize the execution
|
||||||
|
error before applying public error projection once. On normal return, freeze
|
||||||
|
attribution and the returned result/error, clean up the child, then classify
|
||||||
|
and deliver the existing terminal event. Preserve normal observation of
|
||||||
|
request conversion and preparation failures. Handle all early returns; install
|
||||||
|
cleanup before any user callback can panic.
|
||||||
|
6. Do not change provider timeouts, retry eligibility, repair limits, scheduling,
|
||||||
|
or existing result timing conventions to implement this feature. A timer
|
||||||
|
expiring while terminal observation runs is not another execution failure.
|
||||||
|
|
||||||
|
Validation: narrow helper tests cover disabled scopes, local expiry, parent
|
||||||
|
deadline/cancellation, inherited operation causes, unrelated errors, and nil
|
||||||
|
completion. Add public `Run` tests proving the marker survives phase wrapping
|
||||||
|
and agrees with an ordinary deadline terminal event. Provider-only deadline
|
||||||
|
errors while the operation child is active must not gain the marker.
|
||||||
|
|
||||||
|
## Stage 3: Retain Controls Across Split Preparation
|
||||||
|
|
||||||
|
Outcome: prepared execution carries the selected duration and notification,
|
||||||
|
without keeping a live operation scope while idle.
|
||||||
|
|
||||||
|
1. Extend `preparedExecutionPayload` with the duration and internal notification.
|
||||||
|
`PrepareExecution` copies them from the converted request only after its
|
||||||
|
ordinary successful preparation. Preparation itself uses its supplied
|
||||||
|
context, without starting the selected timer or calling the notification.
|
||||||
|
2. Extend payload clearing for both fields. Preserve the existing atomic claim,
|
||||||
|
owner checks, foreign/reused-handle behavior, and durable redacted details.
|
||||||
|
A discard or completed/unwound claim drops PromptKit's callback reference.
|
||||||
|
3. After a successful `Runner.RunPrepared` claim, establish the scope with the
|
||||||
|
retained duration, before the retained observer's start delivery, credential
|
||||||
|
revalidation, and admission. Defer payload and scope cleanup before callback
|
||||||
|
work. A failed claim neither starts notification nor changes existing silent
|
||||||
|
invalid-handle behavior.
|
||||||
|
4. Pass the child through execution and finalize the returned error with the
|
||||||
|
same helper before cleanup and facade terminal classification. Keep this
|
||||||
|
lifecycle in the internal prepared runner rather than borrowing duration or
|
||||||
|
callback state from an unclaimed handle in the facade.
|
||||||
|
5. `RunPrepared`'s facade projects the resulting error and emits its one existing
|
||||||
|
terminal event after resource release. Do not reapply the timer at the facade
|
||||||
|
or retain the preparation context. Namespace views keep their shared-owner
|
||||||
|
behavior and use the handle's original selections.
|
||||||
|
|
||||||
|
Validation: extend prepared-execution coverage to show that preparation and an
|
||||||
|
idle delay longer than the selected duration do not consume its execution
|
||||||
|
budget; later execution does. Reassignment of the original request does not
|
||||||
|
change the retained duration/callback. Cover earlier caller deadlines and a
|
||||||
|
shared caller deadline across preparation and execution. Existing claim-race,
|
||||||
|
discard, foreign-owner, credential, and error-identity tests remain applicable.
|
||||||
|
Notification invocation itself is completed in Stage 4.
|
||||||
|
|
||||||
|
## Stage 4: Notify From The Actual Shared Prepared Snapshot
|
||||||
|
|
||||||
|
Outcome: ordinary and prepared execution expose one detached logical request
|
||||||
|
before first generation, without repeating preparation.
|
||||||
|
|
||||||
|
1. Thread the internal notification from `Runner.Run` and the claimed payload
|
||||||
|
into the existing shared `executePreparedRun` entry. At that entry, preparation
|
||||||
|
is complete and logical admission is held on both paths.
|
||||||
|
2. When selected, check the operation context, invoke the internal adapter with
|
||||||
|
the actual retained prepared state, and check context again before entering
|
||||||
|
generation. A context failure at this boundary follows the existing
|
||||||
|
generation operation-error wrapping, preserving context identity and gaining
|
||||||
|
local-timeout identity only through the scope finalizer.
|
||||||
|
3. Invoke once per executing logical request, not per transport retry or repair.
|
||||||
|
Do not call `Prepare`, `Details`, artifact readers, renderers, or schema
|
||||||
|
preparation again to produce the notification. With nil `OnPrepared`, avoid
|
||||||
|
constructing a public details copy.
|
||||||
|
4. Keep the prepared state's API key redacted. Generation still receives its
|
||||||
|
private direct credential through the existing separate path. Preserve the
|
||||||
|
complete assembled messages, schema, settings, input hashes, session,
|
||||||
|
ancestry, and preparation metadata in the delivered public value.
|
||||||
|
5. Leave logical admission held; do not release/reacquire it around notification.
|
||||||
|
Invoke application code without holding the handle mutex or a capacity
|
||||||
|
manager lock. Preserve panic propagation while defers release the lease,
|
||||||
|
timer, and claimed payload. A panic does not make a handle reusable.
|
||||||
|
|
||||||
|
Validation: one ordinary and one split public test establish ordering before
|
||||||
|
generation and exact callback selection. Add focused context-before/after
|
||||||
|
notification tests and an internal cleanup regression if the public capacity
|
||||||
|
probe alone cannot demonstrate payload cleanup. More extensive ownership and
|
||||||
|
failure assertions belong to Stage 6, not duplicate tests here.
|
||||||
|
|
||||||
|
## Stage 5: Verify Continuous Budgets Across Execution Phases
|
||||||
|
|
||||||
|
Outcome: tests demonstrate that one scope reaches all relevant collaborators,
|
||||||
|
with correct attribution and observation on failure.
|
||||||
|
|
||||||
|
1. Add focused tests for preparation expiry through a context-aware injected
|
||||||
|
artifact reader, generation-capacity waiting on a real limited backend,
|
||||||
|
interrupted built-in retry backoff, and structural repair expiry. Verify
|
||||||
|
standard context matching, local marker, and retained phase identity where
|
||||||
|
that phase already supplies one. Cover ordinary and split paths without
|
||||||
|
mechanically duplicating every case on both.
|
||||||
|
2. Use a narrow internal runner test for validation-phase expiry, since validation
|
||||||
|
collaborators are not a public injection API. Check the same deadline reaches
|
||||||
|
preparation, validation, and generation; do not add a new public testing seam.
|
||||||
|
3. Test that notification time consumes the same budget and that expiry there
|
||||||
|
prevents generation. Cover an earlier parent deadline and parent cancellation
|
||||||
|
separately from a provider's own deadline. Preserve original injected error
|
||||||
|
identities and do not relabel a non-context error after timer expiry.
|
||||||
|
4. On both ordinary and split public execution, exercise a terminal observer
|
||||||
|
that outlasts the operation budget after successful completion. The result
|
||||||
|
stays successful and the terminal remains successful. Include the timeout
|
||||||
|
failure case to ensure public error mapping happens before terminal delivery.
|
||||||
|
5. After expiry, prove capacity remains usable and the caller parent stays
|
||||||
|
active. Respect the current capacity model: logical admission may reject
|
||||||
|
immediately, while admitted work can wait for an active generation permit.
|
||||||
|
Do not introduce a new blocking admission queue to satisfy a test label.
|
||||||
|
|
||||||
|
Use channels for ordering and `testing/synctest` for controlled elapsed-time
|
||||||
|
scenarios where its bubble rules apply. This repository's Go minimum supports
|
||||||
|
it. Prefer in-memory fixtures and controlled transports; avoid real short sleeps
|
||||||
|
or close-to-deadline assertions. Reuse the existing generation tests for retry
|
||||||
|
policy rather than building a second retry matrix. There is no production clock
|
||||||
|
framework, live provider probe, allocation threshold, or performance gate here.
|
||||||
|
|
||||||
|
Validation: run focused use-case/generation tests and the new public execution
|
||||||
|
control tests under the race detector. The new cases should protect deadline
|
||||||
|
propagation and classification, not private helper call sequences.
|
||||||
|
|
||||||
|
## Stage 6: Verify Notification Ownership And Failure Lifecycles
|
||||||
|
|
||||||
|
Outcome: applications can safely retain diagnostic details without altering
|
||||||
|
execution or acquiring new lifecycle guarantees.
|
||||||
|
|
||||||
|
1. Add a representative continuation request containing history, follow-up
|
||||||
|
messages, appended cache-control metadata, input hashes, session, settings
|
||||||
|
with nested provider values, and a structured schema. Inside `OnPrepared`,
|
||||||
|
retain and mutate the delivered mutable values; the model request, captured
|
||||||
|
exchange, and later prepared-handle `Details()` remain unchanged. Use
|
||||||
|
synthetic direct/environment credentials and verify their values are absent
|
||||||
|
from diagnostic details while the existing client credential path still works.
|
||||||
|
2. Demonstrate snapshot fidelity when mutable input/schema sources change after
|
||||||
|
preparation and before execution. Reuse existing frozen-preparation fixtures;
|
||||||
|
notification must describe the executed snapshot, not reread the changed
|
||||||
|
sources. A later provider failure or cancellation may return no result, while
|
||||||
|
the application still retains the delivered details.
|
||||||
|
3. Verify absence of notification for preparation failure and ordinary admission
|
||||||
|
rejection, without requiring extra reads to produce diagnostics. For split
|
||||||
|
execution cover invalid/foreign/discarded/reused handles and pre-notification
|
||||||
|
cancellation. `Details()`, `Prepare`, `ValidateRequest`, and preparation of an
|
||||||
|
executable handle never notify or start this execution timer.
|
||||||
|
4. Verify that retries and structural repair do not repeat notification. Reuse
|
||||||
|
an existing retry-plus-repair integration scenario with an added notification
|
||||||
|
assertion rather than duplicating its full event/usage test elsewhere.
|
||||||
|
5. Recover a notification panic only in test code, then verify ordinary capacity
|
||||||
|
is reusable and a claimed handle stays consumed. Do not require a terminal
|
||||||
|
event after panic or impose a prohibition on callback reentrancy. A capacity
|
||||||
|
one probe documents held admission rather than promising nested success.
|
||||||
|
|
||||||
|
Validation: run affected public contract and private payload-cleanup tests under
|
||||||
|
the race detector. Mutable callback state in concurrent tests must be protected
|
||||||
|
by the test; do not make PromptKit serialize application callbacks globally.
|
||||||
|
|
||||||
|
## Stage 7: Verify Coordinated Turns And Observer Composition
|
||||||
|
|
||||||
|
Outcome: coordination obtains the controls through ordinary `Run`, retaining
|
||||||
|
its existing branching and delivery semantics.
|
||||||
|
|
||||||
|
1. Check the coordinator's existing request-value copy preserves both new
|
||||||
|
controls while it replaces `ContinueFrom` and composes observers. No separate
|
||||||
|
coordinator timeout, executor, or public prepare/run sequence is needed.
|
||||||
|
2. Add a controlled-time, two-continuation success case. Each turn finishes
|
||||||
|
within its own timeout while their combined execution exceeds one turn's
|
||||||
|
duration. Callback details contain each turn's assembled history; no timer
|
||||||
|
from an earlier turn controls a later turn.
|
||||||
|
3. In the next-turn callback, verify the parent context is active and the prior
|
||||||
|
child context has been cleaned up. Run an explicitly selected assessment on
|
||||||
|
the same backend with concurrency one and queue capacity zero, using its own
|
||||||
|
timeout. This differs from `OnPrepared`, where admission is still held.
|
||||||
|
4. Exercise a locally timed-out continuation: coordination returns its ordinary
|
||||||
|
operational error with the marker, does not fabricate an outcome or completed
|
||||||
|
exchange, and leaves the parent and a sibling continuation usable. Retained
|
||||||
|
details are available if notification occurred before failure. Do not promise
|
||||||
|
them if execution stopped earlier.
|
||||||
|
5. Compose request and coordinator observers with an explicit execution group.
|
||||||
|
Check one ordinary lifecycle per selected run, normal terminal usage/attempt
|
||||||
|
totals, and no implicit membership for assessment calls. Reuse existing
|
||||||
|
final-turn and parent-cancellation/undelivered-result cases to verify that
|
||||||
|
cleanup and slow terminal observation have not changed delivery semantics.
|
||||||
|
|
||||||
|
Validation: run the coordinator, observation, group, and new execution-control
|
||||||
|
contract cases together under the race detector. Keep tests at the public
|
||||||
|
boundary for the assembled behavior; the internal coordinator loop need not
|
||||||
|
be redesigned solely because its ordinary run gained options.
|
||||||
|
|
||||||
|
## Stage 8: Document, Demonstrate, And Validate The Assembled Feature
|
||||||
|
|
||||||
|
Outcome: the implementation and its supported consumer workflow are complete,
|
||||||
|
with documentation in the canonical owners.
|
||||||
|
|
||||||
|
1. Finish GoDoc on `RunRequest`, `ErrOperationTimeout`, `Run`, `Prepare`,
|
||||||
|
`ValidateRequest`, `PrepareExecution`, `RunPrepared`, and `PreparedExecution`.
|
||||||
|
Cross-reference the owning field/error contract rather than duplicating its
|
||||||
|
entire definition. Explain conditional notification, held admission,
|
||||||
|
cooperative cancellation, split preparation, callback ownership, and outcome
|
||||||
|
finalization before terminal observation.
|
||||||
|
2. Update the [consumer guide](../consumers/pkg-promptkit.md) with small ordinary,
|
||||||
|
coordinated, and split usage sketches linked to GoDoc. Show a caller-owned
|
||||||
|
retained `PreparedRun` on failure, `errors.Is` checks for both timeout
|
||||||
|
identities, and a separate budget for callback-owned assessment. Do not add
|
||||||
|
persistence, log destinations, redaction policy, or a mandatory diagnostic
|
||||||
|
write gate.
|
||||||
|
3. Extend the maintained [observation example](../../examples/go-library/observe/main.go)
|
||||||
|
and its semantic test with a positive operation budget on continuation,
|
||||||
|
selected prepared-details retention, and a separately budgeted next-turn
|
||||||
|
assessment. Keep it offline, synthetic, and successful; Stage 5/6 tests own
|
||||||
|
failure demonstrations. Do not add another standalone example framework.
|
||||||
|
4. Update [runner documentation](../internal/runner.md),
|
||||||
|
[observation documentation](../internal/observation.md), and the
|
||||||
|
[component inventory](../internal/overview.md) only where responsibilities or
|
||||||
|
lifecycle descriptions changed. No ConfigKit, collection-format, provider
|
||||||
|
payload, or architectural policy change is required by this work.
|
||||||
|
5. Review Audita's six acceptance scenarios and the feature roadmap's validation
|
||||||
|
targets against the completed tests. Consolidate duplicate tests; do not
|
||||||
|
expand this into a full cross-product of all entry points, phases, observers,
|
||||||
|
and callback behaviors. Retain existing no-option regression coverage.
|
||||||
|
6. Select the newest installed Go toolchain per the development guide. Format
|
||||||
|
changed Go files, run `go test -race ./...` and `go vet ./...` with
|
||||||
|
`GOTOOLCHAIN=local` and `GOWORK=off`, and run `git diff --check`. The full Go
|
||||||
|
suite includes the maintained examples; no second example execution gate is
|
||||||
|
needed. Use writable caches as required by the development environment.
|
||||||
|
Validate local links and inspect the full diff for unintended files/secrets.
|
||||||
|
|
||||||
|
Do not edit published v0.22.0 release notes, create a new release/tag, or require
|
||||||
|
CI or external publication to finish the sprint. Release preparation and
|
||||||
|
retirement of these roadmaps are subsequent work.
|
||||||
|
|
||||||
|
## Completion Criteria
|
||||||
|
|
||||||
|
- Ordinary and coordinated runs apply one optional whole-operation budget;
|
||||||
|
split handles retain duration and callback selection without live idle timers.
|
||||||
|
- Deadline attribution survives public wrapping, distinguishes this scope's
|
||||||
|
recorded timer cause, and agrees with terminal observation before cleanup.
|
||||||
|
- Notification supplies a detached redacted view of the actual logical prepared
|
||||||
|
request, once before generation when reached, without repeated preparation.
|
||||||
|
- Failure, cancellation, and panic paths preserve existing resource, handle,
|
||||||
|
error, observation, grouping, and continuation behavior.
|
||||||
|
- The maintained example, canonical documentation, focused tests, full race
|
||||||
|
suite, and vet are complete. No-option use stays compatible.
|
||||||
|
|
||||||
|
No significant open questions remain. The API and nonpositive-duration choices
|
||||||
|
above complete the planning decisions left by the accepted feature roadmap.
|
||||||
Reference in New Issue
Block a user