Prepare v0.22.0 release notes and retire completed roadmaps
This commit is contained in:
@@ -45,6 +45,9 @@ boundary and constraints that framework work must preserve.
|
|||||||
|
|
||||||
## Release Guidance
|
## Release Guidance
|
||||||
|
|
||||||
|
The [v0.22.0 release notes](docs/releases/v0.22.0.md) cover captured conversation
|
||||||
|
exchanges, application-directed turn coordination, and explicit execution groups.
|
||||||
|
|
||||||
The [v0.21.0 release notes](docs/releases/v0.21.0.md) cover ConfigKit v1.2.0
|
The [v0.21.0 release notes](docs/releases/v0.21.0.md) cover ConfigKit v1.2.0
|
||||||
and main-relative companion imports through the existing configuration-source
|
and main-relative companion imports through the existing configuration-source
|
||||||
API.
|
API.
|
||||||
|
|||||||
67
docs/releases/v0.22.0.md
Normal file
67
docs/releases/v0.22.0.md
Normal file
@@ -0,0 +1,67 @@
|
|||||||
|
# Promptkit v0.22.0
|
||||||
|
|
||||||
|
This minor release adds captured conversation exchanges, optional sequential
|
||||||
|
coordination, and explicit execution-group summaries. Applications can continue
|
||||||
|
or branch a conversation without reconstructing rendered history, while keeping
|
||||||
|
response interpretation and workflow decisions in application code.
|
||||||
|
|
||||||
|
## Capture And Continue Conversations
|
||||||
|
|
||||||
|
Set `RunRequest.CaptureExchange` to capture a completed run's effective messages
|
||||||
|
and final assistant content. Supply its `RunResult.Exchange` as `ContinueFrom`
|
||||||
|
on a new request to continue it. Exchanges are reusable, support independent
|
||||||
|
branches and different engines, and expose detached history through `Details()`.
|
||||||
|
Prepared execution also supports capture of the exact snapshot it executes.
|
||||||
|
|
||||||
|
Internal structural-repair instructions and intermediate candidates are not
|
||||||
|
conversation history. A completed result with failed structural validation can
|
||||||
|
still be captured; an operational error returning no result creates no exchange.
|
||||||
|
Follow-ups select their own profiles, output contracts, credentials, and sessions
|
||||||
|
under ordinary request rules. See [capture and continuation guidance](../consumers/pkg-promptkit.md#capture-and-continue-a-conversation)
|
||||||
|
and the [public exchange contract](../../exchange.go).
|
||||||
|
|
||||||
|
## Coordinate Application-Directed Turns
|
||||||
|
|
||||||
|
`Engine.Coordinate` advances one conversation branch using an application callback
|
||||||
|
and an additional-turn budget. The application selects the next request or stops;
|
||||||
|
Promptkit handles turn progression and cancellation. Each completed run releases
|
||||||
|
backend admission before the callback, allowing assessment calls to use the same
|
||||||
|
backend. Stop and budget exhaustion are control-flow outcomes, not judgments of
|
||||||
|
response acceptance. Transport retries and structural repairs remain separate
|
||||||
|
from the turn budget.
|
||||||
|
|
||||||
|
See the [coordination walkthrough](../consumers/pkg-promptkit.md#coordinate-application-directed-turns)
|
||||||
|
and [coordinator GoDoc](../../coordinate.go) for callback and cancellation behavior.
|
||||||
|
|
||||||
|
## Collect Selected Execution Facts
|
||||||
|
|
||||||
|
`ExecutionGroup` collects explicitly selected execution observations, while
|
||||||
|
`ComposeObservers` combines groups with existing diagnostic observers. Detached
|
||||||
|
group snapshots report member identities, terminal coverage, attempt/retry/repair
|
||||||
|
counts, and available token totals without counting a run more than once.
|
||||||
|
Terminal coverage remains distinct from token-usage completeness; membership is
|
||||||
|
not inferred from ancestry or nested calls.
|
||||||
|
|
||||||
|
See [execution-group guidance](../consumers/pkg-promptkit.md#collect-selected-execution-facts),
|
||||||
|
the [group GoDoc](../../execution_group.go), and the updated
|
||||||
|
[offline observation example](../../examples/go-library/observe/main.go).
|
||||||
|
|
||||||
|
## Efficiency And Compatibility
|
||||||
|
|
||||||
|
History assembly and inspection avoid intermediate full-history copies. Observer
|
||||||
|
composition retains storage only for selected callbacks and avoids fan-out
|
||||||
|
closures for zero or one selected callback.
|
||||||
|
|
||||||
|
Capture and coordination are opt-in; existing execution workflows and
|
||||||
|
configuration formats need no migration from v0.21.0. The Go consumer minimum
|
||||||
|
remains 1.27.0, and dependency versions are unchanged. Exported request, prepared,
|
||||||
|
result, and observation structs gained fields; consumers using positional struct
|
||||||
|
literals must switch to keyed literals. Continuation metadata adds `parent_run_id`
|
||||||
|
to prepared/result JSON when applicable; the opaque exchange itself is excluded.
|
||||||
|
Exchanges are in-process capabilities, not serialized restartable sessions.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go get go.maximumdirect.net/promptkit@v0.22.0
|
||||||
|
```
|
||||||
|
|
||||||
|
[COPYRIGHT](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.22.0/COPYRIGHT) | [LICENSE](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.22.0/LICENSE) | [ATTRIBUTION](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.22.0/docs/oss/attribution.md)
|
||||||
@@ -1,169 +0,0 @@
|
|||||||
# Audita request: conversation continuation and application-directed repair
|
|
||||||
|
|
||||||
Status: proposed upstream features, requested by Audita. Public API design and
|
|
||||||
implementation remain for PromptKit to determine.
|
|
||||||
|
|
||||||
This request is based on Audita's use of PromptKit v0.21.0. It proposes two
|
|
||||||
related capabilities: captured conversation continuations, followed by bounded
|
|
||||||
repair driven by application-supplied validation feedback. The goal is to move
|
|
||||||
reusable execution machinery into PromptKit and remove its local equivalents
|
|
||||||
from Audita.
|
|
||||||
|
|
||||||
## Audita's workflow
|
|
||||||
|
|
||||||
Audita polishes transcripts using module-specific proposal, validation, and
|
|
||||||
repair prompts:
|
|
||||||
|
|
||||||
1. A proposal request supplies transcript segments and receives a structured
|
|
||||||
collection of proposed corrections.
|
|
||||||
2. Audita evaluates each correction using deterministic and LLM-backed
|
|
||||||
validators. Accepted corrections are retained for later application.
|
|
||||||
3. Each actionable rejection starts an independent repair branch containing the
|
|
||||||
original rendered request, the full final assistant response, and a follow-up
|
|
||||||
instruction rendered from that module's repair prompt definition.
|
|
||||||
4. The follow-up identifies one correction and supplies its current content,
|
|
||||||
available before-and-after transcript context, and validation findings. The
|
|
||||||
model can revise or withdraw that correction. This response uses a different
|
|
||||||
schema from the original collection of proposals.
|
|
||||||
5. A revision passes through the validators again. Further attempts retain that
|
|
||||||
branch's history and receive updated feedback. Acceptance, withdrawal,
|
|
||||||
exhaustion, or failure ends the branch; independent approvals are preserved.
|
|
||||||
|
|
||||||
Audita owns the meaning of a correction, the validation rules, repair
|
|
||||||
eligibility, and eventual application to the transcript. Conversation handling
|
|
||||||
and the bounded revision lifecycle are useful beyond transcript processing.
|
|
||||||
|
|
||||||
## Existing support and the remaining gap
|
|
||||||
|
|
||||||
PromptKit already supports [structural output repair](../consumers/pkg-promptkit.md#repair-a-structured-result),
|
|
||||||
[appended messages](../consumers/pkg-promptkit.md#append-already-rendered-messages),
|
|
||||||
prepared execution, profiles, execution observations, and backend capacity
|
|
||||||
management. These allow Audita to implement the workflow today.
|
|
||||||
|
|
||||||
Audita still captures exchanges, reconstructs the original prompt, verifies its
|
|
||||||
rendered prefix, assembles branch histories, renders follow-up instructions,
|
|
||||||
and coordinates repeated application validation and revision. The requested
|
|
||||||
APIs would make those responsibilities supported PromptKit capabilities.
|
|
||||||
|
|
||||||
## 1. Captured conversations and independent continuations
|
|
||||||
|
|
||||||
Provide a public way to capture a rendered request and its full final assistant
|
|
||||||
response, then continue that exchange without making the consumer reconstruct
|
|
||||||
the original prompt and check that its messages are unchanged.
|
|
||||||
|
|
||||||
Audita needs to:
|
|
||||||
|
|
||||||
- Preserve the original rendered message content, roles, and cache metadata
|
|
||||||
when continuing an exchange.
|
|
||||||
- Start multiple independent branches from the same exchange. A branch's later
|
|
||||||
instructions and responses must not alter the original exchange or another
|
|
||||||
branch's history.
|
|
||||||
- Render follow-up instructions using ordinary PromptKit prompt definitions and
|
|
||||||
variables, and append them to the selected branch. Audita's repair prompts
|
|
||||||
remain operator-replaceable through its existing PromptKit configuration.
|
|
||||||
- Select a follow-up profile and output contract independently of the initial
|
|
||||||
request. The first response may contain many items while a continuation
|
|
||||||
returns one revised item or a withdrawal.
|
|
||||||
- Retain the relationship between the original exchange, the follow-up prompt
|
|
||||||
definition, and each execution in inspection and observation data.
|
|
||||||
|
|
||||||
An in-process capability is sufficient for Audita. Persistence, restartable
|
|
||||||
sessions, and a conversation database are not required by this request. The
|
|
||||||
existing [prepared execution handle](../../prepared_execution.go) is single-use;
|
|
||||||
the request does not assume that copying or reusing it creates a continuation.
|
|
||||||
|
|
||||||
## 2. Bounded application-directed repair
|
|
||||||
|
|
||||||
Build a reusable revision coordinator on the continuation capability. Audita
|
|
||||||
should supply its assessment and request-building behavior while PromptKit
|
|
||||||
handles the repeated execution lifecycle.
|
|
||||||
|
|
||||||
The coordinator needs to support:
|
|
||||||
|
|
||||||
- Starting with an already-assessed candidate and its rejection findings, so
|
|
||||||
Audita need not repeat the initial batch validation simply to enter repair.
|
|
||||||
- Application-supplied assessment of each revision, including deterministic
|
|
||||||
checks and checks that make further PromptKit calls. Findings may come from
|
|
||||||
multiple validators; their meaning and explanatory content remain with the
|
|
||||||
application.
|
|
||||||
- Application-selected follow-up prompts, variables, profiles, and response
|
|
||||||
contracts. Audita supplies the candidate and context, and interprets the
|
|
||||||
response as a revision or withdrawal. PromptKit need not prescribe an Audita
|
|
||||||
correction schema or a universal feedback format.
|
|
||||||
- Reassessment before a revised candidate can be accepted, with fresh feedback
|
|
||||||
for the next attempt and accumulated history confined to that branch.
|
|
||||||
- A caller-selected additional-revision budget and caller context, with clear
|
|
||||||
terminal outcomes covering acceptance, withdrawal, exhaustion, unavailable
|
|
||||||
results, and interruption. Callers retain decisions about repair eligibility
|
|
||||||
and whether an operational failure should terminate other work.
|
|
||||||
- Attempt observations, execution relationships, and available usage accounting
|
|
||||||
sufficient to report a repair sequence without recreating its lifecycle
|
|
||||||
tracking in Audita.
|
|
||||||
|
|
||||||
The revision budget must remain distinct from transport retry and structural
|
|
||||||
repair budgets. A structurally valid response can still fail Audita's checks;
|
|
||||||
each revision generation can itself need structural repair. Accounting should
|
|
||||||
distinguish those layers and avoid double-counting shared initial work.
|
|
||||||
|
|
||||||
Application validators must be able to call the same engine and backend without
|
|
||||||
being blocked by capacity retained by their parent operation. Coordinating
|
|
||||||
ordinary completed runs is one possible approach. PromptKit should determine
|
|
||||||
the implementation in light of its [execution and admission lifecycle](../internal/runner.md#run-flow).
|
|
||||||
|
|
||||||
This is broader than exposing the current structural-repair hook: the response
|
|
||||||
schema can change between proposal and repair, validation may perform model
|
|
||||||
calls, and each rejected item has its own history and disposition.
|
|
||||||
|
|
||||||
## Ownership boundary
|
|
||||||
|
|
||||||
PromptKit would own captured exchanges, branch continuation, the bounded repair
|
|
||||||
lifecycle, and associated execution observations. Audita would retain:
|
|
||||||
|
|
||||||
- Transcript segments, proposal identities, glossary and protected-term rules,
|
|
||||||
editorial judgments, and meaning checks.
|
|
||||||
- Validator selection and ordering, actionable-rejection policy, feedback
|
|
||||||
content, and module-specific proposal, validation, and repair prompts.
|
|
||||||
- Before-and-after transcript previews, preservation of accepted sibling
|
|
||||||
proposals, application ordering, and final transcript mutation.
|
|
||||||
- Application reports, debug artifact destinations, and CLI behavior.
|
|
||||||
|
|
||||||
The separate idea of reconciling missing or duplicate items in batch responses
|
|
||||||
is outside this request. The two requested capabilities should be usable
|
|
||||||
through the supported public facade without an Audita dependency or a general
|
|
||||||
workflow framework. Exact types, callbacks, and configuration syntax are left
|
|
||||||
to PromptKit's API design.
|
|
||||||
|
|
||||||
## Suggested delivery and acceptance scenarios
|
|
||||||
|
|
||||||
Deliver conversation continuation first, allowing Audita to replace its capture
|
|
||||||
and replay machinery. Then add application-directed repair and replace the
|
|
||||||
corresponding local coordination. Existing prompt definitions and domain
|
|
||||||
validators should remain usable through the new APIs.
|
|
||||||
|
|
||||||
The following scenarios would demonstrate that the capabilities meet Audita's
|
|
||||||
needs:
|
|
||||||
|
|
||||||
1. An initial response contains three proposals. One is accepted; the other two
|
|
||||||
receive independent repair conversations containing the same original
|
|
||||||
exchange. Revising either branch leaves the accepted proposal and the other
|
|
||||||
branch unchanged.
|
|
||||||
2. A follow-up uses a separate prompt definition, profile, and single-item
|
|
||||||
response schema. Inspection identifies the follow-up and its originating
|
|
||||||
exchange.
|
|
||||||
3. A revision fails multiple application checks, receives their updated
|
|
||||||
feedback, and passes on a later attempt. An LLM-backed check can use the same
|
|
||||||
engine and backend even when backend capacity is limited to one.
|
|
||||||
4. Withdrawal, exhausted revision budget, unavailable validation, and
|
|
||||||
cancellation produce distinguishable outcomes. An unaccepted revision is
|
|
||||||
never returned as an accepted candidate, and a recoverable branch failure
|
|
||||||
leaves independent approvals available to the application.
|
|
||||||
5. A revision generation needs both a transport retry and structural repair.
|
|
||||||
Observations distinguish these from application-directed revision attempts
|
|
||||||
and expose the available usage without counting the original exchange once
|
|
||||||
per branch.
|
|
||||||
|
|
||||||
For implementation reference, Audita currently owns these mechanics in
|
|
||||||
`internal/framework/llm/adapter.go`,
|
|
||||||
`internal/framework/structuredoutput/call.go`, and
|
|
||||||
`internal/framework/runner/repair.go`. These are examples of the consumer need,
|
|
||||||
not proposed PromptKit package boundaries or public API shapes.
|
|
||||||
@@ -1,448 +0,0 @@
|
|||||||
# Conversation Continuation And Execution Groups
|
|
||||||
|
|
||||||
Status: target behavior accepted; concrete API choices and implementation steps
|
|
||||||
are recorded in the [implementation plan](implementation.md). This document
|
|
||||||
defines the target behavior and responsibility boundaries. Execution groups are
|
|
||||||
independently usable.
|
|
||||||
|
|
||||||
## Purpose And Responsibility Boundary
|
|
||||||
|
|
||||||
PromptKit will let applications continue logical conversations, optionally
|
|
||||||
coordinate successive turns, and summarize explicitly selected executions.
|
|
||||||
These capabilities remove repeated history reconstruction, turn lifecycle work,
|
|
||||||
and observation aggregation without interpreting prompt or response content.
|
|
||||||
|
|
||||||
[Audita's request](audita-request.md) supplies consumer evidence and acceptance
|
|
||||||
scenarios. Repair, analysis, clarification, and branching exploration should all
|
|
||||||
use the same content-neutral capabilities. New concepts and names must be useful
|
|
||||||
without translating an application's domain into repair terminology. This does
|
|
||||||
not rename or expand PromptKit's existing structural-output repair functionality.
|
|
||||||
|
|
||||||
| Capability | PromptKit owns | Application owns |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| Captured exchanges and continuation | Logical history, reusable branches, ordinary follow-up execution, exchange details and parent metadata | Which exchange to continue and which request to execute |
|
|
||||||
| Optional sequential coordinator | One branch's turn progression, additional-turn budget, callback delivery and cancellation checks | Response interpretation, next action, branch scheduling and broader failure policy |
|
|
||||||
| Optional execution groups | Explicitly selected observations, deduplication and execution-fact summaries | Membership selection, domain identifiers, final-report timing and presentation |
|
|
||||||
|
|
||||||
Continuation is independently usable. The coordinator builds on it; applications
|
|
||||||
can instead use their own loops. Groups work with ordinary runs as well as
|
|
||||||
continuations and require neither capture nor a coordinator. Selecting a group
|
|
||||||
does not change scheduling, capacity, cancellation, retries, or validation.
|
|
||||||
|
|
||||||
The scope is in-process use. Persistence, restartable workflows, cross-process
|
|
||||||
resumption, a conversation registry, batch reconciliation, and a branching-workflow
|
|
||||||
scheduler are outside this feature. Applications schedule branches, join results,
|
|
||||||
and decide whether one branch's failure should affect other work. This initial
|
|
||||||
scope is not a permanent prohibition on future capabilities.
|
|
||||||
|
|
||||||
## Captured Exchanges And Continuation
|
|
||||||
|
|
||||||
### Capture And Availability
|
|
||||||
|
|
||||||
Capture is opt-in per ordinary or prepared execution. Ordinary execution returns
|
|
||||||
the exchange alongside its usual result. A prepared handle retains the capture
|
|
||||||
choice; executing that exact prepared snapshot creates the exchange. Preparation
|
|
||||||
alone creates no completed exchange. Continuation implies capture so its result
|
|
||||||
can itself be continued.
|
|
||||||
|
|
||||||
Capture the request actually executed, including the complete rendered messages
|
|
||||||
with their content, roles, and cache metadata, followed by the full final
|
|
||||||
assistant content returned by the run. Consumers must not reconstruct the request
|
|
||||||
or pair a response with separately prepared data. “Full” means content supported
|
|
||||||
by PromptKit, not a new raw provider-envelope API.
|
|
||||||
|
|
||||||
Capture represents logical conversation history. Internal structural-repair
|
|
||||||
instructions and intermediate candidates are excluded; only the final returned
|
|
||||||
assistant content follows the original request. Existing attempt observations
|
|
||||||
remain separate from conversation content.
|
|
||||||
|
|
||||||
| Execution outcome | Capture behavior |
|
|
||||||
| --- | --- |
|
|
||||||
| Completed response satisfying its output contract | Return the captured exchange. |
|
|
||||||
| Completed response with failed structural validation, including repair exhaustion | Return the exchange with the ordinary validation result intact. |
|
|
||||||
| Operational error with no completed result | Create no new exchange; the preceding exchange remains usable. |
|
|
||||||
| Completed response obtained before cancellation prevents coordinator delivery | Preserve that single undelivered response under the coordinator return contract. |
|
|
||||||
|
|
||||||
Capture does not imply structural validity or application acceptance. A completed
|
|
||||||
response with failed validation follows the normal coordinator callback path,
|
|
||||||
subject to cancellation; it is not reclassified as an operational error.
|
|
||||||
|
|
||||||
### Ownership And Reuse
|
|
||||||
|
|
||||||
An exchange is an immutable, reusable snapshot held by the caller. Continuing it
|
|
||||||
adds the rendered follow-up messages and final response to a new exchange, leaving
|
|
||||||
the source and independent branches unchanged. No previous templates or input
|
|
||||||
files need to be read or rendered again.
|
|
||||||
|
|
||||||
Avoid unnecessary repeated copying of retained history where practical, while
|
|
||||||
preserving independent branches and detached inspection values. Internal sharing
|
|
||||||
of immutable history is an implementation option, not a required representation
|
|
||||||
or a public allocation or complexity guarantee.
|
|
||||||
|
|
||||||
Another engine instance in the same process can continue the exchange, applying
|
|
||||||
its own configuration to the new request. The originating engine is not required.
|
|
||||||
This permits application engine replacement without promising serialized or
|
|
||||||
cross-process resumption.
|
|
||||||
|
|
||||||
Completed exchanges retain no backend admission lease, direct credential, or
|
|
||||||
execution callback. Ordinary Go memory management controls their lifetime; there
|
|
||||||
is no new `Close` or `Discard` lifecycle. The existing
|
|
||||||
[prepared execution handle](../../prepared_execution.go) remains a separate,
|
|
||||||
single-use execution capability.
|
|
||||||
|
|
||||||
### Exchange Details
|
|
||||||
|
|
||||||
Provide explicit access to the held exchange's logical message history, including
|
|
||||||
its final assistant content, originating run identity, and immediate parent
|
|
||||||
reference. Reuse existing run identifiers; no mandatory conversation, branch, or
|
|
||||||
sequence identifiers are introduced.
|
|
||||||
|
|
||||||
Follow the general pattern of `PreparedExecution.Details()`. Details must be
|
|
||||||
generically renderable through ordinary encoders. Reading or modifying returned
|
|
||||||
values must not affect retained history or another branch. Inspection reads the
|
|
||||||
snapshot without rendering, loading inputs, or making provider calls.
|
|
||||||
|
|
||||||
Existing results and preparation details continue to supply ordinary execution
|
|
||||||
information. Exchange inspection does not create a second comprehensive report,
|
|
||||||
a result archive, or an engine-level conversation lookup API. `InspectPrompt`,
|
|
||||||
`InspectProfile`, and `InspectBackend` keep their resource-inspection boundary.
|
|
||||||
Captured message content remains separate from metadata-only observations.
|
|
||||||
|
|
||||||
## Optional Sequential Coordinator
|
|
||||||
|
|
||||||
The coordinator starts from a captured exchange and executes one branch using
|
|
||||||
an application callback, a caller-selected additional-turn budget, and context.
|
|
||||||
The callback selects a new ordinary request, stops, or returns an error. It can
|
|
||||||
keep application state in a closure; PromptKit defines no domain-state or
|
|
||||||
feedback schema.
|
|
||||||
|
|
||||||
### Progression And Budget
|
|
||||||
|
|
||||||
With an active context, invoke the initial callback once, even with a zero-turn
|
|
||||||
budget. It receives the starting exchange and remaining budget; the starting
|
|
||||||
exchange is not a newly delivered response. Existing application findings can
|
|
||||||
be used without repeating earlier assessment.
|
|
||||||
|
|
||||||
After each continuation, deliver its completed response once through the same
|
|
||||||
callback before starting another execution, including the final permitted
|
|
||||||
response. Internal retries and structural repairs do not create extra callbacks.
|
|
||||||
|
|
||||||
| Callback action | Coordinator behavior |
|
|
||||||
| --- | --- |
|
|
||||||
| Stop | Return application-directed completion, including after the last permitted response. |
|
|
||||||
| Continue with budget remaining | Start the selected continuation. |
|
|
||||||
| Continue without budget remaining | Return exhaustion without executing the requested turn. |
|
|
||||||
| Error | Stop and propagate the error, subject to context checks. |
|
|
||||||
|
|
||||||
The budget permits at most N additional continuation executions to start.
|
|
||||||
Starting a continuation consumes one turn even if preparation or generation
|
|
||||||
fails. The initial exchange and callback invocations consume no turns; internal
|
|
||||||
transport retries and structural repairs stay within one turn. The first
|
|
||||||
operational error stops coordination, without refunds or recovery machinery.
|
|
||||||
|
|
||||||
Using the full budget alone does not mean exhaustion: exhaustion means the
|
|
||||||
application requested more work than the budget permits. The budget does not
|
|
||||||
bound provider attempts, cost, or work inside callbacks. Applications wanting
|
|
||||||
no callback work can skip the coordinator entirely.
|
|
||||||
|
|
||||||
### Cancellation And Failures
|
|
||||||
|
|
||||||
Check context immediately before every callback, including the initial callback.
|
|
||||||
If cancellation is already observed, skip delivery and return the context error
|
|
||||||
with its identity preserved. Retain a completed response already obtained but
|
|
||||||
not yet delivered; do not recover raw output from an underlying failed call that
|
|
||||||
returned no completed result.
|
|
||||||
|
|
||||||
Callbacks receive the same context and cooperate with cancellation. Check it
|
|
||||||
again after the callback returns and before starting another execution.
|
|
||||||
Cancellation may race with invocation; PromptKit does not forcibly terminate
|
|
||||||
callback code or promise callback delivery despite cancellation or panic.
|
|
||||||
|
|
||||||
An execution or callback error stops coordination. Preserve ordinary Go error
|
|
||||||
identities, including `errors.Is`/`errors.As`. Do not automatically replay the
|
|
||||||
failed continuation or callback. Ordinary transport retries and structural repair
|
|
||||||
remain execution-internal. Earlier deliveries and application work remain intact;
|
|
||||||
the application owns any broader consequences.
|
|
||||||
|
|
||||||
### Minimal Return Contract
|
|
||||||
|
|
||||||
Return only:
|
|
||||||
|
|
||||||
- An ordinary Go error for execution failure, callback failure, or cancellation.
|
|
||||||
- A small normal outcome distinguishing application-directed stop from budget
|
|
||||||
exhaustion; neither outcome is an error or a claim about content acceptance.
|
|
||||||
- An optional single completed response when cancellation prevented callback
|
|
||||||
delivery, reusing existing response types where practical.
|
|
||||||
|
|
||||||
Delivery means callback invocation, not successful processing or acceptance. If
|
|
||||||
the callback returns an error, its response is already delivered and is neither
|
|
||||||
returned as undelivered nor replayed. Previously delivered responses are not
|
|
||||||
archived in the termination result. No additional failure hierarchy, failure-stage
|
|
||||||
fields, run-ID references, or aggregate reporting is added to this contract.
|
|
||||||
Existing response and observation fields remain available; group summaries are
|
|
||||||
obtained separately.
|
|
||||||
|
|
||||||
## Shared Execution And Accounting Rules
|
|
||||||
|
|
||||||
### Follow-Up Configuration
|
|
||||||
|
|
||||||
A continuation is an ordinary new execution supplied with captured history.
|
|
||||||
Resolve its prompt, profile, credentials, output contract, and request overrides
|
|
||||||
using the receiving engine's existing rules. In particular:
|
|
||||||
|
|
||||||
- Use an explicit request profile or the follow-up prompt's default profile,
|
|
||||||
with ordinary missing-profile behavior when neither is available.
|
|
||||||
- A supplied output contract completely replaces the default; do not merge it
|
|
||||||
with or inherit the preceding execution's contract.
|
|
||||||
- Resolve credentials for the new execution; retained history does not preserve
|
|
||||||
the original direct key.
|
|
||||||
- Select session ID from the new request or follow-up prompt's session template.
|
|
||||||
Do not implicitly inherit it. Applications may explicitly reuse a session ID;
|
|
||||||
ancestry does not depend on one.
|
|
||||||
|
|
||||||
History introduces no additional execution-setting precedence. The follow-up
|
|
||||||
can change model or response schema without altering the original exchange.
|
|
||||||
|
|
||||||
Assemble each continuation's messages in this order:
|
|
||||||
|
|
||||||
1. The captured logical history, including its final assistant content.
|
|
||||||
2. The rendered follow-up prompt definition messages, in definition order.
|
|
||||||
3. The follow-up request's caller-appended messages, in caller order.
|
|
||||||
|
|
||||||
Preserve authored roles and message content; do not move messages based on role.
|
|
||||||
Continuation adds no history-size limit or automatic truncation. Upstream
|
|
||||||
context-window or other request rejections follow ordinary generation-error
|
|
||||||
behavior.
|
|
||||||
|
|
||||||
### Capacity And Application Work
|
|
||||||
|
|
||||||
Compose completed executions. Finish each turn and release its logical admission
|
|
||||||
lease before calling application code. Releasing only the active generation
|
|
||||||
permit is insufficient under the existing [run flow](../internal/runner.md#run-flow).
|
|
||||||
Each later execution acquires ordinary admission.
|
|
||||||
|
|
||||||
The coordinator holds no backend capacity while awaiting callback work.
|
|
||||||
Callbacks may inspect responses, consult data, wait for application input, or
|
|
||||||
call the same engine and backend, including when backend capacity is one.
|
|
||||||
|
|
||||||
### Ancestry And Per-Run Accounting
|
|
||||||
|
|
||||||
Each continuation retains its own originating run ID and exposes its immediate
|
|
||||||
parent exchange's run ID. Parent metadata describes history provenance; group
|
|
||||||
membership describes application-selected accounting. Neither implies the other.
|
|
||||||
A validator can join a continuation's accounting group without extending its
|
|
||||||
conversation.
|
|
||||||
|
|
||||||
Reuse ordinary per-run usage and observations, including attempt, retry, repair,
|
|
||||||
failure-path, and usage-presence/completeness semantics. Preserve the existing
|
|
||||||
[observation boundary](../internal/observation.md): arbitrary nested calls are
|
|
||||||
independent executions. Do not infer membership from a callback's purpose or
|
|
||||||
context.
|
|
||||||
|
|
||||||
If initial run A supplies branches B and C, its usage belongs only to A. B and C
|
|
||||||
do not inherit that cost. Usage reported for resending captured history in B and
|
|
||||||
C still belongs to those new runs. Count selected runs once each; never add
|
|
||||||
terminal totals to the attempt events they already summarize.
|
|
||||||
|
|
||||||
## Optional Execution Groups
|
|
||||||
|
|
||||||
### Explicit Membership And Observer Composition
|
|
||||||
|
|
||||||
Use an observer-compatible collector attached explicitly to selected executions.
|
|
||||||
Prepared requests retain the selection; membership starts when execution emits
|
|
||||||
observations. Preparation alone, discarded handles, and failed claims do not
|
|
||||||
enroll runs through observations.
|
|
||||||
|
|
||||||
Coordinator selection covers its continuation runs. Callback-initiated calls
|
|
||||||
require their own explicit selection. Neither nested context nor a captured
|
|
||||||
starting exchange automatically enrolls another run.
|
|
||||||
|
|
||||||
Support an application's existing diagnostic observer alongside one or more
|
|
||||||
collectors through a small observer-composition helper. Preserve synchronous
|
|
||||||
delivery and existing panic behavior. Do not introduce asynchronous queues,
|
|
||||||
panic recovery, or guaranteed delivery to later observers after an earlier one
|
|
||||||
panics; interrupted delivery can leave accounting incomplete.
|
|
||||||
|
|
||||||
A run may explicitly belong to multiple independent groups. Each group counts
|
|
||||||
its run identity once. Applications obtain operation-level and overall views by
|
|
||||||
attaching the corresponding collectors to each selected run. Overlapping totals
|
|
||||||
are not generally additive; no group hierarchy, inherited membership, or merging
|
|
||||||
facility is needed.
|
|
||||||
|
|
||||||
### Concurrent Snapshots And Lifetime
|
|
||||||
|
|
||||||
Provide detached, internally consistent snapshots from a caller-held group while
|
|
||||||
selected executions and snapshot reads may occur concurrently. Reading or
|
|
||||||
modifying a snapshot must not change the collector or another snapshot. There
|
|
||||||
is no mandatory close, seal, or wait lifecycle. Applications own scheduling and
|
|
||||||
decide when to read or discard the group.
|
|
||||||
|
|
||||||
Distinguish observed runs with terminal observations from those without them.
|
|
||||||
Missing terminal observations mean incomplete accounting; the group does not
|
|
||||||
infer whether work is active, delivery was interrupted, or another cause applies.
|
|
||||||
It promises no terminal delivery beyond ordinary observation semantics.
|
|
||||||
|
|
||||||
“All observed members have terminal observations” does not mean the application
|
|
||||||
operation is complete. More runs can join later, and PromptKit cannot know whether
|
|
||||||
the application selected all relevant calls. The application chooses when a
|
|
||||||
snapshot represents its final report.
|
|
||||||
|
|
||||||
### Summary Contents
|
|
||||||
|
|
||||||
The initial summary exposes:
|
|
||||||
|
|
||||||
- Member run identities and terminal-observation coverage.
|
|
||||||
- Existing provider-attempt, transient-retry, and structural-repair counters.
|
|
||||||
- Available token totals with per-field known/completeness information.
|
|
||||||
|
|
||||||
Aggregate one terminal observation per run. These observations already summarize
|
|
||||||
retries, repairs, and available failure-path usage, including executions returning
|
|
||||||
no ordinary result. Do not reconstruct totals from attempts.
|
|
||||||
|
|
||||||
Keep terminal coverage distinct from usage completeness. A terminal observation
|
|
||||||
may itself have incomplete usage. Two finished runs with fully reported usage
|
|
||||||
provide a useful known subtotal, but a third observed run lacking a terminal
|
|
||||||
event leaves group accounting incomplete. Preserve existing usage semantics and
|
|
||||||
expose missing coverage separately; unavailable evidence must not become known
|
|
||||||
zero.
|
|
||||||
|
|
||||||
Live attempt-level totals and duration aggregation are omitted from the initial
|
|
||||||
implementation. Applications can use existing events for progress displays. If
|
|
||||||
durations are later added, summed execution durations must be distinguished from
|
|
||||||
elapsed group time. Groups supply execution facts, not monetary cost, application
|
|
||||||
outcomes, or an application-wide reporting framework. Applications retain domain
|
|
||||||
mappings, interpretation, and report assembly.
|
|
||||||
|
|
||||||
## Consumer Sketches
|
|
||||||
|
|
||||||
These are pseudocode, not selected exported names or implemented Go APIs.
|
|
||||||
Each begins with an already captured exchange and uses ordinary follow-up request
|
|
||||||
selection. Both omit application-specific error handling, including checks for
|
|
||||||
failed structural validation before schema-specific decoding.
|
|
||||||
|
|
||||||
### Analysis Loop
|
|
||||||
|
|
||||||
The application records responses and selects its next question or decides that
|
|
||||||
its investigation is complete. PromptKit retains history and executes the turns.
|
|
||||||
|
|
||||||
```text
|
|
||||||
analysis = application.start_analysis(initial_exchange)
|
|
||||||
|
|
||||||
choose_next(ctx, turn):
|
|
||||||
if turn.new_result exists:
|
|
||||||
analysis.record(turn.new_result)
|
|
||||||
|
|
||||||
question = application.choose_next_question(analysis)
|
|
||||||
if question is absent:
|
|
||||||
return Stop()
|
|
||||||
|
|
||||||
return Continue(request(
|
|
||||||
prompt_id = "research/investigate",
|
|
||||||
vars = {"question": question}
|
|
||||||
))
|
|
||||||
|
|
||||||
termination, error = coordinate(
|
|
||||||
ctx, start = initial_exchange,
|
|
||||||
additional_turns = 3, next = choose_next
|
|
||||||
)
|
|
||||||
application.present_analysis(analysis, termination, error)
|
|
||||||
```
|
|
||||||
|
|
||||||
### Audita Branch
|
|
||||||
|
|
||||||
Audita starts with an already-assessed candidate and existing findings. Only
|
|
||||||
Audita interprets acceptance, withdrawal, or assessment unavailability.
|
|
||||||
|
|
||||||
```text
|
|
||||||
branch = audita.branch(candidate, existing_findings)
|
|
||||||
|
|
||||||
choose_next(ctx, turn):
|
|
||||||
if turn.new_result exists:
|
|
||||||
response = audita.decode_response(turn.new_result)
|
|
||||||
if audita.is_withdrawal(response):
|
|
||||||
branch.mark_withdrawn()
|
|
||||||
return Stop()
|
|
||||||
|
|
||||||
branch.update_candidate(response)
|
|
||||||
assessment = audita.assess(ctx, engine, branch.candidate)
|
|
||||||
branch.record(assessment)
|
|
||||||
if audita.should_stop(assessment):
|
|
||||||
return Stop()
|
|
||||||
|
|
||||||
return Continue(request(
|
|
||||||
prompt_id = branch.configured_follow_up_prompt,
|
|
||||||
vars = audita.follow_up_variables(branch)
|
|
||||||
))
|
|
||||||
|
|
||||||
termination, error = coordinate(
|
|
||||||
ctx, start = initial_exchange,
|
|
||||||
additional_turns = 3, next = choose_next
|
|
||||||
)
|
|
||||||
audita.record_branch_outcome(branch, termination, error)
|
|
||||||
```
|
|
||||||
|
|
||||||
The initial callback uses existing findings without repeating batch assessment.
|
|
||||||
`audita.assess` may call the same backend after the turn's admission lease is
|
|
||||||
released. Those calls join any group only through their own explicit selection.
|
|
||||||
Other candidates can start independent branches; Audita owns sibling outcomes.
|
|
||||||
|
|
||||||
Both sketches process the final permitted response through the normal callback.
|
|
||||||
Applications retain their received results; the coordinator's termination value
|
|
||||||
is not an archive. These examples should remain usable through the same generic
|
|
||||||
API without manual history assembly or per-turn lifecycle bookkeeping.
|
|
||||||
|
|
||||||
## API And Implementation Ownership
|
|
||||||
|
|
||||||
The [implementation plan](implementation.md) selects the concrete API and steps
|
|
||||||
for reaching this target. Both consumer sketches must remain expressible without
|
|
||||||
manual history assembly or per-turn lifecycle bookkeeping. Execution groups
|
|
||||||
remain independent of capture and coordination; the plan's ordering does not
|
|
||||||
introduce a product dependency between them.
|
|
||||||
|
|
||||||
## Acceptance Scenarios
|
|
||||||
|
|
||||||
Use [Audita's acceptance scenarios](audita-request.md#suggested-delivery-and-acceptance-scenarios)
|
|
||||||
alongside analysis and branching-conversation examples. Validate each capability
|
|
||||||
at its public boundary without adding guarantees beyond the selected behavior.
|
|
||||||
|
|
||||||
### Capture And Continuation
|
|
||||||
|
|
||||||
- Ordinary and prepared opt-in capture uses the actual executed snapshot;
|
|
||||||
preparation alone produces no exchange.
|
|
||||||
- Internal structural repair contributes only the full final assistant content
|
|
||||||
to history. Final validation failure still permits capture; an operational
|
|
||||||
failure without a completed result does not.
|
|
||||||
- Independent branches preserve prior history and outcomes, including after
|
|
||||||
original inputs change or disappear and when continued by another engine.
|
|
||||||
- Follow-up configuration, credentials, session selection, and complete output
|
|
||||||
overrides follow ordinary rules without implicit inheritance.
|
|
||||||
- Details encode generically, expose originating and parent identities, and are
|
|
||||||
detached from retained state; inspection performs no rendering or external reads.
|
|
||||||
|
|
||||||
### Coordination
|
|
||||||
|
|
||||||
- Initial callback, zero-turn behavior, last-response delivery, stop versus
|
|
||||||
exhaustion, and execution-start budget consumption follow the specified rules.
|
|
||||||
- Cancellation before delivery retains only an already-obtained undelivered
|
|
||||||
response; callback errors do not replay delivered responses or earlier work.
|
|
||||||
- Failed structural validation follows normal response delivery, while operational
|
|
||||||
errors preserve Go error identities and stop progression.
|
|
||||||
- Same-backend callback calls at capacity one proceed after logical admission
|
|
||||||
release. Internal retries/repairs do not create extra conversation callbacks.
|
|
||||||
- Both consumer sketches work without PromptKit interpreting content or owning
|
|
||||||
application outcomes, branch scheduling, or a response archive.
|
|
||||||
|
|
||||||
### Observations And Groups
|
|
||||||
|
|
||||||
- Per-run usage and failure diagnostics preserve ordinary presence/completeness
|
|
||||||
semantics and parent references distinguish turns and branches.
|
|
||||||
- Explicit ordinary, prepared, coordinator, and callback membership works alongside
|
|
||||||
an existing observer; unrelated nested calls remain independent.
|
|
||||||
- Concurrent collection and snapshots preserve detached, consistent results.
|
|
||||||
More members may join after a snapshot with full terminal coverage.
|
|
||||||
- Overlapping groups count each run once within each group. Shared initial work,
|
|
||||||
newly reported replay usage, and attempt/terminal events are not double counted.
|
|
||||||
- Known subtotals, explicit zeros, incomplete usage, and missing terminal coverage
|
|
||||||
remain distinguishable, including on failure paths.
|
|
||||||
- Observer composition preserves synchronous delivery and panic behavior without
|
|
||||||
stronger terminal-delivery promises.
|
|
||||||
@@ -1,659 +0,0 @@
|
|||||||
# Conversation Continuation And Execution Groups Implementation Plan
|
|
||||||
|
|
||||||
## Objective And Sprint Rules
|
|
||||||
|
|
||||||
Implement the accepted [feature roadmap](conversation-continuation.md), using
|
|
||||||
[Audita's request](audita-request.md) as consumer evidence. The feature roadmap
|
|
||||||
owns product behavior and boundaries; this plan selects the API and work order.
|
|
||||||
Existing structural-output repair remains a separate, unchanged capability.
|
|
||||||
|
|
||||||
Each numbered stage is one Luna-sized implementation prompt. Execute them in
|
|
||||||
order during one end-to-end sprint. Intermediate stages need not compile or
|
|
||||||
constitute independently releasable changes. Do not build compatibility shims or
|
|
||||||
temporary implementations solely to make a stage green. Carry a concise record
|
|
||||||
of remaining wiring into the next stage; the final stage must leave the whole
|
|
||||||
repository coherent and validated. There are no intermediate human-review,
|
|
||||||
external-publication, provider-access, or dependency-release gates.
|
|
||||||
|
|
||||||
Start with [development guidance](../development.md). Follow the applicable
|
|
||||||
[agent](../policy/agents.md), [architecture](../policy/architecture.md),
|
|
||||||
[testing](../policy/testing.md), and [documentation](../policy/documentation.md)
|
|
||||||
policies. Preserve the accepted [configuration boundary](../adr/0001-configuration-boundary.md).
|
|
||||||
This sprint needs no configuration format, dependency, Go minimum, release
|
|
||||||
tooling, or licensing changes. It does not authorize tagging or publication.
|
|
||||||
|
|
||||||
Use real fast internal collaborators and replace only external boundaries in
|
|
||||||
tests. Select focused checks during stages and run the complete checks at the
|
|
||||||
end. Do not duplicate existing retry, repair, schema, or capacity test matrices.
|
|
||||||
Protect the new interactions with those mechanisms.
|
|
||||||
|
|
||||||
## Current Baseline And Integration Points
|
|
||||||
|
|
||||||
- The root facade in `engine.go` converts public requests, masks inherited
|
|
||||||
observation state, invokes the internal engine, maps errors, and finishes
|
|
||||||
observations after the runner releases admission. Preserve this boundary.
|
|
||||||
- `internal/usecase/runner.go` has shared resolution and completion preparation
|
|
||||||
phases. `completePreparationWithStructuredOutput` currently combines rendered
|
|
||||||
definition messages with appended messages and hashes the resulting prompt.
|
|
||||||
`executePreparedRun` is shared by ordinary and prepared execution and returns
|
|
||||||
the final `RawOutput`, including after exhausted structural repair.
|
|
||||||
- `internal/usecase/prepared_execution.go` retains a single-claim payload,
|
|
||||||
validation plan, direct credential, retry budget, and observer separately from
|
|
||||||
durable prepared details. Claims remain bound to the originating execution
|
|
||||||
owner. Details survive execution and discard.
|
|
||||||
- `internal/domain/message.go` owns message cloning and cache-control ownership.
|
|
||||||
Reuse it rather than inventing a second normalization contract.
|
|
||||||
- `observations.go`, `internal/domain/observation.go`, and
|
|
||||||
`internal/observation/recorder.go` own value-only events, run IDs, terminal
|
|
||||||
counters, and presence-aware usage. Terminal observations already include
|
|
||||||
completed work preceding operational failure. Do not total result usage or
|
|
||||||
attempt events on top of terminal totals.
|
|
||||||
- `internal/engine/engine.go` applies namespace views to new caller selections.
|
|
||||||
Captured history requires no original-engine lookups or namespace rewriting.
|
|
||||||
- Root conversion, `json.go`, formatting, and contract tests must account for
|
|
||||||
additions without changing existing timing, credential, or error behavior.
|
|
||||||
|
|
||||||
## Selected Public API
|
|
||||||
|
|
||||||
Implement these names and shapes in the root `promptkit` package. Code below
|
|
||||||
specifies additions and complete new types, not replacement definitions for
|
|
||||||
existing types. Keep all implementation machinery under `internal/`.
|
|
||||||
|
|
||||||
### Capture, Continuation, And Ancestry
|
|
||||||
|
|
||||||
```go
|
|
||||||
// Add to RunRequest.
|
|
||||||
CaptureExchange bool
|
|
||||||
ContinueFrom *Exchange `json:"-" yaml:"-"`
|
|
||||||
|
|
||||||
// Add to RunResult.
|
|
||||||
Exchange *Exchange `json:"-" yaml:"-"`
|
|
||||||
ParentRunID string `json:"parent_run_id,omitempty"`
|
|
||||||
|
|
||||||
// Add to PreparedRun.
|
|
||||||
ParentRunID string `json:"parent_run_id,omitempty"`
|
|
||||||
|
|
||||||
// Add to ExecutionEvent, following its existing field/tag style.
|
|
||||||
ParentRunID string
|
|
||||||
|
|
||||||
// Exchange has private state only.
|
|
||||||
type Exchange struct { /* private immutable snapshot reference */ }
|
|
||||||
|
|
||||||
type ExchangeDetails struct {
|
|
||||||
RunID string `json:"run_id"`
|
|
||||||
ParentRunID string `json:"parent_run_id,omitempty"`
|
|
||||||
Messages []RenderedMessage `json:"messages"`
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e *Exchange) Details() ExchangeDetails
|
|
||||||
```
|
|
||||||
|
|
||||||
- `Run` with `CaptureExchange: true` returns `result.Exchange`. A non-nil
|
|
||||||
`ContinueFrom` supplies history and implies capture, regardless of the Boolean.
|
|
||||||
No separate continuation execution method is necessary.
|
|
||||||
- `Prepare`, `ValidateRequest`, and `PrepareExecution` accept the same request
|
|
||||||
fields and assemble the same effective messages. They create no completed
|
|
||||||
exchange. `PrepareExecution` retains capture selection and ancestry privately
|
|
||||||
for `RunPrepared`; plain preparation only returns its ordinary metadata.
|
|
||||||
- Nil `ContinueFrom` means no history. A non-nil, zero-value exchange is not a
|
|
||||||
captured exchange and fails as `ErrInvalidRequest` before source/model work.
|
|
||||||
`Details` on a nil or zero-value exchange returns a zero details value,
|
|
||||||
matching the prepared-handle accessor pattern.
|
|
||||||
- Details contain the complete logical history ending in the final assistant
|
|
||||||
content. They are detached and encodable with ordinary encoders. Give the
|
|
||||||
opaque handle constant `String` and `GoString` representations, following
|
|
||||||
`PreparedExecution`. Serializing ordinary results does not serialize the
|
|
||||||
capability or automatically disclose the additional conversation history;
|
|
||||||
callers explicitly encode `Details()` when desired. Do not add deserialization,
|
|
||||||
persistence, reconstruction, lookup, or lifecycle APIs.
|
|
||||||
- `ParentRunID` is the immediate source exchange's originating run ID, or empty
|
|
||||||
for initial runs. Expose it in prepared details, completed results, exchange
|
|
||||||
details, and ordinary observation events. Reuse the run ID already assigned to
|
|
||||||
the execution. It is metadata, not a provider request parameter, session ID,
|
|
||||||
group ID, or new source of execution settings.
|
|
||||||
- The follow-up's `PromptID`, definition hash, selected profile/backend, and input
|
|
||||||
hashes describe the new request. The rendered-prompt hash covers its complete
|
|
||||||
effective message sequence and selected session, using the existing hash
|
|
||||||
function. Do not hash capture flags or run identities into content equality.
|
|
||||||
|
|
||||||
### Optional Coordinator
|
|
||||||
|
|
||||||
```go
|
|
||||||
type ConversationTurn struct {
|
|
||||||
Exchange *Exchange
|
|
||||||
Result *RunResult
|
|
||||||
RemainingTurns int
|
|
||||||
}
|
|
||||||
|
|
||||||
type CoordinateRequest struct {
|
|
||||||
Start *Exchange
|
|
||||||
AdditionalTurns int
|
|
||||||
Next func(context.Context, ConversationTurn) (*RunRequest, error)
|
|
||||||
Observer ExecutionObserver
|
|
||||||
}
|
|
||||||
|
|
||||||
type CoordinationOutcome string
|
|
||||||
|
|
||||||
const (
|
|
||||||
CoordinationStopped CoordinationOutcome = "stopped"
|
|
||||||
CoordinationExhausted CoordinationOutcome = "exhausted"
|
|
||||||
)
|
|
||||||
|
|
||||||
type CoordinationResult struct {
|
|
||||||
Outcome CoordinationOutcome `json:"outcome,omitempty"`
|
|
||||||
Undelivered *RunResult `json:"undelivered,omitempty"`
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e *Engine) Coordinate(ctx context.Context, req CoordinateRequest) (CoordinationResult, error)
|
|
||||||
```
|
|
||||||
|
|
||||||
- `Next` returning `(nil, nil)` stops. A non-nil request asks to continue; an
|
|
||||||
error stops with that error. No separate next-action enum or domain-outcome
|
|
||||||
taxonomy is needed. A zero request pointed to by a non-nil pointer is still
|
|
||||||
a continuation request, subject to ordinary validation if execution starts.
|
|
||||||
- `Start` must hold a captured exchange, `Next` must be non-nil, and
|
|
||||||
`AdditionalTurns` must be nonnegative; invalid values match `ErrInvalidRequest`.
|
|
||||||
A nil/uninitialized engine follows the existing `ErrInvalidConfig` convention.
|
|
||||||
Zero is a useful budget, not a default that expands into extra turns.
|
|
||||||
- Initial callback state has `Exchange: Start`, `Result: nil`, and the full
|
|
||||||
remaining budget. Later states contain the newly completed result and exchange
|
|
||||||
with the remaining start budget, including zero after the last permitted run.
|
|
||||||
The initial exchange is not presented as a newly delivered response.
|
|
||||||
- The coordinator copies each selected request value and sets its `ContinueFrom`
|
|
||||||
to the current exchange. It owns this one branch's progression, so a callback's
|
|
||||||
`ContinueFrom` does not redirect that branch. The rest of the request uses
|
|
||||||
ordinary rules. Callers wanting independent branches use `Run` themselves.
|
|
||||||
- Compose the request's observer first and the coordinator's observer second.
|
|
||||||
The latter selects all continuation runs, not the starting run or nested
|
|
||||||
callback calls. Applications explicitly compose additional group observers.
|
|
||||||
- On error leave `Outcome` empty. Set `Undelivered` only for a completed new
|
|
||||||
response withheld from callback invocation because cancellation was observed.
|
|
||||||
That result includes its exchange. A callback that was invoked has received
|
|
||||||
its response even if it errors or cancels. Return no result archive, latest
|
|
||||||
delivered response, failure-stage taxonomy, or coordinator accounting totals.
|
|
||||||
- Callbacks receive caller-owned public values. Retain the coordinator's current
|
|
||||||
immutable exchange separately, so replacing fields in delivered values cannot
|
|
||||||
rewrite retained history or the next continuation's ancestry.
|
|
||||||
|
|
||||||
### Observer Composition And Execution Groups
|
|
||||||
|
|
||||||
```go
|
|
||||||
func ComposeObservers(observers ...ExecutionObserver) ExecutionObserver
|
|
||||||
|
|
||||||
// Zero value ready for use; contains private synchronization/state only.
|
|
||||||
type ExecutionGroup struct { /* private collector */ }
|
|
||||||
|
|
||||||
func (g *ExecutionGroup) Observe(event ExecutionEvent)
|
|
||||||
func (g *ExecutionGroup) Snapshot() ExecutionGroupSummary
|
|
||||||
|
|
||||||
type ExecutionGroupMember struct {
|
|
||||||
RunID string `json:"run_id"`
|
|
||||||
TerminalObserved bool `json:"terminal_observed"`
|
|
||||||
}
|
|
||||||
|
|
||||||
type ExecutionGroupSummary struct {
|
|
||||||
Members []ExecutionGroupMember `json:"members"`
|
|
||||||
TerminalCoverageComplete bool `json:"terminal_coverage_complete"`
|
|
||||||
Attempts uint64 `json:"attempts"`
|
|
||||||
TransientRetries uint64 `json:"transient_retries"`
|
|
||||||
RepairCalls uint64 `json:"repair_calls"`
|
|
||||||
Usage ObservedTokenUsage `json:"usage"`
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
- Attach a group with `Observer: group.Observe`, or use `ComposeObservers` for
|
|
||||||
diagnostics and multiple groups. No request group field, implicit context
|
|
||||||
membership, constructor, registry, close, seal, or wait method is needed.
|
|
||||||
- Composition copies the supplied function slice, skips nil functions, and
|
|
||||||
invokes the remainder synchronously in supplied order. With no non-nil
|
|
||||||
functions return nil. Do not deduplicate functions or recover panics; a panic
|
|
||||||
prevents delivery to later observers. Closures retain their ordinary identity.
|
|
||||||
- Groups enroll a run when any ordinary event for that run is observed. The
|
|
||||||
first terminal event seen for that identity contributes counters and usage
|
|
||||||
exactly once, including when it is the first observed event. Repeated delivery
|
|
||||||
must not change totals or regress terminal coverage. Handling contradictory
|
|
||||||
caller-fabricated events is not a new public validation contract.
|
|
||||||
- Summary counters and token subtotals use terminal observations only. Pending
|
|
||||||
runs contribute no invented attempt counts or known zero usage. Members without
|
|
||||||
terminals remain identifiable. `TerminalCoverageComplete` says only that every
|
|
||||||
currently observed member has a terminal observation; it says nothing about
|
|
||||||
future members or application completion.
|
|
||||||
- For each token field, sum known terminal values; `Known` is the OR of terminal
|
|
||||||
`Known` flags and `Complete` is the AND of terminal `Complete` flags. These flags
|
|
||||||
describe received terminal subtotals; missing terminal coverage is separate.
|
|
||||||
If members exist but none has a terminal, usage is unknown/incomplete. An empty
|
|
||||||
group has complete terminal coverage and known, complete zero totals. Preserve
|
|
||||||
explicit provider zero and existing per-run failure-path usage semantics.
|
|
||||||
- Snapshots are detached and internally consistent under concurrent observation
|
|
||||||
and reads. There is no promised member ordering; tests compare by identity.
|
|
||||||
The mutex-bearing group must not be copied after first use, as with ordinary
|
|
||||||
synchronized Go values. Overlapping group totals are not generally additive.
|
|
||||||
|
|
||||||
## Internal Implementation Decisions
|
|
||||||
|
|
||||||
### Immutable History And Execution-Owned Messages
|
|
||||||
|
|
||||||
Put the internal immutable exchange value and history operations in
|
|
||||||
`internal/domain/exchange.go`. The public handle privately references this value;
|
|
||||||
internal packages never import the root facade. Retain only history and the two
|
|
||||||
run identities, not engine owners, definitions, requests, credentials, validation
|
|
||||||
plans, results, admission leases, or observer/callback closures.
|
|
||||||
|
|
||||||
Use a simple immutable parent-plus-turn representation: a parent exchange,
|
|
||||||
detached new-turn messages, the final assistant message, and a cached total
|
|
||||||
message count. Initial exchanges have no parent. New exchanges retain only their
|
|
||||||
new suffix rather than another flat copy of all ancestors. Flatten iteratively
|
|
||||||
into one correctly sized, caller/execution-owned destination when preparing a
|
|
||||||
provider request or returning details; do not recurse through arbitrary depth.
|
|
||||||
Reuse message/cache-control clone helpers at mutable boundaries. This is an
|
|
||||||
internal choice, not a public allocation or asymptotic guarantee. Avoid unrelated
|
|
||||||
history caches, ropes, eviction, limits, or pooling.
|
|
||||||
|
|
||||||
Prepare the new prompt normally, then assemble history, new definition messages,
|
|
||||||
and appended messages in the accepted order before hashing. Captured history
|
|
||||||
is not normalized or rendered again. Save capture/parent state separately in the
|
|
||||||
private prepared operation/payload. Prepared metadata contains `ParentRunID` and
|
|
||||||
the already assembled messages, not the exchange capability or capture settings.
|
|
||||||
Pass the private capture state through the shared execution function. After
|
|
||||||
successful final validation processing, capture the new-turn suffix from the
|
|
||||||
executed snapshot and the exact final `RawOutput`; failed structural validation
|
|
||||||
is still a completed result. Do not capture internal repair instructions or
|
|
||||||
intermediate candidates. On an operational error returning no result, create no
|
|
||||||
exchange. Do not change underlying execution cancellation or partial-result
|
|
||||||
rules to recover an assistant message.
|
|
||||||
|
|
||||||
### Coordinator Placement And Boundary Adapters
|
|
||||||
|
|
||||||
Put the loop in `internal/usecase/coordinate.go`, not in root business logic.
|
|
||||||
Use one small internal function generic over request/result value types, with
|
|
||||||
typed callback and execution functions and a `*domain.Exchange` current state.
|
|
||||||
Its internal turn/termination values contain only the fields projected by the
|
|
||||||
public API. This lets the facade inject ordinary `Engine.Run` and the application
|
|
||||||
callback without importing facade types internally or converting an entire run
|
|
||||||
result to domain and back just for delivery. No generic public API is added.
|
|
||||||
|
|
||||||
The execution adapter accepts `(context, currentExchange, selectedRequest)` and
|
|
||||||
returns `(completedResult, nextExchange, error)`. It creates a fresh public
|
|
||||||
exchange wrapper, copies the request, supplies the current history and composed
|
|
||||||
observer, and calls `e.Run`. Obtain the next internal exchange before delivering
|
|
||||||
the result to application code. The callback adapter projects a fresh public
|
|
||||||
wrapper and the current result into `ConversationTurn`. The shared loop does not
|
|
||||||
inspect results, interpret output, or access generation/capacity directly.
|
|
||||||
|
|
||||||
Use this exact progression:
|
|
||||||
|
|
||||||
1. With no completed response pending initially, check context before callback.
|
|
||||||
2. Invoke `Next` with the current exchange, pending result if any, and remaining
|
|
||||||
budget. Invocation marks that pending result delivered.
|
|
||||||
3. Check context again; cancellation wins over the callback's action/error.
|
|
||||||
Otherwise return a callback error unchanged, stop on nil request, or continue.
|
|
||||||
4. If a request was selected with no remaining budget, return exhaustion before
|
|
||||||
converting/preparing that request. Otherwise check context before execution,
|
|
||||||
consume one start from the budget, and call the execution adapter.
|
|
||||||
5. Return an execution error unchanged; retain no partial output. On a completed
|
|
||||||
result, update the private current exchange and repeat from step 1.
|
|
||||||
|
|
||||||
At step 1, cancellation with a pending new completed response returns that
|
|
||||||
single response in `Undelivered`; cancellation before the initial callback
|
|
||||||
returns none. Callback errors and cancellation observed after invocation return
|
|
||||||
no undelivered response. Panics propagate naturally without recovery. These
|
|
||||||
context checks do not forcibly interrupt callbacks or eliminate races.
|
|
||||||
|
|
||||||
Calling ordinary `Run` per turn ensures its logical admission and terminal
|
|
||||||
observation finish before `Next`. Do not wrap the whole coordinator in an
|
|
||||||
admission lease or an execution recorder. Nested calls remain independent.
|
|
||||||
|
|
||||||
### Group Collector Placement And Cost
|
|
||||||
|
|
||||||
Implement the collector in `internal/observation/group.go`, using domain events
|
|
||||||
and summary values. The root group is a thin adapter with private internal state;
|
|
||||||
root summary/member types are independent public declarations, not aliases.
|
|
||||||
Store a map keyed by run identity, a terminal-seen bit per member, and aggregate
|
|
||||||
terminal counters/usage under one mutex. Do not retain full event histories,
|
|
||||||
provider content, or per-attempt arrays. Event updates use map lookup rather than
|
|
||||||
rescanning all members. Snapshot copies the map's members and totals under the
|
|
||||||
lock without invoking callbacks. Observer composition is also internal logic
|
|
||||||
with thin root conversion; do not round-trip every event through both public and
|
|
||||||
domain representations simply to fan out callbacks.
|
|
||||||
|
|
||||||
## Stage 1 — Implement Immutable Exchange Storage
|
|
||||||
|
|
||||||
**Status:** Complete.
|
|
||||||
|
|
||||||
**Scope:** internal history value and ownership only.
|
|
||||||
|
|
||||||
1. Add the internal exchange representation and construction/flattening helpers
|
|
||||||
described above. Keep fields private and expose only the internal accessors
|
|
||||||
needed for run identities, message count, capture, and detached details.
|
|
||||||
2. Clone only new mutable suffix data when capturing; safely share immutable
|
|
||||||
parents. Preserve all role, content, empty-content, and cache metadata values.
|
|
||||||
3. Add focused internal domain tests for initial history, a second turn, two
|
|
||||||
independent branches, and mutation of source messages/cache control and
|
|
||||||
returned copies. Include a many-turn flattening case without asserting a
|
|
||||||
specific private layout or allocation count.
|
|
||||||
|
|
||||||
**Validation:** `go test ./internal/domain` when the package is buildable.
|
|
||||||
These tests own focused history behavior; the final suite verifies its assembly.
|
|
||||||
|
|
||||||
## Stage 2 — Add Public Handles And History-Aware Preparation
|
|
||||||
|
|
||||||
**Status:** Complete; capture state is carried through execution.
|
|
||||||
|
|
||||||
**Scope:** facade request conversion and the shared preparation path; generation
|
|
||||||
capture is the next stage.
|
|
||||||
|
|
||||||
1. Add `Exchange`, `ExchangeDetails`, their accessor and opaque formatting, the
|
|
||||||
request fields, and the prepared/result/observation ancestry fields selected
|
|
||||||
above. Add private domain request and prepared-operation capture state.
|
|
||||||
2. Extend `toDomainRunRequest` to translate a valid handle and capture choice.
|
|
||||||
Reject non-nil empty handles as invalid requests; do not impose engine binding
|
|
||||||
on exchanges. Preserve existing appended-message validation and copying.
|
|
||||||
3. Update completion to prepend captured messages before newly rendered messages
|
|
||||||
and caller suffixes. Compute hashes only after final assembly, leaving the
|
|
||||||
follow-up's settings, session, profile selection, and schema rules ordinary.
|
|
||||||
4. Update `PreparedRun` conversion and JSON behavior for optional `ParentRunID`.
|
|
||||||
Keep capture/private history references out of returned prepared details.
|
|
||||||
`ValidateRequest` inherits this through its existing preparation delegation.
|
|
||||||
5. Extend preparation contract coverage to assert exact order, roles, cache
|
|
||||||
metadata, hash sensitivity to prior history, and unchanged definition hashes.
|
|
||||||
Check nil/empty handle behavior and that preparation performs no model work.
|
|
||||||
|
|
||||||
**Validation:** focused root preparation/JSON tests and affected use-case tests.
|
|
||||||
Do not add a separate preparation or validation algorithm for continuation.
|
|
||||||
|
|
||||||
## Stage 3 — Capture Ordinary And Prepared Executions
|
|
||||||
|
|
||||||
**Status:** Complete.
|
|
||||||
|
|
||||||
**Scope:** completed-result creation and prepared payload lifetime.
|
|
||||||
|
|
||||||
1. Carry capture/parent state through `preparedOperation` and
|
|
||||||
`preparedExecutionPayload` to the shared `executePreparedRun` path. Keep it
|
|
||||||
out of the retained details object so discard can release execution-only
|
|
||||||
references. Preserve claim ownership, validation-plan freezing, direct-key
|
|
||||||
clearing, observer identity, and retry budgets.
|
|
||||||
2. Create the exchange from the request actually executed and exact final response
|
|
||||||
after the ordinary execution has a result. Initial capture is opt-in;
|
|
||||||
continuation always captures. Attach it and `ParentRunID` in result conversion.
|
|
||||||
3. Capture final failed validation/repair exhaustion as well as valid results.
|
|
||||||
On generation, validation-operation, or repair-generation errors returning no
|
|
||||||
result, create no exchange and leave the source exchange reusable.
|
|
||||||
4. Keep `RunResult.Exchange` excluded from ordinary JSON/YAML and preserve existing
|
|
||||||
result timing marshal/unmarshal behavior. Details are the explicit content
|
|
||||||
encoding path. The exchange's private state stores no execution target,
|
|
||||||
credential, result archive, callback, or lease.
|
|
||||||
5. Add focused tests for default no-capture, ordinary capture, prepared capture,
|
|
||||||
discarded/failed-claim silence, final repaired content without repair chatter,
|
|
||||||
failed validation capture, and operational-error absence of a new exchange.
|
|
||||||
Reuse existing prepared lifecycle and mutating-client fixtures.
|
|
||||||
|
|
||||||
**Validation:** targeted root capture, prepared-execution, and repair tests plus
|
|
||||||
`go test ./internal/usecase` as the touched package becomes coherent.
|
|
||||||
|
|
||||||
## Stage 4 — Propagate Ancestry Through Ordinary Observations
|
|
||||||
|
|
||||||
**Status:** Complete.
|
|
||||||
|
|
||||||
**Scope:** immediate-parent metadata without accounting changes.
|
|
||||||
|
|
||||||
1. Add parent metadata to the internal recorder's event construction and the root
|
|
||||||
event projection. Set it before ordinary start delivery and before a prepared
|
|
||||||
start after successful claim. Preserve it when selection identities update.
|
|
||||||
A small internal setter before `Start` is sufficient; do not add context-based
|
|
||||||
ancestry inheritance or new run IDs.
|
|
||||||
2. Propagate parent identity through start, attempt, retry-delay, and terminal
|
|
||||||
events. Initial runs and unrelated nested calls keep an empty parent. Preserve
|
|
||||||
terminal error classification and public error identities.
|
|
||||||
3. Cover observed ordinary/prepared continuation, preparation silence, operational
|
|
||||||
failure after continuation selection, and an independent nested call using
|
|
||||||
existing observation fixtures. Verify no history/content enters events.
|
|
||||||
|
|
||||||
**Validation:** focused observation and prepared-observation contract tests and
|
|
||||||
`go test ./internal/observation ./internal/generation`.
|
|
||||||
|
|
||||||
## Stage 5 — Verify Independent Continuation At The Public Boundary
|
|
||||||
|
|
||||||
**Status:** Complete.
|
|
||||||
|
|
||||||
**Scope:** assembled continuation behavior before adding coordination.
|
|
||||||
|
|
||||||
1. Add an external-package contract test that captures an initial exchange and
|
|
||||||
executes two branches, then continues one branch again. Assert exact outgoing
|
|
||||||
messages and immediate ancestry without changing the initial or sibling data.
|
|
||||||
2. Continue the exchange through another engine, including a scoped view. Select
|
|
||||||
different follow-up prompt/profile/output settings and session rules, and
|
|
||||||
verify that neither previous settings nor original direct credentials leak
|
|
||||||
into the new execution. Use synthetic credentials, not real environment data.
|
|
||||||
3. Remove or alter original artifact files after initial capture; continuation
|
|
||||||
still uses captured history. New follow-up inputs retain ordinary reads and
|
|
||||||
errors. Exercise prepared follow-up freezing separately from exchange reuse.
|
|
||||||
4. Mutate details and received result fields and use a mutating injected client
|
|
||||||
to prove ownership. Encode details generically and check exact final assistant
|
|
||||||
content, including empty/whitespace content when the output contract allows it.
|
|
||||||
5. Verify overlapping continuation runs can share the source exchange under the
|
|
||||||
race detector. Compare behavior, not a required scheduling order.
|
|
||||||
|
|
||||||
**Validation:** focused root continuation/capture tests under `-race`. This stage
|
|
||||||
must exercise ordinary shared preparation/execution rather than a mock history
|
|
||||||
reconstruction implemented by the test itself.
|
|
||||||
|
|
||||||
## Stage 6 — Implement The Internal Sequential Coordinator
|
|
||||||
|
|
||||||
**Status:** Complete.
|
|
||||||
|
|
||||||
**Scope:** the small generic loop and its deterministic state-machine tests.
|
|
||||||
|
|
||||||
1. Add the internal loop and turn/termination values described above, injecting
|
|
||||||
typed callback and run functions. Do not add a generic workflow framework or
|
|
||||||
a new generation collaborator.
|
|
||||||
2. Implement initial callback delivery, turn-start consumption, final-response
|
|
||||||
delivery, explicit stop versus exhaustion, and first-error termination.
|
|
||||||
3. Implement context checks and the single undelivered-response slot exactly as
|
|
||||||
specified. Do not prepare an exhausted request or replay delivered work.
|
|
||||||
4. Unit-test zero turns, stop on last response versus request for another turn,
|
|
||||||
execution and callback errors preserving `errors.Is`/`errors.As`, cancellation
|
|
||||||
before initial callback, cancellation withholding a completed result, and
|
|
||||||
cancellation/error after callback invocation. Use deterministic hooks, not
|
|
||||||
sleeps. Add one panic propagation check without terminal-delivery promises.
|
|
||||||
|
|
||||||
**Validation:** focused `go test ./internal/usecase -run Coordinate` (name tests
|
|
||||||
accordingly). The loop tests use cheap execution functions; provider behavior
|
|
||||||
remains covered by integration tests.
|
|
||||||
|
|
||||||
## Stage 7 — Expose Coordination Through The Facade
|
|
||||||
|
|
||||||
**Status:** Complete; public coordinator tests passed with observer composition.
|
|
||||||
|
|
||||||
**Scope:** public types, argument checks, and ordinary-execution adapters.
|
|
||||||
|
|
||||||
1. Add the selected coordinator types and `Engine.Coordinate`, with GoDoc for
|
|
||||||
callback state, budgets, cancellation, errors, and normal outcomes.
|
|
||||||
2. Validate coordinator arguments without looking up old resources. Project
|
|
||||||
callback state through fresh public handle wrappers. Preserve current private
|
|
||||||
exchange state independently of any callback mutation.
|
|
||||||
3. Supply the run adapter using `Engine.Run` with the receiving engine/view.
|
|
||||||
Set current history on a value copy; the coordinator does not mutate the
|
|
||||||
callback's `RunRequest`. Never add enclosing admission or an observer recorder.
|
|
||||||
4. Wire its observer selection to composition in Stage 9; do not build a second
|
|
||||||
temporary observer mechanism to make this stage independently complete.
|
|
||||||
5. Add basic public tests for invalid arguments, zero-budget initial stop,
|
|
||||||
ordinary continuation, callback error identity, and a callback modifying the
|
|
||||||
result/handle it received without changing subsequent retained history.
|
|
||||||
|
|
||||||
**Validation:** focused coordinator tests when wiring is buildable. Carry the
|
|
||||||
explicit observer-composition dependency forward, without an intermediate gate.
|
|
||||||
|
|
||||||
## Stage 8 — Verify Coordinator Capacity And Cancellation Boundaries
|
|
||||||
|
|
||||||
**Status:** Complete; focused race validation passed with observer composition.
|
|
||||||
|
|
||||||
**Scope:** the subtle assembled execution lifecycle.
|
|
||||||
|
|
||||||
1. Configure one backend with concurrency one and waiting capacity zero. Have the
|
|
||||||
callback make a same-engine/backend validation run after receiving a turn.
|
|
||||||
Assert it succeeds, proving the logical admission lease as well as the active
|
|
||||||
permit was released. Use ordinary admission, not test-only capacity bypasses.
|
|
||||||
2. Deliver a structurally invalid completed response normally, including one
|
|
||||||
whose structural repairs were exhausted. The application callback decides
|
|
||||||
whether to stop or select a new request; PromptKit adds no acceptance meaning.
|
|
||||||
3. Exercise operational failure stopping one coordinator while a sibling exchange
|
|
||||||
remains usable. Verify typed provider/capacity errors cross unchanged.
|
|
||||||
4. Trigger cancellation after a completed run but before callback delivery using
|
|
||||||
a terminal observer; retain its result/exchange once as undelivered. Separately
|
|
||||||
cancel inside the callback and assert no replay or undelivered duplicate.
|
|
||||||
5. Check that internal retry/repair attempts create only one delivery for that
|
|
||||||
continuation and do not consume extra turns. Reuse existing deterministic
|
|
||||||
HTTP/repair fixtures, not live providers.
|
|
||||||
|
|
||||||
**Validation:** focused root coordination tests under `-race`, with bounded test
|
|
||||||
contexts as failure guards, not timing-dependent synchronization.
|
|
||||||
|
|
||||||
## Stage 9 — Add Observer Composition
|
|
||||||
|
|
||||||
**Status:** Complete.
|
|
||||||
|
|
||||||
**Scope:** fan-out helper and final coordinator-observer wiring.
|
|
||||||
|
|
||||||
1. Implement internal synchronous fan-out over a copied function slice and the
|
|
||||||
root `ComposeObservers` wrapper. Keep public/domain adapters proportionate;
|
|
||||||
a small internal generic callback-composition helper can operate directly on
|
|
||||||
either event value type without repeated conversion.
|
|
||||||
2. Honor nil filtering, supplied order, no function deduplication, and ordinary
|
|
||||||
panic propagation. The helper holds no lock while invoking application code.
|
|
||||||
3. Complete coordinator observer composition, selecting each request observer
|
|
||||||
before the coordinator observer. Do not change original callback request
|
|
||||||
values. Callback-initiated calls remain explicitly selected independently.
|
|
||||||
4. Test nil/empty composition, caller mutation of the original observer slice,
|
|
||||||
ordered delivery, early panic, and simultaneous independent executions. Do
|
|
||||||
not promise stronger callback ordering across different runs.
|
|
||||||
|
|
||||||
**Validation:** focused composition and coordinator observer tests, including
|
|
||||||
the root observation-boundary tests.
|
|
||||||
|
|
||||||
## Stage 10 — Implement Terminal-Based Group Aggregation
|
|
||||||
|
|
||||||
**Status:** Complete.
|
|
||||||
|
|
||||||
**Scope:** internal collector, usage semantics, and concurrent snapshots.
|
|
||||||
|
|
||||||
1. Add domain summary/member values and the zero-ready internal group collector
|
|
||||||
under `internal/observation`. Implement map enrollment, terminal deduplication,
|
|
||||||
incremental terminal counters, and detached snapshots under one mutex.
|
|
||||||
2. Implement the selected known/completeness and coverage rules for all five
|
|
||||||
token fields. Do not sum `RunResult.Usage`, reconstruct usage from attempt
|
|
||||||
events, or convert missing terminals into zero evidence.
|
|
||||||
3. Cover empty groups, members without terminals, terminal-only enrollment,
|
|
||||||
duplicate terminal and repeated ordinary events, explicit zeros, wholly
|
|
||||||
unknown fields, partially known terminal usage, and terminal failures retaining
|
|
||||||
earlier completed usage. Use small event values rather than an HTTP test for
|
|
||||||
every arithmetic combination.
|
|
||||||
4. Concurrently observe independent runs and read snapshots, checking each
|
|
||||||
snapshot's counts, coverage, and member identities agree. Mutate a returned
|
|
||||||
snapshot and prove later snapshots are independent. Do not require snapshot
|
|
||||||
member ordering or introspect locks.
|
|
||||||
|
|
||||||
**Validation:** `go test -race ./internal/observation` with focused new group
|
|
||||||
tests. No background worker, callback invocation, or collector lifecycle is added.
|
|
||||||
|
|
||||||
## Stage 11 — Expose Groups And Verify Explicit Membership
|
|
||||||
|
|
||||||
**Status:** Complete.
|
|
||||||
|
|
||||||
**Scope:** thin public collector adapter and assembled accounting.
|
|
||||||
|
|
||||||
1. Add the public group/member/summary types and root event/summary conversions.
|
|
||||||
A private zero-ready internal group field keeps the public zero value usable.
|
|
||||||
Document synchronization, snapshot ownership, and no-copy-after-use semantics.
|
|
||||||
2. Exercise `group.Observe` alone and composed with a diagnostic observer and a
|
|
||||||
second group. Cover ordinary runs without capture, captured runs, prepared
|
|
||||||
execution, coordinator continuations, and separately selected callback calls.
|
|
||||||
3. Assert preparation/discard/failed claim creates no membership. The initial
|
|
||||||
exchange is not automatically enrolled by coordination; unselected nested
|
|
||||||
calls remain absent. Parent metadata and group membership remain independent.
|
|
||||||
4. In one assembled scenario, select initial A and branches B/C, plus a validator
|
|
||||||
into overlapping groups. Include a built-in transient retry, structural
|
|
||||||
repair, and a failure after completed usage. Check counters and known subtotals
|
|
||||||
once per identity, including replay usage reported by the provider for B/C.
|
|
||||||
5. A composed observer panic may leave an observed member without a terminal.
|
|
||||||
Verify snapshots remain usable, expose missing coverage, and do not manufacture
|
|
||||||
completion. Snapshotting inside an observer is safe because group locks are
|
|
||||||
not held while calling other observers.
|
|
||||||
|
|
||||||
**Validation:** focused root group/coordinator/observation tests under `-race`.
|
|
||||||
Reuse existing reliable-execution fixtures and avoid duplicating their complete
|
|
||||||
retry/repair matrices.
|
|
||||||
|
|
||||||
## Stage 12 — Demonstrate The Consumer Workflows
|
|
||||||
|
|
||||||
**Status:** Complete.
|
|
||||||
|
|
||||||
**Scope:** practical API use and one maintained offline example.
|
|
||||||
|
|
||||||
1. Extend `examples/go-library/observe` and its semantic assertions to demonstrate
|
|
||||||
captured proposal history, a coordinator follow-up, an application-owned
|
|
||||||
assessment call, and explicit group selection alongside diagnostic observation.
|
|
||||||
Keep its deterministic transport and existing retry/usage protections; no paid
|
|
||||||
calls or real credentials are needed.
|
|
||||||
2. The example's assessment remains application policy, not a library validator
|
|
||||||
hook. Use a closure to retain assessed state, and show stop versus exhausted
|
|
||||||
budget without returning an application verdict from PromptKit.
|
|
||||||
3. Add concise consumer-guide examples for direct branching and prepared capture,
|
|
||||||
plus both roadmap sketches translated to the selected callback shape. The
|
|
||||||
analysis sketch records a new result then asks another question or returns nil.
|
|
||||||
The Audita-shaped sketch uses existing findings initially, assesses each new
|
|
||||||
result, and either returns nil or an ordinary follow-up request. Domain-specific
|
|
||||||
functions remain illustrative caller code.
|
|
||||||
4. Show group use with ordinary/prepared requests and `CoordinateRequest.Observer`,
|
|
||||||
including explicit selection for a nested validator and an independent group
|
|
||||||
summary. Demonstrate checking structural validation before schema-specific
|
|
||||||
decoding; stop/exhaustion does not imply content acceptance.
|
|
||||||
|
|
||||||
**Validation:** `go test ./examples/go-library/...` and affected public example
|
|
||||||
tests. Review snippets against actual declarations; full copyable setup stays in
|
|
||||||
the maintained example rather than duplicated in prose.
|
|
||||||
|
|
||||||
## Stage 13 — Finalize Contracts, Documentation, And Sprint Validation
|
|
||||||
|
|
||||||
**Status:** Complete. Full offline race suite, vet, and diff checks passed with Go 1.27.3.
|
|
||||||
|
|
||||||
**Scope:** complete the target and remove sprint-only rough edges.
|
|
||||||
|
|
||||||
1. Review every new public declaration and GoDoc against this plan and the feature
|
|
||||||
roadmap. Confirm ordinary APIs still use existing error, timing, prepared-claim,
|
|
||||||
credential, namespace, usage, and cancellation behavior. Verify result/details
|
|
||||||
serialization and constant opaque formatting without introducing a restartable
|
|
||||||
exchange format or redacting caller-authored message content.
|
|
||||||
2. Update the canonical consumer guide, `doc.go` where orientation needs it, and
|
|
||||||
affected internal runner, observation, and component-overview documents.
|
|
||||||
Document capacity release for conversation callbacks separately from ordinary
|
|
||||||
per-attempt observer callback constraints. Configuration and integration docs
|
|
||||||
need only changes necessary to describe the implemented message composition;
|
|
||||||
ancestry is not an upstream wire field or configuration setting.
|
|
||||||
3. Remove temporary helpers, placeholder implementations, unused experimental
|
|
||||||
API choices, and intermediate compile/test failures. Do not remove unrelated
|
|
||||||
planned/deferred work or publish a release in this sprint. Release-note/version
|
|
||||||
selection and roadmap retirement can follow during release preparation.
|
|
||||||
4. With the newest locally available Go toolchain selected as the development
|
|
||||||
guide prescribes, format changed Go files, then run:
|
|
||||||
|
|
||||||
```sh
|
|
||||||
go test -race ./...
|
|
||||||
go vet ./...
|
|
||||||
git diff --check
|
|
||||||
```
|
|
||||||
|
|
||||||
The full suite includes the maintained examples; do not add a redundant
|
|
||||||
separate build or provider connectivity gate. Check added/changed local
|
|
||||||
documentation links, using the existing tooling where practical rather than
|
|
||||||
introducing another release validator.
|
|
||||||
5. Review the complete diff and Git status for unintended files or credentials.
|
|
||||||
Confirm no new dependency, workspace override, configuration parser, provider
|
|
||||||
parameter, or strengthened restriction/guarantee slipped into the work.
|
|
||||||
Commit completed units with plain-English messages under the agent policy.
|
|
||||||
|
|
||||||
**Completion evidence:** the implemented facade supports ordinary/prepared
|
|
||||||
capture, cross-engine independent continuations, coordinator callbacks after
|
|
||||||
admission release, cancellation's single undelivered response, and independent
|
|
||||||
observer-based groups. Public behavior tests cover those boundaries and terminal
|
|
||||||
usage completeness; the complete offline race suite and vet pass. No product or
|
|
||||||
API decisions are deferred to an implementation stage.
|
|
||||||
@@ -1,177 +0,0 @@
|
|||||||
# Conversation Continuation Review Remediation
|
|
||||||
|
|
||||||
## Review Scope
|
|
||||||
|
|
||||||
Reviewed all 13 stages of the [implementation plan](implementation.md) against
|
|
||||||
the [accepted target](conversation-continuation.md), implementation, tests, and
|
|
||||||
documentation. Baseline: `5c135c5`, including the implementation commits after
|
|
||||||
`5cc3fa0`.
|
|
||||||
|
|
||||||
The implementation retains the intended boundaries: immutable parent-linked
|
|
||||||
history, ordinary preparation and execution for continuations, a separate
|
|
||||||
sequential coordinator, and explicitly selected observer-based groups. No runtime
|
|
||||||
correctness defect or inadvertently strengthened restriction was identified.
|
|
||||||
The initial finding concerns the durable public documentation in Stage 13.
|
|
||||||
|
|
||||||
Validation completed with the locally installed Go 1.27.3 toolchain,
|
|
||||||
`GOTOOLCHAIN=local`, `GOWORK=off`, and a writable temporary build cache:
|
|
||||||
|
|
||||||
- `go test -race ./...` passed, including the maintained examples.
|
|
||||||
- `go vet ./...` passed.
|
|
||||||
- `git diff --check` passed.
|
|
||||||
|
|
||||||
### Algorithmic Follow-Up
|
|
||||||
|
|
||||||
A subsequent static audit of all 13 stages, at the same implementation baseline,
|
|
||||||
identified the two efficiency opportunities below. No additional ordinary
|
|
||||||
correctness bug or avoidable quadratic-time algorithm was found. No real-world
|
|
||||||
performance data was collected.
|
|
||||||
|
|
||||||
The planned algorithms remain fixed: suffix-only immutable capture, iterative
|
|
||||||
history flattening, sequential coordination, synchronous observer delivery, and
|
|
||||||
map-based incremental group aggregation. The findings address extra allocation
|
|
||||||
and copying around those choices, not replacements for them. Complexity statements
|
|
||||||
describe implementation costs, not new public performance guarantees.
|
|
||||||
|
|
||||||
For orientation, capture is linear in the new turn's message count; flattening is
|
|
||||||
linear in retained message count; coordinator bookkeeping is constant per turn,
|
|
||||||
excluding application callbacks and ordinary execution. Group updates are
|
|
||||||
expected constant time per event for fixed-length run identities, with linear
|
|
||||||
membership storage and linear full snapshots. An observer dispatch is linear in
|
|
||||||
its selected callbacks. These implementations do not rescan all prior turns or
|
|
||||||
group members on each incremental update.
|
|
||||||
|
|
||||||
## 1. Complete The Public Exchange Capture And Continuation Contract
|
|
||||||
|
|
||||||
**Priority:** P2. **Status:** Complete. **Plan stage:** 13.
|
|
||||||
|
|
||||||
**Completion:** Expanded the public request, exchange, details, and prepared-handle
|
|
||||||
GoDoc and linked the consumer walkthrough to those owners. Checked comments
|
|
||||||
against capture, preparation, and repair behavior; local links and diff verified.
|
|
||||||
|
|
||||||
**Locations:** [request fields](../../types.go),
|
|
||||||
[exchange declarations and accessor](../../exchange.go), and
|
|
||||||
[consumer guidance](../consumers/pkg-promptkit.md#capture-and-continue-a-conversation).
|
|
||||||
|
|
||||||
The `CaptureExchange` and `ContinueFrom` GoDoc currently provides only a short
|
|
||||||
description of opt-in capture, invalid handles, and fresh follow-up settings.
|
|
||||||
The `Exchange` and `Details` comments describe reuse and detached history, but do
|
|
||||||
not define its exact contents or explain that inspection uses only retained
|
|
||||||
state. Important selected semantics remain documented only in temporary roadmaps
|
|
||||||
or prose/internal guidance:
|
|
||||||
|
|
||||||
- History contains the effective original request and exact final assistant
|
|
||||||
content, preserving roles and cache metadata; intermediate structural-repair
|
|
||||||
candidates and corrective instructions are excluded.
|
|
||||||
- Continuation assembles captured history, rendered follow-up definition
|
|
||||||
messages, and caller-appended messages in that order, without re-rendering or
|
|
||||||
reading the original inputs. It adds no history-size limit or truncation.
|
|
||||||
- `PrepareExecution` retains capture selection for execution of its snapshot;
|
|
||||||
preparation and concrete-request validation alone produce no completed exchange.
|
|
||||||
- `Details` reads the retained snapshot without rendering, filesystem reads, or
|
|
||||||
provider calls. Its run identities identify the originating execution and
|
|
||||||
immediate parent execution.
|
|
||||||
|
|
||||||
These behaviors are implemented. The gap is that the exported declarations and
|
|
||||||
GoDoc do not yet own their complete contract as required by the documentation
|
|
||||||
policy and the final implementation stage. Retiring the roadmap would otherwise
|
|
||||||
leave consumers and future maintainers without a complete canonical reference.
|
|
||||||
|
|
||||||
**Required remediation:** Document these existing semantics at the appropriate
|
|
||||||
request fields, exchange type/details, and prepared-execution declarations, using
|
|
||||||
GoDoc cross-references instead of repeating the same contract across declarations.
|
|
||||||
Link the new consumer walkthrough to those public owners. Preserve the selected
|
|
||||||
scope: do not add persistence, limits, stronger serialization guarantees, or
|
|
||||||
private storage/allocation promises while documenting the behavior.
|
|
||||||
|
|
||||||
**Verification:** Check the comments against the existing capture, continuation,
|
|
||||||
prepared-execution, ownership, and repair tests and implementation. Validate the
|
|
||||||
affected documentation links and diff. This is documentation remediation; it
|
|
||||||
does not require changing runtime behavior or duplicating existing tests.
|
|
||||||
|
|
||||||
## 2. Avoid Intermediate History Copies During Assembly And Inspection
|
|
||||||
|
|
||||||
**Priority:** P3. **Status:** Complete. **Plan stages:** 2 and 5.
|
|
||||||
|
|
||||||
**Completion:** Preparation now fills one history-plus-suffix buffer. Inspection
|
|
||||||
projects indexed history directly into public messages, cloning exposed cache
|
|
||||||
metadata once. The parent-linked representation and public types are unchanged.
|
|
||||||
Focused race-enabled history, continuation, preparation, and ownership tests passed.
|
|
||||||
|
|
||||||
**Locations:** `completePreparationWithStructuredOutput` in
|
|
||||||
[the runner](../../internal/usecase/runner.go), `Exchange.Messages` in
|
|
||||||
[history storage](../../internal/domain/exchange.go), and
|
|
||||||
[`Exchange.Details`](../../exchange.go) with `fromDomainRenderedMessages` in
|
|
||||||
[facade conversion](../../convert.go).
|
|
||||||
|
|
||||||
Let H be the captured history's message count and S the newly rendered and
|
|
||||||
caller-appended message count. Continuation preparation currently:
|
|
||||||
|
|
||||||
1. allocates and copies S messages into the new suffix;
|
|
||||||
2. materializes H history messages into a detached intermediate slice; and
|
|
||||||
3. allocates the final H + S slice and copies both intermediate slices again.
|
|
||||||
|
|
||||||
Thus assembly copies 2H + 2S message entries, rather than the H + S needed to
|
|
||||||
populate the final execution-owned buffer. Each pass also clones non-nil
|
|
||||||
cache-control structs. The public `Details` path similarly materializes a
|
|
||||||
detached domain history and then traverses it again to allocate public messages
|
|
||||||
and clone cache controls for a second time. Strings share immutable content;
|
|
||||||
these are message-header and cache-struct copies, not duplicate text-byte buffers.
|
|
||||||
|
|
||||||
**Complexity:** Current and proposed assembly are both Θ(H + S) in time and
|
|
||||||
auxiliary space; details remain Θ(H). The opportunity removes an extra linear
|
|
||||||
pass and temporary allocations, not an asymptotic quadratic defect. Successive
|
|
||||||
full-history requests still entail work proportional to the sum of their history
|
|
||||||
lengths. This audit does not propose truncation or a different
|
|
||||||
conversation protocol.
|
|
||||||
|
|
||||||
**Potential refactoring:** Add an internal fill/flatten helper that writes history
|
|
||||||
and the new message parts directly into one correctly sized final domain slice.
|
|
||||||
For inspection, consider an internal indexed traversal/projection helper that
|
|
||||||
fills the public destination without first constructing a complete domain slice.
|
|
||||||
Keep the immutable parent-linked representation, separate public types, and
|
|
||||||
detachment of every exposed mutable cache-control value. Prioritize the simple
|
|
||||||
assembly change; avoid adding a general projection framework solely to eliminate
|
|
||||||
the inspection intermediate.
|
|
||||||
|
|
||||||
**Verification:** Reuse history ordering, cache-control mutation, branching,
|
|
||||||
prepared-execution, and mutating-client tests. Review the copy/allocation paths
|
|
||||||
statically. Do not add public allocation guarantees or exact-allocation tests.
|
|
||||||
|
|
||||||
## 3. Size Observer Storage For Selected Callbacks
|
|
||||||
|
|
||||||
**Priority:** P3. **Status:** Complete. **Plan stage:** 9.
|
|
||||||
|
|
||||||
**Completion:** Observer composition counts selected callbacks before allocating,
|
|
||||||
sizes retained storage to that count, and handles zero/single selections without
|
|
||||||
a fan-out closure. Sparse-selection ownership and duplicate-delivery tests pass.
|
|
||||||
The complete offline race suite, maintained examples, vet, and diff checks passed
|
|
||||||
with Go 1.27.3 after all remediations.
|
|
||||||
|
|
||||||
**Location:** [`Compose`](../../internal/observation/compose.go).
|
|
||||||
|
|
||||||
`Compose` allocates `selected` with capacity equal to every supplied observer
|
|
||||||
slot before filtering nil callbacks. The returned closure retains that backing
|
|
||||||
array even when only a small subset of slots survives. An all-nil input also
|
|
||||||
allocates the full temporary array before returning nil.
|
|
||||||
|
|
||||||
Let K be the number of supplied slots and A the number of non-nil callbacks,
|
|
||||||
including intentionally repeated callbacks. Construction is Θ(K) time with
|
|
||||||
Θ(K) allocated storage, and a nonempty composition retains Θ(K) storage.
|
|
||||||
Dispatch is already Θ(A) and does not need changing. Selected-function storage
|
|
||||||
can instead be Θ(A), with constant storage for A = 0, while preserving Θ(K)
|
|
||||||
construction time.
|
|
||||||
|
|
||||||
**Potential refactoring:** Count non-nil callbacks first, then allocate exactly A
|
|
||||||
slots and fill them in order. This trades a second linear scan for avoiding
|
|
||||||
storage proportional to disabled slots. Alternatively, append into an initially
|
|
||||||
nil slice for a simpler one-pass filter with amortized Θ(K) construction and
|
|
||||||
Θ(A) retained storage, accepting growth allocations for dense inputs. A
|
|
||||||
zero/single-callback fast path can also avoid an unnecessary fan-out closure.
|
|
||||||
The usual coordinator supplies only two slots, so its benefit is small; this is
|
|
||||||
a low-priority general-helper scalability opportunity, not a release blocker.
|
|
||||||
|
|
||||||
**Verification:** Preserve copied function values, nil skipping, duplicate
|
|
||||||
delivery, supplied order, and ordinary panic propagation. Existing composition
|
|
||||||
and prepared-observer ownership tests protect those contracts; no benchmark or
|
|
||||||
new performance gate is required.
|
|
||||||
Reference in New Issue
Block a user