promptkit/docs/roadmap/conversation-continuation.md

# Conversation Continuation And Execution Groups

Status: target behavior accepted; ready for API and staged implementation planning.
No implementation is authorized by this document. Exact public types and
signatures remain to be designed. Execution groups are independently deliverable.

## Purpose And Responsibility Boundary

PromptKit will let applications continue logical conversations, optionally
coordinate successive turns, and summarize explicitly selected executions.
These capabilities remove repeated history reconstruction, turn lifecycle work,
and observation aggregation without interpreting prompt or response content.

[Audita's request](audita-request.md) supplies consumer evidence and acceptance
scenarios. Repair, analysis, clarification, and branching exploration should all
use the same content-neutral capabilities. New concepts and names must be useful
without translating an application's domain into repair terminology. This does
not rename or expand PromptKit's existing structural-output repair functionality.

| Capability | PromptKit owns | Application owns |
| --- | --- | --- |
| Captured exchanges and continuation | Logical history, reusable branches, ordinary follow-up execution, exchange details and parent metadata | Which exchange to continue and which request to execute |
| Optional sequential coordinator | One branch's turn progression, additional-turn budget, callback delivery and cancellation checks | Response interpretation, next action, branch scheduling and broader failure policy |
| Optional execution groups | Explicitly selected observations, deduplication and execution-fact summaries | Membership selection, domain identifiers, final-report timing and presentation |

Continuation is independently usable. The coordinator builds on it; applications
can instead use their own loops. Groups work with ordinary runs as well as
continuations and require neither capture nor a coordinator. Selecting a group
does not change scheduling, capacity, cancellation, retries, or validation.

The scope is in-process use. Persistence, restartable workflows, cross-process
resumption, a conversation registry, batch reconciliation, and a branching-workflow
scheduler are outside this feature. Applications schedule branches, join results,
and decide whether one branch's failure should affect other work. This initial
scope is not a permanent prohibition on future capabilities.

## Captured Exchanges And Continuation

### Capture And Availability

Capture is opt-in per ordinary or prepared execution. Ordinary execution returns
the exchange alongside its usual result. A prepared handle retains the capture
choice; executing that exact prepared snapshot creates the exchange. Preparation
alone creates no completed exchange. Continuation implies capture so its result
can itself be continued.

Capture the request actually executed, including the complete rendered messages
with their content, roles, and cache metadata, followed by the full final
assistant content returned by the run. Consumers must not reconstruct the request
or pair a response with separately prepared data. “Full” means content supported
by PromptKit, not a new raw provider-envelope API.

Capture represents logical conversation history. Internal structural-repair
instructions and intermediate candidates are excluded; only the final returned
assistant content follows the original request. Existing attempt observations
remain separate from conversation content.

| Execution outcome | Capture behavior |
| --- | --- |
| Completed response satisfying its output contract | Return the captured exchange. |
| Completed response with failed structural validation, including repair exhaustion | Return the exchange with the ordinary validation result intact. |
| Operational error with no completed result | Create no new exchange; the preceding exchange remains usable. |
| Completed response obtained before cancellation prevents coordinator delivery | Preserve that single undelivered response under the coordinator return contract. |

Capture does not imply structural validity or application acceptance. A completed
response with failed validation follows the normal coordinator callback path,
subject to cancellation; it is not reclassified as an operational error.

### Ownership And Reuse

An exchange is an immutable, reusable snapshot held by the caller. Continuing it
adds the rendered follow-up messages and final response to a new exchange, leaving
the source and independent branches unchanged. No previous templates or input
files need to be read or rendered again.

Avoid unnecessary repeated copying of retained history where practical, while
preserving independent branches and detached inspection values. Internal sharing
of immutable history is an implementation option, not a required representation
or a public allocation or complexity guarantee.

Another engine instance in the same process can continue the exchange, applying
its own configuration to the new request. The originating engine is not required.
This permits application engine replacement without promising serialized or
cross-process resumption.

Completed exchanges retain no backend admission lease, direct credential, or
execution callback. Ordinary Go memory management controls their lifetime; there
is no new `Close` or `Discard` lifecycle. The existing
[prepared execution handle](../../prepared_execution.go) remains a separate,
single-use execution capability.

### Exchange Details

Provide explicit access to the held exchange's logical message history, including
its final assistant content, originating run identity, and immediate parent
reference. Reuse existing run identifiers; no mandatory conversation, branch, or
sequence identifiers are introduced.

Follow the general pattern of `PreparedExecution.Details()`. Details must be
generically renderable through ordinary encoders. Reading or modifying returned
values must not affect retained history or another branch. Inspection reads the
snapshot without rendering, loading inputs, or making provider calls.

Existing results and preparation details continue to supply ordinary execution
information. Exchange inspection does not create a second comprehensive report,
a result archive, or an engine-level conversation lookup API. `InspectPrompt`,
`InspectProfile`, and `InspectBackend` keep their resource-inspection boundary.
Captured message content remains separate from metadata-only observations.

## Optional Sequential Coordinator

The coordinator starts from a captured exchange and executes one branch using
an application callback, a caller-selected additional-turn budget, and context.
The callback selects a new ordinary request, stops, or returns an error. It can
keep application state in a closure; PromptKit defines no domain-state or
feedback schema.

### Progression And Budget

With an active context, invoke the initial callback once, even with a zero-turn
budget. It receives the starting exchange and remaining budget; the starting
exchange is not a newly delivered response. Existing application findings can
be used without repeating earlier assessment.

After each continuation, deliver its completed response once through the same
callback before starting another execution, including the final permitted
response. Internal retries and structural repairs do not create extra callbacks.

| Callback action | Coordinator behavior |
| --- | --- |
| Stop | Return application-directed completion, including after the last permitted response. |
| Continue with budget remaining | Start the selected continuation. |
| Continue without budget remaining | Return exhaustion without executing the requested turn. |
| Error | Stop and propagate the error, subject to context checks. |

The budget permits at most N additional continuation executions to start.
Starting a continuation consumes one turn even if preparation or generation
fails. The initial exchange and callback invocations consume no turns; internal
transport retries and structural repairs stay within one turn. The first
operational error stops coordination, without refunds or recovery machinery.

Using the full budget alone does not mean exhaustion: exhaustion means the
application requested more work than the budget permits. The budget does not
bound provider attempts, cost, or work inside callbacks. Applications wanting
no callback work can skip the coordinator entirely.

### Cancellation And Failures

Check context immediately before every callback, including the initial callback.
If cancellation is already observed, skip delivery and return the context error
with its identity preserved. Retain a completed response already obtained but
not yet delivered; do not recover raw output from an underlying failed call that
returned no completed result.

Callbacks receive the same context and cooperate with cancellation. Check it
again after the callback returns and before starting another execution.
Cancellation may race with invocation; PromptKit does not forcibly terminate
callback code or promise callback delivery despite cancellation or panic.

An execution or callback error stops coordination. Preserve ordinary Go error
identities, including `errors.Is`/`errors.As`. Do not automatically replay the
failed continuation or callback. Ordinary transport retries and structural repair
remain execution-internal. Earlier deliveries and application work remain intact;
the application owns any broader consequences.

### Minimal Return Contract

Return only:

- An ordinary Go error for execution failure, callback failure, or cancellation.
- A small normal outcome distinguishing application-directed stop from budget
  exhaustion; neither outcome is an error or a claim about content acceptance.
- An optional single completed response when cancellation prevented callback
  delivery, reusing existing response types where practical.

Delivery means callback invocation, not successful processing or acceptance. If
the callback returns an error, its response is already delivered and is neither
returned as undelivered nor replayed. Previously delivered responses are not
archived in the termination result. No additional failure hierarchy, failure-stage
fields, run-ID references, or aggregate reporting is added to this contract.
Existing response and observation fields remain available; group summaries are
obtained separately.

## Shared Execution And Accounting Rules

### Follow-Up Configuration

A continuation is an ordinary new execution supplied with captured history.
Resolve its prompt, profile, credentials, output contract, and request overrides
using the receiving engine's existing rules. In particular:

- Use an explicit request profile or the follow-up prompt's default profile,
  with ordinary missing-profile behavior when neither is available.
- A supplied output contract completely replaces the default; do not merge it
  with or inherit the preceding execution's contract.
- Resolve credentials for the new execution; retained history does not preserve
  the original direct key.
- Select session ID from the new request or follow-up prompt's session template.
  Do not implicitly inherit it. Applications may explicitly reuse a session ID;
  ancestry does not depend on one.

History introduces no additional execution-setting precedence. The follow-up
can change model or response schema without altering the original exchange.

Assemble each continuation's messages in this order:

1. The captured logical history, including its final assistant content.
2. The rendered follow-up prompt definition messages, in definition order.
3. The follow-up request's caller-appended messages, in caller order.

Preserve authored roles and message content; do not move messages based on role.
Continuation adds no history-size limit or automatic truncation. Upstream
context-window or other request rejections follow ordinary generation-error
behavior.

### Capacity And Application Work

Compose completed executions. Finish each turn and release its logical admission
lease before calling application code. Releasing only the active generation
permit is insufficient under the existing [run flow](../internal/runner.md#run-flow).
Each later execution acquires ordinary admission.

The coordinator holds no backend capacity while awaiting callback work.
Callbacks may inspect responses, consult data, wait for application input, or
call the same engine and backend, including when backend capacity is one.

### Ancestry And Per-Run Accounting

Each continuation retains its own originating run ID and exposes its immediate
parent exchange's run ID. Parent metadata describes history provenance; group
membership describes application-selected accounting. Neither implies the other.
A validator can join a continuation's accounting group without extending its
conversation.

Reuse ordinary per-run usage and observations, including attempt, retry, repair,
failure-path, and usage-presence/completeness semantics. Preserve the existing
[observation boundary](../internal/observation.md): arbitrary nested calls are
independent executions. Do not infer membership from a callback's purpose or
context.

If initial run A supplies branches B and C, its usage belongs only to A. B and C
do not inherit that cost. Usage reported for resending captured history in B and
C still belongs to those new runs. Count selected runs once each; never add
terminal totals to the attempt events they already summarize.

## Optional Execution Groups

### Explicit Membership And Observer Composition

Use an observer-compatible collector attached explicitly to selected executions.
Prepared requests retain the selection; membership starts when execution emits
observations. Preparation alone, discarded handles, and failed claims do not
enroll runs through observations.

Coordinator selection covers its continuation runs. Callback-initiated calls
require their own explicit selection. Neither nested context nor a captured
starting exchange automatically enrolls another run.

Support an application's existing diagnostic observer alongside one or more
collectors through a small observer-composition helper. Preserve synchronous
delivery and existing panic behavior. Do not introduce asynchronous queues,
panic recovery, or guaranteed delivery to later observers after an earlier one
panics; interrupted delivery can leave accounting incomplete.

A run may explicitly belong to multiple independent groups. Each group counts
its run identity once. Applications obtain operation-level and overall views by
attaching the corresponding collectors to each selected run. Overlapping totals
are not generally additive; no group hierarchy, inherited membership, or merging
facility is needed.

### Concurrent Snapshots And Lifetime

Provide detached, internally consistent snapshots from a caller-held group while
selected executions and snapshot reads may occur concurrently. Reading or
modifying a snapshot must not change the collector or another snapshot. There
is no mandatory close, seal, or wait lifecycle. Applications own scheduling and
decide when to read or discard the group.

Distinguish observed runs with terminal observations from those without them.
Missing terminal observations mean incomplete accounting; the group does not
infer whether work is active, delivery was interrupted, or another cause applies.
It promises no terminal delivery beyond ordinary observation semantics.

“All observed members have terminal observations” does not mean the application
operation is complete. More runs can join later, and PromptKit cannot know whether
the application selected all relevant calls. The application chooses when a
snapshot represents its final report.

### Summary Contents

The initial summary exposes:

- Member run identities and terminal-observation coverage.
- Existing provider-attempt, transient-retry, and structural-repair counters.
- Available token totals with per-field known/completeness information.

Aggregate one terminal observation per run. These observations already summarize
retries, repairs, and available failure-path usage, including executions returning
no ordinary result. Do not reconstruct totals from attempts.

Keep terminal coverage distinct from usage completeness. A terminal observation
may itself have incomplete usage. Two finished runs with fully reported usage
provide a useful known subtotal, but a third observed run lacking a terminal
event leaves group accounting incomplete. Preserve existing usage semantics and
expose missing coverage separately; unavailable evidence must not become known
zero.

Live attempt-level totals and duration aggregation are omitted from the initial
implementation. Applications can use existing events for progress displays. If
durations are later added, summed execution durations must be distinguished from
elapsed group time. Groups supply execution facts, not monetary cost, application
outcomes, or an application-wide reporting framework. Applications retain domain
mappings, interpretation, and report assembly.

## Consumer Sketches

These are pseudocode, not selected exported names or implemented Go APIs.
Each begins with an already captured exchange and uses ordinary follow-up request
selection. Both omit application-specific error handling, including checks for
failed structural validation before schema-specific decoding.

### Analysis Loop

The application records responses and selects its next question or decides that
its investigation is complete. PromptKit retains history and executes the turns.

```text
analysis = application.start_analysis(initial_exchange)

choose_next(ctx, turn):
    if turn.new_result exists:
        analysis.record(turn.new_result)

    question = application.choose_next_question(analysis)
    if question is absent:
        return Stop()

    return Continue(request(
        prompt_id = "research/investigate",
        vars = {"question": question}
    ))

termination, error = coordinate(
    ctx, start = initial_exchange,
    additional_turns = 3, next = choose_next
)
application.present_analysis(analysis, termination, error)
```

### Audita Branch

Audita starts with an already-assessed candidate and existing findings. Only
Audita interprets acceptance, withdrawal, or assessment unavailability.

```text
branch = audita.branch(candidate, existing_findings)

choose_next(ctx, turn):
    if turn.new_result exists:
        response = audita.decode_response(turn.new_result)
        if audita.is_withdrawal(response):
            branch.mark_withdrawn()
            return Stop()

        branch.update_candidate(response)
        assessment = audita.assess(ctx, engine, branch.candidate)
        branch.record(assessment)
        if audita.should_stop(assessment):
            return Stop()

    return Continue(request(
        prompt_id = branch.configured_follow_up_prompt,
        vars = audita.follow_up_variables(branch)
    ))

termination, error = coordinate(
    ctx, start = initial_exchange,
    additional_turns = 3, next = choose_next
)
audita.record_branch_outcome(branch, termination, error)
```

The initial callback uses existing findings without repeating batch assessment.
`audita.assess` may call the same backend after the turn's admission lease is
released. Those calls join any group only through their own explicit selection.
Other candidates can start independent branches; Audita owns sibling outcomes.

Both sketches process the final permitted response through the normal callback.
Applications retain their received results; the coordinator's termination value
is not an archive. These examples should remain usable through the same generic
API without manual history assembly or per-turn lifecycle bookkeeping.

## API Planning And Delivery

No product-level decisions remain open. The companion implementation plan should
settle exact public types and signatures before implementation, within the
accepted behavior above:

- Capture opt-in syntax and exchange representation. Prefer a request option;
  separate capture methods remain viable. Prefer a details accessor, while a
  public value is viable if it preserves ownership and generic rendering.
- Continuation request shape and placement of immediate-parent metadata, using
  the message assembly order defined above.
- Callback state, next-action types, normal outcomes, and the optional
  undelivered-response result.
- Collector attachment, observer composition, and summary field representation.
  Start with the observer-compatible API. Explicit group fields remain a fallback
  only if concrete ordinary/prepared/coordinator/callback usage sketches expose
  awkwardness; either approach uses existing observation machinery.

Check the concrete API against both consumer sketches and ordinary and prepared
execution, including capture selection, exchange access, coordinator actions and
results, and observer composition. Keep these choices consistent across the
staged plan rather than leaving each implementation stage to choose its own
public contract.

Deliver in separable stages:

1. **Capture and independent continuation:** expose execution-derived exchanges,
   reusable branches, fresh follow-up settings, parent metadata, and details.
   This lets consumers remove reconstruction and prefix-verification machinery.
2. **Optional sequential coordinator:** build on continuation and ordinary
   execution, validation, capacity, and observations. Refine both consumer
   sketches against the concrete API.
3. **Optional execution groups, on an independent track:** provide collection,
   composition, snapshots, and aggregation. This can ship separately and need
   not wait for or delay continuation or coordination.

These stages guide a subsequent implementation plan; they do not authorize code
changes or impose an ordering dependency on execution groups.

## Acceptance Scenarios

Use [Audita's acceptance scenarios](audita-request.md#suggested-delivery-and-acceptance-scenarios)
alongside analysis and branching-conversation examples. Validate each capability
at its public boundary without adding guarantees beyond the selected behavior.

### Capture And Continuation

- Ordinary and prepared opt-in capture uses the actual executed snapshot;
  preparation alone produces no exchange.
- Internal structural repair contributes only the full final assistant content
  to history. Final validation failure still permits capture; an operational
  failure without a completed result does not.
- Independent branches preserve prior history and outcomes, including after
  original inputs change or disappear and when continued by another engine.
- Follow-up configuration, credentials, session selection, and complete output
  overrides follow ordinary rules without implicit inheritance.
- Details encode generically, expose originating and parent identities, and are
  detached from retained state; inspection performs no rendering or external reads.

### Coordination

- Initial callback, zero-turn behavior, last-response delivery, stop versus
  exhaustion, and execution-start budget consumption follow the specified rules.
- Cancellation before delivery retains only an already-obtained undelivered
  response; callback errors do not replay delivered responses or earlier work.
- Failed structural validation follows normal response delivery, while operational
  errors preserve Go error identities and stop progression.
- Same-backend callback calls at capacity one proceed after logical admission
  release. Internal retries/repairs do not create extra conversation callbacks.
- Both consumer sketches work without PromptKit interpreting content or owning
  application outcomes, branch scheduling, or a response archive.

### Observations And Groups

- Per-run usage and failure diagnostics preserve ordinary presence/completeness
  semantics and parent references distinguish turns and branches.
- Explicit ordinary, prepared, coordinator, and callback membership works alongside
  an existing observer; unrelated nested calls remain independent.
- Concurrent collection and snapshots preserve detached, consistent results.
  More members may join after a snapshot with full terminal coverage.
- Overlapping groups count each run once within each group. Shared initial work,
  newly reported replay usage, and attempt/terminal events are not double counted.
- Known subtotals, explicit zeros, incomplete usage, and missing terminal coverage
  remain distinguishable, including on failure paths.
- Observer composition preserves synchronous delivery and panic behavior without
  stronger terminal-delivery promises.