promptkit/docs/roadmap/conversation-continuation.md

# Conversation Continuation And Application-Directed Interaction

Status: feature roadmap under discussion; accepted directions and remaining
open decisions are identified below. No implementation is authorized by this
document. Recommendations remain proposals unless explicitly marked accepted.

## North Star: Content-Neutral, Generally Useful Capabilities

**Accepted direction:** evaluate each capability for its usefulness across
different downstream applications. PromptKit must be able to implement it and
explain its value without assuming or inferring the meaning of the content
passed through prompts and responses.

PromptKit can own conversation and execution mechanics: captured messages,
continuations, independent branches, execution relationships, budgets,
cancellation, observations, and usage. Applications own the interpretation of
responses and their reasons for choosing the next action. Repair, analysis,
clarification, and branching exploration are examples of consumer workflows
that could use the same mechanisms.

Decision 3 selects an optional sequential coordinator built on independently
usable continuation primitives. Its value is generally useful execution and
conversation management, rather than implementation of Audita's repair loop.
It executes application-directed interactions without needing to understand why
the application requests another turn or stops. Applications retain orchestration
across branches; that initial scope does not prohibit future capabilities.

Apply two questions to proposed public concepts and implementation machinery:

1. Can their value be explained using conversation and execution concepts alone?
2. Can different applications use them without translating their domain into
   repair-specific concepts?

Audita supplies concrete evidence of need and acceptance examples; its workflow
does not define PromptKit's abstraction. Terms such as rejected candidate,
accepted revision, and withdrawal belong to the consuming application. This
direction concerns the new capabilities and does not rename or expand
PromptKit's existing structural-output repair functionality.

## Purpose And Scope

This roadmap records PromptKit's design questions arising from
[Audita's feature request](audita-request.md). That request owns the consumer
workflow and requested acceptance scenarios; this document owns the proposed
PromptKit direction, discussion order, and eventual implementation sequencing.

The proposed capabilities are:

- Capture a rendered request and its final assistant content, then continue it
  using ordinary configured prompts without reconstructing the original request.
- Allow independent continuations from one captured exchange.
- Provide an optional sequential coordinator for bounded looping using
  application-supplied next actions and follow-up requests.

Continuation has direct value by removing consumer capture, reconstruction,
prefix comparison, and history assembly. The coordinator centralizes turn
progression, execution relationships, cancellation, budgets, and diagnostics.
Its value should come from removing that repeated lifecycle work, beyond merely
replacing a consumer's loop counter.
Audita would continue owning candidate selection, domain validation, feedback
meaning, branch scheduling, accepted sibling proposals, and transcript changes.
Its domain-specific repair code would not move wholesale into PromptKit.

The request needs in-process use. Persistent sessions, restartable workflows,
conversation storage, and batch-item reconciliation are outside its scope.
The existing [prepared execution handle](../../prepared_execution.go) remains a
single-use execution capability; continuation needs its own semantics.

## Decisions In Discussion Order

### 1. What Does A Captured Exchange Contain?

**Status: core decision accepted; remaining lifecycle details open.** Capture
represents the logical conversation. Structural-output repair attempts and
corrective instructions are internal execution details, not conversation turns.

**Accepted approach:** capture the complete rendered request, preserving message
content, roles, and cache metadata, followed by the final assistant content
returned by the run. When structural repair succeeds, retain only the final
corrected assistant content, excluding intermediate candidates and corrective
instructions from the logical conversation. This supplies reusable context for
subsequent turns while keeping transport attempts and structural repair separate
from application turns. It also matches Audita's requested history.

Capture is opt-in. A continuation would append its rendered follow-up messages
and final response to produce a new exchange, leaving the source exchange and
other branches unchanged. Previously rendered history would not require the
original input files or templates to be rendered again.

**Alternative considered, not selected:** retain the corrective messages and
responses that led to the final output. This provides more execution detail,
but increases retained content and carries internal repair instructions into
later model context.
Fine-grained execution tracing is outside this feature's purpose. Existing
attempt and usage observations remain separate from logical conversation
content; this decision adds no requirement to capture internal repair text.

**Details to settle:** capture availability after structural validation
exhaustion or an operational failure; how callers request capture for ordinary
and prepared runs; and the ownership and engine-binding of captured exchanges.
The request's “full final assistant response” should refer to the content
supported by PromptKit, without silently adding a raw provider-envelope API.

### 2. How Are Follow-Up Execution Settings Selected?

**Status: accepted.** A continuation supplies existing conversational context
to a normally configured new execution. Conversation history does not introduce
another source of execution-setting inheritance.

**Accepted approach:** preserve conversation content while resolving the new
profile and output contract through the follow-up prompt and ordinary request
overrides. The follow-up can therefore use a different model and response
schema without modifying the original exchange. Explicit request overrides use
ordinary precedence, including complete replacement of the output contract
when supplied. Reuse existing selection and precedence rules.

In particular, prefer the follow-up prompt's default profile when no explicit
profile is supplied, with the ordinary missing-profile behavior if neither is
available. Do not retain the original direct credential merely to retain
conversation history; resolve credential supply for the new execution using
the ordinary credential rules.

Select the session ID using the new request's explicit value or the follow-up
prompt's session template under the existing rules. Do not implicitly inherit
the preceding execution's session ID. An application can explicitly supply the
same session ID across turns. Conversation relationships are recorded separately
and do not depend on a shared session ID.

**Alternative considered, not selected:** inherit the originating execution's
profile and settings unless overridden. This is convenient for repeated turns
against the same model, but adds another precedence source and can make a
follow-up prompt's defaults surprising. Inheriting an old output contract is
especially unsuitable when successive turns expect different response shapes.
An explicit inheritance option is not selected for this feature; reconsider it
only if ordinary selection leaves a demonstrated usability gap.

**Implementation planning detail:** specify message assembly order for captured
history, follow-up definition messages, and caller-appended messages, preserving
authored roles. This does not reopen the accepted execution-setting precedence.

### 3. Should PromptKit Coordinate Turns, And Where Does Application Work Run?

**Status: accepted; exact public types and accounting details remain open.** Provide
independently usable continuation primitives and an optional sequential
coordinator for one branch. The application selects each next request or decides
to stop. PromptKit manages turn progression and execution bookkeeping under the
caller's chosen additional-turn budget and context.

Applications can create several independent continuations or coordinators from
the same exchange. Scheduling branches, joining results, and deciding whether a
branch failure should stop other work remain application responsibilities in
this initial scope. Consumers can also use continuation primitives in their own
loops without adopting the coordinator.

The current [run flow](../internal/runner.md#run-flow) retains logical admission
through structural validation and repair. Releasing an active generation permit
alone does not release that admission lease.

**Accepted execution lifecycle:** compose completed executions. Finish a turn
and release its logical admission lease before invoking application work. Each
later turn receives ordinary admission. The coordinator holds no backend
capacity while awaiting application work. Callbacks can inspect responses,
consult application data, wait for application input, or make model calls against
the same engine and backend, including at capacity one. PromptKit need not
interpret the purpose of that work.

**Alternatives considered, not selected:**

- Provide only continuation and observation primitives. This would keep the
  public API smaller, but leave repeated lifecycle and accounting coordination
  downstream. Independent primitive use remains supported alongside the selected
  coordinator.
- Provide a coordinator for an entire branching tree. This could manage more
  work, but introduces scheduling, concurrency, joining, and failure-policy
  questions without demonstrated shared answers across consumers. Applications
  can build those workflows from the selected branch-level capabilities.

**Details to settle:** exact request/stop and outcome types, and turn-budget
consumption belong to Decision 4. That decision records accepted response
delivery, zero-turn budgets, cancellation, and minimal error/result semantics.
Execution association and usage accounting belong to Decision 5. The sketches
below illustrate the selected responsibility split; they do not settle the
remaining public contracts.

#### Illustrative Consumer Sketches

These are pseudocode, not implemented Go APIs or selected exported names.
`coordinate`, `Continue`, `Stop`, and callback state fields are illustrative.
`request` represents an ordinary PromptKit request, with the selection rules
accepted in Decision 2. Each sketch starts from an already captured exchange.
The callback can keep application state in a closure; PromptKit need not define
a universal domain-state or feedback schema.

**Analysis loop:** the application examines the latest response and selects a
question or decides that its investigation is complete.

```text
analysis = application.start_analysis(initial_exchange)

choose_next(ctx, turn):
    if turn.new_result exists:
        analysis.record(turn.new_result)

    question = application.choose_next_question(analysis)
    if question is absent:
        return Stop()

    return Continue(request(
        prompt_id = "research/investigate",
        vars = {"question": question}
    ))

termination, error = coordinate(
    ctx, start = initial_exchange,
    additional_turns = 3, next = choose_next
)
application.present_analysis(analysis, termination, error)
```

PromptKit preserves the history and performs each selected execution. It does
not decide whether the response answered the question, whether the evidence is
sufficient, or what to investigate next. The application callback may consult
other sources or call the engine before returning its next action.

**Audita branch:** the application starts with one already-assessed correction
and its rejection findings. It interprets and assesses each new response; the
coordinator uses the same generic controls as the analysis loop.

```text
branch = audita.branch(candidate, existing_findings)

choose_next(ctx, turn):
    if turn.new_result exists:
        response = audita.decode_response(turn.new_result)
        if audita.is_withdrawal(response):
            branch.mark_withdrawn()
            return Stop()

        branch.update_candidate(response)
        assessment = audita.assess(ctx, engine, branch.candidate)
        branch.record(assessment)
        if audita.should_stop(assessment):
            return Stop()

    return Continue(request(
        prompt_id = branch.configured_follow_up_prompt,
        vars = audita.follow_up_variables(branch)
    ))

termination, error = coordinate(
    ctx, start = initial_exchange,
    additional_turns = 3, next = choose_next
)
audita.record_branch_outcome(branch, termination, error)
```

The first callback uses existing findings, so initial batch validation is not
repeated. `audita.assess` may make model calls after the turn's admission lease
has been released. Only Audita decides that a correction is accepted, withdrawn,
or unavailable, and retains those meanings in its own state. Further corrections
can start separate coordinators from the same initial exchange; accepted sibling
corrections and other branches remain with Audita.

Both sketches omit application-specific error handling and depict normal turn
progression. The final permitted response follows the same callback path as
earlier responses, as accepted in Decision 4. That decision also defines zero-turn
budgets, cancellation around callback delivery, and a minimal termination result.
The sketches do not imply an archive of responses in that result: applications
already receive each completed response through the callback. Completed work
must not be mistaken for application acceptance of its content.
The concrete interface should let both examples use the same coordination
mechanism without manual message-history assembly or per-turn lifecycle
bookkeeping. Callback model-call association remains part of Decision 5.

### 4. How Are Application Next Actions And Execution Outcomes Represented?

**Status: response delivery, exhaustion, zero-turn budgets, cancellation, and
minimal error/result semantics accepted; turn-budget consumption remains open.**
The coordinator uses content-neutral next actions and distinguishes
application-directed completion from exhaustion.

**Accepted normal flow:** deliver each completed response once through the
ordinary application callback, including the last permitted response. If the
callback returns `Stop`, report application-directed completion. If it requests
another turn and none remain, report exhaustion without executing that request.
Using the entire budget does not itself imply exhaustion; exhaustion means the
application requested additional work that the budget would not permit.

This gives consumers one response-processing path, including any application
validation, without a separate path for the final response. Internal transport
retries and structural repairs do not produce additional conversation-response
callbacks. Cancellation and operational errors can interrupt normal delivery as
specified below. `Stop` and other API spellings remain illustrative until public
types are selected.

**Alternative considered, not selected:** stop immediately when the turn budget
is used and expose the final response only in the report. This avoids another
callback but requires separate consumer processing of the final response.

**Accepted zero-turn behavior:** with an active context, invoke the initial
callback once even when the additional-turn budget is zero. The callback receives
the starting exchange and knows that no additional turns remain; this is not
delivery of a newly generated response. It can use existing application state
without repeating earlier assessment.

- `Stop` produces application-directed completion.
- Requesting another turn produces exhaustion without executing the request.
- Returning an error produces an operational failure, subject to the
  cancellation checks below.

Zero bounds coordinator-driven continuation executions, not work inside the
application callback. The callback can still consult data or make model calls.
Applications wanting to disable all such work can skip invoking the coordinator.
The alternative of treating zero as an immediate no-op with no callback is not
selected: it would introduce special behavior and would not distinguish an
already-complete interaction from one requesting more work.

**Accepted cancellation behavior:** check the caller's context immediately
before callback invocation, including the initial callback. If cancellation is
already observed, do not invoke the callback; return the context error with its
identity preserved. If the coordinator already obtained a completed response
that has not been delivered to the callback, preserve that single response in
the optional result described below. This does not imply application processing
or acceptance, or require recovering a response that a failed underlying
execution did not return.

Cancellation can race with callback invocation. The callback receives the same
context and is responsible for cooperating with cancellation during its work.
PromptKit checks the context again after the callback returns and before starting
another execution. It does not forcibly terminate callback code. Thus ordinary
response delivery is once per completed turn during normal progression, while
cancellation can interrupt that progression. Preserving available completed work
on interruption does not create a separate final-response processing path during
normal budget exhaustion.

The alternative of invoking the callback after cancellation with a canceled
context is not selected. It could help callbacks that only record results, but
callbacks may perform substantial application work or model calls; PromptKit
cannot assume they are merely bookkeeping.

**Accepted operational-failure behavior:** each continuation is an ordinary
execution supplied with captured conversation history. Applications receive
completed responses through the callback before another continuation begins.
An execution or callback error stops coordination and propagates to the caller,
preserving useful Go error identities. PromptKit does not automatically replay
the failed continuation or callback. Existing transport retries and structural
repair remain within ordinary execution. Earlier callback deliveries and
application work remain intact; applications decide whether the failure affects
other branches or the overall workflow.

**Accepted minimal return contract:** provide only the following information:

- An ordinary Go error for execution failures, callback failures, and
  cancellation, preserving `errors.Is`/`errors.As` behavior.
- A small outcome value distinguishing application-directed stopping from
  budget exhaustion on normal termination. These outcomes are not errors.
- An optional completed response when cancellation prevented its delivery to
  the callback. Reuse existing response types where practical.

Delivery means the callback was invoked with the response; it does not imply
the application finished processing or accepted it. If that callback returns
an error, the response is not classified as undelivered or automatically
delivered again. The coordinator returns no archive of previously delivered
responses. A small optional result accommodates the undelivered-response case;
exact Go declarations remain for implementation planning.

Failure-stage fields, additional run-ID references, a separate failure
classification hierarchy, and aggregate reporting are not part of this minimal
termination contract. They require a demonstrated consumer need before addition.
This does not remove fields from existing response values or observation APIs.
Decision 5 separately evaluates accounting, and Decision 6 inspection, without
presuming that either needs to expand the coordinator's termination result.

Audita retains its own meanings for acceptance, rejection, withdrawal, and
assessment unavailability. A normal stop does not assert any of those outcomes.

**Alternative considered, not selected for normal termination:** use callback
errors to signal ordinary termination as well as operational failure. This can
reduce the number of result types, but makes normal control flow harder to
distinguish from failure. Normal stopping uses an explicit action, while failures
use ordinary Go errors. A broader structured termination report was also
considered and not selected: its additional metadata is unnecessary for the
demonstrated minimal contract.

**Details to settle:** what consumes an additional-turn budget when preparation
or generation fails. Keep that budget distinct from structural repair and
transport retry. Exact next-action, outcome, and optional-result types remain
implementation planning details. Whether a completed run with failed structural
validation produces a reusable exchange remains a Decision 1 question; the
operational-error decision does not reclassify that validation result.

### 5. How Are Related Executions And Usage Accounted For?

**Status: open.** Define accounting for the initial exchange, each continuation,
its structural repairs and transport retries, and model calls made by
application callbacks, both for standalone continuations and coordinated turns.

The current [observation boundary](../internal/observation.md) keeps nested
invocations independent. A callback's arbitrary model calls cannot simply be
assumed to belong to the enclosing interaction's totals.

**Recommendation:** preserve independent run accounting and add explicit
relationships between exchanges, branches, sequences, and associated runs.
Choose a small association mechanism, such as a scoped observation facility or
explicit execution references, so consumers need not rebuild lifecycle tracking.
Shared initial work should be referenced by branches and counted once when
reporting the combined operation. Report available usage with its existing
presence/completeness meaning; do not promise complete cost accounting.

**Alternative:** have PromptKit automatically capture every nested run during an
application callback. This minimizes explicit association but can include
unrelated work and changes the existing invocation-isolation boundary.
Another simpler option is to report continuation usage only and let applications
attach callback accounting; that leaves more work downstream and needs checking
against Audita's stated reporting requirement.

**Details to settle:** association of concurrent callback calls, observer
composition, failure-path diagnostics, and which totals include which child
executions. A turn budget would bound continuation attempts, not all provider
calls performed by structural repair, transport retries, or application work.

### 6. Where Should Continuation Inspection Live?

**Status: open.** Clarify the request's references to inspection without changing
the established resource-inspection boundary.

**Recommendation:** expose originating exchange, follow-up prompt, and execution
relationships in continuation details, prepared execution metadata, sequence
reports, and observations as appropriate. Keep `InspectPrompt`, `InspectProfile`,
and `InspectBackend` focused on their configured resources. Reuse the existing
approach to generically renderable inspection data where new details are exposed.
Any proposed report should demonstrate its consumer value; Decision 4's minimal
termination contract does not promise a sequence report or response archive.

**Alternative:** introduce a dedicated conversation inspection API. This may
be useful if a retained exchange needs a richer view, but should be justified
against a simple details method. Extending resource inspection with execution
history would give those APIs a different responsibility.

**Details to settle:** the minimum inspectable metadata, how callers explicitly
access captured message content, and separation of content-bearing details from
metadata-only observations. Public type names and serialization details belong
in implementation planning after this boundary is settled.

## Proposed Delivery And Validation

Resolve the decisions above before selecting exact public signatures or writing
an implementation plan. The recommended sequence is:

1. Deliver capture and independent continuation through the public facade,
   allowing Audita to remove its reconstruction and prefix-verification work.
2. Build the optional sequential coordinator on that capability, reusing
   ordinary execution, validation, capacity, and observation machinery. Refine
   both consumer sketches against the selected public contracts to verify that
   the same mechanism removes useful repeated work in each application.

Use [Audita's acceptance scenarios](audita-request.md#suggested-delivery-and-acceptance-scenarios)
as consumer checks, alongside analysis and branching-conversation examples that
exercise the same APIs without repair-specific concepts. Additional proposed
coverage should establish:

- The selected conversation history when the initial run needed structural
  repair, including use of its full final assistant content.
- Continuation after the original input files change or disappear, without
  altering the captured history.
- Independent branches retaining their own history and outcomes.
- Assessment calls against the same backend at capacity one, without capacity
  retained by the coordinator blocking them.
- Associated validator usage and shared initial usage reported without
  double-counting, including incomplete usage and operational failures.

These are planning targets, not new implemented guarantees. Keep additional
mechanisms and constraints tied to the selected decisions and demonstrated
consumer needs.