promptkit/docs/roadmap/conversation-continuation.md

Conversation Continuation And Application-Directed Interaction

Status: feature roadmap under discussion; accepted directions and remaining open decisions are identified below. No implementation is authorized by this document. Recommendations remain proposals unless explicitly marked accepted.

North Star: Content-Neutral, Generally Useful Capabilities

Accepted direction: evaluate each capability for its usefulness across different downstream applications. PromptKit must be able to implement it and explain its value without assuming or inferring the meaning of the content passed through prompts and responses.

PromptKit can own conversation and execution mechanics: captured messages, continuations, independent branches, execution relationships, budgets, cancellation, observations, and usage. Applications own the interpretation of responses and their reasons for choosing the next action. Repair, analysis, clarification, and branching exploration are examples of consumer workflows that could use the same mechanisms.

Decision 3 selects an optional sequential coordinator built on independently usable continuation primitives. Its value is generally useful execution and conversation management, rather than implementation of Audita's repair loop. It executes application-directed interactions without needing to understand why the application requests another turn or stops. Applications retain orchestration across branches; that initial scope does not prohibit future capabilities.

Apply two questions to proposed public concepts and implementation machinery:

1. Can their value be explained using conversation and execution concepts alone?
2. Can different applications use them without translating their domain into repair-specific concepts?

Audita supplies concrete evidence of need and acceptance examples; its workflow does not define PromptKit's abstraction. Terms such as rejected candidate, accepted revision, and withdrawal belong to the consuming application. This direction concerns the new capabilities and does not rename or expand PromptKit's existing structural-output repair functionality.

Purpose And Scope

This roadmap records PromptKit's design questions arising from Audita's feature request. That request owns the consumer workflow and requested acceptance scenarios; this document owns the proposed PromptKit direction, discussion order, and eventual implementation sequencing.

The proposed capabilities are:

• Capture a rendered request and its final assistant content, then continue it using ordinary configured prompts without reconstructing the original request.
• Allow independent continuations from one captured exchange.
• Provide an optional sequential coordinator for bounded looping using application-supplied next actions and follow-up requests.

Continuation has direct value by removing consumer capture, reconstruction, prefix comparison, and history assembly. The coordinator centralizes turn progression, execution relationships, cancellation, budgets, and diagnostics. Its value should come from removing that repeated lifecycle work, beyond merely replacing a consumer's loop counter. Audita would continue owning candidate selection, domain validation, feedback meaning, branch scheduling, accepted sibling proposals, and transcript changes. Its domain-specific repair code would not move wholesale into PromptKit.

The request needs in-process use. Persistent sessions, restartable workflows, conversation storage, and batch-item reconciliation are outside its scope. The existing prepared execution handle remains a single-use execution capability; continuation needs its own semantics.

Decisions In Discussion Order

1. What Does A Captured Exchange Contain?

Status: core decision accepted; remaining lifecycle details open. Capture represents the logical conversation. Structural-output repair attempts and corrective instructions are internal execution details, not conversation turns.

Accepted approach: capture the complete rendered request, preserving message content, roles, and cache metadata, followed by the final assistant content returned by the run. When structural repair succeeds, retain only the final corrected assistant content, excluding intermediate candidates and corrective instructions from the logical conversation. This supplies reusable context for subsequent turns while keeping transport attempts and structural repair separate from application turns. It also matches Audita's requested history.

Capture is opt-in. A continuation would append its rendered follow-up messages and final response to produce a new exchange, leaving the source exchange and other branches unchanged. Previously rendered history would not require the original input files or templates to be rendered again.

Alternative considered, not selected: retain the corrective messages and responses that led to the final output. This provides more execution detail, but increases retained content and carries internal repair instructions into later model context. Fine-grained execution tracing is outside this feature's purpose. Existing attempt and usage observations remain separate from logical conversation content; this decision adds no requirement to capture internal repair text.

Details to settle: capture availability after structural validation exhaustion or an operational failure; how callers request capture for ordinary and prepared runs; and the ownership and engine-binding of captured exchanges. The request's “full final assistant response” should refer to the content supported by PromptKit, without silently adding a raw provider-envelope API.

2. How Are Follow-Up Execution Settings Selected?

Status: accepted. A continuation supplies existing conversational context to a normally configured new execution. Conversation history does not introduce another source of execution-setting inheritance.

Accepted approach: preserve conversation content while resolving the new profile and output contract through the follow-up prompt and ordinary request overrides. The follow-up can therefore use a different model and response schema without modifying the original exchange. Explicit request overrides use ordinary precedence, including complete replacement of the output contract when supplied. Reuse existing selection and precedence rules.

In particular, prefer the follow-up prompt's default profile when no explicit profile is supplied, with the ordinary missing-profile behavior if neither is available. Do not retain the original direct credential merely to retain conversation history; resolve credential supply for the new execution using the ordinary credential rules.

Select the session ID using the new request's explicit value or the follow-up prompt's session template under the existing rules. Do not implicitly inherit the preceding execution's session ID. An application can explicitly supply the same session ID across turns. Conversation relationships are recorded separately and do not depend on a shared session ID.

Alternative considered, not selected: inherit the originating execution's profile and settings unless overridden. This is convenient for repeated turns against the same model, but adds another precedence source and can make a follow-up prompt's defaults surprising. Inheriting an old output contract is especially unsuitable when successive turns expect different response shapes. An explicit inheritance option is not selected for this feature; reconsider it only if ordinary selection leaves a demonstrated usability gap.

Implementation planning detail: specify message assembly order for captured history, follow-up definition messages, and caller-appended messages, preserving authored roles. This does not reopen the accepted execution-setting precedence.

3. Should PromptKit Coordinate Turns, And Where Does Application Work Run?

Status: accepted; exact callback and reporting contracts remain open. Provide independently usable continuation primitives and an optional sequential coordinator for one branch. The application selects each next request or decides to stop. PromptKit manages turn progression and execution bookkeeping under the caller's chosen additional-turn budget and context.

Applications can create several independent continuations or coordinators from the same exchange. Scheduling branches, joining results, and deciding whether a branch failure should stop other work remain application responsibilities in this initial scope. Consumers can also use continuation primitives in their own loops without adopting the coordinator.

The current run flow retains logical admission through structural validation and repair. Releasing an active generation permit alone does not release that admission lease.

Accepted execution lifecycle: compose completed executions. Finish a turn and release its logical admission lease before invoking application work. Each later turn receives ordinary admission. The coordinator holds no backend capacity while awaiting application work. Callbacks can inspect responses, consult application data, wait for application input, or make model calls against the same engine and backend, including at capacity one. PromptKit need not interpret the purpose of that work.

Alternatives considered, not selected:

• Provide only continuation and observation primitives. This would keep the public API smaller, but leave repeated lifecycle and accounting coordination downstream. Independent primitive use remains supported alongside the selected coordinator.
• Provide a coordinator for an entire branching tree. This could manage more work, but introduces scheduling, concurrency, joining, and failure-policy questions without demonstrated shared answers across consumers. Applications can build those workflows from the selected branch-level capabilities.

Details to settle: callback cancellation boundaries, exact request/stop and outcome types, and error/report semantics belong to Decision 4. That decision records the accepted normal delivery of the last permitted response. Execution association and usage accounting belong to Decision 5. The sketches below illustrate the selected responsibility split; they do not settle the remaining public contracts.

Illustrative Consumer Sketches

These are pseudocode, not implemented Go APIs or selected exported names. coordinate, Continue, Stop, and callback state fields are illustrative. request represents an ordinary PromptKit request, with the selection rules accepted in Decision 2. Each sketch starts from an already captured exchange. The callback can keep application state in a closure; PromptKit need not define a universal domain-state or feedback schema.

Analysis loop: the application examines the latest response and selects a question or decides that its investigation is complete.

analysis = application.start_analysis(initial_exchange)

choose_next(ctx, turn):
    if turn.new_result exists:
        analysis.record(turn.new_result)

    question = application.choose_next_question(analysis)
    if question is absent:
        return Stop()

    return Continue(request(
        prompt_id = "research/investigate",
        vars = {"question": question}
    ))

report, error = coordinate(
    ctx, start = initial_exchange,
    additional_turns = 3, next = choose_next
)
application.present_analysis(analysis, report, error)

PromptKit preserves the history and performs each selected execution. It does not decide whether the response answered the question, whether the evidence is sufficient, or what to investigate next. The application callback may consult other sources or call the engine before returning its next action.

Audita branch: the application starts with one already-assessed correction and its rejection findings. It interprets and assesses each new response; the coordinator uses the same generic controls as the analysis loop.

branch = audita.branch(candidate, existing_findings)

choose_next(ctx, turn):
    if turn.new_result exists:
        response = audita.decode_response(turn.new_result)
        if audita.is_withdrawal(response):
            branch.mark_withdrawn()
            return Stop()

        branch.update_candidate(response)
        assessment = audita.assess(ctx, engine, branch.candidate)
        branch.record(assessment)
        if audita.should_stop(assessment):
            return Stop()

    return Continue(request(
        prompt_id = branch.configured_follow_up_prompt,
        vars = audita.follow_up_variables(branch)
    ))

report, error = coordinate(
    ctx, start = initial_exchange,
    additional_turns = 3, next = choose_next
)
audita.record_branch_outcome(branch, report, error)

The first callback uses existing findings, so initial batch validation is not repeated. audita.assess may make model calls after the turn's admission lease has been released. Only Audita decides that a correction is accepted, withdrawn, or unavailable, and retains those meanings in its own state. Further corrections can start separate coordinators from the same initial exchange; accepted sibling corrections and other branches remain with Audita.

Both sketches omit application-specific error handling and depict normal turn progression. The final permitted response follows the same callback path as earlier responses, as accepted in Decision 4. Callback errors, zero-turn budgets, and cancellation around delivery remain open there. Completed work must not be mistaken for application acceptance of its content. The concrete interface should let both examples use the same coordination mechanism without manual message-history assembly or per-turn lifecycle bookkeeping. Callback model-call association remains part of Decision 5.

4. How Are Application Next Actions And Execution Outcomes Represented?

Status: normal response delivery and exhaustion decision accepted; remaining outcome and failure details open. The coordinator uses content-neutral next actions and distinguishes application-directed completion from exhaustion.

Accepted normal flow: deliver each completed response once through the ordinary application callback, including the last permitted response. If the callback returns Stop, report application-directed completion. If it requests another turn and none remain, report exhaustion without executing that request. Using the entire budget does not itself imply exhaustion; exhaustion means the application requested additional work that the budget would not permit.

This gives consumers one response-processing path, including any application validation, without a separate path for the final response. Internal transport retries and structural repairs do not produce additional conversation-response callbacks. Cancellation and operational-failure delivery semantics remain open; this normal-flow decision does not settle those cases. Stop and other API spellings remain illustrative until public types are selected.

Alternative considered, not selected: stop immediately when the turn budget is used and expose the final response only in the report. This avoids another callback but requires separate consumer processing of the final response.

Recommendation: let the application request another turn or indicate that the interaction should stop, separately from reporting an operational error. PromptKit need not know whether the application is repairing, investigating, or exploring. Permit starting from an existing exchange and application state without forcing the application to repeat earlier work.

Return execution diagnostics that preserve completed work when an interaction stops. A completed model call does not imply application acceptance of its content. Audita would map its own acceptance, rejection, withdrawal, and unavailable-assessment outcomes to the selected generic controls. Cancellation and operational errors should retain useful Go error identities. Applications decide whether a stopped branch affects independent work.

Alternative considered, not selected for normal termination: use callback errors to signal ordinary termination as well as operational failure. This can reduce the number of result types, but makes normal control flow harder to distinguish from failure. The accepted normal flow uses an explicit stop action; the operational-error contract remains to be selected.

Details to settle: zero-turn budgets; cancellation before callback delivery, during callbacks, and before subsequent execution; exact next-action and outcome types; result-plus-error behavior; and what consumes an additional-turn budget when preparation or generation fails. Keep that budget distinct from structural repair and transport retry. Whether a failed execution produces a reusable exchange remains a Decision 1 question.

5. How Are Related Executions And Usage Accounted For?

Status: open. Define accounting for the initial exchange, each continuation, its structural repairs and transport retries, and model calls made by application callbacks, both for standalone continuations and coordinated turns.

The current observation boundary keeps nested invocations independent. A callback's arbitrary model calls cannot simply be assumed to belong to the enclosing interaction's totals.

Recommendation: preserve independent run accounting and add explicit relationships between exchanges, branches, sequences, and associated runs. Choose a small association mechanism, such as a scoped observation facility or explicit execution references, so consumers need not rebuild lifecycle tracking. Shared initial work should be referenced by branches and counted once when reporting the combined operation. Report available usage with its existing presence/completeness meaning; do not promise complete cost accounting.

Alternative: have PromptKit automatically capture every nested run during an application callback. This minimizes explicit association but can include unrelated work and changes the existing invocation-isolation boundary. Another simpler option is to report continuation usage only and let applications attach callback accounting; that leaves more work downstream and needs checking against Audita's stated reporting requirement.

Details to settle: association of concurrent callback calls, observer composition, failure-path diagnostics, and which totals include which child executions. A turn budget would bound continuation attempts, not all provider calls performed by structural repair, transport retries, or application work.

6. Where Should Continuation Inspection Live?

Status: open. Clarify the request's references to inspection without changing the established resource-inspection boundary.

Recommendation: expose originating exchange, follow-up prompt, and execution relationships in continuation details, prepared execution metadata, sequence reports, and observations as appropriate. Keep InspectPrompt, InspectProfile, and InspectBackend focused on their configured resources. Reuse the existing approach to generically renderable inspection data where new details are exposed.

Alternative: introduce a dedicated conversation inspection API. This may be useful if a retained exchange needs a richer view, but should be justified against a simple details method. Extending resource inspection with execution history would give those APIs a different responsibility.

Details to settle: the minimum inspectable metadata, how callers explicitly access captured message content, and separation of content-bearing details from metadata-only observations. Public type names and serialization details belong in implementation planning after this boundary is settled.

Proposed Delivery And Validation

Resolve the decisions above before selecting exact public signatures or writing an implementation plan. The recommended sequence is:

1. Deliver capture and independent continuation through the public facade, allowing Audita to remove its reconstruction and prefix-verification work.
2. Build the optional sequential coordinator on that capability, reusing ordinary execution, validation, capacity, and observation machinery. Refine both consumer sketches against the selected public contracts to verify that the same mechanism removes useful repeated work in each application.

Use Audita's acceptance scenarios as consumer checks, alongside analysis and branching-conversation examples that exercise the same APIs without repair-specific concepts. Additional proposed coverage should establish:

• The selected conversation history when the initial run needed structural repair, including use of its full final assistant content.
• Continuation after the original input files change or disappear, without altering the captured history.
• Independent branches retaining their own history and outcomes.
• Assessment calls against the same backend at capacity one, without capacity retained by the coordinator blocking them.
• Associated validator usage and shared initial usage reported without double-counting, including incomplete usage and operational failures.

These are planning targets, not new implemented guarantees. Keep additional mechanisms and constraints tied to the selected decisions and demonstrated consumer needs.