From de2a7fcd6f5f19db864ac25b93fbedc3dc55e0bd Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Wed, 11 Nov 2026 21:12:13 +0000 Subject: [PATCH] Record sequential coordinator decision and consumer sketches --- docs/roadmap/conversation-continuation.md | 186 +++++++++++++++++----- 1 file changed, 144 insertions(+), 42 deletions(-) diff --git a/docs/roadmap/conversation-continuation.md b/docs/roadmap/conversation-continuation.md index 165ca9f..b247f52 100644 --- a/docs/roadmap/conversation-continuation.md +++ b/docs/roadmap/conversation-continuation.md @@ -18,11 +18,12 @@ responses and their reasons for choosing the next action. Repair, analysis, clarification, and branching exploration are examples of consumer workflows that could use the same mechanisms. -Coordination and looping remain open questions. Neither is presumed necessary -or excluded. Assess their value as generally useful capabilities, rather than -as an implementation of Audita's repair loop. A coordinator, if selected, -would execute application-directed interactions without needing to understand -why the application requests another turn or stops. +Decision 3 selects an optional sequential coordinator built on independently +usable continuation primitives. Its value is generally useful execution and +conversation management, rather than implementation of Audita's repair loop. +It executes application-directed interactions without needing to understand why +the application requests another turn or stops. Applications retain orchestration +across branches; that initial scope does not prohibit future capabilities. Apply two questions to proposed public concepts and implementation machinery: @@ -48,12 +49,14 @@ The proposed capabilities are: - Capture a rendered request and its final assistant content, then continue it using ordinary configured prompts without reconstructing the original request. - Allow independent continuations from one captured exchange. -- Consider coordination and bounded looping using application-supplied next - actions and follow-up requests. +- Provide an optional sequential coordinator for bounded looping using + application-supplied next actions and follow-up requests. Continuation has direct value by removing consumer capture, reconstruction, -prefix comparison, and history assembly. A coordinator could also remove -repeated lifecycle work, if its general value justifies its public contract. +prefix comparison, and history assembly. The coordinator centralizes turn +progression, execution relationships, cancellation, budgets, and diagnostics. +Its value should come from removing that repeated lifecycle work, beyond merely +replacing a consumer's loop counter. Audita would continue owning candidate selection, domain validation, feedback meaning, branch scheduling, accepted sibling proposals, and transcript changes. Its domain-specific repair code would not move wholesale into PromptKit. @@ -137,41 +140,139 @@ authored roles. This does not reopen the accepted execution-setting precedence. ### 3. Should PromptKit Coordinate Turns, And Where Does Application Work Run? -**Status: open.** Evaluate whether a coordinator or loop provides generally -useful behavior beyond continuation primitives. If selected, determine its -scope and a lifecycle that permits application callbacks to call the same -engine and backend, including with capacity limited to one. +**Status: accepted; exact callback and reporting contracts remain open.** Provide +independently usable continuation primitives and an optional sequential +coordinator for one branch. The application selects each next request or decides +to stop. PromptKit manages turn progression and execution bookkeeping under the +caller's chosen additional-turn budget and context. + +Applications can create several independent continuations or coordinators from +the same exchange. Scheduling branches, joining results, and deciding whether a +branch failure should stop other work remain application responsibilities in +this initial scope. Consumers can also use continuation primitives in their own +loops without adopting the coordinator. The current [run flow](../internal/runner.md#run-flow) retains logical admission through structural validation and repair. Releasing an active generation permit alone does not release that admission lease. -**Recommendation:** assess the value across repair, analysis, and branching -conversation examples before choosing the abstraction. If PromptKit provides -coordination, compose completed executions: finish a turn and release its -admission lease before invoking application work. Each later turn receives -ordinary admission. A coordinator would hold no backend capacity while awaiting -application work. This reuses existing execution machinery and permits -callbacks to make their own model calls without interpreting their purpose. +**Accepted execution lifecycle:** compose completed executions. Finish a turn +and release its logical admission lease before invoking application work. Each +later turn receives ordinary admission. The coordinator holds no backend +capacity while awaiting application work. Callbacks can inspect responses, +consult application data, wait for application input, or make model calls against +the same engine and backend, including at capacity one. PromptKit need not +interpret the purpose of that work. -**Alternative:** provide continuation and observation primitives while leaving -looping to consumers. This keeps PromptKit's API smaller, but may leave repeated -execution and accounting coordination downstream. Compare the actual consumer -work under both approaches. Extending the structural-repair loop is another -possible implementation route, but would require changing lease ownership and -decoupling it from repair-specific assumptions. +**Alternatives considered, not selected:** -**Details to settle:** whether to provide coordination, looping, or both; scope -over a single branch versus multiple branches; callback cancellation boundaries; -and the sequence of request construction, execution, and application decisions. -Applications retain the meaning of those decisions and policy for stopping -other work after a failure. +- Provide only continuation and observation primitives. This would keep the + public API smaller, but leave repeated lifecycle and accounting coordination + downstream. Independent primitive use remains supported alongside the selected + coordinator. +- Provide a coordinator for an entire branching tree. This could manage more + work, but introduces scheduling, concurrency, joining, and failure-policy + questions without demonstrated shared answers across consumers. Applications + can build those workflows from the selected branch-level capabilities. + +**Details to settle:** callback cancellation boundaries, exact request/stop and +outcome types, final-turn delivery at budget exhaustion, and error/report +semantics belong to Decision 4. Execution association and usage accounting belong +to Decision 5. The sketches below illustrate the selected responsibility split; +they do not settle those public contracts. + +#### Illustrative Consumer Sketches + +These are pseudocode, not implemented Go APIs or selected exported names. +`coordinate`, `Continue`, `Stop`, and callback state fields are illustrative. +`request` represents an ordinary PromptKit request, with the selection rules +accepted in Decision 2. Each sketch starts from an already captured exchange. +The callback can keep application state in a closure; PromptKit need not define +a universal domain-state or feedback schema. + +**Analysis loop:** the application examines the latest response and selects a +question or decides that its investigation is complete. + +```text +analysis = application.start_analysis(initial_exchange) + +choose_next(ctx, turn): + if turn.new_result exists: + analysis.record(turn.new_result) + + question = application.choose_next_question(analysis) + if question is absent: + return Stop() + + return Continue(request( + prompt_id = "research/investigate", + vars = {"question": question} + )) + +report, error = coordinate( + ctx, start = initial_exchange, + additional_turns = 3, next = choose_next +) +application.present_analysis(analysis, report, error) +``` + +PromptKit preserves the history and performs each selected execution. It does +not decide whether the response answered the question, whether the evidence is +sufficient, or what to investigate next. The application callback may consult +other sources or call the engine before returning its next action. + +**Audita branch:** the application starts with one already-assessed correction +and its rejection findings. It interprets and assesses each new response; the +coordinator uses the same generic controls as the analysis loop. + +```text +branch = audita.branch(candidate, existing_findings) + +choose_next(ctx, turn): + if turn.new_result exists: + response = audita.decode_response(turn.new_result) + if audita.is_withdrawal(response): + branch.mark_withdrawn() + return Stop() + + branch.update_candidate(response) + assessment = audita.assess(ctx, engine, branch.candidate) + branch.record(assessment) + if audita.should_stop(assessment): + return Stop() + + return Continue(request( + prompt_id = branch.configured_follow_up_prompt, + vars = audita.follow_up_variables(branch) + )) + +report, error = coordinate( + ctx, start = initial_exchange, + additional_turns = 3, next = choose_next +) +audita.record_branch_outcome(branch, report, error) +``` + +The first callback uses existing findings, so initial batch validation is not +repeated. `audita.assess` may make model calls after the turn's admission lease +has been released. Only Audita decides that a correction is accepted, withdrawn, +or unavailable, and retains those meanings in its own state. Further corrections +can start separate coordinators from the same initial exchange; accepted sibling +corrections and other branches remain with Audita. + +Both sketches omit application-specific error handling and depict normal turn +progression. They do not prescribe how callback errors or the final result at +the budget boundary are delivered; Decision 4 must specify those paths so the +application can handle completed work without mistaking it for accepted content. +The concrete interface should let both examples use the same coordination +mechanism without manual message-history assembly or per-turn lifecycle +bookkeeping. Callback model-call association remains part of Decision 5. ### 4. How Are Application Next Actions And Execution Outcomes Represented? -**Status: open.** Subject to Decision 3, define content-neutral next actions and -distinguish application-directed completion, budget exhaustion, interruption, -and operational failure. +**Status: open.** For the coordinator selected in Decision 3, define +content-neutral next actions and distinguish application-directed completion, +budget exhaustion, interruption, and operational failure. **Recommendation:** let the application request another turn or indicate that the interaction should stop, separately from reporting an operational error. @@ -194,22 +295,22 @@ small explicit next-action result without introducing domain-specific statuses. **Details to settle:** next-action and outcome vocabulary, callback/result types, result-plus-error behavior, zero-budget behavior, and what consumes an additional-turn budget when preparation or generation fails. Keep such a budget -distinct from structural repair and transport retry. If coordination is left -to consumers, limit this decision to continuation execution outcomes. +distinct from structural repair and transport retry. Specify how an application +receives the last completed response when no further turns remain, including +whether and how its callback is invoked at that boundary. ### 5. How Are Related Executions And Usage Accounted For? **Status: open.** Define accounting for the initial exchange, each continuation, its structural repairs and transport retries, and model calls made by -application callbacks, whether or not PromptKit supplies a coordinator. +application callbacks, both for standalone continuations and coordinated turns. The current [observation boundary](../internal/observation.md) keeps nested invocations independent. A callback's arbitrary model calls cannot simply be assumed to belong to the enclosing interaction's totals. **Recommendation:** preserve independent run accounting and add explicit -relationships between exchanges, branches, and associated runs, with sequence -relationships if coordination is selected. +relationships between exchanges, branches, sequences, and associated runs. Choose a small association mechanism, such as a scoped observation facility or explicit execution references, so consumers need not rebuild lifecycle tracking. Shared initial work should be referenced by branches and counted once when @@ -256,9 +357,10 @@ an implementation plan. The recommended sequence is: 1. Deliver capture and independent continuation through the public facade, allowing Audita to remove its reconstruction and prefix-verification work. -2. If selected after evaluating general usefulness, build coordination and/or - bounded looping on that capability, reusing ordinary execution, validation, - capacity, and observation machinery. +2. Build the optional sequential coordinator on that capability, reusing + ordinary execution, validation, capacity, and observation machinery. Refine + both consumer sketches against the selected public contracts to verify that + the same mechanism removes useful repeated work in each application. Use [Audita's acceptance scenarios](audita-request.md#suggested-delivery-and-acceptance-scenarios) as consumer checks, alongside analysis and branching-conversation examples that