Record sequential coordinator decision and consumer sketches

This commit is contained in:
2026-11-11 21:12:13 +00:00
parent f20020d9f4
commit de2a7fcd6f

View File

@@ -18,11 +18,12 @@ responses and their reasons for choosing the next action. Repair, analysis,
clarification, and branching exploration are examples of consumer workflows clarification, and branching exploration are examples of consumer workflows
that could use the same mechanisms. that could use the same mechanisms.
Coordination and looping remain open questions. Neither is presumed necessary Decision 3 selects an optional sequential coordinator built on independently
or excluded. Assess their value as generally useful capabilities, rather than usable continuation primitives. Its value is generally useful execution and
as an implementation of Audita's repair loop. A coordinator, if selected, conversation management, rather than implementation of Audita's repair loop.
would execute application-directed interactions without needing to understand It executes application-directed interactions without needing to understand why
why the application requests another turn or stops. the application requests another turn or stops. Applications retain orchestration
across branches; that initial scope does not prohibit future capabilities.
Apply two questions to proposed public concepts and implementation machinery: Apply two questions to proposed public concepts and implementation machinery:
@@ -48,12 +49,14 @@ The proposed capabilities are:
- Capture a rendered request and its final assistant content, then continue it - Capture a rendered request and its final assistant content, then continue it
using ordinary configured prompts without reconstructing the original request. using ordinary configured prompts without reconstructing the original request.
- Allow independent continuations from one captured exchange. - Allow independent continuations from one captured exchange.
- Consider coordination and bounded looping using application-supplied next - Provide an optional sequential coordinator for bounded looping using
actions and follow-up requests. application-supplied next actions and follow-up requests.
Continuation has direct value by removing consumer capture, reconstruction, Continuation has direct value by removing consumer capture, reconstruction,
prefix comparison, and history assembly. A coordinator could also remove prefix comparison, and history assembly. The coordinator centralizes turn
repeated lifecycle work, if its general value justifies its public contract. progression, execution relationships, cancellation, budgets, and diagnostics.
Its value should come from removing that repeated lifecycle work, beyond merely
replacing a consumer's loop counter.
Audita would continue owning candidate selection, domain validation, feedback Audita would continue owning candidate selection, domain validation, feedback
meaning, branch scheduling, accepted sibling proposals, and transcript changes. meaning, branch scheduling, accepted sibling proposals, and transcript changes.
Its domain-specific repair code would not move wholesale into PromptKit. Its domain-specific repair code would not move wholesale into PromptKit.
@@ -137,41 +140,139 @@ authored roles. This does not reopen the accepted execution-setting precedence.
### 3. Should PromptKit Coordinate Turns, And Where Does Application Work Run? ### 3. Should PromptKit Coordinate Turns, And Where Does Application Work Run?
**Status: open.** Evaluate whether a coordinator or loop provides generally **Status: accepted; exact callback and reporting contracts remain open.** Provide
useful behavior beyond continuation primitives. If selected, determine its independently usable continuation primitives and an optional sequential
scope and a lifecycle that permits application callbacks to call the same coordinator for one branch. The application selects each next request or decides
engine and backend, including with capacity limited to one. to stop. PromptKit manages turn progression and execution bookkeeping under the
caller's chosen additional-turn budget and context.
Applications can create several independent continuations or coordinators from
the same exchange. Scheduling branches, joining results, and deciding whether a
branch failure should stop other work remain application responsibilities in
this initial scope. Consumers can also use continuation primitives in their own
loops without adopting the coordinator.
The current [run flow](../internal/runner.md#run-flow) retains logical admission The current [run flow](../internal/runner.md#run-flow) retains logical admission
through structural validation and repair. Releasing an active generation permit through structural validation and repair. Releasing an active generation permit
alone does not release that admission lease. alone does not release that admission lease.
**Recommendation:** assess the value across repair, analysis, and branching **Accepted execution lifecycle:** compose completed executions. Finish a turn
conversation examples before choosing the abstraction. If PromptKit provides and release its logical admission lease before invoking application work. Each
coordination, compose completed executions: finish a turn and release its later turn receives ordinary admission. The coordinator holds no backend
admission lease before invoking application work. Each later turn receives capacity while awaiting application work. Callbacks can inspect responses,
ordinary admission. A coordinator would hold no backend capacity while awaiting consult application data, wait for application input, or make model calls against
application work. This reuses existing execution machinery and permits the same engine and backend, including at capacity one. PromptKit need not
callbacks to make their own model calls without interpreting their purpose. interpret the purpose of that work.
**Alternative:** provide continuation and observation primitives while leaving **Alternatives considered, not selected:**
looping to consumers. This keeps PromptKit's API smaller, but may leave repeated
execution and accounting coordination downstream. Compare the actual consumer
work under both approaches. Extending the structural-repair loop is another
possible implementation route, but would require changing lease ownership and
decoupling it from repair-specific assumptions.
**Details to settle:** whether to provide coordination, looping, or both; scope - Provide only continuation and observation primitives. This would keep the
over a single branch versus multiple branches; callback cancellation boundaries; public API smaller, but leave repeated lifecycle and accounting coordination
and the sequence of request construction, execution, and application decisions. downstream. Independent primitive use remains supported alongside the selected
Applications retain the meaning of those decisions and policy for stopping coordinator.
other work after a failure. - Provide a coordinator for an entire branching tree. This could manage more
work, but introduces scheduling, concurrency, joining, and failure-policy
questions without demonstrated shared answers across consumers. Applications
can build those workflows from the selected branch-level capabilities.
**Details to settle:** callback cancellation boundaries, exact request/stop and
outcome types, final-turn delivery at budget exhaustion, and error/report
semantics belong to Decision 4. Execution association and usage accounting belong
to Decision 5. The sketches below illustrate the selected responsibility split;
they do not settle those public contracts.
#### Illustrative Consumer Sketches
These are pseudocode, not implemented Go APIs or selected exported names.
`coordinate`, `Continue`, `Stop`, and callback state fields are illustrative.
`request` represents an ordinary PromptKit request, with the selection rules
accepted in Decision 2. Each sketch starts from an already captured exchange.
The callback can keep application state in a closure; PromptKit need not define
a universal domain-state or feedback schema.
**Analysis loop:** the application examines the latest response and selects a
question or decides that its investigation is complete.
```text
analysis = application.start_analysis(initial_exchange)
choose_next(ctx, turn):
if turn.new_result exists:
analysis.record(turn.new_result)
question = application.choose_next_question(analysis)
if question is absent:
return Stop()
return Continue(request(
prompt_id = "research/investigate",
vars = {"question": question}
))
report, error = coordinate(
ctx, start = initial_exchange,
additional_turns = 3, next = choose_next
)
application.present_analysis(analysis, report, error)
```
PromptKit preserves the history and performs each selected execution. It does
not decide whether the response answered the question, whether the evidence is
sufficient, or what to investigate next. The application callback may consult
other sources or call the engine before returning its next action.
**Audita branch:** the application starts with one already-assessed correction
and its rejection findings. It interprets and assesses each new response; the
coordinator uses the same generic controls as the analysis loop.
```text
branch = audita.branch(candidate, existing_findings)
choose_next(ctx, turn):
if turn.new_result exists:
response = audita.decode_response(turn.new_result)
if audita.is_withdrawal(response):
branch.mark_withdrawn()
return Stop()
branch.update_candidate(response)
assessment = audita.assess(ctx, engine, branch.candidate)
branch.record(assessment)
if audita.should_stop(assessment):
return Stop()
return Continue(request(
prompt_id = branch.configured_follow_up_prompt,
vars = audita.follow_up_variables(branch)
))
report, error = coordinate(
ctx, start = initial_exchange,
additional_turns = 3, next = choose_next
)
audita.record_branch_outcome(branch, report, error)
```
The first callback uses existing findings, so initial batch validation is not
repeated. `audita.assess` may make model calls after the turn's admission lease
has been released. Only Audita decides that a correction is accepted, withdrawn,
or unavailable, and retains those meanings in its own state. Further corrections
can start separate coordinators from the same initial exchange; accepted sibling
corrections and other branches remain with Audita.
Both sketches omit application-specific error handling and depict normal turn
progression. They do not prescribe how callback errors or the final result at
the budget boundary are delivered; Decision 4 must specify those paths so the
application can handle completed work without mistaking it for accepted content.
The concrete interface should let both examples use the same coordination
mechanism without manual message-history assembly or per-turn lifecycle
bookkeeping. Callback model-call association remains part of Decision 5.
### 4. How Are Application Next Actions And Execution Outcomes Represented? ### 4. How Are Application Next Actions And Execution Outcomes Represented?
**Status: open.** Subject to Decision 3, define content-neutral next actions and **Status: open.** For the coordinator selected in Decision 3, define
distinguish application-directed completion, budget exhaustion, interruption, content-neutral next actions and distinguish application-directed completion,
and operational failure. budget exhaustion, interruption, and operational failure.
**Recommendation:** let the application request another turn or indicate that **Recommendation:** let the application request another turn or indicate that
the interaction should stop, separately from reporting an operational error. the interaction should stop, separately from reporting an operational error.
@@ -194,22 +295,22 @@ small explicit next-action result without introducing domain-specific statuses.
**Details to settle:** next-action and outcome vocabulary, callback/result types, **Details to settle:** next-action and outcome vocabulary, callback/result types,
result-plus-error behavior, zero-budget behavior, and what consumes an result-plus-error behavior, zero-budget behavior, and what consumes an
additional-turn budget when preparation or generation fails. Keep such a budget additional-turn budget when preparation or generation fails. Keep such a budget
distinct from structural repair and transport retry. If coordination is left distinct from structural repair and transport retry. Specify how an application
to consumers, limit this decision to continuation execution outcomes. receives the last completed response when no further turns remain, including
whether and how its callback is invoked at that boundary.
### 5. How Are Related Executions And Usage Accounted For? ### 5. How Are Related Executions And Usage Accounted For?
**Status: open.** Define accounting for the initial exchange, each continuation, **Status: open.** Define accounting for the initial exchange, each continuation,
its structural repairs and transport retries, and model calls made by its structural repairs and transport retries, and model calls made by
application callbacks, whether or not PromptKit supplies a coordinator. application callbacks, both for standalone continuations and coordinated turns.
The current [observation boundary](../internal/observation.md) keeps nested The current [observation boundary](../internal/observation.md) keeps nested
invocations independent. A callback's arbitrary model calls cannot simply be invocations independent. A callback's arbitrary model calls cannot simply be
assumed to belong to the enclosing interaction's totals. assumed to belong to the enclosing interaction's totals.
**Recommendation:** preserve independent run accounting and add explicit **Recommendation:** preserve independent run accounting and add explicit
relationships between exchanges, branches, and associated runs, with sequence relationships between exchanges, branches, sequences, and associated runs.
relationships if coordination is selected.
Choose a small association mechanism, such as a scoped observation facility or Choose a small association mechanism, such as a scoped observation facility or
explicit execution references, so consumers need not rebuild lifecycle tracking. explicit execution references, so consumers need not rebuild lifecycle tracking.
Shared initial work should be referenced by branches and counted once when Shared initial work should be referenced by branches and counted once when
@@ -256,9 +357,10 @@ an implementation plan. The recommended sequence is:
1. Deliver capture and independent continuation through the public facade, 1. Deliver capture and independent continuation through the public facade,
allowing Audita to remove its reconstruction and prefix-verification work. allowing Audita to remove its reconstruction and prefix-verification work.
2. If selected after evaluating general usefulness, build coordination and/or 2. Build the optional sequential coordinator on that capability, reusing
bounded looping on that capability, reusing ordinary execution, validation, ordinary execution, validation, capacity, and observation machinery. Refine
capacity, and observation machinery. both consumer sketches against the selected public contracts to verify that
the same mechanism removes useful repeated work in each application.
Use [Audita's acceptance scenarios](audita-request.md#suggested-delivery-and-acceptance-scenarios) Use [Audita's acceptance scenarios](audita-request.md#suggested-delivery-and-acceptance-scenarios)
as consumer checks, alongside analysis and branching-conversation examples that as consumer checks, alongside analysis and branching-conversation examples that