diff --git a/README.md b/README.md index 7af961e..f8fdb03 100644 --- a/README.md +++ b/README.md @@ -45,6 +45,9 @@ boundary and constraints that framework work must preserve. ## Release Guidance +The [v0.22.0 release notes](docs/releases/v0.22.0.md) cover captured conversation +exchanges, application-directed turn coordination, and explicit execution groups. + The [v0.21.0 release notes](docs/releases/v0.21.0.md) cover ConfigKit v1.2.0 and main-relative companion imports through the existing configuration-source API. diff --git a/docs/releases/v0.22.0.md b/docs/releases/v0.22.0.md new file mode 100644 index 0000000..d859ec2 --- /dev/null +++ b/docs/releases/v0.22.0.md @@ -0,0 +1,67 @@ +# Promptkit v0.22.0 + +This minor release adds captured conversation exchanges, optional sequential +coordination, and explicit execution-group summaries. Applications can continue +or branch a conversation without reconstructing rendered history, while keeping +response interpretation and workflow decisions in application code. + +## Capture And Continue Conversations + +Set `RunRequest.CaptureExchange` to capture a completed run's effective messages +and final assistant content. Supply its `RunResult.Exchange` as `ContinueFrom` +on a new request to continue it. Exchanges are reusable, support independent +branches and different engines, and expose detached history through `Details()`. +Prepared execution also supports capture of the exact snapshot it executes. + +Internal structural-repair instructions and intermediate candidates are not +conversation history. A completed result with failed structural validation can +still be captured; an operational error returning no result creates no exchange. +Follow-ups select their own profiles, output contracts, credentials, and sessions +under ordinary request rules. See [capture and continuation guidance](../consumers/pkg-promptkit.md#capture-and-continue-a-conversation) +and the [public exchange contract](../../exchange.go). + +## Coordinate Application-Directed Turns + +`Engine.Coordinate` advances one conversation branch using an application callback +and an additional-turn budget. The application selects the next request or stops; +Promptkit handles turn progression and cancellation. Each completed run releases +backend admission before the callback, allowing assessment calls to use the same +backend. Stop and budget exhaustion are control-flow outcomes, not judgments of +response acceptance. Transport retries and structural repairs remain separate +from the turn budget. + +See the [coordination walkthrough](../consumers/pkg-promptkit.md#coordinate-application-directed-turns) +and [coordinator GoDoc](../../coordinate.go) for callback and cancellation behavior. + +## Collect Selected Execution Facts + +`ExecutionGroup` collects explicitly selected execution observations, while +`ComposeObservers` combines groups with existing diagnostic observers. Detached +group snapshots report member identities, terminal coverage, attempt/retry/repair +counts, and available token totals without counting a run more than once. +Terminal coverage remains distinct from token-usage completeness; membership is +not inferred from ancestry or nested calls. + +See [execution-group guidance](../consumers/pkg-promptkit.md#collect-selected-execution-facts), +the [group GoDoc](../../execution_group.go), and the updated +[offline observation example](../../examples/go-library/observe/main.go). + +## Efficiency And Compatibility + +History assembly and inspection avoid intermediate full-history copies. Observer +composition retains storage only for selected callbacks and avoids fan-out +closures for zero or one selected callback. + +Capture and coordination are opt-in; existing execution workflows and +configuration formats need no migration from v0.21.0. The Go consumer minimum +remains 1.27.0, and dependency versions are unchanged. Exported request, prepared, +result, and observation structs gained fields; consumers using positional struct +literals must switch to keyed literals. Continuation metadata adds `parent_run_id` +to prepared/result JSON when applicable; the opaque exchange itself is excluded. +Exchanges are in-process capabilities, not serialized restartable sessions. + +```sh +go get go.maximumdirect.net/promptkit@v0.22.0 +``` + +[COPYRIGHT](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.22.0/COPYRIGHT) | [LICENSE](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.22.0/LICENSE) | [ATTRIBUTION](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.22.0/docs/oss/attribution.md) diff --git a/docs/roadmap/audita-request.md b/docs/roadmap/audita-request.md deleted file mode 100644 index 4dcd54b..0000000 --- a/docs/roadmap/audita-request.md +++ /dev/null @@ -1,169 +0,0 @@ -# Audita request: conversation continuation and application-directed repair - -Status: proposed upstream features, requested by Audita. Public API design and -implementation remain for PromptKit to determine. - -This request is based on Audita's use of PromptKit v0.21.0. It proposes two -related capabilities: captured conversation continuations, followed by bounded -repair driven by application-supplied validation feedback. The goal is to move -reusable execution machinery into PromptKit and remove its local equivalents -from Audita. - -## Audita's workflow - -Audita polishes transcripts using module-specific proposal, validation, and -repair prompts: - -1. A proposal request supplies transcript segments and receives a structured - collection of proposed corrections. -2. Audita evaluates each correction using deterministic and LLM-backed - validators. Accepted corrections are retained for later application. -3. Each actionable rejection starts an independent repair branch containing the - original rendered request, the full final assistant response, and a follow-up - instruction rendered from that module's repair prompt definition. -4. The follow-up identifies one correction and supplies its current content, - available before-and-after transcript context, and validation findings. The - model can revise or withdraw that correction. This response uses a different - schema from the original collection of proposals. -5. A revision passes through the validators again. Further attempts retain that - branch's history and receive updated feedback. Acceptance, withdrawal, - exhaustion, or failure ends the branch; independent approvals are preserved. - -Audita owns the meaning of a correction, the validation rules, repair -eligibility, and eventual application to the transcript. Conversation handling -and the bounded revision lifecycle are useful beyond transcript processing. - -## Existing support and the remaining gap - -PromptKit already supports [structural output repair](../consumers/pkg-promptkit.md#repair-a-structured-result), -[appended messages](../consumers/pkg-promptkit.md#append-already-rendered-messages), -prepared execution, profiles, execution observations, and backend capacity -management. These allow Audita to implement the workflow today. - -Audita still captures exchanges, reconstructs the original prompt, verifies its -rendered prefix, assembles branch histories, renders follow-up instructions, -and coordinates repeated application validation and revision. The requested -APIs would make those responsibilities supported PromptKit capabilities. - -## 1. Captured conversations and independent continuations - -Provide a public way to capture a rendered request and its full final assistant -response, then continue that exchange without making the consumer reconstruct -the original prompt and check that its messages are unchanged. - -Audita needs to: - -- Preserve the original rendered message content, roles, and cache metadata - when continuing an exchange. -- Start multiple independent branches from the same exchange. A branch's later - instructions and responses must not alter the original exchange or another - branch's history. -- Render follow-up instructions using ordinary PromptKit prompt definitions and - variables, and append them to the selected branch. Audita's repair prompts - remain operator-replaceable through its existing PromptKit configuration. -- Select a follow-up profile and output contract independently of the initial - request. The first response may contain many items while a continuation - returns one revised item or a withdrawal. -- Retain the relationship between the original exchange, the follow-up prompt - definition, and each execution in inspection and observation data. - -An in-process capability is sufficient for Audita. Persistence, restartable -sessions, and a conversation database are not required by this request. The -existing [prepared execution handle](../../prepared_execution.go) is single-use; -the request does not assume that copying or reusing it creates a continuation. - -## 2. Bounded application-directed repair - -Build a reusable revision coordinator on the continuation capability. Audita -should supply its assessment and request-building behavior while PromptKit -handles the repeated execution lifecycle. - -The coordinator needs to support: - -- Starting with an already-assessed candidate and its rejection findings, so - Audita need not repeat the initial batch validation simply to enter repair. -- Application-supplied assessment of each revision, including deterministic - checks and checks that make further PromptKit calls. Findings may come from - multiple validators; their meaning and explanatory content remain with the - application. -- Application-selected follow-up prompts, variables, profiles, and response - contracts. Audita supplies the candidate and context, and interprets the - response as a revision or withdrawal. PromptKit need not prescribe an Audita - correction schema or a universal feedback format. -- Reassessment before a revised candidate can be accepted, with fresh feedback - for the next attempt and accumulated history confined to that branch. -- A caller-selected additional-revision budget and caller context, with clear - terminal outcomes covering acceptance, withdrawal, exhaustion, unavailable - results, and interruption. Callers retain decisions about repair eligibility - and whether an operational failure should terminate other work. -- Attempt observations, execution relationships, and available usage accounting - sufficient to report a repair sequence without recreating its lifecycle - tracking in Audita. - -The revision budget must remain distinct from transport retry and structural -repair budgets. A structurally valid response can still fail Audita's checks; -each revision generation can itself need structural repair. Accounting should -distinguish those layers and avoid double-counting shared initial work. - -Application validators must be able to call the same engine and backend without -being blocked by capacity retained by their parent operation. Coordinating -ordinary completed runs is one possible approach. PromptKit should determine -the implementation in light of its [execution and admission lifecycle](../internal/runner.md#run-flow). - -This is broader than exposing the current structural-repair hook: the response -schema can change between proposal and repair, validation may perform model -calls, and each rejected item has its own history and disposition. - -## Ownership boundary - -PromptKit would own captured exchanges, branch continuation, the bounded repair -lifecycle, and associated execution observations. Audita would retain: - -- Transcript segments, proposal identities, glossary and protected-term rules, - editorial judgments, and meaning checks. -- Validator selection and ordering, actionable-rejection policy, feedback - content, and module-specific proposal, validation, and repair prompts. -- Before-and-after transcript previews, preservation of accepted sibling - proposals, application ordering, and final transcript mutation. -- Application reports, debug artifact destinations, and CLI behavior. - -The separate idea of reconciling missing or duplicate items in batch responses -is outside this request. The two requested capabilities should be usable -through the supported public facade without an Audita dependency or a general -workflow framework. Exact types, callbacks, and configuration syntax are left -to PromptKit's API design. - -## Suggested delivery and acceptance scenarios - -Deliver conversation continuation first, allowing Audita to replace its capture -and replay machinery. Then add application-directed repair and replace the -corresponding local coordination. Existing prompt definitions and domain -validators should remain usable through the new APIs. - -The following scenarios would demonstrate that the capabilities meet Audita's -needs: - -1. An initial response contains three proposals. One is accepted; the other two - receive independent repair conversations containing the same original - exchange. Revising either branch leaves the accepted proposal and the other - branch unchanged. -2. A follow-up uses a separate prompt definition, profile, and single-item - response schema. Inspection identifies the follow-up and its originating - exchange. -3. A revision fails multiple application checks, receives their updated - feedback, and passes on a later attempt. An LLM-backed check can use the same - engine and backend even when backend capacity is limited to one. -4. Withdrawal, exhausted revision budget, unavailable validation, and - cancellation produce distinguishable outcomes. An unaccepted revision is - never returned as an accepted candidate, and a recoverable branch failure - leaves independent approvals available to the application. -5. A revision generation needs both a transport retry and structural repair. - Observations distinguish these from application-directed revision attempts - and expose the available usage without counting the original exchange once - per branch. - -For implementation reference, Audita currently owns these mechanics in -`internal/framework/llm/adapter.go`, -`internal/framework/structuredoutput/call.go`, and -`internal/framework/runner/repair.go`. These are examples of the consumer need, -not proposed PromptKit package boundaries or public API shapes. diff --git a/docs/roadmap/conversation-continuation.md b/docs/roadmap/conversation-continuation.md deleted file mode 100644 index 1249a70..0000000 --- a/docs/roadmap/conversation-continuation.md +++ /dev/null @@ -1,448 +0,0 @@ -# Conversation Continuation And Execution Groups - -Status: target behavior accepted; concrete API choices and implementation steps -are recorded in the [implementation plan](implementation.md). This document -defines the target behavior and responsibility boundaries. Execution groups are -independently usable. - -## Purpose And Responsibility Boundary - -PromptKit will let applications continue logical conversations, optionally -coordinate successive turns, and summarize explicitly selected executions. -These capabilities remove repeated history reconstruction, turn lifecycle work, -and observation aggregation without interpreting prompt or response content. - -[Audita's request](audita-request.md) supplies consumer evidence and acceptance -scenarios. Repair, analysis, clarification, and branching exploration should all -use the same content-neutral capabilities. New concepts and names must be useful -without translating an application's domain into repair terminology. This does -not rename or expand PromptKit's existing structural-output repair functionality. - -| Capability | PromptKit owns | Application owns | -| --- | --- | --- | -| Captured exchanges and continuation | Logical history, reusable branches, ordinary follow-up execution, exchange details and parent metadata | Which exchange to continue and which request to execute | -| Optional sequential coordinator | One branch's turn progression, additional-turn budget, callback delivery and cancellation checks | Response interpretation, next action, branch scheduling and broader failure policy | -| Optional execution groups | Explicitly selected observations, deduplication and execution-fact summaries | Membership selection, domain identifiers, final-report timing and presentation | - -Continuation is independently usable. The coordinator builds on it; applications -can instead use their own loops. Groups work with ordinary runs as well as -continuations and require neither capture nor a coordinator. Selecting a group -does not change scheduling, capacity, cancellation, retries, or validation. - -The scope is in-process use. Persistence, restartable workflows, cross-process -resumption, a conversation registry, batch reconciliation, and a branching-workflow -scheduler are outside this feature. Applications schedule branches, join results, -and decide whether one branch's failure should affect other work. This initial -scope is not a permanent prohibition on future capabilities. - -## Captured Exchanges And Continuation - -### Capture And Availability - -Capture is opt-in per ordinary or prepared execution. Ordinary execution returns -the exchange alongside its usual result. A prepared handle retains the capture -choice; executing that exact prepared snapshot creates the exchange. Preparation -alone creates no completed exchange. Continuation implies capture so its result -can itself be continued. - -Capture the request actually executed, including the complete rendered messages -with their content, roles, and cache metadata, followed by the full final -assistant content returned by the run. Consumers must not reconstruct the request -or pair a response with separately prepared data. “Full” means content supported -by PromptKit, not a new raw provider-envelope API. - -Capture represents logical conversation history. Internal structural-repair -instructions and intermediate candidates are excluded; only the final returned -assistant content follows the original request. Existing attempt observations -remain separate from conversation content. - -| Execution outcome | Capture behavior | -| --- | --- | -| Completed response satisfying its output contract | Return the captured exchange. | -| Completed response with failed structural validation, including repair exhaustion | Return the exchange with the ordinary validation result intact. | -| Operational error with no completed result | Create no new exchange; the preceding exchange remains usable. | -| Completed response obtained before cancellation prevents coordinator delivery | Preserve that single undelivered response under the coordinator return contract. | - -Capture does not imply structural validity or application acceptance. A completed -response with failed validation follows the normal coordinator callback path, -subject to cancellation; it is not reclassified as an operational error. - -### Ownership And Reuse - -An exchange is an immutable, reusable snapshot held by the caller. Continuing it -adds the rendered follow-up messages and final response to a new exchange, leaving -the source and independent branches unchanged. No previous templates or input -files need to be read or rendered again. - -Avoid unnecessary repeated copying of retained history where practical, while -preserving independent branches and detached inspection values. Internal sharing -of immutable history is an implementation option, not a required representation -or a public allocation or complexity guarantee. - -Another engine instance in the same process can continue the exchange, applying -its own configuration to the new request. The originating engine is not required. -This permits application engine replacement without promising serialized or -cross-process resumption. - -Completed exchanges retain no backend admission lease, direct credential, or -execution callback. Ordinary Go memory management controls their lifetime; there -is no new `Close` or `Discard` lifecycle. The existing -[prepared execution handle](../../prepared_execution.go) remains a separate, -single-use execution capability. - -### Exchange Details - -Provide explicit access to the held exchange's logical message history, including -its final assistant content, originating run identity, and immediate parent -reference. Reuse existing run identifiers; no mandatory conversation, branch, or -sequence identifiers are introduced. - -Follow the general pattern of `PreparedExecution.Details()`. Details must be -generically renderable through ordinary encoders. Reading or modifying returned -values must not affect retained history or another branch. Inspection reads the -snapshot without rendering, loading inputs, or making provider calls. - -Existing results and preparation details continue to supply ordinary execution -information. Exchange inspection does not create a second comprehensive report, -a result archive, or an engine-level conversation lookup API. `InspectPrompt`, -`InspectProfile`, and `InspectBackend` keep their resource-inspection boundary. -Captured message content remains separate from metadata-only observations. - -## Optional Sequential Coordinator - -The coordinator starts from a captured exchange and executes one branch using -an application callback, a caller-selected additional-turn budget, and context. -The callback selects a new ordinary request, stops, or returns an error. It can -keep application state in a closure; PromptKit defines no domain-state or -feedback schema. - -### Progression And Budget - -With an active context, invoke the initial callback once, even with a zero-turn -budget. It receives the starting exchange and remaining budget; the starting -exchange is not a newly delivered response. Existing application findings can -be used without repeating earlier assessment. - -After each continuation, deliver its completed response once through the same -callback before starting another execution, including the final permitted -response. Internal retries and structural repairs do not create extra callbacks. - -| Callback action | Coordinator behavior | -| --- | --- | -| Stop | Return application-directed completion, including after the last permitted response. | -| Continue with budget remaining | Start the selected continuation. | -| Continue without budget remaining | Return exhaustion without executing the requested turn. | -| Error | Stop and propagate the error, subject to context checks. | - -The budget permits at most N additional continuation executions to start. -Starting a continuation consumes one turn even if preparation or generation -fails. The initial exchange and callback invocations consume no turns; internal -transport retries and structural repairs stay within one turn. The first -operational error stops coordination, without refunds or recovery machinery. - -Using the full budget alone does not mean exhaustion: exhaustion means the -application requested more work than the budget permits. The budget does not -bound provider attempts, cost, or work inside callbacks. Applications wanting -no callback work can skip the coordinator entirely. - -### Cancellation And Failures - -Check context immediately before every callback, including the initial callback. -If cancellation is already observed, skip delivery and return the context error -with its identity preserved. Retain a completed response already obtained but -not yet delivered; do not recover raw output from an underlying failed call that -returned no completed result. - -Callbacks receive the same context and cooperate with cancellation. Check it -again after the callback returns and before starting another execution. -Cancellation may race with invocation; PromptKit does not forcibly terminate -callback code or promise callback delivery despite cancellation or panic. - -An execution or callback error stops coordination. Preserve ordinary Go error -identities, including `errors.Is`/`errors.As`. Do not automatically replay the -failed continuation or callback. Ordinary transport retries and structural repair -remain execution-internal. Earlier deliveries and application work remain intact; -the application owns any broader consequences. - -### Minimal Return Contract - -Return only: - -- An ordinary Go error for execution failure, callback failure, or cancellation. -- A small normal outcome distinguishing application-directed stop from budget - exhaustion; neither outcome is an error or a claim about content acceptance. -- An optional single completed response when cancellation prevented callback - delivery, reusing existing response types where practical. - -Delivery means callback invocation, not successful processing or acceptance. If -the callback returns an error, its response is already delivered and is neither -returned as undelivered nor replayed. Previously delivered responses are not -archived in the termination result. No additional failure hierarchy, failure-stage -fields, run-ID references, or aggregate reporting is added to this contract. -Existing response and observation fields remain available; group summaries are -obtained separately. - -## Shared Execution And Accounting Rules - -### Follow-Up Configuration - -A continuation is an ordinary new execution supplied with captured history. -Resolve its prompt, profile, credentials, output contract, and request overrides -using the receiving engine's existing rules. In particular: - -- Use an explicit request profile or the follow-up prompt's default profile, - with ordinary missing-profile behavior when neither is available. -- A supplied output contract completely replaces the default; do not merge it - with or inherit the preceding execution's contract. -- Resolve credentials for the new execution; retained history does not preserve - the original direct key. -- Select session ID from the new request or follow-up prompt's session template. - Do not implicitly inherit it. Applications may explicitly reuse a session ID; - ancestry does not depend on one. - -History introduces no additional execution-setting precedence. The follow-up -can change model or response schema without altering the original exchange. - -Assemble each continuation's messages in this order: - -1. The captured logical history, including its final assistant content. -2. The rendered follow-up prompt definition messages, in definition order. -3. The follow-up request's caller-appended messages, in caller order. - -Preserve authored roles and message content; do not move messages based on role. -Continuation adds no history-size limit or automatic truncation. Upstream -context-window or other request rejections follow ordinary generation-error -behavior. - -### Capacity And Application Work - -Compose completed executions. Finish each turn and release its logical admission -lease before calling application code. Releasing only the active generation -permit is insufficient under the existing [run flow](../internal/runner.md#run-flow). -Each later execution acquires ordinary admission. - -The coordinator holds no backend capacity while awaiting callback work. -Callbacks may inspect responses, consult data, wait for application input, or -call the same engine and backend, including when backend capacity is one. - -### Ancestry And Per-Run Accounting - -Each continuation retains its own originating run ID and exposes its immediate -parent exchange's run ID. Parent metadata describes history provenance; group -membership describes application-selected accounting. Neither implies the other. -A validator can join a continuation's accounting group without extending its -conversation. - -Reuse ordinary per-run usage and observations, including attempt, retry, repair, -failure-path, and usage-presence/completeness semantics. Preserve the existing -[observation boundary](../internal/observation.md): arbitrary nested calls are -independent executions. Do not infer membership from a callback's purpose or -context. - -If initial run A supplies branches B and C, its usage belongs only to A. B and C -do not inherit that cost. Usage reported for resending captured history in B and -C still belongs to those new runs. Count selected runs once each; never add -terminal totals to the attempt events they already summarize. - -## Optional Execution Groups - -### Explicit Membership And Observer Composition - -Use an observer-compatible collector attached explicitly to selected executions. -Prepared requests retain the selection; membership starts when execution emits -observations. Preparation alone, discarded handles, and failed claims do not -enroll runs through observations. - -Coordinator selection covers its continuation runs. Callback-initiated calls -require their own explicit selection. Neither nested context nor a captured -starting exchange automatically enrolls another run. - -Support an application's existing diagnostic observer alongside one or more -collectors through a small observer-composition helper. Preserve synchronous -delivery and existing panic behavior. Do not introduce asynchronous queues, -panic recovery, or guaranteed delivery to later observers after an earlier one -panics; interrupted delivery can leave accounting incomplete. - -A run may explicitly belong to multiple independent groups. Each group counts -its run identity once. Applications obtain operation-level and overall views by -attaching the corresponding collectors to each selected run. Overlapping totals -are not generally additive; no group hierarchy, inherited membership, or merging -facility is needed. - -### Concurrent Snapshots And Lifetime - -Provide detached, internally consistent snapshots from a caller-held group while -selected executions and snapshot reads may occur concurrently. Reading or -modifying a snapshot must not change the collector or another snapshot. There -is no mandatory close, seal, or wait lifecycle. Applications own scheduling and -decide when to read or discard the group. - -Distinguish observed runs with terminal observations from those without them. -Missing terminal observations mean incomplete accounting; the group does not -infer whether work is active, delivery was interrupted, or another cause applies. -It promises no terminal delivery beyond ordinary observation semantics. - -“All observed members have terminal observations” does not mean the application -operation is complete. More runs can join later, and PromptKit cannot know whether -the application selected all relevant calls. The application chooses when a -snapshot represents its final report. - -### Summary Contents - -The initial summary exposes: - -- Member run identities and terminal-observation coverage. -- Existing provider-attempt, transient-retry, and structural-repair counters. -- Available token totals with per-field known/completeness information. - -Aggregate one terminal observation per run. These observations already summarize -retries, repairs, and available failure-path usage, including executions returning -no ordinary result. Do not reconstruct totals from attempts. - -Keep terminal coverage distinct from usage completeness. A terminal observation -may itself have incomplete usage. Two finished runs with fully reported usage -provide a useful known subtotal, but a third observed run lacking a terminal -event leaves group accounting incomplete. Preserve existing usage semantics and -expose missing coverage separately; unavailable evidence must not become known -zero. - -Live attempt-level totals and duration aggregation are omitted from the initial -implementation. Applications can use existing events for progress displays. If -durations are later added, summed execution durations must be distinguished from -elapsed group time. Groups supply execution facts, not monetary cost, application -outcomes, or an application-wide reporting framework. Applications retain domain -mappings, interpretation, and report assembly. - -## Consumer Sketches - -These are pseudocode, not selected exported names or implemented Go APIs. -Each begins with an already captured exchange and uses ordinary follow-up request -selection. Both omit application-specific error handling, including checks for -failed structural validation before schema-specific decoding. - -### Analysis Loop - -The application records responses and selects its next question or decides that -its investigation is complete. PromptKit retains history and executes the turns. - -```text -analysis = application.start_analysis(initial_exchange) - -choose_next(ctx, turn): - if turn.new_result exists: - analysis.record(turn.new_result) - - question = application.choose_next_question(analysis) - if question is absent: - return Stop() - - return Continue(request( - prompt_id = "research/investigate", - vars = {"question": question} - )) - -termination, error = coordinate( - ctx, start = initial_exchange, - additional_turns = 3, next = choose_next -) -application.present_analysis(analysis, termination, error) -``` - -### Audita Branch - -Audita starts with an already-assessed candidate and existing findings. Only -Audita interprets acceptance, withdrawal, or assessment unavailability. - -```text -branch = audita.branch(candidate, existing_findings) - -choose_next(ctx, turn): - if turn.new_result exists: - response = audita.decode_response(turn.new_result) - if audita.is_withdrawal(response): - branch.mark_withdrawn() - return Stop() - - branch.update_candidate(response) - assessment = audita.assess(ctx, engine, branch.candidate) - branch.record(assessment) - if audita.should_stop(assessment): - return Stop() - - return Continue(request( - prompt_id = branch.configured_follow_up_prompt, - vars = audita.follow_up_variables(branch) - )) - -termination, error = coordinate( - ctx, start = initial_exchange, - additional_turns = 3, next = choose_next -) -audita.record_branch_outcome(branch, termination, error) -``` - -The initial callback uses existing findings without repeating batch assessment. -`audita.assess` may call the same backend after the turn's admission lease is -released. Those calls join any group only through their own explicit selection. -Other candidates can start independent branches; Audita owns sibling outcomes. - -Both sketches process the final permitted response through the normal callback. -Applications retain their received results; the coordinator's termination value -is not an archive. These examples should remain usable through the same generic -API without manual history assembly or per-turn lifecycle bookkeeping. - -## API And Implementation Ownership - -The [implementation plan](implementation.md) selects the concrete API and steps -for reaching this target. Both consumer sketches must remain expressible without -manual history assembly or per-turn lifecycle bookkeeping. Execution groups -remain independent of capture and coordination; the plan's ordering does not -introduce a product dependency between them. - -## Acceptance Scenarios - -Use [Audita's acceptance scenarios](audita-request.md#suggested-delivery-and-acceptance-scenarios) -alongside analysis and branching-conversation examples. Validate each capability -at its public boundary without adding guarantees beyond the selected behavior. - -### Capture And Continuation - -- Ordinary and prepared opt-in capture uses the actual executed snapshot; - preparation alone produces no exchange. -- Internal structural repair contributes only the full final assistant content - to history. Final validation failure still permits capture; an operational - failure without a completed result does not. -- Independent branches preserve prior history and outcomes, including after - original inputs change or disappear and when continued by another engine. -- Follow-up configuration, credentials, session selection, and complete output - overrides follow ordinary rules without implicit inheritance. -- Details encode generically, expose originating and parent identities, and are - detached from retained state; inspection performs no rendering or external reads. - -### Coordination - -- Initial callback, zero-turn behavior, last-response delivery, stop versus - exhaustion, and execution-start budget consumption follow the specified rules. -- Cancellation before delivery retains only an already-obtained undelivered - response; callback errors do not replay delivered responses or earlier work. -- Failed structural validation follows normal response delivery, while operational - errors preserve Go error identities and stop progression. -- Same-backend callback calls at capacity one proceed after logical admission - release. Internal retries/repairs do not create extra conversation callbacks. -- Both consumer sketches work without PromptKit interpreting content or owning - application outcomes, branch scheduling, or a response archive. - -### Observations And Groups - -- Per-run usage and failure diagnostics preserve ordinary presence/completeness - semantics and parent references distinguish turns and branches. -- Explicit ordinary, prepared, coordinator, and callback membership works alongside - an existing observer; unrelated nested calls remain independent. -- Concurrent collection and snapshots preserve detached, consistent results. - More members may join after a snapshot with full terminal coverage. -- Overlapping groups count each run once within each group. Shared initial work, - newly reported replay usage, and attempt/terminal events are not double counted. -- Known subtotals, explicit zeros, incomplete usage, and missing terminal coverage - remain distinguishable, including on failure paths. -- Observer composition preserves synchronous delivery and panic behavior without - stronger terminal-delivery promises. diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md deleted file mode 100644 index 8861bd7..0000000 --- a/docs/roadmap/implementation.md +++ /dev/null @@ -1,659 +0,0 @@ -# Conversation Continuation And Execution Groups Implementation Plan - -## Objective And Sprint Rules - -Implement the accepted [feature roadmap](conversation-continuation.md), using -[Audita's request](audita-request.md) as consumer evidence. The feature roadmap -owns product behavior and boundaries; this plan selects the API and work order. -Existing structural-output repair remains a separate, unchanged capability. - -Each numbered stage is one Luna-sized implementation prompt. Execute them in -order during one end-to-end sprint. Intermediate stages need not compile or -constitute independently releasable changes. Do not build compatibility shims or -temporary implementations solely to make a stage green. Carry a concise record -of remaining wiring into the next stage; the final stage must leave the whole -repository coherent and validated. There are no intermediate human-review, -external-publication, provider-access, or dependency-release gates. - -Start with [development guidance](../development.md). Follow the applicable -[agent](../policy/agents.md), [architecture](../policy/architecture.md), -[testing](../policy/testing.md), and [documentation](../policy/documentation.md) -policies. Preserve the accepted [configuration boundary](../adr/0001-configuration-boundary.md). -This sprint needs no configuration format, dependency, Go minimum, release -tooling, or licensing changes. It does not authorize tagging or publication. - -Use real fast internal collaborators and replace only external boundaries in -tests. Select focused checks during stages and run the complete checks at the -end. Do not duplicate existing retry, repair, schema, or capacity test matrices. -Protect the new interactions with those mechanisms. - -## Current Baseline And Integration Points - -- The root facade in `engine.go` converts public requests, masks inherited - observation state, invokes the internal engine, maps errors, and finishes - observations after the runner releases admission. Preserve this boundary. -- `internal/usecase/runner.go` has shared resolution and completion preparation - phases. `completePreparationWithStructuredOutput` currently combines rendered - definition messages with appended messages and hashes the resulting prompt. - `executePreparedRun` is shared by ordinary and prepared execution and returns - the final `RawOutput`, including after exhausted structural repair. -- `internal/usecase/prepared_execution.go` retains a single-claim payload, - validation plan, direct credential, retry budget, and observer separately from - durable prepared details. Claims remain bound to the originating execution - owner. Details survive execution and discard. -- `internal/domain/message.go` owns message cloning and cache-control ownership. - Reuse it rather than inventing a second normalization contract. -- `observations.go`, `internal/domain/observation.go`, and - `internal/observation/recorder.go` own value-only events, run IDs, terminal - counters, and presence-aware usage. Terminal observations already include - completed work preceding operational failure. Do not total result usage or - attempt events on top of terminal totals. -- `internal/engine/engine.go` applies namespace views to new caller selections. - Captured history requires no original-engine lookups or namespace rewriting. -- Root conversion, `json.go`, formatting, and contract tests must account for - additions without changing existing timing, credential, or error behavior. - -## Selected Public API - -Implement these names and shapes in the root `promptkit` package. Code below -specifies additions and complete new types, not replacement definitions for -existing types. Keep all implementation machinery under `internal/`. - -### Capture, Continuation, And Ancestry - -```go -// Add to RunRequest. -CaptureExchange bool -ContinueFrom *Exchange `json:"-" yaml:"-"` - -// Add to RunResult. -Exchange *Exchange `json:"-" yaml:"-"` -ParentRunID string `json:"parent_run_id,omitempty"` - -// Add to PreparedRun. -ParentRunID string `json:"parent_run_id,omitempty"` - -// Add to ExecutionEvent, following its existing field/tag style. -ParentRunID string - -// Exchange has private state only. -type Exchange struct { /* private immutable snapshot reference */ } - -type ExchangeDetails struct { - RunID string `json:"run_id"` - ParentRunID string `json:"parent_run_id,omitempty"` - Messages []RenderedMessage `json:"messages"` -} - -func (e *Exchange) Details() ExchangeDetails -``` - -- `Run` with `CaptureExchange: true` returns `result.Exchange`. A non-nil - `ContinueFrom` supplies history and implies capture, regardless of the Boolean. - No separate continuation execution method is necessary. -- `Prepare`, `ValidateRequest`, and `PrepareExecution` accept the same request - fields and assemble the same effective messages. They create no completed - exchange. `PrepareExecution` retains capture selection and ancestry privately - for `RunPrepared`; plain preparation only returns its ordinary metadata. -- Nil `ContinueFrom` means no history. A non-nil, zero-value exchange is not a - captured exchange and fails as `ErrInvalidRequest` before source/model work. - `Details` on a nil or zero-value exchange returns a zero details value, - matching the prepared-handle accessor pattern. -- Details contain the complete logical history ending in the final assistant - content. They are detached and encodable with ordinary encoders. Give the - opaque handle constant `String` and `GoString` representations, following - `PreparedExecution`. Serializing ordinary results does not serialize the - capability or automatically disclose the additional conversation history; - callers explicitly encode `Details()` when desired. Do not add deserialization, - persistence, reconstruction, lookup, or lifecycle APIs. -- `ParentRunID` is the immediate source exchange's originating run ID, or empty - for initial runs. Expose it in prepared details, completed results, exchange - details, and ordinary observation events. Reuse the run ID already assigned to - the execution. It is metadata, not a provider request parameter, session ID, - group ID, or new source of execution settings. -- The follow-up's `PromptID`, definition hash, selected profile/backend, and input - hashes describe the new request. The rendered-prompt hash covers its complete - effective message sequence and selected session, using the existing hash - function. Do not hash capture flags or run identities into content equality. - -### Optional Coordinator - -```go -type ConversationTurn struct { - Exchange *Exchange - Result *RunResult - RemainingTurns int -} - -type CoordinateRequest struct { - Start *Exchange - AdditionalTurns int - Next func(context.Context, ConversationTurn) (*RunRequest, error) - Observer ExecutionObserver -} - -type CoordinationOutcome string - -const ( - CoordinationStopped CoordinationOutcome = "stopped" - CoordinationExhausted CoordinationOutcome = "exhausted" -) - -type CoordinationResult struct { - Outcome CoordinationOutcome `json:"outcome,omitempty"` - Undelivered *RunResult `json:"undelivered,omitempty"` -} - -func (e *Engine) Coordinate(ctx context.Context, req CoordinateRequest) (CoordinationResult, error) -``` - -- `Next` returning `(nil, nil)` stops. A non-nil request asks to continue; an - error stops with that error. No separate next-action enum or domain-outcome - taxonomy is needed. A zero request pointed to by a non-nil pointer is still - a continuation request, subject to ordinary validation if execution starts. -- `Start` must hold a captured exchange, `Next` must be non-nil, and - `AdditionalTurns` must be nonnegative; invalid values match `ErrInvalidRequest`. - A nil/uninitialized engine follows the existing `ErrInvalidConfig` convention. - Zero is a useful budget, not a default that expands into extra turns. -- Initial callback state has `Exchange: Start`, `Result: nil`, and the full - remaining budget. Later states contain the newly completed result and exchange - with the remaining start budget, including zero after the last permitted run. - The initial exchange is not presented as a newly delivered response. -- The coordinator copies each selected request value and sets its `ContinueFrom` - to the current exchange. It owns this one branch's progression, so a callback's - `ContinueFrom` does not redirect that branch. The rest of the request uses - ordinary rules. Callers wanting independent branches use `Run` themselves. -- Compose the request's observer first and the coordinator's observer second. - The latter selects all continuation runs, not the starting run or nested - callback calls. Applications explicitly compose additional group observers. -- On error leave `Outcome` empty. Set `Undelivered` only for a completed new - response withheld from callback invocation because cancellation was observed. - That result includes its exchange. A callback that was invoked has received - its response even if it errors or cancels. Return no result archive, latest - delivered response, failure-stage taxonomy, or coordinator accounting totals. -- Callbacks receive caller-owned public values. Retain the coordinator's current - immutable exchange separately, so replacing fields in delivered values cannot - rewrite retained history or the next continuation's ancestry. - -### Observer Composition And Execution Groups - -```go -func ComposeObservers(observers ...ExecutionObserver) ExecutionObserver - -// Zero value ready for use; contains private synchronization/state only. -type ExecutionGroup struct { /* private collector */ } - -func (g *ExecutionGroup) Observe(event ExecutionEvent) -func (g *ExecutionGroup) Snapshot() ExecutionGroupSummary - -type ExecutionGroupMember struct { - RunID string `json:"run_id"` - TerminalObserved bool `json:"terminal_observed"` -} - -type ExecutionGroupSummary struct { - Members []ExecutionGroupMember `json:"members"` - TerminalCoverageComplete bool `json:"terminal_coverage_complete"` - Attempts uint64 `json:"attempts"` - TransientRetries uint64 `json:"transient_retries"` - RepairCalls uint64 `json:"repair_calls"` - Usage ObservedTokenUsage `json:"usage"` -} -``` - -- Attach a group with `Observer: group.Observe`, or use `ComposeObservers` for - diagnostics and multiple groups. No request group field, implicit context - membership, constructor, registry, close, seal, or wait method is needed. -- Composition copies the supplied function slice, skips nil functions, and - invokes the remainder synchronously in supplied order. With no non-nil - functions return nil. Do not deduplicate functions or recover panics; a panic - prevents delivery to later observers. Closures retain their ordinary identity. -- Groups enroll a run when any ordinary event for that run is observed. The - first terminal event seen for that identity contributes counters and usage - exactly once, including when it is the first observed event. Repeated delivery - must not change totals or regress terminal coverage. Handling contradictory - caller-fabricated events is not a new public validation contract. -- Summary counters and token subtotals use terminal observations only. Pending - runs contribute no invented attempt counts or known zero usage. Members without - terminals remain identifiable. `TerminalCoverageComplete` says only that every - currently observed member has a terminal observation; it says nothing about - future members or application completion. -- For each token field, sum known terminal values; `Known` is the OR of terminal - `Known` flags and `Complete` is the AND of terminal `Complete` flags. These flags - describe received terminal subtotals; missing terminal coverage is separate. - If members exist but none has a terminal, usage is unknown/incomplete. An empty - group has complete terminal coverage and known, complete zero totals. Preserve - explicit provider zero and existing per-run failure-path usage semantics. -- Snapshots are detached and internally consistent under concurrent observation - and reads. There is no promised member ordering; tests compare by identity. - The mutex-bearing group must not be copied after first use, as with ordinary - synchronized Go values. Overlapping group totals are not generally additive. - -## Internal Implementation Decisions - -### Immutable History And Execution-Owned Messages - -Put the internal immutable exchange value and history operations in -`internal/domain/exchange.go`. The public handle privately references this value; -internal packages never import the root facade. Retain only history and the two -run identities, not engine owners, definitions, requests, credentials, validation -plans, results, admission leases, or observer/callback closures. - -Use a simple immutable parent-plus-turn representation: a parent exchange, -detached new-turn messages, the final assistant message, and a cached total -message count. Initial exchanges have no parent. New exchanges retain only their -new suffix rather than another flat copy of all ancestors. Flatten iteratively -into one correctly sized, caller/execution-owned destination when preparing a -provider request or returning details; do not recurse through arbitrary depth. -Reuse message/cache-control clone helpers at mutable boundaries. This is an -internal choice, not a public allocation or asymptotic guarantee. Avoid unrelated -history caches, ropes, eviction, limits, or pooling. - -Prepare the new prompt normally, then assemble history, new definition messages, -and appended messages in the accepted order before hashing. Captured history -is not normalized or rendered again. Save capture/parent state separately in the -private prepared operation/payload. Prepared metadata contains `ParentRunID` and -the already assembled messages, not the exchange capability or capture settings. -Pass the private capture state through the shared execution function. After -successful final validation processing, capture the new-turn suffix from the -executed snapshot and the exact final `RawOutput`; failed structural validation -is still a completed result. Do not capture internal repair instructions or -intermediate candidates. On an operational error returning no result, create no -exchange. Do not change underlying execution cancellation or partial-result -rules to recover an assistant message. - -### Coordinator Placement And Boundary Adapters - -Put the loop in `internal/usecase/coordinate.go`, not in root business logic. -Use one small internal function generic over request/result value types, with -typed callback and execution functions and a `*domain.Exchange` current state. -Its internal turn/termination values contain only the fields projected by the -public API. This lets the facade inject ordinary `Engine.Run` and the application -callback without importing facade types internally or converting an entire run -result to domain and back just for delivery. No generic public API is added. - -The execution adapter accepts `(context, currentExchange, selectedRequest)` and -returns `(completedResult, nextExchange, error)`. It creates a fresh public -exchange wrapper, copies the request, supplies the current history and composed -observer, and calls `e.Run`. Obtain the next internal exchange before delivering -the result to application code. The callback adapter projects a fresh public -wrapper and the current result into `ConversationTurn`. The shared loop does not -inspect results, interpret output, or access generation/capacity directly. - -Use this exact progression: - -1. With no completed response pending initially, check context before callback. -2. Invoke `Next` with the current exchange, pending result if any, and remaining - budget. Invocation marks that pending result delivered. -3. Check context again; cancellation wins over the callback's action/error. - Otherwise return a callback error unchanged, stop on nil request, or continue. -4. If a request was selected with no remaining budget, return exhaustion before - converting/preparing that request. Otherwise check context before execution, - consume one start from the budget, and call the execution adapter. -5. Return an execution error unchanged; retain no partial output. On a completed - result, update the private current exchange and repeat from step 1. - -At step 1, cancellation with a pending new completed response returns that -single response in `Undelivered`; cancellation before the initial callback -returns none. Callback errors and cancellation observed after invocation return -no undelivered response. Panics propagate naturally without recovery. These -context checks do not forcibly interrupt callbacks or eliminate races. - -Calling ordinary `Run` per turn ensures its logical admission and terminal -observation finish before `Next`. Do not wrap the whole coordinator in an -admission lease or an execution recorder. Nested calls remain independent. - -### Group Collector Placement And Cost - -Implement the collector in `internal/observation/group.go`, using domain events -and summary values. The root group is a thin adapter with private internal state; -root summary/member types are independent public declarations, not aliases. -Store a map keyed by run identity, a terminal-seen bit per member, and aggregate -terminal counters/usage under one mutex. Do not retain full event histories, -provider content, or per-attempt arrays. Event updates use map lookup rather than -rescanning all members. Snapshot copies the map's members and totals under the -lock without invoking callbacks. Observer composition is also internal logic -with thin root conversion; do not round-trip every event through both public and -domain representations simply to fan out callbacks. - -## Stage 1 — Implement Immutable Exchange Storage - -**Status:** Complete. - -**Scope:** internal history value and ownership only. - -1. Add the internal exchange representation and construction/flattening helpers - described above. Keep fields private and expose only the internal accessors - needed for run identities, message count, capture, and detached details. -2. Clone only new mutable suffix data when capturing; safely share immutable - parents. Preserve all role, content, empty-content, and cache metadata values. -3. Add focused internal domain tests for initial history, a second turn, two - independent branches, and mutation of source messages/cache control and - returned copies. Include a many-turn flattening case without asserting a - specific private layout or allocation count. - -**Validation:** `go test ./internal/domain` when the package is buildable. -These tests own focused history behavior; the final suite verifies its assembly. - -## Stage 2 — Add Public Handles And History-Aware Preparation - -**Status:** Complete; capture state is carried through execution. - -**Scope:** facade request conversion and the shared preparation path; generation -capture is the next stage. - -1. Add `Exchange`, `ExchangeDetails`, their accessor and opaque formatting, the - request fields, and the prepared/result/observation ancestry fields selected - above. Add private domain request and prepared-operation capture state. -2. Extend `toDomainRunRequest` to translate a valid handle and capture choice. - Reject non-nil empty handles as invalid requests; do not impose engine binding - on exchanges. Preserve existing appended-message validation and copying. -3. Update completion to prepend captured messages before newly rendered messages - and caller suffixes. Compute hashes only after final assembly, leaving the - follow-up's settings, session, profile selection, and schema rules ordinary. -4. Update `PreparedRun` conversion and JSON behavior for optional `ParentRunID`. - Keep capture/private history references out of returned prepared details. - `ValidateRequest` inherits this through its existing preparation delegation. -5. Extend preparation contract coverage to assert exact order, roles, cache - metadata, hash sensitivity to prior history, and unchanged definition hashes. - Check nil/empty handle behavior and that preparation performs no model work. - -**Validation:** focused root preparation/JSON tests and affected use-case tests. -Do not add a separate preparation or validation algorithm for continuation. - -## Stage 3 — Capture Ordinary And Prepared Executions - -**Status:** Complete. - -**Scope:** completed-result creation and prepared payload lifetime. - -1. Carry capture/parent state through `preparedOperation` and - `preparedExecutionPayload` to the shared `executePreparedRun` path. Keep it - out of the retained details object so discard can release execution-only - references. Preserve claim ownership, validation-plan freezing, direct-key - clearing, observer identity, and retry budgets. -2. Create the exchange from the request actually executed and exact final response - after the ordinary execution has a result. Initial capture is opt-in; - continuation always captures. Attach it and `ParentRunID` in result conversion. -3. Capture final failed validation/repair exhaustion as well as valid results. - On generation, validation-operation, or repair-generation errors returning no - result, create no exchange and leave the source exchange reusable. -4. Keep `RunResult.Exchange` excluded from ordinary JSON/YAML and preserve existing - result timing marshal/unmarshal behavior. Details are the explicit content - encoding path. The exchange's private state stores no execution target, - credential, result archive, callback, or lease. -5. Add focused tests for default no-capture, ordinary capture, prepared capture, - discarded/failed-claim silence, final repaired content without repair chatter, - failed validation capture, and operational-error absence of a new exchange. - Reuse existing prepared lifecycle and mutating-client fixtures. - -**Validation:** targeted root capture, prepared-execution, and repair tests plus -`go test ./internal/usecase` as the touched package becomes coherent. - -## Stage 4 — Propagate Ancestry Through Ordinary Observations - -**Status:** Complete. - -**Scope:** immediate-parent metadata without accounting changes. - -1. Add parent metadata to the internal recorder's event construction and the root - event projection. Set it before ordinary start delivery and before a prepared - start after successful claim. Preserve it when selection identities update. - A small internal setter before `Start` is sufficient; do not add context-based - ancestry inheritance or new run IDs. -2. Propagate parent identity through start, attempt, retry-delay, and terminal - events. Initial runs and unrelated nested calls keep an empty parent. Preserve - terminal error classification and public error identities. -3. Cover observed ordinary/prepared continuation, preparation silence, operational - failure after continuation selection, and an independent nested call using - existing observation fixtures. Verify no history/content enters events. - -**Validation:** focused observation and prepared-observation contract tests and -`go test ./internal/observation ./internal/generation`. - -## Stage 5 — Verify Independent Continuation At The Public Boundary - -**Status:** Complete. - -**Scope:** assembled continuation behavior before adding coordination. - -1. Add an external-package contract test that captures an initial exchange and - executes two branches, then continues one branch again. Assert exact outgoing - messages and immediate ancestry without changing the initial or sibling data. -2. Continue the exchange through another engine, including a scoped view. Select - different follow-up prompt/profile/output settings and session rules, and - verify that neither previous settings nor original direct credentials leak - into the new execution. Use synthetic credentials, not real environment data. -3. Remove or alter original artifact files after initial capture; continuation - still uses captured history. New follow-up inputs retain ordinary reads and - errors. Exercise prepared follow-up freezing separately from exchange reuse. -4. Mutate details and received result fields and use a mutating injected client - to prove ownership. Encode details generically and check exact final assistant - content, including empty/whitespace content when the output contract allows it. -5. Verify overlapping continuation runs can share the source exchange under the - race detector. Compare behavior, not a required scheduling order. - -**Validation:** focused root continuation/capture tests under `-race`. This stage -must exercise ordinary shared preparation/execution rather than a mock history -reconstruction implemented by the test itself. - -## Stage 6 — Implement The Internal Sequential Coordinator - -**Status:** Complete. - -**Scope:** the small generic loop and its deterministic state-machine tests. - -1. Add the internal loop and turn/termination values described above, injecting - typed callback and run functions. Do not add a generic workflow framework or - a new generation collaborator. -2. Implement initial callback delivery, turn-start consumption, final-response - delivery, explicit stop versus exhaustion, and first-error termination. -3. Implement context checks and the single undelivered-response slot exactly as - specified. Do not prepare an exhausted request or replay delivered work. -4. Unit-test zero turns, stop on last response versus request for another turn, - execution and callback errors preserving `errors.Is`/`errors.As`, cancellation - before initial callback, cancellation withholding a completed result, and - cancellation/error after callback invocation. Use deterministic hooks, not - sleeps. Add one panic propagation check without terminal-delivery promises. - -**Validation:** focused `go test ./internal/usecase -run Coordinate` (name tests -accordingly). The loop tests use cheap execution functions; provider behavior -remains covered by integration tests. - -## Stage 7 — Expose Coordination Through The Facade - -**Status:** Complete; public coordinator tests passed with observer composition. - -**Scope:** public types, argument checks, and ordinary-execution adapters. - -1. Add the selected coordinator types and `Engine.Coordinate`, with GoDoc for - callback state, budgets, cancellation, errors, and normal outcomes. -2. Validate coordinator arguments without looking up old resources. Project - callback state through fresh public handle wrappers. Preserve current private - exchange state independently of any callback mutation. -3. Supply the run adapter using `Engine.Run` with the receiving engine/view. - Set current history on a value copy; the coordinator does not mutate the - callback's `RunRequest`. Never add enclosing admission or an observer recorder. -4. Wire its observer selection to composition in Stage 9; do not build a second - temporary observer mechanism to make this stage independently complete. -5. Add basic public tests for invalid arguments, zero-budget initial stop, - ordinary continuation, callback error identity, and a callback modifying the - result/handle it received without changing subsequent retained history. - -**Validation:** focused coordinator tests when wiring is buildable. Carry the -explicit observer-composition dependency forward, without an intermediate gate. - -## Stage 8 — Verify Coordinator Capacity And Cancellation Boundaries - -**Status:** Complete; focused race validation passed with observer composition. - -**Scope:** the subtle assembled execution lifecycle. - -1. Configure one backend with concurrency one and waiting capacity zero. Have the - callback make a same-engine/backend validation run after receiving a turn. - Assert it succeeds, proving the logical admission lease as well as the active - permit was released. Use ordinary admission, not test-only capacity bypasses. -2. Deliver a structurally invalid completed response normally, including one - whose structural repairs were exhausted. The application callback decides - whether to stop or select a new request; PromptKit adds no acceptance meaning. -3. Exercise operational failure stopping one coordinator while a sibling exchange - remains usable. Verify typed provider/capacity errors cross unchanged. -4. Trigger cancellation after a completed run but before callback delivery using - a terminal observer; retain its result/exchange once as undelivered. Separately - cancel inside the callback and assert no replay or undelivered duplicate. -5. Check that internal retry/repair attempts create only one delivery for that - continuation and do not consume extra turns. Reuse existing deterministic - HTTP/repair fixtures, not live providers. - -**Validation:** focused root coordination tests under `-race`, with bounded test -contexts as failure guards, not timing-dependent synchronization. - -## Stage 9 — Add Observer Composition - -**Status:** Complete. - -**Scope:** fan-out helper and final coordinator-observer wiring. - -1. Implement internal synchronous fan-out over a copied function slice and the - root `ComposeObservers` wrapper. Keep public/domain adapters proportionate; - a small internal generic callback-composition helper can operate directly on - either event value type without repeated conversion. -2. Honor nil filtering, supplied order, no function deduplication, and ordinary - panic propagation. The helper holds no lock while invoking application code. -3. Complete coordinator observer composition, selecting each request observer - before the coordinator observer. Do not change original callback request - values. Callback-initiated calls remain explicitly selected independently. -4. Test nil/empty composition, caller mutation of the original observer slice, - ordered delivery, early panic, and simultaneous independent executions. Do - not promise stronger callback ordering across different runs. - -**Validation:** focused composition and coordinator observer tests, including -the root observation-boundary tests. - -## Stage 10 — Implement Terminal-Based Group Aggregation - -**Status:** Complete. - -**Scope:** internal collector, usage semantics, and concurrent snapshots. - -1. Add domain summary/member values and the zero-ready internal group collector - under `internal/observation`. Implement map enrollment, terminal deduplication, - incremental terminal counters, and detached snapshots under one mutex. -2. Implement the selected known/completeness and coverage rules for all five - token fields. Do not sum `RunResult.Usage`, reconstruct usage from attempt - events, or convert missing terminals into zero evidence. -3. Cover empty groups, members without terminals, terminal-only enrollment, - duplicate terminal and repeated ordinary events, explicit zeros, wholly - unknown fields, partially known terminal usage, and terminal failures retaining - earlier completed usage. Use small event values rather than an HTTP test for - every arithmetic combination. -4. Concurrently observe independent runs and read snapshots, checking each - snapshot's counts, coverage, and member identities agree. Mutate a returned - snapshot and prove later snapshots are independent. Do not require snapshot - member ordering or introspect locks. - -**Validation:** `go test -race ./internal/observation` with focused new group -tests. No background worker, callback invocation, or collector lifecycle is added. - -## Stage 11 — Expose Groups And Verify Explicit Membership - -**Status:** Complete. - -**Scope:** thin public collector adapter and assembled accounting. - -1. Add the public group/member/summary types and root event/summary conversions. - A private zero-ready internal group field keeps the public zero value usable. - Document synchronization, snapshot ownership, and no-copy-after-use semantics. -2. Exercise `group.Observe` alone and composed with a diagnostic observer and a - second group. Cover ordinary runs without capture, captured runs, prepared - execution, coordinator continuations, and separately selected callback calls. -3. Assert preparation/discard/failed claim creates no membership. The initial - exchange is not automatically enrolled by coordination; unselected nested - calls remain absent. Parent metadata and group membership remain independent. -4. In one assembled scenario, select initial A and branches B/C, plus a validator - into overlapping groups. Include a built-in transient retry, structural - repair, and a failure after completed usage. Check counters and known subtotals - once per identity, including replay usage reported by the provider for B/C. -5. A composed observer panic may leave an observed member without a terminal. - Verify snapshots remain usable, expose missing coverage, and do not manufacture - completion. Snapshotting inside an observer is safe because group locks are - not held while calling other observers. - -**Validation:** focused root group/coordinator/observation tests under `-race`. -Reuse existing reliable-execution fixtures and avoid duplicating their complete -retry/repair matrices. - -## Stage 12 — Demonstrate The Consumer Workflows - -**Status:** Complete. - -**Scope:** practical API use and one maintained offline example. - -1. Extend `examples/go-library/observe` and its semantic assertions to demonstrate - captured proposal history, a coordinator follow-up, an application-owned - assessment call, and explicit group selection alongside diagnostic observation. - Keep its deterministic transport and existing retry/usage protections; no paid - calls or real credentials are needed. -2. The example's assessment remains application policy, not a library validator - hook. Use a closure to retain assessed state, and show stop versus exhausted - budget without returning an application verdict from PromptKit. -3. Add concise consumer-guide examples for direct branching and prepared capture, - plus both roadmap sketches translated to the selected callback shape. The - analysis sketch records a new result then asks another question or returns nil. - The Audita-shaped sketch uses existing findings initially, assesses each new - result, and either returns nil or an ordinary follow-up request. Domain-specific - functions remain illustrative caller code. -4. Show group use with ordinary/prepared requests and `CoordinateRequest.Observer`, - including explicit selection for a nested validator and an independent group - summary. Demonstrate checking structural validation before schema-specific - decoding; stop/exhaustion does not imply content acceptance. - -**Validation:** `go test ./examples/go-library/...` and affected public example -tests. Review snippets against actual declarations; full copyable setup stays in -the maintained example rather than duplicated in prose. - -## Stage 13 — Finalize Contracts, Documentation, And Sprint Validation - -**Status:** Complete. Full offline race suite, vet, and diff checks passed with Go 1.27.3. - -**Scope:** complete the target and remove sprint-only rough edges. - -1. Review every new public declaration and GoDoc against this plan and the feature - roadmap. Confirm ordinary APIs still use existing error, timing, prepared-claim, - credential, namespace, usage, and cancellation behavior. Verify result/details - serialization and constant opaque formatting without introducing a restartable - exchange format or redacting caller-authored message content. -2. Update the canonical consumer guide, `doc.go` where orientation needs it, and - affected internal runner, observation, and component-overview documents. - Document capacity release for conversation callbacks separately from ordinary - per-attempt observer callback constraints. Configuration and integration docs - need only changes necessary to describe the implemented message composition; - ancestry is not an upstream wire field or configuration setting. -3. Remove temporary helpers, placeholder implementations, unused experimental - API choices, and intermediate compile/test failures. Do not remove unrelated - planned/deferred work or publish a release in this sprint. Release-note/version - selection and roadmap retirement can follow during release preparation. -4. With the newest locally available Go toolchain selected as the development - guide prescribes, format changed Go files, then run: - - ```sh - go test -race ./... - go vet ./... - git diff --check - ``` - - The full suite includes the maintained examples; do not add a redundant - separate build or provider connectivity gate. Check added/changed local - documentation links, using the existing tooling where practical rather than - introducing another release validator. -5. Review the complete diff and Git status for unintended files or credentials. - Confirm no new dependency, workspace override, configuration parser, provider - parameter, or strengthened restriction/guarantee slipped into the work. - Commit completed units with plain-English messages under the agent policy. - -**Completion evidence:** the implemented facade supports ordinary/prepared -capture, cross-engine independent continuations, coordinator callbacks after -admission release, cancellation's single undelivered response, and independent -observer-based groups. Public behavior tests cover those boundaries and terminal -usage completeness; the complete offline race suite and vet pass. No product or -API decisions are deferred to an implementation stage. diff --git a/docs/roadmap/remediation.md b/docs/roadmap/remediation.md deleted file mode 100644 index 4b04c60..0000000 --- a/docs/roadmap/remediation.md +++ /dev/null @@ -1,177 +0,0 @@ -# Conversation Continuation Review Remediation - -## Review Scope - -Reviewed all 13 stages of the [implementation plan](implementation.md) against -the [accepted target](conversation-continuation.md), implementation, tests, and -documentation. Baseline: `5c135c5`, including the implementation commits after -`5cc3fa0`. - -The implementation retains the intended boundaries: immutable parent-linked -history, ordinary preparation and execution for continuations, a separate -sequential coordinator, and explicitly selected observer-based groups. No runtime -correctness defect or inadvertently strengthened restriction was identified. -The initial finding concerns the durable public documentation in Stage 13. - -Validation completed with the locally installed Go 1.27.3 toolchain, -`GOTOOLCHAIN=local`, `GOWORK=off`, and a writable temporary build cache: - -- `go test -race ./...` passed, including the maintained examples. -- `go vet ./...` passed. -- `git diff --check` passed. - -### Algorithmic Follow-Up - -A subsequent static audit of all 13 stages, at the same implementation baseline, -identified the two efficiency opportunities below. No additional ordinary -correctness bug or avoidable quadratic-time algorithm was found. No real-world -performance data was collected. - -The planned algorithms remain fixed: suffix-only immutable capture, iterative -history flattening, sequential coordination, synchronous observer delivery, and -map-based incremental group aggregation. The findings address extra allocation -and copying around those choices, not replacements for them. Complexity statements -describe implementation costs, not new public performance guarantees. - -For orientation, capture is linear in the new turn's message count; flattening is -linear in retained message count; coordinator bookkeeping is constant per turn, -excluding application callbacks and ordinary execution. Group updates are -expected constant time per event for fixed-length run identities, with linear -membership storage and linear full snapshots. An observer dispatch is linear in -its selected callbacks. These implementations do not rescan all prior turns or -group members on each incremental update. - -## 1. Complete The Public Exchange Capture And Continuation Contract - -**Priority:** P2. **Status:** Complete. **Plan stage:** 13. - -**Completion:** Expanded the public request, exchange, details, and prepared-handle -GoDoc and linked the consumer walkthrough to those owners. Checked comments -against capture, preparation, and repair behavior; local links and diff verified. - -**Locations:** [request fields](../../types.go), -[exchange declarations and accessor](../../exchange.go), and -[consumer guidance](../consumers/pkg-promptkit.md#capture-and-continue-a-conversation). - -The `CaptureExchange` and `ContinueFrom` GoDoc currently provides only a short -description of opt-in capture, invalid handles, and fresh follow-up settings. -The `Exchange` and `Details` comments describe reuse and detached history, but do -not define its exact contents or explain that inspection uses only retained -state. Important selected semantics remain documented only in temporary roadmaps -or prose/internal guidance: - -- History contains the effective original request and exact final assistant - content, preserving roles and cache metadata; intermediate structural-repair - candidates and corrective instructions are excluded. -- Continuation assembles captured history, rendered follow-up definition - messages, and caller-appended messages in that order, without re-rendering or - reading the original inputs. It adds no history-size limit or truncation. -- `PrepareExecution` retains capture selection for execution of its snapshot; - preparation and concrete-request validation alone produce no completed exchange. -- `Details` reads the retained snapshot without rendering, filesystem reads, or - provider calls. Its run identities identify the originating execution and - immediate parent execution. - -These behaviors are implemented. The gap is that the exported declarations and -GoDoc do not yet own their complete contract as required by the documentation -policy and the final implementation stage. Retiring the roadmap would otherwise -leave consumers and future maintainers without a complete canonical reference. - -**Required remediation:** Document these existing semantics at the appropriate -request fields, exchange type/details, and prepared-execution declarations, using -GoDoc cross-references instead of repeating the same contract across declarations. -Link the new consumer walkthrough to those public owners. Preserve the selected -scope: do not add persistence, limits, stronger serialization guarantees, or -private storage/allocation promises while documenting the behavior. - -**Verification:** Check the comments against the existing capture, continuation, -prepared-execution, ownership, and repair tests and implementation. Validate the -affected documentation links and diff. This is documentation remediation; it -does not require changing runtime behavior or duplicating existing tests. - -## 2. Avoid Intermediate History Copies During Assembly And Inspection - -**Priority:** P3. **Status:** Complete. **Plan stages:** 2 and 5. - -**Completion:** Preparation now fills one history-plus-suffix buffer. Inspection -projects indexed history directly into public messages, cloning exposed cache -metadata once. The parent-linked representation and public types are unchanged. -Focused race-enabled history, continuation, preparation, and ownership tests passed. - -**Locations:** `completePreparationWithStructuredOutput` in -[the runner](../../internal/usecase/runner.go), `Exchange.Messages` in -[history storage](../../internal/domain/exchange.go), and -[`Exchange.Details`](../../exchange.go) with `fromDomainRenderedMessages` in -[facade conversion](../../convert.go). - -Let H be the captured history's message count and S the newly rendered and -caller-appended message count. Continuation preparation currently: - -1. allocates and copies S messages into the new suffix; -2. materializes H history messages into a detached intermediate slice; and -3. allocates the final H + S slice and copies both intermediate slices again. - -Thus assembly copies 2H + 2S message entries, rather than the H + S needed to -populate the final execution-owned buffer. Each pass also clones non-nil -cache-control structs. The public `Details` path similarly materializes a -detached domain history and then traverses it again to allocate public messages -and clone cache controls for a second time. Strings share immutable content; -these are message-header and cache-struct copies, not duplicate text-byte buffers. - -**Complexity:** Current and proposed assembly are both Θ(H + S) in time and -auxiliary space; details remain Θ(H). The opportunity removes an extra linear -pass and temporary allocations, not an asymptotic quadratic defect. Successive -full-history requests still entail work proportional to the sum of their history -lengths. This audit does not propose truncation or a different -conversation protocol. - -**Potential refactoring:** Add an internal fill/flatten helper that writes history -and the new message parts directly into one correctly sized final domain slice. -For inspection, consider an internal indexed traversal/projection helper that -fills the public destination without first constructing a complete domain slice. -Keep the immutable parent-linked representation, separate public types, and -detachment of every exposed mutable cache-control value. Prioritize the simple -assembly change; avoid adding a general projection framework solely to eliminate -the inspection intermediate. - -**Verification:** Reuse history ordering, cache-control mutation, branching, -prepared-execution, and mutating-client tests. Review the copy/allocation paths -statically. Do not add public allocation guarantees or exact-allocation tests. - -## 3. Size Observer Storage For Selected Callbacks - -**Priority:** P3. **Status:** Complete. **Plan stage:** 9. - -**Completion:** Observer composition counts selected callbacks before allocating, -sizes retained storage to that count, and handles zero/single selections without -a fan-out closure. Sparse-selection ownership and duplicate-delivery tests pass. -The complete offline race suite, maintained examples, vet, and diff checks passed -with Go 1.27.3 after all remediations. - -**Location:** [`Compose`](../../internal/observation/compose.go). - -`Compose` allocates `selected` with capacity equal to every supplied observer -slot before filtering nil callbacks. The returned closure retains that backing -array even when only a small subset of slots survives. An all-nil input also -allocates the full temporary array before returning nil. - -Let K be the number of supplied slots and A the number of non-nil callbacks, -including intentionally repeated callbacks. Construction is Θ(K) time with -Θ(K) allocated storage, and a nonempty composition retains Θ(K) storage. -Dispatch is already Θ(A) and does not need changing. Selected-function storage -can instead be Θ(A), with constant storage for A = 0, while preserving Θ(K) -construction time. - -**Potential refactoring:** Count non-nil callbacks first, then allocate exactly A -slots and fill them in order. This trades a second linear scan for avoiding -storage proportional to disabled slots. Alternatively, append into an initially -nil slice for a simpler one-pass filter with amortized Θ(K) construction and -Θ(A) retained storage, accepting growth allocations for dense inputs. A -zero/single-callback fast path can also avoid an unnecessary fan-out closure. -The usual coordinator supplies only two slots, so its benefit is small; this is -a low-priority general-helper scalability opportunity, not a release blocker. - -**Verification:** Preserve copied function values, nil skipping, duplicate -delivery, supplied order, and ordinary panic propagation. Existing composition -and prepared-observer ownership tests protect those contracts; no benchmark or -new performance gate is required.