@@ -48,7 +48,7 @@ implementing changes. It owns implementation discipline and local commit rules.
| Output, cache, resume, or debug artifacts | [Run State Internals](internal/state.md), [Operations](operations.md), and [Configuration](config.md) | These separate implementation details, operator behavior, and configuration contracts. |
| External input formats, artifact schemas, or durable output files | [Integration Contracts](integrations/) | Integration documents define external and durable data contracts. |
| Repository-specific release commands or diagnostics | [Source Releases](release.md) | The runbook supplies publication commands and recovery procedures. |
| Proposed or unimplemented behavior | [Roadmap](roadmap/) | Future work belongs only in roadmap documentation until implemented. |
| Proposed or unimplemented behavior | [Native migration roadmap](roadmap/promptkit.md) and [separate backlog](roadmap/future.md) | The master roadmap owns the native cutover; the backlog holds other future work. Implemented behavior belongs in its canonical documentation. |
For an existing subsystem, also inspect its focused tests and the package-local
The decisions in items 1–6, the subsequent CommandKit discussion, and the
simplified cache/output designs are incorporated into the
[master native migration roadmap](promptkit.md). It owns the selected target,
unresolved choices, sequencing, acceptance criteria, and implementation status.
The [CommandKit discussion record](commandkit-migration-discussion.md) retains
the related CLI, preflight, output, debug, and migration rationale. The master
roadmap replaces earlier checkpoint matching/provenance proposals with the
accepted-artifact cache and retires mandatory manifests, extensive receipts,
and durable execution ledgers. The upstream message-content hash remains pending;
incorporation does not claim it is already available or implemented.
This document retains the discussion, rationale, and evidence, including item 7's
completed Audita comparison. Item-level questions and follow-ups below record
the discussion as it developed; they are not a second implementation plan.
The goal is to make Notarius fully PromptKit-native and adopt modern PromptKit
capabilities as fully as practical. Application ownership of a policy does not
necessarily require application ownership of its implementation. Evaluate
remaining machinery against this goal, including opportunities for upstream
features where PromptKit does not yet meet an application need.
No application changes have been implemented through this discussion or the
roadmap rewrite. Significant changes to accepted architecture need successor ADRs
under the [documentation policy](../policy/documentation.md#architecture-decision-records).
Current boundaries are described in [Architecture](../policy/architecture.md)
and [LLM Runtime Internals](../internal/llm.md).
## 1. LLM API And Diagnostic Boundary
**Discussion / decision:** Direction agreed: remove the Notarius PromptKit
adapter and compose the application directly from native PromptKit execution,
typed artifact operations, and pipeline orchestration. A small application
function connects generation to configured validation and acceptance. The design
below incorporates items 2–6 and the Audita review; exact signatures and migration
details remain open. No application changes have been implemented.
### Background And Intent
Notarius and related applications, including Audita, originally developed their
own LLM interfaces or used the OpenAI SDK directly. PromptKit emerged by
extracting their duplicated mechanics into a shared library. Notarius adopted
an early version and predates many of the library's modern capabilities.
The maintainer clarified that duplication of PromptKit capabilities, including
slight behavioral differences, does not reflect an intentional requirement for
an independent Notarius implementation. Existing wrappers, tests, and historical
decisions are migration evidence, not sufficient reasons to preserve that
duplication. Identify actual application needs before choosing what to retain.
Missing reusable capabilities may justify upstream feature requests. The
discussion is not constrained to what v0.23.2 can replace immediately; any
resulting change to the adoption target remains to be selected.
### Application Structure
| Part | Responsibility |
| --- | --- |
| Application composition | Load configuration, assemble native prompt sources, and construct one shared PromptKit engine for producers, validators, and continuation runs. |
| Artifact-family modules | Supply prompt inputs and response requirements, interpret responses, and apply domain rules. |
| Pipeline execution | Connect generation to configured validation, decide acceptance or fallback, and manage dependencies and bounded workers. |
| PromptKit | Prepare and execute requests, maintain conversations, enforce native budgets and admission, and report execution facts. |
| Application records and CLI | Store artifacts and accepted-artifact cache entries, attach pipeline meaning to execution facts, and present results through CommandKit. |
Engine construction becomes ordinary application wiring using item 2's embedded
defaults and operator sources. Profile selection is expressed through native
requests. U3 now places shared loaders and engine construction in `internal/cli`
and the producer helper beside pipeline assessment. Retire the emptied
`internal/framework/llm` package rather than retaining a constructor-only package.
The [master roadmap](promptkit.md#module-execution-and-assessment-u3-resolved) owns
the resolved design; implementation is pending.
Use native requests, results, exchanges, errors, and observers directly in
model-executing application code. Pure domain models, codecs, source-reference
resolution, and deterministic transformations need no PromptKit dependency.
Retain application interfaces for extracting artifacts, validating typed
candidates, encoding output, and loading accepted cached artifacts. Deterministic
stages continue to operate without conversation machinery.
### Connect Modules To Pipeline Assessment
The current [spell extractor](../../internal/modules/dnd/extract/spells/extractor.go)
prepares inputs, invokes the private completion interface, interprets a response,
and cache eligibility. Detailed response contracts, budget migration, and
the open policy questions below remain to be resolved. No application changes
have been implemented.
### Agreed Flow
For cacheable operations, item 5's initial preparation and lookup precede this
flow. The steps below describe execution on a miss.
1. Prepare the source context, references, and any request-local candidate-handle
mapping once for the producer operation. Run its initial native request with
exchange capture when continuation is enabled. Capture is independent of
detailed debug recording.
2. Handle the native call's terminal success or failure, decode a usable response,
and perform the module's deterministic interpretation and safety checks. Run
the configured validation chain on a materialized candidate. A module can
identify actionable defects before a candidate is ready for that chain.
3. If correction is needed, call `Engine.Coordinate` with the initial exchange
and a configured `AdditionalTurns` budget. The callback selects a native
repair prompt and supplies actionable findings as variables. PromptKit retains
the original exchange and each subsequent repair turn.
4. Assess every delivered response, including the last permitted turn. Rebuild
the typed candidate from the fixed operation inputs and the revised response;
do not repeatedly apply revisions to an already modified artifact. A changed
candidate receives the complete configured validation chain again, including
validators that approved an earlier version.
5. Stop on acceptance or a terminal condition. After exhaustion, Notarius decides
whether to reject, fail, or evaluate an explicitly supported safe fallback.
Native `stopped` and `exhausted` outcomes describe coordination, not acceptance.
Cancellation never turns an undelivered response into an accepted candidate.
Start with one conversation per producer operation: a chunk-plan generation,
one lane's extraction of one chunk, or one lane's normalization pass. This aligns
with Notarius's whole-output validation boundary. Audita's per-proposal branches
are useful evidence, but per-entity repair would require its own combination and
validation design. It is not necessary to adopt native continuation.
Repair prompts should supply a new user task, with enduring principles in the
original system instructions. Avoid replaying the initial task or embedding
initial-response-only constraints in system messages. Begin with complete
replacement model responses; for normalization, the response can remain a
reconciliation plan that Notarius applies to construct the durable artifact.
A different repair schema or profile is supported but not yet selected. Keep
source context and candidate-handle meanings stable throughout the conversation.
Accumulated history increases context and token use across turns; the semantic
turn budget should reflect that tradeoff.
### Machinery To Remove Or Adapt
| Current machinery | Agreed direction |
| --- | --- |
| `CorrectionProtocolSingleResponseV1`, raw-response `ModelCandidate`, and `SemanticCorrection` | Retire the latest-response-only transport protocol and its copy/validation plumbing. Keep the typed domain candidate and use native `RunResult` / `Exchange` for continuation state. |
| Adapter code that rebuilds the original request and appends an assistant/user pair | Remove with the adapter. Native continuation assembles history; native repair definitions render findings. |
| `runProducerAttempts` combining operational, structural, module-requested, and validator-semantic retries | Replace its LLM retry loop with native coordination plus application assessment and terminal disposition. Retain useful stage provenance, not a second turn counter or general retry engine. |
| Outer LLM retries after transport or structural repair exhaustion | Remove by default and settle the failure after PromptKit's budgets are exhausted. A semantic turn addresses actionable domain defects; it should not silently restart a failed provider execution. Any additional recovery policy needs an explicit application purpose. |
| `executeValidationChain`'s repeated LLM validator invocations | Keep the ordered chain and result aggregation; move transport and response-shape recovery into each validator's native execution. A validator execution failure remains distinct from a rejected producer candidate. |
| `NormalizeRetry` and `producerRetryDirective` | Replace requests to rerun the whole module with assessment findings and explicit fallback information. Preserve the domain meaning of a safe partial result. |
| `CorrectionRequest`'s hard-coded retry-message construction | Keep useful ordered collection/deduplication of actionable findings, and render the repair assignment through a native prompt. Operator messages and reason codes remain diagnostic data, not automatic model instructions. |
| Correction capability declarations and preparation checks | Retire checks tied to `single_response_v1`; prepare the actual native initial/repair definitions and response requirements. Deterministic operations need no synthetic exchange. |
Native transient retries, structural repair, and semantic additional turns remain
distinct controls, but Notarius need not implement all three loops. Existing
producer and validator `retries` settings cannot be mechanically renamed: their
current budgets cover different behavior. Non-LLM failure handling should be
reviewed separately rather than routed through conversation coordination.
### Validation And Safe Fallback Still Have Application Meaning
content, and optional debug capture. The CommandKit output decision below removes
mandatory durable execution provenance. No application changes have been
implemented.
### Ownership
| Responsibility | Owner |
| --- | --- |
| Configuration provenance, resource inspection, and request preparation | ConfigKit and PromptKit through their native APIs. |
| Standard configuration/prompt commands, preview and smoke commands, report formatting and visibility | CommandKit and its ckconfig/ckprompt integrations. |
| Provider attempts, transport retries, structural repairs, execution failures, and token observations | PromptKit observations and results; `ExecutionGroup` for selected execution totals. |
| Step/lane/chunk/validator attribution and pipeline progress meaning | Notarius. |
Item 2's native underlay direction has a working application precedent. Audita
also offers evidence for ordinary embedded filesystems and per-request schema
selection. U2 now selects ordinary embedded assets and prompt-declared contracts
by default, reserving per-request replacement for demonstrated runtime needs.
The subsequent item 3 discussion selects native continuation and accumulated
conversation history. Keeping domain validation local does not require keeping
conversation assembly or the turn loop local. Item 3 owns the agreed Notarius
flow and the remaining repair-prompt, response-contract, and policy details;
the comparison alone did not settle those details.
Item 4 now records agreement to use native admission with bounded application
workers. The revised item 5 selects an accepted-artifact cache and a focused
upstream message-content hash instead of comprehensive execution fingerprints.
Item 6 adopts native execution observations and CommandKit presentation;
the [CommandKit output decision](commandkit-migration-discussion.md#4-reports-machine-output-and-outcomes)
selects artifact files and a small native completion report.
The two requested capabilities already work with v0.23.2; no new PromptKit
feature is needed merely to provide replaceable defaults or domain-directed
continuation. Any remaining upstream request should identify the specific unmet
requirement after reconsidering Notarius's historical behavior.
### Verification
Read the production composition, adapter, coordinator, domain repair callbacks,
prompt/schema definitions, and focused behavior tests. The following checks
passed from the Audita checkout:
```sh
go test ./internal/framework/llm ./internal/prompts ./internal/framework/responseschema ./internal/framework/runner ./internal/framework/proposal_generation
go test ./cmd/audita -run '^TestSubprocess(CorrectionRepairConversation|CancellationDuringCorrectionRepair|RepairDisabledAndOutputSelections)$' -count=1
```
The subprocess checks exercise assembled configuration, real native continuation
@@ -751,7 +768,7 @@ reopen the architecture. IDs retain continuity with the earlier roadmap.
| ID | Remaining choice or dependency | Resolve for |
| --- | --- | --- |
| U1 | Supplying PromptKit API/release for the requested message-content hash; verify the final compatible dependency set against the selected baseline. The upstream request is pending. | Track in Phase 1; required to finish cache integration in Phase 5. |
| U1 | Review the message-content hash API/release documented upstream in v0.24.0, including its ConfigKit dependency; verify the final compatible dependency set against the selected baseline. | Track in Phase 1; required to finish cache integration in Phase 5. |
| U4 | Role-specific initial/repair definitions, response contracts, and profile selection within complete replacement and fixed operation inputs. | Phases 3–4. |
| U5 | Producer/validator retry-setting migration to native controls and semantic budgets: field names/defaults, non-LLM failure handling, and any justified recovery beyond native exhaustion. | Phases 3–4 and configuration migration. |
| U6 | Detailed terminal/fallback policy, especially whether an earlier safe candidate survives a later failed turn. No implicit best-so-far acceptance. | Phases 3–4. |
@@ -787,7 +804,7 @@ major boundary changes in successor ADRs without rewriting accepted decisions.
### Phase 1: Establish The Migration Baseline
- Record the selected dependency set and track U1 against the upstream request.
- Record the selected dependency set and review U1 against the upstream release.
Keep the cache dependency explicit without blocking independent work.
- Map affected ADRs, current callers, contracts, and removal inventory. Carry
forward Audita evidence without adopting its application wrappers.
@@ -795,8 +812,9 @@ major boundary changes in successor ADRs without rewriting accepted decisions.
execution, assets/schemas, CommandKit, debug, cache, or output ownership.
preparation/execution, continuation, observations, and admission.
### Audita Comparison
The comparison was completed on 2026-11-18 against Audita revision
`da5b8913d1206a8e8f0a784b2322760385548a60`, using PromptKit v0.23.2,
ConfigKit v1.2.1, CommandKit v0.7.0, ckconfig v0.9.0, and ckprompt v0.10.0.
The observations below concern that revision; linked sibling files may evolve.
| Evidence | Lesson carried into this roadmap |
| --- | --- |
| [Engine assembly](../../../audita/internal/framework/llm/runtime.go), [source tests](../../../audita/internal/framework/llm/engine_sources_test.go), and [assembly failure tests](../../../audita/internal/framework/llm/runtime_assembly_test.go) | One captured engine, native underlays and main-source-relative companions, whole-record replacement, preserved file contexts, strict peer conflicts, and no silent fallback on operator-source errors. |
| [Adapter tests](../../../audita/internal/framework/llm/adapter_test.go) | Ordinary embedded schemas and complete per-request output contracts work with operator prompt replacement. U2 chooses prompt-declared defaults for Notarius; Audita's schema registry and per-call overrides are not requirements to copy. |
| [Native coordination bridge](../../../audita/internal/framework/llm/coordinate.go), [repair flow](../../../audita/internal/framework/runner/repair.go), [conversation tests](../../../audita/internal/framework/llm/conversation_test.go), and [repair tests](../../../audita/internal/framework/runner/repair_test.go) | Captured exchanges, accumulated history, complete candidate revalidation, final-turn assessment, and canceled/undelivered response handling. |
| [Prompt guidance](../../../audita/docs/internal/prompts.md) | Durable system instructions, an initial user task, and subsequent repair assignments suit native continuation. |
| [Observations](../../../audita/internal/framework/llm/observations.go) and [CLI composition](../../../audita/internal/cli/app_wiring.go) | Native execution totals with application scope and explicit nested-validator enrollment; standard CommandKit suites using shared loaders. |
Audita repairs individual transcript corrections in independent branches;
Notarius's selected unit is a whole producer operation and typed artifact.
Audita's wrappers, response-schema registry, per-proposal policies, and absence
of a cache do not define Notarius requirements.
At the reviewed revision, focused LLM, prompt/schema, runner, and proposal-generation
tests passed, together with subprocess checks for correction conversations,
cancellation during repair, and repair-disabled/output selections using a local
fake provider. These historical checks do not establish live-model quality or
completion of Notarius's migration acceptance checks.
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.