Files
notarius/docs/roadmap/promptkit.md

977 lines
65 KiB
Markdown
Raw Normal View History

# Native PromptKit And CommandKit Migration
## Purpose And Status
Make Notarius fully native to PromptKit, ConfigKit, and CommandKit. Use upstream
configuration, execution, continuation, admission, inspection, commands, and
presentation directly. Retain the application work of producing, assessing,
publishing, and reusing typed artifacts. Historical duplication of library
behavior is not a requirement to preserve it.
This roadmap incorporates the [PromptKit architecture discussion](promptkit-architecture-discussion.md)
and all six items of the [CommandKit discussion](commandkit-migration-discussion.md),
including the subsequent accepted-artifact cache and artifact-first output
decisions. The notepads retain rationale and Audita evidence; this document owns
the selected target, unresolved choices, implementation sequence, removal
inventory, and acceptance criteria. All implementation phases are pending.
Recording these decisions has not changed application behavior.
Complete one coordinated cutover for configuration, PromptKit resources, and the
CLI. Adopt native public conventions fully; preserving old CLI behavior is a
non-goal. Supply migration guidance and maintained examples instead of aliases,
parallel parsers, or compatibility wrappers for legacy behavior. CommandKit is
part of the main migration, not a later follow-up. Release preparation and
publication remain separate work.
## Baseline And Dependency Target
At Notarius revision `12eeb6369e40746b2c18c55397a289a1e1b5908d`, the application
uses configuration version 5 and Go 1.27. Its dependencies include old-path
PromptKit v0.12.0 and ConfigKit v0.5.0, new-path ConfigKit v1.2.0, go-yaml
v4.0.0-rc.6, and two external PromptKit provider catalogs.
The selected reviewed baseline is:
| Module | Version |
| --- | --- |
| `go.maximumdirect.net/commandkit` | `v0.7.0` |
| `go.maximumdirect.net/ckconfig` | `v0.9.0` |
| `go.maximumdirect.net/ckprompt` | `v0.10.0` |
| `go.maximumdirect.net/configkit` | `v1.2.1` |
| `go.maximumdirect.net/promptkit` | `v0.23.3`, pending the message-content hash addition below |
Keep Go's consumer minimum at 1.27, adopt stable go-yaml v4, and resolve compatible
validation dependencies. PromptKit v0.23.3 supersedes the initial v0.23.2 target
for its timeout/cancellation fixes. These are selected versions, not a claim
that they are the latest releases. The local tag review and integration guidance
are recorded in [CommandKit item 1](commandkit-migration-discussion.md#1-adoption-scope-and-dependency-baseline).
The tagged [release notes][release], [migration guide][migration],
[format reference][formats], and [Go consumer guide][consumer] supply the baseline.
The cache requires the proposed native `MessageContentHash`. The
[upstream request](../../../promptkit/docs/roadmap/notarius-request.md) is pending;
v0.23.3's `RenderedPromptHash` is not a substitute. Confirm the supplying API and
tagged release, then verify the assembled dependency set. Independent composition
and pipeline work can proceed while that addition is pending; the completed
cache must use the upstream capability. Do not add a local hash-policy fork.
Neither a cross-backend global limit nor a comprehensive execution fingerprint
is required. Remove obsolete imports and catalog dependencies when unused;
avoid permanent `replace` directives and update [attribution](../oss/attribution.md)
under the [OSS policy](../policy/oss.md).
Audita's reviewed v0.23.2 integration demonstrates native source assembly,
continuation, observations, and CommandKit wiring. Adopt those mechanisms
without copying its `RoleClient`, structured-output interface, or opaque exchange
wrapper. Its per-proposal repair branches and lack of checkpoints do not define
Notarius's artifact and recovery policies.
## Target Architecture
### Application Structure And Ownership
| Part | Responsibility |
| --- | --- |
| Application composition | Load configuration and native sources; construct one shared PromptKit engine for producers, validators, and continuation runs. |
| Artifact-family modules | Prepare domain inputs, select prompt IDs and response requirements, interpret responses, and apply deterministic domain rules. |
| Pipeline execution | Connect generation to configured validation and terminal policy; manage ordered dependencies, typed handoffs, bounded workers, and stable result ordering. |
| PromptKit | Prepare and execute requests; maintain captured exchanges and conversations; enforce native retry, repair, turn, and admission budgets; report execution facts. |
| Application state and records | Publish artifacts, cache final accepted artifacts, provide a small completion summary, and capture optional debug evidence with pipeline context. |
| ConfigKit and CommandKit | ConfigKit owns configuration mechanics; CommandKit and ckconfig/ckprompt own standardized commands and presentation. Notarius supplies loaders, request builders, report content, and outcome mapping. |
Use native PromptKit requests, results, exchanges, errors, and observers in
model-executing application code. Remove the private provider-neutral completion
API. Pure domain models, codecs, source-reference resolution, and deterministic
transformations need no PromptKit dependency. Application interfaces continue to
express artifact production, typed validation, output encoding, and state access.
#### Module Execution And Assessment (U3 Resolved)
Keep the existing typed stage boundaries and use native PromptKit values in
model-executing code. Place engine construction and shared loaders in focused
files in the `internal/cli` composition root. It constructs one native engine
for the run. Retire `internal/framework/llm` once its adapter, scheduler, asset
registry, and reporting responsibilities disappear; do not retain a package
solely to hold a small constructor or introduce another LLM service interface.
Place one small shared producer helper alongside assessment in
`internal/framework/pipeline`. Connect modules to it through ordinary functions:
1. The pipeline supplies an operation-scoped assessment callback capturing the
configured validator chain and acceptance/terminal policy.
2. The module prepares source material, references, and any normalization handle
mappings once. It builds the native initial request and supplies response
interpretation and correction-request functions.
3. The helper receives those functions, the shared engine, and relevant
cache/debug collaborators. For cacheable work it prepares the initial request
and looks up the accepted-artifact cache; on a miss it executes the retained
request. Otherwise it uses ordinary native execution.
4. For each executed response, the helper invokes module interpretation and
pipeline assessment. When correction is appropriate, it supplies the module's
next request to native coordination. PromptKit owns conversation state and
turn iteration; the helper does not implement another retry engine.
5. Return a typed application outcome. Cache only final accepted artifacts under
the initial request's key. A cache hit follows the existing direct-reuse policy
without model execution or renewed semantic assessment.
Request builders return `promptkit.RunRequest`; interpreters consume native
results. Candidates, findings, and acceptance outcomes remain application-owned.
Keep operation-specific state in local variables or closures, without an
execution-plan language, replacement completion API, or copied conversation model.
Use one pipeline assessment implementation for the initial candidate and every
correction, including the final permitted response. Do not repeat final validation
merely because the helper returns to the outer runner. Deterministic stages keep
their ordinary typed production/validation path without synthetic requests or
conversation machinery. LLM validators call the shared engine directly and return
domain findings; they do not automatically acquire a producer correction loop.
Modules receive operation data and collaborators, not physical storage roots or
CommandKit reporters. Pure domain packages retain artifact models, codecs,
reference resolution, and deterministic rules. `internal/preflight` retains its
separate known-dependency checks. Exact argument grouping and helper signatures
can be refined against representative extraction and normalization during
implementation; ownership and placement are settled. Role-specific prompts,
budgets, and fallback policy remain U4–U6. Implementation is pending.
Retain the fixed `input -> chunk -> extract -> merge -> normalize -> output`
shape, ordered steps and generated references, typed artifact-family boundaries,
and deterministic public ordering. Keep modules independent of physical output,
cache, and debug roots. Native adoption does not turn the pipeline into a general
workflow engine or redesign durable D&D artifacts.
### Command Surface And Native Inputs
Install the application `pipeline` family alongside ckconfig and ckprompt:
| Command | Responsibility |
| --- | --- |
| `pipeline list` | Discover configured pipelines. |
| `pipeline show <pipeline-id>` | Report resolved steps, lanes, modules, validators, and reference relationships. |
| `pipeline validate <pipeline-id>` | Check composition and known prompt/profile dependencies offline. |
| `pipeline run <pipeline-id> --input-file PATH` | Execute the pipeline and publish artifacts. |
| `run <pipeline-id> --input-file PATH` | Native alias for `pipeline run`, with the same handler and inputs. |
Use `commandkit.Alias("run", "pipeline run")`, not a forwarding command. Use
`notarius run` in quick starts and document both routes. Install the full selected
standard suite: native help/version; Bash, Zsh, and Fish completion; configuration
scopes, validation, show, sources, explain, and diff; PromptKit resource listings
and inspection for prompts, profiles, backends, namespaces, and collections;
requirements discovery/validation and prepared-request preview; and explicit
reachability, authentication, text, structured-output, and aggregate smoke tests.
Include upstream convenience aliases and useful completion hints.
Use native typed declarations and supplied-value semantics. Defaults apply to
omission, not explicit empty/false/zero values. Preserve native command-path,
operand, repeated-flag, and `--` parsing conventions. Check every occurrence of
a singleton flag and use the last successfully parsed value.
| Concern | Selected input |
| --- | --- |
| Configuration | Native `--config`, `--overlay`, and supported application `--set POINTER=JSON`; scope named `app`. |
| Pipeline input | Required pipeline-ID operand and `--input-file PATH`. |
| Lanes | Repeatable `--lane ID`; omission selects the normal lane set. Commas are literal values, not separators. |
| Output | Application `--output-dir PATH`. |
| Chunk-plan cache | `--chunk-cache auto\|bypass\|refresh`. |
| Accepted-artifact cache | Reuse/bypass/refresh controls; spelling, scope, and defaults remain U8. |
| References | Repeatable `--reference SELECTOR=PATH` and `--without-reference SELECTOR`. |
| Presentation | Native `--format`, `--quiet`, `--verbose`, `--debug`, and `--progress`. |
| Capture destination | Application `--debug-dir PATH`, with the semantics below. |
Keep worker counts and semantic budgets in configuration, adjustable through
application `--set`; their fields/defaults remain open. Use ckprompt's reusable
profile, session, and execution flags, including model, temperature, timeout,
reasoning effort, and transient retries, for run-wide overrides across producers,
validators, and continuations. Omission preserves normal selection; explicitly
empty reasoning effort clears inheritance. Use native `--llm-profile` and
`--llm-session-id`, retiring the separate reasoning-clearing control. Native
transient retries remain distinct from semantic additional-turn budgets.
Individual prompt selection and named prompt inputs belong on standard PromptKit
commands; pipeline modules build their own requests.
Notarius retains reference-selector expansion, specificity, required slots, and
external/generated-reference conflict checks. Pipeline reference selectors are
not individual-request `--llm-input-file` bindings. Remove custom dispatch,
assembled help, standard flag parsing, comma-separated `--only`, and bespoke
`--json` switches. Replace `config effective` with `config show` and `config
profile` with `promptkit show profile`. Old flags are migration evidence only.
### Application Composition, Loaders, And Preflight
One constructor registers application commands, the native alias, ckconfig's
`app` and `promptkit` scopes, ckprompt, and native help/version/completion/reporting.
It does not load configuration, build an engine, or initialize run storage.
Help, version, and completion work without application configuration.
Share two loading responsibilities across application and standard commands:
1. Select application configuration through explicit `--config`, then existing
environment/discovery policy. Let ConfigKit apply overlays, imports,
inheritance, overrides, defaults, normalization, and structural validation.
Retain its native result and selected main-source context. Validate an
explicitly empty configuration path rather than falling through to discovery.
2. Build the native engine from that application's source context using the source
assembly below. Across commands, `--config` selects the application document;
the PromptKit scope inspects the engine assembled for it. Request overrides
do not mutate captured engine configuration. Do not emulate unsupported
arbitrary `--set` overrides for the PromptKit scope.
Use small invocation-owned state through CommandKit's facilities to reuse loaded
values. A pipeline shares one engine across all roles. `config diff` may load two
distinct configurations within one invocation: keep their selections separate,
including overlays/overrides. No persistent loader cache or configuration hash
framework is needed.
Reuse the application resolver for module/options checks, typed compatibility,
ordered dependencies, validator chains, and reference bindings. `pipeline list`
only enumerates configuration; `show` renders resolved composition rather than
building an engine to expand every profile. ConfigKit supplies loaded values and
native provenance/secret-aware inspection, not domain pipeline semantics.
Centralize known prompt/profile dependency checks in `internal/preflight`. It
accepts the resolved pipeline, module declarations, and native engine. Check
every statically known dependency, including LLM validators and correction
prompts, plus applicable/default profiles. Partial profiles alone are not errors.
Share identifiers and selection logic with execution; no independent inventory,
asset registry, or synthetic request execution is needed. Dynamic selections
receive their final checks at runtime. Findings identify pipeline, step/lane,
stage/validator, and resource where applicable.
`pipeline validate` and `run` call the same resolution/preflight functions;
handlers do not invoke other commands or parse their reports. Keep `config
validate` focused on configuration and remove its pipeline-specific mode.
Offline preflight reads configuration and prompt resources but needs no transcript,
reference contents, credential values, or provider calls. Concrete PromptKit
preparation/validation owns completed-request checks, including schemas and
credentials; provider smoke tests require explicit invocation.
Preflight does not discover configuration, register modules, build engines,
initialize storage, or render CLI reports. Modules do not import it. Use native
request builders for standard commands and add application callbacks only where
needed, preserving supplied operator values. Output/cache/debug allocation
belongs to the run path, with debug timing selected below.
### Configuration, Sources, And Prompt Assets
Introduce application configuration version 6 as the only supported version.
Preserve unrelated discovery, pipeline structure, and resolution behavior.
ConfigKit continues to own loading and composition under
[ADR-0020](../adr/0020-delegate-configuration-mechanics-to-configkit.md).
Retire `promptkit.profile_file`, `profile_dir`, and `local_backend` in favor of
native companion sources; do not introduce a separate `promptkit.config_file`.
Retire `concurrency.total_llm` and revise retry/worker settings as described below.
Their replacement defaults and detailed migration treatment remain open.
Use one source assembly for runtime, preflight, and inspection:
1. PromptKit's bundled OpenRouter underlay.
2. Notarius's embedded native prompts and profiles under `collections.notarius`.
3. An optional, higher-priority operator underlay containing `prompts.yml` and
recursive `.yml`/`.yaml` files under `prompts/`, beside the main application
configuration actually selected by ConfigKit.
Anchor companion discovery to that selected main source, whether explicit or
discovered. Configurations in the same directory share companions; there is no
working-directory fallback. The two companion locations are strict peers, so
filenames do not resolve conflicting operator records. Missing companions are
fine; loading errors do not silently fall back to embedded defaults. Preserve
native imports and source-relative resource paths.
Native declarations become the sole prompt assembly mechanism. Keep the embedded
content tree and module ownership of prompt meaning, response requirements, and
domain interpretation. Remove prompt/fallback-profile registration in
`AssetRegistry`, per-module `PromptAssetManifest` plumbing, `ModulePromptFS`,
shared-name maps, synthetic `sharedassets` mounts, and manual prompt enumeration
for checkpoint fingerprints. The resolved schema design below also removes
schema registration and filesystem-flattening machinery.
Convert maintained resources, examples, and fixtures to native identified lists,
`config:` source controls, and sibling inheritance (`extends: {id: parent}`).
Same-ID replacement supplies a complete record; distinct inherited variants can
customize an existing resource. Explain native profile-null/backend fallback
semantics. The optional PromptKit source header is version 2, independently of
Notarius's application version. Do not translate legacy resource formats or
synthesize compatibility aliases.
Select prompts by ID alone; remove obsolete PromptKit version selectors and
related request/debug metadata. Retain current local resource IDs and use exact
qualified references, such as `notarius/dnd.spells` and
`notarius/dnd-extraction`. Keep the extraction profile's existing model/settings
and its reference to the bundled unqualified `openrouter` backend. Inherited
application profiles use sibling local IDs in the same collection. The bundled
inventory differs from the old external catalogs and selects no model for an
application request; document intentional replacements for removed catalog IDs.
The bundled backend supplies no concurrency limit. Application/operator limits
need explicit native configuration; application-supplied defaults remain U7.
Preserve prompt prose, message ordering, roles, and cache boundaries where their
meaning carries forward. Review and deliberately revise initial/repair prompts
for accumulated conversation history; preserving every old byte or retry message
is not a migration requirement. Shared content should use native filesystem
contexts and imports rather than reconstructed virtual trees.
#### Prompt Assets And Response Schemas (U2 Resolved)
Use the existing root `assets` package's ordinary `embed.FS` directly for native
prompt sources and response schemas, with `fs.Sub` only where useful. Shared
message files use their actual paths within the native filesystem context.
Remove synthetic mounts, file copying, flattened schema namespaces, and module
asset manifests. Retire `readonlyfs` when its prompt-mounting and asset-flattening
consumers disappear; no replacement filesystem abstraction is needed.
Declare response format, validation mode, schema path, and structural-repair
budget in native prompt definitions by default. Modules select prompt IDs, supply
inputs, and interpret responses. Initial and correction prompts may reference
the same schema file. Execution, inspection, and prepared-request preview then
use the same declared contract without repeating schema selection in Go.
Role-specific prompt contents remain U4, and retry defaults remain U5.
Use a request-level native `OutputContract` only for a demonstrated runtime-
dependent contract. It replaces the whole configured contract, so supply the
intended format, validation mode, schema, and repair policy together. Do not
introduce a wrapper or per-call registry solely to reproduce Audita's approach.
Operator prompt replacements must still produce responses the selected module
can interpret; that module boundary does not require duplicating each prompt's
schema selection in Go.
Default to native `WithSchemaFS(assets.FS(), ".")`. Schema paths are relative to
that selected root, for example `dnd/spells/schemas/dnd_spells_llm.v1.json`.
They are separate from message-file source contexts: prompt underlays do not
implicitly overlay schema files. When an operator configures an external schema
directory, select PromptKit's native directory source instead of the embedded
root. It replaces the whole schema source and must supply the schemas used by
the selected prompts. Do not build an application filesystem overlay to mix
embedded and external schemas. Transparent layering would be a separate upstream
capability, not a requirement for this migration. Concrete configuration binding
is an implementation detail within this selected source policy.
Keep private response schemas as self-contained JSON documents. PromptKit owns
loading, compilation, structural validation, provider structured-output settings,
and structural repair. Remove Notarius's response-schema byte registry, copied
schema wrappers, defensive-copy layer, digest calculation, and duplicate
structural-validation machinery. Native prepared snapshots provide diagnostic
schema details; no schema fingerprint is added to the message-content cache key.
Keep typed decoding, source/reference checks, reconciliation safety, semantic
validators, artifact codecs, and durable artifact schemas in Notarius. Response
schemas describe model output; artifact schemas describe published and consumed
domain artifacts. Preserve identifiers and versions belonging to actual artifact
contracts. Retire private schema keys, registration records, hashes, and copied
identity/version/name metadata whose only consumers were adapters, checkpoints,
or reporting. No independent schema metadata system replaces them.
Implementation is pending. Verify native preparation of embedded and operator
prompt definitions, shared-file paths, schema selection from embedded and external
roots, and complete request-contract replacement where actually used. Missing or
invalid schemas should follow native preparation behavior. No new schema check
requiring synthetic requests is added to offline pipeline preflight.
### Native Production, Continuation, And Assessment
Start with one conversation per producer operation: a chunk-plan generation,
one lane's extraction of one chunk, or one lane's normalization pass. Use
complete replacement model responses. Normalization may continue to return a
reconciliation plan that deterministic code applies to produce the artifact.
Per-entity branching is not part of this migration.
The application flow is:
1. Prepare source context, references, and candidate-handle mappings once. Build
the native initial request. For cacheable work, prepare and check the cache
before execution as described below; execute the retained request on a miss.
Otherwise use ordinary `Engine.Run`. Capture an exchange when continuation
or debug inspection needs one.
2. Consume the native usable response or terminal failure, decode, and apply
module interpretation and safety checks. A module can supply actionable
findings before it has a materialized candidate. Run configured artifact
validation on each candidate.
3. For actionable correction, call `Engine.Coordinate` with the captured exchange
and `AdditionalTurns`. Its callback selects a repair prompt and supplies
findings; PromptKit retains the initial exchange and subsequent turns.
4. Interpret every delivered revised response and rebuild from the fixed original
inputs. Run the complete configured validation chain again for changed
candidates, including validators that previously approved. Assess the final
permitted response before declaring exhaustion.
5. Stop on acceptance or a terminal condition. Apply application rejection,
failure, or supported safe-fallback policy after exhaustion. Native `stopped`
and `exhausted` outcomes do not themselves mean application acceptance.
Repair prompts supply a new user task; enduring principles belong in the original
system instructions. Keep source/reference and candidate-handle meanings stable
throughout the conversation. Avoid replaying the original task and constraints
that permit only an initial response. Accumulated history increases context use;
select the semantic turn budget accordingly. Separate repair profiles or schemas
are supported but not yet selected.
Remove `single_response_v1`, raw-response `ModelCandidate`/`SemanticCorrection`
transport, manual assistant/user replay, and correction-protocol capability
checks. Replace `runProducerAttempts`' LLM retry loop with native coordination and
application assessment. Adapt `NormalizeRetry` and `producerRetryDirective` to
findings and explicit fallback information. Retain useful ordered collection and
deduplication of actionable findings, rendered through a native repair prompt.
Reason codes and operator messages are not automatically model instructions.
Transport retries, structural repair, and semantic additional turns remain
separate controls. Preserve the ability to override native transport retries,
using the upstream default when omitted. Remove blanket outer LLM retries after
native transport/structural budgets are exhausted, including repeated LLM
validator invocations in `executeValidationChain`. Keep the ordered validator
chain and aggregation. A validator execution failure is distinct from rejection
of a producer candidate. Existing producer/validator `retries` settings cannot be
mechanically renamed; decide their migration and non-LLM failure handling
explicitly. Use native `OperationTimeout` when that operation scope is needed,
without adding an arbitrary default or replacing broader application deadlines.
Keep source grounding, reference resolution, candidate isolation, domain
invariants, and reconciliation safety checks. Unknown or overlapping handles
remain invalid. A supported safe fallback may retain safe merges while leaving
unsafe groups separate; it must pass configured artifact validation and terminal
policy. After structural-repair exhaustion it can be assessed directly without
restarting the same model operation. Whether an earlier safe candidate can
survive a later failed turn remains open; do not silently introduce best-so-far
acceptance.
Preserve rejection versus unavailable/incomplete validation. A structurally
valid candidate may advance under the existing `warn_continue` policy with
explicit incomplete-validation outcome information, but it and its affected
derivatives are ineligible for accepted-artifact cache reuse or publication. A
completed rejection never advances. Cancellation cannot make an undelivered
response accepted. Keep conversation state in process, outside ordinary
artifact/cache payloads; explicitly requested detailed debug is separate.
### Admission And Pipeline Workers
Remove `Scheduler`, `scheduledClient`, and custom global queue/permit plumbing.
Retire the separate cross-backend ceiling and the default/maximum extraction
worker coupling to `concurrency.total_llm`. Keep bounded extraction workers and
bounded lane continuations with independent defaults to be selected.
Use one shared native engine across all roles. Native capacity belongs to the
engine and backend ID: profiles selecting the same ID share its pool; different
backend IDs or separate engines do not automatically share limits, even for the
same endpoint. Endpoint-only profiles and backends without a positive limit
remain unrestricted by native backend admission.
For limited backends, the reviewed baseline bounds active model attempts by
`concurrency_limit` and admitted operations by that limit plus `queue_capacity`.
Ordinary execution admission spans preparation, retries, and repairs; split
`PrepareExecution` itself reserves no capacity, with admission occurring during
`RunPrepared`. Active permits are released during retry waits.
A completed continuation turn releases admission before assessment, allowing a
nested validator to use the same backend. Configure native limits at the intended
capacity boundary and handle `ErrCapacityExceeded` through application policy.
Aggregate concurrency across independent backends may exceed the former global
ceiling. That change is intentional. Bounded pipeline workers are application
work controls, not a substitute cross-backend LLM ceiling. Defaults, retired-field
migration, and role-specific admission-failure dispositions remain open.
### Accepted-Artifact Cache
Replace the checkpoint subsystem with a cache around the complete Notarius
producer operation. Its value is an accepted answer to the original task. This
avoids repeating expensive model work after a downstream failure without caching
every cheap deterministic pipeline step.
```text
Prepare initial request -> obtain MessageContentHash -> look up cache/<hash>.json
Hit: decode the accepted artifact and continue the pipeline
Miss: execute the retained request and complete the Notarius operation
-> interpret, validate, and perform application-directed corrections
-> on final acceptance, store the artifact under the INITIAL hash
```
Use the proposed upstream hash of ordered message roles and exact text. Include
message boundaries and order, excluding model/settings, session/execution IDs,
timing, provider cache hints, and other non-content metadata. Add no stage,
prompt-ID, library-version, whole-configuration, schema-policy, or implementation
fingerprint. Actual input or schema text rendered into messages participates as
message content; metadata outside those messages does not.
| Change | Intended reuse behavior |
| --- | --- |
| Fix a downstream stage | Earlier artifacts remain reusable when their initial messages match. |
| Change initial message text, roles, order, or boundaries | Cache miss. |
| Change a prompt ID, model, or settings without changing messages | Cache hit remains possible. |
| Change deterministic interpretation or validation | Existing accepted artifacts remain reusable; explicit bypass/refresh applies the new policy. |
Obtain the key before model execution through native preparation. Preserve the
prepared snapshot for execution on a miss using `PrepareExecution`/`RunPrepared`
where supported, and discard the handle on a hit. `OnPrepared` execution hooks
alone cannot short-circuit execution. v0.23.3's `RenderedPromptHash` includes
session identity and cache hints; it is not the requested hash. The upstream
request in U1 must land before completing this cache integration.
Keep the initial key throughout corrections. Cache the final Notarius artifact
through its existing codec, not an intermediate response, `RunResult`, exchange,
or prepared handle. For normalization, save the accepted normalized artifact
after applying the reconciliation plan, not the proposed plan itself. Save only
after the entire Notarius operation succeeds, including domain validation and
any application-directed corrections. Failed, rejected, and validation-incomplete
work and ineligible derivatives are not cacheable.
A hit loads the accepted artifact directly, with ordinary decoding/usability
checks; absent or unusable entries are misses. It makes no model calls, rebuilds
no result/exchange, replays no corrections, and performs no new semantic
validation merely because the artifact was cached. It makes no claim of acceptance
under changed validation policy and contributes no historical token usage to
current execution totals.
PromptKit owns everything inside each native request, including structural
validation, transport retries, and repairs. Notarius consumes its public response
or terminal failure and owns domain acceptance. Internal PromptKit attempts do
not determine cache eligibility. Native diagnostic evidence does not transfer
that ownership. No cached-response injection, replay API, exchange restoration,
or PromptKit-owned write-through cache is requested.
Use a small filesystem lookup/write facility and ordinary temporary-file/rename
publication. `cache/<hash>.json` contains the artifact, with no required manifest,
acceptance ledger, historical provenance, or versioned checkpoint envelope.
Lookup does not require original output or debug directories. Remove checkpoint
identities, dependency fingerprints, invocation-wide barriers, and recovery
bookkeeping. Preserve the separate canonical chunk-plan cache and its materialized
chunk validation. Cache directory configuration, reuse/bypass/refresh controls,
and handling of old checkpoint files remain U8; old `--resume`/`--recompute-step`
conventions are not compatibility requirements.
### Inspection And Execution Bookkeeping
Delegate configuration/resource inspection, requirements, concrete preview and
validation, and explicit smoke diagnostics to ckconfig/ckprompt through the shared
loaders. Remove effective-YAML parsing, the profile-inspector wrapper, and copied
effective-parameter types. A configured profile may be partial; use native
prepared/result details for the actual execution target, including overrides.
Standard commands retain their native reports, support/applicability distinctions,
and failure behavior. Smoke status zero alone does not prove a check passed.
Use ordinary `Run` with `OnPrepared` when actual request details are needed for
diagnostics; split preparation has a separate purpose for cache lookup. Native
observations provide terminal facts even without a returned `RunResult`.
`OnPrepared` is absent on preparation/admission failures and is not a completion
record. Keep synchronous callbacks lightweight, compose capture/accounting
observers, and synchronize shared state where operations overlap.
Use native `RunID`/`ParentRunID` with application step/lane/chunk/validator scope.
Remove `LLMExecutionSummary`, duplicate usage types, conversions, cloning,
independent accumulators, debug completion wrappers, synthetic call IDs, and
adapter profile-recording/forwarding. Native snapshots serve actual-target
inspection; do not reconstruct a durable profile inventory.
Use `ExecutionGroup` for selected diagnostic totals, explicitly enrolling initial,
continuation, and nested validator executions without overlapping double counts.
Preserve terminal coverage and unknown-versus-zero usage. Reuse adds no historical
usage to current totals. These facts support diagnostics and capture, not a
mandatory consumer execution ledger. Notarius's acceptance, fallback, rejection,
incomplete-validation, publication, and reuse facts feed the completion summary
and debug overview without independently maintained reporting state.
### Artifact Delivery And Completion Output
Artifacts are the primary deliverable: publish accepted domain artifacts as files
in the output directory. Preserve intrinsic domain content and artifact schemas.
CLI `--format` controls the completion report, not payload encoding. Consumers
need no run manifest, metadata bundle, persisted receipt, or execution history to
use the artifacts. Retire reporting-only metadata and mandatory manifests instead
of rebuilding them with native values.
Submit one small structured value through `Reporter.Report`:
| Field | Meaning |
| --- | --- |
| `pipeline` | Selected pipeline identifier. |
| `artifact_count` | Deliverable artifacts, including reused artifacts; excludes intermediates, cache entries, and debug files. |
| `output_directory` | Absolute published artifact directory. |
| `debug_directory` | Optional absolute completed run-specific capture directory; omitted when capture is disabled. |
Supply a native text template for the ordinary successful case:
```text
Success: dnd-session produced 6 artifacts in /work/results/run-123
```
When capture is available, append `Debug capture: <run-directory>`. Provide the
path rather than mirroring the bundle. JSON/YAML render the same structured
value through native serializers. A plain string would render natively as text
but lose named fields in machine output; the template gives both presentations
without parallel encoders. No report-version, run-ID, token, profile, warning-count,
or validation-history collection is required.
When a completed run includes rejected work or permitted incomplete validation,
qualify the human summary and structured value with the rejection count and/or
incomplete-validation fact. Keep these facts visible without verbose/debug mode;
small outcome qualifications do not require per-producer histories. Exact names
of those conditional fields are implementation details.
Wire results to stdout and diagnostics to stderr:
| Mode | Stdout | Stderr |
| --- | --- | --- |
| Ordinary | Brief completion report. | Failure explanation when needed. |
| `--verbose` | Same completion report. | Native information, warnings, and failures. |
| `--quiet` | Nothing. | Failure explanation when needed. |
| `--debug` | Completion report with capture path when enabled. | Native diagnostics and default progress. |
Use native mode ordering (last enabled mode wins), explicit progress selection,
and quiet suppression. Quiet still permits failure explanations and does not
prevent rendering errors from failing the invocation. `--format json --verbose`
keeps stdout valid report JSON. Standard configuration/PromptKit commands retain
their own native values, with no universal Notarius envelope. Artifact-quality
advisories retain their domain classification.
Optional saved reports use shell redirection, without a report-file flag or
automatic duplicate persistence:
```sh
notarius run dnd-session --input-file transcript.json --format json > run-report.json
```
Publish artifacts and finish requested capture before submitting a successful
completion report. Use native outcomes: status 0 for command completion, 1 for
operational/cancellation/publication/presentation failure, 2 for invalid invocation
or identifiable invalid configuration, and 3 for native `config diff --check`
differences. Preserve ckconfig's structured classification and underlying causes;
not every preflight failure is invalid input. Command completion is distinct from
domain acceptance: permitted rejections/incomplete validation qualify the report,
while terminal pipeline failure fails the command.
On failure, supply a useful native explanation and nonzero status, without a new
structured error-envelope protocol. `--format` does not serialize diagnostics.
Retain CommandKit's separation of operation causes from output/diagnostic delivery
failures. If publication succeeded before capture/report delivery failed, identify
the published path and available debug location. Published artifacts remain valid;
do not rerun model work or rewrite execution history. Subprocess callers check
status before decoding a successful report; failed writes may leave partial stdout.
### Debug Capture And Optional Diagnostics
Capture the **most convenient** debug representation, rather than optimizing
for conciseness, exhaustive completeness, or strict deduplication. Native snapshots
and readable views may repeat evidence. Reuse application decisions and native
execution facts instead of maintaining a second ledger or transcript model.
Debug formats are diagnostic data, not stable consumer contracts.
#### Controls And Location
Bind an application boolean to native `--debug`. Capture requires that boolean
and a nonempty effective directory; a directory alone does not enable capture.
Select `--debug-dir PATH` over `debug.directory` over `/tmp/notarius-debug`.
Resolve nonempty configuration-relative paths against their source and CLI-relative
paths against the invocation directory. Recognize explicit empty values first:
`--debug-dir ""` or `debug.directory: ""` disables capture without fallback.
A nonempty CLI override can re-enable capture over an empty configured root.
An unusable nonempty destination fails requested capture rather than selecting
another directory. Remove the configuration-relative `./notarius-debug` default.
Capture selection is independent of the final reporting mode. `--debug --quiet`
captures while suppressing stdout; `--quiet --debug` selects debug verbosity and
capture. `--debug=false` disables capture through native binding semantics.
Empty-directory debug retains native verbosity. No separate capture flag or
capture-depth switch is needed.
#### Lifecycle And Failure Handling
Allocate the run bundle after application configuration and run identity are
available, before engine construction and pipeline preflight. Argument/config-load
failures may have no bundle. Keep allocation out of shared loaders.
Create `<debug-root>/<run-id>/` and update `<debug-root>/latest` to a relative
symlink to that run at session start. It points to the newly started session even
if that session fails. Do not retarget at completion; concurrent starts use the
last successful link update. Runs without capture leave it untouched.
One application recorder accepts initial context, native evidence, and application
decisions, then finalizes once with pipeline/publication/capture outcomes. Persist
as operations advance, without holding the whole pipeline's content in memory.
Keep debug separate from output/cache and do not automatically delete bundles.
Requested capture is required work: allocation/link failures stop before model
work; recording failures surface at the next normal application boundary;
finalization failure fails the command. Observer callbacks retain errors for
boundary checks because they cannot return errors. An existing pipeline failure
remains primary, with capture failures reported additionally and causes retained.
Already-written data and published artifacts remain available. Capture failures
never cause model retries. Finalize before submitting the success report.
#### Inventory And Depth
Replace separate summary/trace categories with a compact run overview,
configuration/composition snapshots, parsed source, and per-operation records.
Use native execution IDs with application scope for navigation. `latest` is the
selected link name; other filenames and serialized application fields remain
implementation choices.
| Capture | Selected contents and depth |
| --- | --- |
| Run overview | Application version, run/pipeline identity, timing, outcome, publication location, effective invocation selections/overrides, and navigation. Terminal facts come from the same application outcome used by the CLI. |
| Configuration and composition | Native application and PromptKit effective snapshots, normally once each, plus resolved steps, lanes, modules, validators, and references. |
| Parsed source | Generic input document with unit identities and content, independent of the original input file. If parsing fails, retain the input location and available diagnostics. |
| Operation inputs | Actual source/chunk and reference content, incoming artifacts, prompt bindings/variables, and fixed original normalization candidates/handle mappings. Include directly even when duplicated elsewhere. |
| Prepared requests | Native `PreparedRun`, including settings, messages, and exposed output-contract/schema details when available. |
| Execution events | Native per-execution events in occurrence order, including attempts/retries/repairs, timing, usage availability, and terminal metadata. |
| Results and responses | Native `RunResult` plus exact `RawOutput` in a separate editor-readable text file. |
| Conversations | Full `Exchange.Details()` for every application-visible turn, including rejected turns, plus simple readable role-labeled messages. Repeated prefixes are acceptable. Use prepared messages when no completed exchange exists and label the representation. |
| Interpretation and assessment | Candidate via its codec before acceptance; decode/safety failures; each validator's findings and correction guidance; application disposition and links to LLM validator executions. |
| Normalization and corrections | Proposed reconciliation plan and applied candidate; constructed correction inputs even if the next preparation fails. |
| Deterministic stages and reuse | Chunk plans/materialized chunks, merged/normalized artifacts, and the loaded cached artifact with the current hit decision. No reconstructed history or fabricated execution on reuse. |
Use indented JSON for native structured snapshots and existing domain codecs.
Retain selected content in full, with no arbitrary truncation. Enable exchange
capture where debug inspection requires it, including single-turn calls;
continuation's in-process exchange needs remain independent of durable capture.
Keep operation inputs self-contained enough for direct investigation without
requiring original inputs, a cache entry, or reconstruction of deltas. Do not
archive every original file or serialize the pipeline's entire internal state.
On failure, retain available evidence, where execution stopped, errors, and
application disposition. Preparation/admission failure may have no `OnPrepared`;
operational failure may have no `RunResult`. Capture a coordination `Undelivered`
response as unassessed. Native internal-attempt metadata is useful but intermediate
provider response bodies are not required. No provider wrapper or complete
provider-transcript guarantee is part of this design.
#### Data Handling And Diagnostic Commands
Operators secure debug data as they secure ordinary application inputs and
outputs, for example with filesystem permissions. Debug contains internal data;
no separate security/retention model or sanitized-bundle promise is required.
Do not directly print/serialize ConfigKit-marked secrets, and preserve native
credential omissions through secret-aware inspection. Do not heuristically scan,
scrub, or rewrite prompts, source material, model output, or arbitrary strings.
Inspection/diagnostic commands use native reports and do not automatically create
pipeline debug bundles. Operators may redirect those reports. Neither debug mode
nor capture initiates provider probes. Explicit smoke checks retain native
support, applicability, and outcome distinctions.
## Unresolved Choices
The architecture and public direction above are selected. Resolve each remaining
choice before its dependent work; independent work need not wait. Routine API
placement and serialization choices are implementation details, not reasons to
reopen the architecture. IDs retain continuity with the earlier roadmap.
| ID | Remaining choice or dependency | Resolve for |
| --- | --- | --- |
| U1 | Supplying PromptKit API/release for the requested message-content hash; verify the final compatible dependency set against the selected baseline. The upstream request is pending. | Track in Phase 1; required to finish cache integration in Phase 5. |
| U4 | Role-specific initial/repair definitions, response contracts, and profile selection within complete replacement and fixed operation inputs. | Phases 3–4. |
| U5 | Producer/validator retry-setting migration to native controls and semantic budgets: field names/defaults, non-LLM failure handling, and any justified recovery beyond native exhaustion. | Phases 3–4 and configuration migration. |
| U6 | Detailed terminal/fallback policy, especially whether an earlier safe candidate survives a later failed turn. No implicit best-so-far acceptance. | Phases 3–4. |
| U7 | Independent worker defaults; application-supplied native backend/queue limits, if any; retired `total_llm` migration and admission-failure handling by role. | Phase 4. |
| U8 | Cache directory configuration and reuse/bypass/refresh controls (names, defaults, scope); old-checkpoint treatment and unusable-entry diagnostics. Hash inputs, lookup timing, accepted value, and ownership are settled. | Design with Phase 3; finish in Phase 5. |
| U9 | Observer/group enrollment and application scope; concrete conditional report fields and debug filenames/serialization within the agreed contracts; publication-layout/consumer migration details preserving intrinsic artifact schemas. | Implementation details in Phases 3–5; no new receipt/manifest inventory or capture-policy decision required. |
**U2 resolved:** Ordinary embedded assets, prompt-declared output contracts,
native schema-source selection, and removal of duplicate schema/filesystem
machinery are selected in [the schema design](#prompt-assets-and-response-schemas-u2-resolved).
Implementation remains in Phase 2; role-specific prompts and retry budgets remain
U4/U5.
**U3 resolved:** Typed stage boundaries, composition-root engine construction,
and the shared pipeline producer helper with module functions and a pipeline
assessment callback are selected in [the execution design](#module-execution-and-assessment-u3-resolved).
Implement in Phase 3 and refine signatures during Phase 4 without reopening
ownership. U4–U6 still own role-specific behavior.
**U10 resolved:** CommandKit versions, capabilities, commands, loaders/preflight,
output, debug policy, and coordinated cutover sequencing are selected above.
The former later CommandKit phase is retired. CLI compatibility is a non-goal;
existing artifact schemas remain domain contracts. Any actual artifact-format
change discovered during implementation needs an explicit compatibility decision,
not an assumption that native CLI adoption authorizes it.
## Implementation Sequence
All six phases are pending. Each delivers a reviewable outcome, not a separate
release or compatibility layer. Tests and canonical documentation accompany
behavior changes; the last phase verifies integration and completeness. Record
major boundary changes in successor ADRs without rewriting accepted decisions.
### Phase 1: Establish The Migration Baseline
- Record the selected dependency set and track U1 against the upstream request.
Keep the cache dependency explicit without blocking independent work.
- Map affected ADRs, current callers, contracts, and removal inventory. Carry
forward Audita evidence without adopting its application wrappers.
- Preserve U4–U9 with their dependent phases; do not reopen resolved module
execution, assets/schemas, CommandKit, debug, cache, or output ownership.
**Completion evidence:** reviewed dependency baseline, upstream dependency, scope,
and outstanding choices are explicit. This phase does not claim that the hash
already exists or that the migration is implemented.
### Phase 2: Build Native Composition, Sources, And Preflight
- Implement the resolved U2 design, update available dependencies, introduce
configuration version 6, convert native embedded/companion resources, and use
native schema loading with embedded/default or operator-selected roots.
- Build the small CommandKit application, shared ConfigKit/engine loaders,
standard suites, pipeline declarations/native alias, and `internal/preflight`.
- Add maintained embedded-only and operator-override examples. Remove duplicated
loading and asset assembly as callers migrate.
**Completion evidence:** runtime/inspection/preflight share intended definitions;
source contexts, shared message paths, prompt-declared output contracts, selected
schema roots, strict peer conflicts, and replacement work offline. Help,
version, and completion need no configuration. Preflight checks known dependencies
without credentials or synthetic requests. `config diff` handles two selections.
### Phase 3: Prove A Representative Native Pipeline
- Implement the resolved U3 collaboration and settle relevant U4–U6 details.
Connect module request/interpretation functions to the pipeline assessment
callback through the shared producer helper; refine concrete signatures here.
- Exercise an LLM validator on the same engine and normalization with fixed
handles and fresh plan application. Select actual observer/group enrollment.
- Run through the new command surface and alias, using the small completion
report, native outcomes/streams, and useful debug capture from the outset.
Begin U8/U9 integration without a provisional local message hash.
**Completion evidence:** fake-provider workflows exercise accumulated history,
full candidate revalidation through the same assessor, final-turn acceptance
without duplicate final validation, selected exhaustion/fallback, and useful
failures. The executable publishes artifacts and reports them through native
presentation without the private completion/correction API.
### Phase 4: Migrate All Roles And Remove Superseded Machinery
- Migrate every LLM-backed chunker, extractor, normalizer, and validator; preserve
deterministic paths and ordered validation. Complete U4–U7 across roles.
- Remove the adapter, copied completion contracts, manual replay/capability checks,
outer LLM retries, global scheduler/ceiling, and coupled worker settings.
- Finish native command/inspection migration and remove custom dispatch, flags,
encoders, verbosity handling, and obsolete asset registration. Retire the
emptied `internal/framework/llm` package after moving engine assembly to the
composition root and producer collaboration to the pipeline.
**Completion evidence:** every role uses the shared native engine and selected
admission policy. No production path depends on the private completion API,
latest-response-only correction, global ceiling, or legacy CLI infrastructure.
Obsolete tests retire with the machinery; surviving domain requirements remain.
### Phase 5: Complete Cache, Output, And Capture Integration
- Resolve U1's supplying release and complete U8. Implement lookup at initial
preparation, accepted-artifact storage under that hash, explicit cache controls,
and deliberate old-checkpoint handling. Keep canonical chunk caching separate.
- Complete the artifact-first publication/report contract and U9 details. Remove
checkpoint fingerprint graphs, mandatory manifests/receipts, durable execution
ledgers, and duplicate accounting; do not retain them to populate summaries.
- Complete capture inventory, per-operation evidence, directory/empty-value
behavior, `latest`, and recording/finalization failure handling.
**Completion evidence:** downstream failure/fix/retry reuses accepted predecessors;
message changes miss and settings-only changes can hit; corrected answers use the
initial key; hits make no model calls or fresh validation claims. Artifacts need
no metadata bundle. Machine output and debug capture obey the selected contracts,
including failures after publication and quiet-mode capture.
### Phase 6: Verify And Document The Coordinated Cutover
- Complete the acceptance checks and removal inventory, including dependency
cleanup, maintained examples, and the assembled command surface.
- Finish release migration guidance, attribution, successor ADRs, and affected
canonical documentation. Reconcile future-work notes and notepad status.
- Confirm no compatibility scaffold, deferred CommandKit adoption, local hash
policy, or obsolete metadata requirement remains in the finished migration.
**Completion evidence:** PromptKit, ConfigKit, and CommandKit adoption, accepted-
artifact reuse, artifact delivery, diagnostics, and maintained offline workflows
are verified together. Remaining work is explicitly separate future scope.
Release preparation/publication is governed separately by release policy.
## Removal Inventory
| Retire or replace | Retained application responsibility |
| --- | --- |
| `StructuredLLMClient`, private request/response types, adapter conversions, emptied `internal/framework/llm` package | Composition-root engine construction, native module request/interpretation functions, and pipeline assessment/helper. |
| `AssetRegistry`, `PromptAssetManifest`, `ModulePromptFS`, synthetic mounts, flattened schema namespaces, `readonlyfs`, manual fingerprint enumeration | Ordinary embedded content, native source assembly, and module meaning. |
| Private response-schema registry/wrappers, byte copies, digest and reporting-only identity metadata, duplicate structural validation | Native prompt output contracts and schema loading; application interpretation and durable artifact contracts. |
| `single_response_v1`, raw `ModelCandidate`/`SemanticCorrection`, manual replay, correction capability checks | Typed candidates, actionable findings, native exchanges, assessment. |
| LLM retry loops in `runProducerAttempts`/`executeValidationChain`, module rerun directives | Ordered validators, domain acceptance/rejection/fallback. |
| `Scheduler`, `scheduledClient`, `concurrency.total_llm`, worker coupling | Bounded pipeline work and native backend admission. |
| Checkpoint stage/dependency/configuration fingerprints, invocation-wide barriers, manifests, acceptance/provenance envelopes | Small accepted-artifact cache using the upstream message hash, ordinary load checks, explicit controls. |
| Inspection wrappers, copied effective parameters, usage summaries/conversions/accumulators, debug completion wrapper, synthetic IDs, profile recorder | Native facts with application scope for diagnostics. |
| Custom dispatch/help/version/parsing, command-specific loading, copied inspection, legacy aliases/flags, parallel encoders and verbosity handling | Native declarations/loaders, pipeline handlers, domain input checks. |
| Mandatory run manifests, reporting-only metadata bundles, extensive receipts and durable execution ledgers | Domain artifact files, one small native completion report, optional debug evidence. |
| Summary/trace hierarchy, heuristic debug-content scrubbing, custom transcript/deduplication machinery | Convenient run/operation capture and native secret/credential omissions. |
## Acceptance And Verification
Use real CommandKit, ConfigKit, and PromptKit in representative assembled flows
with a fake at the provider boundary. Keep the suite deterministic, offline, and
independent of real credentials. Test domain interpretation/assessment at their
package boundaries; use subprocess tests for streams, exit status, alias wiring,
and representative machine output. Do not duplicate upstream parser/renderer/
retry tests or freeze complete help text and native debug snapshots.
| Area | Required evidence |
| --- | --- |
| Configuration/assets | Version 6/new resources work; old formats intentionally rejected; companions/native imports retain main-source context; peer conflicts fail; same-ID replacement, real shared-file paths, prompt-declared contracts, and embedded/external schema roots work; necessary request overrides replace the complete contract. |
| Commands/loaders | Required operands, native alias, supplied-value distinctions, references, and run-wide overrides reach intended handlers/roles; help/version/completion need no config; diff selections remain distinct. |
| Preflight/inspection | Same definitions across execution and inspection; missing statically selected resources fail with context; partial profiles alone pass; no provider calls, transcript, or credential requirement for offline preflight. |
| Domain workflow | Representative extraction and normalization preserve source/reference identity, fixed handles, safe plan application, ordered validators, artifact schemas, and stable output ordering; one assessor checks initial/corrected candidates without an extra final pass; deterministic stages and LLM validators need no producer conversation loop. |
| Continuation/failures | Accumulated history, full revalidation and final-turn assessment; no blanket retries beyond native exhaustion; validator failure differs from rejection; selected fallback, incomplete-validation, and cancellation policy hold. |
| Admission/workers | Shared backend pools across roles, independent backends, bounded pipeline work, nested validators, queue exhaustion, and cancellation without leaked capacity or a global ceiling. |
| Accepted-artifact cache | Downstream recovery; initial-message keys; corrected final artifact stored late under the initial key; changed messages miss; settings/ID-only changes can hit; no model/revalidation on hits; unusable entries miss; failed/rejected/incomplete work and ineligible derivatives excluded; no required history. |
| Cache migration | Selected reuse/bypass/refresh and old-record treatment work; invocation configuration no longer blocks lookup; canonical chunk caching retains its distinct policy. |
| Output/outcomes | Artifact files independent of report format; native text/JSON/YAML summary and qualified outcomes; no mandatory metadata bundle; quiet suppression, verbose stderr, debug path; native nonzero failures and useful explanations. Publication survives later capture/report failure without model reruns. |
| Execution observations | Initial/continuation/validator totals preserve unknown-versus-zero usage without double counts; result-less failures observable; reuse does not count historical work; native execution success remains distinct from acceptance. |
| Debug controls/lifecycle | Default/CLI/config/explicit-empty paths and native mode ordering; quiet capture; allocation before engine/preflight; start-time relative `latest`, no completion retarget; disabled runs leave it untouched; capture failures preserve primary causes and publication facts. |
| Debug evidence | Available input/source/native snapshots, readable text, all visible turns, assessments, normalization and deterministic outputs, cache hits, and failed-run evidence; no invented provider transcript; ConfigKit secrets/native credentials omitted, no heuristic content scrubbing. |
| End-to-end migration | Maintained minimal and complete D&D workflows, session routing, reference behavior, raw artifacts, reports, and migration examples work through the new executable. No postponed CommandKit phase. |
Run focused checks during implementation and the repository-wide checks for the
completed cutover under the [testing policy](../policy/testing.md):
```sh
go test ./...
go vet ./...
go build ./cmd/notarius
```
Check maintained examples, dependencies, attribution, documentation links, and
removal coverage. Live provider smoke and human model-quality evaluation remain
optional, outside the default suite. These migration checks add no permanent
release gates. Documentation completion does not claim that implementation
acceptance checks have passed.
## Migration Guidance And Documentation
Provide practical release guidance for operators and subprocess callers:
- New commands/flags, native `run` alias, `--format`, stdout/stderr, outcomes, and
saving optional reports by redirection. No legacy CLI compatibility promise.
- Version 6 and native resources, embedded-only versus companion examples in
appropriate directories, imports, qualified IDs, replacement/inheritance,
native local backends, removed catalogs/versions, and separate source versions.
- Native admission and worker defaults, removed global limit, retry-budget
migration, accumulated-history costs, and selected role failure policy.
- Accepted-artifact caching, message-based misses, settings-only reuse, refresh/
bypass for changed interpretation or validation, and old-checkpoint handling.
- Artifact delivery without mandatory manifests/receipts; preserve intrinsic
artifact schemas and document any explicitly selected format change.
- Debug root/default/empty behavior, `latest`, native mode ordering, failure
behavior, and ordinary operator responsibility for application data.
Update canonical [configuration](../config.md), [CLI](../cli.md),
[operations](../operations.md), [internal overview](../internal/overview.md),
[LLM](../internal/llm.md), [pipeline](../internal/pipeline.md),
[modules](../internal/modules.md), [state](../internal/state.md), and affected
integration/consumer guides as behavior lands. Remove superseded manifest/receipt
contracts then; keep current-behavior documentation accurate until implementation.
Update [architecture policy](../policy/architecture.md) for native dependencies
and changed scheduler, retry, asset, cache, and reporting boundaries.
Record significant changes in successor ADRs, including
[asset assembly](../adr/0011-centralize-llm-assets.md),
[validation retries](../adr/0014-feedback-aware-validation-retries.md),
[module retries](../adr/0016-feedback-aware-module-requested-retries.md),
[PromptKit configuration](../adr/0019-adopt-native-promptkit-configuration.md),
admission, caching, and CLI/reporting ownership. Preserve accepted decision text
and update supersession status under [documentation policy](../policy/documentation.md#architecture-decision-records).
Notepads retain discussion history; this roadmap owns implementation status.
[release]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.3/docs/releases/v0.23.3.md
[migration]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.3/docs/consumers/configuration-migration.md
[formats]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.3/docs/formats.md
[consumer]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.3/docs/consumers/pkg-promptkit.md