104 KiB
Native PromptKit And CommandKit Migration
Purpose And Status
Make Notarius fully native to PromptKit, ConfigKit, and CommandKit. Use upstream configuration, execution, continuation, admission, inspection, commands, and presentation directly. Retain the application work of producing, assessing, publishing, and reusing typed artifacts. Historical duplication of library behavior is not a requirement to preserve it.
This is the feature roadmap for PromptKit, ConfigKit, CommandKit, accepted-artifact caching, and artifact-first output. It owns the selected target, decision status, removal inventory, acceptance criteria, and supporting evidence. The earlier discussion documents are retired; Git history preserves the discussion. The implementation plan owns the ordered, prompt-sized stages. The native cutover and canonical documentation are implemented, with completion evidence recorded in that plan. The final whole-roadmap acceptance audit remains pending; implementation markers do not substitute for that verification. Current behavior belongs in Configuration, CLI Reference, Operations, and native migration guidance. Unrelated future work remains in the separate backlog.
Notarius predates many modern PromptKit capabilities: the library grew out of mechanics shared by Notarius and related applications. Historical duplication and slight behavioral differences do not express an intentional application requirement. Evaluate remaining machinery against actual domain needs, and consider upstream features when a reusable capability is missing.
Complete one coordinated cutover for configuration, PromptKit resources, and the CLI. Adopt native public conventions fully; preserving old CLI behavior is a non-goal. Supply migration guidance and maintained examples instead of aliases, parallel parsers, or compatibility wrappers for legacy behavior. CommandKit is part of the main migration, not a later follow-up. Release preparation and publication remain separate work.
Adopting the kits together lets standard inspection and execution share the same loaders and native facts. It avoids building temporary command and reporting adapters while those boundaries are already being replaced.
Baseline And Dependency Target
At Notarius revision 12eeb6369e40746b2c18c55397a289a1e1b5908d, the application
used configuration version 5 and Go 1.27. Its dependencies included old-path
PromptKit v0.12.0 and ConfigKit v0.5.0, new-path ConfigKit v1.2.0, go-yaml
v4.0.0-rc.6, and two external PromptKit provider catalogs.
The selected reviewed baseline is:
| Module | Version |
|---|---|
go.maximumdirect.net/commandkit |
v0.7.0 |
go.maximumdirect.net/ckconfig |
v0.10.0 |
go.maximumdirect.net/ckprompt |
v0.11.0 |
go.maximumdirect.net/configkit |
v1.4.0 |
go.maximumdirect.net/promptkit |
v0.24.0 |
Keep Go's consumer minimum at 1.27.0. The reviewed set resolves stable
go.yaml.in/yaml/v4 v4.0.0 and github.com/santhosh-tekuri/jsonschema/v6 v6.0.3.
Published tags and dependency manifests were verified, and the modules resolved
together without local replacements in an isolated consumer module. Maintained
composed-CLI and offline-preparation examples and focused message-hash,
configuration-handoff, and schema tests passed without live provider calls.
Those checks established the upstream baseline; subsequent Notarius implementation
evidence is recorded in the plan, with final acceptance pending. See the
current integration boundary and
integration version guidance
and tagged release notes, migration guide,
format reference, and Go consumer guide.
U1 is resolved: PromptKit v0.24.0 supplies the prepared MessageContentHash
required by the cache, and ConfigKit v1.4.0 supplies typed result handoff and
focused inspection. Adopt these with the selected CommandKit integrations as
one baseline. The migration includes one application-owned load,
promptkit-rooted resources, native schema configuration, and the request API
changes described below; it is not just a dependency-version update. PromptKit's
v0.23.3 timeout and cancellation fixes remain included. Do not add a local
hash-policy fork or substitute RenderedPromptHash for message-content identity.
Neither a cross-backend global limit nor a comprehensive execution fingerprint
is required. Remove obsolete imports and catalog dependencies when unused;
avoid permanent replace directives and update attribution
under the OSS policy.
Audita's reviewed v0.23.2 integration demonstrates native source assembly,
continuation, observations, and CommandKit wiring. Adopt those mechanisms
without copying its RoleClient, structured-output interface, or opaque exchange
wrapper. Its per-proposal repair branches and lack of checkpoints do not define
Notarius's artifact and recovery policies.
Target Architecture
Application Structure And Ownership
| Part | Responsibility |
|---|---|
| Application composition | Load application and PromptKit settings together through ConfigKit; hand the completed result to one shared PromptKit engine for producers, validators, and continuation runs. |
| Artifact-family modules | Prepare domain inputs, select prompt IDs and response requirements, interpret responses, apply deterministic domain rules, and provide supported fallbacks. |
| Pipeline execution | Connect generation to configured validation and strict/permissive handling; manage ordered dependencies, typed handoffs, bounded workers, and stable result ordering. |
| PromptKit | Prepare and execute requests; maintain captured exchanges and conversations; enforce native retry, repair, turn, and admission budgets; report execution facts. |
| Application state and records | Publish artifacts, cache final accepted artifacts, provide a small completion summary, and capture optional debug evidence with pipeline context. |
| ConfigKit and CommandKit | ConfigKit owns configuration mechanics; CommandKit and ckconfig/ckprompt own standardized commands and presentation. Notarius supplies loaders, request builders, report content, and outcome mapping. |
Use native PromptKit requests, results, exchanges, errors, and observers in model-executing application code. Remove the private provider-neutral completion API. Pure domain models, codecs, source-reference resolution, and deterministic transformations need no PromptKit dependency. Application interfaces continue to express artifact production, typed validation, output encoding, and state access.
Module Execution And Assessment (U3 Resolved)
Keep the existing typed stage boundaries and use native PromptKit values in
model-executing code. Place engine construction and shared loaders in focused
files in the internal/cli composition root. It constructs one native engine
for the run. Retire internal/framework/llm once its adapter, scheduler, asset
registry, and reporting responsibilities disappear; do not retain a package
solely to hold a small constructor or introduce another LLM service interface.
Place one small shared producer helper alongside assessment in
internal/framework/pipeline. Connect modules to it through ordinary functions:
- The pipeline supplies an operation-scoped assessment callback capturing the configured validator chain and acceptance criteria. It applies the selected strict/permissive mode when no acceptable result is obtained.
- The module prepares source material, references, and any normalization handle mappings once. It builds the native initial request and supplies response interpretation, correction-request, and supported fallback functions.
- The helper receives those functions, the shared engine, and relevant cache/debug collaborators. For cacheable work it prepares the initial request and looks up the accepted-artifact cache; on a miss it executes the retained request. Otherwise it uses ordinary native execution.
- For each usable executed response, the helper invokes module interpretation and pipeline assessment. Recognize native errors and failed validation results as terminal failures before interpretation. When domain correction is appropriate, it supplies the module's next request to native coordination. PromptKit owns conversation state and turn iteration; the helper does not implement another retry engine.
- Return a typed application outcome. Cache only final accepted artifacts under the initial request's key. A cache hit follows the existing direct-reuse policy without model execution or renewed semantic assessment.
Request builders return promptkit.RunRequest; interpreters consume native
results. Candidates, findings, and acceptance outcomes remain application-owned.
Keep operation-specific state in local variables or closures, without an
execution-plan language, replacement completion API, or copied conversation model.
Use one pipeline assessment implementation for the initial candidate and every correction, including the final permitted response. Do not repeat final validation merely because the helper returns to the outer runner. Deterministic stages keep their ordinary typed production/validation path without synthetic requests or conversation machinery. LLM validators call the shared engine directly and return domain findings; they do not automatically acquire a producer correction loop.
Modules receive operation data and collaborators, not physical storage roots or
CommandKit reporters. Pure domain packages retain artifact models, codecs,
reference resolution, and deterministic rules. internal/preflight retains its
separate known-dependency checks. Exact argument grouping and helper signatures
can be refined against representative extraction and normalization during
implementation; ownership and placement are settled. U4 selects prompt roles and
guidance ownership below; U5 selects retry controls and the correction budget.
U6 selects strict-by-default handling and module-owned fallbacks below;
configuration precedence and fallback behavior are implemented. See
Pipeline Internals and the plan's completed operation/
module work; final acceptance remains pending.
Retain the fixed input -> chunk -> extract -> merge -> normalize -> output
shape, ordered steps and generated references, typed artifact-family boundaries,
and deterministic public ordering. Keep modules independent of physical output,
cache, and debug roots. Native adoption does not turn the pipeline into a general
workflow engine or redesign durable D&D artifacts.
Command Surface And Native Inputs
Install the application pipeline family alongside ckconfig and ckprompt:
| Command | Responsibility |
|---|---|
pipeline list |
Discover configured pipelines. |
pipeline show <pipeline-id> |
Report resolved steps, lanes, modules, validators, and reference relationships. |
pipeline validate <pipeline-id> |
Check composition and known prompt/profile dependencies offline. |
pipeline run <pipeline-id> --input-file PATH |
Execute the pipeline and publish artifacts. |
run <pipeline-id> --input-file PATH |
Native alias for pipeline run, with the same handler and inputs. |
Use commandkit.Alias("run", "pipeline run"), not a forwarding command. Use
notarius run in quick starts and document both routes. Install the full selected
standard suite: native help/version; Bash, Zsh, and Fish completion; configuration
scopes, validation, show, sources, explain, and diff; PromptKit resource listings
and inspection for prompts, profiles, backends, namespaces, and collections;
requirements discovery/validation and prepared-request preview; and explicit
reachability, authentication, text, structured-output, and aggregate smoke tests.
Include upstream convenience aliases and useful completion hints.
Use native typed declarations and supplied-value semantics. Preserve the
distinction between omission and explicit empty/false/zero values, then apply
each field's selected default or empty-value policy. Preserve native command-path,
operand, repeated-flag, and -- parsing conventions. Check every occurrence of
a singleton flag and use the last successfully parsed value.
| Concern | Selected input |
|---|---|
| Configuration | Native --config, --overlay, and supported application --set POINTER=JSON; scope named app. |
| Pipeline input | Required pipeline-ID operand and --input-file PATH. |
| Lanes | Repeatable --lane ID; omission selects the normal lane set. Commas are literal values, not separators. |
| Output | Application --output-dir PATH. |
| Cache | --cache auto|bypass|refresh overrides caching for the entire invocation, including chunk generation. Otherwise resolve module-binding cache, pipeline cache, then auto. |
| References | Repeatable --reference SELECTOR=PATH and --without-reference SELECTOR. |
| Presentation | Native --format, --quiet, --verbose, --debug, and --progress. |
| Capture destination | Application --debug-dir PATH, with the semantics below. |
Keep the worker limit and semantic budgets in configuration, adjustable through
application --set. U5 selects max_additional_turns; U7 selects
concurrency.workers with a default of 64.
Use ckprompt's reusable profile, session, and execution flags, including model,
temperature, timeout, reasoning effort, and transient retries, for run-wide
overrides across producers, validators, and continuations. Omission preserves
normal selection; explicitly
empty reasoning effort clears inheritance. Use native --llm-profile and
--llm-session-id, retiring the separate reasoning-clearing control. Native
transient retries remain distinct from semantic additional-turn budgets.
Individual prompt selection and named prompt inputs belong on standard PromptKit
commands; pipeline modules build their own requests.
Notarius retains reference-selector expansion, specificity, required slots, and
external/generated-reference conflict checks. Pipeline reference selectors are
not individual-request --llm-input-file bindings. Remove custom dispatch,
assembled help, standard flag parsing, comma-separated --only, and bespoke
--json switches. Replace config effective with config show and config profile with promptkit show profile. Old flags are migration evidence only.
Application Composition, Loaders, And Preflight
One constructor registers application commands, the native alias, ckconfig's
app and promptkit scopes, ckprompt, and native help/version/completion/reporting.
It does not load configuration, build an engine, or initialize run storage.
Help, version, and completion work without application configuration.
Share one application-owned configuration load and a separate engine-construction step across application and standard commands:
- Select application configuration through explicit
--config, then existing environment/discovery policy. Embedpromptkit.Settingsunderyaml:"promptkit"in the application configuration and compose its ConfigKit definition withpromptkit.Configure. Assemble the native sources below before callingconfigkit.LoadLayeredonce for that selection. Let ConfigKit apply overlays, imports, inheritance, overrides, defaults, normalization, and structural validation. Retain its native result and selected main-source context. Validate an explicitly empty configuration path rather than falling through to discovery. - Pass the successful whole result to
promptkit.NewFromConfig, with nativeRuntimeConfigfor runtime client/timeout/transient-retry options. Construction captures the resolved resources without loading sources or running ConfigKit callbacks again. SelectBundledOpenRouterUnderlaybefore loading; this constructor does not implicitly add the bundle. Across commands,--configselects the application document. Request overrides do not mutate captured engine configuration or the result's frozen inspection evidence.
Use ConfigKit's native typed-value and subtree-inspection handoff. ckconfig's
application scope and ckprompt's focused /promptkit inspection share that same
result; use native adapters, including FromConfigKitSubtree where a focused
scope is needed. Keep native configuration paths, override capabilities, and
whole-load source/diagnostic evidence. A focused view is not a separately loaded
document. Remove separate PromptKit source loading, reparsing, and inspection
conversion rather than wrapping the new handoff in an application abstraction.
Use small invocation-owned state through CommandKit's facilities to reuse loaded
results and constructed engines. A pipeline shares one engine across all roles.
config diff may load two distinct configurations within one invocation: keep
their selections separate, including overlays/overrides. No persistent loader
cache or configuration hash framework is needed.
Reuse the application resolver for module/options checks, typed compatibility,
ordered dependencies, validator chains, and reference bindings. pipeline list
only enumerates configuration; show renders resolved composition rather than
building an engine to expand every profile. ConfigKit supplies loaded values and
native provenance/secret-aware inspection, not domain pipeline semantics.
Centralize known prompt/profile dependency checks in internal/preflight. It
accepts the resolved pipeline, module declarations, and native engine. Check
every statically known dependency, including LLM validators and correction
prompts, plus applicable/default profiles. Partial profiles alone are not errors.
Share identifiers and selection logic with execution; no independent inventory,
asset registry, or synthetic request execution is needed. Dynamic selections
receive their final checks at runtime. Findings identify pipeline, step/lane,
stage/validator, and resource where applicable.
pipeline validate and run call the same resolution/preflight functions;
handlers do not invoke other commands or parse their reports. Keep config validate focused on configuration and remove its pipeline-specific mode.
Offline preflight reads configuration and prompt resources but needs no transcript,
reference contents, credential values, or provider calls. Concrete PromptKit
preparation/validation owns completed-request checks, including schemas and
credentials; provider smoke tests require explicit invocation.
Preflight does not discover configuration, register modules, build engines, initialize storage, or render CLI reports. Modules do not import it. Use native request builders for standard commands and add application callbacks only where needed, preserving supplied operator values. Output/cache/debug allocation belongs to the run path, with debug timing selected below.
Configuration, Sources, And Prompt Assets
Introduce application configuration version 6 as the only supported version.
Preserve unrelated discovery, pipeline structure, and resolution behavior.
ConfigKit continues to own loading and composition under
ADR-0020.
Retire promptkit.profile_file, profile_dir, and local_backend in favor of
native companion sources; do not introduce a separate promptkit.config_file.
Retire concurrency.total_llm and revise retry/worker settings as described below.
U5 selects max_additional_turns and removes application structural-repair
controls; U7 selects the shared worker pool and retired global-limit migration.
U8 replaces the nested legacy cache settings with root-level cache_directory
and pipeline/module-binding cache modes.
Use one source assembly in the shared ConfigKit load for runtime, preflight, and inspection:
- PromptKit's bundled OpenRouter underlay.
- Notarius's embedded native prompts and profiles under
promptkit.collections.notarius. - An optional, higher-priority operator underlay containing
prompts.ymland recursive.yml/.yamlfiles underprompts/, beside the main application configuration actually selected by ConfigKit.
These are ordered underlays of the single application load. The selected main document and its imports, selected overlay, and explicit configuration overrides participate through ConfigKit's native precedence. Do not introduce a second PromptKit main document or apply overlays/overrides again during engine construction.
Complete documents place native resources and schema_root beneath promptkit.
Use SettingsInput for wrapperless settings fragments and CollectionInput for
collection fragments; discovered fragments need an explicit destination because
discovery helpers return root-mounted inputs. Keep resource IDs and exact
references unchanged by the configuration path. Old root-level prompt/profile
lists are not consumed as PromptKit resources; do not translate them implicitly.
Anchor companion discovery to that selected main source, whether explicit or discovered. Configurations in the same directory share companions; there is no working-directory fallback. The two companion locations are strict peers, so filenames do not resolve conflicting operator records. Missing companions are fine; loading errors do not silently fall back to embedded defaults. Preserve native imports and source-relative resource paths.
Native declarations become the sole prompt assembly mechanism. Keep the embedded
content tree and module ownership of prompt meaning, response requirements, and
domain interpretation. Remove prompt/fallback-profile registration in
AssetRegistry, per-module PromptAssetManifest plumbing, ModulePromptFS,
shared-name maps, synthetic sharedassets mounts, and manual prompt enumeration
for checkpoint fingerprints. The resolved schema design below also removes
schema registration and filesystem-flattening machinery.
Convert maintained resources, examples, and fixtures to native identified lists,
config: source controls, and sibling inheritance (extends: {id: parent}).
Same-ID replacement supplies a complete record; distinct inherited variants can
customize an existing resource. Explain native profile-null/backend fallback
semantics. Notarius owns application version 6 through its ConfigKit definition;
PromptKit adds no resource-version rule. Retire the planned PromptKit version-2
source header. Collection release_version is optional descriptive
metadata, not a selector or validation gate. Do not translate legacy resource
formats or synthesize compatibility aliases.
Select prompts by ID alone; remove obsolete PromptKit version selectors and
related request/debug metadata. Retain current local resource IDs and use exact
qualified references, such as notarius/dnd.spells and
notarius/dnd-extraction. Keep the extraction profile's existing model/settings
and its reference to the bundled unqualified openrouter backend. Inherited
application profiles use sibling local IDs in the same collection. The bundled
inventory differs from the old external catalogs and selects no model for an
application request; document intentional replacements for removed catalog IDs.
The bundled backend supplies no concurrency limit. Under U7, Notarius injects no
backend concurrency or queue defaults; operators use native backend configuration.
Preserve prompt prose, message ordering, roles, and cache boundaries where their meaning carries forward. Review and deliberately revise initial/repair prompts for accumulated conversation history; preserving every old byte or retry message is not a migration requirement. Shared content should use native filesystem contexts and imports rather than reconstructed virtual trees.
Prompt Assets And Response Schemas (U2 Resolved)
Use the existing root assets package's ordinary embed.FS directly for native
prompt sources and response schemas, with fs.Sub only where useful. Shared
message files use their actual paths within the native filesystem context.
Remove synthetic mounts, file copying, flattened schema namespaces, and module
asset manifests. Retire readonlyfs when its prompt-mounting and asset-flattening
consumers disappear; no replacement filesystem abstraction is needed.
Declare response format, validation mode, and schema path in native prompt definitions by default. Omit structural-repair attempt settings in bundled prompts to inherit PromptKit's upstream default under U5. Modules select prompt IDs, supply inputs, and interpret responses. U4 selects the same schema file for each initial/correction pair. Execution, inspection, and prepared-request preview then use the same declared contract without repeating schema selection in Go. U4 selects prompt roles and contracts; exact prose is implementation work. U5 selects budget ownership and defaults.
Use a request-level native *promptkit.OutputContractInput only for a demonstrated
runtime-dependent contract. It replaces the whole configured contract, so supply
the intended format, validation mode, and schema together while leaving structural
repair at its native default by leaving RepairAttempts nil. OutputContract
remains the resolved type in native prepared details. Do not inject a repair count
merely to construct the override, or introduce a wrapper or per-call registry
solely to reproduce Audita's approach. Operator prompt replacements must still
produce responses the selected module can interpret; that module boundary does
not require duplicating each prompt's schema selection in Go.
Declare the bundled collection's native schema_root in an embedded ConfigKit
source whose filesystem context points at the assets root. Bundled prompts use
output.schema_path relative to that root, for example
dnd/spells/schemas/dnd_spells_llm.v1.json. An operator can replace
promptkit.collections.notarius.schema_root with an external directory supplied
from the operator configuration's filesystem context. That replaces the whole
root for those schema-path lookups and must supply their selected schemas;
missing files do not fall back to the embedded root. Do not retain the removed
WithSchemaFS, WithSchemaFile, or Config.SchemaDir APIs or introduce an
application filesystem overlay.
Support native prompt-owned inline schema and source-relative schema_file
selections as well, including operator prompts packaged with their own schemas.
Exactly one schema selection applies in JSON Schema mode. A nonnull collection
root takes precedence over promptkit.schema_root; an absent/null collection
root inherits it. Roots and file references retain the effective contributing
ConfigKit source's filesystem/base directory. There is no implicit working-
directory root. Request replacements use inline Schema or SchemaPath under
the selected prompt's applicable root; they do not inherit a configured
schema_file context. Native source composition owns these rules.
Keep private response schemas as self-contained JSON documents. PromptKit owns loading, compilation, structural validation, provider structured-output settings, and structural repair. It loads and compiles schemas during preparation and retains them in prepared execution. Remove Notarius's response-schema byte registry, copied schema wrappers, defensive-copy layer, digest calculation, and duplicate structural-validation machinery. Native prepared snapshots provide diagnostic schema details; no schema fingerprint is added to the message-content cache key.
Keep typed decoding, source/reference checks, reconciliation safety, semantic validators, artifact codecs, and durable artifact schemas in Notarius. Response schemas describe model output; artifact schemas describe published and consumed domain artifacts. Preserve identifiers and versions belonging to actual artifact contracts. Retire private schema keys, registration records, hashes, and copied identity/version/name metadata whose only consumers were adapters, checkpoints, or reporting. No independent schema metadata system replaces them.
Native preparation is implemented and covered by asset/operator-schema tests recorded in the plan. Final verification must confirm embedded and operator prompt definitions, shared-file paths, schema selection from embedded and external roots, source-relative/inline selection, and complete request-contract replacement where actually used. Missing or invalid schemas follow native preparation behavior. No new schema check requiring synthetic requests is added to offline pipeline preflight.
Native Production, Continuation, And Assessment
Start with one conversation per producer operation: a chunk-plan generation, one lane's extraction of one chunk, or one lane's normalization pass. Use complete replacement model responses. Normalization may continue to return a reconciliation plan that deterministic code applies to produce the artifact. Per-entity branching is not part of this migration.
Prompt Roles And Correction Guidance (U4 Resolved)
Give each correctable producer an initial native prompt and a domain-specific
correction prompt. Retain existing initial IDs and identify correction IDs
explicitly in the module, for example notarius/dnd.spells.repair. Preflight uses
those same declarations to check known prompt dependencies. Shared correction
wording can live in ordinary asset files; distinct domain/operation templates
supply the task and response expectations without a universal application-aware
correction protocol.
The initial request supplies source material, references, and the task. Enduring domain principles belong in its system instructions. The correction template supplies a new user assignment with the current candidate's actionable findings; PromptKit carries the preceding exchange. Do not rerender the original source or manually replay prior responses. Review initial wording so later correction assignments remain valid. Include additional application-derived context only when needed to interpret the findings. Keep source/reference meanings and normalization candidate handles fixed throughout the conversation.
Use the same declared response schema for each initial/correction pair:
| Role | Response contract |
|---|---|
| LLM chunk planning | Complete replacement chunk plan. |
| Extraction | Complete replacement extracted artifact candidate. |
| Normalization | Complete replacement reconciliation plan, applied afresh to the fixed original candidates. |
| LLM validation | The validator's assessment response; no separate semantic correction conversation by default. |
No correction envelope, patch language, or special repair-result type is needed. PromptKit retains structural validation and repair inside each native execution. Notarius assesses the response returned by that public boundary, independently of how many internal attempts produced it.
Use the same native default profile for an initial/correction pair. Preserve explicit application profile selections and run-wide execution overrides across both requests. An operator may select a different default profile in a correction prompt, subject to explicit request overrides. Validators retain their own profile selection. Pass native selections and overrides without copying resolved settings or adding a Notarius-wide repair-model configuration layer.
The component that detects a defect owns its actionable correction guidance:
- Each rejecting validator supplies guidance describing the defect and the constraints a corrected answer must satisfy. A deterministic validator authors it alongside the check; an LLM validator requests an actionable explanation in its prompt and returns it through the application validation contract.
- Preserve the explicit
CorrectionGuidanceboundary separately from operator diagnostics. Guidance need not prescribe an exact replacement, and validators do not construct the complete correction prompt. - Module interpretation and normalization safety checks supply their own guidance when defects arise before an artifact candidate can be materialized.
- The pipeline collects guidance in configured validator order. The module's domain-specific template renders it into the correction assignment. Neither the pipeline helper nor the template duplicates validators' defect-detection logic. Reason codes and operator messages are not automatically instructions.
Assess the same candidate through every applicable configured validator before
requesting correction. A rejection does not short-circuit later validators;
collect all actionable findings they return. Reassess every revised candidate
through the complete applicable chain, including earlier approvals and the final
permitted response. The existing validator-owned guidance and ordered-chain
semantics remain useful; replace the hard-coded CorrectionRequest message
builder with native template rendering and retain useful ordered aggregation.
Applicability requires usable input: if interpretation cannot construct a typed candidate, its module supplies guidance and artifact validators cannot yet run. Cancellation or fatal execution failures can interrupt assessment. An unavailable validator assessment is a separate outcome, not a detected defect or an instruction to change the artifact. Do not convert provider errors into correction guidance; apply U6's strict/permissive handling when no acceptable result is obtained.
Ownership, response contracts, and default profile behavior are resolved. Refine exact template prose, correction IDs, and necessary operation-specific inputs within implemented module boundaries. The plan records producer/prompt/assessment coverage; final verification remains pending. U5 selects semantic budgets and U6 selects strict-by-default terminal handling and fallbacks.
Execution And Assessment Flow
The application flow is:
- Prepare source context, references, and candidate-handle mappings once. Build
the native initial request. For cacheable work, prepare and check the cache
before execution as described below; execute the retained request on a miss.
Otherwise use ordinary
Engine.Run. Capture an exchange when continuation or debug inspection needs one. - Check the native error and public validation status before treating a response
as usable. PromptKit may return
ValidationFailedwith a nil Go error after exhausting structural repair; that is a terminal failure, not a candidate for application structural repair or cache storage. Decode a usable response and apply module interpretation and safety checks. A module can supply actionable findings before it has a materialized candidate. Assess each candidate through all applicable configured validators and collect their actionable guidance before requesting correction, as selected in U4. - For actionable correction, call
Engine.Coordinatewith the captured exchange and the resolvedmax_additional_turnsas nativeAdditionalTurns. Its callback selects a repair prompt and supplies findings; PromptKit retains the initial exchange and subsequent turns. - Apply the same native-result check, interpretation, and assessment to every delivered revised response, rebuilding from the fixed original inputs. Run the complete applicable validation chain even when a revision produces an unchanged candidate, including validators that previously approved. Assess the final permitted response before declaring exhaustion.
- Stop on acceptance or a terminal condition. If no acceptable result is obtained,
fail in strict mode or produce the module's supported fallback in permissive
mode. Native
stoppedandexhaustedoutcomes do not themselves mean application acceptance. Cancellation stops work without constructing a fallback.
Accumulated history increases context use; U5's allowance bounds additional semantic correction while stopping as soon as the candidate is accepted.
Remove single_response_v1, raw-response ModelCandidate/SemanticCorrection
transport, manual assistant/user replay, and correction-protocol capability
checks. Replace runProducerAttempts' LLM retry loop with native coordination and
application assessment. Adapt NormalizeRetry and producerRetryDirective to
findings and explicit fallback information. Retain useful ordered collection and
deduplication of actionable findings, rendered through a native repair prompt.
Reason codes and operator messages are not automatically model instructions.
Retry Controls And Correction Budget (U5 Resolved)
Delegate transport retries and structural repair to PromptKit. Notarius selects only the allowance for application-directed semantic correction:
| Concern | Control and selected default |
|---|---|
| Temporary provider failures | Native PromptKit transient retries; inherit the upstream default when omitted and preserve native execution overrides. |
| Structurally invalid model output | Native output-contract policy; bundled prompts and application requests inherit PromptKit's upstream structural-repair default. |
| Actionable domain defects | Application max_additional_turns, passed directly to native AdditionalTurns; application default 2. |
max_additional_turns is a nonnegative maximum, excluding the initial producer
response. Resolve producer-binding override, then pipeline override, then the
application default of 2. Omission inherits; explicit 0 disables semantic
correction. It applies to LLM-backed producers. Validators do not receive a
separate semantic continuation budget, and deterministic stages do not acquire
conversation machinery. Keep the field in configuration, adjustable through the
existing application --set mechanism, without another dedicated CLI flag.
For example, a producer binding can override its pipeline allowance:
extract:
module: dnd/spells
max_additional_turns: 2
Two additional turns are an allowance, not mandatory executions. An accepted initial candidate needs none; an accepted first correction needs only one. A second turn is available when actionable defects remain and continuation is appropriate. Assess every delivered response, including the final permitted turn, and stop on acceptance or a terminal condition. PromptKit enforces the turn limit; do not add an application counter or retry loop. Internal transport attempts and structural repairs do not consume this semantic-turn allowance. The default is an application policy choice, not a measured claim about how frequently corrections will be needed.
For structural repair, omit output.repair_attempts from all bundled initial,
correction, and validator definitions. Remove Notarius's pipeline/binding/
validator structured_output_repair_attempts fields and resolution plumbing.
Do not replace them with a hard-coded default, including zero, in request
construction or a native output-contract override. Explicit operator
customization remains available through native PromptKit prompt configuration.
PromptKit v0.24.0 defaults omitted/null budgets to one structural repair for
basic, json, and json_schema, and zero for none. These are upstream
defaults, not Notarius-pinned counts; Notarius deliberately inherits future
changes. Nil request budgets inherit the native default, while explicit zero
disables repair. Notarius still declares response format,
validation mode, and schema; PromptKit owns the mechanics of obtaining a valid
response. At the public boundary, handle a usable result or terminal failure,
including a returned result reporting ValidationFailed with a nil Go error.
Apply that boundary to producers, continuations, and LLM validators; consume the
native terminal status without inspecting or reproducing internal attempts.
Do not restart the operation to extend PromptKit's exhausted repair budget.
Remove blanket outer LLM retries, including repeated LLM validator invocations in
executeValidationChain. Keep ordered candidate assessment and distinguish a
validator execution failure from a domain rejection. Use native OperationTimeout
when that operation scope is needed, without adding an arbitrary default or
replacing broader application deadlines. Cancellation stops work.
Remove generic retries for non-LLM operations as well. Parsing, chunk materialization, deterministic merging, normalization-plan application, and encoding run without an outer attempt loop. Repeating those operations against unchanged inputs ordinarily cannot correct a defect. Return operation errors through normal pipeline failure handling; apply the selected terminal policy to a rejected deterministic candidate instead of regenerating the identical value. Cancellation stops work.
A demonstrated transient I/O recovery need belongs with that specific operation, where repeatability and failure conditions are understood. Do not retain a generic stage retry setting or framework in anticipation of filesystem or future remote- storage failures. After a fix, an operator can rerun the pipeline using the accepted-artifact cache for successful predecessors while cheap deterministic preparation runs again.
Retire producer and validator retries fields entirely, together with the generic
outer attempt loop. Migration guidance maps intent: a producer allowance intended
for semantic correction becomes max_additional_turns; native transport controls
and prompt-owned structural customization cover their own concerns. Remove
non-LLM retry settings without a replacement generic allowance. Do not promise
equivalent total provider-attempt counts. U5 is resolved; U6 selects strict-by-default
handling and module-owned fallbacks. Native producer/assessment integration and
removal of outer retries are implemented; see the plan's completion evidence.
Terminal Outcomes And Module Fallbacks (U6 Resolved)
Use two modes when an operation does not obtain a successful, validated result:
| Mode | Terminal behavior |
|---|---|
| Strict | Require a successful, validated result; otherwise fail the pipeline. |
| Permissive | Produce the module's supported fallback, record a warning, and continue with that artifact. |
The pipeline selects the mode; the module owns the fallback algorithm. Replace
the failure-category policy object and separate fail_run, reject_output, and
warn_continue choices with this selection. An empty artifact is a supported
fallback where the domain permits it; no separate omit-output action is needed.
The application default is strict for every role, including normalization.
Modules do not declare their own defaults. Permissive handling requires an
explicit configuration selection; normalization has no automatic exception.
Use mode: strict or mode: permissive at pipeline level, with an optional
mode override on each configured producer module binding. Resolve the module
binding first, then the pipeline setting, then the application default strict.
Omission inherits; either explicit value overrides the broader setting. A strict
pipeline can therefore permit fallback for one module, and a permissive pipeline
can require strict handling for an essential module. This is binding configuration,
not a default declared by the module implementation. Validators report findings;
the producer's resolved mode governs disposition. Use ordinary configuration and
the existing --set mechanism, without a dedicated mode flag.
Attempt application-directed correction when appropriate within the selected budget before terminal handling. Neither mode extends PromptKit's native retry or repair budgets or adds retries to deterministic operations. Preserve the distinction between a detected defect and an unavailable assessment for correction guidance and diagnostics; neither establishes ordinary acceptance. Cancellation remains cancellation and does not initiate fallback work. If a supported fallback cannot be constructed, fail execution.
Fallbacks must satisfy the structural artifact contract and invariants required by downstream stages. They need not satisfy every ordinary semantic acceptance criterion whose failure led to fallback. Do not rerun the complete ordinary validator chain as a condition of using the fallback or claim normal validation passed. Keep the necessary checks with module fallback construction; no separate configurable fallback-validation framework is needed.
Use these domain-specific fallbacks:
| Module | Fallback direction |
|---|---|
| D&D spell extraction | A valid empty spell artifact, so that the failed chunk contributes no extracted spells. |
| D&D transcript chunking | One chunk containing the entire transcript, preserving source order and references. This is the default fallback; no deterministic slicing machinery or sizing policy is needed for this migration. |
| Normalization | Apply only safe consolidation groups from the final usable response afresh to the fixed originals, retaining everything else. If that response is unusable, use the deterministic merge of the input chunk artifacts. Reuse an already available merged input. |
Keep source grounding, reference resolution, candidate isolation, and reconciliation safety checks. Unknown or overlapping handles remain invalid; partial normalization applies only compatible safe groups and leaves other original candidates separate. Use the final response and fixed original inputs, without recovering an earlier candidate or combining proposals across turns. Concrete module algorithms must make their supported fallback and required invariants explicit. Refine module-specific construction without introducing a generic fallback language. Whole-transcript chunking deliberately favors simple machinery for a rarely expected fallback; it does not promise bounded downstream request size.
Fallbacks may be published and consumed downstream, but they and dependent results affected by their incompleteness are ineligible for accepted-artifact cache lookup or storage. Carry the small application outcome facts needed to skip that reuse; do not add dependency fingerprints or a provenance ledger. Fully validated, unaffected predecessors remain reusable, allowing later runs to retry work that previously needed fallback.
Record why fallback was used and expose that qualification in the ordinary completion report, even when diagnostic verbosity hides individual warnings. Use the same application outcome for reporting and optional debug detail. Keep conversation state in process, outside ordinary artifact/cache payloads; explicitly requested detailed debug is separate. U6 is resolved; module-specific construction remains implementation work.
Admission And Pipeline Workers (U7 Resolved)
Remove Scheduler, scheduledClient, and custom global queue/permit plumbing.
Retire the separate cross-backend ceiling and the default/maximum extraction
worker coupling to concurrency.total_llm. Use one shared pool for ready pipeline
jobs, configured by the positive integer concurrency.workers, default 64.
Omission uses the default; explicit 1 provides serial job execution. Keep this
in application configuration, adjustable through --set, without a dedicated
CLI flag or separate extraction/lane worker settings.
The pool serves both kinds of work:
| Job | Work covered by one worker slot |
|---|---|
| Extraction | One chunk-and-lane extraction, including its validation and semantic correction. |
| Lane completion | A ready lane's merge and normalization, including their validation and correction. |
The limit counts total active jobs across both kinds, with no separate reserved capacity. The existing pipeline coordinator tracks dependencies and makes lane completion ready after that lane's extraction finishes. A worker never holds a slot while waiting for prerequisite jobs. Validators and correction turns execute within the current job instead of acquiring another worker slot. Preserve stable public result ordering regardless of completion order. Input preparation, chunking, and final publication retain their sequential placement around lane execution. This remains execution of the fixed pipeline, not a general workflow scheduler.
Count application jobs, not OS threads or all process goroutines. Do not derive
the default from CPU count or GOMAXPROCS; model-backed jobs often wait for I/O
or native admission. The pool bounds active application state and preparation as
well as model-backed work. 64 supplies practical headroom for backend utilization
while remaining well below the reviewed native default queue capacity; it is not
a benchmark-derived optimum or a guarantee of backend saturation. Operators may
tune it for workload and host resources independently of backend settings.
Use one shared native engine across all roles. Native capacity belongs to the engine and backend ID: profiles selecting the same ID share its pool; different backend IDs or separate engines do not automatically share limits, even for the same endpoint. Endpoint-only profiles and backends without a positive limit remain unrestricted by native backend admission.
Inject no Notarius-owned backend concurrency or queue defaults. Preserve native
operator configuration and inherit upstream defaults rather than deriving backend
settings from the worker limit. In the reviewed PromptKit baseline, omitted/zero
backend concurrency is unlimited and has no queue; with positive concurrency,
omitted queue capacity defaults to 1024. Explicit queue capacity, including zero,
uses native semantics. These are upstream defaults, not values pinned by Notarius.
With an unlimited backend, active jobs can generate concurrent calls up to the
application's workload; the worker default does not imply a separate LLM ceiling.
For limited backends, the reviewed baseline bounds active model attempts by
concurrency_limit and admitted operations by that limit plus queue_capacity.
Ordinary execution admission spans preparation, retries, and repairs; split
PrepareExecution itself reserves no capacity, with admission occurring during
RunPrepared. Active permits are released during retry waits.
A completed continuation turn releases admission before assessment, allowing a
nested validator to use the same backend. Configure native limits at the intended
capacity boundary. Worker counts need not be less than backend concurrency;
native admission can queue waiting operations.
Handle ErrCapacityExceeded through U6's existing strict/permissive policy:
- A producer admission failure fails in strict mode or uses the module's supported fallback in permissive mode.
- A validator admission failure means assessment is unavailable; apply the producing module's resolved mode. Do not turn it into a semantic rejection or a correction instruction.
- Cancellation stops work and does not initiate fallback.
Preserve the error and backend identity in diagnostics. Add no application retry, backoff, or second admission queue. A full native queue rejects admission without calling the model; the large default queue makes this unusual with modest worker counts, but explicit operator limits remain authoritative. Use ordinary fallback reporting and cache exclusions when permissive handling continues.
Aggregate concurrency across independent backends may exceed the former global ceiling. That change is intentional. Bounded pipeline workers are application work controls, not a substitute cross-backend LLM ceiling.
Retire concurrency.stage_workers.extract along with concurrency.total_llm.
Migration guidance selects concurrency.workers independently and configures
each backend natively. Existing explicit extraction-worker values can inform the
new shared limit, but its scope now includes lane completion. Do not automatically
translate a cross-backend ceiling into per-backend limits or preserve worker
inheritance/caps from that field. Provide a native limited-backend example with
usable queue capacity, without requiring operators to override the upstream
queue default. U7 is implemented with focused worker/admission and assembled
coverage recorded in the plan; final acceptance remains pending.
Accepted-Artifact Cache (U8 Resolved)
Replace both the checkpoint subsystem and the separate source-keyed chunk-plan store with one cache around the complete Notarius producer operation. Its value is an accepted answer to the original task, including an accepted chunk plan for LLM-backed chunk generation. This avoids repeating expensive model work after a downstream failure without caching every cheap deterministic pipeline step.
Prepare initial request -> obtain MessageContentHash -> look up <cache_directory>/<hash>.json
Hit: decode the accepted artifact and continue the pipeline
Miss: execute the retained request and complete the Notarius operation
-> interpret, validate, and perform application-directed corrections
-> on final acceptance, store the eligible artifact under the INITIAL hash
Use the upstream message-content hash of ordered message roles and exact text. Include message boundaries and order, excluding model/settings, session/execution IDs, timing, provider cache hints, and other non-content metadata. Add no stage, prompt-ID, library-version, whole-configuration, schema-policy, or implementation fingerprint. Actual input or schema text rendered into messages participates as message content; metadata outside those messages does not.
| Change | Intended reuse behavior |
|---|---|
| Fix a downstream stage | Earlier artifacts remain reusable when their initial messages match. |
| Change initial message text, roles, order, or boundaries | Cache miss. |
| Change a prompt ID, model, or settings without changing messages | Cache hit remains possible. |
| Change deterministic interpretation or validation | Existing accepted artifacts remain reusable; explicit bypass/refresh applies the new policy. |
Call PrepareExecution before model execution and obtain the key from the returned
handle's Details().MessageContentHash. Treat this native value as
opaque and filename-safe. Preserve the prepared snapshot for execution on a miss
using RunPrepared, and discard the handle on a hit. OnPrepared execution hooks
alone cannot short-circuit execution. RenderedPromptHash includes session
identity and cache hints; it is not the cache key. U1's reviewed release supplies
the required API, with no local hash implementation or extra key dimensions.
Keep the initial key throughout corrections. Cache the final Notarius artifact
through its existing codec, not an intermediate response, RunResult, exchange,
or prepared handle. For normalization, save the accepted normalized artifact
after applying the reconciliation plan, not the proposed plan itself. Save only
after the entire Notarius operation succeeds, including domain validation and
any application-directed corrections. Failed, rejected, and validation-incomplete
work is not cacheable. U6's fallbacks and dependent results affected by their
incompleteness are ineligible for both lookup and storage, even if the downstream
operation itself validates successfully. An unrelated accepted predecessor remains
reusable. This requires a small eligibility fact, not a dependency fingerprint graph.
A hit loads the accepted artifact directly, with ordinary decoding/usability checks; absent or unusable entries are misses. It makes no model calls, rebuilds no result/exchange, replays no corrections, and performs no new semantic validation merely because the artifact was cached. It makes no claim of acceptance under changed validation policy and contributes no historical token usage to current execution totals.
For an LLM chunker, cache the final accepted plan after materialization, configured validation, and any application-directed corrections succeed. On a hit, decode the plan and deterministically materialize chunks from the current source. Retain ordinary structural and source-reference usability checks; an unusable plan is a miss. Do not rerun semantic validators merely because the plan was cached. The plan remains a domain result, without the old provenance envelope or a cached copy of the full transcript. Deterministic chunkers execute normally; do not manufacture a PromptKit request or a second hash policy to cache them.
Retire the guarantee of one canonical partition per source regardless of the configured chunking request. Matching initial messages reuse the same accepted plan; changed prompt/reference/input text can produce a miss and a different partition. Model/settings-only changes can still hit under the shared hash policy. Source identity alone no longer selects a plan. This deliberately supersedes the policy in ADR-0005 as the migration lands; preserve the accepted ADR and record a successor decision. Chunk-plan semantics and deterministic materialization remain application-owned; their storage and reuse use the same mechanism as other accepted results.
PromptKit owns everything inside each native request, including structural validation, transport retries, and repairs. Notarius consumes its public response or terminal failure and owns domain acceptance. Internal PromptKit attempts do not determine cache eligibility. Native diagnostic evidence does not transfer that ownership. No cached-response injection, replay API, exchange restoration, or PromptKit-owned write-through cache is requested.
Cache Configuration And Reuse Controls
Use root-level cache_directory for the shared cache root. Omission or an empty
value selects os.UserCacheDir()/notarius: normally ~/.cache/notarius on Linux,
respecting XDG_CACHE_HOME. An explicit directory is the exact root, independent
of output/debug roots and run IDs. Document /var/cache/notarius as the convention
for an explicitly configured system-wide installation. Directory selection does
not enable or disable caching; modes govern cache I/O.
Use cache: auto|bypass|refresh at pipeline level, with optional module-binding
overrides. Resolve an explicitly supplied --cache first, then module-binding
cache, then pipeline cache, then default auto. Omission inherits. An explicit
CLI value governs all cacheable work in the invocation, including chunk generation;
without it, module overrides take precedence over the pipeline setting. Ordinary
configuration and --set can select more specific bindings. No independent
recording switch, separate chunk-cache control, or per-step recomputation selector
is needed.
| Mode | Read existing entries | Store newly accepted results |
|---|---|---|
auto (default) |
Yes | Yes |
bypass |
No | No |
refresh |
No | Yes |
Modes never override the accepted-result eligibility rules. refresh replaces an
entry only after a newly accepted result is obtained; failure or fallback leaves
an existing entry untouched. bypass performs no cache I/O or directory creation.
Rerunning with default auto reuses successful predecessors and executes previously
unsuccessful work without a separate resume request.
Use a small filesystem lookup/write facility and ordinary temporary-file/rename
publication. <cache_directory>/<hash>.json contains the accepted artifact or
chunk plan, with no required manifest, acceptance ledger, historical provenance,
or versioned checkpoint envelope. Lookup does not require original output or
debug directories.
Missing entries are ordinary misses. Unreadable, malformed, or unusable entries are misses with useful diagnostics. Cache write failures warn and preserve the successful application result; do not rerun model work to compensate for a failed cache write. Cache I/O failures do not fail otherwise successful pipeline work. Invalid configuration still follows ordinary configuration-error handling. No expiration policy, garbage collector, or cache migration command is required.
Migration From Both Legacy Caches
Start the unified cache empty and populate it through ordinary execution. Ignore legacy checkpoint and source-keyed chunk-plan records; do not import, reinterpret, rewrite, or automatically delete them, even when their directories are beneath the new root. They do not supply the new message-content keys. Operators may remove obsolete directories when convenient.
Retire cache.checkpoints and cache.chunk_plans, including their separate
directories and checkpoint recording switch. Remove --resume,
--recompute-step, and the separate chunk-cache flag, including the previously
planned --chunk-cache. Release guidance maps those intentions to root-level
cache_directory, pipeline/module cache, and invocation-wide --cache.
Explain automatic reuse, first-run population, and the move from a canonical
source partition to request-content-based plan reuse.
Remove checkpoint identities, dependency fingerprints, invocation-wide barriers, recovery bookkeeping, and the separate canonical-plan lookup/publication policy. Retain domain plan interpretation and materialization without retaining a second cache mechanism. U8 is implemented using U1's selected upstream hash capability; store, producer/plan, and assembled recovery coverage is recorded in the plan.
Inspection And Execution Bookkeeping
Delegate configuration/resource inspection, requirements, concrete preview and validation, and explicit smoke diagnostics to ckconfig/ckprompt through the shared loaders. Remove effective-YAML parsing, the profile-inspector wrapper, and copied effective-parameter types. A configured profile may be partial; use native prepared/result details for the actual execution target, including overrides. Standard commands retain their native reports, support/applicability distinctions, and failure behavior. Smoke status zero alone does not prove a check passed.
For ordinary Run and coordinated turns, use OnPrepared to retain actual
request details for diagnostics. For cacheable operations, retain the already
available PreparedExecution.Details() before discarding or executing the handle.
That snapshot is available on a cache hit even though no execution or OnPrepared
callback occurs. Debug capture must not trigger another preparation or model call.
Native observations provide terminal facts even without a returned RunResult.
OnPrepared is absent on preparation/admission failures and is not a completion
record. Keep synchronous callbacks lightweight, compose capture/accounting
observers, and synchronize shared state where operations overlap. OnPrepared
runs while admission is held; persist retained evidence at normal operation
boundaries.
Use native RunID/ParentRunID with application step/lane/chunk/validator scope.
Remove LLMExecutionSummary, duplicate usage types, conversions, cloning,
independent accumulators, debug completion wrappers, synthetic call IDs, and
adapter profile-recording/forwarding. Native snapshots serve actual-target
inspection; do not reconstruct a durable profile inventory.
Use one native ExecutionGroup per invocation for diagnostic totals. Explicitly
enroll every actual initial, continuation, and nested LLM validator execution once,
with application step/lane/chunk/validator context attached to observations.
Use native aggregation directly, without application-maintained totals or a
hierarchy of overlapping groups. Preserve terminal coverage and unknown-versus-zero
usage. Cache hits create no execution and add no historical usage to current totals.
Concrete observer composition, enrollment plumbing, and context attachment remain
implementation choices within this contract. These facts support diagnostics and
capture, not a mandatory consumer execution ledger. Notarius's application outcome
supplies publication and fallback qualifications for the completion summary;
assessment, failure, and reuse details support optional diagnostics and debug
capture without independently maintained reporting state.
Artifact Delivery And Completion Output
Artifacts are the primary deliverable: publish accepted domain artifacts and
permitted module fallbacks as files in the output directory. Preserve intrinsic
domain content and artifact schemas.
CLI --format controls the completion report, not payload encoding. Consumers
need no run manifest, metadata bundle, persisted receipt, or execution history to
use the artifacts. Retire reporting-only metadata and mandatory manifests instead
of rebuilding them with native values.
Preserve the existing run-specific destination and domain artifact paths:
<output-root>/<run-id>/
lanes/<safe-lane-id>.json
chunk-map.json # when explicitly enabled and available
evidence-context.json # when explicitly enabled and available
Keep the existing lane filename mapping and intrinsic payload schemas. Remove
the mandatory index.json, manifest.json, and reporting-only rejected.json,
warnings.json, and diagnostics.json sidecars. Their relevant diagnostic content
belongs in ordinary diagnostics and optional debug capture; retain intrinsic domain
content in the artifacts themselves. output_directory identifies the absolute
run-specific directory, not the configured output root.
Consumers check process status, optionally decode the small completion report, and read documented artifact paths beneath the published directory. For configurable pipelines, the pipeline configuration identifies expected lanes. Migration guidance replaces receipt/index traversal with those paths, without introducing a replacement discovery manifest or a payload-schema change.
Submit one small structured value through Reporter.Report:
| Field | Meaning |
|---|---|
pipeline |
Selected pipeline identifier. |
artifact_count |
Deliverable artifacts, including reused artifacts; excludes intermediates, cache entries, and debug files. |
output_directory |
Absolute published artifact directory. |
debug_directory |
Optional absolute completed run-specific capture directory; omitted when capture is disabled. |
fallback_count |
Number of producer operations that emitted a fallback; omitted when zero. |
Supply a native text template for the ordinary successful case:
Success: dnd-session produced 6 artifacts in /work/results/run-123
When capture is available, append Debug capture: <run-directory>. Provide the
path rather than mirroring the bundle. JSON/YAML render the same structured
value through native serializers. A plain string would render natively as text
but lose named fields in machine output; the template gives both presentations
without parallel encoders. No report-version, run-ID, token, profile, warning-count,
or validation-history collection is required.
When a completed run used a fallback, qualify the human summary and structured
value with that fact, even if native diagnostic verbosity suppresses individual
warnings. Count each producer operation that actually emitted a fallback once;
do not count rejected attempts, individual validator failures, or downstream
artifacts affected by that fallback. JSON/YAML include fallback_count only when
positive. The native text template can render:
Success with fallback: dnd-session produced 6 artifacts in /work/results/run-123
2 operations used fallback.
Do not present the run as if all operations obtained normally validated results. Detailed reasons belong in optional diagnostics/debug capture, without per-producer histories in the completion report. Quiet mode continues to suppress stdout.
Wire results to stdout and diagnostics to stderr:
| Mode | Stdout | Stderr |
|---|---|---|
| Ordinary | Brief completion report. | Failure explanation when needed. |
--verbose |
Same completion report. | Native information, warnings, and failures. |
--quiet |
Nothing. | Failure explanation when needed. |
--debug |
Completion report with capture path when enabled. | Native diagnostics and default progress. |
Use native mode ordering (last enabled mode wins), explicit progress selection,
and quiet suppression. Quiet still permits failure explanations and does not
prevent rendering errors from failing the invocation. --format json --verbose
keeps stdout valid report JSON. Standard configuration/PromptKit commands retain
their own native values, with no universal Notarius envelope. Artifact-quality
advisories retain their domain classification.
Optional saved reports use shell redirection, without a report-file flag or automatic duplicate persistence:
notarius run dnd-session --input-file transcript.json --format json > run-report.json
Publish artifacts and finish requested capture before submitting a successful
completion report. Use native outcomes: status 0 for command completion, 1 for
operational/cancellation/publication/presentation failure, 2 for invalid invocation
or identifiable invalid configuration, and 3 for native config diff --check
differences. Preserve ckconfig's structured classification and underlying causes;
not every preflight failure is invalid input. A permissive run that completes with
supported fallbacks succeeds with an explicit fallback qualification. Strict-mode
failure, inability to construct a supported fallback, and other terminal pipeline
failures fail the command.
On failure, supply a useful native explanation and nonzero status, without a new
structured error-envelope protocol. --format does not serialize diagnostics.
Retain CommandKit's separation of operation causes from output/diagnostic delivery
failures. If publication succeeded before capture/report delivery failed, identify
the published path and available debug location. Published artifacts remain valid;
do not rerun model work or rewrite execution history. Subprocess callers check
status before decoding a successful report; failed writes may leave partial stdout.
Debug Capture And Optional Diagnostics
Capture the most convenient debug representation, rather than optimizing for conciseness, exhaustive completeness, or strict deduplication. Native snapshots and readable views may repeat evidence. Reuse application decisions and native execution facts instead of maintaining a second ledger or transcript model. Debug formats are diagnostic data, not stable consumer contracts.
Controls And Location
Bind an application boolean to native --debug. Capture requires that boolean
and a nonempty effective directory; a directory alone does not enable capture.
Select --debug-dir PATH over debug.directory over /tmp/notarius-debug.
Resolve nonempty configuration-relative paths against their source and CLI-relative
paths against the invocation directory. Recognize explicit empty values first:
--debug-dir "" or debug.directory: "" disables capture without fallback.
A nonempty CLI override can re-enable capture over an empty configured root.
An unusable nonempty destination fails requested capture rather than selecting
another directory. Remove the configuration-relative ./notarius-debug default.
Capture selection is independent of the final reporting mode. --debug --quiet
captures while suppressing stdout; --quiet --debug selects debug verbosity and
capture. --debug=false disables capture through native binding semantics.
Empty-directory debug retains native verbosity. No separate capture flag or
capture-depth switch is needed.
Lifecycle And Failure Handling
Allocate the run bundle after application configuration and run identity are available, before engine construction and pipeline preflight. Argument/config-load failures may have no bundle. Keep allocation out of shared loaders.
Create <debug-root>/<run-id>/ and update <debug-root>/latest to a relative
symlink to that run at session start. It points to the newly started session even
if that session fails. Do not retarget at completion; concurrent starts use the
last successful link update. Runs without capture leave it untouched.
One application recorder accepts initial context, native evidence, and application decisions, then finalizes once with pipeline/publication/capture outcomes. Persist as operations advance, without holding the whole pipeline's content in memory. Keep debug separate from output/cache and do not automatically delete bundles.
Requested capture is required work: allocation/link failures stop before model work; recording failures surface at the next normal application boundary; finalization failure fails the command. Observer callbacks retain errors for boundary checks because they cannot return errors. An existing pipeline failure remains primary, with capture failures reported additionally and causes retained. Already-written data and published artifacts remain available. Capture failures never cause model retries. Finalize before submitting the success report.
Inventory And Depth
Replace separate summary/trace categories with a compact run overview,
configuration/composition snapshots, parsed source, and per-operation records.
Organize operation directories by meaningful pipeline scope, with native execution
IDs within those operations. Provide a readable run overview and clearly named
structured snapshots and text views. latest is the selected link name; exact
debug filenames, directory nesting, and serialized application fields are explicitly
delegated to implementation. Evaluate them by ease of investigating an operation;
do not freeze them as a public format or introduce another debug framework.
| Capture | Selected contents and depth |
|---|---|
| Run overview | Application version, run/pipeline identity, timing, outcome, publication location, effective invocation selections/overrides, and navigation. Terminal facts come from the same application outcome used by the CLI. |
| Configuration and composition | Native application and PromptKit effective snapshots, normally once each, plus resolved steps, lanes, modules, validators, and references. |
| Parsed source | Generic input document with unit identities and content, independent of the original input file. If parsing fails, retain the input location and available diagnostics. |
| Operation inputs | Actual source/chunk and reference content, incoming artifacts, prompt bindings/variables, and fixed original normalization candidates/handle mappings. Include directly even when duplicated elsewhere. |
| Prepared requests | Native PreparedRun, including settings, messages, and exposed output-contract/schema details when available. |
| Execution events | Native per-execution events in occurrence order, including attempts/retries/repairs, timing, usage availability, and terminal metadata. |
| Results and responses | Native RunResult plus exact RawOutput in a separate editor-readable text file. |
| Conversations | Full Exchange.Details() for every application-visible turn, including rejected turns, plus simple readable role-labeled messages. Repeated prefixes are acceptable. Use prepared messages when no completed exchange exists and label the representation. |
| Interpretation and assessment | Candidate via its codec before acceptance; decode/safety failures; each validator's findings and correction guidance; application disposition, any fallback artifact and reason, and links to LLM validator executions. |
| Normalization and corrections | Proposed reconciliation plan and applied candidate; constructed correction inputs even if the next preparation fails. |
| Deterministic stages and reuse | Chunk plans/materialized chunks, merged/normalized artifacts, and the loaded cached artifact with the current hit decision. No reconstructed history or fabricated execution on reuse. |
Use indented JSON for native structured snapshots and existing domain codecs. Retain selected content in full, with no arbitrary truncation. Enable exchange capture where debug inspection requires it, including single-turn calls; continuation's in-process exchange needs remain independent of durable capture. Keep operation inputs self-contained enough for direct investigation without requiring original inputs, a cache entry, or reconstruction of deltas. Do not archive every original file or serialize the pipeline's entire internal state.
On failure, retain available evidence, where execution stopped, errors, and
application disposition. Preparation/admission failure may have no OnPrepared;
operational failure may have no RunResult. Capture a coordination Undelivered
response as unassessed. Native internal-attempt metadata is useful but intermediate
provider response bodies are not required. No provider wrapper or complete
provider-transcript guarantee is part of this design.
Data Handling And Diagnostic Commands
Operators secure debug data as they secure ordinary application inputs and outputs, for example with filesystem permissions. Debug contains internal data; no separate security/retention model or sanitized-bundle promise is required. Do not directly print/serialize ConfigKit-marked secrets, and preserve native credential omissions through secret-aware inspection. Do not heuristically scan, scrub, or rewrite prompts, source material, model output, or arbitrary strings.
Inspection/diagnostic commands use native reports and do not automatically create pipeline debug bundles. Operators may redirect those reports. Neither debug mode nor capture initiates provider probes. Explicit smoke checks retain native support, applicability, and outcome distinctions.
Decision Status
U1–U10 are resolved. There are no remaining open architectural or dependency choices in this roadmap. Native loading/execution/commands, accepted-result reuse, artifact-first reporting, capture, examples, and documentation are implemented. Observer wiring, private debug representations, and routine API placement are concrete implementation choices, not deferred architecture. The final assembled acceptance/removal/link audit remains pending in the implementation plan. IDs retain continuity with the earlier roadmap.
U1 resolved: The reviewed dependency set
selects PromptKit v0.24.0, ConfigKit v1.4.0, ckconfig v0.10.0, ckprompt v0.11.0,
and CommandKit v0.7.0. Native MessageContentHash satisfies the cache requirement;
typed result handoff and focused inspection enable one application-owned load.
The native resource layout, schema selections, and OutputContractInput
API are implemented. Upstream and focused assembled checks passed as recorded
in the plan; the final full acceptance audit remains pending.
U2 resolved: Ordinary embedded assets, prompt-declared output contracts, native schema-source selection, and removal of duplicate schema/filesystem machinery are selected in the schema design. The native assets and schema selection are implemented with preparation and operator-contract coverage in the plan. U4 selects prompt roles/contracts, and U5 selects native repair delegation and the application correction allowance.
U3 resolved: Typed stage boundaries, composition-root engine construction, and the shared pipeline producer helper with module functions and a pipeline assessment callback are selected in the execution design. The typed operation/helper signatures are implemented within that ownership. U4/U5 select prompt/guidance and retry behavior; U6 selects terminal handling, configuration precedence, and module-owned fallbacks.
U4 resolved: Domain-specific initial/correction prompt pairs, shared response schemas, default profile behavior, validator-owned guidance, and complete applicable-chain assessment are selected in the prompt and guidance design. Native prompt pairs and operation-specific inputs are implemented within that ownership. U5 selects budgets; U6 selects strict/permissive handling.
U5 resolved: max_additional_turns defaults to 2, with binding over
pipeline precedence and explicit zero disabling correction. Native transient and
structural-repair defaults remain upstream-owned; bundled prompts and application
requests do not pin structural repair. The retry-control design
also removes generic non-LLM retries, all producer/validator retries fields,
and the generic outer attempt loop. Any demonstrated transient I/O recovery stays
with its specific operation. U6 selects terminal handling.
U6 resolved: Strict/permissive handling and module-owned fallbacks
replace failure-category policies. Fallbacks satisfy artifact/downstream invariants
without asserting ordinary semantic acceptance; use only the final response and
fixed originals. Fallback and affected downstream work remain outside accepted-
artifact caching, and completion reports disclose fallback use. Strict is the
application default, including normalization; there are no module-declared
defaults. mode resolves module binding, then pipeline, then that default.
Chunking falls back to one whole-transcript chunk; spell extraction uses its empty
artifact, and normalization uses safe final-response groups or merged input.
Module-owned construction and terminal-mode coverage are implemented; see the
plan and D&D guide.
U7 resolved: One pipeline worker pool and native admission
replace the separate worker pools and application LLM scheduler.
concurrency.workers defaults to 64 and bounds ready extraction/lane completion
jobs together, independently of CPU count and backend settings. PromptKit owns
backend capacity with no injected application defaults; admission failures use
U6's mode without extra retries or queues. Retire total_llm and the old extraction
worker field with explicit migration guidance.
U8 resolved: The unified accepted-result cache
includes LLM chunk plans and replaces both legacy caches. Root-level
cache_directory selects a per-user default or an explicit root, with
/var/cache/notarius documented for system-wide installation. Mode precedence is
explicit --cache, module-binding cache, pipeline cache, then auto.
auto reads/writes, bypass does neither, and refresh writes newly accepted
results without reading. Cache I/O is best effort; legacy records remain untouched
and are not imported. U1 supplies the selected native hash API.
U9 resolved: Execution bookkeeping
uses one native execution group per invocation, with all actual initial,
continuation, and LLM validator executions enrolled once and application scope
attached. Completion output adds only
optional positive fallback_count, counting producer operations that emitted
fallbacks. Preserve run-specific domain artifact paths and schemas; retire index,
manifest, and reporting-only sidecars and migrate consumers to documented paths.
Observer wiring and debug filenames/layout are explicitly
implementation choices, not unresolved architecture or stable debug contracts.
Implemented native observations/capture/publication and reporting tests are
recorded in the plan; final acceptance remains pending.
U10 resolved: CommandKit versions, capabilities, commands, loaders/preflight, output, and debug policy are implemented as selected above. CommandKit belongs to the coordinated cutover. CLI compatibility is a non-goal; existing artifact schemas remain domain contracts. Any actual artifact-format change discovered during implementation needs an explicit compatibility decision, not an assumption that native CLI adoption authorizes it.
Implementation Planning
The implementation plan owns execution order, repository touchpoints, stage acceptance criteria, and validation commands. This roadmap defines the complete target and acceptance criteria. Release preparation and publication remain separate work.
Removal Inventory
| Retire or replace | Retained application responsibility |
|---|---|
| Separate application/PromptKit loading, duplicated source interpretation and inspection conversion | One composed ConfigKit load and native result handoff to engine construction and focused inspection. |
StructuredLLMClient, private request/response types, adapter conversions, emptied internal/framework/llm package |
Composition-root engine construction, native module request/interpretation functions, and pipeline assessment/helper. |
AssetRegistry, PromptAssetManifest, ModulePromptFS, synthetic mounts, flattened schema namespaces, readonlyfs, manual fingerprint enumeration |
Ordinary embedded content, native source assembly, and module meaning. |
| Private response-schema registry/wrappers, byte copies, digest and reporting-only identity metadata, duplicate structural validation, old schema constructor options | Native prompt output contracts, schema selections and roots; application interpretation and durable artifact contracts. |
single_response_v1, raw ModelCandidate/SemanticCorrection, manual replay, correction capability checks |
Typed candidates, actionable findings, native exchanges, assessment. |
Generic runProducerAttempts outer loop, repeated validator executions in executeValidationChain, all producer/validator retries fields, module rerun directives, hard-coded CorrectionRequest message builder |
Ordinary deterministic execution, native LLM execution/continuation, ordered validators, validator/module-owned guidance, native domain correction templates, and domain acceptance/rejection/fallback. |
Application structured_output_repair_attempts fields/resolution and explicit bundled output.repair_attempts |
Native upstream structural-repair default, with operator customization through native prompt configuration. |
Failure-category validation_policy fields and separate fail_run/reject_output/warn_continue actions |
Strict/permissive selection, module-owned fallback construction, and explicit fallback outcomes. |
Scheduler, scheduledClient, concurrency.total_llm, concurrency.stage_workers.extract, separate extraction/lane worker pools |
One ready-job pool controlled by concurrency.workers, default 64, and native backend admission. |
| Checkpoint stage/dependency/configuration fingerprints, invocation-wide barriers, manifests, acceptance/provenance envelopes | Small accepted-artifact cache using the upstream message hash, ordinary load checks, explicit controls. |
Separate source-keyed chunk-plan store, canonical-plan provenance/publication policy, nested legacy cache configuration, --resume, --recompute-step, separate chunk-cache flag |
Unified message-content lookup/storage, domain chunk-plan interpretation/materialization, cache_directory, pipeline/module cache, and invocation-wide --cache. |
| Inspection wrappers, copied effective parameters, usage summaries/conversions/accumulators, debug completion wrapper, synthetic IDs, profile recorder | Native facts with application scope for diagnostics. |
| Custom dispatch/help/version/parsing, command-specific loading, copied inspection, legacy aliases/flags, parallel encoders and verbosity handling | Native declarations/loaders, pipeline handlers, domain input checks. |
Mandatory index.json/manifest.json, reporting-only rejected.json/warnings.json/diagnostics.json, extensive receipts and durable execution ledgers |
Preserved domain artifact paths and schemas, one small native completion report with optional fallback_count, ordinary diagnostics, and optional debug evidence. |
| Summary/trace hierarchy, heuristic debug-content scrubbing, custom transcript/deduplication machinery | Convenient run/operation capture and native secret/credential omissions. |
Acceptance And Verification
Use real CommandKit, ConfigKit, and PromptKit in representative assembled flows with a fake at the provider boundary. Keep the suite deterministic, offline, and independent of real credentials. Test domain interpretation/assessment at their package boundaries; use subprocess tests for streams, exit status, alias wiring, and representative machine output. Do not duplicate upstream parser/renderer/ retry tests or freeze complete help text and native debug snapshots.
| Area | Required evidence |
|---|---|
| Configuration/assets | Application version 6 and resources beneath promptkit work without a separate PromptKit resource-version gate or legacy translation; companions/native imports retain main-source context; peer conflicts fail; same-ID replacement, real shared-file paths, prompt-declared contracts, embedded/external roots and source-relative/inline schemas work. Root replacement has no missing-file fallback. Necessary OutputContractInput overrides replace the complete contract. |
| Commands/loaders | One composed ConfigKit load supplies engine construction and native inspection, including explicit bundled resources, source evidence, and focused paths; no duplicate PromptKit load. Required operands, native alias, supplied-value distinctions, references, and run-wide overrides reach intended handlers/roles; help/version/completion need no config; diff selections remain distinct. |
| Preflight/inspection | Same definitions across execution and inspection; missing statically selected resources fail with context; partial profiles alone pass; no provider calls, transcript, or credential requirement for offline preflight. |
| Domain workflow | Representative extraction and normalization preserve source/reference identity, fixed handles, safe plan application, ordered validators, artifact schemas, and stable output ordering; one assessor checks initial/corrected candidates without an extra final pass; deterministic stages and LLM validators need no producer conversation loop. |
| Prompt/guidance contracts | Domain-specific initial/correction pairs use the same response schema and default profile; explicit profile/execution overrides reach both; operator correction-profile defaults work when not overridden; all applicable validators assess the same candidate despite earlier rejection and reassess each delivered revision, including an unchanged candidate; their actionable findings reach the correction template; module interpretation defects and unavailable assessments remain distinct. |
| Retry controls | max_additional_turns resolves binding, pipeline, then application default 2; explicit zero disables correction; acceptance stops early and the final allowed turn is assessed. Bundled declarations/application overrides leave structural repair unspecified, native operator customization remains effective, and tests do not freeze upstream default attempt counts. |
| Non-LLM failure handling | Deterministic errors propagate without generic retries; rejected deterministic candidates reach terminal policy without regeneration; cancellation stops work. Producer/validator retries fields and the generic outer attempt loop are retired. |
| Continuation/failures | Accumulated history, full revalidation and final-turn assessment; native failed validation with a nil Go error reaches terminal handling before domain interpretation or caching in producer, continuation, and validator paths. No blanket retries beyond native exhaustion; validator failure differs from rejection; no earlier-candidate recovery or cross-turn proposal union; cancellation stops without fallback work. |
| Strict/permissive outcomes | mode resolves module binding, pipeline, then strict; omission inherits and either explicit value overrides. Strict is the default for every role, including normalization, without module-specific defaults. Strict failure when no acceptable result is obtained; permissive handling produces module fallback with required artifact/downstream invariants, without rerunning the ordinary full validation chain or claiming normal acceptance. Chunking fallback is one whole-transcript chunk preserving source order/references. Fallback construction failure fails execution; successful fallback use is disclosed in human/machine completion reports. |
| Admission/workers | One shared extraction/lane completion pool obeys concurrency.workers, default 64; explicit 1 runs serially without dependency or nested-validator deadlock. Only ready jobs occupy slots; result ordering remains stable. Backend limits remain independent with no application-injected defaults. Producer/validator admission failures use U6's mode, preserve useful diagnostics, and add no retries; cancellation releases capacity. No separate global LLM ceiling. |
| Accepted-artifact cache | Downstream recovery; initial-message keys; corrected final artifact stored late under the initial key; changed messages miss; settings/ID-only changes can hit; no model/revalidation on hits; unusable entries miss. Failed/rejected/incomplete work is not stored; fallbacks and affected downstream work skip lookup/storage while unaffected accepted predecessors remain reusable; no required history. |
| Chunk-plan reuse | Final accepted plan stored under the initial request hash after materialization/validation/correction; hits rematerialize with structural/source-reference usability checks and no renewed semantic validation. Changed messages can select a new partition; deterministic chunkers run without synthetic cache requests. |
| Cache controls and migration | Root-level directory default/explicit selection; mode precedence CLI, module binding, pipeline, then auto. auto reads/writes; bypass does no I/O or directory creation; refresh only replaces entries after new acceptance. Unusable entries miss; cache I/O failures preserve successful pipeline results. Both legacy stores remain untouched and unread; no conversion or separate canonical-plan mechanism. |
| Output/outcomes | Preserved run-specific domain paths, lane filename mapping, and payload schemas; optional chunk-map/evidence-context publication remains. No index/manifest/reporting sidecars. Native text/JSON/YAML summary includes positive fallback_count and omits zero; count emitting producer operations, not attempts, validator failures, or affected derivatives. Quiet suppression, verbose stderr, debug path, and useful nonzero failures hold. Publication survives later capture/report failure without model reruns. |
| Execution observations | One native invocation group enrolls actual initial/continuation/LLM validator executions once, with application scope; native totals preserve unknown-versus-zero usage without application accumulators or overlapping-group aggregation. Result-less failures remain observable; cache hits add no executions/history; native execution success remains distinct from acceptance. |
| Debug controls/lifecycle | Default/CLI/config/explicit-empty paths and native mode ordering; quiet capture; allocation before engine/preflight; start-time relative latest, no completion retarget; disabled runs leave it untouched; capture failures preserve primary causes and publication facts. |
| Debug evidence | Available input/source/native snapshots, readable text, all visible turns, assessments, normalization and deterministic outputs, cache hits, and failed-run evidence. Cache hits retain prepared details without an execution event, duplicate preparation, or model call; no invented provider transcript. ConfigKit secrets/native credentials omitted, no heuristic content scrubbing. |
| End-to-end migration | Maintained minimal and complete D&D workflows, session routing, reference behavior, raw artifacts, reports, and migration examples work through the new executable. No postponed CommandKit adoption. |
Run focused checks during implementation and the repository-wide checks for the completed cutover under the testing policy:
go test ./...
go vet ./...
go build ./cmd/notarius
Check maintained examples, dependencies, attribution, documentation links, and removal coverage. Live provider smoke and human model-quality evaluation remain optional, outside the default suite. These migration checks add no permanent release gates. Documentation completion does not claim that implementation acceptance checks have passed.
Migration Guidance And Documentation
Provide practical release guidance for operators and subprocess callers:
- New commands/flags, native
runalias,--format, stdout/stderr, outcomes, and saving optional reports by redirection. No legacy CLI compatibility promise. - Application version 6 and native resources beneath
promptkit, embedded-only versus companion examples in appropriate directories, imports, qualified IDs, replacement/inheritance, native local backends, and removed catalogs/version selectors. Explain native schema selections and collection/application roots, external root replacement, and the removal of a separate PromptKit source-version requirement. - Shared
concurrency.workersdefault64, ready-job scope, explicit serial execution, native backend concurrency/queue settings, and removal of the global limit and old extraction-worker field without automatic backend translation. Explain strict/permissive handling of producer and validator admission failures. max_additional_turnsdefault/precedence/zero behavior, and intent-based migration from LLMretries. Explain retirement of all producer/validatorretriesfields and generic non-LLM retries, upstream-owned structural defaults, removal of application repair controls and bundled counts, native operator customization, accumulated-history costs, the universal strict default, pipelinemodewith module-binding overrides, and each module's fallback behavior and downstream guarantees, including the whole-transcript chunking fallback.- Unified accepted-artifact and chunk-plan caching, message-based misses,
settings-only reuse, root-level
cache_directoryand the per-user/system-wide conventions, pipeline/modulecache, and invocation-wide--cacheprecedence. Explain refresh/bypass, best-effort I/O, fallback/affected-work exclusion, removal of both legacy mechanisms without importing/deleting records, and retirement of canonical-source partition reuse. - Artifact delivery through preserved run-specific domain paths and lane filename
mapping, without index/manifest/reporting sidecars or receipt traversal. Document
fallback_countand its omission/counting rules; consumers use the published directory and pipeline-defined lanes. Preserve intrinsic artifact schemas and document any explicitly selected format change. - Debug root/default/empty behavior,
latest, native mode ordering, failure behavior, and ordinary operator responsibility for application data.
Update canonical configuration, CLI, operations, internal overview, LLM, pipeline, modules, state, and affected integration/consumer guides as behavior lands. Remove superseded manifest/receipt contracts then; keep current-behavior documentation accurate until implementation. Update architecture policy for native dependencies and changed scheduler, retry, asset, cache, and reporting boundaries.
Record significant changes in successor ADRs, including asset assembly, validation retries, module retries, PromptKit configuration, the canonical chunk-plan policy, admission, caching, and CLI/reporting ownership. Preserve accepted decision text and update supersession status under documentation policy. This roadmap owns the target and decision rationale; the implementation plan owns execution stages and their status. Git history retains earlier discussions.
Evidence And Reading
These sources support the selected design, not a claim of completed Notarius implementation. Sibling links assume the local repositories are available; the tagged PromptKit links below preserve the reviewed baseline.
Native Library Guidance
- CommandKit consumer guide and CLI conventions: module selection, native aliases/inputs, shared loaders, reporting modes, outcomes, and delivery failures.
- ckconfig and ckprompt: standard suites supplied with application loaders and request builders.
- ConfigKit v1.4.0 release notes: typed result handoff and focused inspection over the original load's provenance and resource contexts.
- ckconfig v0.10.0 and ckprompt v0.11.0: the selected integration compatibility and API changes.
- Shared-loading guidance and the composed CLI example: composed rules, explicit bundled resources, one application result, native engine construction, and focused standard scopes.
- PromptKit consumer guide and capacity behavior: native preparation/execution, continuation, observations, and admission.
Audita Comparison
The comparison was completed on 2026-11-18 against Audita revision
da5b8913d1206a8e8f0a784b2322760385548a60, using PromptKit v0.23.2,
ConfigKit v1.2.1, CommandKit v0.7.0, ckconfig v0.9.0, and ckprompt v0.10.0.
The observations below concern that revision; linked sibling files may evolve.
| Evidence | Lesson carried into this roadmap |
|---|---|
| Engine assembly, source tests, and assembly failure tests | One captured engine, native underlays and main-source-relative companions, whole-record replacement, preserved file contexts, strict peer conflicts, and no silent fallback on operator-source errors. |
| Adapter tests | Ordinary embedded schemas and complete per-request output contracts work with operator prompt replacement. U2 chooses prompt-declared defaults for Notarius; Audita's schema registry and per-call overrides are not requirements to copy. |
| Native coordination bridge, repair flow, conversation tests, and repair tests | Captured exchanges, accumulated history, complete candidate revalidation, final-turn assessment, and canceled/undelivered response handling. |
| Prompt guidance | Durable system instructions, an initial user task, and subsequent repair assignments suit native continuation. |
| Observations and CLI composition | Native execution totals with application scope and explicit nested-validator enrollment; standard CommandKit suites using shared loaders. |
Audita repairs individual transcript corrections in independent branches; Notarius's selected unit is a whole producer operation and typed artifact. Audita's wrappers, response-schema registry, per-proposal policies, and absence of a cache do not define Notarius requirements.
At the reviewed revision, focused LLM, prompt/schema, runner, and proposal-generation tests passed, together with subprocess checks for correction conversations, cancellation during repair, and repair-disabled/output selections using a local fake provider. These historical checks do not establish live-model quality or completion of Notarius's migration acceptance checks.