Rewrite roadmap for PromptKit-native architecture

This commit is contained in:
2026-11-19 15:27:08 +00:00
parent 12eeb6369e
commit d17d261d62
2 changed files with 539 additions and 220 deletions

View File

@@ -1,245 +1,565 @@
# PromptKit v0.23.2 Adoption
# PromptKit-Native Architecture Migration
Open questions about the retained application machinery are tracked in the
[architecture discussion notepad](promptkit-architecture-discussion.md).
Those proposals remain pending until reconciled into this roadmap.
## Purpose And Status
## Purpose
Make Notarius fully PromptKit-native: use upstream configuration, execution,
continuation, admission, inspection, and accounting directly, and retain the
application work of producing, assessing, storing, and reusing typed artifacts.
Historical duplication of PromptKit behavior is not a requirement to preserve it.
Replace Notarius's PromptKit v0.12 integration with a native v0.23.2
integration. Delegate configuration composition, resource inspection, request
preparation, and model execution to PromptKit and ConfigKit. Keep application
machinery where it implements a distinct Notarius decision, such as typed
artifact handling, stage acceptance, cross-backend scheduling, or checkpoint
reuse.
This is the implementation roadmap for the agreed directions in items 1–6 of
[the architecture discussion record](promptkit-architecture-discussion.md).
That document retains the discussion and Audita evidence. This roadmap owns the
selected target, unresolved choices, implementation sequence, and acceptance
criteria. All implementation phases below are pending; rewriting the roadmap
has not changed application behavior.
This is a hard cutover for both the Notarius application configuration and
PromptKit resource definitions. Provide migration guidance and maintained
examples; support only the new formats in the resulting application. This
roadmap defines the PromptKit cutover. Operator interface standardization
belongs to the subsequent CommandKit adoption.
The core migration is a hard cutover for application configuration and PromptKit
resource definitions. Supply migration guidance and maintained examples; support
only the new formats in the resulting application. CommandKit adoption remains a
subsequent delivery phase with its ownership accounted for throughout this plan.
Release preparation and publication are separate work.
## Reviewed Baseline
## Baseline And Dependency Target
Notarius at `65b8e8f0a2d60041c18a29a3c33afc5c689468df` uses application
configuration version 5, Go 1.27, new-path ConfigKit v1.2.0, and old-path
PromptKit v0.12.0. The latter brings in a separate old-path ConfigKit v0.5.0
and two external provider catalogs. Notarius and PromptKit currently share
the YAML v4 release candidate.
At Notarius revision `12eeb6369e40746b2c18c55397a289a1e1b5908d`, the application
uses configuration version 5 and Go 1.27. Its dependencies include old-path
PromptKit v0.12.0 and ConfigKit v0.5.0, new-path ConfigKit v1.2.0, go-yaml
v4.0.0-rc.6, and two external PromptKit provider catalogs.
The target is `go.maximumdirect.net/promptkit@v0.23.2`, published at
`382ccfb14b9630fd8f48c8364dbe2b6159637d71`. The tagged [release
notes][release], [migration guide][migration], [format reference][formats],
and [Go consumer guide][consumer] are the primary upstream references.
The initial dependency target remains `go.maximumdirect.net/promptkit@v0.23.2`
(tagged revision `382ccfb14b9630fd8f48c8364dbe2b6159637d71`), with shared
ConfigKit v1.2.1, stable go-yaml v4, and compatible validation dependencies.
The Go consumer minimum remains 1.27. The tagged [release notes][release],
[migration guide][migration], [format reference][formats], and
[Go consumer guide][consumer] supply the upstream baseline.
The main baseline seams are:
Confirm the final compatible tagged dependency set before implementation.
A demonstrated unmet application need may justify an upstream request and a
later target; the agreed design does not currently require a new PromptKit
feature. In particular, neither a cross-backend global limit nor a comprehensive
execution fingerprint is a prerequisite. Remove obsolete imports and catalog
dependencies when unused; do not retain a compatibility fork or permanent
`replace` directive. Update attribution under the [OSS policy](../policy/oss.md).
- `internal/core/config` resolves `promptkit.profile_file` and
`profile_dir` and interprets the optional `local_backend` object.
`promptkit_profiles.go` builds the operator layers for execution and
inspection; `asset_registry.go` supplies the embedded layer.
- `assets/` and `examples/llm-profiles/` use keyed resource definitions.
Completion requests and diagnostics carry a PromptKit prompt version.
- `internal/framework/llm` does split preparation and execution to retain
debug material; it projects a backend-completed profile report and walks
effective YAML to discover versioned prompts for checkpoint identity.
- The pipeline's producer-attempt loop coordinates domain and stage policy.
The scheduled LLM client enforces the application-wide limit independently
of any PromptKit backend policy.
Audita's reviewed v0.23.2 integration demonstrates native source assembly,
continuation, observations, and CommandKit wiring. Adopt those mechanisms
without copying its `RoleClient`, structured-output interface, or opaque exchange
wrapper. Its per-proposal repair branches and lack of checkpoints do not define
Notarius's artifact and recovery policies.
## Target End State
## Target Architecture
### Dependency And Application Configuration Cutover
### Application Structure And Ownership
Use the vanity PromptKit import path and shared ConfigKit v1.2.1, stable
go-yaml v4, and the resulting compatible validation dependencies. Remove the
old-path imports and catalog dependencies once unused. Use tagged modules
without a compatibility fork or permanent replacement. The consumer minimum
remains Go 1.27. Update attribution for the selected and removed components
under the [OSS policy](../policy/oss.md).
| Part | Responsibility |
| --- | --- |
| Application composition | Load configuration and native sources; construct one shared PromptKit engine for producers, validators, and continuation runs. |
| Artifact-family modules | Prepare domain inputs, select prompt IDs and response requirements, interpret responses, and apply deterministic domain rules. |
| Pipeline execution | Connect generation to configured validation and terminal policy; manage ordered dependencies, typed handoffs, bounded workers, and stable result ordering. |
| PromptKit | Prepare and execute requests; maintain captured exchanges and conversations; enforce native retry, repair, turn, and admission budgets; report execution facts. |
| Application state and records | Store artifacts, apply checkpoint reuse policy, preserve acceptance provenance, and attach pipeline context to execution facts. |
| ConfigKit and CommandKit | ConfigKit owns configuration mechanics; CommandKit and ckconfig/ckprompt own standardized commands and presentation. Notarius supplies loaders, request builders, report content, and outcome mapping. |
Introduce application configuration version 6 as the only supported application
version. Version 6 retains unrelated pipeline structure, discovery, and
resolution behavior. ConfigKit continues to own application loading and
composition under [ADR-0020](../adr/0020-delegate-configuration-mechanics-to-configkit.md).
Retire `profile_file`, `profile_dir`, and `local_backend` in favor of native
PromptKit companion files. There is no separate `promptkit.config_file` selector.
Preserve the ability to override transport retries through the PromptKit engine
option; leave the upstream default in effect when the operator does not
override it.
Use native PromptKit requests, results, exchanges, errors, and observers in
model-executing application code. Remove the private provider-neutral completion
API. Pure domain models, codecs, source-reference resolution, and deterministic
transformations need no PromptKit dependency. Application interfaces continue to
express artifact production, typed validation, output encoding, and state access.
### Companion Files And Embedded Defaults
A small shared application function connects module request construction and
response interpretation to pipeline validation and terminal policy. It calls
native `Run` and `Coordinate` and returns a typed application outcome. It owns no
replacement completion API, conversation protocol, or general retry engine.
Exact signatures and helper placement remain open. The current
`internal/framework/llm` package may shrink to engine assembly and focused helpers
or cease to be a separate package.
Adopt Audita's companion-file convention. Use the directory of the main
Notarius configuration actually selected by ConfigKit, whether explicit or
discovered. Load the optional `prompts.yml` and recursive `.yml` and `.yaml`
files under `prompts/` beside it as one higher-priority operator underlay
using native ConfigKit imports. Do not add a working-directory fallback
for companion discovery. Configurations in the same directory share those
companions. Missing companions are fine; loading errors follow upstream
semantics rather than silently falling back to embedded assets. Native
imports and source-relative resource paths govern any files reached from the
companions. The two companion locations are strict peers; filenames do not
set precedence between conflicting operator records.
Retain the fixed `input -> chunk -> extract -> merge -> normalize -> output`
shape, ordered steps and generated references, typed artifact-family boundaries,
and deterministic public ordering. Keep modules independent of physical output,
cache, and debug roots. Native adoption does not turn the pipeline into a general
workflow engine or redesign durable D&D artifacts.
Use one source assembly for runtime, preflight, and inspection: PromptKit's
bundled OpenRouter underlay, Notarius's embedded underlay, then the operator
underlay when present. Embed all Notarius default prompts and profiles under
`collections.notarius` in the application underlay. Operator companion files
can replace individual defaults by defining complete records with the same
local IDs in that same collection. They can also define new resources and
local backends using the native format.
### Configuration, Sources, And Prompt Assets
The bundled OpenRouter inventory differs from the former external catalogs
and supplies no backend-local concurrency limit. It does not implicitly
select a model for any Notarius request. Migrate deployments that use removed
profiles or backends to intentionally selected native records. The
application-wide LLM ceiling remains independent of backend-local settings.
Introduce application configuration version 6 as the only supported version.
Preserve unrelated discovery, pipeline structure, and resolution behavior.
ConfigKit continues to own loading and composition under
[ADR-0020](../adr/0020-delegate-configuration-mechanics-to-configkit.md).
Retire `promptkit.profile_file`, `profile_dir`, and `local_backend` in favor of
native companion sources; do not introduce a separate `promptkit.config_file`.
Retire `concurrency.total_llm` and revise retry/worker settings as described below.
Their replacement defaults and detailed migration treatment remain open.
### Native Prompt And Profile Resources
Use one source assembly for runtime, preflight, and inspection:
Convert maintained prompts, profiles, examples, and fixtures to identified
lists, with `config:` for source controls and native sibling inheritance
(`extends: {id: parent}`). Under native default composition, a later same-ID
record replaces the earlier record in full. Explain complete replacements,
distinct inherited variants, and the new backend fallback semantics for
profile nulls.
The optional PromptKit source specification header is version 2; it is not the
Notarius application version. Do not translate legacy resource files,
synthesize aliases, or reconstruct old recursive same-ID overlays.
1. PromptKit's bundled OpenRouter underlay.
2. Notarius's embedded native prompts and profiles under `collections.notarius`.
3. An optional, higher-priority operator underlay containing `prompts.yml` and
recursive `.yml`/`.yaml` files under `prompts/`, beside the main application
configuration actually selected by ConfigKit.
Select prompts by ID alone and remove the obsolete PromptKit version selector
and metadata from Notarius's request, debug, and reconciliation identity
plumbing. Review affected diagnostic fields and document deliberate changes.
Keep the current local resource IDs and use their qualified forms in
lookups and references, for example `notarius/dnd.spells` and
`notarius/dnd-extraction`. Update prompt profile references, application
profile selections, and inspection examples to use the qualified IDs.
The `dnd-extraction` profile retains its current model and settings and
continues to reference the unqualified bundled `openrouter` backend. A
profile that extends it belongs in the same collection and uses its local
sibling ID in `extends`. Retain message roles, bytes, order, cache boundaries,
and intended output/repair behavior while changing the declaration format.
Keep module-owned asset selection and embedded filesystem contexts intact.
Verify maintained private response schemas work under upstream's
self-contained-schema rule; this is not a redesign of durable D&D artifacts.
Anchor companion discovery to that selected main source, whether explicit or
discovered. Configurations in the same directory share companions; there is no
working-directory fallback. The two companion locations are strict peers, so
filenames do not resolve conflicting operator records. Missing companions are
fine; loading errors do not silently fall back to embedded defaults. Preserve
native imports and source-relative resource paths.
### Inspection And Reuse Identity
Native declarations become the sole prompt assembly mechanism. Keep the embedded
content tree and module ownership of prompt meaning, response requirements, and
domain interpretation. Remove prompt/fallback-profile registration in
`AssetRegistry`, per-module `PromptAssetManifest` plumbing, `ModulePromptFS`,
shared-name maps, synthetic `sharedassets` mounts, and manual prompt enumeration
for checkpoint fingerprints. Schema-related registry responsibilities remain
subject to the explicit schema decision below.
Use PromptKit's captured configuration reports for effective configuration
and provenance. Use its resource inventory and inspection APIs for exact
identities, presence-aware resource fields, and captured prompt hashes, rather
than deriving an inventory from rendered configuration YAML. Make only the
changes needed to keep the existing `config effective`, `config explain`, and
`config profile` commands usable after the cutover. Report PromptKit's
resolved profile resource rather than rebuilding the former
`effective_model_params` view or designing a combined profile/backend report.
Retain the existing disclosure boundary, including redaction of
credential-shaped provider parameters. Defer new operator commands and report
surfaces to the CommandKit work.
Convert maintained resources, examples, and fixtures to native identified lists,
`config:` source controls, and sibling inheritance (`extends: {id: parent}`).
Same-ID replacement supplies a complete record; distinct inherited variants can
customize an existing resource. Explain native profile-null/backend fallback
semantics. The optional PromptKit source header is version 2, independently of
Notarius's application version. Do not translate legacy resource formats or
synthesize compatibility aliases.
Retain a private, versioned Notarius checkpoint identity based on the engine's
captured state. Use native resource enumeration and prompt hashes, keeping
the effective execution inputs needed to distinguish reuse safely. A diff of
effective values alone cannot detect edits to a referenced message file.
Version the fingerprint for the cutover so obsolete identity inputs cannot
cause a false hit; do not reread mutable sources or expose the digest inputs.
Keep run manifests based on the profiles actually used, not the inventory.
Select prompts by ID alone; remove obsolete PromptKit version selectors and
related request/debug metadata. Retain current local resource IDs and use exact
qualified references, such as `notarius/dnd.spells` and
`notarius/dnd-extraction`. Keep the extraction profile's existing model/settings
and its reference to the bundled unqualified `openrouter` backend. Inherited
application profiles use sibling local IDs in the same collection. The bundled
inventory differs from the old external catalogs and selects no model for an
application request; document intentional replacements for removed catalog IDs.
The bundled backend supplies no concurrency limit. Application/operator limits
need explicit native configuration; application-supplied defaults remain U7.
### Offline Preflight
Preserve prompt prose, message ordering, roles, and cache boundaries where their
meaning carries forward. Review and deliberately revise initial/repair prompts
for accumulated conversation history; preserving every old byte or retry message
is not a migration requirement. Shared content should use native filesystem
contexts and imports rather than reconstructed virtual trees.
Keep `config validate` focused on validating the resolved Notarius application
configuration. With `--pipeline`, also resolve the selected stages and lanes
against the module catalog. For an LLM-backed selection, load the same
effective PromptKit definitions used by `run`, even when the pipeline does
not explicitly select a profile. Let PromptKit validate and capture those
definitions, including companion sources and referenced message files.
Check the prompt and profile IDs known from pipeline resolution, including a
prompt's default profile when the prompt is known and its default is the
effective selection. Do not simulate stages to discover runtime selections.
PromptKit owns private model-response validation and structural repair. Native
schemas resolve against the configured schema root, separately from message-file
contexts, and v0.23.2 requires self-contained schemas loaded during preparation.
A request override replaces the complete output contract; it must retain the
intended schema, validation mode, and repair policy. A prompt hash alone does not
establish schema equality.
Preflight requires no transcript, credential values, or provider calls. Its
success confirms configuration and known resource selection, not that every
concrete request is runnable. A valid PromptKit profile can be partial. Let
PromptKit check the completed target, required credentials, inputs, and schema
during request preparation. Use `InspectRequirements` to explain recognized
dependencies when useful; use `Prepare` or `ValidateRequest` for an actual
request when no-generation validation is needed. Do not construct synthetic
requests or reimplement target resolution for offline preflight.
The custom read-only filesystem, schema root/path selection, module-supplied
request contracts versus prompt-declared contracts, wrapper removal, and useful
schema metadata remain open. An ordinary embedded schema filesystem, as used by
Audita, is a candidate rather than a selected requirement. Durable artifact
schemas and their validators remain distinct from private response schemas.
### Native Execution And Retained Pipeline Policy
### Native Production, Continuation, And Assessment
Use ordinary `Engine.Run` for structured completion and use `OnPrepared`
where opt-in debug capture needs the actual prepared request. The callback
runs after admission, before generation, and is not delivered for failed
preparation or admission. Keep the callback lightweight and use native
observations for safe terminal execution facts, including attempts, repairs,
and available token usage. Where a diagnostic needs totals over selected
executions, use `ExecutionGroup` rather than inventing a second accumulator.
Use `OperationTimeout` where a completion calls for that scope of budget;
do not add a new default deadline or replace application deadlines that also
cover work outside the PromptKit call.
Start with one conversation per producer operation: a chunk-plan generation,
one lane's extraction of one chunk, or one lane's normalization pass. Use
complete replacement model responses. Normalization may continue to return a
reconciliation plan that deterministic code applies to produce the artifact.
Per-entity branching is not part of this migration.
PromptKit owns provider admission, transport retries, and structural repair.
Keep the Notarius cross-backend scheduler because it serves a different
budget from backend or pipeline worker limits. Keep the adapter's typed
decode, actual-run provenance, and provider-neutral error and diagnostic
mapping. PromptKit execution types stay within that integration boundary.
In particular, preserve the distinction between invalid structured
output, provider failure, capacity rejection, and cancellation. A configured
repair override still selects a complete upstream output contract; it must not
accidentally drop the selected schema or validation mode.
The application flow is:
Notarius owns stage attempts, validation and fallback decisions, and
publication of accepted artifacts. Keep its latest-response-only correction
policy using native appended messages, as defined by
[ADR-0014](../adr/0014-feedback-aware-validation-retries.md) and
[ADR-0016](../adr/0016-feedback-aware-module-requested-retries.md).
`Exchange` and `Coordinate` are for workflows that benefit from retained
conversation history. Adopting them for stage retries would change what the
model sees and which work consumes an attempt. They are not needed to execute
the existing retry policy.
1. Prepare source context, references, and candidate-handle mappings once. Build
the native initial request and call `Engine.Run` with exchange capture when
continuation is enabled. Capture is independent of detailed debug recording.
2. Check native structural validation, decode, and apply module interpretation
and safety checks. A module can supply actionable findings before it has a
materialized candidate. Run configured artifact validation on each candidate.
3. For actionable correction, call `Engine.Coordinate` with the captured exchange
and `AdditionalTurns`. Its callback selects a repair prompt and supplies
findings; PromptKit retains the initial exchange and subsequent turns.
4. Interpret every delivered revised response and rebuild from the fixed original
inputs. Run the complete configured validation chain again for changed
candidates, including validators that previously approved. Assess the final
permitted response before declaring exhaustion.
5. Stop on acceptance or a terminal condition. Apply application rejection,
failure, or supported safe-fallback policy after exhaustion. Native `stopped`
and `exhausted` outcomes do not themselves mean application acceptance.
## Migration And Verification
Repair prompts supply a new user task; enduring principles belong in the original
system instructions. Keep source/reference and candidate-handle meanings stable
throughout the conversation. Avoid replaying the original task and constraints
that permit only an initial response. Accumulated history increases context use;
select the semantic turn budget accordingly. Separate repair profiles or schemas
are supported but not yet selected.
Update the maintained minimal and complete D&D examples to distinguish
embedded-only operation from companion configuration. Use separate example
directories if necessary to demonstrate those deployments independently.
Document how to move existing PromptKit sources into the companion layout
or import them from it, declare a local backend, and replace or extend an
embedded resource. Explain qualified IDs, retired catalog IDs, the changed
profile report, and any new checkpoint cold misses. Update the canonical
configuration, CLI, operations, and integration guides when behavior lands.
Record the changed decisions about source precedence and inspection in a
successor to
[ADR-0019](../adr/0019-adopt-native-promptkit-configuration.md) rather than
rewriting its accepted decision text. Release preparation is separate.
Remove `single_response_v1`, raw-response `ModelCandidate`/`SemanticCorrection`
transport, manual assistant/user replay, and correction-protocol capability
checks. Replace `runProducerAttempts`' LLM retry loop with native coordination and
application assessment. Adapt `NormalizeRetry` and `producerRetryDirective` to
findings and explicit fallback information. Retain useful ordered collection and
deduplication of actionable findings, rendered through a native repair prompt.
Reason codes and operator messages are not automatically model instructions.
Verify the dependency graph and offline examples after the cutover. Use
focused behavior tests for the new format and intentional rejection of the
old one, main-relative companion discovery, and native relative imports.
Verify that a companion can replace a qualified embedded prompt or profile
without changing unrelated defaults, and that preflight, inspection, and
runtime use the same effective definitions. Verify that pipeline preflight
catches invalid effective definitions and unknown selected IDs without
requiring inputs or credentials, and does not reject a profile merely for
being partial. Exercise preparation, generation, structural repair, and
semantic retries through the maintained fake-provider integration. Check
that real message or target changes affect checkpoint reuse, and that session
routing, cache controls, diagnostics, and durable outputs keep their existing
meaning. Live provider checks are useful for deployment confidence, not a
requirement of the offline suite.
Transport retries, structural repair, and semantic additional turns remain
separate controls. Preserve the ability to override native transport retries,
using the upstream default when omitted. Remove blanket outer LLM retries after
native transport/structural budgets are exhausted, including repeated LLM
validator invocations in `executeValidationChain`. Keep the ordered validator
chain and aggregation. A validator execution failure is distinct from rejection
of a producer candidate. Existing producer/validator `retries` settings cannot be
mechanically renamed; decide their migration and non-LLM failure handling
explicitly. Use native `OperationTimeout` when that operation scope is needed,
without adding an arbitrary default or replacing broader application deadlines.
## Follow-Up: CommandKit
Keep source grounding, reference resolution, candidate isolation, domain
invariants, and reconciliation safety checks. Unknown or overlapping handles
remain invalid. A supported safe fallback may retain safe merges while leaving
unsafe groups separate; it must pass configured artifact validation and terminal
policy. After structural-repair exhaustion it can be assessed directly without
restarting the same model operation. Whether an earlier safe candidate can
survive a later failed turn remains open; do not silently introduce best-so-far
acceptance.
The planned follow-up adopts `go.maximumdirect.net/commandkit` with its
`ckconfig` and `ckprompt` integrations and replaces Notarius-owned
configuration and PromptKit commands where applicable. That work owns the
standardized CLI and decisions about exposing resource lists, configuration
diffs, request previews, and explicit provider smoke diagnostics. Leave
those commands and the CommandKit dependencies to the follow-up. This
deferral does not change offline pipeline preflight, internal use of native
inspection and observation APIs, or existing run diagnostics.
Preserve rejection versus unavailable/incomplete validation. A structurally
valid candidate may advance under the existing `warn_continue` policy with
explicit incomplete-validation provenance, but it and its affected derivatives
cannot be loaded from or published to reusable checkpoint/cache state. A
completed rejection never advances. Cancellation cannot make an undelivered
response accepted. Keep conversation state in process, outside ordinary
manifests and checkpoints; explicitly requested detailed debug is separate.
### Admission And Pipeline Workers
Remove `Scheduler`, `scheduledClient`, and custom global queue/permit plumbing.
Retire the separate cross-backend ceiling and the default/maximum extraction
worker coupling to `concurrency.total_llm`. Keep bounded extraction workers and
bounded lane continuations with independent defaults to be selected.
Use one shared native engine across all roles. Native capacity belongs to the
engine and backend ID: profiles selecting the same ID share its pool; different
backend IDs or separate engines do not automatically share limits, even for the
same endpoint. Endpoint-only profiles and backends without a positive limit
remain unrestricted by native backend admission.
For limited backends, v0.23.2 bounds active model attempts by `concurrency_limit`
and admitted operations by that limit plus `queue_capacity`. Admission spans
preparation, retries, and repairs; active permits are released during retry waits.
A completed continuation turn releases admission before assessment, allowing a
nested validator to use the same backend. Configure native limits at the intended
capacity boundary and handle `ErrCapacityExceeded` through application policy.
Aggregate concurrency across independent backends may exceed the former global
ceiling. That change is intentional. Bounded pipeline workers are application
work controls, not a substitute cross-backend LLM ceiling. Defaults, retired-field
migration, and role-specific admission-failure dispositions remain open.
### Checkpoint Reuse For Successful Work
Checkpoints support recovery after a downstream failure: fixing a later stage
should not require rerunning earlier successful work. Use this conceptual key:
```text
logical stage identity + input-data identity + prompt ID, where applicable
```
Scope logical identity to the relevant step/lane/stage and artifact kind, without
hashing complete configuration or implementations. Inputs include actual source
or chunk data, reference material, and consumed upstream artifact content.
Deterministic stages need no synthetic prompt ID. Content digests can represent
inputs and verify payload integrity without fingerprinting the execution setup.
| Change | Intended reuse behavior |
| --- | --- |
| Fix a later failed stage | Reuse successful predecessors with matching logical work and inputs. |
| Edit prompt content while keeping its ID | Reuse; explicit recomputation applies the edit. |
| Change model, generation settings, implementation, or validation policy | Reuse otherwise usable completed work; explicit recomputation applies improvements. |
| Change the prompt ID or consumed source/reference/upstream data | No matching checkpoint. |
| Request explicit recomputation | Recompute the selected work and its dependents. |
Keep recording separate from reuse. The existing controls are
[`--resume` and `--recompute-step`](../cli.md#run); migrate their compatibility
behavior to this policy. A matching key does not make an unreadable, corrupted,
or incompatible artifact usable. Retain acceptance/completion status, integrity,
and decode/consumer compatibility checks. Exact schema hash equality is not
required for a compatible artifact.
Only accepted, completely validated work is reusable. Preserve original
execution and acceptance provenance; do not rerun semantic validators solely
because their configuration changed or claim the old artifact passed a new
policy. Retain the exclusion of rejected, structurally invalid, and
validation-incomplete work and its ineligible derivatives. The accepted tradeoff
is reuse of results produced by prompts, settings, or code that have improved.
Remove whole PromptKit configuration, catalog/library versions, model settings,
prompt-content hashes, and component-policy fingerprints as matching criteria.
Native hashes may remain diagnostic metadata. Revise storage/lookup scoping so
an invocation-wide pipeline/configuration digest cannot still block recovery.
There is no need for an upstream semantic-equality feature, persisted conversation,
native prepared-execution handle, or temporary conservative fingerprint.
Keep the separate source-addressed canonical chunk-plan cache policy, including
current validation of materialized chunks. Concrete checkpoint identifiers,
input representation, old-record transition, and incompatible-payload reporting
remain open.
### Inspection, Bookkeeping, And Presentation
Use native configuration/resource inspection rather than effective-YAML parsing,
a profile-inspector wrapper, or copied effective-model-parameter types. A profile
can be partial; its inspection is not the completed execution target. Use actual
prepared/result data for execution provenance, including request overrides, and
record the profiles/targets used rather than the available catalog.
Keep application pipeline preflight for modules, typed artifact compatibility,
reference bindings, and known prompt/profile selections. `config validate` checks
resolved application configuration; with a pipeline selection, load the same
native definitions used by execution, including a known prompt's default profile
when effective. Do not simulate stages to discover dynamic selections. Preflight
needs no transcript, credential values, or provider call. Use native requirement
inspection for explanation and concrete `Prepare`/`ValidateRequest` for completed
target, input, schema, and credential checks when requested. Partial profiles
alone are not preflight failures.
Use ordinary `Run` with `OnPrepared` for needed actual request details and optional
debug capture. Remove forced `PrepareExecution`/`RunPrepared` solely for recording.
`OnPrepared` runs after preparation/admission and before generation; it is absent
on preparation/admission failure and cannot be a terminal record. Native
observations supply terminal facts even when a failed call returns no `RunResult`.
Keep callbacks lightweight and synchronize shared observation state as required.
Use native `RunID`/`ParentRunID` plus application step/lane/chunk/validator scope.
Remove duplicate `LLMExecutionSummary`, usage types, enum conversions, cloning,
independent accumulators, debug completion wrappers, and redundant synthetic
call IDs. Replace adapter profile-recording/forwarding with only the projection
needed at application boundaries. Preserve native failure distinctions while
Notarius chooses domain outcomes and exit-status mapping.
Use `ExecutionGroup` for totals over selected executions. Enroll the initial run,
continuation runs, and nested validator executions explicitly as applicable.
Preserve terminal coverage and usage completeness: unknown usage is not zero.
Avoid counting both attempt and terminal aggregates or adding overlapping groups.
Provider execution success remains separate from domain acceptance.
Retain application records of acceptance, rejection, validation completeness,
fallback, generation versus reuse, and pipeline outcome. Reused artifacts retain
original provenance; the current invocation records reuse without counting old
token usage as current work. Keep small deliberate projections for the
[published manifest](../integrations/json-output.md) and
[run-result receipt](../integrations/run-result.md), with explicit compatibility
decisions. Do not serialize complete upstream runtime types as durable contracts.
Notarius retains debug placement, opt-in capture, retention, and disclosure
handling, including credential redaction. Native callbacks provide the data;
ordinary provenance retains only needed fields, and sensitive response/history
content stays out of ordinary records. Debug is not checkpoint input.
CommandKit with ckconfig/ckprompt is the selected owner of standardized
configuration/resource commands, requirements, validation, preview, opt-in smoke
diagnostics, formatting, visibility, and progress presentation. Notarius supplies
loaders, request builders, report content, pipeline meaning, and outcome mapping.
Keep machine-output receipts deliberate and presentation delivery failures
distinguishable from operation outcomes. Until that follow-up lands, make only
the changes needed to keep existing inspection commands usable through native
APIs; do not build temporary custom commands or a new reporting framework.
## Unresolved Choices
The directions above are selected. The following details are not. Resolve and
record each choice before implementing the work that depends on it; independent
work need not wait for every choice. Candidates here are not implicit defaults.
| ID | Choice still needed | Resolve for |
| --- | --- | --- |
| U1 | Final compatible tagged PromptKit/ConfigKit/dependency set; any specific unmet need requiring an upstream request. v0.23.2 is the initial target, not a restriction on useful upstream work. | Phase 1; revisit if the target or upstream needs change. |
| U2 | Remaining custom read-only filesystem responsibilities; schema root/path and assembly, prompt-declared versus per-request contracts, removable wrappers, and metadata useful for manifests/artifact compatibility. | Phase 2 source and schema assembly. |
| U3 | Concrete module/pipeline signatures and placement of engine assembly, request construction, interpretation, assessment, and focused helpers. | Phase 3 representative workflow, refined during Phase 4. |
| U4 | Initial and repair prompt definitions, response contracts, and profile selection within complete replacement and fixed operation inputs. | Phases 3–4 for each producer role. |
| U5 | Mapping producer/validator retry settings to native controls; names, defaults, compatibility, non-LLM failure handling, and any explicitly justified recovery beyond native exhaustion. | Phases 3–4 and configuration migration. |
| U6 | Detailed terminal/fallback dispositions, especially whether any earlier safe candidate survives a later failed turn. | Phases 3–4; no implicit best-so-far acceptance. |
| U7 | Independent worker defaults, native backend/queue defaults where supplied, retired `total_llm` migration, and admission-failure handling by role. | Phase 4 shared-engine cutover. |
| U8 | Logical checkpoint/prompt identifiers, consumed-input representation, storage scoping, old-record transition, and load-compatibility reporting. | Phase 5 recovery migration. |
| U9 | Minimal durable fields and their compatibility treatment; observer/group enrollment, application scope, and debug write/disclosure handling. | Design with Phase 3, complete in Phase 5. |
| U10 | CommandKit/ckconfig/ckprompt versions, command exposure and compatibility, and application wiring for the follow-up. | Phase 7; do not recreate its machinery earlier. |
## Implementation Sequence
Each phase is pending. Phases describe reviewable outcomes and dependencies,
not separate releases. The supported core result is the hard cutover; any
intermediate migration scaffolding must retire before that result is complete.
Update affected canonical documentation as behavior lands. Record significant
boundary decisions in successor ADRs without rewriting accepted decision text.
### Phase 1: Establish The Migration Baseline
- Confirm U1 against the tagged APIs and dependency graph. Carry forward the
Audita evidence without treating its wrappers as requirements.
- Map the changed architecture and ADRs, and identify concrete decisions needed
for the next phase. Keep unresolved entries visible until actually settled.
- Identify the production callers and old contracts covered by the removal
inventory below; use that inventory to track complete cutover coverage.
**Completion evidence:** selected dependency baseline and decision/implementation
scope are recorded; unresolved choices have a clear dependent phase. No new
upstream feature is assumed necessary without an identified requirement.
### Phase 2: Assemble Native Sources And A Shared Engine
- Resolve U2, update dependencies, and implement version 6 and companion-source
composition. Convert embedded resources and establish the schema loading path.
- Construct the shared native engine and use the same assembly for execution,
preflight, and inspection. Remove duplicate prompt assembly as callers migrate.
- Add maintained embedded-only and operator-override examples with the new format.
**Completion evidence:** offline preparation proves resource selection, shared
content/schema resolution, source precedence, and operator replacement. Invalid
sources fail rather than silently selecting defaults. Preflight needs no provider
credentials or synthetic stage execution.
### Phase 3: Prove Native Production And Assessment
- Resolve U3 and the relevant U4–U6/U9 choices for a representative extraction
workflow. Connect native generation, domain interpretation, configured
validation, and continuation through the small application function.
- Exercise a configured LLM validator on the same engine. Attach native observers,
application scope, and selected execution-group membership from the start.
- Exercise normalization's fixed candidate handles and fresh plan application,
including the selected terminal/fallback policy.
**Completion evidence:** fake-provider workflows demonstrate accumulated history,
full revalidation, final-turn acceptance, correct exhaustion/failure handling,
and typed application outcomes without the private completion/correction API.
### Phase 4: Migrate All Roles And Retire Runtime Machinery
- Apply the established flow to every LLM-backed chunker, extractor, normalizer,
and validator. Preserve deterministic stage paths and ordered validation.
- Complete U4–U7 across roles. Migrate retry controls, remove the global scheduler,
and give bounded pipeline workers independent configuration.
- Remove the adapter, copied completion contracts, correction-protocol checks,
outer LLM retry loops, and redundant asset registration once unused.
**Completion evidence:** all production roles share native execution and admission;
no application path still depends on the retired completion API, global ceiling,
or latest-response-only correction. Native failures and cancellation settle
through the selected role policy. Tests for obsolete machinery retire with it.
### Phase 5: Simplify Recovery And Execution Records
- Resolve U8–U9. Implement logical-work/input/prompt checkpoint matching and revise
storage scoping so unrelated configuration changes cannot defeat recovery.
- Implement and document old-checkpoint handling, artifact usability checks,
original acceptance provenance, and explicit recomputation behavior.
- Complete native inspection, observation/group accounting, minimal durable
projections, and optional debug capture; remove the duplicate bookkeeping.
**Completion evidence:** a downstream-failure/fix/resume scenario reuses successful
predecessors; settings changes do not spuriously invalidate them, consumed-data
changes do, and reused work does not count as current execution. Published records
follow the selected compatibility policy and preserve failure/usage distinctions.
### Phase 6: Verify And Document The Core Cutover
- Complete the acceptance checks below and the removal inventory. Verify the
dependency graph contains no unused old-path PromptKit/ConfigKit or catalogs.
- Finish migration guidance, maintained examples, attribution, successor ADRs,
and canonical documentation updates. Reconcile affected future-work notes.
- Keep existing inspection commands usable with native reports. Document that
the standardized CommandKit command surface remains the subsequent phase.
**Completion evidence:** the core native architecture, configuration migration,
checkpoint policy, diagnostics, and maintained offline workflows are verified;
remaining work is explicitly the CommandKit follow-up or separately identified
future work, not hidden adapter or retry compatibility machinery.
### Phase 7: Adopt CommandKit Presentation
- Resolve U10 and adopt CommandKit with ckconfig/ckprompt. Supply application and
engine loaders plus request builders for the selected diagnostic commands.
- Replace applicable Notarius-owned configuration/PromptKit commands and standard
presentation while preserving deliberate machine-output contracts and domain
outcome mapping. Keep provider smoke diagnostics explicitly opt-in.
**Completion evidence:** standard commands use upstream integrations and the same
native source assembly as execution; application preflight and published receipts
retain their intended meaning. This phase completes the presentation target and
is not a prerequisite for completing the core PromptKit cutover.
## Removal Inventory
| Retire or replace | Retained application responsibility |
| --- | --- |
| `StructuredLLMClient`, private structured-completion request/response types, adapter conversions | Native request construction and typed domain interpretation. |
| Prompt/profile `AssetRegistry` plumbing, `PromptAssetManifest`, `ModulePromptFS`, synthetic shared mounts, manual fingerprint enumeration | Embedded content and module semantics; schema responsibilities resolved under U2. |
| `single_response_v1`, raw `ModelCandidate`/`SemanticCorrection`, manual replay, correction capability checks | Typed candidates, actionable findings, native exchanges, and assessment. |
| LLM retry portions of `runProducerAttempts` and `executeValidationChain`, module rerun directives | Ordered validators, acceptance/rejection/fallback policy, and useful provenance. |
| `Scheduler`, `scheduledClient`, `concurrency.total_llm`, worker coupling | Bounded pipeline work and native per-backend admission. |
| Broad configuration/prompt/catalog/implementation checkpoint fingerprints and invocation-wide lookup barriers | Logical work, consumed-input identity, load checks, and explicit recomputation. |
| Inspection wrappers, copied effective parameters, `LLMExecutionSummary` and usage conversions/accumulators, debug completion wrapper, redundant call IDs, adapter profile recorder | Native facts with application scope and selected durable projections. |
| Custom standard inspection commands/presentation in the CommandKit phase | Application loaders, request builders, report content, and outcome mapping. |
## Acceptance And Verification
Use focused behavioral tests with real deterministic collaborators, a native
engine, and injected clients or a fake HTTP provider. Test domain interpretation
and assessment directly; pipeline tests can use fake stages. Replace tests that
preserve removed mechanics rather than reproducing a private LLM interface for
testing. Default tests remain offline and independent of real credentials.
| Area | Required evidence |
| --- | --- |
| Configuration and assets | Version 6/new resources work; legacy formats are rejected intentionally; main-relative companions and native imports work independently of working directory; peer conflicts fail; same-ID replacement affects only the intended default; ordering/cache controls and selected schema contracts survive assembly. |
| Preflight and inspection | Runtime, inspection, and preflight share effective definitions; unknown statically selected IDs and invalid definitions fail offline; partial profiles alone pass; completed target checks occur on concrete requests; actual-run overrides appear in provenance. |
| Native artifact workflow | Representative extraction and normalization produce typed candidates, invoke ordered validators, and preserve source/reference identity, safe plan application, artifact-schema checks, and stable output ordering. |
| Continuation and failures | Multiple turns accumulate history with fixed input/handle meanings; every revised candidate and final allowed response is assessed; native transport/structural exhaustion is not silently retried outside PromptKit; validator failure differs from rejection; selected safe fallback and cancellation policy hold. |
| Admission and workers | All roles share the selected backend pool; independent backends remain independent; pipeline work stays bounded; queue exhaustion, nested validators, and cancellation settle without leaked capacity or a retained global ceiling. |
| Checkpoints | Successful predecessors survive downstream fixes and prompt/model/validation-setting changes; changed prompt IDs or consumed source/reference/upstream data miss; recompute forces selected work/dependents; corrupt/incompatible or unaccepted/incomplete artifacts and ineligible derivatives are unusable; compatible schema changes do not require exact hash equality. |
| Reuse provenance | Original acceptance and execution provenance survive resume without new semantic validation claims or historical token usage counted as current; recording remains separate from reuse; canonical chunk-plan caching retains its distinct policy. |
| Execution records | Initial/continuation/nested validator accounting preserves terminal coverage and unknown-versus-zero usage without double counting; failures without a result remain observable; native success is distinct from acceptance; debug is opt-in and ordinary records exclude sensitive response/history content. |
| Published behavior | Maintained minimal and complete D&D workflows, session routing, artifact output, manifest/receipt compatibility, and selected migration diagnostics work through the assembled CLI. CommandKit-phase tests cover its own command/presentation changes. |
Run focused checks while implementing and the repository-wide commands for the
completed application cutover, following the [testing policy](../policy/testing.md):
```sh
go test ./...
go vet ./...
go build ./cmd/notarius
```
Check maintained examples, dependency removal, links, and attribution alongside
those commands. Live provider checks and human model-quality evaluation can
inform deployment and prompt refinement; they are not requirements of the
deterministic offline suite. Planning/documentation completion does not claim
that these implementation acceptance checks have already passed.
## Migration Guidance And Documentation
Maintain examples that distinguish embedded-only operation from companion
configuration, using separate directories when needed. Explain moving/importing
old sources, qualified IDs, complete replacement versus inheritance, native local
backends, removed catalogs and prompt versions, and the separate application and
PromptKit source versions. Document worker/global-limit changes, retry-budget
migration, accumulated conversation costs, and the selected failure policy.
Explain successful-work checkpoint reuse, explicit recomputation to apply prompt
or setting improvements, consumed-data misses, original validation provenance,
and the chosen old-record transition. Document changed inspection and durable
fields deliberately rather than promising incidental old adapter output.
Update the canonical [configuration](../config.md), [CLI](../cli.md),
[operations](../operations.md), [internal overview](../internal/overview.md),
[LLM](../internal/llm.md), [pipeline](../internal/pipeline.md),
[modules](../internal/modules.md), [state](../internal/state.md), and affected
integration/consumer guides when their behavior changes. Update the
[architecture policy](../policy/architecture.md) to make native PromptKit an
explicit execution dependency and remove the superseded scheduler, retry,
asset-registration, and fingerprint constraints.
Record significant ownership and contract changes in successor ADRs, including
those affecting [asset assembly](../adr/0011-centralize-llm-assets.md),
[validation retries](../adr/0014-feedback-aware-validation-retries.md),
[module-requested retries](../adr/0016-feedback-aware-module-requested-retries.md),
and [PromptKit configuration](../adr/0019-adopt-native-promptkit-configuration.md).
Also record the changed admission and checkpoint matching decisions. Preserve
accepted decision text and update supersession status as appropriate under the
[documentation policy](../policy/documentation.md#architecture-decision-records).
The discussion record supplies rationale and evidence; this roadmap tracks
implementation status until the corresponding behavior lands.
[release]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.2/docs/releases/v0.23.2.md
[migration]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.23.2/docs/consumers/configuration-migration.md