Replace checkpoint proposals with accepted-artifact caching

This commit is contained in:
2026-11-20 03:10:16 +00:00
parent 98ba7e1f55
commit 305a88e8f4
2 changed files with 255 additions and 154 deletions

View File

@@ -30,15 +30,19 @@ explicit-empty directory behavior are resolved below. Item 6 selects verificatio
migration guidance, and incorporation into the master roadmap; that incorporation
is still pending.
The major architectural decisions are settled. Remaining choices are run-report
fields/versioning and durable record formats (item 4/U9). Function signatures,
bundle filenames other than `latest`, and similar implementation details do not
The cache follow-up in item 4 replaces the earlier checkpoint design with an
accepted-artifact cache keyed by initial message content. The output/report
contract has been reopened around consumer needs; it is not merely waiting for
field names and versions. Function signatures, bundle filenames other than
`latest`, and similar implementation details do not
reopen the selected architecture. Broader PromptKit choices remain with the
master roadmap.
The PromptKit architecture and checkpoint decisions remain in force. This
discussion develops the CommandKit portion of that migration and its interactions
with U1 (dependencies), U9 (records/reporting), and U10 (CommandKit adoption).
The PromptKit ownership direction remains in force. The revised
[cache decision](promptkit-architecture-discussion.md#5-accepted-artifact-cache)
supersedes earlier checkpoint matching and provenance requirements. Master-roadmap
incorporation is pending, including U1 (the upstream content-hash capability),
U8 (cache migration), U9 (records/reporting), and U10 (CommandKit adoption).
## Agreed Proposal
@@ -60,10 +64,10 @@ Documenting the transition replaces a requirement to keep the old CLI working.
Update canonical CLI and consumer documentation when the behavior lands.
Application responsibilities such as producing artifacts, reporting publication
outcomes, and managing checkpoint/debug storage remain meaningful. Their existing
outcomes, and managing cache/debug storage remain meaningful. Their existing
CLI expression is not protected by an implicit compatibility exception. This
guiding rule does not itself choose stored artifact or checkpoint formats; those
remain separate application decisions in the master roadmap.
guiding rule does not itself choose stored artifact formats. The revised cache
decision below selects simple artifact storage rather than checkpoint envelopes.
### Why Adopt Together
@@ -88,7 +92,7 @@ spending migration effort recreating existing CLI conventions.
| Command hosting, standard input/help mechanics, common report formatting and visibility | CommandKit. |
| Standard configuration and PromptKit inspection commands | ckconfig and ckprompt, supplied with application loaders and request builders. |
| Configuration composition and model execution | ConfigKit and PromptKit, using the native boundaries already selected. |
| Pipeline execution and listing, pipeline-specific preflight, artifact publication, checkpoints, and domain outcomes | Notarius application handlers and framework. |
| Pipeline execution and listing, pipeline-specific preflight, artifact publication, accepted-artifact caching, and domain outcomes | Notarius application handlers and framework. |
| Receipt content, debug artifact lifecycle, and application outcome meaning | Notarius, expressed through CommandKit's reporting and outcome conventions. |
Adopt the standard capability suite selected in item 1, including upstream
@@ -104,9 +108,9 @@ meaning and selects report content. CommandKit supplies presentation mechanics.
Two application needs already identified for deliberate treatment are:
- The current `run --json` publishes a receipt after output publication. Item 4
replaces its presentation with one compact application report rendered natively
for humans and machines. Exact fields/versioning remain with U9; reports must
describe the actual publication and domain outcome.
records native reporting mechanics, but the output/report contract has been
reopened around artifact delivery and optional metadata. Its earlier mandatory
run-report proposal is subject to that discussion.
- Item 5 uses native `--debug` for diagnostic verbosity and for capture when an
effective debug directory is available, including an application default.
Capture selection remains independent of the final reporting mode.
@@ -191,8 +195,11 @@ are selected native features; this does not reintroduce legacy Notarius aliases.
- Retain application handlers for pipeline execution and discovery, with native
commands, inputs, and result presentation as selected in items 2–5.
No upstream feature request has been identified as a blocker for this scope.
Evaluate any later demonstrated gap on its actual application need. On roadmap
The standard capability suite needs no identified upstream addition. The later
cache decision does require a native message-content hash, which is not the
existing v0.23.3 rendered-prompt hash. Confirm the supplying API/release before
implementing cache lookup; the version table remains the reviewed baseline, not
a claim that the new hash is already available. On roadmap
reconciliation, incorporate these versions into U1, replace U10's open standard
capability selection with this decision, and incorporate the application wiring
and presentation decisions from items 2–5. Preserve the remaining record choices
@@ -268,7 +275,7 @@ omitted; application validation still decides whether a supplied value is valid.
| Lane selection | Repeatable `--lane ID`; omission selects the normal pipeline lane set. Replace comma-separated `--only`. |
| Output placement | Application-owned `--output-dir PATH`. |
| Chunk-plan caching | `--chunk-cache auto\|bypass\|refresh`. |
| Checkpoint reuse | Keep `--resume` and `--recompute-step ID` with the agreed recovery policy. |
| Accepted-artifact cache | Replace checkpoint-centric controls with cache reuse/bypass/refresh behavior; concrete CLI spelling remains to be selected. `--resume` and `--recompute-step` are migration context, not compatibility requirements. |
| Presentation | Native `--format`, `--quiet`, `--verbose`, `--debug`, and `--progress`. |
| Configuration-valued tuning | Supported application `--set POINTER=JSON` overrides, including worker and semantic-budget settings once their configuration design is settled. |
@@ -500,8 +507,8 @@ preserving supplied operator values. Pipeline modules construct their own domain
requests; share concrete request-building functions when useful without
simulating pipeline execution to populate an inspection command.
Output publication, checkpoints, debug capture, and execution records belong to
the run path. Loaders do not initialize those resources; item 5 selects debug
Output publication, artifact caching, debug capture, and execution records belong
to the run path. Loaders do not initialize those resources; item 5 selects debug
allocation timing. Remove command-specific loading paths, copied configuration
and resource inspection, eager runtime setup, and duplicated preflight
orchestration. No broader application/package reorganization is required by this
@@ -514,10 +521,56 @@ Evidence: [native suite and loader ownership](../../../commandkit/docs/consumers
## 4. Reports, Machine Output, And Outcomes
**Discussion / decision:** Agreed. Use native CommandKit reports for command
results, native outcomes for command completion, and a compact application-owned
run report for artifact discovery and domain status. Distinguish pipeline results,
command completion, and presentation delivery. Implementation is pending.
**Discussion / decision:** Native reporting and outcome mechanics remain selected.
The output/report contract is reopened: consumers primarily need resulting
artifacts, metadata may be optional, failures need useful nonzero outcomes, and
operators/developers use debug capture for detailed investigation. The earlier
run-report and durable-metadata proposals below remain discussion material for
that review, not requirements to preserve or expand. No new report/manifest
schemas have been selected. Finalize the cache decision first; return to the
consumer output/report contract afterward. Implementation is pending.
### Accepted-Artifact Cache And PromptKit Ownership
**Agreed follow-up:** Rename and replace the checkpoint subsystem with the
[accepted-artifact cache](promptkit-architecture-discussion.md#5-accepted-artifact-cache).
That decision owns the matching and storage policy; this section records its
CommandKit and execution-boundary implications.
The cached value is an accepted answer to the original task. Compute the key
from the initial prepared request using a proposed upstream `MessageContentHash`
over ordered message roles and exact text, excluding settings, session/execution
IDs, timestamps, and provider cache hints. Do not hash the entire prepared
request or introduce another Notarius-owned fingerprint policy.
Look up `cache/<hash>.json` before model execution. A hit loads the final accepted
Notarius artifact directly, subject only to ordinary decoding/usability checks.
It does not execute a model request, reconstruct a result/exchange, replay
corrections, or repeat semantic validation merely because the artifact was cached.
On a miss, execute the complete Notarius operation: native PromptKit calls,
domain interpretation and validation, and any application-directed correction
passes. Only after final acceptance save the artifact under the initial hash,
even if correction required several turns. For normalization, cache the accepted
normalized artifact after applying the plan, not the model's proposed plan.
Rejected, failed, and validation-incomplete work is not published to the cache.
PromptKit owns structural response validation, internal retries, repairs, and
other mechanics inside each request execution. Notarius consumes a usable
response or terminal failure at the public boundary and owns domain acceptance.
Internal PromptKit attempts do not determine cache eligibility. Native debug
evidence about those attempts does not transfer their ownership to Notarius.
The upstream requirement is message-content hashing. It does not require native
cached-response injection, replay, exchange restoration, or write-through
response caching. A small Notarius filesystem lookup/write facility stores the
accepted artifact without checkpoint manifests, dependency fingerprints,
acceptance ledgers, or required execution provenance. A cache hit is not a new
model execution or a claim of validation under a changed policy.
Concrete cache flags and legacy-checkpoint handling remain implementation-planning
choices. The output/report discussion must not reintroduce cache metadata solely
to populate a report. The master roadmap still needs this replacement policy.
### One Report Value For Human And Machine Output
@@ -633,10 +686,10 @@ status reflects presentation failure while the published artifacts remain a
fact. Do not rewrite pipeline history as failed execution or automatically rerun
model work because report delivery failed.
Notarius owns stage/lane/chunk progress, domain acceptance, and checkpoint reuse.
Notarius owns stage/lane/chunk progress, domain acceptance, and cache reuse.
PromptKit supplies native execution observations and `ExecutionGroup` totals;
CommandKit presents the selected facts. Explicitly enroll the relevant initial,
continuation, and validator executions without double counting. Checkpoint reuse
continuation, and validator executions without double counting. Cache reuse
does not add historical token usage to current execution totals, and unavailable
usage remains distinguishable from zero.
@@ -742,7 +795,7 @@ context where useful:
| `Observer` | Execution identities, attempt/retry/repair metadata, timing, usage completeness, and terminal metadata. |
| Returned `RunResult` | Available model output and native validation result. |
| `Exchange.Details()` | Explicit inspection of captured conversation history. |
| Application decisions | Domain acceptance/rejection, fallback, validation completeness, and checkpoint reuse. |
| Application decisions | Domain acceptance/rejection, fallback, validation completeness, and cache reuse. |
Compose capture and accounting observers. Remove the debug completion wrapper,
copied execution summaries, synthetic call identities, and duplicate terminal
@@ -757,12 +810,14 @@ Capture only available evidence. Preparation/admission failures may have no
`OnPrepared` notification; observer events have metadata, not response bodies;
operational failures may return no `RunResult`. Do not present the bundle as a
complete provider transcript. Use ordinary `Run` with native hooks, without
forcing split preparation/execution solely for recording.
forcing split preparation/execution solely for recording. The cache's lookup
before execution is a separate reason to use native preparation; `OnPrepared`
notifications alone do not provide cache short-circuiting.
Conversation capture required for continuation remains in-process execution
state. Persisting its history requires enabled debug capture. Normal manifests,
checkpoints, and run reports retain their minimal projections independently of
debug selection.
state. Persisting its history requires enabled debug capture. Cached values are
accepted domain artifacts, not conversation state or provenance records. The
separate output/report contract remains under discussion.
### Capture Lifecycle And Failure Policy
@@ -783,7 +838,7 @@ Treat link-update failures under the requested-capture failure policy below.
One application-owned recorder records initial context, accepts native execution
evidence and application decisions, and finalizes once with the known pipeline,
publication, and capture outcomes. Debug remains separate from output/cache,
never supplies checkpoint reuse, and is not automatically deleted.
never supplies cache reuse, and is not automatically deleted.
Requested capture is required work:
@@ -891,8 +946,9 @@ application outcome used for the CLI report. Reference detailed records where
useful; include convenient summary fields without maintaining a second execution
ledger. Configuration and resolved composition normally need one snapshot each.
Use native execution IDs with application scope to connect records.
Reused work has a restored-payload snapshot and reuse decision linked to original
provenance, not a fabricated model execution.
Reused work has a loaded-artifact snapshot and current cache-hit decision, not
a fabricated model execution. Historical provenance is not required in cache
files or reconstructed solely for debug capture.
Retain enough available context to investigate a run that fails before publishing
output. Do not depend entirely on a successful output manifest or automatically
@@ -905,7 +961,7 @@ competing transcript model or deduplication system to eliminate repeated history
**Resolved:** Use one default capture depth preserving every application-visible
turn, its useful inputs and outputs, and the decisions made about it. An operator
should be able to inspect a failed operation without reconstructing deltas,
finding a checkpoint, or extracting model text from escaped JSON strings.
finding a cache entry, or extracting model text from escaped JSON strings.
Convenience governs both capture and inspection; direct native serialization
and a simple readable rendering of the same values can both be useful.
@@ -988,9 +1044,9 @@ correction inputs even if the next execution fails during preparation; successfu
preparation captures the actual rendered correction in its native messages.
Retain deterministic stage outputs too: chunk plans and materialized chunks,
merged artifacts, and normalized artifacts. For checkpoint reuse, retain the
restored payload and reuse decision linked to original provenance. These records
do not fabricate a current execution or new validation of reused work.
merged artifacts, and normalized artifacts. For cache reuse, retain the loaded
artifact and current hit decision without requiring historical provenance. These
records do not fabricate a current execution or new validation of reused work.
#### Failed Runs And Native Availability
@@ -1046,9 +1102,11 @@ Select a small set of workflows that collectively establish:
flags, supplied-value distinctions, and overrides mapped correctly.
- Reports, diagnostics, and exit outcomes follow item 4, distinguishing domain
rejection, command failure, and presentation failure after publication.
- A downstream failure followed by resume reuses successful predecessors under
the simpler checkpoint policy, with explicit recomputation and consumed-input
changes handled as selected in the master roadmap.
- A downstream failure followed by another invocation reuses accepted artifacts
under the revised cache policy. Final corrected artifacts are stored under the
initial message hash; changed messages miss, settings-only changes can hit,
and hits neither make model calls nor repeat domain validation. Rejected or
validation-incomplete work is never stored as an accepted cache value.
- Debug capture follows item 5's directory selection, reporting-mode independence,
`latest` update timing, useful evidence, and allocation/write/finalization
failure behavior.
@@ -1090,17 +1148,17 @@ old-to-new migration map covering:
- Application and PromptKit configuration changes, source discovery, profiles,
and companion files.
- Native backend admission, independent worker settings, and revised retry controls.
- Checkpoint reuse, explicit recomputation, and the selected treatment of existing
checkpoint files.
- Accepted-artifact cache reuse, explicit refresh/bypass, and the selected
treatment of old checkpoint files.
- Machine reports, exit outcomes, and the distinction between successful command
execution and accepted domain results.
- Debug capture under `/tmp/notarius-debug`, `latest`, and the interaction between
capture and reporting modes.
CLI backward compatibility remains a non-goal; migration guidance does not imply
compatibility shims or preservation of old command behavior. Existing checkpoint
and published-artifact compatibility remain explicit application decisions about
stored work and consumer data, separate from CLI syntax.
compatibility shims or preservation of old command behavior. Legacy checkpoint
handling and published-artifact compatibility remain explicit application
decisions, without assuming that old checkpoint envelopes must survive.
Maintain runnable examples for embedded defaults and operator overrides. Update
canonical CLI, configuration, operations, integration/consumer, and internal
@@ -1121,7 +1179,7 @@ master roadmap. Use this phase sequence:
including reporting and useful debug capture.
4. Migrate remaining roles and remove the adapter, scheduler, retry machinery,
and custom CLI infrastructure.
5. Complete checkpoint migration, durable records, and capture behavior.
5. Complete cache migration, the selected output/report contract, and capture behavior.
6. Verify the assembled application and finish migration documentation.
Each phase delivers a reviewable result with appropriate verification. Tests and
@@ -1130,8 +1188,10 @@ completeness. The phases do not require separate releases or an intermediate
compatibility framework.
Resolve U1's dependency baseline and U10's CommandKit scope from items 1–4. Narrow
U9 to remaining durable-field and accounting choices, and debug implementation
U9 to the reopened consumer output/report contract, and debug implementation
details within item 5's resolved inventory, depth, and directory controls.
Update U1 for the upstream message-content hash and replace U8's checkpoint
identity/provenance plan with the accepted-artifact cache decision.
Reconcile removal inventory and acceptance criteria, including old
inspection-command and machine-output compatibility assumptions. Preserve
unresolved schema handling, role-specific retry/fallback policy, worker defaults,

View File

@@ -2,11 +2,18 @@
## Purpose And Status
The decisions in items 1–6 have been incorporated into the
The original decisions in items 1–6 were incorporated into the
[PromptKit-native architecture migration roadmap](promptkit.md). That roadmap
is the implementation plan and owns current unresolved choices, sequencing,
acceptance criteria, and implementation status.
The subsequent CommandKit discussion replaces item 5's checkpoint proposal with
the accepted-artifact cache below and clarifies PromptKit's execution ownership.
Those updates have not yet been incorporated into the master roadmap. They
supersede its earlier checkpoint matching and provenance requirements. The
[CommandKit notepad](commandkit-migration-discussion.md) records the related CLI,
debug, and output/report discussion; the output/report contract is being revisited.
This document retains the discussion, rationale, and evidence, including item 7's
completed Audita comparison. Item-level questions and follow-ups below record
the discussion as it developed; they are not a second implementation plan.
@@ -57,7 +64,7 @@ resulting change to the adoption target remains to be selected.
| Artifact-family modules | Supply prompt inputs and response requirements, interpret responses, and apply domain rules. |
| Pipeline execution | Connect generation to configured validation, decide acceptance or fallback, and manage dependencies and bounded workers. |
| PromptKit | Prepare and execute requests, maintain conversations, enforce native budgets and admission, and report execution facts. |
| Application records and CLI | Store artifacts and checkpoints, attach pipeline meaning to execution facts, and present results through CommandKit. |
| Application records and CLI | Store artifacts and accepted-artifact cache entries, attach pipeline meaning to execution facts, and present results through CommandKit. |
Engine construction becomes ordinary application wiring using item 2's embedded
defaults and operator sources. Profile selection is expressed through native
@@ -69,8 +76,8 @@ Use native requests, results, exchanges, errors, and observers directly in
model-executing application code. Pure domain models, codecs, source-reference
resolution, and deterministic transformations need no PromptKit dependency.
Retain application interfaces for extracting artifacts, validating typed
candidates, encoding output, and loading checkpoints. Deterministic stages
continue to operate without conversation machinery.
candidates, encoding output, and loading accepted cached artifacts. Deterministic
stages continue to operate without conversation machinery.
### Connect Modules To Pipeline Assessment
@@ -116,8 +123,9 @@ that wrapper stack around native calls.
Attach pipeline scope through native callbacks and observers. Use native
`ExecutionGroup` totals and small application projections for manifests and
optional debug artifacts, following item 6. Checkpoint matching and durable
acceptance provenance remain the application responsibilities selected in item 5.
optional debug artifacts, following item 6 and the reopened output/report
discussion. Item 5 now selects a simple accepted-artifact cache, without required
durable acceptance or execution-provenance records.
Audita demonstrates shared engine assembly, native continuation, and CommandKit
integration. Its `RoleClient`, structured-output interface, and opaque exchange
@@ -225,20 +233,23 @@ exchange wrapper are not part of the selected direction.
The flow and removal/adaptation direction below are agreed. Begin with one
conversation per producer operation and complete replacement model responses.
Retain application assessment, safe-fallback decisions, artifact validation,
and checkpoint eligibility. Detailed response contracts, budget migration, and
and cache eligibility. Detailed response contracts, budget migration, and
the open policy questions below remain to be resolved. No application changes
have been implemented.
### Agreed Flow
For cacheable operations, item 5's initial preparation and lookup precede this
flow. The steps below describe execution on a miss.
1. Prepare the source context, references, and any request-local candidate-handle
mapping once for the producer operation. Run its initial native request with
exchange capture when continuation is enabled. Capture is independent of
detailed debug recording.
2. Inspect native structural validation, decode the response, and perform the
module's deterministic interpretation and safety checks. Run the configured
validation chain on a materialized candidate. A module can identify actionable
defects before a candidate is ready for that chain.
2. Handle the native call's terminal success or failure, decode a usable response,
and perform the module's deterministic interpretation and safety checks. Run
the configured validation chain on a materialized candidate. A module can
identify actionable defects before a candidate is ready for that chain.
3. If correction is needed, call `Engine.Coordinate` with the initial exchange
and a configured `AdditionalTurns` budget. The callback selects a native
repair prompt and supplies actionable findings as variables. PromptKit retains
@@ -295,18 +306,27 @@ plan checks, typed application rules, and candidate isolation. In particular,
normalization must still reject unknown or overlapping candidate handles and
apply only safe groups. A final fallback can retain safe merges while leaving
unsafe groups separate; it must go through configured artifact validation and
terminal policy before it can advance. After native structural-repair exhaustion,
a supported deterministic fallback can be assessed directly instead of first
terminal policy before it can advance. After a native terminal failure covered
by the selected fallback policy, a supported deterministic fallback can be assessed
directly instead of first
restarting the same model operation through a feedback-free module retry. Which
earlier safe candidate, if any, may survive a later failed turn remains an explicit
policy question; do not introduce implicit best-so-far acceptance.
Retain the distinction between a rejection and unavailable validation. Under the
current `warn_continue` policy, incomplete validation can advance a structurally
valid candidate but prevents checkpoint reuse/publication, including for derived
results. Continuation adoption does not itself change that policy, safe-fallback
eligibility, or durable provenance. Keep raw conversation state in process and
out of ordinary manifests/checkpoints; explicit detailed debug remains separate.
valid candidate but prevents accepted-artifact cache reuse/publication, including
for derived results. Continuation adoption does not itself change that policy
or safe-fallback eligibility. Keep raw conversation state in process and
out of cached artifacts; explicit detailed debug remains separate. Item 5
supersedes checkpoint-specific publication and provenance machinery.
PromptKit owns everything within an individual request execution, including
structural validation, internal retries, and repairs. Notarius consumes the
public result or failure and owns domain interpretation, validation, and any
application-directed correction passes. Do not inspect or reproduce internal
PromptKit attempts to decide cache eligibility; cache only the final accepted
Notarius artifact after its complete application process succeeds.
PromptKit should own validation and repair of the private model response.
Notarius's serialized schema validator currently receives the codec-produced
@@ -334,7 +354,7 @@ and the native continuation evidence in item 7.
- Failure handling for non-LLM operations and any justified recovery beyond
native execution budgets.
- Item 2's deferred filesystem and schema assembly choices, item 5's remaining
checkpoint implementation details, and item 6's durable bookkeeping design.
cache implementation details, and the reopened output/report contract.
**Follow-up:** Resolve the remaining details above, then reconcile the adoption
roadmap. Record the changed architecture in a successor to
@@ -342,7 +362,7 @@ roadmap. Record the changed architecture in a successor to
[ADR-0016](../adr/0016-feedback-aware-module-requested-retries.md).
Migration checks should cover accumulated history, fixed handle mappings,
revalidation, final-turn acceptance, native budget exhaustion, safe fallback,
validator failure, cancellation, and checkpoint eligibility. Replace tests of
validator failure, cancellation, and cache eligibility. Replace tests of
the retired message-reconstruction protocol rather than preserving its mechanics.
## 4. Global Scheduling And Admission
@@ -415,103 +435,118 @@ Evidence: [scheduled client](../../internal/framework/llm/scheduled_client.go),
**Follow-up:** Reconcile the adoption roadmap's scheduler-preservation language
and record the changed ownership in an ADR. Update architecture, configuration,
operations, and implementation documentation when the migration lands. Continue
with item 5's execution identity and checkpoint reuse discussion.
with item 5's accepted-artifact cache discussion.
## 5. Execution Identity And Checkpoint Reuse
## 5. Accepted-Artifact Cache
**Discussion / decision:** Direction agreed: resuming preserves successful stage
results for the same logical work and data; explicit recomputation applies
changed execution behavior. Use a simple Notarius-owned matching policy instead
of comprehensive execution fingerprints. No application changes have been
implemented.
**Discussion / decision:** Agreed. Rename and replace the checkpoint subsystem
with a simple cache surrounding the complete Notarius producer operation.
Compute its key early from the initial prepared request's message contents;
store its value late, after Notarius has accepted the final artifact. This
supersedes both comprehensive execution fingerprints and the earlier
stage/input/prompt-ID matching proposal. Implementation is pending.
### Purpose And Matching Rule
### Purpose And Key
Checkpoints preserve a successful producer's result when a dependent stage fails.
After fixing the failed stage, an operator should be able to retry it without
paying to rerun the successful producer. Matching every configuration parameter
or implementation revision would undermine that recovery purpose.
The cached value is an accepted answer to the original task. After a downstream
failure, rerunning the pipeline can reuse that artifact without repeating the
expensive model work that produced it. Cheap deterministic work elsewhere in
the pipeline does not need a general stage-checkpoint framework.
The agreed conceptual key is:
Use a PromptKit-provided `MessageContentHash` (proposed name) over the initial
request's ordered messages, including each message's role and exact text. Exclude
model settings, session IDs, execution IDs, timestamps, and provider cache hints.
Do not add whole-configuration, stage, prompt-ID, library-version, or validation-
policy fingerprints to this matching rule.
Changing the actual initial messages changes the key. Changing a prompt ID,
model, or setting without changing those messages does not. Fixing deterministic
interpretation or validation may therefore require explicit refresh/bypass to
replace a previously accepted artifact; a hit does not claim fresh validation.
This is a concrete upstream hash requirement. The reviewed PromptKit v0.23.3
`RenderedPromptHash` includes session identity and provider cache hints, while
hashing the entire `PreparedRun` would include volatile execution metadata.
Neither is the selected key. Expose the content hash through native preparation
rather than implementing another message-hashing policy in Notarius. Confirm
the supplying upstream API/release during roadmap incorporation; do not claim
the proposed field already exists or silently substitute a broader fingerprint.
### Compute Early, Store After Acceptance
```text
logical stage identity + input-data identity + prompt ID, where applicable
Prepare initial request -> obtain message-content hash -> look up cache/<hash>.json
Hit: decode the accepted Notarius artifact and continue the pipeline
Miss: execute the complete Notarius operation
-> native PromptKit calls as needed
-> domain interpretation, validation, and correction passes
-> on final acceptance, save the artifact under the initial hash
```
Scope the checkpoint to the relevant step/lane/stage and artifact kind so that
shared prompt IDs do not mix unrelated results. These are logical identifiers,
not hashes of implementations or complete configuration. Deterministic stages
need no synthetic prompt identity.
Lookup occurs before model execution. Use native preparation to inspect the
initial request; preserve that prepared request for execution on a miss where
the native API supports it. This is a reason to prepare before deciding whether
to execute, distinct from preparing solely to obtain logging details.
Input data includes the source or chunk being processed, reference material,
and upstream artifacts actually consumed. Matching only the original source
would miss changes to generated dependencies. Content digests can compactly
identify these inputs and check stored payload integrity without requiring an
exact fingerprint of the execution environment.
Retain the initial hash throughout application-directed correction. Save the
final accepted artifact under that hash, not under a later correction request's
hash. A subsequent run can then reuse the accepted answer without replaying the
failed turns that led to it.
| Change | Reuse behavior |
| --- | --- |
| Fix the implementation of a later failed stage | Preserve earlier successful stages with matching work and inputs. |
| Edit prompt text while retaining its ID | Reuse; explicit recomputation applies the edit. |
| Change model, generation settings, or implementation/validation policy | Reuse otherwise compatible completed work; explicit recomputation applies the change. |
| Change the prompt ID or data consumed by the stage | No matching checkpoint under this rule. |
| Explicitly request recomputation | Recompute the selected work and its dependents. |
The value is the final Notarius artifact, serialized through its existing codec.
For normalization, it is the accepted normalized artifact after plan application,
not the model's proposed reconciliation plan. Do not store an intermediate raw
response, `RunResult`, exchange, or prepared-execution handle as the cache value.
The existing controls are `--resume` and `--recompute-step`, documented in the
[CLI reference](../cli.md#run) and
[selective recompute workflow](../operations.md#selective-recompute). Their
current strict compatibility checks still need migration to this agreed policy.
The accepted tradeoff is that a checkpoint may preserve a successful result
produced by code, prompts, or settings that have since improved.
Only publish after the whole Notarius process succeeds, including configured
domain validation and any application-directed retries/correction passes.
Rejected, failed, and validation-incomplete results are not cacheable. Retain
the existing exclusion of derivatives of validation-incomplete work without
restoring checkpoint manifests or fingerprint graphs to implement it.
### Matching, Loading, And Provenance
A hit loads the accepted artifact directly. Ordinary decoding and usability
checks apply; absent or unusable entries are misses. Do not execute a model
request, reconstruct native results/exchanges, replay corrections, or repeat
semantic validation merely because the value came from cache. Do not count
historical model work as current execution.
A matching key selects a checkpoint; it does not make an unreadable or
incompatible artifact usable. Retain completion/acceptance status, integrity
checks, and the ability to decode and consume the stored artifact. Exact schema
hash equality is not required when the artifact remains compatible.
### PromptKit Ownership Boundary
Only accepted, completely validated results are reusable. Rejected, structurally
invalid, or validation-incomplete results and their ineligible derivatives
remain outside reusable state. Acceptance on resume refers to the earlier
execution: do not automatically rerun semantic validators merely because their
configuration changed, or claim the artifact passed the new validation policy.
Preserve original provenance; explicit recomputation applies changed behavior.
PromptKit owns the mechanics within each request execution, including structural
response validation, internal retries, and repairs. Notarius handles a usable
response or terminal failure at the public call boundary, then applies its own
domain interpretation, validation, correction, and acceptance process.
Useful execution metadata can remain available for explanation without becoming
a matching requirement. Neither complete conversation history nor native
prepared-execution handles are needed to locate and reuse a completed artifact.
The separate source-addressed canonical chunk-plan policy remains distinct.
Internal PromptKit attempts are not inputs to the cache design. Native debug
observations may expose evidence about them, but Notarius does not inspect or
reproduce those mechanics to determine cache eligibility. The upstream addition
needed here is the message-content hash; no cached-response injection, replay,
exchange restoration, or PromptKit-owned write-through cache is required.
### Simplification And Ownership
### Storage And Removal
- Remove the broad PromptKit configuration fingerprint from checkpoint matching,
including effective-configuration parsing and whole-catalog enumeration.
- Remove library/catalog versions, model settings, prompt-content hashes, and
component-policy fingerprints as automatic invalidators under this policy.
- Keep straightforward input/dependency identity, successful-stage records,
artifact loading checks, storage, and explicit recomputation in Notarius.
- Keep PromptKit hashes available for native inspection and diagnostics without
making them checkpoint prerequisites.
Use a small filesystem lookup/write facility with `cache/<hash>.json` containing
the accepted artifact. Ordinary temporary-file/rename publication suffices to
avoid exposing partial entries. Cache lookup must not require original output
or debug directories, acceptance ledgers, execution provenance, or a separate
versioned checkpoint envelope. Debug capture can record the current hit and
loaded artifact without requiring historical metadata in the cache file.
Comprehensive upstream execution identity is no longer required for this
design. It does not require a later PromptKit release or a temporary conservative
configuration fingerprint. Item 5 supersedes the earlier proposal to make an
upstream semantic-equality feature part of the migration's checkpoint design.
Retire checkpoint-specific identities, invocation-wide lookup barriers,
stage/dependency fingerprints, manifests, and recovery bookkeeping. Do not
preserve that machinery solely to cache cheap deterministic stages. The existing
standalone canonical chunk-plan cache remains a separate concern.
### Remaining Implementation Work
The old `--resume`/`--recompute-step` conventions are migration context, not a
compatibility requirement. Select straightforward cache reuse/bypass/refresh
controls and legacy-file handling during implementation planning; do not infer
new flag spellings or a migration framework from this decision.
Define the concrete logical stage/prompt identifiers and consumed-input
representation. Revise checkpoint lookup and storage scoping so that an
invocation-wide pipeline/configuration digest cannot still prevent reuse of
successful predecessors. Decide how existing checkpoint records transition to
the simpler format and how incompatible payloads are reported.
**Follow-up:** Reconcile the adoption roadmap and record the changed reuse
contract in an ADR. Update current-behavior documentation when implemented.
Verify recovery after a downstream fix, reuse across prompt/settings changes,
misses after consumed-data changes, explicit recomputation, and rejection of
unusable or unaccepted stored artifacts. Continue with item 6.
**Follow-up:** Incorporate this design and the upstream hash requirement into the
master roadmap. Verify that a downstream failure leaves earlier accepted
artifacts reusable, corrected answers are stored under the initial hash, changed
messages miss, settings-only changes can hit, and a hit makes no model calls or
new semantic-validation claims. Keep output/report contract decisions separate.
## 6. Inspection And Execution Bookkeeping
@@ -529,7 +564,7 @@ No application changes have been implemented.
| Standard configuration/prompt commands, preview and smoke commands, report formatting and visibility | CommandKit and its ckconfig/ckprompt integrations. |
| Provider attempts, transport retries, structural repairs, execution failures, and token observations | PromptKit observations and results; `ExecutionGroup` for selected execution totals. |
| Step/lane/chunk/validator attribution and pipeline progress meaning | Notarius. |
| Acceptance, rejection, fallback, checkpoint reuse, and pipeline outcome | Notarius. |
| Acceptance, rejection, fallback, accepted-artifact reuse, and pipeline outcome | Notarius. |
| Artifact storage, durable provenance, receipts, and optional debug storage/disclosure | Notarius. |
### Native Inspection And Operator Presentation
@@ -592,15 +627,21 @@ artifact, which validators assessed it, whether continuation or a safe fallback
was used, and whether the artifact was generated now or reused. PromptKit cannot
infer that meaning from provider execution alone.
Under item 5, a reused artifact retains its original execution and acceptance
provenance. The current invocation records the reuse decision without counting
the earlier execution's token usage as current work or claiming new validation.
Under the revised item 5, cache files contain accepted artifacts without required
historical execution or acceptance records. A current debug record can identify
reuse and show the loaded artifact without counting earlier token usage as
current work or claiming new validation.
Retain small, deliberate application projections for the
The output/report contract is reopened in the CommandKit discussion; the earlier
manifest and receipt proposals below are not requirements to retain metadata
without a consumer need. They do not impose any additional cache records.
Previously proposed small application projections for the
[published manifest](../integrations/json-output.md) and
[run-result receipt](../integrations/run-result.md), with explicit compatibility
decisions for changed fields. Durable contracts should not become raw upstream
type dumps or require parallel runtime representations of every native result.
decisions for changed fields, remain subject to that review. Durable contracts
should not become raw upstream type dumps or require parallel runtime
representations of every native result.
Keep application debug artifact placement, opt-in capture, retention, and
disclosure handling. Native callbacks supply execution data; they do not own
@@ -627,7 +668,7 @@ remaining upstream request should identify a concrete unmet requirement.
significant boundary changes in successor ADRs. Update current-behavior
documentation when implemented. Verify failure reporting without a result,
continuation/validator accounting, unknown usage, actual-target provenance,
checkpoint reuse accounting, and optional debug capture during implementation.
cache reuse accounting, and optional debug capture during implementation.
## 7. Compare Audita's PromptKit Workflow
@@ -766,7 +807,7 @@ and [prompt guidance](../../../audita/docs/internal/prompts.md).
individual correction within a proposal response. Notarius must deliberately
choose its correction unit rather than assume per-entity branching.
- Audita has no resume/checkpoint/manifest system. It cannot settle Notarius's
reuse identity or persisted validation-provenance requirements.
cache design; item 5 records that application decision independently.
Evidence: [LLM runtime](../../../audita/docs/internal/llm-runtime.md),
[observations](../../../audita/internal/framework/llm/observations.go),
@@ -786,10 +827,10 @@ flow and the remaining repair-prompt, response-contract, and policy details;
the comparison alone did not settle those details.
Item 4 now records agreement to use native admission with bounded application
workers. Item 5 selects simple application-owned recovery matching and removes
the upstream execution-fingerprint prerequisite. Item 6 adopts native execution
observations and CommandKit presentation, retaining application attribution,
domain outcomes, and durable provenance.
workers. The revised item 5 selects an accepted-artifact cache and a focused
upstream message-content hash instead of comprehensive execution fingerprints.
Item 6 adopts native execution observations and CommandKit presentation;
consumer output/report requirements are being revisited separately.
The two requested capabilities already work with v0.23.2; no new PromptKit
feature is needed merely to provide replaceable defaults or domain-directed
continuation. Any remaining upstream request should identify the specific unmet