Replace checkpoint proposals with accepted-artifact caching

This commit is contained in:
2026-11-20 03:10:16 +00:00
parent 98ba7e1f55
commit 305a88e8f4
2 changed files with 255 additions and 154 deletions

View File

@@ -30,15 +30,19 @@ explicit-empty directory behavior are resolved below. Item 6 selects verificatio
migration guidance, and incorporation into the master roadmap; that incorporation
is still pending.
The major architectural decisions are settled. Remaining choices are run-report
fields/versioning and durable record formats (item 4/U9). Function signatures,
bundle filenames other than `latest`, and similar implementation details do not
The cache follow-up in item 4 replaces the earlier checkpoint design with an
accepted-artifact cache keyed by initial message content. The output/report
contract has been reopened around consumer needs; it is not merely waiting for
field names and versions. Function signatures, bundle filenames other than
`latest`, and similar implementation details do not
reopen the selected architecture. Broader PromptKit choices remain with the
master roadmap.
The PromptKit architecture and checkpoint decisions remain in force. This
discussion develops the CommandKit portion of that migration and its interactions
with U1 (dependencies), U9 (records/reporting), and U10 (CommandKit adoption).
The PromptKit ownership direction remains in force. The revised
[cache decision](promptkit-architecture-discussion.md#5-accepted-artifact-cache)
supersedes earlier checkpoint matching and provenance requirements. Master-roadmap
incorporation is pending, including U1 (the upstream content-hash capability),
U8 (cache migration), U9 (records/reporting), and U10 (CommandKit adoption).
## Agreed Proposal
@@ -60,10 +64,10 @@ Documenting the transition replaces a requirement to keep the old CLI working.
Update canonical CLI and consumer documentation when the behavior lands.
Application responsibilities such as producing artifacts, reporting publication
outcomes, and managing checkpoint/debug storage remain meaningful. Their existing
outcomes, and managing cache/debug storage remain meaningful. Their existing
CLI expression is not protected by an implicit compatibility exception. This
guiding rule does not itself choose stored artifact or checkpoint formats; those
remain separate application decisions in the master roadmap.
guiding rule does not itself choose stored artifact formats. The revised cache
decision below selects simple artifact storage rather than checkpoint envelopes.
### Why Adopt Together
@@ -88,7 +92,7 @@ spending migration effort recreating existing CLI conventions.
| Command hosting, standard input/help mechanics, common report formatting and visibility | CommandKit. |
| Standard configuration and PromptKit inspection commands | ckconfig and ckprompt, supplied with application loaders and request builders. |
| Configuration composition and model execution | ConfigKit and PromptKit, using the native boundaries already selected. |
| Pipeline execution and listing, pipeline-specific preflight, artifact publication, checkpoints, and domain outcomes | Notarius application handlers and framework. |
| Pipeline execution and listing, pipeline-specific preflight, artifact publication, accepted-artifact caching, and domain outcomes | Notarius application handlers and framework. |
| Receipt content, debug artifact lifecycle, and application outcome meaning | Notarius, expressed through CommandKit's reporting and outcome conventions. |
Adopt the standard capability suite selected in item 1, including upstream
@@ -104,9 +108,9 @@ meaning and selects report content. CommandKit supplies presentation mechanics.
Two application needs already identified for deliberate treatment are:
- The current `run --json` publishes a receipt after output publication. Item 4
replaces its presentation with one compact application report rendered natively
for humans and machines. Exact fields/versioning remain with U9; reports must
describe the actual publication and domain outcome.
records native reporting mechanics, but the output/report contract has been
reopened around artifact delivery and optional metadata. Its earlier mandatory
run-report proposal is subject to that discussion.
- Item 5 uses native `--debug` for diagnostic verbosity and for capture when an
effective debug directory is available, including an application default.
Capture selection remains independent of the final reporting mode.
@@ -191,8 +195,11 @@ are selected native features; this does not reintroduce legacy Notarius aliases.
- Retain application handlers for pipeline execution and discovery, with native
commands, inputs, and result presentation as selected in items 2–5.
No upstream feature request has been identified as a blocker for this scope.
Evaluate any later demonstrated gap on its actual application need. On roadmap
The standard capability suite needs no identified upstream addition. The later
cache decision does require a native message-content hash, which is not the
existing v0.23.3 rendered-prompt hash. Confirm the supplying API/release before
implementing cache lookup; the version table remains the reviewed baseline, not
a claim that the new hash is already available. On roadmap
reconciliation, incorporate these versions into U1, replace U10's open standard
capability selection with this decision, and incorporate the application wiring
and presentation decisions from items 2–5. Preserve the remaining record choices
@@ -268,7 +275,7 @@ omitted; application validation still decides whether a supplied value is valid.
| Lane selection | Repeatable `--lane ID`; omission selects the normal pipeline lane set. Replace comma-separated `--only`. |
| Output placement | Application-owned `--output-dir PATH`. |
| Chunk-plan caching | `--chunk-cache auto\|bypass\|refresh`. |
| Checkpoint reuse | Keep `--resume` and `--recompute-step ID` with the agreed recovery policy. |
| Accepted-artifact cache | Replace checkpoint-centric controls with cache reuse/bypass/refresh behavior; concrete CLI spelling remains to be selected. `--resume` and `--recompute-step` are migration context, not compatibility requirements. |
| Presentation | Native `--format`, `--quiet`, `--verbose`, `--debug`, and `--progress`. |
| Configuration-valued tuning | Supported application `--set POINTER=JSON` overrides, including worker and semantic-budget settings once their configuration design is settled. |
@@ -500,8 +507,8 @@ preserving supplied operator values. Pipeline modules construct their own domain
requests; share concrete request-building functions when useful without
simulating pipeline execution to populate an inspection command.
Output publication, checkpoints, debug capture, and execution records belong to
the run path. Loaders do not initialize those resources; item 5 selects debug
Output publication, artifact caching, debug capture, and execution records belong
to the run path. Loaders do not initialize those resources; item 5 selects debug
allocation timing. Remove command-specific loading paths, copied configuration
and resource inspection, eager runtime setup, and duplicated preflight
orchestration. No broader application/package reorganization is required by this
@@ -514,10 +521,56 @@ Evidence: [native suite and loader ownership](../../../commandkit/docs/consumers
## 4. Reports, Machine Output, And Outcomes
**Discussion / decision:** Agreed. Use native CommandKit reports for command
results, native outcomes for command completion, and a compact application-owned
run report for artifact discovery and domain status. Distinguish pipeline results,
command completion, and presentation delivery. Implementation is pending.
**Discussion / decision:** Native reporting and outcome mechanics remain selected.
The output/report contract is reopened: consumers primarily need resulting
artifacts, metadata may be optional, failures need useful nonzero outcomes, and
operators/developers use debug capture for detailed investigation. The earlier
run-report and durable-metadata proposals below remain discussion material for
that review, not requirements to preserve or expand. No new report/manifest
schemas have been selected. Finalize the cache decision first; return to the
consumer output/report contract afterward. Implementation is pending.
### Accepted-Artifact Cache And PromptKit Ownership
**Agreed follow-up:** Rename and replace the checkpoint subsystem with the
[accepted-artifact cache](promptkit-architecture-discussion.md#5-accepted-artifact-cache).
That decision owns the matching and storage policy; this section records its
CommandKit and execution-boundary implications.
The cached value is an accepted answer to the original task. Compute the key
from the initial prepared request using a proposed upstream `MessageContentHash`
over ordered message roles and exact text, excluding settings, session/execution
IDs, timestamps, and provider cache hints. Do not hash the entire prepared
request or introduce another Notarius-owned fingerprint policy.
Look up `cache/<hash>.json` before model execution. A hit loads the final accepted
Notarius artifact directly, subject only to ordinary decoding/usability checks.
It does not execute a model request, reconstruct a result/exchange, replay
corrections, or repeat semantic validation merely because the artifact was cached.
On a miss, execute the complete Notarius operation: native PromptKit calls,
domain interpretation and validation, and any application-directed correction
passes. Only after final acceptance save the artifact under the initial hash,
even if correction required several turns. For normalization, cache the accepted
normalized artifact after applying the plan, not the model's proposed plan.
Rejected, failed, and validation-incomplete work is not published to the cache.
PromptKit owns structural response validation, internal retries, repairs, and
other mechanics inside each request execution. Notarius consumes a usable
response or terminal failure at the public boundary and owns domain acceptance.
Internal PromptKit attempts do not determine cache eligibility. Native debug
evidence about those attempts does not transfer their ownership to Notarius.
The upstream requirement is message-content hashing. It does not require native
cached-response injection, replay, exchange restoration, or write-through
response caching. A small Notarius filesystem lookup/write facility stores the
accepted artifact without checkpoint manifests, dependency fingerprints,
acceptance ledgers, or required execution provenance. A cache hit is not a new
model execution or a claim of validation under a changed policy.
Concrete cache flags and legacy-checkpoint handling remain implementation-planning
choices. The output/report discussion must not reintroduce cache metadata solely
to populate a report. The master roadmap still needs this replacement policy.
### One Report Value For Human And Machine Output
@@ -633,10 +686,10 @@ status reflects presentation failure while the published artifacts remain a
fact. Do not rewrite pipeline history as failed execution or automatically rerun
model work because report delivery failed.
Notarius owns stage/lane/chunk progress, domain acceptance, and checkpoint reuse.
Notarius owns stage/lane/chunk progress, domain acceptance, and cache reuse.
PromptKit supplies native execution observations and `ExecutionGroup` totals;
CommandKit presents the selected facts. Explicitly enroll the relevant initial,
continuation, and validator executions without double counting. Checkpoint reuse
continuation, and validator executions without double counting. Cache reuse
does not add historical token usage to current execution totals, and unavailable
usage remains distinguishable from zero.
@@ -742,7 +795,7 @@ context where useful:
| `Observer` | Execution identities, attempt/retry/repair metadata, timing, usage completeness, and terminal metadata. |
| Returned `RunResult` | Available model output and native validation result. |
| `Exchange.Details()` | Explicit inspection of captured conversation history. |
| Application decisions | Domain acceptance/rejection, fallback, validation completeness, and checkpoint reuse. |
| Application decisions | Domain acceptance/rejection, fallback, validation completeness, and cache reuse. |
Compose capture and accounting observers. Remove the debug completion wrapper,
copied execution summaries, synthetic call identities, and duplicate terminal
@@ -757,12 +810,14 @@ Capture only available evidence. Preparation/admission failures may have no
`OnPrepared` notification; observer events have metadata, not response bodies;
operational failures may return no `RunResult`. Do not present the bundle as a
complete provider transcript. Use ordinary `Run` with native hooks, without
forcing split preparation/execution solely for recording.
forcing split preparation/execution solely for recording. The cache's lookup
before execution is a separate reason to use native preparation; `OnPrepared`
notifications alone do not provide cache short-circuiting.
Conversation capture required for continuation remains in-process execution
state. Persisting its history requires enabled debug capture. Normal manifests,
checkpoints, and run reports retain their minimal projections independently of
debug selection.
state. Persisting its history requires enabled debug capture. Cached values are
accepted domain artifacts, not conversation state or provenance records. The
separate output/report contract remains under discussion.
### Capture Lifecycle And Failure Policy
@@ -783,7 +838,7 @@ Treat link-update failures under the requested-capture failure policy below.
One application-owned recorder records initial context, accepts native execution
evidence and application decisions, and finalizes once with the known pipeline,
publication, and capture outcomes. Debug remains separate from output/cache,
never supplies checkpoint reuse, and is not automatically deleted.
never supplies cache reuse, and is not automatically deleted.
Requested capture is required work:
@@ -891,8 +946,9 @@ application outcome used for the CLI report. Reference detailed records where
useful; include convenient summary fields without maintaining a second execution
ledger. Configuration and resolved composition normally need one snapshot each.
Use native execution IDs with application scope to connect records.
Reused work has a restored-payload snapshot and reuse decision linked to original
provenance, not a fabricated model execution.
Reused work has a loaded-artifact snapshot and current cache-hit decision, not
a fabricated model execution. Historical provenance is not required in cache
files or reconstructed solely for debug capture.
Retain enough available context to investigate a run that fails before publishing
output. Do not depend entirely on a successful output manifest or automatically
@@ -905,7 +961,7 @@ competing transcript model or deduplication system to eliminate repeated history
**Resolved:** Use one default capture depth preserving every application-visible
turn, its useful inputs and outputs, and the decisions made about it. An operator
should be able to inspect a failed operation without reconstructing deltas,
finding a checkpoint, or extracting model text from escaped JSON strings.
finding a cache entry, or extracting model text from escaped JSON strings.
Convenience governs both capture and inspection; direct native serialization
and a simple readable rendering of the same values can both be useful.
@@ -988,9 +1044,9 @@ correction inputs even if the next execution fails during preparation; successfu
preparation captures the actual rendered correction in its native messages.
Retain deterministic stage outputs too: chunk plans and materialized chunks,
merged artifacts, and normalized artifacts. For checkpoint reuse, retain the
restored payload and reuse decision linked to original provenance. These records
do not fabricate a current execution or new validation of reused work.
merged artifacts, and normalized artifacts. For cache reuse, retain the loaded
artifact and current hit decision without requiring historical provenance. These
records do not fabricate a current execution or new validation of reused work.
#### Failed Runs And Native Availability
@@ -1046,9 +1102,11 @@ Select a small set of workflows that collectively establish:
flags, supplied-value distinctions, and overrides mapped correctly.
- Reports, diagnostics, and exit outcomes follow item 4, distinguishing domain
rejection, command failure, and presentation failure after publication.
- A downstream failure followed by resume reuses successful predecessors under
the simpler checkpoint policy, with explicit recomputation and consumed-input
changes handled as selected in the master roadmap.
- A downstream failure followed by another invocation reuses accepted artifacts
under the revised cache policy. Final corrected artifacts are stored under the
initial message hash; changed messages miss, settings-only changes can hit,
and hits neither make model calls nor repeat domain validation. Rejected or
validation-incomplete work is never stored as an accepted cache value.
- Debug capture follows item 5's directory selection, reporting-mode independence,
`latest` update timing, useful evidence, and allocation/write/finalization
failure behavior.
@@ -1090,17 +1148,17 @@ old-to-new migration map covering:
- Application and PromptKit configuration changes, source discovery, profiles,
and companion files.
- Native backend admission, independent worker settings, and revised retry controls.
- Checkpoint reuse, explicit recomputation, and the selected treatment of existing
checkpoint files.
- Accepted-artifact cache reuse, explicit refresh/bypass, and the selected
treatment of old checkpoint files.
- Machine reports, exit outcomes, and the distinction between successful command
execution and accepted domain results.
- Debug capture under `/tmp/notarius-debug`, `latest`, and the interaction between
capture and reporting modes.
CLI backward compatibility remains a non-goal; migration guidance does not imply
compatibility shims or preservation of old command behavior. Existing checkpoint
and published-artifact compatibility remain explicit application decisions about
stored work and consumer data, separate from CLI syntax.
compatibility shims or preservation of old command behavior. Legacy checkpoint
handling and published-artifact compatibility remain explicit application
decisions, without assuming that old checkpoint envelopes must survive.
Maintain runnable examples for embedded defaults and operator overrides. Update
canonical CLI, configuration, operations, integration/consumer, and internal
@@ -1121,7 +1179,7 @@ master roadmap. Use this phase sequence:
including reporting and useful debug capture.
4. Migrate remaining roles and remove the adapter, scheduler, retry machinery,
and custom CLI infrastructure.
5. Complete checkpoint migration, durable records, and capture behavior.
5. Complete cache migration, the selected output/report contract, and capture behavior.
6. Verify the assembled application and finish migration documentation.
Each phase delivers a reviewable result with appropriate verification. Tests and
@@ -1130,8 +1188,10 @@ completeness. The phases do not require separate releases or an intermediate
compatibility framework.
Resolve U1's dependency baseline and U10's CommandKit scope from items 1–4. Narrow
U9 to remaining durable-field and accounting choices, and debug implementation
U9 to the reopened consumer output/report contract, and debug implementation
details within item 5's resolved inventory, depth, and directory controls.
Update U1 for the upstream message-content hash and replace U8's checkpoint
identity/provenance plan with the accepted-artifact cache decision.
Reconcile removal inventory and acceptance criteria, including old
inspection-command and machine-output compatibility assumptions. Preserve
unresolved schema handling, role-specific retry/fallback policy, worker defaults,