Replace checkpoint proposals with accepted-artifact caching
This commit is contained in:
@@ -30,15 +30,19 @@ explicit-empty directory behavior are resolved below. Item 6 selects verificatio
|
||||
migration guidance, and incorporation into the master roadmap; that incorporation
|
||||
is still pending.
|
||||
|
||||
The major architectural decisions are settled. Remaining choices are run-report
|
||||
fields/versioning and durable record formats (item 4/U9). Function signatures,
|
||||
bundle filenames other than `latest`, and similar implementation details do not
|
||||
The cache follow-up in item 4 replaces the earlier checkpoint design with an
|
||||
accepted-artifact cache keyed by initial message content. The output/report
|
||||
contract has been reopened around consumer needs; it is not merely waiting for
|
||||
field names and versions. Function signatures, bundle filenames other than
|
||||
`latest`, and similar implementation details do not
|
||||
reopen the selected architecture. Broader PromptKit choices remain with the
|
||||
master roadmap.
|
||||
|
||||
The PromptKit architecture and checkpoint decisions remain in force. This
|
||||
discussion develops the CommandKit portion of that migration and its interactions
|
||||
with U1 (dependencies), U9 (records/reporting), and U10 (CommandKit adoption).
|
||||
The PromptKit ownership direction remains in force. The revised
|
||||
[cache decision](promptkit-architecture-discussion.md#5-accepted-artifact-cache)
|
||||
supersedes earlier checkpoint matching and provenance requirements. Master-roadmap
|
||||
incorporation is pending, including U1 (the upstream content-hash capability),
|
||||
U8 (cache migration), U9 (records/reporting), and U10 (CommandKit adoption).
|
||||
|
||||
## Agreed Proposal
|
||||
|
||||
@@ -60,10 +64,10 @@ Documenting the transition replaces a requirement to keep the old CLI working.
|
||||
Update canonical CLI and consumer documentation when the behavior lands.
|
||||
|
||||
Application responsibilities such as producing artifacts, reporting publication
|
||||
outcomes, and managing checkpoint/debug storage remain meaningful. Their existing
|
||||
outcomes, and managing cache/debug storage remain meaningful. Their existing
|
||||
CLI expression is not protected by an implicit compatibility exception. This
|
||||
guiding rule does not itself choose stored artifact or checkpoint formats; those
|
||||
remain separate application decisions in the master roadmap.
|
||||
guiding rule does not itself choose stored artifact formats. The revised cache
|
||||
decision below selects simple artifact storage rather than checkpoint envelopes.
|
||||
|
||||
### Why Adopt Together
|
||||
|
||||
@@ -88,7 +92,7 @@ spending migration effort recreating existing CLI conventions.
|
||||
| Command hosting, standard input/help mechanics, common report formatting and visibility | CommandKit. |
|
||||
| Standard configuration and PromptKit inspection commands | ckconfig and ckprompt, supplied with application loaders and request builders. |
|
||||
| Configuration composition and model execution | ConfigKit and PromptKit, using the native boundaries already selected. |
|
||||
| Pipeline execution and listing, pipeline-specific preflight, artifact publication, checkpoints, and domain outcomes | Notarius application handlers and framework. |
|
||||
| Pipeline execution and listing, pipeline-specific preflight, artifact publication, accepted-artifact caching, and domain outcomes | Notarius application handlers and framework. |
|
||||
| Receipt content, debug artifact lifecycle, and application outcome meaning | Notarius, expressed through CommandKit's reporting and outcome conventions. |
|
||||
|
||||
Adopt the standard capability suite selected in item 1, including upstream
|
||||
@@ -104,9 +108,9 @@ meaning and selects report content. CommandKit supplies presentation mechanics.
|
||||
Two application needs already identified for deliberate treatment are:
|
||||
|
||||
- The current `run --json` publishes a receipt after output publication. Item 4
|
||||
replaces its presentation with one compact application report rendered natively
|
||||
for humans and machines. Exact fields/versioning remain with U9; reports must
|
||||
describe the actual publication and domain outcome.
|
||||
records native reporting mechanics, but the output/report contract has been
|
||||
reopened around artifact delivery and optional metadata. Its earlier mandatory
|
||||
run-report proposal is subject to that discussion.
|
||||
- Item 5 uses native `--debug` for diagnostic verbosity and for capture when an
|
||||
effective debug directory is available, including an application default.
|
||||
Capture selection remains independent of the final reporting mode.
|
||||
@@ -191,8 +195,11 @@ are selected native features; this does not reintroduce legacy Notarius aliases.
|
||||
- Retain application handlers for pipeline execution and discovery, with native
|
||||
commands, inputs, and result presentation as selected in items 2–5.
|
||||
|
||||
No upstream feature request has been identified as a blocker for this scope.
|
||||
Evaluate any later demonstrated gap on its actual application need. On roadmap
|
||||
The standard capability suite needs no identified upstream addition. The later
|
||||
cache decision does require a native message-content hash, which is not the
|
||||
existing v0.23.3 rendered-prompt hash. Confirm the supplying API/release before
|
||||
implementing cache lookup; the version table remains the reviewed baseline, not
|
||||
a claim that the new hash is already available. On roadmap
|
||||
reconciliation, incorporate these versions into U1, replace U10's open standard
|
||||
capability selection with this decision, and incorporate the application wiring
|
||||
and presentation decisions from items 2–5. Preserve the remaining record choices
|
||||
@@ -268,7 +275,7 @@ omitted; application validation still decides whether a supplied value is valid.
|
||||
| Lane selection | Repeatable `--lane ID`; omission selects the normal pipeline lane set. Replace comma-separated `--only`. |
|
||||
| Output placement | Application-owned `--output-dir PATH`. |
|
||||
| Chunk-plan caching | `--chunk-cache auto\|bypass\|refresh`. |
|
||||
| Checkpoint reuse | Keep `--resume` and `--recompute-step ID` with the agreed recovery policy. |
|
||||
| Accepted-artifact cache | Replace checkpoint-centric controls with cache reuse/bypass/refresh behavior; concrete CLI spelling remains to be selected. `--resume` and `--recompute-step` are migration context, not compatibility requirements. |
|
||||
| Presentation | Native `--format`, `--quiet`, `--verbose`, `--debug`, and `--progress`. |
|
||||
| Configuration-valued tuning | Supported application `--set POINTER=JSON` overrides, including worker and semantic-budget settings once their configuration design is settled. |
|
||||
|
||||
@@ -500,8 +507,8 @@ preserving supplied operator values. Pipeline modules construct their own domain
|
||||
requests; share concrete request-building functions when useful without
|
||||
simulating pipeline execution to populate an inspection command.
|
||||
|
||||
Output publication, checkpoints, debug capture, and execution records belong to
|
||||
the run path. Loaders do not initialize those resources; item 5 selects debug
|
||||
Output publication, artifact caching, debug capture, and execution records belong
|
||||
to the run path. Loaders do not initialize those resources; item 5 selects debug
|
||||
allocation timing. Remove command-specific loading paths, copied configuration
|
||||
and resource inspection, eager runtime setup, and duplicated preflight
|
||||
orchestration. No broader application/package reorganization is required by this
|
||||
@@ -514,10 +521,56 @@ Evidence: [native suite and loader ownership](../../../commandkit/docs/consumers
|
||||
|
||||
## 4. Reports, Machine Output, And Outcomes
|
||||
|
||||
**Discussion / decision:** Agreed. Use native CommandKit reports for command
|
||||
results, native outcomes for command completion, and a compact application-owned
|
||||
run report for artifact discovery and domain status. Distinguish pipeline results,
|
||||
command completion, and presentation delivery. Implementation is pending.
|
||||
**Discussion / decision:** Native reporting and outcome mechanics remain selected.
|
||||
The output/report contract is reopened: consumers primarily need resulting
|
||||
artifacts, metadata may be optional, failures need useful nonzero outcomes, and
|
||||
operators/developers use debug capture for detailed investigation. The earlier
|
||||
run-report and durable-metadata proposals below remain discussion material for
|
||||
that review, not requirements to preserve or expand. No new report/manifest
|
||||
schemas have been selected. Finalize the cache decision first; return to the
|
||||
consumer output/report contract afterward. Implementation is pending.
|
||||
|
||||
### Accepted-Artifact Cache And PromptKit Ownership
|
||||
|
||||
**Agreed follow-up:** Rename and replace the checkpoint subsystem with the
|
||||
[accepted-artifact cache](promptkit-architecture-discussion.md#5-accepted-artifact-cache).
|
||||
That decision owns the matching and storage policy; this section records its
|
||||
CommandKit and execution-boundary implications.
|
||||
|
||||
The cached value is an accepted answer to the original task. Compute the key
|
||||
from the initial prepared request using a proposed upstream `MessageContentHash`
|
||||
over ordered message roles and exact text, excluding settings, session/execution
|
||||
IDs, timestamps, and provider cache hints. Do not hash the entire prepared
|
||||
request or introduce another Notarius-owned fingerprint policy.
|
||||
|
||||
Look up `cache/<hash>.json` before model execution. A hit loads the final accepted
|
||||
Notarius artifact directly, subject only to ordinary decoding/usability checks.
|
||||
It does not execute a model request, reconstruct a result/exchange, replay
|
||||
corrections, or repeat semantic validation merely because the artifact was cached.
|
||||
|
||||
On a miss, execute the complete Notarius operation: native PromptKit calls,
|
||||
domain interpretation and validation, and any application-directed correction
|
||||
passes. Only after final acceptance save the artifact under the initial hash,
|
||||
even if correction required several turns. For normalization, cache the accepted
|
||||
normalized artifact after applying the plan, not the model's proposed plan.
|
||||
Rejected, failed, and validation-incomplete work is not published to the cache.
|
||||
|
||||
PromptKit owns structural response validation, internal retries, repairs, and
|
||||
other mechanics inside each request execution. Notarius consumes a usable
|
||||
response or terminal failure at the public boundary and owns domain acceptance.
|
||||
Internal PromptKit attempts do not determine cache eligibility. Native debug
|
||||
evidence about those attempts does not transfer their ownership to Notarius.
|
||||
|
||||
The upstream requirement is message-content hashing. It does not require native
|
||||
cached-response injection, replay, exchange restoration, or write-through
|
||||
response caching. A small Notarius filesystem lookup/write facility stores the
|
||||
accepted artifact without checkpoint manifests, dependency fingerprints,
|
||||
acceptance ledgers, or required execution provenance. A cache hit is not a new
|
||||
model execution or a claim of validation under a changed policy.
|
||||
|
||||
Concrete cache flags and legacy-checkpoint handling remain implementation-planning
|
||||
choices. The output/report discussion must not reintroduce cache metadata solely
|
||||
to populate a report. The master roadmap still needs this replacement policy.
|
||||
|
||||
### One Report Value For Human And Machine Output
|
||||
|
||||
@@ -633,10 +686,10 @@ status reflects presentation failure while the published artifacts remain a
|
||||
fact. Do not rewrite pipeline history as failed execution or automatically rerun
|
||||
model work because report delivery failed.
|
||||
|
||||
Notarius owns stage/lane/chunk progress, domain acceptance, and checkpoint reuse.
|
||||
Notarius owns stage/lane/chunk progress, domain acceptance, and cache reuse.
|
||||
PromptKit supplies native execution observations and `ExecutionGroup` totals;
|
||||
CommandKit presents the selected facts. Explicitly enroll the relevant initial,
|
||||
continuation, and validator executions without double counting. Checkpoint reuse
|
||||
continuation, and validator executions without double counting. Cache reuse
|
||||
does not add historical token usage to current execution totals, and unavailable
|
||||
usage remains distinguishable from zero.
|
||||
|
||||
@@ -742,7 +795,7 @@ context where useful:
|
||||
| `Observer` | Execution identities, attempt/retry/repair metadata, timing, usage completeness, and terminal metadata. |
|
||||
| Returned `RunResult` | Available model output and native validation result. |
|
||||
| `Exchange.Details()` | Explicit inspection of captured conversation history. |
|
||||
| Application decisions | Domain acceptance/rejection, fallback, validation completeness, and checkpoint reuse. |
|
||||
| Application decisions | Domain acceptance/rejection, fallback, validation completeness, and cache reuse. |
|
||||
|
||||
Compose capture and accounting observers. Remove the debug completion wrapper,
|
||||
copied execution summaries, synthetic call identities, and duplicate terminal
|
||||
@@ -757,12 +810,14 @@ Capture only available evidence. Preparation/admission failures may have no
|
||||
`OnPrepared` notification; observer events have metadata, not response bodies;
|
||||
operational failures may return no `RunResult`. Do not present the bundle as a
|
||||
complete provider transcript. Use ordinary `Run` with native hooks, without
|
||||
forcing split preparation/execution solely for recording.
|
||||
forcing split preparation/execution solely for recording. The cache's lookup
|
||||
before execution is a separate reason to use native preparation; `OnPrepared`
|
||||
notifications alone do not provide cache short-circuiting.
|
||||
|
||||
Conversation capture required for continuation remains in-process execution
|
||||
state. Persisting its history requires enabled debug capture. Normal manifests,
|
||||
checkpoints, and run reports retain their minimal projections independently of
|
||||
debug selection.
|
||||
state. Persisting its history requires enabled debug capture. Cached values are
|
||||
accepted domain artifacts, not conversation state or provenance records. The
|
||||
separate output/report contract remains under discussion.
|
||||
|
||||
### Capture Lifecycle And Failure Policy
|
||||
|
||||
@@ -783,7 +838,7 @@ Treat link-update failures under the requested-capture failure policy below.
|
||||
One application-owned recorder records initial context, accepts native execution
|
||||
evidence and application decisions, and finalizes once with the known pipeline,
|
||||
publication, and capture outcomes. Debug remains separate from output/cache,
|
||||
never supplies checkpoint reuse, and is not automatically deleted.
|
||||
never supplies cache reuse, and is not automatically deleted.
|
||||
|
||||
Requested capture is required work:
|
||||
|
||||
@@ -891,8 +946,9 @@ application outcome used for the CLI report. Reference detailed records where
|
||||
useful; include convenient summary fields without maintaining a second execution
|
||||
ledger. Configuration and resolved composition normally need one snapshot each.
|
||||
Use native execution IDs with application scope to connect records.
|
||||
Reused work has a restored-payload snapshot and reuse decision linked to original
|
||||
provenance, not a fabricated model execution.
|
||||
Reused work has a loaded-artifact snapshot and current cache-hit decision, not
|
||||
a fabricated model execution. Historical provenance is not required in cache
|
||||
files or reconstructed solely for debug capture.
|
||||
|
||||
Retain enough available context to investigate a run that fails before publishing
|
||||
output. Do not depend entirely on a successful output manifest or automatically
|
||||
@@ -905,7 +961,7 @@ competing transcript model or deduplication system to eliminate repeated history
|
||||
**Resolved:** Use one default capture depth preserving every application-visible
|
||||
turn, its useful inputs and outputs, and the decisions made about it. An operator
|
||||
should be able to inspect a failed operation without reconstructing deltas,
|
||||
finding a checkpoint, or extracting model text from escaped JSON strings.
|
||||
finding a cache entry, or extracting model text from escaped JSON strings.
|
||||
Convenience governs both capture and inspection; direct native serialization
|
||||
and a simple readable rendering of the same values can both be useful.
|
||||
|
||||
@@ -988,9 +1044,9 @@ correction inputs even if the next execution fails during preparation; successfu
|
||||
preparation captures the actual rendered correction in its native messages.
|
||||
|
||||
Retain deterministic stage outputs too: chunk plans and materialized chunks,
|
||||
merged artifacts, and normalized artifacts. For checkpoint reuse, retain the
|
||||
restored payload and reuse decision linked to original provenance. These records
|
||||
do not fabricate a current execution or new validation of reused work.
|
||||
merged artifacts, and normalized artifacts. For cache reuse, retain the loaded
|
||||
artifact and current hit decision without requiring historical provenance. These
|
||||
records do not fabricate a current execution or new validation of reused work.
|
||||
|
||||
#### Failed Runs And Native Availability
|
||||
|
||||
@@ -1046,9 +1102,11 @@ Select a small set of workflows that collectively establish:
|
||||
flags, supplied-value distinctions, and overrides mapped correctly.
|
||||
- Reports, diagnostics, and exit outcomes follow item 4, distinguishing domain
|
||||
rejection, command failure, and presentation failure after publication.
|
||||
- A downstream failure followed by resume reuses successful predecessors under
|
||||
the simpler checkpoint policy, with explicit recomputation and consumed-input
|
||||
changes handled as selected in the master roadmap.
|
||||
- A downstream failure followed by another invocation reuses accepted artifacts
|
||||
under the revised cache policy. Final corrected artifacts are stored under the
|
||||
initial message hash; changed messages miss, settings-only changes can hit,
|
||||
and hits neither make model calls nor repeat domain validation. Rejected or
|
||||
validation-incomplete work is never stored as an accepted cache value.
|
||||
- Debug capture follows item 5's directory selection, reporting-mode independence,
|
||||
`latest` update timing, useful evidence, and allocation/write/finalization
|
||||
failure behavior.
|
||||
@@ -1090,17 +1148,17 @@ old-to-new migration map covering:
|
||||
- Application and PromptKit configuration changes, source discovery, profiles,
|
||||
and companion files.
|
||||
- Native backend admission, independent worker settings, and revised retry controls.
|
||||
- Checkpoint reuse, explicit recomputation, and the selected treatment of existing
|
||||
checkpoint files.
|
||||
- Accepted-artifact cache reuse, explicit refresh/bypass, and the selected
|
||||
treatment of old checkpoint files.
|
||||
- Machine reports, exit outcomes, and the distinction between successful command
|
||||
execution and accepted domain results.
|
||||
- Debug capture under `/tmp/notarius-debug`, `latest`, and the interaction between
|
||||
capture and reporting modes.
|
||||
|
||||
CLI backward compatibility remains a non-goal; migration guidance does not imply
|
||||
compatibility shims or preservation of old command behavior. Existing checkpoint
|
||||
and published-artifact compatibility remain explicit application decisions about
|
||||
stored work and consumer data, separate from CLI syntax.
|
||||
compatibility shims or preservation of old command behavior. Legacy checkpoint
|
||||
handling and published-artifact compatibility remain explicit application
|
||||
decisions, without assuming that old checkpoint envelopes must survive.
|
||||
|
||||
Maintain runnable examples for embedded defaults and operator overrides. Update
|
||||
canonical CLI, configuration, operations, integration/consumer, and internal
|
||||
@@ -1121,7 +1179,7 @@ master roadmap. Use this phase sequence:
|
||||
including reporting and useful debug capture.
|
||||
4. Migrate remaining roles and remove the adapter, scheduler, retry machinery,
|
||||
and custom CLI infrastructure.
|
||||
5. Complete checkpoint migration, durable records, and capture behavior.
|
||||
5. Complete cache migration, the selected output/report contract, and capture behavior.
|
||||
6. Verify the assembled application and finish migration documentation.
|
||||
|
||||
Each phase delivers a reviewable result with appropriate verification. Tests and
|
||||
@@ -1130,8 +1188,10 @@ completeness. The phases do not require separate releases or an intermediate
|
||||
compatibility framework.
|
||||
|
||||
Resolve U1's dependency baseline and U10's CommandKit scope from items 1–4. Narrow
|
||||
U9 to remaining durable-field and accounting choices, and debug implementation
|
||||
U9 to the reopened consumer output/report contract, and debug implementation
|
||||
details within item 5's resolved inventory, depth, and directory controls.
|
||||
Update U1 for the upstream message-content hash and replace U8's checkpoint
|
||||
identity/provenance plan with the accepted-artifact cache decision.
|
||||
Reconcile removal inventory and acceptance criteria, including old
|
||||
inspection-command and machine-output compatibility assumptions. Preserve
|
||||
unresolved schema handling, role-specific retry/fallback policy, worker defaults,
|
||||
|
||||
Reference in New Issue
Block a user