Files
notarius/docs/internal/state.md

194 lines
11 KiB
Markdown
Raw Normal View History

# Run State Internals
This guide explains physical cache, capture, and publication collaborators.
[Operations](../operations.md) owns layout, permissions, retention, and recovery
procedures; [Configuration](../config.md#concurrency-output-cache-and-debug) owns
settings/defaults; [CLI Reference](../cli.md#completion-report) owns reporting.
[Pipeline Internals](pipeline.md) owns assessment, workers, and dependency eligibility.
## Composition And Independent State
[The run handler](../../internal/cli/pipeline_run.go) is the physical-path
composition root. It obtains loaded configuration and a safe run identity,
allocates requested capture before engine/preflight, resolves/materializes/
prepares, and then constructs the cache/recorder collaborators for the runner.
Shared inspection loaders create no run storage.
Modules receive explicit operation/cache/evidence interfaces and data, not roots
or CommandKit reporters. Output, accepted-result cache, and optional capture
remain independent. None is a required metadata bundle for another.
In-memory normalized artifacts support handoffs/evidence; they are not a
persisted resume ledger.
## Prepared Result Reuse
[RunProducer](../../internal/framework/pipeline/producer.go) owns native request
selection and cache interaction for an eligible model operation:
1. Prepare the initial request once with `PrepareExecution`, retain its detached
details and opaque `MessageContentHash`.
2. For a read-enabled selection, decode a found entry through the expected
durable codec and current-source usability checks. On a usable hit, discard
the prepared handle and return the artifact without model work.
3. On a miss, execute that exact handle with `RunPrepared`, then use ordinary
interpretation, complete assessment, and native continuation.
4. Store only final ordinary acceptance, encoded as the applied typed artifact
or checked logical plan, under the initial key. Dispose of unused
prepared handles on every exit.
[Typed bindings](../../internal/framework/pipeline/runner_cache.go) attach codec
and source-evidence checks; [chunk-plan bindings](../../internal/framework/pipeline/runner_chunk_plan.go)
decode and rematerialize against the current document. A hit has no ordinary
semantic revalidation, native execution event, restored exchange, or historical
usage. Normalizers store applied domain results, not reconciliation proposals.
Deterministic operations stay uncached without manufactured requests.
The key is the native initial message-content value, not a configuration,
policy, model/session, implementation, or source-only fingerprint. Rendered
message changes can change selection; settings-only changes need not.
Correction retains the initial key. Public reuse choices and their consequences
belong in [Operations](../operations.md#accepted-result-cache-and-recovery).
Reuse eligibility is separate from the selected cache mode. Fallback and actual
dependent work perform neither lookup nor storage, even if they later produce
otherwise accepted output. Bypass/ineligible work uses ordinary native execution
without cache-specific split preparation. Capture reuses available prepared
details; it never adds preparation to explain a hit.
## Filesystem Store And Failure Handling
[artifactcache.FilesystemStore](../../internal/framework/artifactcache/filesystem.go)
implements the narrow `contracts.ArtifactCache` interface. Construction retains
the exact selected root without creating it. Lookup checks one confined regular
JSON entry; missing data is an ordinary miss. Stored bytes are the unchanged
accepted artifact, without provenance/execution envelopes or alternate key
dimensions. Physical placement belongs in
[Operations](../operations.md#state-surfaces).
The store honors the resolved read/write mode. Refresh does not read or remove
previous acceptance in anticipation of execution. Only a newly supplied accepted
value can replace it. Failed refresh execution or a failed cache write preserves
the old entry.
Legacy data is not scanned, converted, rewritten, or deleted.
Publication uses [fileio](../../internal/core/fileio/fileio.go)'s confined atomic
process-permission writer. New cache objects use process/deployment creation
permissions; there is no permission convergence on an existing root.
[ADR-0017](../adr/0017-defer-cache-access-policy-to-deployment.md) owns that decision.
[Cache integration](../../internal/framework/pipeline/producer_cache.go) treats
unreadable/malformed/codec-unusable entries as diagnostic misses. Store failures
become scoped process warnings without failing successful production or calling
the provider again. Artifact encoding, caller cancellation, requested evidence,
and observer failures retain their own error boundaries; they are not swallowed
as best-effort storage failures.
Accepted predecessors can survive downstream failure and be reused by an ordinary
rerun. There is no checkpoint loader, resume barrier, forced-recompute policy, or
canonical partition selected solely by source digest. New message selections
populate independently; an accepted plan is rematerialized, not restored with a
historic execution claim.
## Artifact Publication And Native Reporting
The runner returns logical deliverable files only after successful execution/
encoding. The [JSON encoder](../../internal/modules/generic/output/json/encoder.go)
owns domain-file selection and optional auxiliary inclusion.
[JSON output](../integrations/json-output.md),
[chunk maps](../integrations/chunk-map.md), and
[evidence context](../integrations/evidence-context.md) own logical paths/schemas,
not the store or recorder.
[writeOutputFiles](../../internal/cli/run_output.go) validates every logical
path before allocating the run directory, refuses an existing destination, and
writes files with confined atomic helpers. This is per-file publication, not
an all-files transaction. A later write can leave a partial new directory;
existing siblings are preserved. No rollback deletes available artifacts.
After complete publication, the run handler finalizes requested capture before
[reportRunResult](../../internal/cli/run_result.go) submits the small native
completion value. The report is not automatically persisted or required to decode
domain files. No index, manifest, receipt, or process-diagnostic sidecar is emitted.
Artifact count counts returned deliverables, including selected available auxiliaries,
not cache/debug/intermediate files.
Post-publication capture/render/delivery failure remains nonzero with available
locations. Valid artifacts remain; model work and capture terminalization are not
repeated. Native delivery evidence is separate from the operation cause, and
failed stdout may be partial. Public outcomes and caller responsibilities belong
in [CLI Reference](../cli.md#output-streams-and-exit-statuses) and the
[subprocess guide](../consumers/subprocess.md).
## Capture Lifecycle
[Debug selection](../../internal/cli/run_debug.go) preserves native boolean and
supplied directory semantics independently of verbosity. The run path alone
calls [debug.Start](../../internal/framework/debug/recorder.go).
[Bundle allocation](../../internal/core/debugbundle/bundle.go) validates identity,
creates a new absolute per-run directory, and atomically updates a relative
latest link at start. Failure can return an available directory for explanation/
retention. Disabled capture creates nothing; completion never retargets the link.
One synchronized recorder writes evidence incrementally, retaining lifecycle
state and the first recording error rather than accumulating whole-run content.
Callback failures are exposed at the next boundary that can return an error.
`Finalize` attempts terminal evidence once, joins primary and capture causes,
and retains its result. Repeated calls do not rewrite evidence or retarget latest.
A later presentation failure does not change capture into another terminal run.
Allocation/link/record/finalization failures never trigger model retries or a
fallback directory. Available files are retained.
## Native And Application Evidence
[Run capture](../../internal/cli/run_capture.go) snapshots the original secret-aware
application/PromptKit configuration and source evidence. Scoped pipeline capture
records source/reference/incoming artifacts, normalization originals/mappings/
proposals, candidates, complete findings/guidance, fallback/deterministic outcomes,
and cache selection/current artifact bytes.
[NativeCapture](../../internal/framework/pipeline/native_capture.go) composes
existing observers and preparation callbacks. Preparation callbacks retain
detached details cheaply; ordinary boundaries write prepared contracts, native
results, raw response text, and full readable exchanges. Events are written
incrementally. Scope includes operation/step/lane/chunk/validator identity to
avoid concurrent evidence collisions.
One invocation `ExecutionGroup` enrolls actual initial, continuation, and nested
validator executions once. Native identity, terminal coverage, and unknown usage
remain native facts, not a copied profile/attempt accounting model.
Hits can retain prepared evidence without execution or usage. Result-less failures
retain availability/cause evidence; undelivered responses are explicitly unassessed.
Native credential omissions and ConfigKit marked-secret paths are preserved.
Arbitrary source/prompt/response text is not heuristically rewritten or made safe
for general logging. Exact evidence names/nesting/serialized forms are private,
not a public protocol or reconstruction requirement. Debug is neither cache
input nor an artifact-discovery interface; [Operations](../operations.md#debug-bundles)
owns handling.
## Focused Verification
- [Filesystem cache](../../internal/framework/artifactcache/filesystem_test.go)
and [permissions](../../internal/framework/artifactcache/filesystem_permissions_unix_test.go)
cover raw bytes, modes, atomic replacement, diagnostics, confinement, deployment
permissions, and unchanged legacy files.
- [Producer cache](../../internal/framework/pipeline/producer_cache_test.go),
[typed runner cache](../../internal/framework/pipeline/runner_cache_test.go), and
[plan cache](../../internal/framework/pipeline/chunk_plan_cache_test.go) cover
initial handles/keys, final acceptance, current-source checks, and failed refresh.
- [Recorder](../../internal/framework/debug/recorder_test.go) and
[allocation](../../internal/core/debugbundle/bundle_test.go) cover incremental
failure retention, concurrent starts, and one terminalization.
- [Native evidence](../../internal/framework/pipeline/native_capture_test.go) and
[configuration capture](../../internal/cli/run_capture_test.go) cover available
content, exact sensitivity, observer composition, and no duplicate preparation.
- [Output writes](../../internal/cli/run_id_test.go) and
[native result boundaries](../../internal/cli/run_result_command_test.go) cover
collisions, partial directories, capture/presentation failure, and retained causes.
- [Assembled recovery](../../internal/cli/recovery_contract_test.go) covers
downstream fix/rerun, initial-key correction reuse, qualified fallback, actual
generated consumers, and debug on hits/failure.
Use focused pipeline/cache/recorder/CLI checks for state-boundary changes.
[Development](../development.md#validation) owns wider validation.