Update native operations, consumer migration, and version checks

This commit is contained in:
2026-11-22 06:32:22 +00:00
parent 5e58327032
commit 2c27695df9
18 changed files with 675 additions and 967 deletions

View File

@@ -10,6 +10,9 @@ Notarius uses native CommandKit commands with ckconfig and ckprompt. The
and smoke semantics. The examples below show Notarius-specific selections.
Use help for every installed command's current options.
For retired command/configuration mappings and receipt-consumer changes, see
[native migration guidance](consumers/native-migration.md).
## Command Summary
~~~text

View File

@@ -6,6 +6,9 @@ source selection, and selectable modules. The sole application version is
[Operations](operations.md) owns physical state and recovery; integration
contracts own artifact payloads.
Upgrading an older deployment or caller? Use the
[native migration guide](consumers/native-migration.md).
## Configuration Discovery And Precedence
Configuration-consuming commands select one main file:

View File

@@ -1,142 +1,83 @@
# Consuming The Complete D&D Pipeline
Use this workflow when an orchestrator runs the maintained complete D&D
pipeline and consumes its structured JSON artifacts. The generic
[subprocess consumer guide](subprocess.md) owns process-level responsibilities;
this guide connects that workflow to the complete D&D configuration, its
Seriatim input, and its artifact inventory.
This guide applies the [subprocess workflow](subprocess.md) to the maintained
complete D&D configuration. [CLI Reference](../cli.md) owns invocation and the
native report; [Configuration](../config.md) owns composition;
[Published JSON Output](../integrations/json-output.md) owns logical paths.
[Migration guidance](native-migration.md) covers older orchestrators.
The [CLI reference](../cli.md), [configuration reference](../config.md),
[run-result receipt](../integrations/run-result.md), and
[published JSON output contract](../integrations/json-output.md) remain the
canonical definitions of those public interfaces.
## Prepare The Deployment
## Prepare And Validate The Deployment
Start from [examples/dnd-complete/config.yml](../../examples/dnd-complete/config.yml).
Preserve its imported pipeline fragments, native companion, and reference files,
or update the paths deliberately. It uses every implemented D&D family, ordered
generated handoffs, and selected chunk-map/evidence output.
Start from the maintained
[complete D&D configuration](../../examples/dnd-complete.config.yml). It uses
the `dnd-session` pipeline and demonstrates every implemented D&D lane, ordered
artifact handoffs, campaign references, chunk-map publication, and evidence
context.
Native embedded defaults and the maintained companion already provide profile/
backend definitions. A deployment can supply variants rather than being required
to recreate them. YAML paths retain the contributing declaration's origin;
CLI reference paths use the process working directory. Absolute caller paths
are useful for services. Keep credentials in the intended environment, not
files, arguments, sessions, or logs.
A deployment must provide its own PromptKit profile and campaign reference
files. Use absolute paths for service and subprocess deployments. In
particular, observe these different resolution rules:
Preflight without credentials or inference:
- reference and PromptKit profile paths in YAML are resolved relative to the
file that supplies their surviving declaration;
- reference paths passed with `--reference` are resolved relative to the
Notarius process working directory.
~~~sh
notarius config validate --config /absolute/path/to/notarius.yml
notarius pipeline validate dnd-session --config /absolute/path/to/notarius.yml
~~~
Do not copy the repository example's relative profile path into a deployment
without preserving its configuration-relative layout. The complete path and
profile rules are defined in [Configuration](../config.md).
## Supply The Transcript And References
Preflight the deployed configuration before processing sessions and whenever
it changes:
Use the [Seriatim JSON contract](../integrations/seriatim.md). Preserve segment
IDs because artifact citations use those source-unit ranges. If the caller
maintains a final trimmed transcript tier, select that same session content
consistently for later consumers.
```sh
notarius config validate \
--config /absolute/path/to/notarius.yml \
--pipeline dnd-session
```
Run with explicit input/output and pipeline-level references:
Provide credentials through the environment or the documented configuration
mechanism. Do not put credentials in command arguments, generated
configuration, or logs.
## Supply The Transcript
The complete pipeline consumes a Seriatim JSON document. The
[Seriatim input contract](../integrations/seriatim.md) defines its required
metadata, segments, and validation rules. Preserve segment IDs: D&D artifact
citations use those segment IDs as source-unit ranges.
When the caller maintains several transcript tiers, use the final trimmed JSON
transcript so extraction operates on the same session content presented to
later consumers. For example, Narratio identifies this implemented artifact as
`narratio.transcript.final_trimmed` and normally stores it at
`transcripts/final.trimmed.json`.
Notarius generates a stable prompt session from the resolved input module and
the exact input bytes. An ordinary orchestrator should not pass `--session-id`.
Use that override only when intentionally changing the routing relationship
between invocations; it is not a credential or output identity.
## Run Notarius
Invoke the pipeline with explicit absolute paths and request its
machine-readable receipt:
```sh
~~~sh
notarius run dnd-session \
--config /absolute/path/to/notarius.yml \
--input /absolute/path/to/transcripts/final.trimmed.json \
--input-file /absolute/path/to/transcripts/final.trimmed.json \
--output-dir /absolute/path/to/notarius-output \
--reference party=/absolute/path/to/references/party.txt \
--reference players=/absolute/path/to/references/players.txt \
--reference glossary=/absolute/path/to/references/glossary.txt \
--reference spell_catalog=/absolute/path/to/references/spells.json \
--json
```
--format json
~~~
Each unqualified reference is pipeline-scoped: Notarius supplies it to every
selected D&D target that declares the slot. A deployment may omit an optional
reference it does not maintain, and may use the lane- or binding-qualified
forms from the [CLI reference](../cli.md#reference-selectors) for an exceptional
override. The registry, scene-description, combat-turn, and NPC-occurrence
references declared between ordered steps in the complete configuration are
generated artifacts. Do not pass those handoffs on the CLI.
Only supply optional references the deployment maintains. Unqualified selectors
reach every selected target declaring the slot; narrower overrides follow the
[selector contract](../cli.md#reference-selectors). Generated registry, scene,
combat, and NPC-occurrence handoffs stay in pipeline configuration, not CLI file
bindings.
The caller should:
The ordinary input-based session is suitable without an override. Use
`--llm-session-id` only to intentionally change its native selection.
Sessions correlate provider routing; they are not credential or output identities.
- capture stdout and stderr separately;
- propagate cancellation and impose an operator-appropriate timeout;
- wait for process completion before interpreting stdout; and
- retain stderr for diagnosis without copying secrets or transcript content
into other logs.
Capture stdout/stderr separately, propagate cancellation, impose an appropriate
caller deadline, and wait for completion. Parse a successful report only after
status zero; failed writes can leave partial stdout. Nonzero status after
publication can still leave valid artifacts for deliberate recovery.
Only exit status 0 permits decoding stdout as a receipt. Ignore stdout after a
nonzero exit because a failed receipt write can leave partial bytes. The
[CLI reference](../cli.md#output-streams-and-exit-statuses) defines the complete
stream and exit-status contract.
## Locate And Read Artifacts
## Discover The Published Bundle
Use the report's absolute `output_directory`, not a newest-directory scan.
The report's exact fields and fallback qualification belong in
[CLI Reference](../cli.md#completion-report). There is no receipt schema,
index/manifest traversal, or descriptor inventory to load.
Decode the successful stdout document as a supported run-result schema. For
the current contract, `schema_version` is `notarius.run-result.v2`. Tolerate
unknown fields allowed by that version, but reject an unsupported schema
version.
Use the receipt's absolute `output_directory` as the exact run-specific bundle
root. Do not scan the output root for its newest directory, guess a run ID, or
construct a bundle path. Resolve `index_file` beneath `output_directory` and
reject an absolute logical path or any result that escapes the bundle root.
The complete configuration uses the application validation defaults. A caller
that requires fully validated D&D artifacts must also require receipt
`validation_status: approved`; a successful `incomplete` result reflects the
configured validator-failure continuation policy and carries its bounded
validator provenance in `validation_summaries`.
Read `index.json` and locate each requested lane in `output_files` by its exact
`lane_id`. Do not guess a lane filename. Before decoding a payload:
1. resolve its descriptor's relative `file` beneath the bundle root with the
same confinement check;
2. verify the descriptor's media type and schema identity against the linked
artifact contract; and
3. decode the payload according to that contract.
The [published JSON output contract](../integrations/json-output.md) defines
the index and bundle layout. Treat all paths obtained from a decoded external
document as untrusted until confined to their documented root.
The complete configuration uses safe literal lane IDs: each table entry below
is at `lanes/<lane-id>.json` beneath that run directory. Decode the configured
domain contract directly. All ten lane artifacts plus the two enabled/available
auxiliaries give twelve deliverables; a count is not a substitute for checking
the files required by the caller.
## Complete Artifact Inventory
When every configured lane is accepted, the complete example publishes these
lane artifacts:
| Lane ID | Purpose | Canonical contract |
| --- | --- | --- |
| `item-registry` | Canonical registry of encountered items and currency. | [Item registry](../integrations/dnd-item-registry-artifacts.md) |
@@ -150,72 +91,52 @@ lane artifacts:
| `location-occurrences` | Source-grounded location occurrences. | [Location occurrences](../integrations/dnd-location-occurrence-artifacts.md) |
| `enemy-events` | Source-grounded enemy combat events. | [Enemy events](../integrations/dnd-enemy-event-artifacts.md) |
The JSON encoder always publishes these bundle-management files:
The complete configuration also selects two pipeline-wide files:
| File | Purpose |
| --- | --- |
| `index.json` | Discovery document for lane and pipeline-wide artifacts. |
| `manifest.json` | Run provenance and result summaries. |
| `rejected.json` | Rejected pipeline outputs. |
| `warnings.json` | Actionable process-degradation warnings. |
| `diagnostics.json` | Advisory and observation findings for accepted artifacts. |
- [`chunk-map.json`](../integrations/chunk-map.md), the materialized plan/chunks.
- [`evidence-context.json`](../integrations/evidence-context.md), the union of
selected cited units and their configured surrounding window.
The complete configuration also requests two pipeline-wide artifacts:
They are not lanes and need no discovery descriptor. Their intrinsic payload
schemas are unchanged. Evidence is reading context, not authoritative lane
attribution; surrounding units do not widen citations in lane artifacts.
No index, manifest, rejection/warning/diagnostic sidecars are required.
- [`chunk-map.json`](../integrations/chunk-map.md), the accepted chunk plan and
chunk metadata; and
- [`evidence-context.json`](../integrations/evidence-context.md), a reading
excerpt containing the union of selected cited source units and the
configured surrounding window.
## Decide Consumer Acceptance
Discover both from their top-level `index.json` descriptors rather than
treating them as lanes. Evidence context is convenient reading material, not
authoritative provenance; citations in the normalized lane payloads remain the
evidence contract.
Process completion does not establish the caller's required artifact set,
semantic quality, or fresh validation under changed policy. A complete-workflow
caller should check all ten required lane files against their contracts. A
subset consumer can choose a narrower set deliberately. Empty valid domain
collections remain distinguishable from missing/incompatible files.
Every optional or lane file is published only when its corresponding artifact
is available. A successful process does not guarantee that all configured
lanes were accepted.
Explicit permissive execution can complete with qualified fallback. Decide
whether to accept that and inspect positive `fallback_count` when relevant.
The count identifies emitting operations, not individual affected lanes.
Fallbacks satisfy intrinsic/downstream invariants but not ordinary semantic
approval; valid downstream results affected by them also avoid cache reuse.
## Decide What Counts As Consumer Success
Default cache reuse does not repeat semantic assessment. For a deliberate model/
policy experiment requiring fresh inference, select bypass/refresh under
[configuration controls](../config.md#producer-controls). Do not look for retired
validation ledgers or treat optional debug serialization as an API.
Exit status 0 means Notarius completed the pipeline and published its result
bundle. The receipt or bundle may still report warnings, rejected outputs, or
missing lane descriptors. A downstream consumer must define its own required
artifact set explicitly.
A caller that claims to consume the complete D&D workflow should normally
require all ten lane IDs in the table and verify each descriptor's expected
contract. If any required lane is missing, rejected, or incompatible, fail the
caller's extraction step while retaining the Notarius bundle for diagnosis. A
consumer that needs only a subset may define and document a narrower policy.
Keep the successful receipt with the complete published bundle. Retain
`manifest.json`, `rejected.json`, `warnings.json`, and captured process logs as
required by the caller's provenance, diagnosis, and retention policies. Avoid
selectively copying payload files without also preserving enough index and
manifest information to identify their originating run and contracts.
The transcript, lane artifacts, evidence context, manifest, debug data, and
logs can all contain private campaign information. Apply the same access,
publication, and retention controls used for the source transcript.
Retain useful domain files and caller-owned invocation/report evidence. Capture
failure after publication does not invalidate completed artifact writes, but
requires deliberate caller recovery rather than parsing partial success output.
A failed publication can leave an incomplete directory. Transcript-derived
artifacts, evidence, cache, debug, and streams can contain private campaign data;
apply the caller's sharing and retention choices.
## Consumer Checklist
- Validate the deployed Notarius configuration and `dnd-session` pipeline.
- Pass the final trimmed Seriatim JSON transcript with stable segment IDs.
- Use absolute configuration, input, output-root, profile, and reference paths
in service deployments.
- Capture stdout and stderr separately and enforce cancellation and timeout.
- Parse stdout only after exit status 0.
- Accept only supported receipt, index, and artifact schema versions while
tolerating permitted unknown fields.
- Use the receipt's `output_directory`; never guess the run directory.
- Confine `index_file` and every descriptor path to the published bundle root.
- Discover lanes by `lane_id` and verify descriptor compatibility before
decoding payloads.
- Enforce an explicit required-lane policy and inspect rejections and warnings.
- Preserve the receipt and sufficient bundle provenance for every retained
artifact.
- Protect all transcript-derived files and diagnostic streams as sensitive
campaign data.
- Validate application configuration and the selected pipeline offline.
- Preserve Seriatim source-unit IDs.
- Pass source, output root, and optional campaign references as first-class inputs.
- Keep generated handoffs configured; avoid external/generated collisions.
- Separate streams, propagate cancellation, and check status before parsing.
- Use the native report directory and configured domain/auxiliary paths.
- Decode supported payload contracts and check the caller's explicit required set.
- Decide whether qualified fallback and cached acceptance meet caller needs.
- Retain only the artifact/report/evidence records useful to the caller.
- Treat source-derived data as sensitive; do not rely on a sanitized debug bundle.

View File

@@ -0,0 +1,143 @@
# Migrating To Native Configuration And Commands
This guide maps retired Notarius interfaces to the implemented native workflow.
It is migration material: the old spellings below are not accepted aliases.
[Configuration](../config.md), [CLI Reference](../cli.md), and
[Operations](../operations.md) own the current contracts. Domain artifact schemas
and lane filename mapping are unchanged.
## Update Caller Arguments
| Former interface | Current equivalent / intent |
| --- | --- |
| `notarius --version` | `notarius version`. |
| `pipelines list` | `pipeline list`. |
| `config validate --pipeline ID` | `pipeline validate ID`; native `config validate` checks the application document. |
| `config effective` | `config show`; scopes are `app` and `promptkit`. |
| Positional `config explain POINTER` | `config explain --path POINTER`; use native list/history options for discovery. |
| `config profile ID` | `promptkit show profile QUALIFIED_ID`; this reports stored fields, not a copied completed target. |
| `--config-overlay NAME` | `--overlay NAME`. |
| Run `--input PATH` | `--input-file PATH`. |
| `--only a,b` | Repeat literal `--lane a --lane b`; commas are no longer separators. |
| `--json` | `--format json`; YAML/text are also native report formats. |
| `--session-id ID` | `--llm-session-id ID`. |
| `--reasoning-effort VALUE` | `--llm-reasoning-effort VALUE`. |
| `--clear-reasoning-effort` | Explicit `--llm-reasoning-effort=`; omission retains native inheritance. |
| `--resume`, `--recompute-step`, separate `--chunk_cache`/`--chunk-cache` | Ordinary accepted-result reuse, with invocation-wide `--cache auto\|bypass\|refresh`. No explicit resume or selective checkpoint barrier. |
`run` and `pipeline run` are two native routes to the same command. Put flags
after the complete command path. Singleton occurrences are all parsed, with the
last successfully parsed value retained. False/zero/empty supplied values are not
omission. Canonical `--llm-*` execution overrides reach producers, correction,
and validators; shorter ckprompt aliases apply only to their installed PromptKit
commands. Help works offline on subcommands.
## Migrate Configuration And Resources
Version 6 is the sole application version; bumping the header alone does not
translate old fields. Use the maintained
[minimal](../../examples/dnd-minimal/config.yml) or
[complete](../../examples/dnd-complete/config.yml) configuration as a starting point.
- Replace `promptkit.profile_file`, `profile_dir`, and `local_backend` wrappers
with native settings beneath the main file's `promptkit`, or wrapperless
`prompts.yml` / recursive `prompts/` companions beside that selected main file.
Companion files form strict peers, not filename-priority layers.
- Use identified `backends`, `profiles`, and `prompts` lists with literal `id`
values and collections. The embedded profile is
`notarius/dnd-extraction`; initial/correction prompt IDs are also qualified,
for example `notarius/dnd.spells` and `notarius/dnd.spells.repair`.
- The root bundled backend is `openrouter`. Retired external provider catalogs
are not loaded as parallel sources; declare needed backends/profiles through
native resources. Do not infer backend identity from an endpoint or model.
- Use native root `config` imports and preserve declaration-relative paths.
Remove old per-prompt resource-version wrappers such as `v1`; resource IDs no
longer select those versions.
- A later same-ID resource replaces the whole earlier record. Use a distinct
sibling with `extends: {id: parent}` to retain fields, not a partial same-ID
replacement or self-inheritance. Profile/backend references to collection
resources use qualified IDs; sibling inheritance is local to its list.
- Preserve deliberate null/empty/zero choices under native semantics. Null
endpoint/key-environment values can allow backend fallback; empty strings can
clear those sources. Inspect native effective values/history rather than
assuming old wrapper null behavior.
- Keep prompt-declared schemas. Inline schemas, source-relative `schema_file`,
and explicit-root `schema_path` are native alternatives. Replacing a collection
schema root does not retain missing-file fallback to embedded resources.
Review [native resource and schema rules](../config.md#promptkit-profiles) before
reusing an old profile file. The complete example's
[companion](../../examples/dnd-complete/prompts.yml) demonstrates current inheritance.
## Separate Retry Intent, Mode, And Capacity
Producer `retries` intended for semantic correction becomes
`max_additional_turns`, excluding the initial response. Binding overrides pipeline;
omission inherits and explicit zero disables semantic correction. Native growing
continuation replaces latest-only manual replay, and each revised candidate gets
the full applicable assessment chain, including its final permitted turn.
Remove validator `retries`, generic deterministic retries,
`structured_output_repair_attempts`, and the application `promptkit.transient_retries`
wrapper. PromptKit owns transient failures and structural repair. Run-wide
transient customization uses `--llm-transient-retries`; structural customization
belongs in native prompt output contracts. Bundled prompts leave repair counts
unspecified. Former total provider-attempt parity is not promised.
Replace failure-category `validation_policy` and its `fail_run`, `reject_output`,
and `warn_continue` choices with producer `mode: strict|permissive`. Strict is
the default, including normalization. Permissive explicitly requests a supported
module fallback, not incomplete ordinary acceptance. Cancellation does not fall
back; unsupported fallback construction fails. Completion discloses actual
fallback emitters independently of warning visibility.
Replace `concurrency.total_llm` and `concurrency.stage_workers.extract` with
`concurrency.workers`, covering ready extraction and lane completion together.
Choose native backend capacity independently. There is no cross-backend global
LLM ceiling, inferred per-backend translation, or extra capacity queue/retry.
The [configuration reference](../config.md#concurrency-output-cache-and-debug)
owns current defaults and controls.
## Replace Both Legacy Caches
Replace nested `cache.chunk_plans` / `cache.checkpoints` directories and recording
switches with root `cache_directory` and pipeline/binding `cache`. An explicit
invocation `--cache` overrides every binding. Auto reads/stores acceptance;
bypass does neither; refresh skips reads and replaces only after new acceptance.
The initial prepared native message-content hash selects final accepted artifacts
and LLM plans. A hit is decoded/usability-checked, not semantically revalidated.
Model/settings/policy changes can still hit; use refresh/bypass for experiments
requiring fresh work. Source identity alone no longer fixes chunk boundaries.
Fallback and actually affected consumers skip reuse; unaffected predecessors
remain reusable after downstream failure. Failed refresh preserves old acceptance.
Do not copy, convert, or delete legacy records as an upgrade prerequisite. They
remain ignored and untouched, and the new cache populates through ordinary work.
[Operations](../operations.md#accepted-result-cache-and-recovery) owns recovery
and deployment-permission guidance.
## Replace Receipt And Sidecar Traversal
Stop expecting `notarius.run-result.v2`, `index_file`, validation histories,
warning counts, `index.json`, `manifest.json`, or reporting-only rejection/
warning/diagnostic files. Check process status, optionally decode the
[native completion report](../cli.md#completion-report), then read configured
lane/auxiliary [domain paths](../integrations/json-output.md).
Use the report's absolute run directory, not a guessed run ID or newest-folder
scan. Count actual deliverables rather than old normalized-output totals.
The report does not establish fresh validation or model quality; decide whether
positive fallback qualification is acceptable. Failed stdout can be partial,
and post-publication failure can leave valid artifacts with a nonzero status.
Debug remains explicitly selected and independent of native visibility modes.
Directory-only configuration does not enable it; explicit empty disables it.
The relative `latest` link updates at start, not completion. Full selected
evidence replaces the old summary/trace hierarchy; filenames/JSON shapes remain
private and arbitrary application content is not heuristically sanitized.
Do not replace receipt traversal with a debug-layout dependency or a new manifest.
Update callers using the [subprocess guide](subprocess.md) or
[complete D&D workflow](dnd-pipeline.md). These changes do not publish a new release
or rewrite historical release notes.

View File

@@ -1,115 +1,96 @@
# Using Notarius As A Subprocess
Use this workflow when an orchestrator runs Notarius and consumes its published
artifacts. The [CLI reference](../cli.md) owns invocation syntax and exit
statuses, while the [run-result receipt](../integrations/run-result.md) and
[Published JSON Output contract](../integrations/json-output.md) own the
durable result formats.
Use this workflow to run Notarius and consume domain files. [CLI Reference](../cli.md)
owns commands, reports, and statuses; [Published JSON Output](../integrations/json-output.md)
owns logical paths and payload links; [Operations](../operations.md) owns state
and recovery. See the [complete D&D workflow](dnd-pipeline.md) for its inventory
or [migration guidance](native-migration.md) for older callers.
For the maintained complete D&D workflow, including its transcript input,
configured lane inventory, and downstream acceptance checklist, see
[Consuming The Complete D&D Pipeline](dnd-pipeline.md).
## Prepare And Run
## Run And Check The Process
Use an argument vector with explicit paths appropriate to the deployment.
Absolute paths avoid dependence on a service's working directory. Keep stdout
and stderr separate, and choose caller cancellation/deadline behavior.
Optionally preflight a selected configuration and pipeline before work starts:
Validate the application and pipeline offline:
```sh
notarius config validate --config /path/to/notarius.yml --config-overlay production --pipeline pipeline-id
```
~~~sh
notarius config validate --config /absolute/path/to/notarius.yml
notarius pipeline validate pipeline-id \
--config /absolute/path/to/notarius.yml --overlay production
~~~
Omit `--config-overlay` when the composed base configuration is the intended
selection; it is independent from any `--llm-profile` run override.
Omit the overlay for the composed base. Pipeline validation needs no transcript,
reference bytes, credentials, provider call, or state directory.
Invoke the run with explicit paths and machine-readable output. Capture
standard output and standard error separately; do not combine them before
processing the result.
Then run with first-class source and output inputs:
```sh
~~~sh
notarius run pipeline-id \
--config /path/to/notarius.yml \
--config-overlay production \
--input /path/to/source.json \
--output-dir /path/to/output-root \
--json
```
--config /absolute/path/to/notarius.yml --overlay production \
--input-file /absolute/path/to/source.json \
--output-dir /absolute/path/to/output-root --format json
~~~
Use absolute paths for supplied input, configuration, output-root, and
reference files. Pass each external reference as its own argument-vector pair;
do not construct and invoke a shell command. An unqualified reference selector
supplies that file to every compatible selected target. Lane and stage
qualification are available for exceptional overrides, while generated
same-run references remain part of configured pipeline composition. The
[CLI reference](../cli.md#reference-selectors) owns the exact selector and
precedence contract.
Pass external references as separate arguments, for example
`--reference party=/absolute/path/to/party.txt`. Unqualified selectors reach
compatible selected targets; narrower selectors are exceptional overrides.
Generated same-run handoffs stay configured, not CLI files. The
[reference selector contract](../cli.md#reference-selectors) owns matching and
precedence.
The maintained D&D subprocess workflow uses this facility for campaign context:
Supply credentials through the intended process environment, not arguments,
session IDs, generated files, or logs. Omitted session selection uses the
input-based default; use the [native override](../cli.md#run-wide-llm-overrides)
only when intentionally changing it.
```sh
notarius run dnd-session \
--config /absolute/path/to/notarius.yml \
--input /absolute/path/to/transcripts/final.trimmed.json \
--output-dir /absolute/path/to/notarius-output \
--reference party=/absolute/path/to/references/party.txt \
--reference players=/absolute/path/to/references/players.txt \
--reference glossary=/absolute/path/to/references/glossary.txt \
--reference spell_catalog=/absolute/path/to/references/spells.json \
--json
```
## Check Completion Before Parsing
Only pass the external references available to and desired by the deployment.
Notarius generates a stable prompt session for the resolved input module and
exact input bytes; reference changes do not change it. Pass **--session-id**
only when intentionally grouping different invocations under a different
session. Supply credentials through Notarius's documented configuration and
environment mechanisms, never as command-line arguments or generated
secret-bearing configuration. In particular, a session identifier is
provider-visible and is not a credential mechanism.
Wait for completion. Only successful status permits interpreting stdout as a
successful native report. Failed delivery can leave partial bytes; retain stderr
and available locations rather than parsing a structured error envelope.
Wait for the process before interpreting standard output. Only an exit status
of 0 permits decoding the receipt. On a nonzero exit, retain standard error for
diagnosis and ignore all standard-output bytes: a failed receipt write may have
left a partial document.
Optionally decode the [small completion report](../cli.md#completion-report).
Its `output_directory` is the exact run root; do not scan for the newest folder
or guess a run ID. The report is not a persisted receipt, schema-version envelope,
index, or validation/execution ledger.
## Discover Required Artifacts
A completed run may contain supported fallback. The caller decides whether that
is acceptable; inspect positive `fallback_count` when relevant. It counts
emitting operations, not affected files. Reuse performs ordinary usability checks,
not fresh semantic assessment. Callers needing new inference/assessment after
policy changes deliberately select bypass/refresh under the
[cache controls](../config.md#producer-controls).
Decode the successful receipt and accept the schema versions supported by the
caller. Use its `output_directory` as the bundle root. For the production JSON
output, resolve `index_file` under that root with a confinement check and reject
an absolute path or a result that escapes the root.
## Read Required Domain Files
Read the resulting `index.json` and locate each artifact by `lane_id`, not by a
guessed filename. Before decoding a selected payload, verify its descriptor's
media type and schema identity against the relevant published artifact
contract. The JSON bundle contract links to the available lane contracts.
Define required/optional lanes from the selected pipeline. Read their
[documented paths](../integrations/json-output.md#bundle-layout) beneath the
report directory and decode using the configured artifact contracts. For custom
lane IDs, apply the unchanged filename mapping; collisions are command failures.
No receipt/index traversal, manifest descriptors, or reporting sidecars are needed.
If `index.json` has an `evidence_context` descriptor, treat it as a
pipeline-wide artifact rather than a lane entry. Verify its six descriptor
fields before decoding the linked file according to the [Published Evidence
Context contract](../integrations/evidence-context.md). Decode its top-level
source-unit array as a reading excerpt. Obtain authoritative citations and lane
provenance from the normalized lane artifacts; the excerpt has neither and its
nearby units do not widen a lane artifact's cited source reference.
Optional chunk-map/evidence files use their documented paths when enabled and
available. Decode supported payload schemas. Evidence is a reading excerpt, not
authoritative lane attribution or a widened citation; normalized lane payloads
retain those contracts.
A zero exit status may still report rejected outputs, warnings, or absent
lanes. The caller decides which lane IDs are required for its own work and
which are optional; it should make that decision explicitly rather than infer
failure from the receipt counts alone.
Process completion, required artifact availability/compatibility, and caller
acceptance requirements are separate checks. A count alone cannot identify
which files were produced.
When complete validation is required, also require receipt
`validation_status: approved` and inspect `validation_summaries`. A successful
run with `validation_status: incomplete` contains a structurally valid result
that advanced after validator execution could not complete under the configured
`warn_continue` policy. It is not reusable checkpoint state and should not be
silently treated as fully reviewed by the caller.
## Preserve Useful Evidence
## Preserve Provenance And Handle Data Carefully
Keep domain files and caller-owned invocation/report records useful for provenance.
Reports can be explicitly redirected; Notarius does not automatically persist a
second report or require a metadata bundle. Debug is optional private evidence,
not a consumer protocol.
Keep the receipt with the published `manifest.json`, and retain
`rejected.json`, `warnings.json`, and `diagnostics.json` when review or later provenance requires
them. Treat the input, output bundle, cache, debug bundle, and captured process
logs as potentially sensitive data. Apply the caller's access controls and
retention policy, and avoid copying secrets into arguments, logs, or
provenance records. An evidence-context artifact contains source-unit text and
metadata and can cover most of an input; preserve and share it only when that
source content is authorized for the recipient.
Post-publication capture/report failure can return nonzero while leaving valid
artifacts. Preserve available files for deliberate recovery, but do not promote
partial stdout or incomplete publication to ordinary consumer success.
[Operations](../operations.md#output-bundles) explains those boundaries.
Inputs, artifacts, evidence, cache, debug, and streams may contain sensitive data.
Apply intended access, sharing, and retention controls. Evidence can cover the
whole source and is not inherently less sensitive.

View File

@@ -40,7 +40,7 @@ implementing changes. It owns implementation discipline and local commit rules.
| --- | --- | --- |
| Finding the package or component that owns current behavior | [Internal Overview](internal/overview.md) | It is the implemented component inventory and routes to focused internals. |
| CLI composition or command behavior | [CLI Internals](internal/cli.md) and [CLI Reference](cli.md) | The internal guide owns composition and command flow; the reference owns public syntax. |
| Building a subprocess caller or changing its result protocol | [Subprocess Consumer Guide](consumers/subprocess.md), [Complete D&D Consumer Guide](consumers/dnd-pipeline.md), [Run Result Receipt](integrations/run-result.md), and [CLI Internals](internal/cli.md) | These separate generic caller workflow, the complete D&D workflow, the durable receipt contract, and CLI implementation behavior. |
| Building a subprocess caller or changing its result protocol | [Subprocess Consumer Guide](consumers/subprocess.md), [Complete D&D Consumer Guide](consumers/dnd-pipeline.md), [Native Completion Report](cli.md#completion-report), and [CLI Internals](internal/cli.md) | These separate generic caller workflow, the complete D&D workflow, the native completion report, and CLI implementation behavior. |
| Configuration loading, resolution, or user-visible configuration behavior | [Configuration Internals](internal/configuration.md) and [Configuration](config.md) | The internal guide owns loading and resolution mechanics; the reference owns the configuration contract. |
| Pipeline resolution or execution | [Pipeline Internals](internal/pipeline.md) | It documents profiles, references, validation, retries, checkpoints, and runner behavior. |
| Production modules or validators | [Module Internals](internal/modules.md), [D&D Module Internals](internal/dnd.md), and [D&D integration contracts](integrations/) | The generic guide owns extension mechanics, the D&D guide owns shared family conventions, and the contracts own durable output shapes. |

View File

@@ -1,9 +1,9 @@
# Accepted Chunk Map
This document defines the optional durable `chunk-map.json` artifact in a
[published JSON bundle](json-output.md). It describes the accepted,
materialized chunk plan used by one run. It is not a lane payload and is never
an input to a later pipeline step.
[published JSON bundle](json-output.md). It describes the materialized chunk
plan used by one run, including a supported qualified fallback. It is not a lane
payload and is never an input to a later pipeline step.
## Contract Identity
@@ -16,8 +16,8 @@ an input to a later pipeline step.
| Schema name | `notarius_source_chunk_map_v1` |
| Schema version | `v1` |
The optional `chunk_map` descriptor in `index.json` identifies this artifact.
Export is controlled by the JSON output binding described in
Read this logical path directly when enabled and available; no discovery
descriptor is required. Export is controlled by the JSON output binding in
[Configuration](../config.md#module-bindings-and-validators).
## Wire Shape
@@ -30,7 +30,7 @@ Every payload has these required fields:
| `source_digest` | Lower-case `sha256:` digest of that source document. |
| `plan_digest` | Lower-case `sha256:` digest of the logical chunk plan. |
| `requested_chunker` | Chunk module selected by the resolved pipeline. |
| `producer` | Original accepted-plan producer. `input_module` and `chunk_module` are required; `llm_profile` is optional. |
| `producer` | Current run's plan component selections. `input_module` and `chunk_module` are required; `llm_profile` is optional and applies only to an LLM-backed chunker. Not historical cache execution provenance. |
| `plan_annotations` | Plan-level annotation namespace map; `{}` when none are present. |
| `chunks` | Non-empty execution-order chunk collection. |
@@ -47,9 +47,11 @@ valid JSON and are retained without interpreting a module-specific namespace.
`chunks` are in execution order. Their indexes are contiguous, start at zero,
and equal their array positions; chunk IDs are unique. The emitted map is built
only after the selected plan has been accepted and materialized against the
source document, so its ranges, unit counts, annotations, and digests describe
that exact plan.
after the selected plan has been materialized against the source document,
so its ranges, unit counts, annotations, and digests describe that exact plan.
Supported fallback still satisfies those invariants without asserting ordinary
semantic approval; run-level qualification belongs to the
[CLI report](../cli.md#completion-report).
The codec rejects malformed JSON, trailing content, unknown fixed-object
fields, invalid identities or digests, invalid annotations, duplicate chunk
@@ -67,12 +69,13 @@ annotations, producer identity, and ordered chunks.
## Publication And Compatibility
The map is present only when a chunk plan was accepted and its export is
enabled. It remains publishable if a later lane is rejected, but is absent when
chunk-plan validation rejects the plan. `requested_chunker` identifies the
current pipeline selection, while `producer` identifies the component that
originally produced the accepted plan; they may differ when an accepted plan is
reused.
The map is published when export is enabled, a materialized plan is available,
and the run reaches publication. Payload fields and schema are unchanged.
`requested_chunker` and `producer` describe the current composition; unified
cache entries retain the logical plan, not an earlier run's producer/profile
ledger. Reuse rematerializes against the current source without semantic
reassessment or claims of historical model execution. A strict failure does not
publish a successful bundle merely to expose this auxiliary.
The map contains structure rather than source content: it excludes transcript
bytes, source-unit metadata, chunk text, private model output, reference

View File

@@ -9,9 +9,8 @@ logical-file discovery is owned by [Published JSON Output](json-output.md).
## Identity And Discovery
When enabled, the JSON bundle contains `evidence-context.json` and an
`index.json` `evidence_context` descriptor with the same six fields as other
pipeline-wide artifact descriptors.
When enabled and available, the JSON bundle contains `evidence-context.json`
at that documented logical path; no index descriptor is required.
| Property | Value |
| --- | --- |
@@ -22,16 +21,16 @@ pipeline-wide artifact descriptors.
| Schema version | `v1` |
| Logical file | `evidence-context.json` |
Consumers must discover the file from the descriptor, verify all six descriptor
fields, and decode only a supported schema version. The descriptor is optional:
its absence means evidence publication was not enabled for that bundle.
Consumers select this path from the configured output policy and decode the
supported payload contract below. Disabled or unavailable evidence is not
published. The identity and JSON shape above remain unchanged.
## Payload
The v1 payload is a top-level JSON array of generic source units. There is no
wrapper, source-level metadata, context grouping, lane identifier, or evidence
reference in the payload. An enabled configuration with no contributing
accepted evidence publishes `[]`.
evidence publishes `[]`.
```json
[
@@ -73,14 +72,16 @@ annotations or reshape source-owned metadata.
## Selection And Citations
The framework obtains direct source references only through typed evidence
projections of accepted normalized artifacts in the configured lane allowlist.
projections of published normalized artifacts in the configured lane allowlist,
including supported fallbacks and reused results.
It validates each reference against the current source document, expands its
range by `window_units` source-unit positions on each side, clamps at document
boundaries, and takes the union of all expanded ranges. The output contains
each selected source unit once in source-document position order, regardless
of numeric unit IDs. Repeated references, overlapping windows, and citations
from multiple lanes do not duplicate a unit. Rejected, failed, absent,
inactive, and unselected lanes contribute nothing.
from multiple lanes do not duplicate a unit. Failed, absent, inactive, and
unselected lane results contribute nothing. Evidence from a supported fallback
does not imply ordinary semantic approval; the completion report qualifies the run.
Normalized lane artifacts remain authoritative for citations and for which lane
cited a range. The excerpt has no lane attribution and must not be used to
@@ -94,10 +95,10 @@ the framework does not truncate the excerpt to meet an arbitrary size limit.
## Consumer Responsibilities And Data Handling
The artifact is additive to the JSON bundle and is not a lane payload,
normalized-output count, checkpoint, or generated reference. Consumers that
do not need it must tolerate an absent descriptor. Consumers that do use it
should validate the descriptor and payload before use, retain the artifact with
The artifact is additive to the JSON bundle and is not a lane payload or
generated reference. It counts as a deliverable when published, not as a lane.
Consumers that do not need it can ignore it. Consumers that do use it should
validate the payload before use, retain the artifact with
its schema identity when needed for a run record, and read citations from the
corresponding normalized lane artifacts.

View File

@@ -1,196 +1,79 @@
# Published JSON Output
This document defines the logical JSON bundle emitted by the production JSON
output encoder. The bundle’s physical destination, atomic publication, and
retention are operational concerns; see [Operations](../operations.md#output-bundles).
Output configuration, including chunk-map and evidence-context publication, belongs in
[Configuration](../config.md#module-bindings-and-validators).
This contract defines logical files emitted by the production JSON output
encoder. [Operations](../operations.md#output-bundles) owns physical placement,
publication failures, permissions, and retention. [Configuration](../config.md#module-bindings-and-validators)
owns output options; the [CLI completion report](../cli.md#completion-report)
identifies a successful run's directory.
## Bundle Layout
All paths below are logical, relative, slash-separated bundle paths. The
encoder always emits the first five JSON files below and adds lane or
pipeline-wide artifact files when their corresponding artifacts are available:
A subprocess caller first obtains the physical bundle root from the
[run-result receipt](run-result.md), then resolves `index.json` beneath that
root for the logical discovery described here.
Paths are relative, slash-separated paths beneath the run-specific directory:
| Path | Purpose |
| --- | --- |
| `index.json` | Entry point that names the other published files and lane payloads. |
| `manifest.json` | Run provenance and result summaries. |
| `rejected.json` | Rejected pipeline outputs. |
| `warnings.json` | Actionable process-degradation warnings. |
| `diagnostics.json` | Accepted-artifact quality advisories and normalization observations. |
| `lanes/<safe-lane-id>.json` | One normalized artifact payload for each lane. |
| `chunk-map.json` | Optional accepted chunk map, when its export is enabled and available. |
| `evidence-context.json` | Optional selected source-unit excerpt, when evidence publication is enabled. |
| `lanes/<safe-lane-id>.json` | One normalized domain artifact per available selected lane, including supported qualified fallback. |
| `chunk-map.json` | Optional materialized chunk map, when enabled and available. |
| `evidence-context.json` | Optional selected source-unit excerpt, when enabled and available. |
JSON files are pretty-printed with a trailing newline. Lane payloads are
accepted only when their media type is `application/json`.
There is no required index, manifest, persisted receipt, rejection file, warning
file, or diagnostic sidecar. Expected lanes come from pipeline configuration;
optional auxiliaries use the paths above. Consumers need no metadata bundle or
debug data to decode artifacts.
## `index.json`
The lane filename mapping is unchanged: trim the ID, replace characters outside
ASCII letters, digits, periods, underscores, and hyphens with underscores,
neutralize `..` sequences, and trim leading/trailing periods and underscores.
An empty resulting name or colliding paths fails encoding. Maintained D&D lane
IDs already map directly to their filenames.
`index.json` is the bundle’s discovery document. An approved run with no
normalized lanes has this valid minimal index:
```json
{
"manifest_file": "manifest.json",
"output_files": [],
"rejected_file": "rejected.json",
"warnings_file": "warnings.json",
"diagnostics_file": "diagnostics.json"
}
```
| Field | Required | Meaning |
| --- | --- | --- |
| `manifest_file` | Yes | Always `manifest.json`. |
| `output_files` | Yes | Lane descriptors sorted by `lane_id`. |
| `rejected_file` | Yes | Always `rejected.json`. |
| `warnings_file` | Yes | Always `warnings.json`. |
| `diagnostics_file` | Yes | Always `diagnostics.json`. |
| `chunk_map` | No | Descriptor for the pipeline-wide `chunk-map.json`; never a lane descriptor. |
| `evidence_context` | No | Descriptor for the pipeline-wide `evidence-context.json`; never a lane descriptor. |
Each lane descriptor has required `lane_id` and `file`. It may also include
`media_type`, `module_key`, `schema_id`, `schema_name`, and `schema_version`
when supplied by the normalized artifact. Each pipeline-wide artifact
descriptor (`chunk_map` or `evidence_context`) contains `artifact_kind`,
`file`, `media_type`, `schema_id`, `schema_name`, and `schema_version`. Their
payloads are defined by the [Accepted Chunk Map contract](chunk-map.md) and
[Published Evidence Context](evidence-context.md), respectively.
The lane path is derived from its lane ID. Characters outside letters, digits,
periods, underscores, and hyphens become underscores; `..` sequences are
neutralized; leading and trailing periods and underscores are removed. A lane
that produces an empty name, or two lanes that produce the same path, makes
output encoding fail.
Files are pretty-printed JSON with a trailing newline. Lane payloads require
`application/json`; the encoder validates the artifact boundary without
wrapping payloads in reporting metadata. Report presentation format does not
change artifact encoding.
## Lane Payloads
Each `lanes/<safe-lane-id>.json` file is the codec-owned normalized JSON for
that lane. Consumers should use the index descriptor’s schema identity rather
than infer a lane schema from its name. The current D&D payload contracts are
[spells](dnd-spell-artifacts.md), [NPC registry](dnd-npc-registry-artifacts.md),
[NPC occurrences](dnd-npc-occurrence-artifacts.md),
[combat turns](dnd-combat-turn-artifacts.md),
[item registry](dnd-item-registry-artifacts.md),
[item occurrences](dnd-item-occurrence-artifacts.md),
[scene descriptions](dnd-scene-description-artifacts.md),
[enemy events](dnd-enemy-event-artifacts.md),
[location registry](dnd-location-registry-artifacts.md), and
[location occurrences](dnd-location-occurrence-artifacts.md).
Each lane file is its codec-owned normalized JSON value. Select the contract
from the configured lane/module, not an absent discovery descriptor. Existing
D&D payload contracts remain unchanged:
## `manifest.json`
- [Spells](dnd-spell-artifacts.md).
- [NPC registry](dnd-npc-registry-artifacts.md) and
[NPC occurrences](dnd-npc-occurrence-artifacts.md).
- [Combat turns](dnd-combat-turn-artifacts.md).
- [Item registry](dnd-item-registry-artifacts.md) and
[item occurrences](dnd-item-occurrence-artifacts.md).
- [Scene descriptions](dnd-scene-description-artifacts.md).
- [Enemy events](dnd-enemy-event-artifacts.md).
- [Location registry](dnd-location-registry-artifacts.md) and
[location occurrences](dnd-location-occurrence-artifacts.md).
`manifest.json` is published provenance, not a copy of lane payloads or a
checkpoint store. Fields without a value may be omitted. Its top-level fields
group into the following externally observable summaries:
Intrinsic fields, identities, source references, and schemas are preserved.
Supported fallback meets intrinsic/downstream invariants without asserting
ordinary semantic approval. Run qualification belongs in the native completion
report, not new payload fields or sidecars.
| Group | Fields |
| --- | --- |
| Run identity and result | `run_id`, `pipeline_id`, `pipeline_digest`, `schema_version`, `validation_status`, `started_at`, `completed_at` |
| Resolved components | `input_module`, `chunker`, `extractors`, `merger`, `normalizer`, `output_encoder`, `artifact_lanes`, `validator_chains`, `module_metadata` |
| Source and references | `source_digests`, `references` |
| Published result summaries | `normalized_outputs`, `rejected_outputs`, `validation_summaries` |
| Execution summaries | `chunk_plan`, `checkpoint_decisions`, `llm_profiles`, `metadata` |
## Auxiliary Artifacts
`references` records provenance such as the target, slot, origin, digest,
media type, size, and generated-artifact identity. It does not contain
reference content. `normalized_outputs` and `rejected_outputs` likewise
summarize results without embedding lane payload bytes. A chunk-plan summary is
provenance for the plan used by this run; cache records, debug artifacts, and
other operational state are not published as bundle files.
[Chunk map](chunk-map.md) and [evidence context](evidence-context.md) are
pipeline-wide artifacts, not lanes. Their existing payload schemas and logical
paths remain; no index descriptor is required.
Each `validation_summaries` entry is a bounded outcome for one producer result.
It has required `status`, `producer_attempt_count`, and `terminal_action`;
the stage and affected step, lane, module, or chunk identity are present when
applicable. `status` is `complete`, `rejected`, or `incomplete`.
`rejecting_validators`, `reason_codes`, and `incomplete_validators` preserve
configured validator order and omit later duplicates. Entries contain no raw
candidate response, correction guidance, validator diagnostic message, or
artifact payload. The same shape may appear as `validation` on an affected
rejection entry.
Enabled, available auxiliaries count as deliverables, including when built
from reused results or supported fallback. Disabled/unavailable auxiliaries are
not published. Evidence can cover the complete source; authoritative citations
remain in normalized lane payloads.
When present, `metadata.session_id` is the effective non-secret routing
correlation identifier used for the run. It can be visible to providers and is
not a substitute for a cache or checkpoint identity. Its generation and
override behavior are defined by the [CLI reference](../cli.md#run).
## Consumer Use And Compatibility
Each `llm_profiles` entry identifies effective, non-secret LLM execution
provenance:
Check process status before decoding a successful native report. Use its absolute
run directory, then the configured lane and selected auxiliary paths. Decode
each file against its domain contract. The [subprocess guide](../consumers/subprocess.md)
and [complete D&D workflow](../consumers/dnd-pipeline.md) explain caller tasks.
| Field | Required | Meaning |
| --- | --- | --- |
| `id` | Yes | Selected PromptKit profile identifier. |
| `provider` | No | Notarius adapter provider identifier. |
| `model` | No | Effective provider model identifier. |
| `backend_id` | No | Effective PromptKit backend registration identifier. Endpoint-only profiles omit it. |
| `reasoning_effort` | No | Effective opaque provider reasoning setting. An empty or explicitly cleared setting is omitted. |
These values describe observed execution; they are not a backend-registration
interface. Entries that differ by backend or effective reasoning remain
distinct even when their profile, provider, and model are otherwise equal.
## Rejections, Warnings, And Diagnostics
`rejected.json` is always an object with a `rejected` array. Each entry has
required `stage` and `message`; `step_id`, `lane_id`, `module_key`, `chunk_id`,
`chunk_index`, `validator_name`, `reason_code`, `attempt_count`, and
`diagnostic_artifact_path` are present only when applicable. An entry may also
contain the bounded `validation` summary described above; the existing singular
validator and reason fields remain the first configured rejection for
compatibility.
`warnings.json` is always the `notarius.warnings.v2` envelope:
```json
{
"schema_version": "notarius.warnings.v2",
"group_count": 0,
"occurrence_count": 0,
"groups": []
}
```
It contains only process warnings. `group_count` is exact, and
`occurrence_count` is the exact sum of its group occurrence counts.
`diagnostics.json` is always the `notarius.diagnostics.v1` envelope:
```json
{
"schema_version": "notarius.diagnostics.v1",
"group_count": 0,
"occurrence_count": 0,
"truncated": false,
"unrepresented_occurrence_count": 0,
"groups": []
}
```
It contains only advisory and observation groups. `group_count` counts groups
represented in `groups`; `occurrence_count` includes both represented and
unrepresented occurrences. When `truncated` is true,
`unrepresented_occurrence_count` is the exact number omitted from group
representation.
Each group has `disposition`, `category`, `reason_code`, framework-owned
`origin`, exact `occurrence_count`, bounded `samples`, and
`omitted_sample_count`. Samples carry safe `scope` and `message`, plus a chunk
ID and zero-based chunk index when applicable. A group retains at most three
distinct samples. The framework fails rather than truncating actionable
warnings beyond 128 groups; it represents at most 256 advisory/observation
groups and records further occurrences through the diagnostic truncation
fields above.
## Compatibility
The index is the authoritative map from a logical lane to its published
payload. Consumers must tolerate omitted optional manifest and descriptor
fields, and should rely on the linked artifact contract for each lane’s JSON
shape. This contract describes the published logical bundle only; it does not
promise a filesystem layout or expose internal state formats.
Removing reporting/discovery files is intentional, not a domain schema or
filename-mapping change. [Migration guidance](../consumers/native-migration.md)
replaces receipt/index traversal. Debug and cache are not public output contracts.
Post-publication failure can leave valid files; failed publication can leave a
partial directory. Neither supplies a successful completion report.

View File

@@ -1,93 +0,0 @@
# Run Result Receipt
`notarius run --json` writes this receipt to standard output when a run
completes successfully. It lets a subprocess caller discover the physical root
of the published output bundle without parsing interactive command output.
Command syntax, streams, and exit statuses are defined in the
[CLI reference](../cli.md); logical files within the bundle are defined in the
[Published JSON Output contract](json-output.md).
## Schema
The current schema version is `notarius.run-result.v2`.
| Field | Required | Meaning |
| --- | --- | --- |
| `schema_version` | Yes | Exactly `notarius.run-result.v2`. |
| `run_id` | Yes | The finalized Notarius run identifier. |
| `pipeline_id` | Yes | The effective pipeline identifier. |
| `output_directory` | Yes | Absolute path to the published, run-specific output bundle. |
| `index_file` | For the production JSON output | Logical path `index.json`; omitted for other output modules. |
| `normalized_output_count` | Yes | Number of final normalized outputs. |
| `rejected_output_count` | Yes | Number of recorded rejected outputs. |
| `warning_group_count` | Yes | Exact number of actionable warning groups. |
| `warning_occurrence_count` | Yes | Exact occurrences represented by actionable warning groups. |
| `diagnostic_group_count` | Yes | Number of represented advisory and observation groups. |
| `diagnostic_occurrence_count` | Yes | Advisory and observation occurrences, including unrepresented occurrences. |
| `diagnostics_truncated` | Yes | Whether advisory/observation group representation was truncated. |
| `validation_status` | Yes | The final run manifest validation status. |
| `validation_summaries` | No | Bounded per-producer validation outcomes; present when producer work ran. |
| `debug_directory` | No | Absolute path to the run-specific debug bundle when requested debug capture completed. |
For the production `json` output module, `index_file` is present only when the
completed run returned exactly one logical output file named `index.json`.
For another output module, its absence does not indicate a failed run.
`validation_status` is `approved`, `rejected`, or `incomplete`; `incomplete`
means one or more otherwise accepted results advanced under validator-failure
`warn_continue`.
```json
{
"schema_version": "notarius.run-result.v2",
"run_id": "run-1770000000000000000-0123456789abcdef0123456789abcdef",
"pipeline_id": "dnd-session",
"output_directory": "/work/results/run-1770000000000000000-0123456789abcdef0123456789abcdef",
"index_file": "index.json",
"normalized_output_count": 6,
"rejected_output_count": 2,
"warning_group_count": 1,
"warning_occurrence_count": 2,
"diagnostic_group_count": 3,
"diagnostic_occurrence_count": 5,
"diagnostics_truncated": false,
"validation_status": "incomplete",
"validation_summaries": [
{
"stage": "extract",
"lane_id": "spells",
"status": "incomplete",
"incomplete_validators": ["dnd/spells/source_refs"],
"producer_attempt_count": 1,
"terminal_action": "warn_continue"
}
]
}
```
## Paths And Bundle Discovery
`output_directory` and `debug_directory`, when present, are lexical absolute
paths. They identify the paths used by Notarius and do not resolve symlinks.
`output_directory` is the run-specific bundle, not the configured output root.
The receipt is a summary and discovery document. Its optional validation
summaries contain only stable status, identity, validator names, reason codes,
attempt counts, and terminal actions. It does not contain lane descriptors,
payloads, manifest payloads, rejection messages, warnings, raw model responses,
correction guidance, or file contents.
For the production JSON output, resolve `index_file` beneath
`output_directory`, reject path escapes, and use the
[Published JSON Output contract](json-output.md) to discover logical files and
lane payloads.
## Delivery And Compatibility
Notarius writes the receipt only after the output bundle has been published and
any requested debug terminal reporting has completed. Standard output is not
transactional: a result-write failure returns a nonzero status and can leave
partial bytes. Consumers must ignore standard output unless the process exits
with status 0.
Future versions may add optional fields to this schema. Consumers must tolerate
unknown fields. An incompatible field or semantic change requires a new
`schema_version` value.

View File

@@ -152,7 +152,7 @@ reporting remains successfully terminalized. The CLI reports a bounded
command-owned error and does not repeat terminal reporting. The receipt remains
a CLI reporting concern rather than a framework or output-module responsibility;
its public contract is the
[run-result receipt](../integrations/run-result.md).
[native completion report](../cli.md#completion-report).
## Failure Mapping And Terminal Reporting

View File

@@ -1,475 +1,292 @@
# Operations
This is the canonical guide for operating Notarius runtime state. The
[CLI reference](cli.md) owns command syntax and exit statuses, while
[Configuration](config.md) owns fields, defaults, and precedence. Maintainers
who need implementation mechanics should read [Run State Internals](internal/state.md).
This guide owns runtime filesystem layout, publication, cache recovery, capture,
provisioning, and retention. [CLI Reference](cli.md) owns invocation and reports;
[Configuration](config.md) owns settings/defaults and source selection.
The [native migration guide](consumers/native-migration.md) maps retired interfaces
to the current workflow.
## Source Deployment
Linux is the supported deployment platform. Install a pinned source release
with the Go version declared in `go.mod` (currently Go 1.27.0):
Linux is supported; macOS is best-effort for development and Windows is
unsupported. Install a pinned source tag with the Go version required by
`go.mod`:
~~~sh
GOWORK=off go install \
gitea.maximumdirect.net/eric/notarius/cmd/notarius@vMAJOR.MINOR.PATCH
notarius version
~~~
Pin the exact tag in deployment automation rather than following a branch.
Use [`notarius --version`](cli.md#command-summary) as a diagnostic after
installation; its syntax and semantics are owned by the [CLI reference](cli.md).
The maintainer publication process, including tag guards and verification,
belongs to [Source Releases](release.md). macOS builds are best-effort for
development, and Windows is unsupported.
Dependency acquisition/private-module access is separate from offline validation.
[Source Releases](release.md) owns maintainer publication commands. This workflow
does not prepare or authorize a release.
## State Surfaces
Each run can use independent roots with different retention and access-control
needs.
Output, cache, and debug have independent roots and lifecycles.
| Surface | Purpose | Created when | Retention |
| --- | --- | --- | --- |
| Output | Durable user-facing result bundle | A pipeline completes and returns logical output files | Keep until consumers no longer need it. |
| Chunk-plan cache | Reconstructible source-addressed plan | The configured cache mode permits cache I/O | Keep while reuse is useful. |
| Checkpoint cache | Reconstructible execution and recovery state | Checkpoint recording is enabled | Keep only while recovery or reuse is useful. |
| Debug bundle | Explicit diagnostic record | A run requests debug collection | Keep only under an intentional sensitive-data retention policy. |
| Surface | Purpose | Allocation / reuse |
| --- | --- | --- |
| Output | Durable domain artifacts | A successful runner result publishes into a new run directory. |
| Accepted-result cache | Reconstructible artifacts and LLM chunk plans | Eligible model-backed work reads/writes according to its resolved mode. |
| Debug | Explicit run/operation evidence | Explicit capture selection with a nonempty effective destination. |
Output, cache, and debug roots are never merged or cleaned automatically. Use
separate locations and permissions for operators or services that must not
share application data.
Directories do not imply one another's enablement or permissions. Configure
separate roots when access or retention needs differ. Notarius performs no
automatic output/debug deletion or cache collection.
## Roots And Permissions
The configured output and debug directories are exact roots. An empty cache
directory selects a per-user root:
Publication and capture use run-specific directories:
~~~
<os.UserCacheDir>/notarius/chunk-plans
<os.UserCacheDir>/notarius/checkpoints
~~~text
<output-root>/<run-id>/
<debug-root>/<run-id>/
<debug-root>/latest -> <run-id>
~~~
The field definitions and configuration examples are in [Configuration](config.md).
On supported Unix systems, output directories and files are created with
requested modes **0755** and **0644**; debug directories and files use
**0700** and **0600**. The operating system's umask may impose stricter output
or debug modes. Chunk-plan and checkpoint caches instead request maximum modes
of **0777** for new directories and **0666** for new files, including atomic
temporary files. The kernel applies the service's umask and applicable default
ACLs when those entries are created.
The unified cache uses its selected exact root independently of run IDs:
## Cache Provisioning
~~~text
<cache-directory>/<opaque-message-content-hash>.json
~~~
Cache locations select where Notarius writes, not who can access them. A
private service can use a restrictive umask:
Omitted/empty cache selection uses the per-user location defined in
[Configuration](config.md#file-shape-and-defaults), normally under Linux's
`XDG_CACHE_HOME` or `~/.cache`. `/var/cache/notarius` is the convention for
an explicitly configured system-wide installation, not an automatically selected
or provisioned root. Output/debug defaults, empty values, and path origins also
belong in that reference.
Output creation requests directory/file modes `0755`/`0644`; debug requests
`0700`/`0600`. The operating system may restrict effective access. Cache creation
requests conventional maxima `0777`/`0666`, including atomic temporary files,
so deployment umask, setgid inheritance, and default ACLs select effective access.
Notarius does not converge existing ownership or modes.
### Cache Provisioning
A private deployment can use a restrictive process umask. A trusted-group
deployment may provision a setgid root or default ACL. These are deployment
choices, not requirements imposed on every operator:
~~~sh
umask 0077
notarius run example --config /etc/notarius/config.yml --input /srv/input.txt
~~~
A trusted-group deployment can provision a setgid cache root and use a
group-oriented umask (or an equivalent default ACL):
~~~sh
install -d -m 2770 -g notarius-operators /srv/notarius/chunk-plans /srv/notarius/checkpoints
# Example provisioning by an authorized administrator.
install -d -m 2770 -g notarius-operators /var/cache/notarius
umask 0007
notarius run example --config /etc/notarius/config.yml --input /srv/input.txt
~~~
Setgid and default ACLs control inherited group ownership and access policy;
Notarius does not copy a parent directory's literal mode bits. Cache state is
reconstructible but can contain source-derived data, so operators own its
access and retention policy. Existing cache inodes retain their prior modes
after an upgrade. Recreate a cache beneath correctly provisioned parents, or
stop writers and perform a deliberate administrative migration when a
whole-tree change is required.
Configure the cache root to use that directory. Setgid/default ACLs govern
inheritance; Notarius does not copy parent literal mode bits, manage accounts/
groups, or chmod/chown old entries. If existing access is unsuitable, operators
can recreate reconstructible state beneath provisioned parents or deliberately
perform an administrative migration.
Cache publication remains atomic and stored records remain validated. Those
guarantees do not add cross-process locking or transactional coordination for
simultaneous writers to one checkpoint identity.
Output bundles can also contain source content when JSON output enables
evidence publication. Apply an appropriate umask and output-root access policy
before enabling that option; the requested output modes alone may not be
suitable for transcript-bearing bundles.
Cache, outputs, and evidence excerpts may contain source-derived data. Select
access and retention accordingly; requested modes alone are not a sharing policy.
Existing confined-path, symlink, atomic-write, and validation boundaries do not
promise cross-process transactions or power-loss durability.
## PromptKit Profile Deployment
Profile deployment has four distinct layers:
One selected application load includes bundled provider resources, embedded
Notarius resources, main-relative operator companions, imports, and an optional
overlay. Resource replacement/inheritance and source origins are defined in
[Configuration](config.md#promptkit-profiles).
| Layer | Owner | Operational role |
| --- | --- | --- |
| Prompts and schemas | Notarius module families, with optional operator prompt overlays | Embedded request and structured-output definitions; operator configuration may recursively overlay prompt fields. Schemas remain application assets. |
| Fallback profiles | Notarius module families | Embedded application defaults, including D&D's `dnd-extraction` profile. |
| Built-in profiles | PromptKit | Upstream catalog entries available when no higher-precedence source defines an ID. |
| Operator configuration | Deployment filesystem | Native profile/backend definitions and prompt overlays selected by `promptkit.profile_file` or `promptkit.profile_dir`. |
Preserve the [complete example's](../examples/dnd-complete/config.yml)
declaration-relative layout or deliberately update paths. Its
[companion](../examples/dnd-complete/prompts.yml) adds native profile/backend
variants; minimal needs no operator companion. Select qualified collection IDs.
Supply credential values through the intended environment, not files, arguments,
session IDs, or logs.
The maintained D&D pipeline uses the workload ID `dnd-extraction`. The
embedded fallback makes that ID usable without an operator file. Production,
development, and local deployments can each overlay settings for the same ID,
retaining inherited values they do not change. Notarius's registered fallback
configuration precedes the selected operator source, and groups compose by
native recursive overlays; a configured Notarius local backend is applied
after the operator source. The configuration fields and source precedence are
owned by [Configuration](config.md#promptkit-profiles).
Use a profile source owned by the service account, keep it readable only by
the intended operator, and supply provider credentials through the service
environment—not in the Notarius configuration or profile YAML. The maintained
[LLM profiles](../examples/llm-profiles/dnd-extraction.yml) are secret-free
and can be copied as a format starting point. Validate a deployment without a
provider call or credentials:
Review the deployment before model work:
~~~sh
notarius config validate --config /etc/notarius/config.yml --pipeline dnd-session
notarius config validate --config /etc/notarius/config.yml
notarius pipeline validate dnd-session --config /etc/notarius/config.yml
notarius config show --scope promptkit --config /etc/notarius/config.yml
notarius promptkit show profile notarius/dnd-extraction \
--config /etc/notarius/config.yml --format json
~~~
An unset optional `api_key_env` reaches the provider without authorization and
may receive a 401 or 403 response.
Offline preflight/inspection needs no credential values or provider calls.
Use sources/explain reports for provenance and prepared preview for rendered
request details. Profile inspection describes stored fields, not a completed
execution target. New invocations observe source edits; captured engines do not
reread configuration/message files. Schema/input reads occur during preparation.
Configured PromptKit profile paths resolve from the file that supplies their
surviving declaration. The complete example's
`./llm-profiles/dnd-extraction.yml` path is therefore relative to that example.
Use absolute paths such as `/etc/notarius/profiles/dnd-extraction.yml` for
services and containers when that is clearer for the deployment.
### Inspect Before Deployment
Use the three configuration reports together when reviewing an operator
source:
~~~sh
notarius config effective --scope promptkit --config /etc/notarius/config.yml
notarius config explain --scope promptkit --config /etc/notarius/config.yml \
/profiles/dnd-extraction/model
notarius config profile --config /etc/notarius/config.yml dnd-extraction
~~~
Each report command constructs an inspection engine from the selected
configuration and registered application assets. Construction validates and
captures the effective definitions; after editing a source, run the command
again to inspect a newly constructed engine. These commands do not resolve
credential values, call a provider, or prepare a request. `config profile`
reports the named profile's resolved target and effective credential metadata,
not whether a credential is available or which profile a particular prompt or
run will select.
Effective and explanation reports can include ordinary inline prompt and
provider-parameter data. Credential-shaped provider parameter keys are
redacted, but the rest of the content is not treated as secret-free. Reports
are written only when requested; Notarius does not automatically persist them
in run output or debug bundles. Operators own their destination, access, and
retention when redirecting or storing reports.
Configuration reports are distinct from a prepared execution or a requested
debug trace. They describe captured definitions and source history, not
rendered request messages, runtime credential values, or an individual
provider interaction. The [CLI reference](cli.md#config-effective) documents
the report syntax and output contracts.
Reports can contain ordinary prompt/provider data. Secret-aware inspection and
native credential omissions do not make arbitrary content safe to share.
Operators select destinations, access, and retention when redirecting reports.
## Run Lifecycle
Use the [run command](cli.md#run) to start a pipeline. A valid invocation loads
and resolves configuration before module preparation and source parsing. It
then performs any permitted cache lookup, executes the pipeline, and publishes
logical output files only after a successful runner result.
A run loads configuration and obtains its identity. Requested capture allocates
before engine construction/preflight. The run resolves/preflights, materializes
references, prepares modules, parses source, performs chunk/lane work with
eligible reuse, encodes domain files, and publishes into a new run directory.
Requested capture finalizes before successful reporting.
On success, the command reports the output bundle path. A run with actionable
process warnings still succeeds and reports warning-group and occurrence counts
on standard error; advisory and observation findings do not produce a warning
line. Errors and
their exit classes are defined in the [CLI reference](cli.md#output-streams-and-exit-statuses).
Argument/configuration failures may have no capture. Later failures retain
available evidence. Publication, required capture, and presentation are separate
boundaries: later failure does not undo completed publication.
[CLI Reference](cli.md#completion-report) owns the completion value and outcomes.
## Validation Retries And Terminal Outcomes
### Acceptance And Fallback
When debug recording is enabled, per-call response artifacts include the
terminal execution summary for retry, repair, timing, and presence-aware usage
diagnosis. Normal output, receipts, and warnings do not aggregate or publish
this diagnostic metadata.
PromptKit owns transient retries and structural repair. Application semantic
continuation carries a growing conversation and assesses each usable candidate
through the complete chain, including the final allowed response. No outer
producer/validator retry loop extends native budgets.
Each producer binding has one outer **retries** budget. It covers complete
producer attempts for operational failures, invalid structured output,
normalizer fallback retry, and semantic correction. It is independent from
PromptKit's structural-repair calls inside one completion, its configured
`promptkit.transient_retries` built-in HTTP attempts, and an LLM-backed
validator's own retry budget. The transient setting defaults to two additional
attempts for eligible provider statuses; set it explicitly to zero when an
operator needs one HTTP attempt per generation. A semantic correction rebuilds
the ordinary producer request and supplies only the latest rejected model
response plus aggregated validator guidance; it is not a conversation replay.
Explicit permissive handling may publish a supported qualified fallback, not
ordinary semantic acceptance. Construction still checks intrinsic artifact,
source, and downstream invariants. Failure to construct fallback fails normally;
cancellation never initiates it. D&D scene planning can fall back to one
whole-source chunk, preserving coverage without promising bounded downstream
request size.
These budgets multiply only by their declared scopes. With two outer producer
retries, two transient retries, and one structural repair, a single producer
can issue up to 18 HTTP attempts: three producer attempts, each with an initial
generation and one repair generation, each allowing three HTTP attempts. This
is an illustration, not a total-deadline or exactly-once guarantee. Every
built-in HTTP attempt has its own generation timeout; caller cancellation
remains the overall authority and interrupts Retry-After waiting. The shared
Notarius scheduler keeps its permit through those waits, so retry budgets can
increase latency and provider cost. Observed usage can remain incomplete when
providers omit accounting fields.
Fallback and actually affected consumers skip cache lookup/storage; unrelated
accepted predecessors remain reusable. The report discloses emitting operations
even when warnings are hidden; affected files and validator calls are not extra
fallbacks. A hit does not establish fresh acceptance under changed policy.
Consumers needing new inference/assessment deliberately choose bypass/refresh.
After the applicable budgets are exhausted, the resolved
[`validation_policy`](config.md#pipelines) determines the result. Structural
failure and semantic rejection normally fail the run; an explicit
`reject_output` records a rejection and allows unrelated work to finish. A
validator execution failure normally uses `warn_continue`, which keeps an
otherwise accepted result in the current run with `incomplete` validation
provenance. It emits one bounded warning for every validator whose execution
budget was exhausted. A corrected result that later passes validation does not
retain diagnostics from abandoned attempts.
Treat a successful process exit as a completed run, not as proof that every
candidate was fully validated. Inspect the receipt's `validation_status`,
`validation_summaries`, rejection count, and warning group and occurrence
counts when an orchestrator requires complete validation. The durable fields
and their meanings are owned by the
[run-result receipt](integrations/run-result.md) and
[published JSON output contract](integrations/json-output.md).
Process warnings retain native visibility; quality advisories are not promoted
to warnings. [Configuration](config.md#producer-controls) owns selected modes,
turn allowances, defaults, and precedence.
## Output Bundles
Each successful run receives a generated safe run identifier and writes beneath:
The encoder defines [logical artifact files](integrations/json-output.md);
the CLI places them beneath the run directory. It checks paths before allocation
and refuses an existing destination without changing it. Individual writes are
atomic, not an all-files transaction. A later write failure can leave a partial
new directory for inspection.
~~~
<output-root>/<run-id>/
~~~
Artifacts are deliverables, not an index, manifest, persisted receipt, or reporting
sidecar. Read configured lane and auxiliary paths beneath the successful report's
directory; do not scan for the newest folder.
The [JSON output contract](integrations/json-output.md) owns the logical files
and their schemas. Before creating the run directory, Notarius validates every
logical output path. It refuses an existing run directory without changing it.
Files are written atomically; if a later write fails, the newly created partial
run directory remains for inspection and is never removed automatically.
Capture/report failure can occur after publication. The command returns nonzero
with available locations; valid artifacts are not removed or regenerated.
Failed stdout can be partial. Do not treat it as a success report or assume a
failed publication directory is complete.
Treat an output bundle as durable user data. Do not use cache-cleanup policy to
remove it. An optional accepted chunk map is also durable output and can carry
source- or model-derived annotations; its content and compatibility contract
are defined in [Accepted Chunk Map](integrations/chunk-map.md). An optional
[evidence context](integrations/evidence-context.md) contains source-unit text
and metadata. It is not a cache or debug artifact: retain it with the output
bundle only for as long as consumers need it, and apply source-content access
controls to the entire bundle. Its selected source-unit excerpt may include
every source unit once when coverage is broad or its configured window is
large, so do not assume a byte or token reduction or reduced sensitivity.
Output is durable user data. Optional [chunk maps](integrations/chunk-map.md)
and [evidence context](integrations/evidence-context.md) are output, not cache/
debug. Evidence can cover every source unit and is not promised smaller or less
sensitive; retain/share it according to consumer needs.
## Chunk-Plan Cache
## Accepted-Result Cache And Recovery
Chunk plans live beneath the selected chunk-plan root:
The cache stores final accepted typed artifacts, including applied normalization,
and accepted LLM chunk plans, not native responses, conversations, or execution
ledgers. Cheap deterministic work runs normally.
~~~
<chunk-plan-root>/<source-sha256-hex>/plan.json
~~~
The key is the initial native prepared `MessageContentHash`, unchanged through
correction. Matching ordered roles/text/boundaries can reuse final acceptance.
Model, session, schema metadata, policy, and implementation add no dimensions;
text actually rendered into messages participates. Use refresh/bypass when a
model or policy experiment needs new inference/assessment.
One validated canonical plan is active for each source digest. The plan stores
boundaries and provenance, not a second copy of the entire source. This
source-addressed policy is recorded in [ADR-0005](adr/0005-cache-canonical-chunk-plans-by-source.md).
Hits perform ordinary codec/current-source usability checks without semantic
reassessment, provider calls, or historical usage claims. Plans rematerialize
against the current source. Source identity alone no longer selects a canonical
partition; changed initial content can select different boundaries.
The configured cache mode controls one invocation:
- **auto** looks for a valid active plan. Missing or invalid state causes a new
plan to be generated; an accepted new plan is atomically published.
- **refresh** skips lookup, generates a plan with the configured chunker, and
atomically replaces the active plan after it is accepted.
- **bypass** performs no chunk-plan cache I/O. It does not resolve or create a
chunk-plan root.
A reused plan is still materialized and validated against the current source.
If a prior plan no longer gives acceptable results, use a refresh run rather
than editing cache files. Deleting a plan is recoverable but can repeat costly
chunking work. A plan accepted only under incomplete validation is not
published, and a rejected cache hit falls through to ordinary generation rather
than becoming a correction candidate.
## Checkpoint Recording, Resume, And Recompute
Checkpoint recording is an explicit configuration choice and is disabled by
default. When enabled, each run records stage transitions and the state needed
for compatible recovery. A run records checkpoints even when it does not ask
to reuse them. Checkpoint payloads can contain source-derived and intermediate
application data, so treat the entire root as sensitive.
Checkpoint loading is separate: [**--resume**](cli.md#run) asks a run to reuse
compatible recorded work. A resume request fails when checkpoint recording is
disabled. Without **--resume**, a recording-enabled run executes normally and
does not load checkpoint state. Compatibility includes the resolved pipeline,
input, selected lanes, runtime overrides, reference provenance, LLM-profile
provenance, the effective PromptKit profile-source fingerprint, and
prepared-component fingerprints. When a local PromptKit backend is configured,
compatibility also includes a non-secret fingerprint of its endpoint. Changing
profile content or the local endpoint causes a cold miss; changing only the
local concurrency limit does not. A changed identity produces a cold miss;
Notarius does not migrate, rewrite, or delete older checkpoint directories.
Reasoning-effort inheritance, replacement, and explicit clearing are distinct
runtime identities, so checkpoints created under one state are not reused by
either of the others.
Only accepted, completely validated chunk, extract, merge, and normalize
results are checkpointed for reuse. Rejected, structurally invalid, and
validation-incomplete producer results remain non-reusable, even when a
`warn_continue` result advanced during its original run. A resumed invocation
therefore reruns that producer rather than treating degraded state as accepted.
Checkpoint state is confined below an identity-specific path:
~~~
<checkpoint-root>/<pipeline-id>/<input-key>-<source-or-input-digest-prefix>/<pipeline-digest-prefix>/<identity-digest-prefix>/
~~~
### Selective Recompute
[**--recompute-step**](cli.md#run) requires both **--resume** and enabled
checkpoint recording. It forces the selected ordered step and every lane that
depends on it through generated artifact references. Unrelated lanes remain
eligible for reuse.
For an earlier producer required by a forced consumer, Notarius requires a
compatible accepted normalized artifact. It validates that artifact before
hydrating it and does not silently rerun the producer. If that state is
missing, rejected, corrupt, non-canonical, or incompatible, the run stops
before its dependent starts. Rerun the required producer deliberately instead
of copying or editing checkpoint files.
## Checkpoint Decisions And Recovery
Checkpoint events classify work as **executed**, **reused**,
**forced_recompute**, or **dependency_invalidated**. Their stable reason codes
are written to run diagnostics and provenance. Use the code, not a copied
error message, to decide what to repair.
| Reason code | Recovery meaning |
| Mode | Recovery behavior |
| --- | --- |
| **loading_disabled** | This invocation did not permit checkpoint loading. |
| **checkpoint_missing**, **checkpoint_path_invalid**, **checkpoint_read_failed**, **checkpoint_decode_failed** | The stored checkpoint could not be located or read safely; normal resume work can execute again. |
| **workspace_schema_incompatible**, **identity_mismatch**, **stage_mismatch**, **step_mismatch**, **lane_mismatch**, **module_mismatch** | Stored state belongs to a different compatible scope or identity; allow a fresh run to create new state. |
| **status_not_reusable** | The recorded operation did not end in reusable state. |
| **dependency_mismatch** | A dependency changed; dependent work is invalidated rather than reused. |
| **artifact_payload_invalid**, **artifact_digest_mismatch**, **artifact_codec_incompatible**, **artifact_not_canonical** | A stored artifact cannot safely be hydrated; rerun the producer instead of modifying the cache. |
| **checkpoint_reused** | A normal checkpoint passed compatibility checks. |
| **accepted_artifact_reused** | A required predecessor's accepted normalized artifact was safely hydrated. |
| **recompute_step** | Selective recomputation deliberately forced this work. |
| `auto` | Read accepted entries; execute misses and store final acceptance. |
| `bypass` | Neither read nor write; no cache directory creation. |
| `refresh` | Skip reads and replace only after new acceptance. |
Reason detail is bounded code-owned text. It is diagnostic information, not a
path-discovery or data-recovery mechanism, and does not contain checkpoint,
source, reference, credential, or environment content.
[Configuration](config.md#producer-controls) owns precedence. No mode overrides
eligibility: failure/rejection/incomplete/canceled/fallback results are not stored.
Failed refresh preserves previous acceptance. Fallback incompleteness excludes
actual dependent work, not unrelated siblings.
Missing entries are ordinary misses. Unreadable, malformed, or unusable entries
miss with diagnostics. Cache I/O failures warn without failing otherwise successful
work or repeating model calls. Artifact encoding, required capture, and invalid
configuration remain their own failure boundaries.
After fixing failed work, rerun normally: accepted predecessors and the chunk
plan can be reused while unsuccessful work executes. There is no resume request
or checkpoint barrier. Refresh obtains new acceptance; bypass isolates an
invocation without updating cache state.
### Legacy State
Legacy checkpoints and source-keyed chunk plans remain ignored and untouched,
even below the new root. They do not supply content keys. No automatic import,
conversion, rewrite, permission migration, or deletion occurs. New entries
populate through eligible execution. Operators may retain/clean obsolete data
deliberately; see [migration guidance](consumers/native-migration.md).
## Debug Bundles
Only a [debug-enabled run](cli.md#run) creates a bundle:
Capture needs explicit selection and a nonempty effective destination.
Directory configuration alone does not enable it. Explicit empty disables it;
a nonempty invocation override can re-enable it. [Configuration](config.md#file-shape-and-defaults)
owns defaults/origins; [CLI Reference](cli.md#run) owns mode ordering.
~~~
<debug-root>/<run-id>/
summary/
trace/
~~~
Allocation creates the run directory and updates `latest` to a relative link
at session start. It reflects the last successful start link update, including a
run that later fails. Completion never retargets it. Concurrent starts retain
separate directories; disabled capture leaves the link untouched. This link is
not an output-discovery API.
The summary contains redacted invocation and resolution information, including
`effective-config.yml` and the separately redacted resolved pipeline, plus run,
final grouped diagnostic, checkpoint, chunk-plan, and terminal reporting artifacts. Attempt
terminal records contain bounded attempt kinds, validator outcomes, policy,
decision, PromptKit repair count, and usage; they do not contain assistant
responses or complete correction messages. The trace contains allowlisted
application diagnostic records and can include source, model, and correction
content. Neither surface is a cache input. Do not treat a debug bundle as safe
to share merely because its configuration summary is redacted. Invocation
metadata omits reasoning effort when it is inherited, records the replacement
value when one is supplied, and records an empty value when inherited reasoning
was explicitly cleared. It records `config_overlay` only when a non-base
configuration overlay was selected.
One recorder writes available evidence incrementally: configuration/composition,
parsed source/fixed inputs, prepared contracts, native events/results/full
exchanges, typed candidates/findings, correction/normalization material, fallbacks,
and cache decisions. Hits can retain prepared/current artifact evidence without
invented executions. Unknown usage stays unknown. Result-less failures retain
available facts; undelivered responses are unassessed.
Notarius never creates debug state without an explicit request and never
automatically deletes a requested bundle. If allocation succeeds, the command
reports its path on both success and later failure. A summary, trace, or
terminal-report persistence failure fails the command while preserving any
already-written diagnostic data for inspection.
Exact files, nesting, and serialized shapes are private diagnostic choices,
not public contracts or exhaustive provider transcripts. Debug never prepares
or executes another request just to capture it.
Requested capture is required work. Allocation/link failure stops before model
work, recording errors surface at normal boundaries, and finalization failure
fails the invocation even after publication. Available evidence and primary/
additional causes remain; capture failure never retries a model or deletes data.
Bundles can contain full source, reference, prompt, response, and intermediate
data. Preserve marked-secret/native credential omissions, without assuming
sanitized content: arbitrary application text is not heuristically scrubbed.
Operators own access, sharing, and retention as for other application data.
## Cleanup
Cleanup is manual and destructive. First inspect the exact leaf directory,
then remove only that leaf; do not use a glob or a parent root as the target.
Inspect exact targets and coordinate with active writers before administrative
cleanup. Prefer moving evidence aside or recoverable removal where practical.
Target only intended run directories or entries, not globs or broad workspace,
home, or root directories.
~~~
rm -rf -- /srv/notarius/output/run-1721300000000000000-0123456789abcdef0123456789abcdef
rm -rf -- /srv/notarius/chunk-plans/0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef
rm -rf -- /srv/notarius/checkpoints/example/seriatim-0123456789abcdef/0123456789abcdef/0123456789abcdef
rm -rf -- /srv/notarius/debug/run-1721300000000000000-0123456789abcdef0123456789abcdef
~~~
Deleting output permanently removes user data. Deleting chunk plans or
checkpoints is recoverable but may repeat expensive provider or pipeline work.
Deleting a debug bundle removes troubleshooting evidence and a retained copy of
application data. Notarius has no cache garbage collector, rollback operation,
or automatic cleanup command.
Output deletion loses user data. Cache deletion is reconstructible but may repeat
expensive work. Debug deletion loses evidence and retained application data;
account for its `latest` link when managing old sessions. Notarius supplies
no automatic retention, cache migration, or collection command.
## Operational Limits
Provider execution settings and the generation timeout come from the selected
PromptKit profile. The invocation-only **--reasoning-effort** and
**--clear-reasoning-effort** controls may replace or clear that profile setting
for all LLM-backed calls in one run without changing the profile. PromptKit
also retries eligible transient HTTP failures with its built-in default
of two additional attempts per generation. This retry work is separate from
structural output repair, which happens within one structured-completion call. Its
effective `structured_output_repair_attempts` limit is resolved from the
selected binding, then the pipeline, then the prompt declaration; see
[module bindings](config.md#module-bindings-and-validators). This is distinct
from Notarius binding **retries**, which rerun the complete module operation
and validation chain and do not consume or replenish the structural-repair
limit. The maintained production prompts declare one repair attempt, paid only
after a structural failure. One structured completion with repair budget **R**
contains at most **R + 1** serial generations. Each generation can now contain
more than one provider call when PromptKit retries an eligible transient
failure. LLM-backed validators have their own corresponding invocation and
repair budgets. These are bounds on attempted work, not a promise that every
call reaches a provider.
Choose caller deadlines and native execution settings for the workload.
Cancellation remains authoritative; native timeout/capacity/provider failures
are not interchangeable with caller cancellation. Retry, repair, and continuation
can increase latency/cost without promising old-wrapper attempt parity.
Timeouts are layered. Caller cancellation is the outer authority. A positive
effective generation timeout adds an inner request deadline to each built-in
generation attempt, while zero disables only that generation deadline. The HTTP
client timeout remains a transport-wide cap. Notarius does not add another
timeout around PromptKit. Retries and repairs are serial within the same caller
context, so their worst-case latency and cost can exceed the structural-repair
bound above; provision run deadlines and provider budgets accordingly.
Credentials remain optional unless the selected
PromptKit profile requires one, in which case preparation fails before a
provider call when its configured credential is unavailable.
The pinned upstream boundary and profile-format links are in
[PromptKit Integration](integrations/pkg-promptkit.md).
Workers bound ready jobs; native engine/backend admission governs configured
backend capacity independently. There is no global cross-backend scheduler or
second admission queue. Nested assessment does not hold a completed producer's
native permit. Endpoint-only/unlimited backends gain no inferred application
backend limit. Capacity failure follows producer mode without another backoff.
Concurrency has two independent layers. Notarius **total_llm** defaults to 16
and is the application-wide provider-call limit shared by all backends,
modules, retries, and validators. PromptKit may impose a narrower admission
limit for the selected backend. The effective active-generation bound is the
intersection of the Notarius limit, any PromptKit backend limit, and work made
available by the pipeline. Built-in OpenRouter profiles use PromptKit's
upstream backend limit; endpoint-only profiles have no PromptKit backend limit
and remain bounded by Notarius. For the configured local backend, a zero
**concurrency_limit** leaves only the Notarius scheduler as a call limit. A
positive value makes the effective active local-generation bound the smaller
of **total_llm** and that local limit, so a local limit of four permits no more
than four active local generations.
The Notarius scheduler admits one logical structured completion and holds that
permit while PromptKit performs its serial corrective calls. PromptKit applies
its selected-backend admission to each provider call; Notarius does not
reacquire a permit or add another scheduler for a repair.
For a positive local limit, PromptKit owns its default waiting capacity and
admission behavior. When a PromptKit backend has admitted all active and queued
work, a new call fails as capacity exhaustion before generation. The adapter
maps that failure to Notarius's existing provider-neutral capacity error and
does not retry it. The calling stage's configured retry policy applies
normally, and the run fails if those attempts are exhausted. Caller
cancellation remains authoritative. Configuration contracts are documented
under [PromptKit profiles](config.md#promptkit-profiles) and
[concurrency](config.md#concurrency-output-cache-and-debug). Extract-worker
limits and actual provider-call limits are independent. Notarius writes local
filesystem state only; remote storage, archival, and retention automation are
outside the implemented CLI.
Every run has an effective prompt session used for provider routing and run
provenance. The generated default is stable for the same input module and raw
input bytes; use [**--session-id**](cli.md#run) only when intentionally grouping
different invocations. Both generated and explicit values can be visible to
providers, manifests, checkpoints, and requested debug bundles. Do not put
credentials or other secrets in an explicit session identifier; command-line
values are not a credential mechanism.
Sessions are provider-visible routing correlations, not secrets. Their default
and override semantics belong in [CLI Reference](cli.md#run-wide-llm-overrides).
State is local filesystem data; remote storage, archival, retention automation,
live model-quality guarantees, and release publication are outside this workflow.

View File

@@ -2,7 +2,7 @@
**Base Policy Version:** 1.7
**Last Updated**: 2026-11-17
**Last Updated**: 2026-11-22
**Upstream Source**: https://gitea.maximumdirect.net/eric/policy/src/commit/593c9b52273c5c879785ae9e995d21026df467b4/docs/policy/release.md
@@ -169,8 +169,8 @@ Do not add routine confirmations for checks the command can perform itself.
### Notarius Gate Selection
Run the default offline Go suite once and build the native `cmd/notarius` with
release version injection, checking its `--version` output. The command check
protects the actual executable entry point and linker-supplied version, which
release version injection, checking its native `version` command output. The
command check protects the actual executable entry point and linker-supplied version, which
package tests alone do not verify. Build products are temporary, not published.
Use `GOWORK=off` so a sibling checkout cannot substitute for declared dependencies.

View File

@@ -25,7 +25,7 @@ without putting secrets in the repository. Prepare the selected toolchain and
dependencies before offline validation where necessary:
```sh
GOWORK=off GOPRIVATE=gitea.maximumdirect.net go mod download
GOWORK=off GOPRIVATE=gitea.maximumdirect.net,go.maximumdirect.net go mod download
```
Choose an unused stable version under the policy's compatibility rules. Add
@@ -103,5 +103,8 @@ development guide and testing policy govern change-specific verification.
A fresh source install or exact-tag checkout/build may be used to investigate
consumer availability after release. Such checks are optional and do not delay
or reverse release completion. `notarius --version` is diagnostic; subprocess
compatibility is defined by the receipt and artifact contracts.
or reverse release completion. `notarius version` is diagnostic; subprocess
compatibility is defined by the [CLI report](cli.md#completion-report) and
[domain artifact contracts](integrations/json-output.md). The
[native migration guide](consumers/native-migration.md) describes the command
and configuration cutover; it does not prepare or authorize a release.

View File

@@ -1943,6 +1943,35 @@ explicit values through declarations, not prose-based change detectors.
## Stage 40: Update Operations And Downstream Consumer Contracts
**Status:** Complete. Updated operations for independent output/cache/debug roots,
per-user and explicitly provisioned system cache placement, best-effort reuse,
failed-refresh preservation, untouched legacy records, fallback qualification/
eligibility, whole-source chunk fallback, and required capture with start-time
`latest`. Replaced receipt/index/manifest workflows with native completion reports
and configured domain paths, including partial stdout and post-publication failure
handling. Retired the superseded receipt contract, repaired current navigation,
and linked new native migration guidance for command/configuration/resource IDs,
retry intent, workers/admission, caches, reporting, and private debug evidence.
JSON/chunk-map/evidence contracts retain payload schemas and logical paths without
requiring descriptors or historical execution provenance on reuse.
The maintained complete native CLI contract now supplies explicit input/output and
pipeline-level campaign/catalog references and verifies all ten domain files plus
both auxiliaries against the twelve-deliverable report using the existing provider
fake. Existing executable fake-HTTP checks retain native format/fallback coverage.
Found and corrected the release validator's retired version flag; its existing
offline test double now rejects the wrong command, demonstrably failing before the
fix and passing afterward. Updated only affected release documentation/command
references, preserving the selected validation gates and adopted policy baseline.
A temporary linked executable reports its injected version via the native command.
Whole-tree tests and vet, executable build, shell syntax, 147 local links/anchors,
and `git diff --check` pass. Removal searches leave receipt-contract references
only in the active plan and preserved historical release/archive records. No
runtime data, domain schema, tagged note, or archive was changed; no real provider,
credential, release, push, or new provisioning/retention requirement was involved.
Full internal/integration/orientation rewrites remain assigned to the next stages.
**Goal:** explain the intentional persistence and subprocess breaking changes.
**Inspect/change:** `docs/operations.md`, `docs/consumers/{subprocess,dnd-pipeline}.md`,