Update native operations, consumer migration, and version checks
This commit is contained in:
@@ -1,142 +1,83 @@
|
||||
# Consuming The Complete D&D Pipeline
|
||||
|
||||
Use this workflow when an orchestrator runs the maintained complete D&D
|
||||
pipeline and consumes its structured JSON artifacts. The generic
|
||||
[subprocess consumer guide](subprocess.md) owns process-level responsibilities;
|
||||
this guide connects that workflow to the complete D&D configuration, its
|
||||
Seriatim input, and its artifact inventory.
|
||||
This guide applies the [subprocess workflow](subprocess.md) to the maintained
|
||||
complete D&D configuration. [CLI Reference](../cli.md) owns invocation and the
|
||||
native report; [Configuration](../config.md) owns composition;
|
||||
[Published JSON Output](../integrations/json-output.md) owns logical paths.
|
||||
[Migration guidance](native-migration.md) covers older orchestrators.
|
||||
|
||||
The [CLI reference](../cli.md), [configuration reference](../config.md),
|
||||
[run-result receipt](../integrations/run-result.md), and
|
||||
[published JSON output contract](../integrations/json-output.md) remain the
|
||||
canonical definitions of those public interfaces.
|
||||
## Prepare The Deployment
|
||||
|
||||
## Prepare And Validate The Deployment
|
||||
Start from [examples/dnd-complete/config.yml](../../examples/dnd-complete/config.yml).
|
||||
Preserve its imported pipeline fragments, native companion, and reference files,
|
||||
or update the paths deliberately. It uses every implemented D&D family, ordered
|
||||
generated handoffs, and selected chunk-map/evidence output.
|
||||
|
||||
Start from the maintained
|
||||
[complete D&D configuration](../../examples/dnd-complete.config.yml). It uses
|
||||
the `dnd-session` pipeline and demonstrates every implemented D&D lane, ordered
|
||||
artifact handoffs, campaign references, chunk-map publication, and evidence
|
||||
context.
|
||||
Native embedded defaults and the maintained companion already provide profile/
|
||||
backend definitions. A deployment can supply variants rather than being required
|
||||
to recreate them. YAML paths retain the contributing declaration's origin;
|
||||
CLI reference paths use the process working directory. Absolute caller paths
|
||||
are useful for services. Keep credentials in the intended environment, not
|
||||
files, arguments, sessions, or logs.
|
||||
|
||||
A deployment must provide its own PromptKit profile and campaign reference
|
||||
files. Use absolute paths for service and subprocess deployments. In
|
||||
particular, observe these different resolution rules:
|
||||
Preflight without credentials or inference:
|
||||
|
||||
- reference and PromptKit profile paths in YAML are resolved relative to the
|
||||
file that supplies their surviving declaration;
|
||||
- reference paths passed with `--reference` are resolved relative to the
|
||||
Notarius process working directory.
|
||||
~~~sh
|
||||
notarius config validate --config /absolute/path/to/notarius.yml
|
||||
notarius pipeline validate dnd-session --config /absolute/path/to/notarius.yml
|
||||
~~~
|
||||
|
||||
Do not copy the repository example's relative profile path into a deployment
|
||||
without preserving its configuration-relative layout. The complete path and
|
||||
profile rules are defined in [Configuration](../config.md).
|
||||
## Supply The Transcript And References
|
||||
|
||||
Preflight the deployed configuration before processing sessions and whenever
|
||||
it changes:
|
||||
Use the [Seriatim JSON contract](../integrations/seriatim.md). Preserve segment
|
||||
IDs because artifact citations use those source-unit ranges. If the caller
|
||||
maintains a final trimmed transcript tier, select that same session content
|
||||
consistently for later consumers.
|
||||
|
||||
```sh
|
||||
notarius config validate \
|
||||
--config /absolute/path/to/notarius.yml \
|
||||
--pipeline dnd-session
|
||||
```
|
||||
Run with explicit input/output and pipeline-level references:
|
||||
|
||||
Provide credentials through the environment or the documented configuration
|
||||
mechanism. Do not put credentials in command arguments, generated
|
||||
configuration, or logs.
|
||||
|
||||
## Supply The Transcript
|
||||
|
||||
The complete pipeline consumes a Seriatim JSON document. The
|
||||
[Seriatim input contract](../integrations/seriatim.md) defines its required
|
||||
metadata, segments, and validation rules. Preserve segment IDs: D&D artifact
|
||||
citations use those segment IDs as source-unit ranges.
|
||||
|
||||
When the caller maintains several transcript tiers, use the final trimmed JSON
|
||||
transcript so extraction operates on the same session content presented to
|
||||
later consumers. For example, Narratio identifies this implemented artifact as
|
||||
`narratio.transcript.final_trimmed` and normally stores it at
|
||||
`transcripts/final.trimmed.json`.
|
||||
|
||||
Notarius generates a stable prompt session from the resolved input module and
|
||||
the exact input bytes. An ordinary orchestrator should not pass `--session-id`.
|
||||
Use that override only when intentionally changing the routing relationship
|
||||
between invocations; it is not a credential or output identity.
|
||||
|
||||
## Run Notarius
|
||||
|
||||
Invoke the pipeline with explicit absolute paths and request its
|
||||
machine-readable receipt:
|
||||
|
||||
```sh
|
||||
~~~sh
|
||||
notarius run dnd-session \
|
||||
--config /absolute/path/to/notarius.yml \
|
||||
--input /absolute/path/to/transcripts/final.trimmed.json \
|
||||
--input-file /absolute/path/to/transcripts/final.trimmed.json \
|
||||
--output-dir /absolute/path/to/notarius-output \
|
||||
--reference party=/absolute/path/to/references/party.txt \
|
||||
--reference players=/absolute/path/to/references/players.txt \
|
||||
--reference glossary=/absolute/path/to/references/glossary.txt \
|
||||
--reference spell_catalog=/absolute/path/to/references/spells.json \
|
||||
--json
|
||||
```
|
||||
--format json
|
||||
~~~
|
||||
|
||||
Each unqualified reference is pipeline-scoped: Notarius supplies it to every
|
||||
selected D&D target that declares the slot. A deployment may omit an optional
|
||||
reference it does not maintain, and may use the lane- or binding-qualified
|
||||
forms from the [CLI reference](../cli.md#reference-selectors) for an exceptional
|
||||
override. The registry, scene-description, combat-turn, and NPC-occurrence
|
||||
references declared between ordered steps in the complete configuration are
|
||||
generated artifacts. Do not pass those handoffs on the CLI.
|
||||
Only supply optional references the deployment maintains. Unqualified selectors
|
||||
reach every selected target declaring the slot; narrower overrides follow the
|
||||
[selector contract](../cli.md#reference-selectors). Generated registry, scene,
|
||||
combat, and NPC-occurrence handoffs stay in pipeline configuration, not CLI file
|
||||
bindings.
|
||||
|
||||
The caller should:
|
||||
The ordinary input-based session is suitable without an override. Use
|
||||
`--llm-session-id` only to intentionally change its native selection.
|
||||
Sessions correlate provider routing; they are not credential or output identities.
|
||||
|
||||
- capture stdout and stderr separately;
|
||||
- propagate cancellation and impose an operator-appropriate timeout;
|
||||
- wait for process completion before interpreting stdout; and
|
||||
- retain stderr for diagnosis without copying secrets or transcript content
|
||||
into other logs.
|
||||
Capture stdout/stderr separately, propagate cancellation, impose an appropriate
|
||||
caller deadline, and wait for completion. Parse a successful report only after
|
||||
status zero; failed writes can leave partial stdout. Nonzero status after
|
||||
publication can still leave valid artifacts for deliberate recovery.
|
||||
|
||||
Only exit status 0 permits decoding stdout as a receipt. Ignore stdout after a
|
||||
nonzero exit because a failed receipt write can leave partial bytes. The
|
||||
[CLI reference](../cli.md#output-streams-and-exit-statuses) defines the complete
|
||||
stream and exit-status contract.
|
||||
## Locate And Read Artifacts
|
||||
|
||||
## Discover The Published Bundle
|
||||
Use the report's absolute `output_directory`, not a newest-directory scan.
|
||||
The report's exact fields and fallback qualification belong in
|
||||
[CLI Reference](../cli.md#completion-report). There is no receipt schema,
|
||||
index/manifest traversal, or descriptor inventory to load.
|
||||
|
||||
Decode the successful stdout document as a supported run-result schema. For
|
||||
the current contract, `schema_version` is `notarius.run-result.v2`. Tolerate
|
||||
unknown fields allowed by that version, but reject an unsupported schema
|
||||
version.
|
||||
|
||||
Use the receipt's absolute `output_directory` as the exact run-specific bundle
|
||||
root. Do not scan the output root for its newest directory, guess a run ID, or
|
||||
construct a bundle path. Resolve `index_file` beneath `output_directory` and
|
||||
reject an absolute logical path or any result that escapes the bundle root.
|
||||
|
||||
The complete configuration uses the application validation defaults. A caller
|
||||
that requires fully validated D&D artifacts must also require receipt
|
||||
`validation_status: approved`; a successful `incomplete` result reflects the
|
||||
configured validator-failure continuation policy and carries its bounded
|
||||
validator provenance in `validation_summaries`.
|
||||
|
||||
Read `index.json` and locate each requested lane in `output_files` by its exact
|
||||
`lane_id`. Do not guess a lane filename. Before decoding a payload:
|
||||
|
||||
1. resolve its descriptor's relative `file` beneath the bundle root with the
|
||||
same confinement check;
|
||||
2. verify the descriptor's media type and schema identity against the linked
|
||||
artifact contract; and
|
||||
3. decode the payload according to that contract.
|
||||
|
||||
The [published JSON output contract](../integrations/json-output.md) defines
|
||||
the index and bundle layout. Treat all paths obtained from a decoded external
|
||||
document as untrusted until confined to their documented root.
|
||||
The complete configuration uses safe literal lane IDs: each table entry below
|
||||
is at `lanes/<lane-id>.json` beneath that run directory. Decode the configured
|
||||
domain contract directly. All ten lane artifacts plus the two enabled/available
|
||||
auxiliaries give twelve deliverables; a count is not a substitute for checking
|
||||
the files required by the caller.
|
||||
|
||||
## Complete Artifact Inventory
|
||||
|
||||
When every configured lane is accepted, the complete example publishes these
|
||||
lane artifacts:
|
||||
|
||||
| Lane ID | Purpose | Canonical contract |
|
||||
| --- | --- | --- |
|
||||
| `item-registry` | Canonical registry of encountered items and currency. | [Item registry](../integrations/dnd-item-registry-artifacts.md) |
|
||||
@@ -150,72 +91,52 @@ lane artifacts:
|
||||
| `location-occurrences` | Source-grounded location occurrences. | [Location occurrences](../integrations/dnd-location-occurrence-artifacts.md) |
|
||||
| `enemy-events` | Source-grounded enemy combat events. | [Enemy events](../integrations/dnd-enemy-event-artifacts.md) |
|
||||
|
||||
The JSON encoder always publishes these bundle-management files:
|
||||
The complete configuration also selects two pipeline-wide files:
|
||||
|
||||
| File | Purpose |
|
||||
| --- | --- |
|
||||
| `index.json` | Discovery document for lane and pipeline-wide artifacts. |
|
||||
| `manifest.json` | Run provenance and result summaries. |
|
||||
| `rejected.json` | Rejected pipeline outputs. |
|
||||
| `warnings.json` | Actionable process-degradation warnings. |
|
||||
| `diagnostics.json` | Advisory and observation findings for accepted artifacts. |
|
||||
- [`chunk-map.json`](../integrations/chunk-map.md), the materialized plan/chunks.
|
||||
- [`evidence-context.json`](../integrations/evidence-context.md), the union of
|
||||
selected cited units and their configured surrounding window.
|
||||
|
||||
The complete configuration also requests two pipeline-wide artifacts:
|
||||
They are not lanes and need no discovery descriptor. Their intrinsic payload
|
||||
schemas are unchanged. Evidence is reading context, not authoritative lane
|
||||
attribution; surrounding units do not widen citations in lane artifacts.
|
||||
No index, manifest, rejection/warning/diagnostic sidecars are required.
|
||||
|
||||
- [`chunk-map.json`](../integrations/chunk-map.md), the accepted chunk plan and
|
||||
chunk metadata; and
|
||||
- [`evidence-context.json`](../integrations/evidence-context.md), a reading
|
||||
excerpt containing the union of selected cited source units and the
|
||||
configured surrounding window.
|
||||
## Decide Consumer Acceptance
|
||||
|
||||
Discover both from their top-level `index.json` descriptors rather than
|
||||
treating them as lanes. Evidence context is convenient reading material, not
|
||||
authoritative provenance; citations in the normalized lane payloads remain the
|
||||
evidence contract.
|
||||
Process completion does not establish the caller's required artifact set,
|
||||
semantic quality, or fresh validation under changed policy. A complete-workflow
|
||||
caller should check all ten required lane files against their contracts. A
|
||||
subset consumer can choose a narrower set deliberately. Empty valid domain
|
||||
collections remain distinguishable from missing/incompatible files.
|
||||
|
||||
Every optional or lane file is published only when its corresponding artifact
|
||||
is available. A successful process does not guarantee that all configured
|
||||
lanes were accepted.
|
||||
Explicit permissive execution can complete with qualified fallback. Decide
|
||||
whether to accept that and inspect positive `fallback_count` when relevant.
|
||||
The count identifies emitting operations, not individual affected lanes.
|
||||
Fallbacks satisfy intrinsic/downstream invariants but not ordinary semantic
|
||||
approval; valid downstream results affected by them also avoid cache reuse.
|
||||
|
||||
## Decide What Counts As Consumer Success
|
||||
Default cache reuse does not repeat semantic assessment. For a deliberate model/
|
||||
policy experiment requiring fresh inference, select bypass/refresh under
|
||||
[configuration controls](../config.md#producer-controls). Do not look for retired
|
||||
validation ledgers or treat optional debug serialization as an API.
|
||||
|
||||
Exit status 0 means Notarius completed the pipeline and published its result
|
||||
bundle. The receipt or bundle may still report warnings, rejected outputs, or
|
||||
missing lane descriptors. A downstream consumer must define its own required
|
||||
artifact set explicitly.
|
||||
|
||||
A caller that claims to consume the complete D&D workflow should normally
|
||||
require all ten lane IDs in the table and verify each descriptor's expected
|
||||
contract. If any required lane is missing, rejected, or incompatible, fail the
|
||||
caller's extraction step while retaining the Notarius bundle for diagnosis. A
|
||||
consumer that needs only a subset may define and document a narrower policy.
|
||||
|
||||
Keep the successful receipt with the complete published bundle. Retain
|
||||
`manifest.json`, `rejected.json`, `warnings.json`, and captured process logs as
|
||||
required by the caller's provenance, diagnosis, and retention policies. Avoid
|
||||
selectively copying payload files without also preserving enough index and
|
||||
manifest information to identify their originating run and contracts.
|
||||
|
||||
The transcript, lane artifacts, evidence context, manifest, debug data, and
|
||||
logs can all contain private campaign information. Apply the same access,
|
||||
publication, and retention controls used for the source transcript.
|
||||
Retain useful domain files and caller-owned invocation/report evidence. Capture
|
||||
failure after publication does not invalidate completed artifact writes, but
|
||||
requires deliberate caller recovery rather than parsing partial success output.
|
||||
A failed publication can leave an incomplete directory. Transcript-derived
|
||||
artifacts, evidence, cache, debug, and streams can contain private campaign data;
|
||||
apply the caller's sharing and retention choices.
|
||||
|
||||
## Consumer Checklist
|
||||
|
||||
- Validate the deployed Notarius configuration and `dnd-session` pipeline.
|
||||
- Pass the final trimmed Seriatim JSON transcript with stable segment IDs.
|
||||
- Use absolute configuration, input, output-root, profile, and reference paths
|
||||
in service deployments.
|
||||
- Capture stdout and stderr separately and enforce cancellation and timeout.
|
||||
- Parse stdout only after exit status 0.
|
||||
- Accept only supported receipt, index, and artifact schema versions while
|
||||
tolerating permitted unknown fields.
|
||||
- Use the receipt's `output_directory`; never guess the run directory.
|
||||
- Confine `index_file` and every descriptor path to the published bundle root.
|
||||
- Discover lanes by `lane_id` and verify descriptor compatibility before
|
||||
decoding payloads.
|
||||
- Enforce an explicit required-lane policy and inspect rejections and warnings.
|
||||
- Preserve the receipt and sufficient bundle provenance for every retained
|
||||
artifact.
|
||||
- Protect all transcript-derived files and diagnostic streams as sensitive
|
||||
campaign data.
|
||||
- Validate application configuration and the selected pipeline offline.
|
||||
- Preserve Seriatim source-unit IDs.
|
||||
- Pass source, output root, and optional campaign references as first-class inputs.
|
||||
- Keep generated handoffs configured; avoid external/generated collisions.
|
||||
- Separate streams, propagate cancellation, and check status before parsing.
|
||||
- Use the native report directory and configured domain/auxiliary paths.
|
||||
- Decode supported payload contracts and check the caller's explicit required set.
|
||||
- Decide whether qualified fallback and cached acceptance meet caller needs.
|
||||
- Retain only the artifact/report/evidence records useful to the caller.
|
||||
- Treat source-derived data as sensitive; do not rely on a sanitized debug bundle.
|
||||
|
||||
143
docs/consumers/native-migration.md
Normal file
143
docs/consumers/native-migration.md
Normal file
@@ -0,0 +1,143 @@
|
||||
# Migrating To Native Configuration And Commands
|
||||
|
||||
This guide maps retired Notarius interfaces to the implemented native workflow.
|
||||
It is migration material: the old spellings below are not accepted aliases.
|
||||
[Configuration](../config.md), [CLI Reference](../cli.md), and
|
||||
[Operations](../operations.md) own the current contracts. Domain artifact schemas
|
||||
and lane filename mapping are unchanged.
|
||||
|
||||
## Update Caller Arguments
|
||||
|
||||
| Former interface | Current equivalent / intent |
|
||||
| --- | --- |
|
||||
| `notarius --version` | `notarius version`. |
|
||||
| `pipelines list` | `pipeline list`. |
|
||||
| `config validate --pipeline ID` | `pipeline validate ID`; native `config validate` checks the application document. |
|
||||
| `config effective` | `config show`; scopes are `app` and `promptkit`. |
|
||||
| Positional `config explain POINTER` | `config explain --path POINTER`; use native list/history options for discovery. |
|
||||
| `config profile ID` | `promptkit show profile QUALIFIED_ID`; this reports stored fields, not a copied completed target. |
|
||||
| `--config-overlay NAME` | `--overlay NAME`. |
|
||||
| Run `--input PATH` | `--input-file PATH`. |
|
||||
| `--only a,b` | Repeat literal `--lane a --lane b`; commas are no longer separators. |
|
||||
| `--json` | `--format json`; YAML/text are also native report formats. |
|
||||
| `--session-id ID` | `--llm-session-id ID`. |
|
||||
| `--reasoning-effort VALUE` | `--llm-reasoning-effort VALUE`. |
|
||||
| `--clear-reasoning-effort` | Explicit `--llm-reasoning-effort=`; omission retains native inheritance. |
|
||||
| `--resume`, `--recompute-step`, separate `--chunk_cache`/`--chunk-cache` | Ordinary accepted-result reuse, with invocation-wide `--cache auto\|bypass\|refresh`. No explicit resume or selective checkpoint barrier. |
|
||||
|
||||
`run` and `pipeline run` are two native routes to the same command. Put flags
|
||||
after the complete command path. Singleton occurrences are all parsed, with the
|
||||
last successfully parsed value retained. False/zero/empty supplied values are not
|
||||
omission. Canonical `--llm-*` execution overrides reach producers, correction,
|
||||
and validators; shorter ckprompt aliases apply only to their installed PromptKit
|
||||
commands. Help works offline on subcommands.
|
||||
|
||||
## Migrate Configuration And Resources
|
||||
|
||||
Version 6 is the sole application version; bumping the header alone does not
|
||||
translate old fields. Use the maintained
|
||||
[minimal](../../examples/dnd-minimal/config.yml) or
|
||||
[complete](../../examples/dnd-complete/config.yml) configuration as a starting point.
|
||||
|
||||
- Replace `promptkit.profile_file`, `profile_dir`, and `local_backend` wrappers
|
||||
with native settings beneath the main file's `promptkit`, or wrapperless
|
||||
`prompts.yml` / recursive `prompts/` companions beside that selected main file.
|
||||
Companion files form strict peers, not filename-priority layers.
|
||||
- Use identified `backends`, `profiles`, and `prompts` lists with literal `id`
|
||||
values and collections. The embedded profile is
|
||||
`notarius/dnd-extraction`; initial/correction prompt IDs are also qualified,
|
||||
for example `notarius/dnd.spells` and `notarius/dnd.spells.repair`.
|
||||
- The root bundled backend is `openrouter`. Retired external provider catalogs
|
||||
are not loaded as parallel sources; declare needed backends/profiles through
|
||||
native resources. Do not infer backend identity from an endpoint or model.
|
||||
- Use native root `config` imports and preserve declaration-relative paths.
|
||||
Remove old per-prompt resource-version wrappers such as `v1`; resource IDs no
|
||||
longer select those versions.
|
||||
- A later same-ID resource replaces the whole earlier record. Use a distinct
|
||||
sibling with `extends: {id: parent}` to retain fields, not a partial same-ID
|
||||
replacement or self-inheritance. Profile/backend references to collection
|
||||
resources use qualified IDs; sibling inheritance is local to its list.
|
||||
- Preserve deliberate null/empty/zero choices under native semantics. Null
|
||||
endpoint/key-environment values can allow backend fallback; empty strings can
|
||||
clear those sources. Inspect native effective values/history rather than
|
||||
assuming old wrapper null behavior.
|
||||
- Keep prompt-declared schemas. Inline schemas, source-relative `schema_file`,
|
||||
and explicit-root `schema_path` are native alternatives. Replacing a collection
|
||||
schema root does not retain missing-file fallback to embedded resources.
|
||||
|
||||
Review [native resource and schema rules](../config.md#promptkit-profiles) before
|
||||
reusing an old profile file. The complete example's
|
||||
[companion](../../examples/dnd-complete/prompts.yml) demonstrates current inheritance.
|
||||
|
||||
## Separate Retry Intent, Mode, And Capacity
|
||||
|
||||
Producer `retries` intended for semantic correction becomes
|
||||
`max_additional_turns`, excluding the initial response. Binding overrides pipeline;
|
||||
omission inherits and explicit zero disables semantic correction. Native growing
|
||||
continuation replaces latest-only manual replay, and each revised candidate gets
|
||||
the full applicable assessment chain, including its final permitted turn.
|
||||
|
||||
Remove validator `retries`, generic deterministic retries,
|
||||
`structured_output_repair_attempts`, and the application `promptkit.transient_retries`
|
||||
wrapper. PromptKit owns transient failures and structural repair. Run-wide
|
||||
transient customization uses `--llm-transient-retries`; structural customization
|
||||
belongs in native prompt output contracts. Bundled prompts leave repair counts
|
||||
unspecified. Former total provider-attempt parity is not promised.
|
||||
|
||||
Replace failure-category `validation_policy` and its `fail_run`, `reject_output`,
|
||||
and `warn_continue` choices with producer `mode: strict|permissive`. Strict is
|
||||
the default, including normalization. Permissive explicitly requests a supported
|
||||
module fallback, not incomplete ordinary acceptance. Cancellation does not fall
|
||||
back; unsupported fallback construction fails. Completion discloses actual
|
||||
fallback emitters independently of warning visibility.
|
||||
|
||||
Replace `concurrency.total_llm` and `concurrency.stage_workers.extract` with
|
||||
`concurrency.workers`, covering ready extraction and lane completion together.
|
||||
Choose native backend capacity independently. There is no cross-backend global
|
||||
LLM ceiling, inferred per-backend translation, or extra capacity queue/retry.
|
||||
The [configuration reference](../config.md#concurrency-output-cache-and-debug)
|
||||
owns current defaults and controls.
|
||||
|
||||
## Replace Both Legacy Caches
|
||||
|
||||
Replace nested `cache.chunk_plans` / `cache.checkpoints` directories and recording
|
||||
switches with root `cache_directory` and pipeline/binding `cache`. An explicit
|
||||
invocation `--cache` overrides every binding. Auto reads/stores acceptance;
|
||||
bypass does neither; refresh skips reads and replaces only after new acceptance.
|
||||
|
||||
The initial prepared native message-content hash selects final accepted artifacts
|
||||
and LLM plans. A hit is decoded/usability-checked, not semantically revalidated.
|
||||
Model/settings/policy changes can still hit; use refresh/bypass for experiments
|
||||
requiring fresh work. Source identity alone no longer fixes chunk boundaries.
|
||||
Fallback and actually affected consumers skip reuse; unaffected predecessors
|
||||
remain reusable after downstream failure. Failed refresh preserves old acceptance.
|
||||
|
||||
Do not copy, convert, or delete legacy records as an upgrade prerequisite. They
|
||||
remain ignored and untouched, and the new cache populates through ordinary work.
|
||||
[Operations](../operations.md#accepted-result-cache-and-recovery) owns recovery
|
||||
and deployment-permission guidance.
|
||||
|
||||
## Replace Receipt And Sidecar Traversal
|
||||
|
||||
Stop expecting `notarius.run-result.v2`, `index_file`, validation histories,
|
||||
warning counts, `index.json`, `manifest.json`, or reporting-only rejection/
|
||||
warning/diagnostic files. Check process status, optionally decode the
|
||||
[native completion report](../cli.md#completion-report), then read configured
|
||||
lane/auxiliary [domain paths](../integrations/json-output.md).
|
||||
|
||||
Use the report's absolute run directory, not a guessed run ID or newest-folder
|
||||
scan. Count actual deliverables rather than old normalized-output totals.
|
||||
The report does not establish fresh validation or model quality; decide whether
|
||||
positive fallback qualification is acceptable. Failed stdout can be partial,
|
||||
and post-publication failure can leave valid artifacts with a nonzero status.
|
||||
|
||||
Debug remains explicitly selected and independent of native visibility modes.
|
||||
Directory-only configuration does not enable it; explicit empty disables it.
|
||||
The relative `latest` link updates at start, not completion. Full selected
|
||||
evidence replaces the old summary/trace hierarchy; filenames/JSON shapes remain
|
||||
private and arbitrary application content is not heuristically sanitized.
|
||||
Do not replace receipt traversal with a debug-layout dependency or a new manifest.
|
||||
|
||||
Update callers using the [subprocess guide](subprocess.md) or
|
||||
[complete D&D workflow](dnd-pipeline.md). These changes do not publish a new release
|
||||
or rewrite historical release notes.
|
||||
@@ -1,115 +1,96 @@
|
||||
# Using Notarius As A Subprocess
|
||||
|
||||
Use this workflow when an orchestrator runs Notarius and consumes its published
|
||||
artifacts. The [CLI reference](../cli.md) owns invocation syntax and exit
|
||||
statuses, while the [run-result receipt](../integrations/run-result.md) and
|
||||
[Published JSON Output contract](../integrations/json-output.md) own the
|
||||
durable result formats.
|
||||
Use this workflow to run Notarius and consume domain files. [CLI Reference](../cli.md)
|
||||
owns commands, reports, and statuses; [Published JSON Output](../integrations/json-output.md)
|
||||
owns logical paths and payload links; [Operations](../operations.md) owns state
|
||||
and recovery. See the [complete D&D workflow](dnd-pipeline.md) for its inventory
|
||||
or [migration guidance](native-migration.md) for older callers.
|
||||
|
||||
For the maintained complete D&D workflow, including its transcript input,
|
||||
configured lane inventory, and downstream acceptance checklist, see
|
||||
[Consuming The Complete D&D Pipeline](dnd-pipeline.md).
|
||||
## Prepare And Run
|
||||
|
||||
## Run And Check The Process
|
||||
Use an argument vector with explicit paths appropriate to the deployment.
|
||||
Absolute paths avoid dependence on a service's working directory. Keep stdout
|
||||
and stderr separate, and choose caller cancellation/deadline behavior.
|
||||
|
||||
Optionally preflight a selected configuration and pipeline before work starts:
|
||||
Validate the application and pipeline offline:
|
||||
|
||||
```sh
|
||||
notarius config validate --config /path/to/notarius.yml --config-overlay production --pipeline pipeline-id
|
||||
```
|
||||
~~~sh
|
||||
notarius config validate --config /absolute/path/to/notarius.yml
|
||||
notarius pipeline validate pipeline-id \
|
||||
--config /absolute/path/to/notarius.yml --overlay production
|
||||
~~~
|
||||
|
||||
Omit `--config-overlay` when the composed base configuration is the intended
|
||||
selection; it is independent from any `--llm-profile` run override.
|
||||
Omit the overlay for the composed base. Pipeline validation needs no transcript,
|
||||
reference bytes, credentials, provider call, or state directory.
|
||||
|
||||
Invoke the run with explicit paths and machine-readable output. Capture
|
||||
standard output and standard error separately; do not combine them before
|
||||
processing the result.
|
||||
Then run with first-class source and output inputs:
|
||||
|
||||
```sh
|
||||
~~~sh
|
||||
notarius run pipeline-id \
|
||||
--config /path/to/notarius.yml \
|
||||
--config-overlay production \
|
||||
--input /path/to/source.json \
|
||||
--output-dir /path/to/output-root \
|
||||
--json
|
||||
```
|
||||
--config /absolute/path/to/notarius.yml --overlay production \
|
||||
--input-file /absolute/path/to/source.json \
|
||||
--output-dir /absolute/path/to/output-root --format json
|
||||
~~~
|
||||
|
||||
Use absolute paths for supplied input, configuration, output-root, and
|
||||
reference files. Pass each external reference as its own argument-vector pair;
|
||||
do not construct and invoke a shell command. An unqualified reference selector
|
||||
supplies that file to every compatible selected target. Lane and stage
|
||||
qualification are available for exceptional overrides, while generated
|
||||
same-run references remain part of configured pipeline composition. The
|
||||
[CLI reference](../cli.md#reference-selectors) owns the exact selector and
|
||||
precedence contract.
|
||||
Pass external references as separate arguments, for example
|
||||
`--reference party=/absolute/path/to/party.txt`. Unqualified selectors reach
|
||||
compatible selected targets; narrower selectors are exceptional overrides.
|
||||
Generated same-run handoffs stay configured, not CLI files. The
|
||||
[reference selector contract](../cli.md#reference-selectors) owns matching and
|
||||
precedence.
|
||||
|
||||
The maintained D&D subprocess workflow uses this facility for campaign context:
|
||||
Supply credentials through the intended process environment, not arguments,
|
||||
session IDs, generated files, or logs. Omitted session selection uses the
|
||||
input-based default; use the [native override](../cli.md#run-wide-llm-overrides)
|
||||
only when intentionally changing it.
|
||||
|
||||
```sh
|
||||
notarius run dnd-session \
|
||||
--config /absolute/path/to/notarius.yml \
|
||||
--input /absolute/path/to/transcripts/final.trimmed.json \
|
||||
--output-dir /absolute/path/to/notarius-output \
|
||||
--reference party=/absolute/path/to/references/party.txt \
|
||||
--reference players=/absolute/path/to/references/players.txt \
|
||||
--reference glossary=/absolute/path/to/references/glossary.txt \
|
||||
--reference spell_catalog=/absolute/path/to/references/spells.json \
|
||||
--json
|
||||
```
|
||||
## Check Completion Before Parsing
|
||||
|
||||
Only pass the external references available to and desired by the deployment.
|
||||
Notarius generates a stable prompt session for the resolved input module and
|
||||
exact input bytes; reference changes do not change it. Pass **--session-id**
|
||||
only when intentionally grouping different invocations under a different
|
||||
session. Supply credentials through Notarius's documented configuration and
|
||||
environment mechanisms, never as command-line arguments or generated
|
||||
secret-bearing configuration. In particular, a session identifier is
|
||||
provider-visible and is not a credential mechanism.
|
||||
Wait for completion. Only successful status permits interpreting stdout as a
|
||||
successful native report. Failed delivery can leave partial bytes; retain stderr
|
||||
and available locations rather than parsing a structured error envelope.
|
||||
|
||||
Wait for the process before interpreting standard output. Only an exit status
|
||||
of 0 permits decoding the receipt. On a nonzero exit, retain standard error for
|
||||
diagnosis and ignore all standard-output bytes: a failed receipt write may have
|
||||
left a partial document.
|
||||
Optionally decode the [small completion report](../cli.md#completion-report).
|
||||
Its `output_directory` is the exact run root; do not scan for the newest folder
|
||||
or guess a run ID. The report is not a persisted receipt, schema-version envelope,
|
||||
index, or validation/execution ledger.
|
||||
|
||||
## Discover Required Artifacts
|
||||
A completed run may contain supported fallback. The caller decides whether that
|
||||
is acceptable; inspect positive `fallback_count` when relevant. It counts
|
||||
emitting operations, not affected files. Reuse performs ordinary usability checks,
|
||||
not fresh semantic assessment. Callers needing new inference/assessment after
|
||||
policy changes deliberately select bypass/refresh under the
|
||||
[cache controls](../config.md#producer-controls).
|
||||
|
||||
Decode the successful receipt and accept the schema versions supported by the
|
||||
caller. Use its `output_directory` as the bundle root. For the production JSON
|
||||
output, resolve `index_file` under that root with a confinement check and reject
|
||||
an absolute path or a result that escapes the root.
|
||||
## Read Required Domain Files
|
||||
|
||||
Read the resulting `index.json` and locate each artifact by `lane_id`, not by a
|
||||
guessed filename. Before decoding a selected payload, verify its descriptor's
|
||||
media type and schema identity against the relevant published artifact
|
||||
contract. The JSON bundle contract links to the available lane contracts.
|
||||
Define required/optional lanes from the selected pipeline. Read their
|
||||
[documented paths](../integrations/json-output.md#bundle-layout) beneath the
|
||||
report directory and decode using the configured artifact contracts. For custom
|
||||
lane IDs, apply the unchanged filename mapping; collisions are command failures.
|
||||
No receipt/index traversal, manifest descriptors, or reporting sidecars are needed.
|
||||
|
||||
If `index.json` has an `evidence_context` descriptor, treat it as a
|
||||
pipeline-wide artifact rather than a lane entry. Verify its six descriptor
|
||||
fields before decoding the linked file according to the [Published Evidence
|
||||
Context contract](../integrations/evidence-context.md). Decode its top-level
|
||||
source-unit array as a reading excerpt. Obtain authoritative citations and lane
|
||||
provenance from the normalized lane artifacts; the excerpt has neither and its
|
||||
nearby units do not widen a lane artifact's cited source reference.
|
||||
Optional chunk-map/evidence files use their documented paths when enabled and
|
||||
available. Decode supported payload schemas. Evidence is a reading excerpt, not
|
||||
authoritative lane attribution or a widened citation; normalized lane payloads
|
||||
retain those contracts.
|
||||
|
||||
A zero exit status may still report rejected outputs, warnings, or absent
|
||||
lanes. The caller decides which lane IDs are required for its own work and
|
||||
which are optional; it should make that decision explicitly rather than infer
|
||||
failure from the receipt counts alone.
|
||||
Process completion, required artifact availability/compatibility, and caller
|
||||
acceptance requirements are separate checks. A count alone cannot identify
|
||||
which files were produced.
|
||||
|
||||
When complete validation is required, also require receipt
|
||||
`validation_status: approved` and inspect `validation_summaries`. A successful
|
||||
run with `validation_status: incomplete` contains a structurally valid result
|
||||
that advanced after validator execution could not complete under the configured
|
||||
`warn_continue` policy. It is not reusable checkpoint state and should not be
|
||||
silently treated as fully reviewed by the caller.
|
||||
## Preserve Useful Evidence
|
||||
|
||||
## Preserve Provenance And Handle Data Carefully
|
||||
Keep domain files and caller-owned invocation/report records useful for provenance.
|
||||
Reports can be explicitly redirected; Notarius does not automatically persist a
|
||||
second report or require a metadata bundle. Debug is optional private evidence,
|
||||
not a consumer protocol.
|
||||
|
||||
Keep the receipt with the published `manifest.json`, and retain
|
||||
`rejected.json`, `warnings.json`, and `diagnostics.json` when review or later provenance requires
|
||||
them. Treat the input, output bundle, cache, debug bundle, and captured process
|
||||
logs as potentially sensitive data. Apply the caller's access controls and
|
||||
retention policy, and avoid copying secrets into arguments, logs, or
|
||||
provenance records. An evidence-context artifact contains source-unit text and
|
||||
metadata and can cover most of an input; preserve and share it only when that
|
||||
source content is authorized for the recipient.
|
||||
Post-publication capture/report failure can return nonzero while leaving valid
|
||||
artifacts. Preserve available files for deliberate recovery, but do not promote
|
||||
partial stdout or incomplete publication to ordinary consumer success.
|
||||
[Operations](../operations.md#output-bundles) explains those boundaries.
|
||||
|
||||
Inputs, artifacts, evidence, cache, debug, and streams may contain sensitive data.
|
||||
Apply intended access, sharing, and retention controls. Evidence can cover the
|
||||
whole source and is not inherently less sensitive.
|
||||
|
||||
Reference in New Issue
Block a user