Update native operations, consumer migration, and version checks

This commit is contained in:
2026-11-22 06:32:22 +00:00
parent 5e58327032
commit 2c27695df9
18 changed files with 675 additions and 967 deletions

View File

@@ -1,9 +1,9 @@
# Accepted Chunk Map
This document defines the optional durable `chunk-map.json` artifact in a
[published JSON bundle](json-output.md). It describes the accepted,
materialized chunk plan used by one run. It is not a lane payload and is never
an input to a later pipeline step.
[published JSON bundle](json-output.md). It describes the materialized chunk
plan used by one run, including a supported qualified fallback. It is not a lane
payload and is never an input to a later pipeline step.
## Contract Identity
@@ -16,8 +16,8 @@ an input to a later pipeline step.
| Schema name | `notarius_source_chunk_map_v1` |
| Schema version | `v1` |
The optional `chunk_map` descriptor in `index.json` identifies this artifact.
Export is controlled by the JSON output binding described in
Read this logical path directly when enabled and available; no discovery
descriptor is required. Export is controlled by the JSON output binding in
[Configuration](../config.md#module-bindings-and-validators).
## Wire Shape
@@ -30,7 +30,7 @@ Every payload has these required fields:
| `source_digest` | Lower-case `sha256:` digest of that source document. |
| `plan_digest` | Lower-case `sha256:` digest of the logical chunk plan. |
| `requested_chunker` | Chunk module selected by the resolved pipeline. |
| `producer` | Original accepted-plan producer. `input_module` and `chunk_module` are required; `llm_profile` is optional. |
| `producer` | Current run's plan component selections. `input_module` and `chunk_module` are required; `llm_profile` is optional and applies only to an LLM-backed chunker. Not historical cache execution provenance. |
| `plan_annotations` | Plan-level annotation namespace map; `{}` when none are present. |
| `chunks` | Non-empty execution-order chunk collection. |
@@ -47,9 +47,11 @@ valid JSON and are retained without interpreting a module-specific namespace.
`chunks` are in execution order. Their indexes are contiguous, start at zero,
and equal their array positions; chunk IDs are unique. The emitted map is built
only after the selected plan has been accepted and materialized against the
source document, so its ranges, unit counts, annotations, and digests describe
that exact plan.
after the selected plan has been materialized against the source document,
so its ranges, unit counts, annotations, and digests describe that exact plan.
Supported fallback still satisfies those invariants without asserting ordinary
semantic approval; run-level qualification belongs to the
[CLI report](../cli.md#completion-report).
The codec rejects malformed JSON, trailing content, unknown fixed-object
fields, invalid identities or digests, invalid annotations, duplicate chunk
@@ -67,12 +69,13 @@ annotations, producer identity, and ordered chunks.
## Publication And Compatibility
The map is present only when a chunk plan was accepted and its export is
enabled. It remains publishable if a later lane is rejected, but is absent when
chunk-plan validation rejects the plan. `requested_chunker` identifies the
current pipeline selection, while `producer` identifies the component that
originally produced the accepted plan; they may differ when an accepted plan is
reused.
The map is published when export is enabled, a materialized plan is available,
and the run reaches publication. Payload fields and schema are unchanged.
`requested_chunker` and `producer` describe the current composition; unified
cache entries retain the logical plan, not an earlier run's producer/profile
ledger. Reuse rematerializes against the current source without semantic
reassessment or claims of historical model execution. A strict failure does not
publish a successful bundle merely to expose this auxiliary.
The map contains structure rather than source content: it excludes transcript
bytes, source-unit metadata, chunk text, private model output, reference

View File

@@ -9,9 +9,8 @@ logical-file discovery is owned by [Published JSON Output](json-output.md).
## Identity And Discovery
When enabled, the JSON bundle contains `evidence-context.json` and an
`index.json` `evidence_context` descriptor with the same six fields as other
pipeline-wide artifact descriptors.
When enabled and available, the JSON bundle contains `evidence-context.json`
at that documented logical path; no index descriptor is required.
| Property | Value |
| --- | --- |
@@ -22,16 +21,16 @@ pipeline-wide artifact descriptors.
| Schema version | `v1` |
| Logical file | `evidence-context.json` |
Consumers must discover the file from the descriptor, verify all six descriptor
fields, and decode only a supported schema version. The descriptor is optional:
its absence means evidence publication was not enabled for that bundle.
Consumers select this path from the configured output policy and decode the
supported payload contract below. Disabled or unavailable evidence is not
published. The identity and JSON shape above remain unchanged.
## Payload
The v1 payload is a top-level JSON array of generic source units. There is no
wrapper, source-level metadata, context grouping, lane identifier, or evidence
reference in the payload. An enabled configuration with no contributing
accepted evidence publishes `[]`.
evidence publishes `[]`.
```json
[
@@ -73,14 +72,16 @@ annotations or reshape source-owned metadata.
## Selection And Citations
The framework obtains direct source references only through typed evidence
projections of accepted normalized artifacts in the configured lane allowlist.
projections of published normalized artifacts in the configured lane allowlist,
including supported fallbacks and reused results.
It validates each reference against the current source document, expands its
range by `window_units` source-unit positions on each side, clamps at document
boundaries, and takes the union of all expanded ranges. The output contains
each selected source unit once in source-document position order, regardless
of numeric unit IDs. Repeated references, overlapping windows, and citations
from multiple lanes do not duplicate a unit. Rejected, failed, absent,
inactive, and unselected lanes contribute nothing.
from multiple lanes do not duplicate a unit. Failed, absent, inactive, and
unselected lane results contribute nothing. Evidence from a supported fallback
does not imply ordinary semantic approval; the completion report qualifies the run.
Normalized lane artifacts remain authoritative for citations and for which lane
cited a range. The excerpt has no lane attribution and must not be used to
@@ -94,10 +95,10 @@ the framework does not truncate the excerpt to meet an arbitrary size limit.
## Consumer Responsibilities And Data Handling
The artifact is additive to the JSON bundle and is not a lane payload,
normalized-output count, checkpoint, or generated reference. Consumers that
do not need it must tolerate an absent descriptor. Consumers that do use it
should validate the descriptor and payload before use, retain the artifact with
The artifact is additive to the JSON bundle and is not a lane payload or
generated reference. It counts as a deliverable when published, not as a lane.
Consumers that do not need it can ignore it. Consumers that do use it should
validate the payload before use, retain the artifact with
its schema identity when needed for a run record, and read citations from the
corresponding normalized lane artifacts.

View File

@@ -1,196 +1,79 @@
# Published JSON Output
This document defines the logical JSON bundle emitted by the production JSON
output encoder. The bundle’s physical destination, atomic publication, and
retention are operational concerns; see [Operations](../operations.md#output-bundles).
Output configuration, including chunk-map and evidence-context publication, belongs in
[Configuration](../config.md#module-bindings-and-validators).
This contract defines logical files emitted by the production JSON output
encoder. [Operations](../operations.md#output-bundles) owns physical placement,
publication failures, permissions, and retention. [Configuration](../config.md#module-bindings-and-validators)
owns output options; the [CLI completion report](../cli.md#completion-report)
identifies a successful run's directory.
## Bundle Layout
All paths below are logical, relative, slash-separated bundle paths. The
encoder always emits the first five JSON files below and adds lane or
pipeline-wide artifact files when their corresponding artifacts are available:
A subprocess caller first obtains the physical bundle root from the
[run-result receipt](run-result.md), then resolves `index.json` beneath that
root for the logical discovery described here.
Paths are relative, slash-separated paths beneath the run-specific directory:
| Path | Purpose |
| --- | --- |
| `index.json` | Entry point that names the other published files and lane payloads. |
| `manifest.json` | Run provenance and result summaries. |
| `rejected.json` | Rejected pipeline outputs. |
| `warnings.json` | Actionable process-degradation warnings. |
| `diagnostics.json` | Accepted-artifact quality advisories and normalization observations. |
| `lanes/<safe-lane-id>.json` | One normalized artifact payload for each lane. |
| `chunk-map.json` | Optional accepted chunk map, when its export is enabled and available. |
| `evidence-context.json` | Optional selected source-unit excerpt, when evidence publication is enabled. |
| `lanes/<safe-lane-id>.json` | One normalized domain artifact per available selected lane, including supported qualified fallback. |
| `chunk-map.json` | Optional materialized chunk map, when enabled and available. |
| `evidence-context.json` | Optional selected source-unit excerpt, when enabled and available. |
JSON files are pretty-printed with a trailing newline. Lane payloads are
accepted only when their media type is `application/json`.
There is no required index, manifest, persisted receipt, rejection file, warning
file, or diagnostic sidecar. Expected lanes come from pipeline configuration;
optional auxiliaries use the paths above. Consumers need no metadata bundle or
debug data to decode artifacts.
## `index.json`
The lane filename mapping is unchanged: trim the ID, replace characters outside
ASCII letters, digits, periods, underscores, and hyphens with underscores,
neutralize `..` sequences, and trim leading/trailing periods and underscores.
An empty resulting name or colliding paths fails encoding. Maintained D&D lane
IDs already map directly to their filenames.
`index.json` is the bundle’s discovery document. An approved run with no
normalized lanes has this valid minimal index:
```json
{
"manifest_file": "manifest.json",
"output_files": [],
"rejected_file": "rejected.json",
"warnings_file": "warnings.json",
"diagnostics_file": "diagnostics.json"
}
```
| Field | Required | Meaning |
| --- | --- | --- |
| `manifest_file` | Yes | Always `manifest.json`. |
| `output_files` | Yes | Lane descriptors sorted by `lane_id`. |
| `rejected_file` | Yes | Always `rejected.json`. |
| `warnings_file` | Yes | Always `warnings.json`. |
| `diagnostics_file` | Yes | Always `diagnostics.json`. |
| `chunk_map` | No | Descriptor for the pipeline-wide `chunk-map.json`; never a lane descriptor. |
| `evidence_context` | No | Descriptor for the pipeline-wide `evidence-context.json`; never a lane descriptor. |
Each lane descriptor has required `lane_id` and `file`. It may also include
`media_type`, `module_key`, `schema_id`, `schema_name`, and `schema_version`
when supplied by the normalized artifact. Each pipeline-wide artifact
descriptor (`chunk_map` or `evidence_context`) contains `artifact_kind`,
`file`, `media_type`, `schema_id`, `schema_name`, and `schema_version`. Their
payloads are defined by the [Accepted Chunk Map contract](chunk-map.md) and
[Published Evidence Context](evidence-context.md), respectively.
The lane path is derived from its lane ID. Characters outside letters, digits,
periods, underscores, and hyphens become underscores; `..` sequences are
neutralized; leading and trailing periods and underscores are removed. A lane
that produces an empty name, or two lanes that produce the same path, makes
output encoding fail.
Files are pretty-printed JSON with a trailing newline. Lane payloads require
`application/json`; the encoder validates the artifact boundary without
wrapping payloads in reporting metadata. Report presentation format does not
change artifact encoding.
## Lane Payloads
Each `lanes/<safe-lane-id>.json` file is the codec-owned normalized JSON for
that lane. Consumers should use the index descriptor’s schema identity rather
than infer a lane schema from its name. The current D&D payload contracts are
[spells](dnd-spell-artifacts.md), [NPC registry](dnd-npc-registry-artifacts.md),
[NPC occurrences](dnd-npc-occurrence-artifacts.md),
[combat turns](dnd-combat-turn-artifacts.md),
[item registry](dnd-item-registry-artifacts.md),
[item occurrences](dnd-item-occurrence-artifacts.md),
[scene descriptions](dnd-scene-description-artifacts.md),
[enemy events](dnd-enemy-event-artifacts.md),
[location registry](dnd-location-registry-artifacts.md), and
[location occurrences](dnd-location-occurrence-artifacts.md).
Each lane file is its codec-owned normalized JSON value. Select the contract
from the configured lane/module, not an absent discovery descriptor. Existing
D&D payload contracts remain unchanged:
## `manifest.json`
- [Spells](dnd-spell-artifacts.md).
- [NPC registry](dnd-npc-registry-artifacts.md) and
[NPC occurrences](dnd-npc-occurrence-artifacts.md).
- [Combat turns](dnd-combat-turn-artifacts.md).
- [Item registry](dnd-item-registry-artifacts.md) and
[item occurrences](dnd-item-occurrence-artifacts.md).
- [Scene descriptions](dnd-scene-description-artifacts.md).
- [Enemy events](dnd-enemy-event-artifacts.md).
- [Location registry](dnd-location-registry-artifacts.md) and
[location occurrences](dnd-location-occurrence-artifacts.md).
`manifest.json` is published provenance, not a copy of lane payloads or a
checkpoint store. Fields without a value may be omitted. Its top-level fields
group into the following externally observable summaries:
Intrinsic fields, identities, source references, and schemas are preserved.
Supported fallback meets intrinsic/downstream invariants without asserting
ordinary semantic approval. Run qualification belongs in the native completion
report, not new payload fields or sidecars.
| Group | Fields |
| --- | --- |
| Run identity and result | `run_id`, `pipeline_id`, `pipeline_digest`, `schema_version`, `validation_status`, `started_at`, `completed_at` |
| Resolved components | `input_module`, `chunker`, `extractors`, `merger`, `normalizer`, `output_encoder`, `artifact_lanes`, `validator_chains`, `module_metadata` |
| Source and references | `source_digests`, `references` |
| Published result summaries | `normalized_outputs`, `rejected_outputs`, `validation_summaries` |
| Execution summaries | `chunk_plan`, `checkpoint_decisions`, `llm_profiles`, `metadata` |
## Auxiliary Artifacts
`references` records provenance such as the target, slot, origin, digest,
media type, size, and generated-artifact identity. It does not contain
reference content. `normalized_outputs` and `rejected_outputs` likewise
summarize results without embedding lane payload bytes. A chunk-plan summary is
provenance for the plan used by this run; cache records, debug artifacts, and
other operational state are not published as bundle files.
[Chunk map](chunk-map.md) and [evidence context](evidence-context.md) are
pipeline-wide artifacts, not lanes. Their existing payload schemas and logical
paths remain; no index descriptor is required.
Each `validation_summaries` entry is a bounded outcome for one producer result.
It has required `status`, `producer_attempt_count`, and `terminal_action`;
the stage and affected step, lane, module, or chunk identity are present when
applicable. `status` is `complete`, `rejected`, or `incomplete`.
`rejecting_validators`, `reason_codes`, and `incomplete_validators` preserve
configured validator order and omit later duplicates. Entries contain no raw
candidate response, correction guidance, validator diagnostic message, or
artifact payload. The same shape may appear as `validation` on an affected
rejection entry.
Enabled, available auxiliaries count as deliverables, including when built
from reused results or supported fallback. Disabled/unavailable auxiliaries are
not published. Evidence can cover the complete source; authoritative citations
remain in normalized lane payloads.
When present, `metadata.session_id` is the effective non-secret routing
correlation identifier used for the run. It can be visible to providers and is
not a substitute for a cache or checkpoint identity. Its generation and
override behavior are defined by the [CLI reference](../cli.md#run).
## Consumer Use And Compatibility
Each `llm_profiles` entry identifies effective, non-secret LLM execution
provenance:
Check process status before decoding a successful native report. Use its absolute
run directory, then the configured lane and selected auxiliary paths. Decode
each file against its domain contract. The [subprocess guide](../consumers/subprocess.md)
and [complete D&D workflow](../consumers/dnd-pipeline.md) explain caller tasks.
| Field | Required | Meaning |
| --- | --- | --- |
| `id` | Yes | Selected PromptKit profile identifier. |
| `provider` | No | Notarius adapter provider identifier. |
| `model` | No | Effective provider model identifier. |
| `backend_id` | No | Effective PromptKit backend registration identifier. Endpoint-only profiles omit it. |
| `reasoning_effort` | No | Effective opaque provider reasoning setting. An empty or explicitly cleared setting is omitted. |
These values describe observed execution; they are not a backend-registration
interface. Entries that differ by backend or effective reasoning remain
distinct even when their profile, provider, and model are otherwise equal.
## Rejections, Warnings, And Diagnostics
`rejected.json` is always an object with a `rejected` array. Each entry has
required `stage` and `message`; `step_id`, `lane_id`, `module_key`, `chunk_id`,
`chunk_index`, `validator_name`, `reason_code`, `attempt_count`, and
`diagnostic_artifact_path` are present only when applicable. An entry may also
contain the bounded `validation` summary described above; the existing singular
validator and reason fields remain the first configured rejection for
compatibility.
`warnings.json` is always the `notarius.warnings.v2` envelope:
```json
{
"schema_version": "notarius.warnings.v2",
"group_count": 0,
"occurrence_count": 0,
"groups": []
}
```
It contains only process warnings. `group_count` is exact, and
`occurrence_count` is the exact sum of its group occurrence counts.
`diagnostics.json` is always the `notarius.diagnostics.v1` envelope:
```json
{
"schema_version": "notarius.diagnostics.v1",
"group_count": 0,
"occurrence_count": 0,
"truncated": false,
"unrepresented_occurrence_count": 0,
"groups": []
}
```
It contains only advisory and observation groups. `group_count` counts groups
represented in `groups`; `occurrence_count` includes both represented and
unrepresented occurrences. When `truncated` is true,
`unrepresented_occurrence_count` is the exact number omitted from group
representation.
Each group has `disposition`, `category`, `reason_code`, framework-owned
`origin`, exact `occurrence_count`, bounded `samples`, and
`omitted_sample_count`. Samples carry safe `scope` and `message`, plus a chunk
ID and zero-based chunk index when applicable. A group retains at most three
distinct samples. The framework fails rather than truncating actionable
warnings beyond 128 groups; it represents at most 256 advisory/observation
groups and records further occurrences through the diagnostic truncation
fields above.
## Compatibility
The index is the authoritative map from a logical lane to its published
payload. Consumers must tolerate omitted optional manifest and descriptor
fields, and should rely on the linked artifact contract for each lane’s JSON
shape. This contract describes the published logical bundle only; it does not
promise a filesystem layout or expose internal state formats.
Removing reporting/discovery files is intentional, not a domain schema or
filename-mapping change. [Migration guidance](../consumers/native-migration.md)
replaces receipt/index traversal. Debug and cache are not public output contracts.
Post-publication failure can leave valid files; failed publication can leave a
partial directory. Neither supplies a successful completion report.

View File

@@ -1,93 +0,0 @@
# Run Result Receipt
`notarius run --json` writes this receipt to standard output when a run
completes successfully. It lets a subprocess caller discover the physical root
of the published output bundle without parsing interactive command output.
Command syntax, streams, and exit statuses are defined in the
[CLI reference](../cli.md); logical files within the bundle are defined in the
[Published JSON Output contract](json-output.md).
## Schema
The current schema version is `notarius.run-result.v2`.
| Field | Required | Meaning |
| --- | --- | --- |
| `schema_version` | Yes | Exactly `notarius.run-result.v2`. |
| `run_id` | Yes | The finalized Notarius run identifier. |
| `pipeline_id` | Yes | The effective pipeline identifier. |
| `output_directory` | Yes | Absolute path to the published, run-specific output bundle. |
| `index_file` | For the production JSON output | Logical path `index.json`; omitted for other output modules. |
| `normalized_output_count` | Yes | Number of final normalized outputs. |
| `rejected_output_count` | Yes | Number of recorded rejected outputs. |
| `warning_group_count` | Yes | Exact number of actionable warning groups. |
| `warning_occurrence_count` | Yes | Exact occurrences represented by actionable warning groups. |
| `diagnostic_group_count` | Yes | Number of represented advisory and observation groups. |
| `diagnostic_occurrence_count` | Yes | Advisory and observation occurrences, including unrepresented occurrences. |
| `diagnostics_truncated` | Yes | Whether advisory/observation group representation was truncated. |
| `validation_status` | Yes | The final run manifest validation status. |
| `validation_summaries` | No | Bounded per-producer validation outcomes; present when producer work ran. |
| `debug_directory` | No | Absolute path to the run-specific debug bundle when requested debug capture completed. |
For the production `json` output module, `index_file` is present only when the
completed run returned exactly one logical output file named `index.json`.
For another output module, its absence does not indicate a failed run.
`validation_status` is `approved`, `rejected`, or `incomplete`; `incomplete`
means one or more otherwise accepted results advanced under validator-failure
`warn_continue`.
```json
{
"schema_version": "notarius.run-result.v2",
"run_id": "run-1770000000000000000-0123456789abcdef0123456789abcdef",
"pipeline_id": "dnd-session",
"output_directory": "/work/results/run-1770000000000000000-0123456789abcdef0123456789abcdef",
"index_file": "index.json",
"normalized_output_count": 6,
"rejected_output_count": 2,
"warning_group_count": 1,
"warning_occurrence_count": 2,
"diagnostic_group_count": 3,
"diagnostic_occurrence_count": 5,
"diagnostics_truncated": false,
"validation_status": "incomplete",
"validation_summaries": [
{
"stage": "extract",
"lane_id": "spells",
"status": "incomplete",
"incomplete_validators": ["dnd/spells/source_refs"],
"producer_attempt_count": 1,
"terminal_action": "warn_continue"
}
]
}
```
## Paths And Bundle Discovery
`output_directory` and `debug_directory`, when present, are lexical absolute
paths. They identify the paths used by Notarius and do not resolve symlinks.
`output_directory` is the run-specific bundle, not the configured output root.
The receipt is a summary and discovery document. Its optional validation
summaries contain only stable status, identity, validator names, reason codes,
attempt counts, and terminal actions. It does not contain lane descriptors,
payloads, manifest payloads, rejection messages, warnings, raw model responses,
correction guidance, or file contents.
For the production JSON output, resolve `index_file` beneath
`output_directory`, reject path escapes, and use the
[Published JSON Output contract](json-output.md) to discover logical files and
lane payloads.
## Delivery And Compatibility
Notarius writes the receipt only after the output bundle has been published and
any requested debug terminal reporting has completed. Standard output is not
transactional: a result-write failure returns a nonzero status and can leave
partial bytes. Consumers must ignore standard output unless the process exits
with status 0.
Future versions may add optional fields to this schema. Consumers must tolerate
unknown fields. An incompatible field or semantic change requires a new
`schema_version` value.