Files
notarius/docs/integrations/chunk-map.md

4.1 KiB

Accepted Chunk Map

This document defines the optional durable chunk-map.json artifact in a published JSON bundle. It describes the materialized chunk plan used by one run, including a supported qualified fallback. It is not a lane payload and is never an input to a later pipeline step.

Contract Identity

Property Value
Artifact kind source/chunk-map
Logical file chunk-map.json
Media type application/json
Schema ID notarius.source.chunk_map
Schema name notarius_source_chunk_map_v1
Schema version v1

Read this logical path directly when enabled and available; no discovery descriptor is required. Export is controlled by the JSON output binding in Configuration.

Wire Shape

Every payload has these required fields:

Field Meaning
source_id Accepted source-document identity.
source_digest Lower-case sha256: digest of that source document.
plan_digest Lower-case sha256: digest of the logical chunk plan.
requested_chunker Chunk module selected by the resolved pipeline.
producer Current run's plan component selections. input_module and chunk_module are required; llm_profile is optional and applies only to an LLM-backed chunker. Not historical cache execution provenance.
plan_annotations Plan-level annotation namespace map; {} when none are present.
chunks Non-empty execution-order chunk collection.

Each chunks entry contains non-empty id, zero-based index, source_ref, positive unit_count, and an explicit annotations map. source_ref contains the same source_id as the top-level value plus positive inclusive start_unit_id and end_unit_id values. Endpoints identify source units; their numeric values do not by themselves establish source-document order.

Annotation namespaces are non-empty trimmed strings. Their values are arbitrary valid JSON and are retained without interpreting a module-specific namespace.

Ordering And Validation

chunks are in execution order. Their indexes are contiguous, start at zero, and equal their array positions; chunk IDs are unique. The emitted map is built after the selected plan has been materialized against the source document, so its ranges, unit counts, annotations, and digests describe that exact plan. Supported fallback still satisfies those invariants without asserting ordinary semantic approval; run-level qualification belongs to the CLI report.

The codec rejects malformed JSON, trailing content, unknown fixed-object fields, invalid identities or digests, invalid annotations, duplicate chunk IDs, non-contiguous indexes, and a plan_digest that does not match the reconstructed logical plan. The checked-in schema defines the strict JSON shape.

Valid Example

The compact source chunk-map fixture is decoded by the production codec and demonstrates an accepted map with annotations, producer identity, and ordered chunks.

Publication And Compatibility

The map is published when export is enabled, a materialized plan is available, and the run reaches publication. Payload fields and schema are unchanged. requested_chunker and producer describe the current composition; unified cache entries retain the logical plan, not an earlier run's producer/profile ledger. Reuse rematerializes against the current source without semantic reassessment or claims of historical model execution. A strict failure does not publish a successful bundle merely to expose this auxiliary.

The map contains structure rather than source content: it excludes transcript bytes, source-unit metadata, chunk text, private model output, reference content, debug data, and filesystem paths. Treat the exported map with the same care as other published output. Publication location and retention are defined in Operations.