notarius/docs/internal/modules.md

# Modules

Production modules live under `internal/modules`. Each module implements one
contract from `internal/framework/contracts`, exposes a `ModuleSpec`, and
registers itself with the matching pipeline registry.

The CLI production catalog currently registers only the modules listed here.

## Contract Pattern

A production module package should provide:

- a stable module key;
- a constructor such as `New`;
- the relevant contract implementation;
- `ModuleSpec`;
- `Register`;
- focused tests for registration, options, contract behavior, and errors.

Module specs should describe capabilities accurately. Resolution uses specs to
reject incompatible pipelines before execution.

Extractor modules that accept auxiliary reference material must declare slots
through both `ReferenceSlots()` and `ModuleSpec().ReferenceSlots`. The runtime
slot list and registry metadata should match so config validation can inspect
slots without constructing extractor instances. A slot declaration names the
slot, whether it is required, accepted media types, whether multiple items are
allowed, and any byte limit. Empty `AcceptedMediaTypes` means any inferred
media type is accepted, though the file must still be UTF-8 text. When a slot
declares accepted media types, Notarius compares the canonical base media type
inferred from the file extension, case-insensitively and without parameters.

Reference content is delivered only to the lane extractor through
`contracts.ExtractionRequest.References`. It is not source evidence and must not
be converted into `SourceRef` values. If a module prompt uses references, load
the prompt bundle with the same declared slots and render with
`RenderUserSystemWithReferences`. Prompt templates may use the `reference`
function for content and the `hasreference` function for conditional sections.
Prompt metadata hashes remain based on template source, not rendered reference
bytes.

Chunk modules receive the structured LLM client through `contracts.ChunkRequest`
when they need model-backed chunking. The pipeline runner validates generic
chunk result invariants before extraction; module-owned policies may be stricter
but must stay within the module package.

## `seriatim` Input

Package: `internal/modules/input/seriatim`

The `seriatim` adapter parses Seriatim transcript JSON into a generic source
document. It owns transcript JSON details, source ID selection, source digest
creation, transcript segment validation, and segment metadata mapping.

Provides:

- `source.transcript`
- `transcript.speaker`
- `transcript.timestamps`

External JSON shape belongs in the Seriatim integration doc.

## `generic` Chunker

Package: `internal/modules/chunk/generic`

The `generic` chunker splits source units into ordered chunks. It validates the
source document, clones source units, assigns chunk IDs such as `chunk-000001`,
and records chunk metadata for start unit, end unit, and unit count.

The pipeline runner canonicalizes chunk units from the source document by ID
before extractors and mergers run. Chunker-owned context should stay in
`SourceChunk.Metadata`.

Options:

- `max_units`: positive integer, default `50`;
- `overlap_units`: non-negative integer, default `0`, and less than
  `max_units`.

Provides:

- `chunks`

## `dnd/scenes` Chunker

Package: `internal/modules/chunk/dnd/scenes`

The `dnd/scenes` chunker uses the structured LLM client to divide transcript
source units into coherent D&D scenes. It renders embedded prompts, loads the
embedded structured response schema, validates model-authored source-unit
boundaries, and converts each scene into a deterministic source chunk.

Requires:

- `source.transcript`

Provides:

- `chunks`
- `chunks.scenes`

Options: none. Non-empty options are rejected.

The chunker enforces full source-unit coverage from the first source unit to the
last, exact source-unit IDs, sequential contiguous scenes, and no overlap. It
assigns chunk IDs such as `scene-000001` and stores scene metadata including
title, primary mode, participants, summary, boundary note, confidence, boundary
unit IDs, and unit count. Boundary caveats become warnings with reason code
`scene_boundary_caveat`. Whitespace-only caveats are treated as malformed
structured output rather than silently dropped.

Malformed model output fails explicitly rather than falling back to another
chunker. The chunker exposes prompt and response-schema provenance through
top-level `module_metadata.chunker` without raw prompts, raw schemas, source
text, or secrets.

## `dnd/spells` Extractor

Package: `internal/modules/extract/dnd/spells`

The `dnd/spells` extractor owns D&D spell-cast artifact semantics. It renders
embedded prompts, loads the embedded structured response schema, calls the
structured LLM client, converts spell-cast responses into artifact candidates,
and supplies deterministic validators.

Requires:

- `chunks`
- `source.transcript`

Provides:

- `dnd.spell_casts`

Artifact type and schema version:

- artifact type: `dnd.spell_cast`
- schema version: `v1`

The extractor adds prompt and response-schema provenance to lane manifest
metadata under `artifact_lanes[].metadata.extractor`. Durable artifact payload
details belong in the
[D&D spell artifact contract](../integrations/dnd-spell-artifacts.md).

The extractor declares optional `roster` and `glossary` reference slots
accepting UTF-8 text without narrowing accepted media types. Its prompt frames
references as supporting disambiguation material only; spell-cast artifacts must
still be grounded in the source transcript.

## D&D Spell Validators

The spell extractor returns two built-in validators:

- `dnd/spells/shape`: rejects malformed payloads and missing required fields.
- `dnd/spells/source_refs`: rejects candidates without valid source references.
  It also emits a warning when the extracted spell name is not found in the
  cited source text.

Reason codes include:

- `invalid_payload`
- `missing_required_field`
- `missing_source_ref`
- `invalid_source_ref`
- `spell_not_near_source`

These validators are supplied by the extractor when no validators are configured
for the lane.

## `appendorder` Merger

Package: `internal/modules/merge/appendorder`

The `appendorder` merger clones and appends candidates in chunk order. It does
not deduplicate or reconcile candidates.

Provides:

- `merged`

## `noop` Normalizer

Package: `internal/modules/normalize/noop`

The `noop` normalizer clones merged candidates and returns them unchanged.

Requires:

- `merged`

Provides:

- `normalized`

## `json` Output

Package: `internal/modules/output/json`

The `json` output encoder converts approved artifacts, rejected artifacts,
warnings, and the run manifest into logical JSON output files. It groups
approved artifacts by artifact type and sanitizes artifact-type file names.

Requires:

- `normalized`

Provides:

- `encoded`

Durable output file shapes belong in the
[JSON output contract](../integrations/json-output.md). Operator behavior
belongs in [Operations](../operations.md).

## Production Registration

Production registration is centralized in `internal/cli/catalog.go`.

Do not make framework code import production modules. The CLI wires production
modules at the application boundary; tests may provide fake registries or fake
catalogs directly.

## Adding A Module

When adding a module, keep source-format and extraction-domain boundaries clear:

- input modules may know external source formats;
- extract modules may know artifact semantics and prompt/schema assets;
- merge and normalize modules own candidate combination and reconciliation;
- output modules own serialization, not diagnostics or CLI reporting.

Update [Development](../policy/development.md), [Configuration](../config.md),
internal docs, integration docs, and examples when the new module becomes
implemented production behavior.