notarius/docs/adr/0003-typed-interfaces-with-two-zone-data-model.md

# ADR-0003: Strongly typed stage interfaces with a two-zone data model

**Status:** Accepted
**Date:** 2026-07-13

## Context

Pipeline stages must exchange source data and extracted artifacts. Universal
source data has one engine-wide meaning, while extracted artifacts have
domain-specific shapes. Passing opaque bytes or `any` between all stages would
make invalid wiring and merge behavior runtime concerns. Requiring JSON at
every handoff would preserve interoperability but discard useful Go type safety
while all modules are in-process.

The framework must also support multiple configured artifact domains, durable
checkpoints, diagnostics, and output encoders without making those consumers
depend on every domain's Go types.

## Decision

Notarius uses two typed data zones followed by one serialized boundary.

### Source zone

Input and chunk stages use conservative, engine-owned document, segment, chunk,
and source-reference types. Their exact Go names are implementation details.
Every segment carries engine-owned source provenance identifying the source
location from which it was produced. Chunks preserve the ordered provenance of
their segments.

Source-format-specific fields remain in input modules or explicitly namespaced
metadata; they do not become framework contracts.

### Domain artifact zone

Each artifact lane has one domain-owned Go artifact type `T`. Its extract,
merge, normalize, and domain-aware validation implementations use generic,
strongly typed contracts over the same `T`. Raw JSON, opaque bytes, and `any`
are not stage-handoff contracts within a lane.

Each registered domain artifact type supplies a codec for `T`. The codec owns:

- stable schema identity and an explicit schema version;
- JSON serialization and deserialization;
- the media type and schema metadata required at serialized boundaries; and
- rejection of data that cannot be represented by the declared artifact
  schema.

An artifact type's JSON representation is a maintained domain contract.
Changing it incompatibly requires a new schema version.

Extract, merge, and normalize may change the contents of `T`, but they do not
change the lane's canonical Go artifact type or artifact schema identity. An
extractor maps any provider- or prompt-specific response type into `T` before
returning. A future lane that requires different artifact types at different
stages requires a new architectural decision.

### Serialized boundary

After normalization, each typed artifact is converted into an engine-owned
serialized artifact containing bytes, media type, and schema metadata. Output
aggregation and output encoders consume this type-erased form. Intermediate
checkpoint and debug encodings do not become stage-handoff contracts.

LLM transport, checkpoints, and opt-in debug recording are also explicit
serialization boundaries. They may encode or decode a typed artifact through
its domain codec, but they do not change the in-memory type used between
extract, merge, normalize, and typed validators. Checkpoint reuse requires a
compatible schema identity and version.

An LLM structured-response schema is a module transport contract and may differ
from the domain artifact schema. The calling module owns the response type and
maps it into the canonical `T`; the artifact codec remains authoritative for
artifact checkpoints and output serialization.

The framework may use private type-erased adapters to store heterogeneous lane
registrations and execute configured domains. Such an adapter must assemble a
type-consistent lane before execution and must not expose `any` or raw payloads
as module-facing handoffs inside the domain artifact zone.

### Construction and dependencies

Every module operation accepts `context.Context`. Modules receive stable runtime
collaborators through an injected dependency set at construction time. In
particular, LLM-using modules receive the application-provided structured LLM
client and do not construct provider clients or bypass shared scheduling.

The application boundary enforces one configurable global upper bound on
in-flight LLM calls across all stages, lanes, retries, and validators.

Configuration options are parsed and validated while a module is constructed,
before that module executes. Per-run data such as source material, references,
session identity, and lane identity remains operation input rather than a
construction dependency.

## Alternatives considered

- Pass raw bytes between stages. This maximizes decoupling but moves wiring,
  parsing, and merge errors to runtime and prevents domain types from being the
  canonical in-process contract.
- Require JSON plus schemas at every stage boundary. This is appropriate for an
  out-of-process boundary, but adds serialization and parsing inside the current
  in-process pipeline. The stable codec contract preserves this upgrade path if
  remote plugins are introduced.
- Use a uniform `Process(any) (any, error)` contract. This simplifies a fully
  dynamic engine but turns incompatible module composition into type assertions
  and runtime failures. The fixed topology does not require that tradeoff.

## Consequences

Domain pipelines gain compile-time handoff safety and explicit merge semantics.
Serialization, schema compatibility, checkpoint decoding, and output erasure
have named owners. Dynamic registration requires a small erased adapter around
each typed lane, and generic stage implementations must be instantiated for a
specific artifact type or behavior rather than manipulating arbitrary JSON.

The engine-owned source model becomes a long-lived contract and must evolve
conservatively. Domain authors must maintain a codec and versioned schema in
addition to their Go artifact type.