notarius/docs/internal/llm.md

# LLM Runtime Internals

`internal/framework/llm` is Notarius’s provider-independent structured
completion boundary. It adapts framework requests to Scriptorium, bounds
provider calls, assembles registered prompt and schema assets, records selected
profiles, and redacts provider errors. The architectural boundary is defined in
[Architecture](../policy/architecture.md#llm-boundary); profile sources,
credentials, and concurrency settings belong in
[Configuration](../config.md#scriptorium-profiles) and
[Configuration](../config.md#concurrency-output-cache-and-debug).

## Structured Completion Boundary

Modules and LLM-backed validators depend only on
`contracts.StructuredLLMClient`. A completion request supplies a prompt ID and
version, optional profile and session IDs, named input material, variables, and
a caller-owned decode target. The successful response returns the validated raw
structured bytes together with non-secret provider, model, profile, and token
metadata.

The caller owns the domain behavior: it chooses the prompt, prepares inputs,
selects the private response schema, and interprets the decoded result. The
adapter does not own source evidence, artifact conversion, normalization, or
durable schemas. Those responsibilities remain with the module and its
[integration contract](../integrations/).

`ScriptoriumClient` validates the request target and prompt identity, maps each
named material to a Scriptorium inline artifact while preserving its origin URI,
forwards session and profile selection, then prepares and runs the prompt. It
returns Scriptorium’s validated raw bytes rather than re-encoding the decoded
target. An empty optional material is represented as one space so its named
input is retained by Scriptorium.

An empty request profile lets the prompt select its configured default. The CLI
prepares every explicitly selected binding profile before a run begins, so a
missing explicit profile fails before stage execution. Calls record the profile
actually selected by Scriptorium; the recorder deduplicates non-secret profile
identity, provider, and model values for manifest use.

## Shared Provider-Call Limit

Production construction creates one Scriptorium client and wraps it in one
scheduled client. The scheduler has a fixed, positive permit limit, serves
queued calls in FIFO order, and removes a queued call when its context is
cancelled. A granted permit is released exactly once on every completion path.

The scheduled wrapper surrounds every `CompleteStructured` call, so concurrent
lanes, pipeline retries, and LLM-backed validators share the same provider-call
ceiling. This ceiling is independent of pipeline worker concurrency; changing
worker counts cannot exceed the configured LLM limit. The configuration field
and its effective default are owned by
[Configuration](../config.md#concurrency-output-cache-and-debug).

## Prompt And Schema Assets

An `AssetRegistry` collects prompt and schema filesystems from production module
families. It flattens registered roots into the Scriptorium filesystems and
rejects invalid roots, unreadable assets, duplicate paths, and missing prompt
or schema files during preparation. The framework’s `promptfs` helper combines
module-owned prompt files with reusable domain fragments without making the
framework depend on D&D content.

Each LLM-backed module owns its prompt declaration, package-specific assets,
and private response schema. Shared D&D wording is owned by the D&D shared
asset package; the detailed D&D conventions are in
[D&D Module Internals](dnd.md). The mounted prompt assets used by a module also
determine its prompt fingerprint. Schema loaders validate JSON, attach identity
and digest metadata, make defensive copies, and expose diagnostics without raw
schema bytes.

Private response schemas validate a model transport envelope. They are not the
durable artifact schema and should not be documented as an external wire
contract. Durable formats and compatibility rules remain in the
[integration contracts](../integrations/).

## Prompt Maintenance And Backend Caching

Prompt message order and shared asset bytes are runtime behavior. Backend cache
reuse depends on the same preceding messages and content, not merely equivalent
meaning. Keep reusable shared assets byte-identical and keep stable material
before the inputs that vary per request wherever a prompt’s declared sequence
supports caching. Preserve the existing manifest order and cache-control hints
when editing a prompt.

D&D extraction manifests place the changing chunk transcript at the end of the
prompt after their reusable context. Scene chunking and NPC normalization use
their own declared message sequences because their inputs and work differ. The
family-specific asset and ordering rules belong in [D&D Module Internals](dnd.md).
Do not add tests that enforce a fixed message-prefix length; prompt-asset tests
should instead verify the meaningful asset sequence, inputs, and cache controls
of the prompt being changed.

## Validation, Repair, And Retries

Scriptorium performs prompt rendering, provider execution, and the prompt’s
structured-output validation. The adapter reports an empty result, validation
failure, empty structured body, or decode failure as
`ErrInvalidStructuredOutput`, while retaining the returned raw bytes and debug
material when they exist. Provider failures remain operational errors rather
than output-validation failures.

Prompt-declared repair is executed within Scriptorium’s structured-output flow.
The current production D&D prompt manifests set repair attempts to zero. That
setting does not replace pipeline retry behavior: a binding’s configured retry
count reruns its stage attempt after an error or rejection, and an exhausted
rejection is a recorded output rather than a provider error. The pipeline owns
attempt lifecycle, validation chains, and retry diagnostics; see
[Pipeline Internals](pipeline.md#validation-retries-and-output) and the
[binding reference](../config.md#module-bindings-and-validators).

## Observability And Redaction

When debug recording is enabled, the pipeline decorates the shared client. The
wrapper records prepared prompt and response material, timing, selected profile
and model, and call identifiers in the run’s debug bundle, including material
available from a failed structured completion. For a successful completion, a
debug-write failure is surfaced; when the completion already failed, its call
error remains the result. Debug-bundle location, retention, and handling are
operational concerns documented in [Operations](../operations.md#debug-bundles).

Run manifests receive selected profile summaries and component identities, not
prompt, schema, source, reference, or response content. Provider error text is
wrapped with prompt context and bearer credentials are redacted before it
crosses the runtime boundary. Known-secret redaction is available to other
runtime collaborators; it does not make prompt or response contents safe for
general logging.

## Failure Boundaries

- Construction fails for missing asset registries, mutually exclusive profile
  sources, invalid asset registration, or a non-positive scheduler limit.
- Preparation failures, unavailable explicit profiles, provider failures, and
  context cancellation propagate to the calling stage with context.
- Malformed or schema-invalid provider output is classified separately as
  invalid structured output so the module or pipeline can apply its own retry
  and rejection policy.
- Domain semantic checks, evidence decisions, and deterministic normalization
  run outside the provider adapter.

## Focused Verification

Read the LLM adapter, scheduler, asset registry, schema loader, and redaction
tests when changing this boundary. Prompt changes also require the owning
module’s asset tests, and retry or debug changes require focused pipeline or
CLI coverage. The focused runtime and D&D checks are:

~~~sh
go test ./internal/framework/llm/... ./internal/modules/dnd/...
~~~