Files
notarius/docs/internal/llm.md

11 KiB

Native LLM Integration Internals

Model-executing code uses native PromptKit types directly. This guide explains Notarius's integration points, not a private completion service or a copied library manual. Architecture defines the boundary; Configuration owns resources/settings; CLI Reference owns request controls; Operations owns state and handling.

Construction And Ownership

The CLI loader hands the original completed ConfigKit application result to promptkit.NewFromConfig. One invocation shares the resulting engine across initial producers, corrections, normalization, and LLM validators. Runtime-only HTTP/client settings and an injected native promptkit.LLMClient remain construction options, not application YAML. There is no provider-neutral completion conversion or second engine per role.

Configuration Internals owns native definition/source assembly and invocation selection reuse. PromptKit owns captured resources, request preparation, native validation/repair, provider transport, conversation state, and backend admission. See the upstream Go consumer contract for those APIs rather than reproducing their mechanics here.

Application contracts remain typed production, candidate assessment, supported fallback, and artifact encoding. Pure domain models, codecs, source resolution, and deterministic transforms do not need PromptKit merely because a surrounding operation invokes a model.

Requests And Fixed Operation Inputs

Model modules receive the shared engine during typed preparation. A module prepares transcript, references, contextual identity projections, and any normalization candidate mapping once. It constructs a promptkit.RunRequest with its selected prompt, profile/session, native artifact inputs, and variables. Response interpretation consumes *promptkit.RunResult and produces the application's typed candidate.

Spell extraction is a representative path: fixed transcript/catalog/NPC projections feed an initial request; native output is decoded and canonically mapped into its durable type. The correction builder selects a paired prompt and supplies current guidance, not another copy of original source/reference material.

Domain identity stays deterministic. Contextual selectors and request-local normalization handles resolve against fixed supplied records; models do not acquire authority over durable IDs, citation meaning, or typed artifact shape. Module Internals and D&D Internals own extension/domain recipes.

Native request overrides preserve supplied values without rewriting configured profiles. The producer helper's request composition preserves operation-owned prompt/input/capture fields and existing callbacks; validators pass native execution options explicitly on their own requests.

Native Production And Application Assessment

RunProducer connects ordinary module interpretation/correction/fallback functions to the pipeline assessor. It uses native Run, or retained prepared execution on the cache path, then checks errors and public validation status before interpretation. Failed native validation can accompany nil Go error and is not a usable semantic candidate.

Every usable initial/revised candidate receives the full applicable configured assessment chain once, including unchanged candidates and the final permitted response. Native coordination owns history and additional-turn iteration. Its initial callback uses already collected findings instead of reassessing the initial result. Module templates own correction assignments; detecting modules/ validators own actionable guidance, separately from diagnostics.

Native execution success is not domain acceptance. Stopped/exhausted outcomes are control facts. Result-less failed continuation does not recover an earlier candidate, and an undelivered response is not assessed or accepted. Caller cancellation stops without fallback. Native timeout/provider/capacity failures remain distinguishable through context state and typed causes.

Resolved producer mode chooses terminal handling. A supported permissive fallback must preserve intrinsic/source/downstream invariants without repeating ordinary assessment or claiming semantic approval. Preparation/capture/encoding failures are not reasons to extend model budgets. Detailed assessment/disposition and handoff eligibility belong in Pipeline Internals.

LLM Validators

The combat-semantics validator builds one native request with its own profile and scene-local input. It uses the same engine and run-wide native execution/session selections, then checks the native error/status before decoding.

Successful interpretation returns a domain judgment and actionable rejection guidance where appropriate. Execution or interpretation failure is unavailable assessment with retained causes, not a semantic defect or invented instructions. Cancellation delivers no judgment. Validators do not coordinate another semantic conversation or rerun themselves to extend structural repair.

A completed producer/continuation releases native admission before assessment, so a nested validator can use that same limited backend. Application workers and native admission remain independent; there is no second scheduler or queue. Capacity handling follows producer mode, not an additional application retry.

Prompt And Schema Assets

assets/notarius.yml imports ordinary embedded initial/correction/validator resources with retained filesystem contexts. The assets package remains a content-only leaf. Prompt declarations own default output format, validation, and schema selections; modules own prompt meaning and small static resource-ID dependencies, not behavioral asset registration.

Each producer pair requests a complete replacement response using its shared schema. Enduring principles live in initial system context; the follow-up supplies current findings and its domain assignment. Normalization applies each complete proposal afresh to fixed originals, not a union of prior groups.

Native preparation loads selected inline/file/root schemas. Intrinsic durable artifact schemas remain application-owned codec contracts and are not removed with duplicate model-response schema machinery. Full source/root/presence rules belong in Configuration and PromptKit formats.

Native structural repair and transient retries remain upstream-owned. Notarius does not pin omitted counts or wrap them in blanket producer/validator retries. Application semantic allowance is a separate resolved control, not another counter around native coordination.

Prepared Execution And Reuse

For eligible cacheable producers, the helper prepares the initial request once, retains its details and opaque message-content hash, and either discards the handle on a usable hit or executes that same handle with RunPrepared. Unused handles are discarded on exits. Continuation retains the initial key.

Hits decode the accepted application artifact/plan without semantic reassessment, exchange restoration, native execution, or historical usage. Normalizers cache applied artifacts, not proposals. Provider-side prompt-cache hints remain native request metadata, distinct from accepted-result identity. State layout and eligibility mechanics belong in Run State Internals, not another LLM cache abstraction. The prepared execution contract owns native handle lifetime and admission behavior.

Native Evidence And Sensitive Data

The runner supplies one invocation execution group. Actual initial, continuation, and nested validator executions enroll once; group observations preserve native identity, terminal coverage, and unknown usage without copied totals or a durable profile inventory.

Native capture composes observers/preparation callbacks rather than replacing them. Retain prepared details cheaply during callbacks, then write at normal boundaries. Cache paths reuse existing details; debug never triggers a second preparation or model call. Shared callback state is synchronized. Available result-less failure facts and unassessed undelivered responses remain distinguishable.

Run capture uses secret-aware configuration views and exact marked-secret omissions, while native snapshots preserve credential omissions. Do not heuristically scrub arbitrary source/prompt/response strings. Selected full evidence can be sensitive; private debug representations are not a consumer serialization contract. Recording failures reach application boundaries without retrying model work. Operations owns handling and State Internals owns recorder lifecycle.

Focused Verification

Use real kits with a fake native provider or local fake HTTP server, not a replacement completion interface. Relevant owners include:

These verify native status/causes, complete assessment, fixed inputs/history, override propagation, cancellation/admission, reuse, and evidence. They do not claim production model quality. Wider checks belong in Development.