Record native architecture decisions and align ownership policy
This commit is contained in:
@@ -2,30 +2,33 @@
|
||||
|
||||
**Base Policy Version:** 1.3
|
||||
|
||||
**Last Updated**: 2026-11-17
|
||||
**Last Updated**: 2026-11-22
|
||||
|
||||
**Upstream Source**: https://gitea.maximumdirect.net/eric/policy/src/commit/593c9b52273c5c879785ae9e995d21026df467b4/docs/policy/architecture.md
|
||||
|
||||
The source above identifies the adopted baseline. Local native-kit architecture
|
||||
decisions below are repository adaptations, not a new upstream policy adoption.
|
||||
|
||||
## Purpose And Ownership
|
||||
|
||||
The adopted policy defines the application's purpose and the architectural
|
||||
constraints contributors must follow. It owns system boundaries, responsibility
|
||||
and dependency rules, established invariants, and explicit non-goals.
|
||||
|
||||
Describe the concrete package inventory and implementation mechanics in internal
|
||||
documentation. Link to canonical public contracts for their exact behavior and
|
||||
to ADRs for decision rationale.
|
||||
Describe concrete package inventory and implementation mechanics in internal
|
||||
documentation. Link to canonical public contracts for exact behavior and to
|
||||
ADRs for decision rationale.
|
||||
|
||||
## Architectural Approach
|
||||
|
||||
Hexagonal architecture is strongly recommended, including encapsulating external
|
||||
integrations and APIs within internal adapters. Keep domain and application
|
||||
logic independent of external integration details, and make dependency direction
|
||||
clear in the adopted policy.
|
||||
Hexagonal architecture is strongly recommended. Keep pure domain logic independent
|
||||
of external integration details and make dependency direction clear. Native kit
|
||||
types are an intentional contract for model execution, configuration composition,
|
||||
and CLI presentation; do not hide them behind a replacement completion API.
|
||||
|
||||
Implement that separation with proportionate, idiomatic code. Choose interfaces
|
||||
and package layout to serve the application's actual boundaries. Use adapters
|
||||
for collaborators with external integration responsibilities.
|
||||
Implement separation with proportionate, idiomatic code. Choose interfaces and
|
||||
package layout to serve demonstrated boundaries. Use adapters where they serve
|
||||
an integration need, not merely to rename an adopted library's public types.
|
||||
|
||||
## Adopted Constraints And Implemented Structure
|
||||
|
||||
@@ -38,16 +41,19 @@ Constraints must reflect deliberately selected requirements and decisions.
|
||||
Do not promote incidental implementation choices into architectural invariants.
|
||||
|
||||
Notarius's fixed pipeline, artifact-family boundaries, and deployment-owned
|
||||
cache permissions are established product decisions. The linked ADRs retain
|
||||
their rationale; the [internal overview](../internal/overview.md) inventories
|
||||
their implementation.
|
||||
cache permissions remain established product decisions. Successor ADRs
|
||||
[0021](../adr/0021-compose-native-kits-and-prompt-declared-resources.md),
|
||||
[0022](../adr/0022-use-native-continuation-and-application-assessment.md),
|
||||
[0023](../adr/0023-unify-accepted-artifact-and-plan-caching.md), and
|
||||
[0024](../adr/0024-publish-artifacts-with-native-reports-and-debug-evidence.md)
|
||||
record the native cutover. Earlier accepted bodies remain historical decisions,
|
||||
with status links identifying superseded portions.
|
||||
|
||||
## System Shape
|
||||
|
||||
Notarius is a small, dependency-light Go application for extracting structured
|
||||
artifacts from source material. It is a general extraction platform whose
|
||||
source formats, extraction domains, validation policies, LLM providers, and
|
||||
output formats are isolated behind explicit boundaries.
|
||||
Notarius is a Go application for configured structured extraction workflows.
|
||||
Source formats, artifact domains, validation, and output encoding remain explicit
|
||||
boundaries.
|
||||
|
||||
The application has one fixed pipeline shape:
|
||||
|
||||
@@ -55,337 +61,308 @@ The application has one fixed pipeline shape:
|
||||
input -> chunk -> extract -> merge -> normalize -> output
|
||||
```
|
||||
|
||||
Pipelines are configured compositions of this shape. They are not arbitrary
|
||||
DAGs or a general workflow language. Every stage remains explicit; general
|
||||
chunking, merging, or normalization behavior must not be hidden inside an
|
||||
extractor.
|
||||
Pipelines are configured compositions of this shape, not arbitrary DAGs or a
|
||||
general workflow language. Every stage remains explicit; chunking, merging, or
|
||||
normalization must not be hidden inside an extractor.
|
||||
|
||||
A stage module is one configured implementation of one pipeline stage. An
|
||||
artifact family is the cohesive domain feature that owns an artifact across
|
||||
the explicit stages and supporting codecs, validators, prompts, identity
|
||||
rules, and reference projections. Artifact-family ownership does not combine
|
||||
stages or alter the fixed pipeline.
|
||||
A stage module implements one pipeline stage. An artifact family owns a cohesive
|
||||
domain feature across its stages, codecs, validators, prompts, identity rules,
|
||||
and reference projections. Family ownership does not combine stages.
|
||||
|
||||
Input and chunking are pipeline-wide. Each selected artifact lane owns its
|
||||
extract, merge, and normalize stages, and the output stage aggregates the run's
|
||||
lane outcomes.
|
||||
extract, merge, and normalize stages. Output aggregates the run's domain results.
|
||||
Ordered steps support generated references without changing the fixed shape.
|
||||
|
||||
Notarius is contract-first without being abstraction-heavy. Interfaces and
|
||||
extension points should protect demonstrated boundaries. New abstraction is not
|
||||
itself an architectural goal.
|
||||
|
||||
Implementation discipline is owned by [Agent Policy](agents.md).
|
||||
Notarius is contract-first without being abstraction-heavy. Extension points
|
||||
protect demonstrated boundaries; new abstraction is not itself an architectural
|
||||
goal. [Agent Policy](agents.md) owns implementation discipline.
|
||||
|
||||
## Layers And Dependency Direction
|
||||
|
||||
The application boundary is the composition root and may depend on concrete
|
||||
implementations. Domain-neutral model and framework layers provide reusable
|
||||
policy, contracts, and orchestration. Concrete input, pipeline, output, and
|
||||
validation extensions depend inward on those generic layers.
|
||||
The composition root may depend on concrete implementations. Domain-neutral
|
||||
models and framework layers provide reusable contracts and orchestration.
|
||||
Concrete input, pipeline, output, and validation extensions depend inward on
|
||||
those generic layers. Generic layers must not depend on production extensions;
|
||||
extensions must not compose the application or own process behavior.
|
||||
|
||||
Semantic reconciliation is one such domain-neutral framework mechanism. It
|
||||
prepares bounded source context, invokes a shared model-judgment protocol,
|
||||
validates proposals, and applies safe plans through typed policies supplied by
|
||||
the consuming artifact family. It does not own domain identity, durable IDs,
|
||||
warning semantics, or artifact construction rules.
|
||||
Semantic reconciliation is a domain-neutral mechanism for fixed candidate
|
||||
preparation, native request construction, proposal decoding, and safe typed
|
||||
application. It does not own domain identity, durable IDs, warning semantics,
|
||||
or artifact construction rules. Native execution uses the shared producer path.
|
||||
|
||||
Generic layers must not depend on production extensions. Concrete extensions
|
||||
must not compose the application or take ownership of process behavior. The
|
||||
current packages implementing these layers are inventoried in
|
||||
[Internal Overview](../internal/overview.md).
|
||||
|
||||
The root `assets` package is a content-only dependency leaf. It may expose a
|
||||
read-only embedded filesystem, but it must contain no business logic and must
|
||||
not depend on `internal` packages or PromptKit. Consumers scope that filesystem
|
||||
to the content they own; the root package is not a behavioral registry or a
|
||||
public extension contract. The rationale and compatibility consequence are
|
||||
recorded in [ADR-0011](../adr/0011-centralize-llm-assets.md).
|
||||
The root `assets` package remains a content-only dependency leaf. It may expose
|
||||
a read-only embedded filesystem but contains no business logic or dependencies
|
||||
on internal packages or PromptKit. Native source assembly retains its filesystem
|
||||
context; the package is not a behavioral registry or public extension contract.
|
||||
This retains [ADR-0011](../adr/0011-centralize-llm-assets.md)'s content boundary.
|
||||
|
||||
The following dependency boundaries are mandatory:
|
||||
|
||||
- extractors and validators do not depend on concrete input adapters;
|
||||
- provider-specific types do not cross the LLM runtime boundary;
|
||||
- external dependency types do not leak across internal package boundaries
|
||||
unless that dependency is the package's explicit contract.
|
||||
- provider-specific transport and authentication remain inside PromptKit;
|
||||
- model-executing framework code, modules, and validators use native PromptKit
|
||||
requests, results, exchanges, errors, and observations directly;
|
||||
- pure domain types, codecs, reference resolution, and deterministic helpers do
|
||||
not acquire PromptKit dependencies merely because surrounding producers use it;
|
||||
- other external dependency types cross internal boundaries only where that
|
||||
dependency is an explicit contract.
|
||||
|
||||
Shared helpers may support demonstrated common needs, but must not move
|
||||
source-format or extraction-domain knowledge into generic framework packages.
|
||||
External dependencies require a clear correctness, security, interoperability,
|
||||
or complexity benefit.
|
||||
Shared helpers support demonstrated common needs without moving source-format
|
||||
or domain knowledge into generic packages. Dependencies require a clear
|
||||
correctness, security, interoperability, or complexity benefit.
|
||||
[Internal Overview](../internal/overview.md) owns the concrete package inventory.
|
||||
|
||||
## Source And Domain Boundaries
|
||||
|
||||
Input modules translate external source formats into the generic source model.
|
||||
Format-specific schemas, fields, and validation remain with the input module
|
||||
and its integration contract.
|
||||
Input modules translate external formats into generic source documents, units,
|
||||
and references. Format-specific fields, schemas, and validation remain with the
|
||||
input module and its integration contract.
|
||||
|
||||
Framework stages operate on source documents, source units, and source
|
||||
references rather than format-specific structures. A source reference identifies
|
||||
an ordered range of generic source units. Framework code preserves those ranges
|
||||
and does not merge or rewrite them unless a stage module explicitly owns that
|
||||
behavior. Every source unit carries a validated self-reference to its containing
|
||||
document and its own unit ID.
|
||||
A source reference identifies an ordered range of generic units. Framework code
|
||||
preserves ranges unless a module explicitly owns their transformation. Every
|
||||
source unit carries a validated self-reference to its document and unit ID.
|
||||
Auxiliary references provide context or disambiguation, not source evidence;
|
||||
they must not be converted into source references.
|
||||
|
||||
Extract modules own artifact semantics, prompt use, response schemas, and
|
||||
domain interpretation. Domain-specific concepts remain in the relevant module,
|
||||
validator, shared domain helper, and artifact contract.
|
||||
Modules own domain semantics, prompt selection, interpretation, deterministic
|
||||
identity, and supported fallback construction. Physical asset centralization does
|
||||
not transfer semantic ownership. Native prompt declarations own default output
|
||||
contracts and schema selections; PromptKit prepares schemas and performs
|
||||
structural validation and repair. Modules declare small static resource
|
||||
dependencies for preflight, not manifests, schema registries, or fingerprints.
|
||||
Intrinsic durable artifact schemas remain application contracts.
|
||||
|
||||
Physical centralization of LLM-facing content does not transfer semantic
|
||||
ownership from those modules. Modules retain their manifests, response-schema
|
||||
identities, prompt ordering, and registration, while reading only their scoped
|
||||
content subtree. Generic framework code remains domain-neutral when it reads
|
||||
its own scoped generic assets from the shared content container.
|
||||
Typed artifact registrations declare one stable kind and exact Go type through
|
||||
extraction, merge, normalization, and semantic validation. Resolution requires a
|
||||
compatible codec and matching kind-specific variants. Private framework erasure
|
||||
reports incompatibility as an error rather than a panic.
|
||||
|
||||
Typed artifact registrations declare one stable artifact kind and exact Go
|
||||
type from extraction through merge, normalization, and semantic validation.
|
||||
Pipeline resolution requires a compatible codec and matching kind-specific
|
||||
variants before a typed lane can be accepted. Framework-owned erasure remains
|
||||
private and must report type incompatibility as an error rather than a panic.
|
||||
A kind may provide a typed evidence projection copying direct generic source
|
||||
references. Preparation proves its exact codec-type compatibility. Evidence is
|
||||
reconstructed from serialized normalized artifacts, including supported fallbacks,
|
||||
without generic code inspecting domain JSON or importing domain types. The output
|
||||
boundary owns publication.
|
||||
|
||||
An artifact kind may additionally provide a typed evidence projection that
|
||||
copies its direct generic source references. Preparation proves that projection
|
||||
matches the artifact codec's exact Go type before retaining it for an output
|
||||
policy. The runner reconstructs evidence only from accepted serialized
|
||||
normalized artifacts, and the output boundary owns any resulting publication.
|
||||
Generic framework code never infers evidence by inspecting domain JSON or
|
||||
depends on domain artifact types.
|
||||
|
||||
Auxiliary references provide context or disambiguation. They are not source
|
||||
evidence and must not be converted into source references.
|
||||
When a model selects an entity, modules supply contextual selections and attach
|
||||
opaque application IDs deterministically after exact resolution. Models do not
|
||||
receive or reproduce opaque application IDs. Reconciliation may expose
|
||||
request-local candidate handles, resolved before typed application and never
|
||||
used as durable identity. Retain
|
||||
[ADR-0012](../adr/0012-resolve-opaque-entity-identifiers-deterministically.md) and
|
||||
[ADR-0013](../adr/0013-use-request-local-candidate-handles-for-semantic-reconciliation.md)'s
|
||||
identity and safety decisions.
|
||||
|
||||
## Pipeline Composition And Ownership
|
||||
|
||||
Module selection is configuration- and registry-driven. The framework resolves
|
||||
named pipeline definitions, applies explicit defaults and runtime overrides,
|
||||
and verifies module availability and capabilities before execution. Structural
|
||||
pipeline choices must not be scattered through conditionals or hidden behind
|
||||
ad hoc command flags.
|
||||
Configuration and registries select modules, explicit defaults, references, and
|
||||
runtime overrides. Validate selected module and validator options before
|
||||
execution. Preparation constructs the complete implementation set; the runner
|
||||
accepts only that prepared set. Construction failures precede source parsing.
|
||||
|
||||
Resolution validates every selected module and validator option set. A separate
|
||||
preparation boundary then constructs the complete input, chunk, lane,
|
||||
validation, and output implementation set in pipeline order. The runner accepts
|
||||
only that prepared set, so construction and dependency failures occur before
|
||||
source parsing or any other module operation.
|
||||
Stage ownership remains explicit:
|
||||
|
||||
Stage ownership is explicit:
|
||||
- input translates external material into the generic source model;
|
||||
- chunk produces source-addressed plans;
|
||||
- extract produces artifacts from chunks;
|
||||
- merge combines extraction outputs;
|
||||
- normalize reconciles merged output;
|
||||
- output encodes domain results and enabled auxiliaries into logical files.
|
||||
|
||||
- input modules convert external material into the generic source model;
|
||||
- chunk modules partition source material for extraction;
|
||||
- extract modules produce domain artifacts from chunks;
|
||||
- merge modules combine accepted extraction outputs;
|
||||
- normalize modules reconcile merged output;
|
||||
- output modules encode accepted results and run outcomes into logical files.
|
||||
|
||||
Chunk modules produce source-addressed chunk plans rather than materialized
|
||||
chunks. The framework validates and materializes those plans into the generic
|
||||
chunk representation before chunk validation and lane execution. Plan reuse is
|
||||
therefore independent of the configured pipeline, module options, references,
|
||||
lanes, validators, and LLM profile: the canonical source digest selects the
|
||||
plan, while the current run still applies its configured chunk validators to
|
||||
the materialized chunks.
|
||||
The framework materializes plans into chunks before candidate assessment and
|
||||
lane execution. LLM plans use the accepted-result cache; hits rematerialize
|
||||
against the current source without new semantic assessment. Source identity alone
|
||||
does not select a canonical partition. Deterministic chunkers execute normally.
|
||||
|
||||
The framework owns orchestration, origin enrichment, aggregation, and handoff
|
||||
provenance. Modules return logical results and classified diagnostics; they do
|
||||
not own CLI reporting, physical output, cache, or
|
||||
debug roots, durable file placement, or checkpoint and debug lifecycle.
|
||||
provenance. Modules return logical results and classified diagnostics, not CLI
|
||||
reports, physical roots, durable placement, or capture lifecycle.
|
||||
|
||||
After pipeline-wide chunking, extraction uses bounded framework concurrency.
|
||||
One run-wide worker pool receives chunk-scoped lane jobs in deterministic
|
||||
chunk-first, lane-second order. A lane may begin its merge and normalize
|
||||
continuation only after all of its extract jobs are terminal; that continuation
|
||||
remains serial within the lane, while bounded continuations for different lanes
|
||||
may overlap. The framework must not create unbounded goroutines per lane or
|
||||
chunk.
|
||||
After pipeline-wide chunking, one pool bounds ready extraction and ready
|
||||
merge/normalize lane-completion jobs together. Extraction dispatch uses stable
|
||||
chunk-first, lane-second order. A continuation becomes ready after its lane's
|
||||
extraction finishes; a worker does not wait for prerequisites while holding a
|
||||
slot. Validation and correction run inside the current job without another
|
||||
worker acquisition. Ordered steps retain their barriers. Do not create unbounded
|
||||
goroutines per lane or chunk.
|
||||
|
||||
Completion timing does not choose public ordering or errors. The coordinator
|
||||
orders accepted artifacts, grouped diagnostics, rejections, checkpoint events, and
|
||||
framework errors by stable pipeline scope. Rejections do not cancel unrelated
|
||||
work. A framework error cancels derived work, prevents undispatched work from
|
||||
starting, waits for started work, and prevents output encoding.
|
||||
Completion timing does not choose public ordering or terminal errors. The
|
||||
coordinator orders artifacts, grouped diagnostics, and errors by stable scope.
|
||||
Cancellation stops dispatch, drains started work, and releases workers and native
|
||||
admission before returning.
|
||||
|
||||
Warnings are process-only signals: configuration degradation, approved fallback,
|
||||
or incomplete configured validation. Quality uncertainty and grounding findings
|
||||
are advisories; successful canonicalization and cleanup are observations.
|
||||
Modules choose that semantic classification, while the framework attaches
|
||||
origin, aggregates groups, enforces bounds, and presents final collections.
|
||||
An ordinary successful run therefore has zero warnings. See
|
||||
[ADR-0015](../adr/0015-separate-process-warnings-from-quality-diagnostics.md)
|
||||
for the decision rationale.
|
||||
Warnings are process signals: configuration degradation, qualified fallback,
|
||||
or incomplete configured work. Quality and grounding findings are advisories;
|
||||
accepted cleanup is observation. Modules own classification, while the framework
|
||||
adds origin and aggregates bounded groups. A non-degraded successful run has no
|
||||
actionable warnings. Retain
|
||||
[ADR-0015](../adr/0015-separate-process-warnings-from-quality-diagnostics.md)'s
|
||||
classification decision, not its retired durable reporting contracts.
|
||||
|
||||
## Validation
|
||||
## Validation And Disposition
|
||||
|
||||
Validation is a framework-managed boundary around outputs from chunk, extract,
|
||||
merge, and normalize stages. Validators receive immutable stage output and
|
||||
make an explicit whole-output decision: approve, reject, fail, or skip when a
|
||||
runtime prerequisite is unavailable.
|
||||
Validation is a framework-managed boundary around chunk, extract, merge, and
|
||||
normalize candidates. Validators receive immutable views and return approve,
|
||||
reject, fail, or unavailable findings. Typed validators receive domain values;
|
||||
chunk validators receive generic chunks; serialized validators receive bytes
|
||||
and schema metadata. Incompatible targets or kinds cannot substitute for one
|
||||
another.
|
||||
|
||||
Typed artifact validators receive the domain value directly. Chunk validators
|
||||
receive source-zone chunks, while serialized validators receive immutable
|
||||
representation bytes and declared schema metadata. A validator registered for
|
||||
one target or artifact kind cannot satisfy an incompatible selection.
|
||||
Assess each usable initial or revised candidate once through every applicable
|
||||
validator in configured order. Rejections do not short-circuit later validators.
|
||||
Earlier approvals and unchanged revisions do not excuse reassessment; the final
|
||||
allowed response also receives the whole chain. Cancellation or fatal framework
|
||||
failures can interrupt assessment. There is no duplicate final assessment.
|
||||
|
||||
The framework runs every applicable validator sequentially in configured order.
|
||||
It aggregates rejections, exhausted validator failures, and skips before the
|
||||
producer policy chooses a disposition. A completed rejection never advances.
|
||||
With no rejection, an exhausted validator failure may fail the run or, under
|
||||
the configured `warn_continue` policy, advance a structurally valid candidate
|
||||
with explicit incomplete-validation provenance. Validators report findings;
|
||||
they do not choose candidate disposition.
|
||||
Actionable correction guidance is distinct from reason codes, operator messages,
|
||||
and failures. The detecting validator or module owns it. The pipeline aggregates
|
||||
it in order; a native module template supplies the correction assignment. Missing
|
||||
required guidance is a contract failure, not an invitation to invent instructions.
|
||||
Unavailable assessment is not a semantic defect or ordinary acceptance.
|
||||
|
||||
A completed rejection supplies a stable reason code for internal provenance
|
||||
and bounded actionable correction guidance for the candidate producer. Reason
|
||||
codes, validator keys, and operator-facing messages remain diagnostic data;
|
||||
they are not model instructions. The framework constructs model-facing retry
|
||||
text only from the semantic guidance and fails the contract rather than
|
||||
inventing or truncating missing guidance.
|
||||
The producer's resolved mode owns terminal disposition. Strict requires
|
||||
acceptance; explicitly selected permissive mode may construct a supported
|
||||
fallback. A fallback must satisfy intrinsic shape and downstream invariants but
|
||||
does not rerun the ordinary validator chain or claim semantic approval. Unsupported
|
||||
or invalid fallback construction fails normally. Caller cancellation never
|
||||
initiates fallback. Exact mode and turn controls belong in [Configuration](../config.md).
|
||||
|
||||
An LLM-backed module may also request a feedback-aware retry when its own
|
||||
deterministic translation or typed safety policy rejects a structurally valid
|
||||
model response. It must supply model-facing guidance separately from its
|
||||
reason code, operator message, and fallback diagnostics, together with the
|
||||
exact `single_response_v1` candidate. A feedback-free module retry remains
|
||||
valid when no exact candidate exists.
|
||||
Default validator chains remain central production composition policy.
|
||||
Configuration can replace a stage-local default, including with an explicit empty
|
||||
chain. Configured order is authoritative. Intrinsic codec/source correctness
|
||||
still applies with an empty chain.
|
||||
|
||||
Default validator chains are production composition policy and are registered
|
||||
centrally by stage and module. Configuration may replace a stage-local default,
|
||||
including with an explicitly empty chain. Configured validator order is
|
||||
authoritative; the framework must not silently reorder it.
|
||||
## Native Execution Boundary
|
||||
|
||||
## LLM Boundary
|
||||
The composition root constructs one shared native PromptKit engine from the
|
||||
application's ConfigKit result. All model roles use that engine. PromptKit owns
|
||||
transport, authentication, transient retries, structural validation and repair,
|
||||
conversation history, turn iteration, and backend admission. Notarius does not
|
||||
pin omitted native budgets or add outer retries to extend them. Deterministic
|
||||
operations have no generic retry budget or synthetic model request.
|
||||
|
||||
Modules and validators use transport-neutral structured completion contracts.
|
||||
Provider request and response types, authentication, transport behavior, and
|
||||
provider error adaptation remain inside the LLM runtime.
|
||||
One small typed producer helper connects module-owned fixed inputs, native
|
||||
request builders, interpretation, and supported fallback functions with the
|
||||
pipeline assessor. Initial/correction pairs use complete replacement responses
|
||||
and the same declared schema. Enduring principles belong in initial system
|
||||
context; correction supplies current findings and a new assignment. Native
|
||||
coordination carries the growing exchange without manual response replay or
|
||||
regeneration of original references. Normalization applies each proposal afresh
|
||||
to fixed originals, without cross-turn unions.
|
||||
|
||||
The caller of the LLM owns prompt selection, prompt inputs, response schema,
|
||||
and interpretation of structured output. Provider adapters do not own source-
|
||||
or domain-specific prompt logic.
|
||||
Check native errors and validation status before interpretation, including failed
|
||||
validation with nil Go error. Native success, stopped, or exhausted status alone
|
||||
does not establish domain acceptance. Do not recover an earlier candidate after
|
||||
a result-less continuation failure or assess an undelivered response. LLM
|
||||
validators execute once per assessment, without a separate semantic correction
|
||||
loop. Calls respect cancellation and applicable native timeouts.
|
||||
|
||||
PromptKit owns bounded structural correction within one structured completion.
|
||||
Notarius owns outer stage attempts, semantic validation, and acceptance policy;
|
||||
the two budgets must remain separate.
|
||||
|
||||
An LLM-backed producer can participate in semantic correction only when it
|
||||
declares `single_response_v1` and returns the exact one response that directly
|
||||
controlled its candidate. On an actionable validator rejection or
|
||||
feedback-capable module retry, the framework rebuilds the ordinary request and
|
||||
appends only the latest defective response as an `assistant` message plus one
|
||||
aggregated `user` correction message. This is a fresh replacement request, not
|
||||
a growing conversation. The retry budgets, terminal policy, and sensitive-data
|
||||
rationale are recorded in
|
||||
[ADR-0014](../adr/0014-feedback-aware-validation-retries.md) and
|
||||
[ADR-0016](../adr/0016-feedback-aware-module-requested-retries.md).
|
||||
|
||||
When a model selects an application entity, callers must supply a contextual
|
||||
selection and deterministically attach the opaque application identity whenever
|
||||
the selection resolves exactly. Models do not receive or reproduce opaque
|
||||
application identifiers. Semantic reconciliation may instead expose
|
||||
contiguous, one-based candidate handles that exist only for one request;
|
||||
deterministic code resolves them before typed application, and they never
|
||||
become durable identity. This is the approved request-local-label application
|
||||
of [ADR-0012](../adr/0012-resolve-opaque-entity-identifiers-deterministically.md)
|
||||
recorded by
|
||||
[ADR-0013](../adr/0013-use-request-local-candidate-handles-for-semantic-reconciliation.md).
|
||||
|
||||
LLM calls and other external operations accept cancellation and respect
|
||||
timeouts. Concurrency control belongs in shared runtime plumbing rather than in
|
||||
individual modules.
|
||||
|
||||
The application-wide LLM scheduler bounds actual provider calls independently
|
||||
of framework worker limits. Every LLM-backed module, retry, and validator uses
|
||||
the single injected scheduled client, including work performed by overlapping
|
||||
lanes. Provider runtime adapters may enforce a narrower backend-specific limit
|
||||
beneath this mandatory application-wide scheduler.
|
||||
Worker limits and native backend admission are independent. Profiles selecting
|
||||
one backend ID share its engine capacity; different IDs do not imply an endpoint
|
||||
limit. Notarius injects no backend concurrency/queue defaults and has no
|
||||
cross-backend scheduler, second admission queue, or capacity retry loop. Completed
|
||||
turns release admission before assessment so nested validators can use the same
|
||||
limited backend. Native failures retain their causes and follow the producer's
|
||||
mode; they do not become correction guidance.
|
||||
|
||||
## Configuration And Provenance
|
||||
|
||||
Configuration loading, precedence, defaults, redaction, and validation are
|
||||
centralized through the ConfigKit-backed boundary. `NOTARIUS_CONFIG` is limited
|
||||
to discovery; field-level environment overlays are not configuration input.
|
||||
Named pipeline definitions make structural composition explicit and discoverable.
|
||||
Explicit command overrides are permitted
|
||||
when they do not obscure the configured pipeline structure.
|
||||
ConfigKit owns loading, imports, composition, overlays, inheritance, source
|
||||
contexts, provenance, and secret-aware inspection. Notarius owns application
|
||||
meaning through supported definition rules and domain resolution, retaining
|
||||
[ADR-0020](../adr/0020-delegate-configuration-mechanics-to-configkit.md).
|
||||
`NOTARIUS_CONFIG` is discovery only; field-level environment overlays are not
|
||||
application configuration. Named pipelines keep structural composition explicit.
|
||||
|
||||
Run preparation fails before stage execution when statically discoverable
|
||||
modules, capabilities, reference bindings, or explicitly selected profiles are
|
||||
invalid or incompatible.
|
||||
One application-owned result supplies engine construction and whole-app/focused
|
||||
inspection. Preserve source contexts and captured selection evidence rather than
|
||||
reparsing settings or maintaining duplicate loaders. Request overrides retain
|
||||
native supplied-value semantics. CommandKit, ckconfig, and ckprompt own standard
|
||||
commands, parsing, presentation, and delivery outcomes. Notarius supplies domain
|
||||
commands and report content, not a universal result envelope.
|
||||
|
||||
Run manifests record enough resolved pipeline, module, source, reference, and
|
||||
LLM provenance to make a run auditable after configuration changes. Manifests
|
||||
record identities and bounded validation summaries rather than secret, raw
|
||||
model, correction, or large payload content.
|
||||
Offline preflight shares domain resolution and checks statically known module,
|
||||
reference, initial/correction/validator prompt, and applicable profile dependencies
|
||||
before execution. It requires no transcript, reference bytes, credentials,
|
||||
provider call, or run storage. Dynamic selections remain runtime checks.
|
||||
Shared loaders allocate no run storage; capture belongs to the run path.
|
||||
|
||||
## Trust Boundaries And Threat Scope
|
||||
|
||||
Where useful, state the trust boundaries and threat scenarios relevant to the
|
||||
application's adopted requirements, along with scenarios it explicitly excludes.
|
||||
For example, distinguish handling untrusted application input from defending
|
||||
against a privileged actor concurrently modifying the host filesystem.
|
||||
Source material and model responses cross application validation boundaries.
|
||||
Host access policy belongs to deployments. Hypothetical privileged actors
|
||||
concurrently modifying the filesystem do not justify new application defenses
|
||||
without an explicit decision. This does not remove adopted correctness boundaries
|
||||
or claim excluded scenarios cannot occur. Raise new threats or guarantees under
|
||||
[Agent Policy](agents.md).
|
||||
|
||||
Threat exclusions define the scope of the adopted requirements. Base defenses
|
||||
and guarantees on those requirements and raise unsettled scope for a deliberate
|
||||
decision under the [agent policy](agents.md).
|
||||
|
||||
Keep this discussion proportionate to the application. Configuration,
|
||||
operations, and internal documents retain ownership of credential supply,
|
||||
runtime handling, and implementation mechanisms under the documentation policy.
|
||||
Configuration, operations, and internal documents own credential supply,
|
||||
runtime handling, and mechanisms under [Documentation Policy](documentation.md).
|
||||
|
||||
## State, Output, And Safety
|
||||
|
||||
Source material and model responses cross application validation boundaries.
|
||||
Host access policy belongs to the deployment; hypothetical privileged actors
|
||||
concurrently changing the filesystem do not justify new application defenses
|
||||
without an explicit decision. This scope statement does not remove the adopted
|
||||
correctness boundaries below or claim excluded scenarios cannot occur. Follow
|
||||
[Agent Policy](agents.md) when a new threat or guarantee needs a decision.
|
||||
Output, cache, and debug retain independent roots and lifecycles:
|
||||
|
||||
Notarius exposes three filesystem surfaces with independent roots and
|
||||
lifecycle:
|
||||
- output is durable domain data; modules encode logical files and the CLI places them;
|
||||
- cache is reconstructible accepted-artifact and LLM-plan state;
|
||||
- debug is explicitly requested run/operation evidence, not a consumer API.
|
||||
|
||||
- output is durable user data; output modules define logical files and the CLI
|
||||
owns their placement;
|
||||
- cache is reconstructible state, with separate chunk-plan and checkpoint
|
||||
families; and
|
||||
- debug is explicitly requested inspection data, combining a redacted summary
|
||||
with a detailed trace.
|
||||
Cache identity is the opaque native initial prepared message-content hash.
|
||||
Add no configuration, schema, model, session, implementation, or dependency
|
||||
dimensions. Store only final accepted codec artifacts or materialized/assessed
|
||||
plans, not responses, exchanges, proposals, provenance envelopes, or historical
|
||||
usage. Hits perform ordinary decoding/source-usability checks without new
|
||||
semantic assessment. Policy/settings changes may still hit; explicit
|
||||
bypass/refresh controls re-execution. Deterministic work is not cached through
|
||||
manufactured requests.
|
||||
|
||||
Chunk plans are keyed only by canonical source digest. Configured checkpoint
|
||||
recording is independent of checkpoint reuse; checkpoints are loaded only for
|
||||
an invocation that explicitly requests resume. Debug is never a cache input and
|
||||
is never created without an explicit request. Pipeline modules receive
|
||||
collaborator interfaces and never physical roots.
|
||||
Fallback and affected dependent work skip lookup and storage; unaffected accepted
|
||||
predecessors remain reusable. Refresh replaces an entry only after new acceptance.
|
||||
Failure preserves an old entry. Cache I/O is best effort and cannot fail otherwise
|
||||
successful work. Legacy caches remain unread and unmodified, with no checkpoint
|
||||
resume, canonical source-only plan store, or automatic migration.
|
||||
|
||||
Deployments own cache access policy. Cache writers request conventional maximum
|
||||
creation modes and let the process umask, setgid inheritance, and default ACLs
|
||||
determine effective access. Notarius owns the separate safety boundary:
|
||||
confined paths, symlink rejection, atomic publication, validation, and
|
||||
recovery. It does not chmod, chown, chgrp, or migrate cache entries to impose
|
||||
an application policy. This rule applies to chunk-plan and checkpoint cache
|
||||
only; output and debug retain their separate contracts. [Operations](../operations.md)
|
||||
owns deployment provisioning and retention guidance.
|
||||
Deployments own cache access policy. Writers request conventional maximum modes
|
||||
and let umask, setgid inheritance, and default ACLs select effective access.
|
||||
Retain confined paths, symlink rejection, atomic publication, validation, and
|
||||
recovery without chmod/chown/chgrp convergence or cache migration.
|
||||
[ADR-0017](../adr/0017-defer-cache-access-policy-to-deployment.md)'s boundary now
|
||||
applies to the unified cache; output and debug retain separate contracts.
|
||||
|
||||
Only accepted, completely validated producer output is reusable checkpoint or
|
||||
chunk-plan state. Rejected, structurally invalid, and validation-incomplete
|
||||
results cannot become cache or checkpoint inputs, even when a
|
||||
`warn_continue` result is allowed to advance in the current run. This
|
||||
ineligibility follows derived merge and normalize results and generated
|
||||
references for the remainder of the run: current-run handoff remains allowed,
|
||||
but no dependent cache or checkpoint may be loaded or published.
|
||||
Preserve intrinsic domain payload schemas and logical paths, including supported
|
||||
fallbacks. Consumers need no index, manifest, persisted receipt, or reporting
|
||||
sidecar. Publish artifacts and finish capture before a native completion report.
|
||||
Disclose fallback use independently of diagnostic verbosity. Delivery failure may
|
||||
leave partial stdout and valid published artifacts; retain those artifacts,
|
||||
useful locations, and distinct operation/delivery causes without model retries.
|
||||
Exact report fields and outcomes belong in [CLI Reference](../cli.md).
|
||||
|
||||
Writes are atomic where practical. Paths for writes, moves, overwrites, and
|
||||
deletion must be narrow and explicit. Notarius never automatically deletes
|
||||
output or requested debug bundles; cache cleanup is explicit and recoverable.
|
||||
One native execution group records actual initial, continuation, and nested
|
||||
validator executions once. Preserve identity, terminal coverage, and
|
||||
unknown-versus-zero usage without synthetic IDs, copied totals, or historical
|
||||
execution claims on cache hits.
|
||||
|
||||
Secrets must not appear in errors, logs, output, cache, debug summaries,
|
||||
traces, manifests, documentation, examples, or redacted configuration. Raw
|
||||
assistant responses and complete correction messages are attempt-local and are
|
||||
excluded from ordinary durable records and summaries; the requested detailed
|
||||
debug trace is the sole diagnostic surface allowed to retain them. Debug
|
||||
collection is allowlisted to application-owned payloads and must not capture
|
||||
unrelated process environment values or filesystem content. Trace data may
|
||||
contain application data and therefore inherits its sensitivity; operators own
|
||||
access controls and retention. Physical layout and operation are defined in
|
||||
[Operations](../operations.md).
|
||||
Debug is never a cache input. Capture requires explicit enablement and a nonempty
|
||||
effective destination; directory configuration alone does not enable it. Requested
|
||||
capture is required work: allocate before engine construction/preflight, record
|
||||
incrementally, and finalize once before successful reporting. Capture failures
|
||||
fail the invocation while retaining available evidence and primary causes.
|
||||
Update the relative `latest` link at session start, never completion.
|
||||
|
||||
Debug reuses prepared details and native evidence without another preparation/
|
||||
model call or parallel conversation ledger. Representations are private
|
||||
diagnostics, not stable serialization contracts or exhaustive provider transcripts.
|
||||
Use ConfigKit secret-aware inspection and preserve native credential omissions.
|
||||
Do not directly serialize marked secrets, unrelated environment, or unselected
|
||||
filesystem content. Do not heuristically scan or rewrite source, prompts,
|
||||
responses, or arbitrary application text. Selected evidence may contain full
|
||||
source/reference data, responses, and exchanges; it is not a sanitized bundle.
|
||||
Operators own access and retention as for other application data.
|
||||
|
||||
Writes are atomic where practical. Write, move, overwrite, and deletion targets
|
||||
must be narrow and explicit. Never automatically delete output or requested debug
|
||||
bundles. Cache cleanup remains explicit and recoverable.
|
||||
[Operations](../operations.md) owns physical layout, provisioning,
|
||||
retention, and recovery guidance.
|
||||
|
||||
## Platform And Distribution
|
||||
|
||||
@@ -404,7 +381,9 @@ Notarius does not aim to provide:
|
||||
|
||||
- an arbitrary workflow graph or general workflow language;
|
||||
- source-format or extraction-domain behavior in generic framework packages;
|
||||
- provider-specific contracts exposed to modules;
|
||||
- provider-specific transport contracts exposed to modules;
|
||||
- structural pipeline composition through ad hoc CLI flags;
|
||||
- implicit cross-stage behavior that bypasses the fixed pipeline;
|
||||
- replacement kit APIs, duplicated schedulers, or compatibility parsers;
|
||||
- a public debug serialization format or mandatory execution ledger;
|
||||
- abstractions introduced solely for hypothetical future complexity.
|
||||
|
||||
Reference in New Issue
Block a user