diff --git a/docs/cli.md b/docs/cli.md index 652757be..ba598786 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -1,299 +1,367 @@ # CLI Reference -This is the canonical reference for the implemented Notarius command-line -interface. For the shortest successful run, see the [README](../README.md). -Configuration fields, discovery rules, and selectable module keys are defined -in [Configuration](config.md); runtime state and recovery procedures are -defined in [Operations](operations.md). +This is the canonical reference for Notarius invocation, command inputs, reports, +and outcome codes. [Configuration](config.md) owns fields, defaults, resource +selection, and module keys; [Operations](operations.md) owns runtime state; +integration contracts own domain payloads. -Configuration-consuming commands share the same selection: explicit `--config`, -then `NOTARIUS_CONFIG`, then working-directory, user, and system searches. A -selected file's loading failure is terminal. Reports use that selected source; -the [configuration reference](config.md#configuration-discovery-and-precedence) -defines the filename groups and value precedence. Use `--config` for a fixed -service or subprocess selection. +Notarius uses native CommandKit commands with ckconfig and ckprompt. The +[upstream CLI reference][native-cli] owns their general parser, report, inspection, +and smoke semantics. The examples below show Notarius-specific selections. +Use help for every installed command's current options. ## Command Summary -~~~ -notarius help -notarius --version -notarius run --input path/to/source.json [--json] [flags] -notarius config validate [--config path/to/config.yml] [--config-overlay name] [--pipeline pipeline-id] [--only lane-a,lane-b] -notarius config effective [--scope notarius|promptkit] [--config path/to/config.yml] [--config-overlay name] -notarius config explain [--scope notarius|promptkit] [--config path/to/config.yml] [--config-overlay name] -notarius config profile [--config path/to/config.yml] [--config-overlay name] -notarius pipelines list [--config path/to/config.yml] [--config-overlay name] [--json] +~~~text +notarius --help +notarius help pipeline run +notarius version +notarius completion bash +notarius pipeline list +notarius pipeline show PIPELINE_ID +notarius pipeline validate PIPELINE_ID +notarius pipeline run PIPELINE_ID --input-file PATH +notarius run PIPELINE_ID --input-file PATH +notarius config scopes +notarius config validate +notarius config show +notarius config sources +notarius config explain +notarius config diff +notarius promptkit list prompts +notarius promptkit show profile PROFILE_ID +notarius promptkit requirements show PROMPT_ID +notarius promptkit requirements validate PROMPT_ID +notarius promptkit show prepared PROMPT_ID +notarius promptkit test KIND ~~~ -Running Notarius without arguments, or with **help**, **--help**, or **-h**, -writes the command summary to standard output and exits with status 0. +Bare invocation, `help`, `--help`, and `-h` display native help. Help also works +on subcommands without their execution-only required inputs. Help, version, and +Bash/Zsh/Fish completion need no configuration, credentials, or run storage. -`notarius --version` is valid only as the sole root argument. It writes exactly -`notarius ` followed by a newline to standard output and exits with -status 0. A tagged `go install` build can report its main-module stable tag, -and controlled builds can inject a stable tag at link time through -`gitea.maximumdirect.net/eric/notarius/internal/buildinfo.Override`; an ordinary -unversioned checkout reports `development`. Invalid injected version content is -a runtime error with exit status 1, while extra `--version` arguments are a -syntax error with exit status 2. This diagnostic does not replace the -[run-result](integrations/run-result.md) or artifact contracts for downstream -compatibility decisions. +`version` reports the application identity; `version --scope commandkit` selects +the library identity. Source-tagged installation and release-linked build identity +remain available; an unversioned checkout reports `development`. Version output +uses native report formats, not a custom root flag. -## run +## Syntax And Supplied Values -~~~ -notarius run --input path/to/source.json [--json] [flags] +Put flags after the **complete command path**, optionally interspersed with its +operands. Value flags accept `--name value` or `--name=value`. Boolean flags +accept presence and explicit values such as `--debug=false`. `--` ends option +parsing; later tokens are operands. + +Singleton flags may repeat; the last successfully parsed occurrence supplies +the value. Every occurrence is parsed, so a malformed earlier value remains an +error. Repeatable flags retain occurrence order and do not split commas. +Omission differs from supplied empty, false, and zero. Semantic validity still +depends on the selected command and native request. See +[registered syntax][syntax] for the shared rules. + +## Configuration Selection + +`--config PATH` fixes the main source; explicit empty fails instead of discovering +another file. Omission uses [configuration discovery](config.md#configuration-discovery-and-precedence). +Commands that support `--overlay NAME` select the native application overlay. +Configuration and pipeline commands also expose repeated `--set POINTER=JSON`: + +~~~sh +notarius config show --config examples/dnd-minimal/config.yml \ + --set /concurrency/workers=1 --path /concurrency/workers --format json ~~~ -The **run** command executes the named pipeline for one input file. The -pipeline ID and **--input** are required. +The value is JSON, so strings require JSON quoting. Preserve shell quoting: -| Flag | Meaning | +~~~sh +notarius pipeline validate dnd-session \ + --config examples/dnd-minimal/config.yml \ + --set '/pipelines/dnd-session/mode="permissive"' \ + --set /pipelines/dnd-session/max_additional_turns=0 +~~~ + +Assignments use non-root mapping paths, not sequence indexes. To change a record +inside a steps/resource list, use an appropriate source/overlay or replace the +complete list. Duplicate or overlapping assignments fail. Version and inheritance +controls are not assignment targets. Overrides follow native loading and recorded +resolution; they are not an alternate YAML parser. PromptKit commands expose +their own native selection/request flags; use help rather than assuming every +configuration flag applies to them. + +## Pipeline Commands + +`pipeline list` lists configured IDs in sorted order without module resolution +or engine construction. `pipeline show PIPELINE_ID` resolves modules, controls, +validator chains, static prompt dependencies, steps/lanes, and reference bindings, +without expanding every LLM profile or reading input/reference content. + +`pipeline validate PIPELINE_ID` performs offline domain resolution and known +prompt/profile dependency checks using the shared load and engine when needed. +It requires no input transcript, reference file bytes, credential values, +provider call, or output/cache/debug directories. Partial profiles remain valid. +This is distinct from native configuration validation: request preparation and +dynamic selections are checked when used. + +Show and validate accept configuration selection and native report controls. +They inspect the complete configured pipeline; lane selection belongs to run: + +~~~sh +notarius pipeline list --config examples/dnd-complete/config.yml --format json +notarius pipeline show dnd-session --config examples/dnd-complete/config.yml +notarius pipeline validate dnd-session --config examples/dnd-complete/config.yml +~~~ + +### Run + +~~~text +notarius run PIPELINE_ID --input-file PATH [flags] +notarius pipeline run PIPELINE_ID --input-file PATH [flags] +~~~ + +`run` is a native alias for `pipeline run`: both resolve to the same declaration +and handler. One pipeline ID and a nonempty `--input-file` are required. + +| Run flag | Meaning | | --- | --- | -| **--config path** | Use this configuration file. When omitted, configuration discovery applies; see [Configuration](config.md). | -| **--config-overlay name** | Select a ConfigKit configuration overlay. Empty and **default** select the composed base configuration. | -| **--input path** | Source input file to process. Required. | -| **--output-dir path** | Override the configured output root for this run. | -| **--json** | Write the successful run-result receipt as JSON to standard output. | -| **--chunk_cache auto\|bypass\|refresh** | Override chunk-plan cache handling for this run. | -| **--resume** | Reuse compatible recorded checkpoints when checkpoint recording is enabled. | -| **--recompute-step step-id** | With **--resume**, recompute the selected ordered step and its dependent lanes. It cannot be combined with **--only**. | -| **--debug** | Retain a debug bundle for this run. | -| **--debug-dir path** | Override the debug-bundle root. Requires **--debug**. | -| **--only lane-a,lane-b** | Run only the selected comma-separated artifact lanes when that selection is valid for the configured pipeline. | -| **--llm-profile id** | Highest-precedence configured profile for selected LLM-backed bindings and validators; it replaces binding and [pipeline](config.md#pipelines) defaults. | -| **--session-id id** | Override the generated prompt session identifier with a non-empty value for LLM-backed module calls. | -| **--reasoning-effort value** | Replace the selected PromptKit profile's reasoning effort for every LLM-backed call in this run. The value must be non-empty and the flag may be specified only once. | -| **--clear-reasoning-effort** | Clear reasoning effort inherited from the selected PromptKit profile for every LLM-backed call in this run. | -| **--reference selector=path** | Add or replace external file reference bindings at pipeline, lane, chunk, or binding scope. Repeatable. | -| **--without-reference selector** | Remove matching configured external reference bindings. Repeatable. | +| `--config PATH`, `--overlay NAME`, `--set POINTER=JSON` | Configuration selection above. | +| `--input-file PATH` | Required source input file. | +| `--lane ID` | Repeat to select literal artifact lane IDs; commas are part of an ID, not separators. | +| `--output-dir PATH` | Nonempty invocation-relative override of the output root. | +| `--cache auto\|bypass\|refresh` | Invocation-wide selection over binding and pipeline cache controls. | +| `--reference SELECTOR=PATH` | Repeatable external reference bind. | +| `--without-reference SELECTOR` | Repeatable external reference unbind. | +| `--debug[=true\|false]` | Native debug verbosity and independent run capture selection. | +| `--debug-dir PATH` | Capture destination override; explicit empty disables capture. | +| `--llm-profile ID`, `--llm-session-id ID` | Native resource/session selections below. | +| `--llm-*` execution controls | Run-wide request overrides below. | +| `--format`, `--quiet`, `--verbose`, `--progress` | Native presentation controls below. | -**--chunk_cache** accepts only **auto**, **bypass**, or **refresh**. -**--debug-dir**, **--output-dir**, **--session-id**, and -**--reasoning-effort**, and **--recompute-step** reject explicit empty values. -**--reasoning-effort** and **--clear-reasoning-effort** are mutually exclusive. -When neither is present, reasoning effort comes from the selected PromptKit -profile. These controls apply to the shared run client, including retries and -LLM-backed validators, and do not modify configuration or profile files. -Persistent reasoning settings remain a PromptKit profile concern. -**--recompute-step** requires **--resume**; checkpoint requirements and reuse -behavior are documented in [Operations](operations.md). +A directory alone does not enable capture. A nonempty debug-directory override +can re-enable capture over an empty configured root, but still needs debug +selection. `--debug --quiet` captures while suppressing stdout; `--quiet --debug` +selects debug visibility and capture. `--debug=false` disables capture. Debug +on inspection commands changes native visibility without allocating a pipeline +bundle. [Configuration](config.md#concurrency-output-cache-and-debug) defines +directory defaults; [Operations](operations.md) owns capture lifecycle. -Every run uses one effective prompt session. Without **--session-id**, Notarius -generates a stable `notarius:v1:` identifier from the trimmed resolved input -module key and the input file's exact raw bytes. The same module and bytes -therefore produce the same identifier, regardless of pipeline, references, -profile, retries, or run settings. An explicit non-empty value replaces that -default. Session identifiers are visible to providers; they are non-secret -correlation identifiers, not credential storage. See -[Operations](operations.md#operational-limits) for privacy and workflow -guidance. - -### Reference selectors - -Use **--reference** only for reference slots declared by the selected -configured targets. Qualification narrows the scope of an override: - -| Form | Target | -| --- | --- | -| slot=path | Every selected target that declares slot. | -| chunk.slot=path | The chunker. | -| lane.slot=path | Every extractor, merger, or normalizer in lane that declares slot. | -| lane.extract.slot=path | The extractor in lane. | -| lane.merge.slot=path | The merger in lane. | -| lane.normalize.slot=path | The normalizer in lane. | - -Pipeline- and lane-scoped selectors are expected to match multiple targets and -fail if they match none. A stage-specific selector fails when its lane is not -selected or its target does not declare the slot. There is no stage-wide -`merge.slot` shorthand; name the lane when targeting a merger. - -CLI bindings override configured external paths. For overlapping CLI -selectors, a binding-specific or chunk selector overrides a lane selector, and -a lane selector overrides a pipeline selector. The last occurrence wins at -equal scope. Binding and unbinding the same concrete target at equal scope is -an error; a narrower bind or unbind may create an intentional exception to a -broader action. - -**--without-reference** uses the same selector forms without `=path` and -removes external bindings only. Neither flag replaces or removes a generated -artifact handoff; an external/generated collision is a resolution error. -Required slots are checked after all effective changes. CLI reference paths -are resolved relative to the process working directory, so subprocess and -service callers should use absolute paths. Slot names, accepted media types, -size limits, requiredness, and configured generated bindings are part of the -[configuration contract](config.md). - -For example, one shared campaign reference can reach every compatible target, -with an optional lane-specific exception: +Example using the maintained minimal configuration and input: +~~~sh +notarius run dnd-session --config examples/dnd-minimal/config.yml \ + --input-file examples/dnd-minimal/transcript.json --format json ~~~ -notarius run dnd-session \ - --input /data/transcript.json \ + +This run uses a configured provider and needs its credentials when required. +Offline validation does not. + +### Run-wide LLM Overrides + +These canonical ckprompt flags reach initial producers, semantic corrections, +and LLM validators on the shared native engine without mutating configuration: + +| Flag | Value | +| --- | --- | +| `--llm-profile` | Exact profile ID; qualified for collection resources. | +| `--llm-session-id` | Session selection. | +| `--llm-endpoint` | Native endpoint override. | +| `--llm-model` | Model override. | +| `--llm-temperature` | Numeric temperature. | +| `--llm-max-tokens` | Integer token limit. | +| `--llm-top-p` | Numeric top-p. | +| `--llm-timeout-seconds` | Integer native request timeout. | +| `--llm-service-tier` | Service tier selection. | +| `--llm-reasoning-effort` | Reasoning selection; explicit empty suppresses inherited reasoning. | +| `--llm-api-key-env` | Credential environment-variable name, not its value. | +| `--llm-transient-retries` | Integer native transient-failure retry selection; explicit zero disables these retries. | +| `--llm-extra-params` | JSON provider-parameter object, preserving false/zero/null values. | + +Omitted values remain unspecified, not the display defaults in help. +`--llm-temperature=0` and `--llm-reasoning-effort=` retain their supplied meaning. +Native target validation and provider-map rules apply; this interface does not +add a repair-model selection, structural-repair count, or generic stage retry. +Use application configuration for semantic turns and native prompt configuration +for structural contracts. + +Omitted profile selection preserves binding/pipeline/prompt precedence. +Explicit `--llm-profile=` clears the application profile selection so native +prompt default selection applies. Without an explicit session flag, Notarius +generates a stable `notarius:v1:` session from the trimmed resolved input module +key and exact raw input bytes, independent of references and pipeline/profile +settings. An explicit value replaces it; explicit empty clears that generated +selection and leaves native prompt/session behavior in control. Sessions are +provider-visible correlation identifiers, not secret storage. + +PromptKit convenience spellings are scoped to `promptkit` commands; run uses +the canonical `--llm-*` names above. [Native request validation][requests] +documents execution controls in detail. + +### Reference Selectors + +Bind declared slots using `--reference SELECTOR=PATH`: + +| Selector | Target | +| --- | --- | +| `slot` | Every selected compatible target declaring the slot. | +| `chunk.slot` | Chunker. | +| `lane.slot` | Compatible extract/merge/normalize bindings in that lane. | +| `lane.extract.slot` | That lane's extractor. | +| `lane.merge.slot` | That lane's merger. | +| `lane.normalize.slot` | That lane's normalizer. | + +Pipeline/lane selectors may match several targets and fail if they match none. +A binding-specific selector requires its selected target to declare the slot. +There is no stage-wide merger shorthand. CLI paths override configured external +paths; narrower selectors beat broader ones, with last occurrence winning at +equal scope. Binding and unbinding the same concrete target at equal scope fails; +narrower actions can make intentional exceptions to broader actions. + +`--without-reference` uses the selector without `=PATH`. It removes external +bindings only. Neither action replaces/removes generated handoffs; collisions +fail resolution and required slots must remain bound. CLI paths use the invocation +working directory. Configuration defines slot compatibility and path origins in +[References And Ordered Handoffs](config.md#references-and-ordered-handoffs). + +For a complete D&D workflow, pipeline-level references are first-class inputs: + +~~~sh +notarius run dnd-session --config examples/dnd-complete/config.yml \ + --input-file examples/dnd-complete/transcript.json \ --reference party=/data/references/party.txt \ - --reference npc-registry.party=/data/references/npc-party-context.txt + --reference glossary=/data/references/glossary.txt \ + --output-dir /data/results --format json ~~~ -### Run output +### Completion Report -Without **--json**, standard output contains the completed pipeline ID, counts -of normalized and rejected outputs, and the output directory. A debug-enabled -run also prints its debug-bundle path to standard output. A successful run with -actionable process warnings reports their group and occurrence counts to -standard error. When the selected output module publishes `warnings.json`, the -summary also reports that durable file's path. Advisory and observation findings -do not produce a warning line. The published JSON bundle -is defined by the [JSON output contract](integrations/json-output.md). +Successful runs publish domain files and finalize requested capture before +reporting one small value: -With **--json**, successful standard output is exactly one -`notarius.run-result.v2` JSON document followed by a newline, with no -human-oriented status or debug-path line. Its fields and compatibility policy -are defined by the [run-result contract](integrations/run-result.md). A caller -must check for exit status 0 before decoding this output; a failed write can -leave incomplete standard-output bytes that are not a result document. +| Field | Meaning | +| --- | --- | +| `pipeline` | Selected pipeline ID. | +| `artifact_count` | Published deliverable files, including reused artifacts and enabled auxiliaries; excludes intermediates/cache/debug. | +| `output_directory` | Absolute run-specific publication directory, not the configured root. | +| `debug_directory` | Optional absolute run capture directory; omitted when disabled. | +| `fallback_count` | Producer operations that emitted fallback, present only when positive. | -Example: +Text summarizes pipeline, count, and path, appending capture location when +available. Fallback success is explicitly qualified, even when warnings are +hidden. Affected downstream artifacts, attempts, and validators are not extra +fallbacks. JSON/YAML serialize the same value; report format does not change +domain artifact encoding. There is no persisted receipt, execution ledger, +universal envelope, or automatic duplicate report file. -~~~ -OPENROUTER_API_KEY=your-api-key \ - go run ./cmd/notarius run dnd-session \ - --config examples/dnd-minimal.config.yml \ - --input examples/seriatim-minimal-transcript.json +Consumers check status, optionally decode the report, and read configured +lane/auxiliary paths. A failed stdout write may leave partial bytes. A failure +after publication can leave valid artifacts; the failure explanation identifies +available paths. [JSON output](integrations/json-output.md) owns logical artifact +paths and payload contracts, not this report. + +## Configuration Commands + +The native scopes are `app` (default) and `promptkit`. Both inspect the same +application-owned load; the latter focuses native PromptKit settings. Use +`config scopes` for supported operations. No inspection command implicitly runs +a pipeline or provider. + +~~~sh +notarius config validate --config examples/dnd-minimal/config.yml +notarius config show --scope app --config examples/dnd-minimal/config.yml +notarius config sources --config examples/dnd-complete/config.yml +notarius config explain --config examples/dnd-minimal/config.yml \ + --path /concurrency/workers --format json +notarius config show --scope promptkit --config examples/dnd-minimal/config.yml +notarius config diff --config examples/dnd-minimal/config.yml \ + --against-config examples/dnd-complete/config.yml --check ~~~ -## config validate +Show supports `--path`; explain supports `--list`, `--path`, and native history/ +filter options. Focused PromptKit paths retain the `/promptkit` prefix, for +example `/promptkit/collections/notarius/profiles`. Diff accepts separate +right-side configuration, overlay, and assignment inputs; differences with +`--check` yield status 3. These commands +report native values and secret-aware source evidence, not copied Notarius +inspection objects. Consult [native configuration commands][configuration] +for complete report and comparison semantics. Use `pipeline validate`, not +configuration validation, for production module/reference/resource preflight. -~~~ -notarius config validate [--config path/to/config.yml] [--config-overlay name] [--pipeline pipeline-id] [--only lane-a,lane-b] +## PromptKit Commands + +The installed native suite includes: + +- listings of prompts, profiles, backends, namespaces, and collections; +- inspection of a named prompt, profile, or backend; +- requirements discovery, concrete request validation, and prepared preview; +- explicit smoke tests: `reachability`, `authentication`, `text_inference`, + `structured_output`, or `all`. + +~~~sh +notarius promptkit list prompts --config examples/dnd-minimal/config.yml \ + --namespace notarius +notarius promptkit show profile notarius/dnd-extraction \ + --config examples/dnd-minimal/config.yml --format json +notarius promptkit requirements show notarius/dnd.spells \ + --config examples/dnd-minimal/config.yml ~~~ -This command loads and validates the selected composed configuration. With **--pipeline**, it also -resolves that pipeline against the production module catalog. **--only** selects -lanes during that resolution and requires **--pipeline**. +Resource/requirements inspection needs no transcript, credential values, or +provider call. Profile reports describe the stored profile before backend +completion and request overrides; partial profiles are valid. Request validation +and `show prepared` actually prepare native requests and may read supplied +resources or require credentials according to the target, but do not call a +model. Prepared reports can contain rendered application data. -Success is written to standard output as either config "" is valid or -config "" is valid for pipeline "". +Use `--llm-var KEY=VALUE`, `--llm-input-text NAME=TEXT`, or +`--llm-input-file NAME=PATH` for concrete prompt inputs. These are not the +pipeline's source input. Namespace flags choose namespaces for short IDs; +already-qualified IDs remain unchanged. Native convenience aliases are installed +only on their applicable PromptKit commands; use help to see them. -Examples: - -~~~ -go run ./cmd/notarius config validate \ - --config examples/dnd-minimal.config.yml \ - --pipeline dnd-session - -OPENROUTER_API_KEY=validation-placeholder \ - go run ./cmd/notarius config validate \ - --config examples/dnd-complete.config.yml \ - --pipeline dnd-session -~~~ - -The placeholder in the second command is sufficient only for offline -validation; it cannot run a provider-backed pipeline. - -## config effective - -~~~ -notarius config effective [--config path/to/config.yml] [--config-overlay name] -~~~ - -This command writes the selected composed configuration as deterministic, -redacted YAML. By default, it reports the Notarius configuration. Set -**--scope promptkit** to report PromptKit's captured effective configuration; -**--scope notarius** is equivalent to the default. Only the PromptKit scope -constructs the PromptKit inspection engine, using the selected configuration -and registered application assets. Engine construction validates and captures -the definitions. Standard output contains only the selected report: the -command does not validate a pipeline, prepare modules, materialize references, -resolve credentials, or contact a provider. PromptKit output can include -ordinary inline prompt and provider-parameter data; credential-shaped provider -parameters are redacted. - -## config explain - -~~~ -notarius config explain [--config path/to/config.yml] [--config-overlay name] -~~~ - -This command writes a redacted, human-readable explanation for one canonical -JSON Pointer. Supply `""` explicitly to explain the root. By default, it -explains Notarius configuration; **--scope promptkit** selects PromptKit -configuration history, while **--scope notarius** is equivalent to the default. -A malformed pointer is a syntax error; a well-formed unknown path is a command -failure. The explanation can describe values displaced by source overlays or -imports. Standard output contains only the selected explanation. PromptKit -scope constructs and validates its captured definitions, and its report can -include ordinary inline prompt and provider-parameter data. - -## config profile - -~~~ -notarius config profile [--config path/to/config.yml] [--config-overlay name] -~~~ - -This command inspects one exact, nonblank PromptKit profile ID and writes one -JSON object followed by a newline: - -The following illustrates the JSON shape; field values vary with the selected -profile and its configured backend. - -~~~json -{"profile_id":"example","api_key_required":true,"effective_model_params":{"backend_id":"openrouter","endpoint":"https://provider.example/v1","model":"example-model","temperature":0,"max_tokens":0,"top_p":0,"timeout_seconds":600,"service_tier":"","reasoning_effort":"","api_key_env":"PROVIDER_API_KEY","extra_params":null}} -~~~ - -The object reports the resolved target, including backend-completed endpoint -and credential-name settings. An empty **backend_id** identifies an -endpoint-only profile. Credential names and **api_key_required** are configured -metadata; the command does not look up credential values or report whether an -environment variable is set. Credential-shaped keys in **extra_params** are -redacted. - -This is the named profile's configured target, not a prediction of provider -defaults or the profile selected by an individual request. A request may omit -its profile and let its prompt select a default, and a run may supply a profile -override; this command applies neither behavior. Zero-valued fields show the -effective value but do not distinguish omission from an explicit zero. Use -[effective configuration](#config-effective) and -[configuration explanation](#config-explain) to inspect source presence and -history. Profile/configuration failures use exit status 1; malformed command -syntax, a blank ID, or an incorrect argument count use exit status 2. No -transcript or model call is involved. - -## pipelines list - -~~~ -notarius pipelines list [--config path/to/config.yml] [--config-overlay name] [--json] -~~~ - -This command lists configured pipeline IDs in sorted order. By default, it -writes one ID per line to standard output. **--json** writes an object shaped as -{"pipelines":[...]} instead. - -~~~ -go run ./cmd/notarius pipelines list \ - --config examples/dnd-minimal.config.yml -~~~ +Smoke tests are explicit and may contact providers or perform inference. +Read support, applicability, and observation results; status zero alone does +not mean every observation passed. Neither debug nor inspection silently +launches a smoke test. Full native mechanics are documented under +[inspection][inspection], [preview][preview], [requirements][requirements], +[resource listings][catalogs], and [smoke tests][smoke]. ## Output Streams And Exit Statuses -Successful commands write their primary result to standard output. Warnings and -errors are written to standard error. +Results go to stdout; failures, native diagnostics, and progress go to stderr. +`--format text|json|yaml` selects report presentation. Default run presentation +uses its human template; other structured native reports ordinarily use YAML. +`--format` does not reformat help, completion scripts, diagnostics, or domain +payloads. -Configuration load errors include ConfigKit's field diagnostics and source -coordinates, followed by a source-ID legend identifying the acquired files or -source labels. This includes imported files and the selected configuration overlay. +Only one visibility mode is active; the last enabled `--quiet`, `--verbose`, +or `--debug` wins. Ordinary mode hides optional warning/info/debug messages; +verbose shows warning/info; debug adds debug and default progress. Explicit +`--progress=false` disables progress, and quiet suppresses it. Quiet suppresses +stdout but retains failure explanations; formatting errors can still fail a +quiet command. Run capture selection remains independent of that final mode. -For **run --json**, actionable process warnings remain on standard error and standard output is a -machine-readable success result only. Syntax and runtime diagnostics remain on -standard error. Parse the result only after the process exits with status 0. +`--format json --verbose` keeps successful stdout report-only JSON while useful +diagnostics go to stderr. No application-wide structured error envelope is +introduced; failed stdout is not a success document. | Status | Meaning | | --- | --- | -| 0 | The command completed successfully, including root help. | -| 1 | Command syntax was valid but configuration loading or validation, pipeline resolution or execution, provider use, output, or requested debug handling failed. | -| 2 | The command or flag syntax was invalid, including unknown commands, missing required arguments, invalid flag values, or invalid flag combinations. | +| `0` | Command completed, including supported permissive fallback with report qualification. | +| `1` | Operational failure, cancellation, publication/capture/presentation failure, or an unclassified/mixed failure. | +| `2` | Invalid invocation or identifiable invalid configuration. | +| `3` | Native configuration differences with diff check mode. | -The root help spellings are the supported help path. Invoking **--help** on -**run**, a **config** subcommand, or **pipelines list** is handled by the flag -parser as a usage error: it writes an error to standard error and exits with -status 2. +Native causes and classification remain intact. Not every resource/preflight +failure is identifiable invalid configuration. Rendering or diagnostic delivery +failure can change an otherwise successful result to failure; callers check the +actual process status. See [native outcomes][outcomes] for library conventions. + +[native-cli]: https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.7.0/docs/cli.md +[syntax]: https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.7.0/docs/cli.md#registered-command-syntax +[configuration]: https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.7.0/docs/cli.md#configuration-scopes +[requests]: https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.7.0/docs/cli.md#concrete-request-validation +[inspection]: https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.7.0/docs/cli.md#promptkit-definition-inspection +[preview]: https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.7.0/docs/cli.md#promptkit-prepared-request-preview +[requirements]: https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.7.0/docs/cli.md#promptkit-requirements +[catalogs]: https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.7.0/docs/cli.md#promptkit-catalogs +[smoke]: https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.7.0/docs/cli.md#promptkit-smoke-testing +[outcomes]: https://forge.maximumdirect.net/base-layer/commandkit/src/tag/v0.7.0/docs/cli.md#outcome-status diff --git a/docs/config.md b/docs/config.md index 337524c9..5e189946 100644 --- a/docs/config.md +++ b/docs/config.md @@ -1,634 +1,371 @@ # Configuration -This is the canonical reference for Notarius configuration. Configuration files -are YAML; the main file must declare version 5. They select pipelines and their modules; -the [CLI reference](cli.md) owns invocation syntax, and -[Operations](operations.md) owns run-state procedures. +This is the canonical reference for Notarius configuration fields, defaults, +source selection, and selectable modules. The sole application version is +`6` (also accepted as `"6"`). [CLI Reference](cli.md) owns invocation syntax; +[Operations](operations.md) owns physical state and recovery; integration +contracts own artifact payloads. ## Configuration Discovery And Precedence -Commands that load configuration choose a file in this order: +Configuration-consuming commands select one main file: -1. a non-empty **--config** CLI value; -2. a non-empty **NOTARIUS_CONFIG** environment value; -3. `notarius.yml`, then `notarius.yaml` in the working directory; -4. the user configuration directory; then -5. the Unix system configuration directories. +1. Explicit `--config`, which must not be empty. +2. Nonblank `NOTARIUS_CONFIG`. +3. `notarius.yml`, then `notarius.yaml`, in the working directory. +4. Go's user configuration directory: first its `notarius/` subdirectory, + trying `notarius.yml`, `notarius.yaml`, `config.yml`, `config.yaml`, + `main.yml`, `main.yaml`; then the two `notarius.*` names directly beneath it. +5. `/usr/local/etc/notarius/`, then `/etc/notarius/`, with those six names; + finally `/etc/notarius.yml` and `/etc/notarius.yaml`. -The user search checks `/notarius/` for `notarius.yml`, -`notarius.yaml`, `config.yml`, `config.yaml`, `main.yml`, and `main.yaml`, in -that order, then `/notarius.yml` and `notarius.yaml`. The system -search uses the same groups under `/usr/local/etc/notarius/` and `/etc/notarius/`, -then `/etc/notarius.yml` and `/etc/notarius.yaml`. Generic filenames in the -working directory are not candidates. Home-directory search is not enabled. +Generic working-directory filenames and home-directory searches are not enabled. +Missing discovery candidates advance the search; a selected missing, unreadable, +or invalid explicit source fails without fallback. Exhausted discovery fails. +Discovery selects a main file rather than layering every candidate. +Use explicit selection for services and subprocesses. -Discovery selects one main file; it does not layer all candidates. A missing -candidate advances the search, but an explicitly selected missing file, access -error, or invalid selected file fails without fallback. An exhausted search -fails. Use an explicit `--config` for services and subprocesses that need a -fixed selection independent of their working directory. +One ConfigKit result contains the application and native PromptKit settings. +Its source layers, from lower to higher priority, are: -For configuration values, precedence is: +1. The bundled OpenRouter resource underlay. +2. Embedded Notarius prompts/profiles and their filesystem-backed schema root. +3. Optional operator companions beside the **actual selected main file**: + `prompts.yml` and recursively selected `*.yml`/`*.yaml` files in `prompts/`. + These form one strict peer group: filenames do not confer priority and + conflicting contributions fail. +4. The selected application main source and its native import graph. +5. The selected ConfigKit overlay. -1. built-in defaults; -2. composed YAML sources, including imports; -3. the selected ConfigKit overlay and inheritance; -4. typed decoding and explicitly supplied configuration-valued run flags; and -5. Notarius resolution and validation. +Companions are wrapperless native PromptKit settings mounted at `/promptkit`. +They are optional only when absent; malformed or unreadable selected content +fails. The loader does not search the invocation directory for substitute +companions. A companion's imports and content-file paths retain its native +source context. Main imports use their declared destinations; main-file native +settings belong beneath `promptkit`. -`--config-overlay` selects a ConfigKit overlay from `overlays/.yml` -beneath the selected configuration root. Empty and `default` select no overlay. -`profiles/.yml` is not a fallback location, and `--config-profile` is no -longer accepted. Configuration overlays are distinct from PromptKit LLM profiles selected by -`llm_profile` or `--llm-profile`. The retired operational environment variables -have no effect; `NOTARIUS_CONFIG` remains a source-discovery input. +Source assembly, inheritance, typed defaults/decoding, explicit configuration +overrides, application normalization, and validation use ConfigKit's native +processing order. Overrides cannot repair earlier parse, composition, version, +inheritance, or decoding errors. The same captured result serves inspection and +engine construction, not a separate profile loader. + +The overlay selector is a name: `development` selects +`overlays/development.yml` under the selected main source's configuration root. +Empty and exact lowercase `default` select the base. A missing selected overlay +fails; there is no overlay stack or extension fallback. Application overlays are +distinct from PromptKit profiles. See [ConfigKit's selected-overlay contract][overlays]. + +Only `NOTARIUS_CONFIG` participates in application environment selection. +Credential environment names are native PromptKit settings, not field-level +application environment overlays. ## Maintained Examples -- [Minimal D&D configuration](../examples/dnd-minimal.config.yml) is a - single-lane Seriatim-to-spell pipeline. -- [Complete D&D configuration](../examples/dnd-complete.config.yml) uses - ordered steps, all implemented D&D lanes, generated references, state - settings, bounded LLM concurrency, and the maintained - [LLM profile](../examples/llm-profiles/dnd-extraction.yml) and imports its - pipeline from [a fragment](../examples/pipelines/dnd-session.yml), which imports - its [first step](../examples/pipelines/dnd-session.describe-session.yml). +- [Minimal](../examples/dnd-minimal/config.yml): embedded resources only, Seriatim + input, generic chunking, and one spell lane. +- [Complete](../examples/dnd-complete/config.yml): ordered steps and all D&D + families, generated references, independent state roots, workers, cache/mode/ + turn controls, and selected domain auxiliaries. Its + [companion](../examples/dnd-complete/prompts.yml) demonstrates native profile + and backend inheritance without replacing native queue defaults. +- The complete [pipeline fragment](../examples/dnd-complete/pipelines/dnd-session.yml) + and [first-step fragment](../examples/dnd-complete/pipelines/dnd-session.describe-session.yml) + retain declaration-relative imports and references. Its development pipeline + selects a derived profile and zero semantic correction turns. The + [development overlay](../examples/dnd-complete/overlays/development.yml) + demonstrates serial workers and cache bypass. -The complete fragment also defines `dnd-session-development`, which inherits -the complete pipeline and profile while setting -`structured_output_repair_attempts: 0`. Select it with: - -```sh -notarius run dnd-session-development \ - --config /path/to/dnd-complete.config.yml \ - --input /path/to/transcript.json -``` - -See the [CLI reference](cli.md#run) for the full invocation contract. This -variant disables PromptKit structural repairs without changing configured producer -retry budgets. When using another profile, operators can instead select -`dnd-session` and set its `llm_profile` to that profile. - -Use these complete files as starting points rather than combining the -illustrative fragments in this reference. +The examples use separate directories so complete-example companions do not +alter minimal-example selection. Use these full files as starting points; +the fragments below illustrate individual settings. ## File Shape And Defaults -Unknown fields, duplicate mapping keys, empty identifiers, and identifiers that -become duplicates after trimming whitespace are rejected. Every top-level field -other than **version** is optional. +Application fields are strict. Unknown application fields, duplicate mapping +keys, blank identifiers, and identifiers duplicated after trimming are rejected. +PromptKit-owned resource records use the [native format][formats], including its +extension-field and strict nested-object rules; arbitrary extension fields are +not automatically secret. -| Field | Type | Default | Rules | -| --- | --- | --- | --- | -| **version** | integer or quoted string | none | Required; must be 5. Maintained files use `version: 5`. | -| **promptkit** | object | none | Profile source and optional local-backend configuration. | -| **pipelines** | map | empty | Maps pipeline IDs to pipeline definitions. | -| **concurrency** | object | see below | Global LLM and extraction limits. | -| **output** | object | see below | Published output settings. | -| **cache** | object | see below | Chunk-plan and checkpoint settings. | -| **debug** | object | see below | Debug-bundle root only; it does not enable capture. | +| Root field | Type | Default / meaning | +| --- | --- | --- | +| `version` | integer or quoted string | Required; `6` only, without translation of older versions. | +| `promptkit` | native settings | Embedded and bundled resources, overlaid by selected sources. | +| `pipelines` | map | Empty; maps IDs to pipeline definitions. | +| `concurrency.workers` | positive integer | `64`; one shared ready-job pool. | +| `output.directory` | nonempty path | `./notarius-output`. | +| `cache_directory` | path | Omitted or empty selects `os.UserCacheDir()/notarius`. | +| `debug.directory` | path | Omitted selects `/tmp/notarius-debug`; explicit empty disables capture. | -Built-in defaults are: +Nonempty relative application destinations and reference paths resolve from the +file contributing the surviving declaration, including imported, overlaid, or +inherited values. A child replacement keeps the child's origin. Built-in relative +destinations use the selected main file's directory; invocation path overrides +use the working directory. Absolute paths remain absolute. -| Field | Default | -| --- | --- | -| **concurrency.total_llm** | 16 | -| **concurrency.stage_workers.extract** | Effective **total_llm** | -| **output.directory** | **./notarius-output** | -| **cache.chunk_plans.mode** | **auto** | -| **cache.chunk_plans.directory** | Empty, selecting the per-user chunk-plan root | -| **cache.checkpoints.enabled** | false | -| **cache.checkpoints.directory** | Empty, selecting the per-user checkpoint root | -| **debug.directory** | **./notarius-debug** | +An empty output directory is invalid. Empty cache selection deliberately chooses +the per-user default; on Linux this normally follows `XDG_CACHE_HOME` or +`~/.cache`. Null is not a universal substitute for omission or empty: effective +types and application validation still apply. Directory selection does not +configure filesystem access policy or enable debug. See +[Operations](operations.md) for lifecycle, permissions, and provisioning. -An empty cache directory in YAML deliberately selects the corresponding -per-user root. An explicit empty output or debug directory is invalid. -Selecting a chunk-plan or checkpoint directory does not configure filesystem -access policy. Cache permissions come from deployment provisioning, process -umask, setgid inheritance, and default ACLs; see [Operations](operations.md#cache-provisioning). +## Imports And Sibling Inheritance -Every nonempty relative configured destination or reference path resolves from -the canonical directory of the file that supplies its surviving declaration. -This includes imported fragments, selected configuration overlays, and values -copied through inheritance: the copied value keeps its original declaring -file. A child-supplied replacement resolves from the child's declaring file. -Built-in relative destinations use the selected main configuration file's -directory; explicit `--output-dir` and `--debug-dir` values use the invocation -working directory. Absolute values remain unchanged. - -Use source-root `config` controls for imports and collection declarations. For -example, an essential fragment should explicitly be required: +Root `config` contains native controls such as imports and sequence declarations. +For example: ~~~yaml config: imports: - - path: pipelines/session.yml + - path: pipelines/dnd-session.yml required: true ~~~ -Imports are optional by default. ConfigKit supports paths, lists, directories, -globs, exclusions, recursive selection, destinations, and sequence-item placement; -its native input-root rules apply. Imported fragments need not declare a version. -The complete example places a mapping fragment at a step-list destination. Nested -application data named `config` is ordinary data, not a control block. +Imports are optional by default. Paths, directory selection, destinations, +sequence-item placement, source containment, and dependency-ready assembly order +follow [ConfigKit's import contract][imports]. Imported fragments need not repeat +the application version. A nested application field named `config` is ordinary +data, not another source-control block. -When migrating application files, rename genuine root `configkit` control -blocks to `config` in the main file and fragments. This is not a compatibility -alias: `configkit` is ordinary data under the new reader. PromptKit profile -documents still use their separate older engine and retain `configkit` controls; -see [PromptKit Profiles](#promptkit-profiles). +Mapping inheritance names a sibling, such as `extends: production` inside +`pipelines`. Identified-list inheritance uses an identity mapping such as +`extends: {id: parent}`. Parents must survive in the same sibling collection; +these are not paths or cross-namespace references. -## Sibling Inheritance +Child mappings recursively overlay parents. Scalars and null replace inherited +values; an empty mapping does not clear existing fields. Lists follow their +native registered/declared composition policy. Pipeline steps are identified by +`id`: native upsert replaces a matching **whole record** in place and appends +new IDs. It does not patch fields of a matching step. Unidentified lists replace +as whole values across layers. Explicit replace with `[]` clears a list; empty +append/upsert does not. Validator lists are not automatically identified. -ConfigKit supports native `extends` composition in mappings and identified -sequence records. In a mapping, `extends: production` selects the named immediate -sibling in that same mapping. In an identified sequence, the reference is an -identity mapping: for steps identified by `id`, `extends: {id: parent}` selects -a record in the same final sequence. Neither form is a path or cross-scope -lookup. See the pinned [native inheritance examples](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/config.md#sibling-inheritance). -Imports and the selected overlay are composed before -inheritance and typed decoding. Explicit CLI assignments follow decoding and -inheritance, before Notarius resolution and validation; they cannot repair an -earlier structural or decoding error. Every definition, including a parent and definitions not -selected for a run, remains subject to ordinary validation. +An identified record's `extends` can retain a distinct sibling's fields before +applying child changes. All surviving records remain real validated records, +not template-only definitions. Every final pipeline step executes and lane IDs +must be unique across the pipeline; inheriting a step's lanes does not remove +them. Pipeline-level inheritance with step upsert is suitable for full variants. -Child mappings recursively overlay parent mappings. Scalars and null replace -inherited values. Unidentified sequences replace as whole values; pipeline -steps are identified by `id` and use native upsert by default: +Omitted fields inherit; explicit zero, false, and empty collections retain their +presence. Null can clear an inherited value but remains subject to that field's +type and fallback semantics. Omitting `extends` retains a lower relationship; +`extends: null` cancels that relationship rather than independently deleting +other composed fields. `extends` is a reserved native configuration key, not +ordinary module-option data. See [native inheritance][inheritance] and +[sequence composition][sequences] for full rules. -~~~yaml -pipelines: - production: - input: {module: seriatim} - llm_profile: dnd-extraction - # Other complete pipeline settings and ordered steps... - development: - extends: production - structured_output_repair_attempts: 0 -~~~ - -Pipeline names are siblings within `pipelines`; nested mappings such as named -artifact lanes also have their own sibling namespace. Distinct imported step IDs -combine in ConfigKit's dependency-ready assembly order, not necessarily import -declaration or acquisition order; duplicate peer IDs conflict. Ready contributions -are selected by their stable source keys. See the pinned -[import assembly contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/config.md#explicit-imports) -for full ordering semantics. In the complete example, the imported describe-session -step precedes the importing body's extract-events step because an imported -dependency is assembled before its importer. Across an overlay or -inheritance, upsert replaces a matching step as a complete record in place and -appends new IDs. It does not merge individual fields of a matching step. An -identified record can itself use native `extends`. Authors may select `replace` -or `append` with a `config.sequences` declaration at the relevant destination; -an explicit replacement with `[]` clears the list, while empty append or upsert -does not. Notarius then validates the resulting pipeline, including its nonempty -step requirement. Validator chains are not automatically identified. Authors -may declare identities for other lists when needed. CLI list assignment replaces -the complete decoded list. - -Record inheritance does not create template-only steps: every record in the -final step list executes and must have artifact lanes. A child that inherits -its parent's named lanes also conflicts with the requirement that lane IDs be -unique across steps; adding differently named lanes does not remove inherited -ones. Use pipeline-level inheritance with step upsert for pipeline variants. -Native identified-record inheritance is also available in compatible collections, -such as an author-identified validator list, subject to ordinary domain validation. - -For example, an overlay can opt into replacing a pipeline's steps: - -~~~yaml -config: - sequences: - - destination: [pipelines, production, steps] - mode: replace -pipelines: - production: - steps: - - id: extract - artifacts: {spells: {extract: {module: dnd/spells}}} -~~~ - -Inherited values count as explicit input for Notarius's existing -presence-sensitive rules: zero, false, and empty-list overrides stay explicit -instead of restoring a default. Null also replaces the inherited value, but -each field's ordinary type and validation rules still apply; null is not -universally valid. Omitting `extends` preserves a relationship supplied by a -lower-priority source, while `extends: null` cancels that relationship. It -does not independently clear other inherited fields. - -`extends` is now a reserved key in every input mapping, including dynamic -module options and named maps. Existing ordinary data or entry names using that -key must be renamed along with any consumer that reads them. Configuration -overlays, pipeline inheritance, and PromptKit's native profile `extends` -inheritance are separate mechanisms. +Explicit configuration assignments use native mapping paths. They cannot walk +sequence indexes, assign the version, or introduce inheritance controls. Mapping +values overlay; list values replace the whole decoded list. Inheritance/defaulting +does not run again afterward. [CLI Reference](cli.md#configuration-selection) +owns assignment syntax. ## PromptKit Profiles -The optional **promptkit** object selects one source of profile definitions and -may register one conventional local OpenAI-compatible backend: +Native `promptkit` settings contain identified `backends`, `profiles`, and +`prompts` lists, plus a `collections` mapping. Notarius embeds its resources +in `collections.notarius`. Runtime IDs are qualified, for example +`notarius/dnd-extraction` and `notarius/dnd.spells`; the bundled root backend +is `openrouter`. Resource IDs are literal, not resource-version paths. +A collection's local record ID omits its namespace; references to resources in +that collection use the qualified runtime ID. + +A wrapperless companion can add a profile variant: ~~~yaml -version: 5 - -promptkit: - profile_dir: ./profiles - # profile_file: ./profiles.yml - transient_retries: 2 - local_backend: - endpoint: http://localhost:8000/v1 - concurrency_limit: 2 +collections: + notarius: + profiles: + - id: campaign + extends: {id: dnd-extraction} + model: example-model ~~~ -| Field | Type | Rules | -| --- | --- | --- | -| **profile_dir** | string | Non-empty directory containing profile files. | -| **profile_file** | string | Non-empty profile file. | -| **transient_retries** | integer | Non-negative additional built-in HTTP attempts for eligible transient provider failures; defaults to 2. Set 0 to disable these retries. | -| **local_backend** | object | Optional registration for the conventional PromptKit backend ID **local**. | -| **local_backend.endpoint** | string | Required when **local_backend** is present; absolute HTTP or HTTPS URL with a host. | -| **local_backend.concurrency_limit** | integer | Optional non-negative limit; defaults to 0. | +The pipeline can then select `llm_profile: notarius/campaign`. In the main +application file, place that same mapping beneath `promptkit`. The parent +reference is local to the same profile list; profile/backend references otherwise +name their intended exact ID. The [complete companion](../examples/dnd-complete/prompts.yml) +shows a runnable inherited backend/profile combination. -Set at most one of **profile_dir** and **profile_file**. Every nonempty relative -configured filesystem path is resolved from the canonical directory of the -file that supplies it. The complete example's -`./llm-profiles/dnd-extraction.yml` value therefore resolves beside that example. -Use absolute paths for services and containers when their deployment layout is -not configuration-relative. +Later same-ID resources replace complete lower records, not selected fields. +Use a distinct inherited sibling to retain a parent's fields; replacing a record +with only one changed field does not implicitly keep the old definition. +Native `extends`, explicit null, list policy, and source contexts apply to +resources just as to other configuration. See [resource inheritance][resource-inheritance]. -An operator source is optional. PromptKit's maintained catalog, Notarius's -registered prompt and fallback assets, and the selected operator source are -assembled in that order; a configured `local_backend` contributes its backend -last. Later source groups recursively overlay earlier groups. Within one group, -inputs compose strictly; directory filenames do not provide priority. For a -requested profile ID, an operator definition can partially override the -embedded fallback while retaining unspecified fields. The maintained -[`dnd-extraction` LLM profile](../examples/llm-profiles/dnd-extraction.yml) -is a secret-free deployment artifact. It also demonstrates a derived -development profile that is not selected by the normal pipeline. -[Operations](operations.md#promptkit-profile-deployment) owns the deployment -workflow and credential-handling guidance. +The embedded D&D profile selects `openrouter`, model `openai/gpt-5.6-luna`, +a `240` second timeout, and `flex` service tier, leaving reasoning unset. +Initial/correction pairs share their bundled default profile. Explicit application +profile selections and run overrides reach producers, validators, and correction +requests; validators can otherwise select their own profile. -When **local_backend** is present, its endpoint is trimmed and must use HTTP or -HTTPS case-insensitively, be absolute, and have a non-empty host. URL paths are -allowed. User information, queries, and fragments are rejected. A zero -**concurrency_limit** leaves the local backend unrestricted inside PromptKit; -a positive value limits simultaneous local generations. The application-wide -**concurrency.total_llm** limit still applies in both cases. Neither local -backend field has an environment override. Omitting **local_backend** registers -nothing and preserves existing built-in and endpoint-only profile behavior. +A native backend supplies endpoint, credential-environment name, optional +capacity/queue controls, provider parameters, and explicit smoke metadata. +Profiles may supply backend or endpoint, model, sampling/token controls, timeout, +service tier, reasoning, credential policy, and provider parameters. Partial +profiles are valid; preparation must complete the actual target. Resource +inspection is not a backend-completed or run-overridden target prediction. -A PromptKit file uses keyed mappings for profiles, backends, and prompts. A -profile selects a backend by its exact ID: +Notarius injects **no backend concurrency or queue defaults**. Native admission +is shared by backend ID within the engine, independently of workers; distinct +IDs do not automatically share capacity even if their endpoints match. An +endpoint-only profile has no backend identity. Configure native backend limits +when desired; the [upstream backend contract][backends] owns defaults and queue +semantics. -~~~yaml -profiles: - local-summary: - backend: local - model: example-model -~~~ +Profile omission, explicit values, and null differ. For example, native null +`endpoint` or `api_key_env` permits backend fallback, while an explicitly empty +string clears those sources; zero sampling values remain intentional. Blank/null +reasoning suppresses the outbound control. Changing backend/model does not +automatically clear unrelated explicit inherited settings. Use the +[native defaults table][profile-defaults], not inferred old-wrapper behavior. -Keep credentials out of the local-backend object. A PromptKit profile may name -its credential environment variable through `api_key_env`; set that variable -only in the run environment. `api_key_required` is supported in profile files; -inspection reports PromptKit's effective metadata but does not check whether -the named environment variable is set. PromptKit owns the -[pinned v0.12.0 format](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.12.0/docs/formats.md), -including native `extends` inheritance and field-presence rules. Notarius -selects source groups and passes definitions through PromptKit. -The [PromptKit upstream boundary](integrations/pkg-promptkit.md) identifies the -supported package API, and [Operations](operations.md#operational-limits) -describes the effective concurrency layers. +Only credential environment **names** belong in files, never values. Native +secret-aware inspection preserves marked secret omissions; arbitrary prompt, +source, or extension data is not promised safe to disclose. -### Native PromptKit Sources And Migration +### Prompt Output Contracts And Schemas -The configured `profile_file` or `profile_dir` is a PromptKit configuration -source. It can contain profiles and backends, as well as prompt overlays. -PromptKit's catalog is followed by registered application prompt/fallback -assets and the selected operator source; Notarius's optional programmatic -`local` backend is applied last. Composition, recursion, exact-key inheritance, -and final-value validation are owned by PromptKit and ConfigKit. Explicit -`null`, zero, false, blank, and omission may have different meanings; an empty -map does not clear an inherited map. +Prompt declarations select response format, validation mode, and schema. Bundled +initial, correction, and validator prompts omit structural-repair counts: +Notarius inherits the upstream default, not an independent application-pinned +count. Native prompt customization can explicitly change that budget. Transient +provider retries are also native, distinct from application semantic turns. +See [native output contracts][output-contract]. -When migrating external files from PromptKit before v0.12, place backend and -profile entries beneath their `backends` and `profiles` mappings, and place -prompt versions beneath `prompts..v1`. Remove embedded prompt -`id`/`version` fields. Replace `base_profile` with native `extends`. There is -no legacy-format parser or alias layer. Keep Notarius's `version: 5` and -`promptkit.profile_file`/`profile_dir` settings unchanged. +Schemas may be inline, `schema_file` relative to its contributing source, or +`schema_path` within an explicit applicable `schema_root`. Exactly one schema +selection is used for JSON Schema mode. Null clears an alternative; switching +forms can require clearing the inherited selection. Native preparation reads +and compiles schemas; configuration/preflight does not read them. -Imports use native ConfigKit declarations relative to the file that declares -them. A file-root layout keeps imported fragments outside a recursively -selected profile directory, avoiding selection of the same definition both by -directory discovery and by import: - -~~~yaml -# notarius.yml -version: 5 -promptkit: - profile_file: ./promptkit/entry.yml -~~~ - -~~~yaml -# promptkit/entry.yml -configkit: - imports: - - path: fragments/profiles.yml -profiles: - dnd-extraction-development: - extends: dnd-extraction - temperature: 0.2 -~~~ - -PromptKit recursively discovers YAML files in a selected directory, and its -inputs compose strictly; there is no filename-priority merger. Imports retain -their declaring source roots, so relative message/content files resolve from -the file that declares them. Use a separate file root for related imports -when directory selection would otherwise load those files a second time. - -The same unified document can declare a custom backend or select a catalog -backend without redeclaring it: - -~~~yaml -backends: - private-compatible: - endpoint: https://llm.example.invalid/v1 - api_key_env: PRIVATE_LLM_API_KEY -profiles: - private-summary: - backend: private-compatible - model: summary-model - catalog-summary: - backend: openrouter - model: openai/gpt-5.6-luna -~~~ - -To override the embedded D&D fallback for a configured local backend, supply -only the changed fields. Explicitly clear inherited settings that do not apply -to the local target: - -~~~yaml -profiles: - dnd-extraction: - backend: local - model: local-model - service_tier: null -~~~ - -The backend can be declared as `backends.local` in the operator source or -registered through the optional `promptkit.local_backend` convenience object. -The [source precedence](#native-promptkit-sources-and-migration) applies when -both contribute. Prompt overlays use the same recursive source composition; for -example, this changes one output setting while retaining the other embedded -definition fields: - -~~~yaml -prompts: - dnd.spells: - v1: - output: - repair_attempts: 0 -~~~ - -Sequences such as prompt messages are replaced as a whole when overridden. -Engine construction validates final effective prompt definitions and captures -message files and static templates. Request preparation loads response schemas -and resolves request variables and input artifacts. Profile preflight needs no -transcript, credential value, or provider call, but can read prompt-content files. - -`notarius config validate --pipeline ` resolves the selected pipeline and -inspects every explicit effective profile without contacting a provider or -requiring credential values. It rejects absent, malformed, or incompatible -profiles before a run prepares modules. Credential availability is checked only -when a generation is prepared. - -## Version 5 Migration - -Version 3 and version 4 files are rejected and are not rewritten at load time. -Use `version: 5`, rename the former `scriptorium` section to `promptkit`, and -convert every module binding to an object containing `module`. Convert every -external reference to `{path: ...}` and move any lane-level references to the -corresponding extraction binding. Put durable operational values in YAML or -use the documented explicit run flags; the former field-level environment -variables are not read. +Embedded Notarius prompts use their collection's filesystem-rooted schemas. +A nonnull collection root takes precedence over the root `promptkit.schema_root`; +absent/null collection roots fall back to that application root. A nonnull blank +root is invalid. Replacing `collections.notarius.schema_root` selects the new +context for its remaining path-based schemas. A missing file does not fall back +to embedded or working-directory roots. Other collections are unaffected. +Schema resources must be self-contained; native schema rules are in +[PromptKit's schema reference][schemas]. ## Concurrency, Output, Cache, And Debug -~~~yaml -concurrency: - total_llm: 2 - stage_workers: - extract: 2 -output: - directory: ./notarius-output -cache: - chunk_plans: - mode: auto - directory: ./notarius-cache/chunk-plans - checkpoints: - enabled: true - directory: ./notarius-cache/checkpoints -debug: - directory: ./notarius-debug -~~~ +`concurrency.workers` bounds active ready extraction and lane-completion jobs +together, including their validation/correction. It must be positive, has no +application maximum, and is independent of CPU count and native backend limits. +Explicit `1` gives serial jobs. Waiting for prerequisites occupies no worker; +nested validators do not request another worker. It is not a cross-backend +provider-call ceiling. -**concurrency.total_llm** must be greater than zero. The only supported -**concurrency.stage_workers** key is **extract**; its value must be from 1 -through **total_llm**. When omitted, it is recalculated from the effective -**total_llm** after imports, overlay selection, and explicit ConfigKit -overrides. +`output.directory`, `cache_directory`, and `debug.directory` select independent +roots. `/var/cache/notarius` is the convention for an explicitly provisioned +system-wide cache, not the per-user default. No directory setting creates a +permission policy or retention service. -**cache.chunk_plans.mode** accepts **auto**, **bypass**, or **refresh**. -**cache.checkpoints.enabled** is a boolean. The CLI can override the output -directory and chunk-plan mode for one run; see [CLI reference](cli.md#run). -Cache directory fields select only locations; Notarius has no cache permission -field or operational environment override. +A configured debug destination alone does not enable capture. Explicit empty +disables it; a nonempty invocation override can re-enable it. Native debug +selection and presentation modes are independent. See [CLI Reference](cli.md#run) +for invocation controls and [Operations](operations.md) for state handling. ## Pipelines -Each **pipelines** entry has a unique, non-empty ID and the following shape: +Each nonblank `pipelines` key defines a configured pipeline. -~~~yaml -pipelines: - dnd-session: - llm_profile: dnd-extraction - input: - module: seriatim - chunk: - module: generic - output: - module: json - artifacts: - spells: - extract: - module: dnd/spells - merge: - module: appendorder - normalize: - module: dnd/spells -~~~ - -| Field | Type | Default | Rules | -| --- | --- | --- | --- | -| **llm_profile** | string | none | Optional non-empty default PromptKit profile ID for selected LLM-backed bindings and validators. An explicitly present blank value is invalid. | -| **structured_output_repair_attempts** | integer | prompt-owned (1 in maintained production prompts) | Optional structural-repair limit from 0 through 3 for selected LLM-backed bindings and validators. Omission leaves the prompt's declared policy in control; explicit 0 disables structural repair at that scope. | -| **validation_policy** | object | see below | Optional terminal policy defaults for producer validation. Its fields inherit independently into chunk, extract, merge, and normalize bindings. | -| **input** | module binding | none | Required. | -| **chunk** | module binding | **generic** | Optional. | -| **output** | module binding | **json** | Optional. | -| **artifacts** | map | none | Compact single-step lane map. | -| **steps** | list | none | Ordered lane definitions. Mutually exclusive with **artifacts**. | -| **references** | map | none | External reference defaults for eligible targets. | - -Use either **artifacts** or **steps**. The compact **artifacts** form is an -implicit single step. An explicit **steps** list must be non-empty; every step -needs a unique non-empty **id**, an **artifacts** map, and may have -**references**. A lane ID must not appear more than once in a pipeline, -including across explicit steps. - -For each selected LLM-backed binding or validator, profile selection occurs -after module, validator, and `--only` lane selection. It uses the -run-level **--llm-profile** value first, then the binding's **llm_profile**, -then the pipeline's **llm_profile**, and finally the PromptKit default. -Deterministic bindings do not receive these defaults or run overrides. - -Structural output repair is resolved after module, validator, and `--only` lane -selection. An object's **structured_output_repair_attempts** value takes -precedence over the pipeline value; otherwise, an LLM-backed binding or -validator inherits the pipeline value. If both are omitted, PromptKit uses the -prompt's declared repair policy. The value must be an integer from 0 through 3; -explicit `null` and non-integer values are invalid. An explicit value on a -deterministic binding or validator is invalid, while a pipeline value simply -does not apply to deterministic selections. - -`validation_policy` controls terminal disposition for one complete producer -attempt and validator chain. It may appear on a pipeline or a **chunk**, -**extract**, **merge**, or **normalize** module binding; input, output, and -validator bindings reject it. Every field is optional and resolves in binding, -pipeline, then application-default order: - -| Field | Values | Default | +| Field | Type | Default / rules | | --- | --- | --- | -| **producer_structural_failure** | **fail_run**, **reject_output** | **fail_run** | -| **semantic_rejection** | **fail_run**, **reject_output** | **fail_run** | -| **validator_failure** | **warn_continue**, **fail_run** | **warn_continue** | +| `llm_profile` | string | Optional qualified profile selection; explicit blank/null is invalid. | +| `mode` | `strict\|permissive` | Producer terminal mode; see below. | +| `max_additional_turns` | nonnegative integer | Semantic correction allowance excluding the initial response. | +| `cache` | `auto\|bypass\|refresh` | Eligible model-result reuse policy. | +| `input` | binding | Required. | +| `chunk` | binding | Defaults to `generic`. | +| `output` | binding | Defaults to `json`. | +| `artifacts` | lane map | Compact single-step form, mutually exclusive with `steps`. | +| `steps` | ordered list | Nonempty when selected; each record has a unique nonblank `id`, `artifacts`, and optional `references`. | +| `references` | map | External reference defaults for compatible targets. | -The policy object and its fields must be non-null, and unknown fields are -rejected. A deterministic producer may not explicitly set -**producer_structural_failure** on its binding, although a pipeline-level -default remains valid for pipelines that include LLM-backed producers. +A lane ID is unique across all steps. Each lane requires `extract`, defaults +`merge` to `appendorder`, and defaults `normalize` to `noop`. References and +validator overrides belong on eligible bindings, not lane-wide fields. +Ordered steps finish before their generated artifacts feed later steps. -After the producer binding's retry budget is exhausted, an invalid structured -response uses **producer_structural_failure**. One or more semantic validator -rejections use **semantic_rejection**; rejection takes precedence over an -exhausted validator failure or skip. With no rejection, an exhausted validator -failure or skip uses **validator_failure**. `reject_output` records the -terminal rejection without advancing that candidate. `warn_continue` is valid -only for validator execution failure: it advances a structurally valid, -otherwise unrejected result with incomplete-validation provenance and without -making it reusable checkpoint state. +For each selected LLM role, profile precedence is explicit invocation override, +binding `llm_profile`, pipeline `llm_profile`, then native prompt default. +Deterministic roles do not receive LLM defaults/overrides and reject explicit +profile assignments. A supplied empty invocation profile clears the application +selection and lets the prompt default apply; configured blank profiles are invalid. -A lane has these fields: +### Producer Controls -| Field | Type | Default | Rules | -| --- | --- | --- | --- | -| **extract** | module binding | none | Required. | -| **merge** | module binding | **appendorder** | Optional. | -| **normalize** | module binding | **noop** | Optional. | -| **references** | map | none | Not supported at lane scope; place references on the eligible binding. | -| **validators** | list | none | Non-empty lane-level lists are rejected. Set validator overrides on a binding instead. | +Chunk, extract, merge, and normalize bindings resolve controls before execution: -Lane-level **references** are not supported. Put a lane extraction reference -under **extract.references**. +| Control | Precedence, highest first | Default | +| --- | --- | --- | +| `mode` | Binding, pipeline, application | `strict` | +| `max_additional_turns` | Binding, pipeline, application | `2` | +| `cache` | Explicit invocation cache, binding, pipeline, application | `auto` | + +Omission inherits. Zero turns disables semantic correction; a positive allowance +is not a requirement to use every turn. The complete applicable validator chain +assesses each usable initial/revised candidate, including the final permitted +response. Native coordination owns conversation/iteration; the allowance does +not count transport retries or structural repair. Deterministic work does not +acquire conversations or generic retries. + +Strict requires an accepted result. Permissive explicitly permits only the +module's supported fallback, with intrinsic shape/source/downstream checks, +without claiming ordinary semantic approval or rerunning the normal chain. +A whole-source D&D scene plan, empty extraction collection where valid, or safe +final normalization groups/merged input are supported directions. Scene +descriptions cannot use an invalid empty collection or fabricated description. +Unsupported fallback construction fails; cancellation stops without fallback. + +Eligible cacheable work uses `auto` to read/store accepted results, `bypass` +for neither, and `refresh` to skip reads and replace only after new acceptance. +Failures preserve earlier acceptance. Fallback and actually affected dependent +work skip both lookup and storage; unrelated predecessors remain reusable. +Settings/policy changes may still reuse old accepted content; select bypass/ +refresh when deliberately applying changed policy. Best-effort cache I/O does +not fail otherwise successful work. [Operations](operations.md) owns recovery +and physical storage, not a checkpoint/resume contract. ## Module Bindings And Validators -Every module and validator binding uses an object, even when it has no fields -beyond its module key: +Every module binding is an object containing `module`, even without options. -~~~yaml -input: - module: seriatim -~~~ +| Binding field | Meaning | +| --- | --- | +| `module` | Registered compatible key; required. | +| `llm_profile` | Optional nonblank qualified profile for an LLM-backed role. | +| `mode`, `max_additional_turns`, `cache` | Resolved producer controls above. | +| `options` | Selected module's strict option object. | +| `references` | Declared slots on chunk/extract/merge/normalize bindings. | +| `validators` | Ordered replacement chain on those producer bindings. | -Use an object for fields: +Omitted validators use the production default. `validators: []` replaces it +with an empty chain; a nonempty list replaces it in listed order. Null is not an +empty list. Validator bindings accept only `module`, optional LLM-backed +`llm_profile`, and `options`; they have no separate producer mode, semantic-turn +allowance, references, or nested validators. Each validator executes once per +candidate assessment. Intrinsic typed codec/source checks remain even with an +empty configured chain. -~~~yaml -extract: - module: dnd/spells - llm_profile: dnd-extraction - retries: 2 - references: - spell_catalog: - path: ./dnd-spell-catalog.json -~~~ +The `json` output options are: -| Binding field | Type | Default | Rules | -| --- | --- | --- | --- | -| **module** | string | none | Required for an object binding. Must be a registered compatible key. | -| **llm_profile** | string | none | Optional non-empty PromptKit profile ID for an LLM-backed binding. It overrides the pipeline default unless the run supplies **--llm-profile**. | -| **structured_output_repair_attempts** | integer | pipeline or prompt-owned (1 in maintained production prompts) | Optional structural-repair limit from 0 through 3 for an LLM-backed binding. It overrides the pipeline value; explicit 0 disables structural repair. | -| **validation_policy** | object | pipeline or application defaults | Optional field-by-field terminal-policy override for a chunk, extract, merge, or normalize binding. | -| **retries** | integer | 0 | Non-negative additional complete producer attempts for chunk, extract, merge, and normalize bindings. This single budget covers operational errors, invalid structured output, module-requested normalization retry, and semantic correction. | -| **options** | object | none | Must satisfy the selected module. | -| **references** | map | none | Valid only on chunk, extract, merge, and normalize bindings. | -| **validators** | list | production chain | Valid only on chunk, extract, merge, and normalize bindings. | +| Option | Default / rules | +| --- | --- | +| `include_chunk_map` | Boolean, default false; publish when enabled and available. | +| `evidence_context` | Omitted disables publication; otherwise a strict object. | +| `evidence_context.enabled` | Required boolean; false permits no other evidence fields. | +| `evidence_context.lanes` | When enabled, required nonempty list of unique trimmed configured lane IDs. | +| `evidence_context.window_units` | When enabled, nonnegative integer, default `3`; zero keeps only directly cited units. | -Omitting **validators** uses the registered chain. **validators: []** selects -an empty chain; a non-empty list replaces the chain in the listed order. -Validator bindings accept only **module**, **llm_profile**, -**structured_output_repair_attempts**, **retries**, and **options**. Their -**retries** value is a non-negative additional validator-execution budget and -is valid only when the selected validator is LLM-backed. A validator retry -rechecks the same immutable candidate; it never regenerates the producer. -They reject -**validation_policy**, **references**, and nested **validators**. Deterministic -validators reject explicit **llm_profile** and -**structured_output_repair_attempts**. -Deterministic module bindings also reject those explicit fields. - -An LLM-backed chunk, extract, merge, or normalize producer with both a -non-empty validator chain and positive **retries** must declare the supported -single-response correction capability. Preparation rejects a configuration -that could require semantic correction from a producer that cannot provide an -exact prior response. A deterministic producer, or an LLM attempt that did -not make a model call, cannot consume a semantic retry after rejection. - -The **json** output module accepts optional **include_chunk_map** and -**evidence_context** settings: - -~~~yaml -output: - module: json - options: - include_chunk_map: true - evidence_context: - enabled: true - window_units: 3 - lanes: - - npc-registry - - spells -~~~ - -**include_chunk_map** is a boolean and defaults to false. It adds the accepted -chunk map when one exists; its wire format is defined in the -[chunk-map contract](integrations/chunk-map.md). - -Omitting **evidence_context** disables evidence publication. When present, it -is an object with these strict fields: - -| Field | Type | Rules | -| --- | --- | --- | -| **enabled** | boolean | Required. `false` permits no other evidence fields. | -| **lanes** | array of strings | Required and non-empty when enabled. Each value is trimmed and must be unique; every value must name a configured pipeline lane. | -| **window_units** | non-negative integer | Optional when enabled; defaults to 3. Zero retains only directly cited units. | - -Unknown outer or nested option fields are rejected, as are incompatible YAML -types. The allowlist remains valid when a run uses lane filtering: a configured -lane that is not active for that invocation simply contributes no evidence. -Evidence publication is opt-in because it can persist source text and metadata. -When enabled, it publishes the selected source-unit excerpt defined by the -[Published Evidence Context contract](integrations/evidence-context.md). +Filtered-out configured lanes contribute no evidence. Unknown or incompatible +options fail. Evidence is opt-in because it can persist source text and metadata. +Logical formats belong to [chunk-map](integrations/chunk-map.md) and +[evidence-context](integrations/evidence-context.md) contracts. ## References And Ordered Handoffs @@ -662,9 +399,9 @@ steps: An artifact selector contains only **step** and **lane**. The producer must be an earlier step and the selected artifact must be compatible with the consumer -slot. A generated binding supplies one accepted normalized artifact; it does -not name a file. A configured generated dependency remains required even when -that consumer slot is otherwise optional. +slot. A generated binding supplies one normalized artifact in memory, including a +supported qualified fallback; it does not name a file. A configured generated +dependency remains required even when that consumer slot is otherwise optional. Pipeline references are configuration defaults. A matching step-local or binding-local external path overrides a pipeline default. CLI reference @@ -719,7 +456,7 @@ normalized combat-turn module may use optional **npc_registry**. Location occurr require **location_registry** for extraction and normalization. Item occurrences require **item_registry** for extraction and normalization. Enemy-event extraction requires all four of its JSON artifact slots; its normalizer requires **npc_registry**. -The [complete example](../examples/dnd-complete.config.yml) shows the ordered +The [complete example](../examples/dnd-complete/config.yml) shows the ordered generated bindings. ## Production Module Keys @@ -770,12 +507,12 @@ Available validator keys are: | Location registry | **extract/dnd/location-registry/shape**, **extract/dnd/location-registry/source_refs**, **extract/dnd/location-registry/source_relatedness**, **normalize/dnd/location-registry/identity** | | Location occurrences | **extract/dnd/location-occurrences/shape**, **extract/dnd/location-occurrences/registry**, **extract/dnd/location-occurrences/source_refs**, **extract/dnd/location-occurrences/source_relatedness**, **normalize/dnd/location-occurrences/invariants** | -`extract/dnd/scene-descriptions/combat_semantics` is not in a production default chain. To opt in, replace the scene extractor validator chain with the current ordered chain plus the semantic validator last, and set a positive producer retry budget if a rejection should request a corrected scene: +`extract/dnd/scene-descriptions/combat_semantics` is not in a production default chain. To opt in, replace the scene extractor validator chain with the current ordered chain plus the semantic validator last, and retain a positive semantic-turn allowance if a rejection should request a corrected scene: ~~~yaml extract: module: dnd/scene-descriptions - retries: 1 + max_additional_turns: 1 validators: - module: generic/valid_json - module: extract/dnd/scene-descriptions/shape @@ -783,14 +520,13 @@ extract: - module: generic/valid_json_schema - module: extract/dnd/scene-descriptions/source_relatedness - module: extract/dnd/scene-descriptions/combat_semantics - retries: 1 ~~~ -An override replaces, rather than extends, the default chain. See [Module Bindings And Validators](#module-bindings-and-validators) for binding, profile, repair, retry, and failure-policy rules. +An override replaces, rather than extends, the default chain. See [Module Bindings And Validators](#module-bindings-and-validators) for binding and profile rules, and [producer controls](#producer-controls) for semantic correction and terminal mode. -The validator retry shown above permits one additional execution against the -same scene candidate when the LLM-backed validator itself fails; it is separate -from both the extractor's producer retry and PromptKit structural repair. +The validator runs once per candidate assessment. Native transient retries and +structural repair remain internal to that execution; there is no separate +validator semantic-turn allowance. When no override is configured, production D&D bindings use the following ordered chains. Each row lists extract then normalize; spell chains are the @@ -813,27 +549,29 @@ Chains are only registered for the D&D extract and normalize modules shown above; select an explicit override when a different compatible chain is required. -## Validation +## Validation And Inspection -Validate a file and one pipeline before running it: +Configuration validation checks the selected native application document. +Pipeline validation additionally resolves production modules/options, validator +targets, references, generated handoffs, and statically known prompt/profile +dependencies. It needs no transcript, reference bytes, credential values, +provider calls, or state directories. Partial profiles pass; request-time data +and dynamic selections remain runtime checks. [CLI Reference](cli.md#pipeline-commands) +owns commands for both boundaries. -~~~sh -go run ./cmd/notarius config validate \ - --config examples/dnd-minimal.config.yml \ - --pipeline dnd-session -~~~ +Native whole-app and focused PromptKit inspection use the same selected result +and secret-aware provenance. Profile inspection describes stored fields, not a +prediction of a completed request; prepared preview is the native request boundary. +See [configuration commands](cli.md#configuration-commands) and +[PromptKit commands](cli.md#promptkit-commands). -Configuration validation rejects invalid YAML, unsupported fields, invalid -defaults or explicit CLI overrides, incompatible module keys, unknown options, -invalid reference bindings, missing required reference slots, invalid validator -overrides, and incompatible generated artifact handoffs. Use -[pipelines list](cli.md#pipelines-list) to inspect configured IDs. - -## Inspection - -Use [config effective](cli.md#config-effective) to write the selected composed -configuration as deterministic, redacted YAML. Use -[config explain](cli.md#config-explain) with a canonical JSON Pointer to inspect -the effective or historical provenance for one path, including a value displaced -by a selected overlay. Neither command resolves a pipeline, prepares modules, -materializes references, or requires provider credentials. +[formats]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/formats.md +[backends]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/formats.md#backend-definitions +[profile-defaults]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/formats.md#defaults-and-overrides +[resource-inheritance]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/formats.md#resource-inheritance +[output-contract]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/formats.md#output-contract +[schemas]: https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.24.0/docs/formats.md#schemas +[imports]: https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.4.0/docs/config.md#explicit-imports +[overlays]: https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.4.0/docs/config.md#selected-overlays +[inheritance]: https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.4.0/docs/config.md#sibling-inheritance +[sequences]: https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.4.0/docs/config.md#strict-document-composition diff --git a/docs/roadmap/implementation.md b/docs/roadmap/implementation.md index d76412b2..ac5ed7b4 100644 --- a/docs/roadmap/implementation.md +++ b/docs/roadmap/implementation.md @@ -1894,6 +1894,28 @@ implemented behavior. Check ADR status links and accepted-body preservation; ## Stage 39: Rewrite Canonical Configuration And CLI References +**Status:** Complete. Rewrote the canonical configuration and CLI references for +version 6, shared native source selection/companions, whole-record resource +replacement and inheritance, null/presence and schema-root semantics, qualified +profiles, workers, semantic turns, strict/permissive modes, and unified reuse +controls. Documented native pipeline/standard commands, supplied/repeated values, +references, complete run-wide execution overrides, presentation/streams, the small +completion report, and outcome classifications. Removed retired syntax and old +example paths rather than preserving compatibility examples. Native default +repair/retry and backend admission remain upstream-owned. Retained domain module, +reference-slot, output-option, and default-validator definitions, checking all 20 +D&D extract/normalize chains against the real resolved complete pipeline. + +Compared declarations, configuration/resolver tests, actual help, maintained +examples, and the selected upstream module documentation. Twenty direct offline +help/completion/configuration/inspection/preflight/diff checks pass with the +OpenRouter credential variable unset; assignment/explicit-zero examples also +pass. All 46 local links/anchors and 20 pinned upstream destinations/anchors +resolve against repository/module sources. Whole-tree tests, vet, a temporary +executable build with `GOWORK=off`, and `git diff --check` pass. No live inference +or real credential was used. Operations, migration, integration, and internal +documentation retain their following assigned stages. + **Goal:** operators can configure and invoke only the implemented native surface. **Inspect/change:** `docs/config.md`, `docs/cli.md`, final command declarations,