Document the new configuration and processing workflow

This commit is contained in:
2026-11-10 17:52:14 +00:00
parent 926fa26a31
commit ba1d5d2367
61 changed files with 832 additions and 765 deletions

View File

@@ -1,192 +1,81 @@
# Audita Configuration
This reference describes the version-2 configuration loader in `internal/core/config`.
Use the maintained [local](../examples/minimal-config.yml) and
[service](../examples/production-config.yml) examples with their separately
selected [local profiles](../examples/profiles/local/) and
[service profiles](../examples/profiles/service/), plus the optional
[testing overlay](../examples/overlays/testing.yml).
The [ConfigKit inheritance example](../examples/configkit-inheritance.yml)
shows a local child backend reusing and overriding sibling settings.
Choose real models for your endpoint; a testing profile makes no cost or quality
guarantee. See the [migration guide](migrations/v2.md) for retired inputs.
Audita uses version 3 application configuration. See the [migration guide](migrations/v3.md)
and [examples](../examples/) for complete projects. Command syntax is owned by
[the CLI reference](cli.md).
## Loading and precedence
## Loading and composition
The application selects one main YAML file: an explicit `--config`, then
`/usr/local/etc/audita/config.yml` if present, then `/etc/audita/config.yml`
if present. Missing system candidates are skipped; an explicit missing or empty
path, inaccessible selected file, or absence of any selected file is an error.
`config validate` requires an explicit file. ConfigKit loads built-in defaults,
the selected file, an optional named overlay from `overlays/<name>.yml` beside the
main file, and only explicitly supplied setting overrides, in that order. Imports
and secret-directory bootstrap are disabled. Unknown fields and missing, null,
or non-integer `version: 2` are errors. Version-1 files require migration; no
compatibility loader or implicit environment configuration remains.
An explicit `--config` selects one main YAML file. Without it, ConfigKit searches
the working directory, user configuration directory, and Unix system directories
for `audita` configuration. If no file is found, built-in defaults apply. An
explicit invalid selection or failure of a selected file is an error; search does
not continue after a selected file fails. An omitted version uses the v3 schema;
a supplied version must be `3`.
After overlay and explicit CLI override precedence is applied, ConfigKit resolves
native sibling inheritance before typed decoding. Any input mapping may use
`extends: <sibling-name>`; the child keeps its key, inherits parent settings, and
overrides them with its own settings. Selected overlay and CLI values therefore
participate in inheritance at their normal precedence. The generic syntax and
resolution rules are defined by ConfigKit v0.7.1's
[sibling inheritance contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.7.1/docs/config.md#sibling-inheritance).
`extends` is reserved structural input in base files, imports, overlays, and
other application input mappings, including mapping values supplied by explicit
CLI setting overrides. ConfigKit consumes it before typed decoding. A previously
valid backend whose ID is literally `extends` cannot be represented under this
rule. Rename that backend to a non-conflicting ID and update every PromptKit
profile that selects it. This compatibility impact means the upgrade does not
promise that every previously valid configuration remains valid.
Inherited values count as explicit input, including zero, false, empty, and
null. Existing field rules still apply: for example, inherited backend
concurrency must be positive, while an inherited queue capacity of zero is
valid. Parent backends undergo the same decoding and application validation as
other backends, whether or not a selected profile uses them. A child
`credential: null` removes the child's inherited credential; it does not remove
the parent's credential reference, which remains subject to ordinary secret
resolution.
Validation runs after overlays, explicit overrides, and derived settings have
resolved. An override can therefore repair an invalid lower-precedence setting
when that setting can be decoded. ConfigKit retains field origins and derivations
for effective-configuration inspection. Its effective YAML redacts resolved
credentials.
Operator sources may use ConfigKit imports. The selected main file provides the
base for an optional `overlays/<name>.yml`, optional `glossary.yml`, and recursive
`glossary/` YAML files. Without a main source, automatic companion files are not
loaded from the working directory. ConfigKit composes sources, resolves native
inheritance, applies explicit overrides, records derived defaults, and validates
the result. Its [configuration reference](https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.2.0/docs/config.md)
owns native composition and provenance semantics.
### PromptKit profiles and operator prompts
`profiles_dir` selects a PromptKit configuration root, resolved against the
selected main file's directory unless it is absolute. The root may contain
keyed `profiles`, `backends`, and `prompts` definitions; it is not limited to
profile-only files. IDs come from literal YAML mapping keys, never filenames.
Multiple definitions may share a document, and a document may contribute to any
of the supported mappings. Directory discovery is recursive, but discovery
order does not establish field precedence. Contributions within each source
group must compose without conflicting values.
PromptKit loads its default catalog and Audita's embedded prompts, followed by
optional `prompts.yml` and recursive `prompts/` YAML files beside the application
main. Native imports, resource collections, inheritance, and replacement apply.
Profiles, backends, models, credentials, admission, and provider parameters belong
to PromptKit configuration; see its [format reference](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.21.0/docs/formats.md).
The engine captures its sources for an invocation. Edit files and run a new
invocation to pick up changes. Inspection does not require model credentials.
PromptKit assembles sources in this order: its selected catalogs and owned
defaults, Audita's embedded prompt/schema inputs, operator documents under
`profiles_dir`, then the application backend mapping. The application mapping
owns each configured endpoint, concurrency limit, queue capacity, and credential
environment lookup. Other supported provider fields can be contributed by the
operator documents. Imports remain disabled. The current maintained examples
select separate roots: [local](../examples/profiles/local/) and
[service](../examples/profiles/service/), so definitions for one backend do not
enter the other example's final configuration.
## Pipeline and glossary selections
Both selected roles must resolve to an existing profile, registered backend,
nonblank effective model, and nonblank endpoint. The normalized effective
endpoint must match the application backend endpoint. A selected profile cannot
introduce a competing `api_key_env` lookup; the application backend credential
supplies the runtime key when required. Other unselected profiles may be
partial, but every final backend, profile, and prompt definition is validated
when the engine is constructed. A malformed or conflicting unused definition
can therefore make the selected root invalid. A missing selected root is an
operator configuration error; an empty root may contribute no definitions.
`glossary_collections` maps names to collections with an ordered `entries` list.
Each entry has an `id`, `name`, and `category`; `description`, `aliases`, and
`plural` provide optional context. Entries are identified within their collection
by `id`. Root `glossaries` and `protected_terms` select collection names. If root
protection is omitted, it derives from the root glossary selection.
Profile IDs and parent IDs are exact, literal keys. PromptKit profiles can use
native `extends` inheritance; child fields override inherited fields, while
unmentioned controls survive changes to model or backend. For example:
`pipeline.stages` is an ordered list of `module` records; supported modules are
`glossary`, `homophones`, `spoken_word`, and `grammar`. The default sequence is
`glossary`, `homophones`, `glossary`, `spoken_word`, `grammar`. Repeated modules
run separately; `[]` runs normalization without model processing. An optional
nonempty, unique `id` identifies a record in evidence. Each record may override
`glossaries`, `protected_terms`, `confidence_threshold`, and fields under `llm`,
`validation`, and `context`. Omitted fields inherit their resolved root values.
Explicit empty lists and null list selections clear a selection.
```yaml
profiles:
"shared~base":
backend: service
model: example-model
temperature: 0.4
top_p: 0.8
"proposal fast":
extends: "shared~base"
model: faster-model
temperature: 0
```
Presence matters. Explicit `temperature: 0` is sent as zero; omission leaves it
unspecified; null suppresses an inherited temperature. Other fields have their
own native null and omission rules. Provider `extra_params` maps merge
recursively, and a null provider value is sent as JSON null. See the pinned
[PromptKit format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.14.0/docs/formats.md)
for the complete field semantics.
Operator documents can also override an embedded prompt/version, including its
messages and definition-level output settings. Shared variables supplied by the
adapter are rendered only when an effective message references them; Audita does
not append omitted shared text. Audita's per-call schemas, structural repair
budget, and downstream domain validators remain in force. The engine captures
effective definitions and message-file contents when constructed, so edits are
observed only by a newly constructed runtime.
### Migrating PromptKit profile documents
For the PromptKit v0.11 format, wrap each profile's fields under
`profiles: {<literal-id>: ...}`, remove inner `id`, and replace
`base_profile` with `extends`. Profile and prompt IDs are mapping keys, so they
need not match source filenames. Review zeros, omitted values, and nulls against
the native presence rules before deploying. Select a root whose final
definitions have valid backend references; eager validation applies even to
unselected definitions. See the complete maintained [local](../examples/minimal-config.yml)
and [service](../examples/production-config.yml) examples and the pinned
[consumer migration guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.14.0/docs/consumers/pkg-promptkit.md#migrate-from-the-previous-configuration-api).
This incompatible profile-format migration is intended for an Audita major
release; no automatic file conversion is provided.
The [CLI inspection views](cli.md#streams-and-exit-status) show selected
controls and field provenance separately from ConfigKit's redacted settings
YAML. Inspection resolves both roles without provider calls.
Each entry receives its selected context and independent protection. Glossary
correction exempts active recognized spellings from protection for that entry;
other protected spellings remain protected. A glossary correction must have at
least one selected entry at processing preflight. Empty definitions and empty
protection remain valid configuration.
## Settings
| Field | Default | Rule |
| Setting | Default | Meaning |
| --- | --- | --- |
| `profiles_dir` | `profiles` | Nonempty PromptKit document root; resolved relative to the selected main file, or absolute. |
| `backends.<id>.endpoint` | None | Required nonempty endpoint for each backend. At least one backend is required. |
| `backends.<id>.concurrency` | `1` | Positive finite capacity; shared by both roles using that backend. |
| `backends.<id>.queue_capacity` | `1024` | Nonnegative; zero disables waiting admission. |
| `backends.<id>.credential` | Unset | Optional environment-variable secret reference; omission or null means unauthenticated use. |
| `llm.proposal_profile` | None | Required nonempty profile ID. |
| `llm.validation_profile` | Final proposal profile | Omission inherits the final proposal selection; explicit empty or null fails. |
| `llm.transient_retries` | `2` | Nonnegative additional attempts per generation. |
| `llm.repair_attempts` | `2` | Additional structural repair generations, from zero through three. |
| `llm.operation_timeout` | `15m` | Positive Go duration string per logical call. |
| `pipeline.modules` | `[glossary, homophones, glossary, spoken_word, grammar]` | Ordered module keys; repeats are meaningful and an explicit empty list runs no correction modules. |
| `output.schema` | `bare-segments` | `bare-segments` or `audita-v1`. See [output formats](integrations/output-files.md). |
| `llm.proposal_profile` | unset | PromptKit profile for proposals; required for processing entries. |
| `llm.validation_profile` | effective proposal profile | Profile for model validation; null clears a selection before fallback. |
| `llm.transient_retries` | `2` | Additional transient attempts per generation. |
| `llm.repair_attempts` | `2` | Structural repairs, from zero through three. |
| `llm.operation_timeout` | `15m` | Positive duration per logical operation. |
| `validation.max_prompt_tokens` | `2048` | Positive validation batching budget. |
| `chunking.target_sections` | Unset | Optional positive target; omission or null selects automatic chunking. |
| `chunking.max_section_tokens` | `8192` | Positive maximum estimate. |
| `chunking.min_section_tokens` | `2048` | Positive and no greater than maximum. |
| `normalization.max_segment_gap` | `4.0` | Nonnegative numeric seconds. |
| `normalization.ellipsis_gap` | `3.5` | Nonnegative numeric seconds. |
| `normalization.max_segment_duration` | `60.0` | Positive numeric seconds. |
| `normalization.max_segment_tokens` | `2048` | Positive token estimate. |
| `thresholds.glossary`, `.homophones`, `.spoken_word`, `.grammar` | `0.8` each | Confidence thresholds in `[0, 1]`. |
| `context.description` | Empty | Trimmed prompt context, at most 500 bytes. |
| `reporting.enabled` | `true` | Write the ordinary three-file report bundle when processing. |
| `reporting.stdout_dir` | `.` | Report directory for stdout transcript output; a `--report-dir` flag can select reports for either transcript destination. |
| `diagnostics.debug` | `false` | Persist detailed debug artifacts when processing. |
| `diagnostics.work_dir` | `/tmp/audita` | Parent of the retained debug run directory; required only when debug is enabled. |
| `thresholds.<module>` | `0.8` | Root confidence threshold for each module, in `[0, 1]`. |
| `context.description` | empty | Prompt context; at most 500 bytes after trimming. |
| `chunking.target_sections` | unset | Optional positive target. |
| `chunking.max_section_tokens`, `chunking.min_section_tokens` | `8192`, `2048` | Positive section bounds; minimum must not exceed maximum. |
| `normalization.max_segment_gap`, `normalization.ellipsis_gap` | `4`, `3.5` | Nonnegative seconds. |
| `normalization.max_segment_duration`, `normalization.max_segment_tokens` | `60`, `2048` | Positive seconds and token estimate. |
| `output.schema` | `bare-segments` | Output format; see [output contracts](integrations/output-files.md). |
| `reporting.enabled`, `reporting.stdout_dir` | `true`, `.` | Ordinary report capture and default directory for stdout output. |
| `diagnostics.work_dir` | `/tmp/audita` | Parent directory for optional debug capture. |
Module keys select correction modules; validator chains remain built into them.
The loader preserves module order and repetitions. Normalization time settings
accept numeric seconds only. The operation timeout uses a duration string.
`reporting.stdout_dir` must be nonblank when it supplies the active report
destination: for stdout output, or for file output when `--report-dir` is
explicitly supplied. File output without that flag uses the transcript's parent,
so the configured value is inactive. Reporting-disabled runs do not select a
report directory. `diagnostics.work_dir` must be nonblank only with debug
enabled. These path requirements are checked when choosing process destinations,
not during configuration inspection.
## Credentials
A backend credential is a ConfigKit secret reference containing an environment
variable name, such as `credential: AUDITA_SERVICE_KEY`. Every reference that
survives overlay and override resolution must name a set, nonempty variable,
including credentials on backends unused by the selected roles. A replacement
reference resolves only the replacement; null removes the reference. General
`AUDITA_*` and `OPENROUTER_API_KEY` variables have no implicit setting effect.
Secret values are available to the runtime through the resolved reference, while
ConfigKit's effective output and provenance contain no resolved secret values.
Current invocation syntax belongs in [CLI](cli.md).
Processing profiles are selected explicitly; no profile or glossary is selected
automatically. The validation selection falls back after root and per-entry
settings are combined. Null does not disable validation. Numeric zero and false
retain their field meaning, subject to validation. Inspect recorded origins and
default dependencies using the [native commands](cli.md).