Document the new configuration and processing workflow
This commit is contained in:
49
README.md
49
README.md
@@ -1,55 +1,34 @@
|
||||
# Audita
|
||||
|
||||
Audita polishes transcript JSON using glossary-aware and LLM-backed correction
|
||||
modules. It validates proposed corrections and produces corrected text plus
|
||||
reports for review and subprocess integration.
|
||||
Audita polishes transcript JSON with glossary-aware and LLM-backed correction
|
||||
modules. It validates proposed edits and produces corrected text and reports for
|
||||
review or subprocess integration.
|
||||
|
||||
## Quickstart
|
||||
|
||||
From a checkout, build with Go 1.26.0 or newer:
|
||||
Build with Go 1.27.0 or newer:
|
||||
|
||||
```sh
|
||||
go build -o ./bin/audita ./cmd/audita
|
||||
```
|
||||
|
||||
Processing requires a reachable [compatible LLM endpoint](docs/integrations/openai-compatible-llm.md)
|
||||
and access to a model you select. The [minimal config](examples/minimal-config.yml)
|
||||
targets an unauthenticated local endpoint; edit its endpoint and
|
||||
[local profiles](examples/profiles/local/) to match your service and models. The
|
||||
[service example](examples/production-config.yml) demonstrates a named credential
|
||||
reference and a shared backend. Set its referenced environment variable before
|
||||
validating it. The [testing overlay](examples/overlays/testing.yml) selects both
|
||||
role profiles, but model cost, speed, and quality depend on your provider choices.
|
||||
|
||||
Check configuration and profile selection offline:
|
||||
Edit the synthetic endpoint and model in [examples/local/prompts.yml](examples/local/prompts.yml)
|
||||
for a reachable OpenAI-compatible service. From `examples/local`, run the built
|
||||
executable (adjust the executable path if needed):
|
||||
|
||||
```sh
|
||||
./bin/audita config validate --config examples/minimal-config.yml
|
||||
./bin/audita config print-effective --config examples/minimal-config.yml --view profiles
|
||||
../../bin/audita config validate --config config.yml
|
||||
../../bin/audita process --config config.yml --output corrected.json transcript.json
|
||||
```
|
||||
|
||||
Run the included small input with the minimal config:
|
||||
|
||||
```sh
|
||||
./bin/audita process examples/tiny-transcript.json \
|
||||
--config examples/minimal-config.yml \
|
||||
--glossary examples/tiny-glossary.yaml \
|
||||
--output corrected.json
|
||||
```
|
||||
|
||||
This makes LLM requests. Explicit setting flags override the selected file and
|
||||
optional overlay; general environment setting overrides are no longer read.
|
||||
By default the command writes `errors.yml`, `recoveries.yml`, and `report.md`
|
||||
beside `corrected.json`. Use `--reporting=false` to suppress those files, or
|
||||
`--debug` to retain detailed run artifacts under the configured debug work
|
||||
directory. Model failures affecting some operations can still yield a completed
|
||||
transcript with applied corrections; the summary and reports identify them.
|
||||
See the [v2 migration guide](docs/migrations/v2.md) for old configurations.
|
||||
Processing makes model requests. Reports are written beside the output by default;
|
||||
recoverable model errors can appear in a completed run. The [example guide](examples/README.md)
|
||||
includes service, overlay, prompt editing, preview, and shared glossary projects.
|
||||
See the [v3 migration guide](docs/migrations/v3.md) for the configuration cutover.
|
||||
|
||||
## Documentation
|
||||
|
||||
- [CLI reference](docs/cli.md)
|
||||
- [Configuration and maintained examples](docs/config.md)
|
||||
- [CLI reference](docs/cli.md) and [configuration](docs/config.md)
|
||||
- [Operations and artifact lifecycle](docs/operations.md)
|
||||
- [Transcript and glossary inputs](docs/integrations/transcript-glossary-files.md)
|
||||
- [Output files and report formats](docs/integrations/output-files.md)
|
||||
|
||||
42
docs/adr/0002-glossary-configuration-and-preflight.md
Normal file
42
docs/adr/0002-glossary-configuration-and-preflight.md
Normal file
@@ -0,0 +1,42 @@
|
||||
# ADR 0002: Glossary configuration and processing readiness
|
||||
|
||||
**Status:** Accepted
|
||||
|
||||
**Date:** 2026-11-10
|
||||
|
||||
## Context
|
||||
|
||||
Operators need to reuse glossary collections across campaigns and select different
|
||||
context and protected terms for each ordered correction. Configuration inspection
|
||||
must remain useful without a transcript, model credentials, or a populated glossary.
|
||||
A processing run must reject an unusable later correction before publishing outputs
|
||||
or spending resources on earlier corrections.
|
||||
|
||||
## Decision
|
||||
|
||||
ConfigKit composes typed, named glossary collections with the application
|
||||
configuration. Each pipeline entry selects context and protection independently;
|
||||
Audita derives consumer views without changing the resolved configuration. For a
|
||||
glossary correction, active recognized spellings are exempt from that entry's
|
||||
protection view. Other protected spellings and validators remain effective.
|
||||
|
||||
After configuration resolution, Audita preflight checks the whole pipeline and
|
||||
coordinates output preparation. It consumes the typed configuration and acquired
|
||||
inputs, without reloading configuration or calling a provider. The output session
|
||||
owns preparation, cleanup, and publication. See the [configuration](../config.md)
|
||||
and [operations](../operations.md) contracts for current behavior.
|
||||
|
||||
## Alternatives considered
|
||||
|
||||
A separately parsed glossary per invocation would duplicate composition and make
|
||||
per-entry selection and provenance harder to express. Requiring nonempty
|
||||
collections in configuration validation would prevent inspection and reuse of
|
||||
valid empty definitions. Preparing outputs or checking each entry only when it
|
||||
runs could leave artifacts or incur model work before finding a later error.
|
||||
|
||||
## Consequences
|
||||
|
||||
Configuration and runtime views have distinct responsibilities. Inspection can
|
||||
show configured selections and origins without presenting derived vocabulary as
|
||||
configuration. Preflight and the output session need a clear handoff, and the
|
||||
processing pipeline must carry each entry's own effective settings and identity.
|
||||
@@ -4,6 +4,8 @@
|
||||
|
||||
- [ADR 0001: Adopt ReleaseKit for release management](0001-releasekit.md) —
|
||||
accepted 2026-10-07.
|
||||
- [ADR 0002: Glossary configuration and processing readiness](0002-glossary-configuration-and-preflight.md) —
|
||||
accepted 2026-11-10.
|
||||
|
||||
Use the
|
||||
[documentation policy's ADR lifecycle](../policy/documentation.md#architecture-decision-records)
|
||||
|
||||
139
docs/cli.md
139
docs/cli.md
@@ -1,116 +1,45 @@
|
||||
# Audita CLI Reference
|
||||
|
||||
The shared version-2 configuration parser and offline profile inspection serve
|
||||
`process`, `config validate`, and `config print-effective`.
|
||||
The [configuration reference](config.md) owns fields, defaults, and validation;
|
||||
the [migration guide](migrations/v2.md) maps retired inputs.
|
||||
Audita uses CommandKit for routing and the ConfigKit and PromptKit integrations
|
||||
for their standard inspection and request commands. Run `audita --help` or
|
||||
command-specific `--help` for the installed command inventory. The adopted
|
||||
[CommandKit consumer guide](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/f369d6a78253b6e6fcb1e023fb2e9c3af1d20ff0/docs/consumers/guide.md)
|
||||
describes the standard command grammar, inspection, comparison, preview,
|
||||
validation, smoke tests, and their provider-specific caveats. Help and version
|
||||
do not load application or model configuration.
|
||||
|
||||
## Commands and help
|
||||
## Processing
|
||||
|
||||
```sh
|
||||
audita process <transcript.json> --glossary <glossary.yml> [flags]
|
||||
audita config validate --config <config.yml> [flags]
|
||||
audita config print-effective [flags]
|
||||
audita version
|
||||
audita process [flags] transcript.json
|
||||
```
|
||||
|
||||
Root invocation, `--help`, `-h`, and command help succeed without loading a config
|
||||
file or resolving secrets. Unknown commands and flags return exit code `2`.
|
||||
`process` accepts exactly one transcript path, either before the flags or after
|
||||
them. With flags first, put the transcript last. Both config commands reject
|
||||
positional arguments and process-only flags.
|
||||
`--config` selects a main configuration and `--overlay` selects its overlay.
|
||||
`--output` writes the transcript to a file; omission selects stdout. The shared
|
||||
`--debug` flag enables retained debug artifacts, and `--quiet` suppresses optional
|
||||
messages. Quiet processing requires an output file or `/dev/null`. See
|
||||
[operations](operations.md) for output and report lifecycle.
|
||||
|
||||
All three commands accept `--config <path>` and `--overlay <name>`. `config validate`
|
||||
requires an explicit nonempty `--config`; the other commands select the explicit
|
||||
file, then `/usr/local/etc/audita/config.yml`, then `/etc/audita/config.yml`.
|
||||
Missing system candidates are skipped; inaccessible selected files are errors.
|
||||
No file means a configuration error. `AUDITA_CONFIG` is ignored. The selected
|
||||
overlay is loaded beside the main file, not used to locate profile files.
|
||||
Explicit process flags override root settings only. `--glossaries` and
|
||||
`--protected-terms` each accept one JSON array of collection names or `null`;
|
||||
`[]` clears the selection. Set pipeline order and per-entry overrides in YAML.
|
||||
Other supplied flags cover profiles, output schema, description, reporting,
|
||||
chunking, normalization, confidence thresholds, validation budget, and debug
|
||||
work directory. `audita process --help` lists the available flags. An omitted flag
|
||||
does not contribute an override. [Configuration](config.md) defines their values
|
||||
and inheritance.
|
||||
|
||||
`audita version` prints `audita <version>` and a newline to stdout, with no
|
||||
application configuration or provider setup. It returns `0` on success with
|
||||
empty stderr. `help version`, `version -h`, and `version --help` print command
|
||||
usage to stdout and return `0`. Help follows standard flag parsing and can
|
||||
short-circuit trailing arguments. Other invalid invocations return `2` with a
|
||||
usage error on stderr. The command uses ReleaseKit's public
|
||||
`buildinfo.Version()` accessor. It reports an injected release identifier when
|
||||
present, otherwise the executable's main-module version when available and not
|
||||
`(devel)`, and `dev` when neither source provides a release identifier.
|
||||
## Inspection and outcomes
|
||||
|
||||
## Setting overrides
|
||||
The application configuration scope uses ConfigKit's native effective values,
|
||||
source evidence, explanation, comparison, and redaction. PromptKit supplies its
|
||||
own configuration and resource scopes and request preparation, validation, and
|
||||
explicit smoke diagnostics. Inspection does not require a processing-ready
|
||||
pipeline or contact a provider. Smoke tests can contact providers and are run
|
||||
only on explicit request.
|
||||
|
||||
Only flags actually supplied override file or overlay settings. `--modules ''`
|
||||
clears the module pipeline, and an explicit numeric zero reaches configuration
|
||||
validation. The shared setting flags are:
|
||||
|
||||
| Setting | Flags |
|
||||
| --- | --- |
|
||||
| Profile selection | `--proposal-profile`, `--validation-profile` |
|
||||
| Pipeline and output | `--modules`, `--output-schema` |
|
||||
| Validation budget | `--validation-max-prompt-tokens` |
|
||||
| Chunking | `--target-sections`, `--max-section-tokens`, `--min-section-tokens` |
|
||||
| Normalization (numeric seconds) | `--normalize-max-segment-gap`, `--normalize-ellipsis-gap`, `--normalize-max-segment-duration`, `--normalize-max-segment-tokens` |
|
||||
| Confidence thresholds | `--glossary-confidence-threshold`, `--homophones-confidence-threshold`, `--spoken-word-confidence-threshold`, `--grammar-confidence-threshold` |
|
||||
| Context | `--transcript-description` |
|
||||
| Reporting | `--reporting[=true\|false]`, `--report-dir DIR` |
|
||||
| Debug | `--debug[=true\|false]`, `--work-dir DIR` |
|
||||
|
||||
`process` alone accepts `--glossary` (required) and `--output`.
|
||||
An omitted or empty output value selects stdout. Nonempty output filenames are
|
||||
used literally, including whitespace.
|
||||
Reporting defaults to enabled. `--reporting=false` disables all three ordinary
|
||||
reports; an explicit false overrides a true file or overlay setting. For stdout
|
||||
output, reports use the effective `reporting.stdout_dir`, which defaults to the
|
||||
invocation working directory. For file output, they go beside the transcript
|
||||
unless `--report-dir` is explicitly supplied. The flag selects the report
|
||||
directory for either transcript destination; a config-file or overlay value for
|
||||
`reporting.stdout_dir` alone affects stdout output only. A selected report
|
||||
directory must be nonblank when reporting is enabled. With reporting disabled,
|
||||
the report destination is inactive and is not probed. `--debug` separately
|
||||
enables retained detailed artifacts under `--work-dir`; selecting `--work-dir`
|
||||
alone does not enable capture. Configuration inspection does not create or
|
||||
probe these output directories.
|
||||
Per-role model, base URL, key, timeout, retry, and concurrency flags have been
|
||||
removed. Models and generation controls belong to PromptKit profiles, backend
|
||||
credentials and capacity to configuration, and retry/deadline settings to the
|
||||
shared LLM section. Legacy configuration environment overrides are ignored.
|
||||
|
||||
## Streams and exit status
|
||||
|
||||
`config validate` loads settings and both selected PromptKit profiles offline,
|
||||
then prints `config is valid` on success. `config print-effective` accepts
|
||||
`--view settings|profiles|explain` and defaults to `settings`. Settings writes
|
||||
ConfigKit's redacted effective-settings YAML. Profiles writes an Audita-owned
|
||||
YAML summary of selected ID, backend, model, sampling controls, per-attempt
|
||||
timeout, service tier, and reasoning effort for each role. It omits credentials,
|
||||
endpoints, and arbitrary extra provider parameters. Explain writes a YAML
|
||||
sequence of field paths, source origins, and recorded derivations without
|
||||
resolved setting values. An inherited field may also include an `inheritance`
|
||||
sequence; each step identifies `from`, `to`, and `reference` paths as JSON
|
||||
pointers. These fields describe the effective inheritance chain without
|
||||
including configuration values. A direct child override reports its own source
|
||||
instead. Debug configuration origins use the same explanation shape. These views
|
||||
share the same settings and profile validation path as `process`. Unknown views
|
||||
are usage errors. The effective settings YAML is for inspection and is not
|
||||
guaranteed reloadable.
|
||||
|
||||
For `process`, `--output` writes the corrected transcript to a file and leaves
|
||||
stdout empty on success. Without it, stdout carries transcript JSON only on
|
||||
success. Enabled reporting creates `errors.yml`, `recoveries.yml`, and
|
||||
`report.md` in the selected directory. Successful stderr begins with
|
||||
`Completed: N corrections applied; E work units with errors; R recoveries.`
|
||||
Report and debug locations follow only when created. Model-operation failures
|
||||
can coexist with a completed run and applied corrections; read the YAML reports
|
||||
for details. Debug artifacts are created only with `--debug`. See the
|
||||
[report contract](integrations/output-files.md).
|
||||
|
||||
Exit codes are `0` for success, `1` for runtime or output failure, and `2` for
|
||||
usage or configuration input errors. Invalid settings and profiles return `2`;
|
||||
broken embedded assets, caller cancellation, fatal processing, and required
|
||||
output failures return `1`. A late report-write failure can follow a published
|
||||
transcript; exit status and report status remain authoritative.
|
||||
Configuration commands inspect profiles without contacting a provider.
|
||||
|
||||
See the [input files](integrations/transcript-glossary-files.md),
|
||||
[subprocess contract](integrations/subprocess.md), and
|
||||
[artifact lifecycle](operations.md) for their respective details.
|
||||
Process stdout contains transcript JSON when no output file is selected. Ordinary
|
||||
reports may record recoverable model failures even when processing completes.
|
||||
Exit status `0` means success, `1` runtime or output failure, `2` invalid usage
|
||||
or input, and native comparison may return `3` for differences. See the
|
||||
[output contract](integrations/output-files.md) and [subprocess contract](integrations/subprocess.md).
|
||||
|
||||
237
docs/config.md
237
docs/config.md
@@ -1,192 +1,81 @@
|
||||
# Audita Configuration
|
||||
|
||||
This reference describes the version-2 configuration loader in `internal/core/config`.
|
||||
Use the maintained [local](../examples/minimal-config.yml) and
|
||||
[service](../examples/production-config.yml) examples with their separately
|
||||
selected [local profiles](../examples/profiles/local/) and
|
||||
[service profiles](../examples/profiles/service/), plus the optional
|
||||
[testing overlay](../examples/overlays/testing.yml).
|
||||
The [ConfigKit inheritance example](../examples/configkit-inheritance.yml)
|
||||
shows a local child backend reusing and overriding sibling settings.
|
||||
Choose real models for your endpoint; a testing profile makes no cost or quality
|
||||
guarantee. See the [migration guide](migrations/v2.md) for retired inputs.
|
||||
Audita uses version 3 application configuration. See the [migration guide](migrations/v3.md)
|
||||
and [examples](../examples/) for complete projects. Command syntax is owned by
|
||||
[the CLI reference](cli.md).
|
||||
|
||||
## Loading and precedence
|
||||
## Loading and composition
|
||||
|
||||
The application selects one main YAML file: an explicit `--config`, then
|
||||
`/usr/local/etc/audita/config.yml` if present, then `/etc/audita/config.yml`
|
||||
if present. Missing system candidates are skipped; an explicit missing or empty
|
||||
path, inaccessible selected file, or absence of any selected file is an error.
|
||||
`config validate` requires an explicit file. ConfigKit loads built-in defaults,
|
||||
the selected file, an optional named overlay from `overlays/<name>.yml` beside the
|
||||
main file, and only explicitly supplied setting overrides, in that order. Imports
|
||||
and secret-directory bootstrap are disabled. Unknown fields and missing, null,
|
||||
or non-integer `version: 2` are errors. Version-1 files require migration; no
|
||||
compatibility loader or implicit environment configuration remains.
|
||||
An explicit `--config` selects one main YAML file. Without it, ConfigKit searches
|
||||
the working directory, user configuration directory, and Unix system directories
|
||||
for `audita` configuration. If no file is found, built-in defaults apply. An
|
||||
explicit invalid selection or failure of a selected file is an error; search does
|
||||
not continue after a selected file fails. An omitted version uses the v3 schema;
|
||||
a supplied version must be `3`.
|
||||
|
||||
After overlay and explicit CLI override precedence is applied, ConfigKit resolves
|
||||
native sibling inheritance before typed decoding. Any input mapping may use
|
||||
`extends: <sibling-name>`; the child keeps its key, inherits parent settings, and
|
||||
overrides them with its own settings. Selected overlay and CLI values therefore
|
||||
participate in inheritance at their normal precedence. The generic syntax and
|
||||
resolution rules are defined by ConfigKit v0.7.1's
|
||||
[sibling inheritance contract](https://gitea.maximumdirect.net/eric/ConfigKit/src/tag/v0.7.1/docs/config.md#sibling-inheritance).
|
||||
|
||||
`extends` is reserved structural input in base files, imports, overlays, and
|
||||
other application input mappings, including mapping values supplied by explicit
|
||||
CLI setting overrides. ConfigKit consumes it before typed decoding. A previously
|
||||
valid backend whose ID is literally `extends` cannot be represented under this
|
||||
rule. Rename that backend to a non-conflicting ID and update every PromptKit
|
||||
profile that selects it. This compatibility impact means the upgrade does not
|
||||
promise that every previously valid configuration remains valid.
|
||||
|
||||
Inherited values count as explicit input, including zero, false, empty, and
|
||||
null. Existing field rules still apply: for example, inherited backend
|
||||
concurrency must be positive, while an inherited queue capacity of zero is
|
||||
valid. Parent backends undergo the same decoding and application validation as
|
||||
other backends, whether or not a selected profile uses them. A child
|
||||
`credential: null` removes the child's inherited credential; it does not remove
|
||||
the parent's credential reference, which remains subject to ordinary secret
|
||||
resolution.
|
||||
|
||||
Validation runs after overlays, explicit overrides, and derived settings have
|
||||
resolved. An override can therefore repair an invalid lower-precedence setting
|
||||
when that setting can be decoded. ConfigKit retains field origins and derivations
|
||||
for effective-configuration inspection. Its effective YAML redacts resolved
|
||||
credentials.
|
||||
Operator sources may use ConfigKit imports. The selected main file provides the
|
||||
base for an optional `overlays/<name>.yml`, optional `glossary.yml`, and recursive
|
||||
`glossary/` YAML files. Without a main source, automatic companion files are not
|
||||
loaded from the working directory. ConfigKit composes sources, resolves native
|
||||
inheritance, applies explicit overrides, records derived defaults, and validates
|
||||
the result. Its [configuration reference](https://forge.maximumdirect.net/base-layer/configkit/src/tag/v1.2.0/docs/config.md)
|
||||
owns native composition and provenance semantics.
|
||||
|
||||
### PromptKit profiles and operator prompts
|
||||
|
||||
`profiles_dir` selects a PromptKit configuration root, resolved against the
|
||||
selected main file's directory unless it is absolute. The root may contain
|
||||
keyed `profiles`, `backends`, and `prompts` definitions; it is not limited to
|
||||
profile-only files. IDs come from literal YAML mapping keys, never filenames.
|
||||
Multiple definitions may share a document, and a document may contribute to any
|
||||
of the supported mappings. Directory discovery is recursive, but discovery
|
||||
order does not establish field precedence. Contributions within each source
|
||||
group must compose without conflicting values.
|
||||
PromptKit loads its default catalog and Audita's embedded prompts, followed by
|
||||
optional `prompts.yml` and recursive `prompts/` YAML files beside the application
|
||||
main. Native imports, resource collections, inheritance, and replacement apply.
|
||||
Profiles, backends, models, credentials, admission, and provider parameters belong
|
||||
to PromptKit configuration; see its [format reference](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.21.0/docs/formats.md).
|
||||
The engine captures its sources for an invocation. Edit files and run a new
|
||||
invocation to pick up changes. Inspection does not require model credentials.
|
||||
|
||||
PromptKit assembles sources in this order: its selected catalogs and owned
|
||||
defaults, Audita's embedded prompt/schema inputs, operator documents under
|
||||
`profiles_dir`, then the application backend mapping. The application mapping
|
||||
owns each configured endpoint, concurrency limit, queue capacity, and credential
|
||||
environment lookup. Other supported provider fields can be contributed by the
|
||||
operator documents. Imports remain disabled. The current maintained examples
|
||||
select separate roots: [local](../examples/profiles/local/) and
|
||||
[service](../examples/profiles/service/), so definitions for one backend do not
|
||||
enter the other example's final configuration.
|
||||
## Pipeline and glossary selections
|
||||
|
||||
Both selected roles must resolve to an existing profile, registered backend,
|
||||
nonblank effective model, and nonblank endpoint. The normalized effective
|
||||
endpoint must match the application backend endpoint. A selected profile cannot
|
||||
introduce a competing `api_key_env` lookup; the application backend credential
|
||||
supplies the runtime key when required. Other unselected profiles may be
|
||||
partial, but every final backend, profile, and prompt definition is validated
|
||||
when the engine is constructed. A malformed or conflicting unused definition
|
||||
can therefore make the selected root invalid. A missing selected root is an
|
||||
operator configuration error; an empty root may contribute no definitions.
|
||||
`glossary_collections` maps names to collections with an ordered `entries` list.
|
||||
Each entry has an `id`, `name`, and `category`; `description`, `aliases`, and
|
||||
`plural` provide optional context. Entries are identified within their collection
|
||||
by `id`. Root `glossaries` and `protected_terms` select collection names. If root
|
||||
protection is omitted, it derives from the root glossary selection.
|
||||
|
||||
Profile IDs and parent IDs are exact, literal keys. PromptKit profiles can use
|
||||
native `extends` inheritance; child fields override inherited fields, while
|
||||
unmentioned controls survive changes to model or backend. For example:
|
||||
`pipeline.stages` is an ordered list of `module` records; supported modules are
|
||||
`glossary`, `homophones`, `spoken_word`, and `grammar`. The default sequence is
|
||||
`glossary`, `homophones`, `glossary`, `spoken_word`, `grammar`. Repeated modules
|
||||
run separately; `[]` runs normalization without model processing. An optional
|
||||
nonempty, unique `id` identifies a record in evidence. Each record may override
|
||||
`glossaries`, `protected_terms`, `confidence_threshold`, and fields under `llm`,
|
||||
`validation`, and `context`. Omitted fields inherit their resolved root values.
|
||||
Explicit empty lists and null list selections clear a selection.
|
||||
|
||||
```yaml
|
||||
profiles:
|
||||
"shared~base":
|
||||
backend: service
|
||||
model: example-model
|
||||
temperature: 0.4
|
||||
top_p: 0.8
|
||||
"proposal fast":
|
||||
extends: "shared~base"
|
||||
model: faster-model
|
||||
temperature: 0
|
||||
```
|
||||
|
||||
Presence matters. Explicit `temperature: 0` is sent as zero; omission leaves it
|
||||
unspecified; null suppresses an inherited temperature. Other fields have their
|
||||
own native null and omission rules. Provider `extra_params` maps merge
|
||||
recursively, and a null provider value is sent as JSON null. See the pinned
|
||||
[PromptKit format reference](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.14.0/docs/formats.md)
|
||||
for the complete field semantics.
|
||||
|
||||
Operator documents can also override an embedded prompt/version, including its
|
||||
messages and definition-level output settings. Shared variables supplied by the
|
||||
adapter are rendered only when an effective message references them; Audita does
|
||||
not append omitted shared text. Audita's per-call schemas, structural repair
|
||||
budget, and downstream domain validators remain in force. The engine captures
|
||||
effective definitions and message-file contents when constructed, so edits are
|
||||
observed only by a newly constructed runtime.
|
||||
|
||||
### Migrating PromptKit profile documents
|
||||
|
||||
For the PromptKit v0.11 format, wrap each profile's fields under
|
||||
`profiles: {<literal-id>: ...}`, remove inner `id`, and replace
|
||||
`base_profile` with `extends`. Profile and prompt IDs are mapping keys, so they
|
||||
need not match source filenames. Review zeros, omitted values, and nulls against
|
||||
the native presence rules before deploying. Select a root whose final
|
||||
definitions have valid backend references; eager validation applies even to
|
||||
unselected definitions. See the complete maintained [local](../examples/minimal-config.yml)
|
||||
and [service](../examples/production-config.yml) examples and the pinned
|
||||
[consumer migration guide](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.14.0/docs/consumers/pkg-promptkit.md#migrate-from-the-previous-configuration-api).
|
||||
This incompatible profile-format migration is intended for an Audita major
|
||||
release; no automatic file conversion is provided.
|
||||
|
||||
The [CLI inspection views](cli.md#streams-and-exit-status) show selected
|
||||
controls and field provenance separately from ConfigKit's redacted settings
|
||||
YAML. Inspection resolves both roles without provider calls.
|
||||
Each entry receives its selected context and independent protection. Glossary
|
||||
correction exempts active recognized spellings from protection for that entry;
|
||||
other protected spellings remain protected. A glossary correction must have at
|
||||
least one selected entry at processing preflight. Empty definitions and empty
|
||||
protection remain valid configuration.
|
||||
|
||||
## Settings
|
||||
|
||||
| Field | Default | Rule |
|
||||
| Setting | Default | Meaning |
|
||||
| --- | --- | --- |
|
||||
| `profiles_dir` | `profiles` | Nonempty PromptKit document root; resolved relative to the selected main file, or absolute. |
|
||||
| `backends.<id>.endpoint` | None | Required nonempty endpoint for each backend. At least one backend is required. |
|
||||
| `backends.<id>.concurrency` | `1` | Positive finite capacity; shared by both roles using that backend. |
|
||||
| `backends.<id>.queue_capacity` | `1024` | Nonnegative; zero disables waiting admission. |
|
||||
| `backends.<id>.credential` | Unset | Optional environment-variable secret reference; omission or null means unauthenticated use. |
|
||||
| `llm.proposal_profile` | None | Required nonempty profile ID. |
|
||||
| `llm.validation_profile` | Final proposal profile | Omission inherits the final proposal selection; explicit empty or null fails. |
|
||||
| `llm.transient_retries` | `2` | Nonnegative additional attempts per generation. |
|
||||
| `llm.repair_attempts` | `2` | Additional structural repair generations, from zero through three. |
|
||||
| `llm.operation_timeout` | `15m` | Positive Go duration string per logical call. |
|
||||
| `pipeline.modules` | `[glossary, homophones, glossary, spoken_word, grammar]` | Ordered module keys; repeats are meaningful and an explicit empty list runs no correction modules. |
|
||||
| `output.schema` | `bare-segments` | `bare-segments` or `audita-v1`. See [output formats](integrations/output-files.md). |
|
||||
| `llm.proposal_profile` | unset | PromptKit profile for proposals; required for processing entries. |
|
||||
| `llm.validation_profile` | effective proposal profile | Profile for model validation; null clears a selection before fallback. |
|
||||
| `llm.transient_retries` | `2` | Additional transient attempts per generation. |
|
||||
| `llm.repair_attempts` | `2` | Structural repairs, from zero through three. |
|
||||
| `llm.operation_timeout` | `15m` | Positive duration per logical operation. |
|
||||
| `validation.max_prompt_tokens` | `2048` | Positive validation batching budget. |
|
||||
| `chunking.target_sections` | Unset | Optional positive target; omission or null selects automatic chunking. |
|
||||
| `chunking.max_section_tokens` | `8192` | Positive maximum estimate. |
|
||||
| `chunking.min_section_tokens` | `2048` | Positive and no greater than maximum. |
|
||||
| `normalization.max_segment_gap` | `4.0` | Nonnegative numeric seconds. |
|
||||
| `normalization.ellipsis_gap` | `3.5` | Nonnegative numeric seconds. |
|
||||
| `normalization.max_segment_duration` | `60.0` | Positive numeric seconds. |
|
||||
| `normalization.max_segment_tokens` | `2048` | Positive token estimate. |
|
||||
| `thresholds.glossary`, `.homophones`, `.spoken_word`, `.grammar` | `0.8` each | Confidence thresholds in `[0, 1]`. |
|
||||
| `context.description` | Empty | Trimmed prompt context, at most 500 bytes. |
|
||||
| `reporting.enabled` | `true` | Write the ordinary three-file report bundle when processing. |
|
||||
| `reporting.stdout_dir` | `.` | Report directory for stdout transcript output; a `--report-dir` flag can select reports for either transcript destination. |
|
||||
| `diagnostics.debug` | `false` | Persist detailed debug artifacts when processing. |
|
||||
| `diagnostics.work_dir` | `/tmp/audita` | Parent of the retained debug run directory; required only when debug is enabled. |
|
||||
| `thresholds.<module>` | `0.8` | Root confidence threshold for each module, in `[0, 1]`. |
|
||||
| `context.description` | empty | Prompt context; at most 500 bytes after trimming. |
|
||||
| `chunking.target_sections` | unset | Optional positive target. |
|
||||
| `chunking.max_section_tokens`, `chunking.min_section_tokens` | `8192`, `2048` | Positive section bounds; minimum must not exceed maximum. |
|
||||
| `normalization.max_segment_gap`, `normalization.ellipsis_gap` | `4`, `3.5` | Nonnegative seconds. |
|
||||
| `normalization.max_segment_duration`, `normalization.max_segment_tokens` | `60`, `2048` | Positive seconds and token estimate. |
|
||||
| `output.schema` | `bare-segments` | Output format; see [output contracts](integrations/output-files.md). |
|
||||
| `reporting.enabled`, `reporting.stdout_dir` | `true`, `.` | Ordinary report capture and default directory for stdout output. |
|
||||
| `diagnostics.work_dir` | `/tmp/audita` | Parent directory for optional debug capture. |
|
||||
|
||||
Module keys select correction modules; validator chains remain built into them.
|
||||
The loader preserves module order and repetitions. Normalization time settings
|
||||
accept numeric seconds only. The operation timeout uses a duration string.
|
||||
`reporting.stdout_dir` must be nonblank when it supplies the active report
|
||||
destination: for stdout output, or for file output when `--report-dir` is
|
||||
explicitly supplied. File output without that flag uses the transcript's parent,
|
||||
so the configured value is inactive. Reporting-disabled runs do not select a
|
||||
report directory. `diagnostics.work_dir` must be nonblank only with debug
|
||||
enabled. These path requirements are checked when choosing process destinations,
|
||||
not during configuration inspection.
|
||||
|
||||
## Credentials
|
||||
|
||||
A backend credential is a ConfigKit secret reference containing an environment
|
||||
variable name, such as `credential: AUDITA_SERVICE_KEY`. Every reference that
|
||||
survives overlay and override resolution must name a set, nonempty variable,
|
||||
including credentials on backends unused by the selected roles. A replacement
|
||||
reference resolves only the replacement; null removes the reference. General
|
||||
`AUDITA_*` and `OPENROUTER_API_KEY` variables have no implicit setting effect.
|
||||
Secret values are available to the runtime through the resolved reference, while
|
||||
ConfigKit's effective output and provenance contain no resolved secret values.
|
||||
|
||||
Current invocation syntax belongs in [CLI](cli.md).
|
||||
Processing profiles are selected explicitly; no profile or glossary is selected
|
||||
automatically. The validation selection falls back after root and per-entry
|
||||
settings are combined. Null does not disable validation. Numeric zero and false
|
||||
retain their field meaning, subject to validation. Inspect recorded origins and
|
||||
default dependencies using the [native commands](cli.md).
|
||||
|
||||
@@ -41,7 +41,7 @@ ownership. Commit discipline and agent Git restrictions are in the
|
||||
|
||||
## Baseline Validation
|
||||
|
||||
Use Go 1.26.0 or newer for source builds. The selected release toolchain is
|
||||
Use Go 1.27.0 or newer for source builds. The selected release toolchain is
|
||||
documented in the [release procedure](release.md#prerequisites). From the
|
||||
repository root:
|
||||
|
||||
|
||||
@@ -1,26 +1,26 @@
|
||||
# OpenAI-Compatible LLM Integration
|
||||
|
||||
Audita uses the pinned [PromptKit](https://gitea.maximumdirect.net/eric/promptkit)
|
||||
Audita uses the pinned [PromptKit](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/171fb11f4e967a1da3af57843c6467954b14ba09)
|
||||
built-in OpenAI-compatible transport. PromptKit owns chat-completions wire framing,
|
||||
provider envelope parsing, transient retries, per-generation timeouts, and JSON
|
||||
Schema repair. A backend's configured base endpoint must support the selected
|
||||
profile's model and structured JSON-schema responses; local configuration
|
||||
inspection does not contact the provider.
|
||||
|
||||
Audita selects a role profile, supplies its registered backend credential, and
|
||||
passes the embedded response schema as a complete per-call output contract. It
|
||||
prepares each call once, checks that the prepared target still matches the
|
||||
inspected backend and endpoint, then executes that same prepared request. A
|
||||
profile cannot introduce a separate credential environment lookup. An absent
|
||||
backend credential permits an unauthenticated local service.
|
||||
Audita selects role profiles independently for each pipeline entry and supplies
|
||||
its domain response schema and per-operation budget. PromptKit owns backend and
|
||||
credential selection, request preparation, transport, retries, repairs, and
|
||||
admission. Audita executes the same prepared handle and retains its own domain
|
||||
validation. Profiles may use native endpoint and credential settings; local
|
||||
inspection does not resolve a credential value or call the provider.
|
||||
|
||||
`llm.transient_retries` counts additional attempts for eligible transient
|
||||
failures; explicit zero makes one attempt. `llm.repair_attempts` counts
|
||||
additional generations for schema-invalid model output. The operation timeout
|
||||
covers preparation, capacity waiting, retries, generation, and repair. Caller
|
||||
cancellation or an earlier deadline also applies. See [configuration](../config.md#settings)
|
||||
for defaults and [PromptKit v0.14.0's consumer contract](https://gitea.maximumdirect.net/eric/promptkit/src/tag/v0.14.0/docs/consumers/pkg-promptkit.md)
|
||||
for the pinned protocol and retry details.
|
||||
for defaults and the [PromptKit consumer contract](https://forge.maximumdirect.net/base-layer/promptkit/src/tag/v0.21.0/docs/consumers/pkg-promptkit.md)
|
||||
for protocol and retry details.
|
||||
|
||||
A model operation that exhausts structural repair, a live-parent local operation
|
||||
timeout, capacity rejection, transport failure, malformed provider response,
|
||||
@@ -31,4 +31,4 @@ continue. Caller cancellation, broken assets or configuration, and unknown
|
||||
internal failures remain fatal and cancel peers. Audita decodes into domain
|
||||
types only after PromptKit validates the JSON. Ordinary reports use safe reason
|
||||
codes and counts rather than provider messages. Full request/response content
|
||||
is written only with `--debug`; see [operations](../operations.md#diagnostics-directory).
|
||||
is written only with `--debug`; see [operations](../operations.md#debug-evidence).
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
The corrected transcript uses the selected `output.schema`: `bare-segments`
|
||||
or `audita-v1`. Its schema and text rules are unchanged by resilience
|
||||
reporting. See [configuration](../config.md#settings) for selection and
|
||||
[operations](../operations.md#required-output-files) for placement and writes.
|
||||
[operations](../operations.md#outputs-and-lifecycle) for placement and writes.
|
||||
|
||||
With reporting enabled, Audita produces three ordinary files:
|
||||
|
||||
@@ -46,7 +46,7 @@ remain in opt-in debug files.
|
||||
## Error and Recovery Records
|
||||
|
||||
An `errors` record with `scope: work_unit` identifies zero-based `stage_index`
|
||||
and `chunk_index`, module key/instance, initial segment range, applied-correction
|
||||
and `chunk_index`, optional `stage_id`, module key/instance, initial segment range, applied-correction
|
||||
count, and a `failures` sequence. Each failure identifies the operation
|
||||
(`proposal`, `validation`, or `validation_input`), optional validator and
|
||||
zero-based batch index, safe reason/message, sorted proposal indices when
|
||||
@@ -59,7 +59,7 @@ For `validation_decisions_invalid`, proposal indices identify only candidates
|
||||
whose required decisions were unavailable. Other candidates in the same batch
|
||||
may still be approved or rejected normally.
|
||||
|
||||
Each recovery identifies its work unit and successful operation, with sorted
|
||||
Each recovery identifies its work unit, optional `stage_id`, and successful operation, with sorted
|
||||
`reasons`, a safe message, available execution counts, and sorted
|
||||
`filtered_proposals` details. One recovery can contain `output_repaired`,
|
||||
`service_recovered`, and `proposals_filtered` together. Filtering covers
|
||||
@@ -79,8 +79,8 @@ is an error even when unexpected indices were also ignored.
|
||||
`report.md` includes run ID, status, corrections, work coverage, error and
|
||||
recovery totals, input/output paths, and full work-unit error and recovery
|
||||
matrices. Rows show every initial chunk with 1-based chunk number and initial
|
||||
segment range; columns follow configured module-instance order, including
|
||||
repeats. Completed error cells are `0` or `1`; recovery cells are operation
|
||||
segment range; columns follow configured pipeline order, including repeated modules. Labels show
|
||||
the one-based position and module, plus the operator ID when supplied. Completed error cells are `0` or `1`; recovery cells are operation
|
||||
counts. `—` means not started and `n*` means interrupted with n facts recorded.
|
||||
An empty stage or chunk set is identified as having no work units. Short reason breakdowns count
|
||||
records containing each reason; a recovery can have several reasons, so those
|
||||
@@ -91,10 +91,9 @@ files. It does not include transcript excerpts or correction lists.
|
||||
|
||||
With `--debug`, `correction-ledger.json` contains proposed candidates and
|
||||
their application, rejection, or interruption details. `llm-execution.json`
|
||||
holds logical calls and PromptKit execution events; `utilization-diagnostics.json`
|
||||
groups backend capacity and timing observations while preserving known and
|
||||
complete usage semantics. Interaction request/response artifacts, selected
|
||||
profiles, configuration provenance, and input snapshots are also debug-only.
|
||||
holds logical calls and PromptKit execution events. Interaction request/response
|
||||
artifacts and input snapshots are also debug-only. Records carry zero-based
|
||||
`stage_index` and optional `stage_id` where an entry is identified.
|
||||
These files are diagnostic evidence, not an alternate ordinary report schema.
|
||||
Each candidate-backed ledger entry includes `proposed_corrected_text`, the
|
||||
literal replacement span, even when it is `""` for a deletion. Applied entries
|
||||
|
||||
@@ -6,15 +6,15 @@ The [CLI reference](../cli.md) owns command syntax, streams, and exit semantics;
|
||||
|
||||
## Invocation
|
||||
|
||||
Use an argument array with the transcript path and required glossary argument,
|
||||
following the [process examples](../cli.md#commands-and-help). For machine workflows,
|
||||
Use an argument array with the transcript path and configured glossary selections,
|
||||
following the [process examples](../cli.md#processing). For machine workflows,
|
||||
select an explicit transcript file destination and a configuration path. Reports
|
||||
go beside the transcript unless an explicit `--report-dir` selects another
|
||||
directory. For stdout output, reports use the effective `reporting.stdout_dir`
|
||||
(defaulting to the invocation working directory), including a `--report-dir`
|
||||
override. A config-file or overlay value for `reporting.stdout_dir` alone does
|
||||
not redirect file-output reports. `--work-dir` selects a debug root, while
|
||||
configuration or `--debug` enables capture; the path flag alone does not.
|
||||
`--debug` enables capture; the path flag alone does not.
|
||||
Supplying a configuration path also makes file selection predictable; only
|
||||
explicitly referenced credential variables are read from the environment.
|
||||
|
||||
@@ -26,7 +26,7 @@ or cancellation in the parent process when needed.
|
||||
## Result Handling
|
||||
|
||||
Interpret the exit result according to the
|
||||
[CLI output and exit contract](../cli.md#streams-and-exit-status).
|
||||
[CLI output and exit contract](../cli.md#inspection-and-outcomes).
|
||||
Consume transcript/report data through the
|
||||
[output-file contract](output-files.md), and treat stderr as human-readable detail.
|
||||
The three reports may be absent after early configuration or destination
|
||||
@@ -38,7 +38,7 @@ transcript may have been written before a later required report failure, so
|
||||
the file alone does not prove completion.
|
||||
|
||||
If processing fails, preserve available evidence and follow
|
||||
[operations recovery](../operations.md#failure-inspection-and-recovery).
|
||||
[operations recovery](../operations.md#recovery).
|
||||
Ordinary reports are independent of optional retained debug evidence. Parent
|
||||
cancellation reaches the child through its context; Audita joins workers and
|
||||
attempts failed-status report finalization before exiting. Uncatchable
|
||||
@@ -49,4 +49,4 @@ file nor a debug directory alone establishes successful completion.
|
||||
|
||||
- [Transcript and glossary inputs](transcript-glossary-files.md)
|
||||
- [LLM endpoint compatibility](openai-compatible-llm.md)
|
||||
- [Configuration inspection commands](../cli.md#commands-and-help)
|
||||
- [Configuration inspection commands](../cli.md#inspection-and-outcomes)
|
||||
|
||||
@@ -28,23 +28,11 @@ The [tiny transcript](../../examples/tiny-transcript.json) is a complete copyabl
|
||||
input. Invalid JSON, unsupported top-level shapes, empty transcripts, invalid field
|
||||
types, duplicate supplied IDs, and the validation failures above are rejected.
|
||||
|
||||
## Glossary YAML Contract
|
||||
## Glossary collections
|
||||
|
||||
The top-level object contains a `glossary` list with at least one entry.
|
||||
|
||||
| Entry field | Type and validation |
|
||||
| --- | --- |
|
||||
| `name` | Non-empty string |
|
||||
| `category` | Non-empty string |
|
||||
| `summary` | Non-empty string |
|
||||
| `aliases` | Optional list of non-empty strings |
|
||||
| `plural` | Optional string |
|
||||
|
||||
The parser checks empty strings without trimming them. Unknown YAML fields are
|
||||
ignored by the glossary parser, unlike the stricter
|
||||
[configuration-file schema](../config.md#loading-and-precedence).
|
||||
|
||||
The [tiny glossary](../../examples/tiny-glossary.yaml) is a complete copyable input.
|
||||
Malformed YAML, an empty/missing glossary, invalid shapes, missing required text,
|
||||
and empty aliases are rejected. These formats describe input acceptance, not
|
||||
additional vocabulary restrictions.
|
||||
Glossary definitions are composed as application configuration, including the
|
||||
optional companion inputs described by the [configuration reference](../config.md).
|
||||
The [local example](../../examples/local/glossary.yml) provides a complete
|
||||
collection. Configuration may contain empty collections; processing preflight
|
||||
requires every glossary-correction entry to select at least one glossary entry.
|
||||
The previous standalone `glossary` file format is not a processing input.
|
||||
|
||||
@@ -9,7 +9,7 @@ CLI wiring, and maintained examples. Definitions of behavior belong to the
|
||||
1. Update fields, defaults, loading, and validation in `internal/core/config`.
|
||||
2. Update file, selected overlay, credential reference, and CLI mapping as
|
||||
applicable. Preserve the precedence defined in the
|
||||
[configuration contract](../config.md#loading-and-precedence), including the
|
||||
[configuration contract](../config.md#loading-and-composition), including the
|
||||
distinct validation and effective-config commands.
|
||||
3. Update redaction when secrets are involved.
|
||||
4. Review configuration tests and related CLI tests for precedence, validation,
|
||||
@@ -29,33 +29,18 @@ CLI wiring, and maintained examples. Definitions of behavior belong to the
|
||||
|
||||
The `version` route uses ReleaseKit's public `buildinfo.Version()` accessor and
|
||||
stays independent of application runtime assembly. Preserve its behavior from
|
||||
the [CLI contract](../cli.md#commands-and-help) when changing the command; the
|
||||
the [CLI contract](../cli.md) when changing the command; the
|
||||
adoption decision is recorded in [ADR 0001](../adr/0001-releasekit.md).
|
||||
|
||||
## Validating Examples
|
||||
|
||||
Keep complete copyable files in [examples](../../examples/) and link to them from
|
||||
their canonical contract documentation. Keep them valid and secret-free.
|
||||
From the repository root, validate the maintained configuration examples:
|
||||
|
||||
```sh
|
||||
go run ./cmd/audita config validate --config examples/minimal-config.yml
|
||||
go run ./cmd/audita config validate --config examples/configkit-inheritance.yml
|
||||
AUDITA_SERVICE_KEY=synthetic-example go run ./cmd/audita config validate --config examples/production-config.yml
|
||||
AUDITA_SERVICE_KEY=synthetic-example go run ./cmd/audita config validate --config examples/production-config.yml --overlay testing
|
||||
go run ./cmd/audita config print-effective --config examples/minimal-config.yml --view profiles
|
||||
go run ./cmd/audita config print-effective --config examples/configkit-inheritance.yml --view explain
|
||||
```
|
||||
|
||||
The local and ConfigKit inheritance files select `profiles/local`; the service
|
||||
file selects `profiles/service`, including the service testing profiles. Keep
|
||||
these roots separate because PromptKit validates every final definition in a
|
||||
selected root, including definitions that are not selected for a role.
|
||||
|
||||
Only referenced credentials read environment values; use a controlled synthetic
|
||||
value when validating the service example. Check affected CLI examples against the
|
||||
parser and use schema tests for maintained transcript/glossary inputs. Example
|
||||
validation should not invoke a live LLM merely to check documentation.
|
||||
Run the commands in the [example guide](../../examples/README.md) from each
|
||||
project's directory. `internal/cli/examples_integration_test.go` exercises the
|
||||
maintained projects with a controlled provider, and the preview fixture can be
|
||||
prepared without inference. Review automatic companion sources and overlays in
|
||||
their actual project directories. Keep service credentials synthetic in tests.
|
||||
|
||||
## Runtime Change Checks
|
||||
|
||||
|
||||
@@ -24,36 +24,27 @@ retain their execution counts and never produce a recovery record.
|
||||
`internal/core/diagnostics.OutputSession` owns active destination checks,
|
||||
transcript preflight, report initialization, regular-file replacement, and direct
|
||||
stream publication. Transcript input is protected from early report writes but
|
||||
may be the final transcript destination. The LLM runtime records successful reads
|
||||
through PromptKit's profile filesystem and supplies those source paths to CLI;
|
||||
may be the final transcript destination. The LLM runtime records consumed PromptKit sources and effective host message
|
||||
files and supplies those source paths to CLI;
|
||||
no second filename-based discovery rule is used.
|
||||
CLI composes it after configuration/profile inspection and before reading
|
||||
inputs or dispatching model work. It does not create ordinary report parents.
|
||||
CLI composes it after configuration, transcript acquisition, and preflight
|
||||
readiness and before dispatching model work. It does not create ordinary report parents.
|
||||
On a final write error it attempts other enabled reports and one failed-status
|
||||
pass; a published transcript is never erased by a later report failure.
|
||||
|
||||
Debug is independent. Only `diagnostics.debug` creates a unique retained child
|
||||
Debug is independent. The shared `--debug` flag enables a unique retained child
|
||||
under `diagnostics.work_dir`. CLI passes empty diagnostics paths and nil writers
|
||||
when debug is off. With debug on, source/normalized snapshots, invocation,
|
||||
effective settings, origins, profiles, initial chunk summary, correction ledger,
|
||||
indexed execution/utilization, and model interactions are written. Runner
|
||||
candidate capture includes proposals later rejected or interrupted. Ledger
|
||||
mapping uses optional string pointers so an empty proposed replacement is
|
||||
serialized for every candidate and an empty applied replacement appears only
|
||||
when that candidate applied; see the [public artifact contract](../integrations/output-files.md#debug-only-files).
|
||||
Debug write failures reach the CLI through an explicit concurrency-safe diagnostic
|
||||
callback and produce redacted stderr messages without aborting processing.
|
||||
Completed validator decisions returned with a fatal batch error remain available
|
||||
to the debug ledger; they never authorize applying an incompletely validated edit.
|
||||
Runner decision records carry the validator's explicit approved, rejected, or
|
||||
unavailable disposition into the ledger. Unavailable candidates have a distinct
|
||||
top-level ledger disposition, while fatal interruptions remain interrupted when
|
||||
no rejection or unavailable decision was returned. Successful validation
|
||||
interaction payloads contain the full decoded response, final content, and
|
||||
reconciliation outcomes; ignored indices do not become ledger candidates.
|
||||
Every debug writer uses all configured backend secrets for redaction, including
|
||||
unselected backend credentials and source snapshots. Debug can still contain
|
||||
sensitive transcript and prompt content.
|
||||
when debug is off. With debug enabled, it captures source and normalized
|
||||
snapshots, invocation, effective settings, correction ledger, LLM execution
|
||||
observations, and model interactions. Candidate capture includes proposals later
|
||||
rejected or interrupted. Ledger mapping preserves empty proposed and applied
|
||||
replacements and distinguishes unavailable candidates from interrupted work.
|
||||
|
||||
A concurrency-safe diagnostic callback reports optional write failures without
|
||||
aborting processing. Diagnostic redaction uses credential source metadata; debug
|
||||
artifacts can still contain sensitive transcript and prompt content. Execution
|
||||
position and optional operator ID are carried separately from generated safe
|
||||
artifact path coordinates.
|
||||
|
||||
## Key Tests
|
||||
|
||||
|
||||
@@ -1,60 +1,22 @@
|
||||
# Audita LLM Runtime
|
||||
|
||||
## Client Boundary
|
||||
`llm.NewRuntime` assembles one captured PromptKit engine using its default
|
||||
catalog, Audita's embedded prompts and schemas, and optional operator sources
|
||||
anchored to the recorded application main context. Captured host sources and
|
||||
message files are passed to output collision checks. An embedded-only diagnostic
|
||||
construction distinguishes shipped asset failures from operator-source failures;
|
||||
it never replaces the requested engine.
|
||||
|
||||
`llm.NewRuntime` assembles one PromptKit engine for both roles. PromptKit's
|
||||
catalogs and defaults are followed by Audita's embedded prompt and schema
|
||||
inputs, operator documents from `profiles_dir`, and the highest-priority
|
||||
application backend mapping. Audita owns backend endpoints, capacity settings,
|
||||
and resolved credentials; native PromptKit layering owns profile inheritance,
|
||||
prompt overrides, and provider-specific fields. Imports are disabled. The
|
||||
runtime inspects both selected roles offline, requiring complete model and
|
||||
endpoint values and rejecting competing profile credential lookups. A
|
||||
role-bound `llm.RoleClient` implements `structuredoutput.Client`; proposal and
|
||||
validator contracts alias that interface. Both roles share engine-local backend
|
||||
capacity.
|
||||
Each resolved pipeline entry receives role-bound clients with its selected
|
||||
profiles and effective budgets. Clients share engine-local backend admission.
|
||||
The adapter uses `PrepareExecution` and `RunPrepared` on the same handle, passes
|
||||
Audita's per-call schema and repair budget, and decodes only validated results.
|
||||
PromptKit owns backend and credential selection. The adapter uses captured
|
||||
credential source names for artifact redaction without inspecting unrelated
|
||||
environment values. See the [integration contract](../integrations/openai-compatible-llm.md)
|
||||
for failure behavior and the [configuration reference](../config.md) for sources.
|
||||
|
||||
Successful construction captures the final definitions and referenced message
|
||||
contents. The runtime does not reload operator documents or message files during
|
||||
calls. If final engine construction fails, an embedded-only diagnostic
|
||||
construction distinguishes shipped asset failures from invalid operator input.
|
||||
An asset failure remains an asset error; when embedded construction succeeds,
|
||||
the original failure remains a configuration error. The diagnostic engine
|
||||
never replaces or executes the requested configuration, and caller cancellation
|
||||
skips this path. The captured inspection also supplies an internal source
|
||||
ledger: all loaded host YAML plus effective, source-relative host
|
||||
`content_file` resources are tracked for output-collision checks.
|
||||
|
||||
For each logical call, the adapter starts an operation context and a diagnostic
|
||||
record before preparation. It supplies the reserved hardening text, the selected
|
||||
profile and resolved backend credential, and the complete embedded schema/repair
|
||||
contract. `PrepareExecution` produces the exact messages and hashes retained in
|
||||
an Audita snapshot. The adapter checks the prepared backend and endpoint before
|
||||
`RunPrepared`. It records available final content, usage, model, and profile
|
||||
metadata and decodes only a validated result. Call hooks expose logical
|
||||
start/finish records and PromptKit execution events for debug utilization.
|
||||
|
||||
The [integration contract](../integrations/openai-compatible-llm.md) describes
|
||||
wire ownership and failure behavior. `structuredoutput.Failure` has a safe
|
||||
message, fixed reason code, typed cause, and recoverable/fatal/canceled class.
|
||||
Counts distinguish provider attempts, transient retries, and structural
|
||||
repairs, including on errors. Secrets from all configured backends, including unselected
|
||||
ones, are available to the diagnostics redaction boundary.
|
||||
|
||||
CLI explain output and debug configuration origins share a value-free projection
|
||||
of effective field origins. The [CLI contract](../cli.md) defines its optional
|
||||
inheritance evidence fields.
|
||||
|
||||
## Orchestration
|
||||
|
||||
CLI processing supplies two role clients from the same engine to the runner.
|
||||
PromptKit's per-backend admission applies to both roles and every generation,
|
||||
retry, and repair. The runner still processes module instances in order,
|
||||
collects concurrent section work deterministically, and cancels remaining
|
||||
section work only on fatal failures. `llm.DiagnosticsWriter` remains the protected
|
||||
request/response artifact writer. `llm.ObservationCollector` captures metadata
|
||||
events under a short mutex and freezes them after runner work joins. Its logical
|
||||
records include preparation failures; summary usage comes from terminal events
|
||||
once per call and retains knownness and completeness for each field. Runner wall
|
||||
and validator timings remain separate from provider attempts. Ordinary reports
|
||||
derive compact runner outcomes; full observations are debug-only.
|
||||
The observation collector tracks logical calls and execution events. Ordinary
|
||||
reports use compact safe outcomes; request and response evidence is optional
|
||||
debug material. The runner preserves ordered pipeline traversal and deterministic
|
||||
section collection while canceling and joining peers on fatal errors.
|
||||
|
||||
@@ -88,4 +88,4 @@ outcomes are defined by the [report contract](../integrations/output-files.md).
|
||||
- `internal/modules/*/module_test.go`
|
||||
- `internal/framework/modules/registry_test.go`
|
||||
- `internal/framework/proposal_generation/*_test.go`
|
||||
- `internal/cli/run_test.go` (pipeline/report integration)
|
||||
- `internal/cli/process_ordered_integration_test.go` (pipeline/report integration)
|
||||
|
||||
@@ -12,7 +12,7 @@ selectable values and defaults belong to
|
||||
|
||||
Configuration validation checks registry support. CLI orchestration resolves the
|
||||
selected encoder, passes the final transcript to it, and routes the encoded bytes
|
||||
to the destination defined by the [CLI contract](../cli.md#streams-and-exit-status).
|
||||
to the destination defined by the [CLI contract](../cli.md#inspection-and-outcomes).
|
||||
An unsupported key fails validation or resolution rather than selecting an
|
||||
implicit fallback.
|
||||
|
||||
|
||||
@@ -14,6 +14,7 @@ Executable and command orchestration:
|
||||
|
||||
Core deterministic components:
|
||||
- `internal/core/config`
|
||||
- `internal/core/glossary`
|
||||
- `internal/core/schema`
|
||||
- `internal/core/normalization`
|
||||
- `internal/core/chunking`
|
||||
@@ -24,6 +25,7 @@ Core deterministic components:
|
||||
|
||||
Framework orchestration and contracts:
|
||||
- `internal/framework/contracts`
|
||||
- `internal/framework/preflight`
|
||||
- `internal/framework/modules`
|
||||
- `internal/framework/proposal_generation`
|
||||
- `internal/framework/proposals`
|
||||
@@ -66,7 +68,7 @@ artifact lifecycle.
|
||||
## Component Responsibilities
|
||||
|
||||
Core components implement configuration defaults/loading/validation/redaction,
|
||||
transcript and glossary schemas, normalization and chunking, output encoding,
|
||||
transcript schemas and derived glossary views, normalization and chunking, output encoding,
|
||||
diagnostics artifact helpers, and public report structures.
|
||||
|
||||
Framework components implement module and validator contracts, proposal
|
||||
|
||||
@@ -4,8 +4,9 @@ The [input contract](../integrations/transcript-glossary-files.md) owns input
|
||||
formats; [configuration](../config.md) owns settings; the
|
||||
[output contract](../integrations/output-files.md) owns serialized results.
|
||||
|
||||
1. CLI filesystem adapters read transcript and glossary files. Schema parsing
|
||||
produces source objects; transcript segments enter canonical normalization.
|
||||
1. CLI loads resolved configuration and transcript input. Preflight checks all
|
||||
selected glossary views and roles before the output session prepares resources.
|
||||
Transcript segments then enter canonical normalization.
|
||||
2. Normalization sorts and merges segments using configured time and token
|
||||
limits, reassigning IDs where needed.
|
||||
3. The runner partitions the normalized transcript exactly once before its
|
||||
|
||||
@@ -116,19 +116,20 @@ is covered by [report assembly](diagnostics-reporting.md).
|
||||
- `internal/validators/*_test.go`
|
||||
- `internal/framework/validators/*_test.go`
|
||||
- `internal/framework/processreport/correction_ledger_test.go`
|
||||
- `internal/cli/run_test.go`
|
||||
- `internal/cli/process_ordered_integration_test.go`
|
||||
|
||||
## Supplemental Validation Context
|
||||
|
||||
Request construction prepares one segment-position lookup and protected vocabulary
|
||||
for all candidates. Glossary selection remains candidate-specific, including the
|
||||
proposed replacement text.
|
||||
Request construction prepares one segment-position lookup and a matcher from the
|
||||
selected context entries for all candidates. Context selection remains candidate-specific,
|
||||
including the proposed replacement text. The independent protection view is used
|
||||
by the deterministic guard and application check.
|
||||
|
||||
Each candidate includes the immediately preceding and following source segments
|
||||
available within its fixed chunk, labeled `preceding` or `following` with speaker,
|
||||
text, and categories, plus glossary entries whose names, aliases, or
|
||||
recognized plurals appear in its original/resulting text or those neighbors.
|
||||
Glossary matching reuses the protected vocabulary; entries retain glossary order.
|
||||
Glossary matching uses the same recognition rules; entries retain selection order.
|
||||
Model-facing candidate and neighboring context omit internal segment IDs.
|
||||
This supplies evidence such as a troupe definition for a proposed `Jesters`
|
||||
correction or an adjacent speaker turn explaining deliberate repetition. A correction
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
|
||||
Audita v2 adopts ConfigKit for configuration and PromptKit for LLM execution.
|
||||
Version-1 configuration files, environment overlays, and retired flags are not
|
||||
accepted. Copy a maintained [local](../../examples/minimal-config.yml) or
|
||||
[service](../../examples/production-config.yml) example, then select real models
|
||||
accepted. Copy a maintained [local](https://gitea.maximumdirect.net/eric/audita/src/tag/v3.1.1/examples/minimal-config.yml) or
|
||||
[service](https://gitea.maximumdirect.net/eric/audita/src/tag/v3.1.1/examples/production-config.yml) example, then select real models
|
||||
in the accompanying `examples/profiles/` files. A testing overlay is shown at
|
||||
`examples/overlays/testing.yml`; its model choice carries no cost, speed, or
|
||||
quality guarantee.
|
||||
@@ -122,7 +122,7 @@ capacity and usage fields retain known/complete flags. Debug runs write
|
||||
`effective-config.redacted.yaml`, `selected-profiles.yaml`,
|
||||
`config-origins.yaml`, `utilization-diagnostics.json`, and
|
||||
`llm-execution.json`. Request and response artifacts remain redacted debugging
|
||||
material. See [operations](../operations.md#diagnostics-directory) for placement
|
||||
material. See [operations](../operations.md#debug-evidence) for placement
|
||||
and paths, [configuration](../config.md) for all settings, and the
|
||||
[CLI reference](../cli.md) for current flags.
|
||||
|
||||
|
||||
23
docs/migrations/v3.md
Normal file
23
docs/migrations/v3.md
Normal file
@@ -0,0 +1,23 @@
|
||||
# Migrating to application configuration v3
|
||||
|
||||
Audita v3 is a hard cutover. Move application settings to `version: 3` and use
|
||||
[the configuration reference](../config.md) and [maintained examples](../../examples/)
|
||||
to construct the new files. The previous configuration and standalone glossary
|
||||
formats are not loaded by the new processing path.
|
||||
|
||||
- Replace `pipeline.modules` with ordered `pipeline.stages` records containing
|
||||
`module` and any per-entry overrides. An explicit empty list runs normalization.
|
||||
- Define named `glossary_collections` in configuration or its companion imports;
|
||||
select them using `glossaries` and `protected_terms`. Remove `--glossary` and
|
||||
`--modules` from invocations.
|
||||
- Move profiles and backends out of the application configuration and retired
|
||||
`profiles_dir` layout into PromptKit resources. Use its current list records,
|
||||
IDs, credential sources, and backend admission settings. Select proposal and
|
||||
validation profiles in application `llm` settings.
|
||||
- Use the shared `--debug` flag for capture; `diagnostics.work_dir` remains the
|
||||
parent directory. Review report and output destinations before running.
|
||||
|
||||
Inspect configuration and resources before processing, then try a controlled
|
||||
small transcript. Processing preflight checks every glossary correction and
|
||||
selected role before preparing outputs or invoking a model. See [operations](../operations.md)
|
||||
for lifecycle and recovery.
|
||||
@@ -1,94 +1,58 @@
|
||||
# Audita Operations
|
||||
|
||||
This guide owns physical output placement and failure handling. The
|
||||
[CLI reference](cli.md) owns flags and streams, [configuration](config.md) owns
|
||||
settings, and the [output contract](integrations/output-files.md) owns schemas.
|
||||
This guide owns output placement and failure handling. The [CLI reference](cli.md)
|
||||
owns invocation, [configuration](config.md) owns settings, and the
|
||||
[output contract](integrations/output-files.md) owns serialized formats.
|
||||
|
||||
## Normal Workflow
|
||||
## Processing
|
||||
|
||||
1. Validate a version-2 configuration and its selected profiles with
|
||||
`audita config validate --config config.yml`.
|
||||
2. Prepare a transcript and glossary following the [input contract](integrations/transcript-glossary-files.md).
|
||||
3. Run `audita process input.json --config config.yml --glossary glossary.yaml --output corrected.json`.
|
||||
4. Check the exit status, stderr summary, corrected transcript, and reports.
|
||||
A completed run can contain recoverable model errors and still apply
|
||||
corrections that passed every required validator.
|
||||
Inspect the application and PromptKit resources, prepare a transcript, then run
|
||||
`audita process` with the desired destination. The [examples](../examples/README.md)
|
||||
show complete projects. Processing preflight checks every configured glossary
|
||||
correction and selected profile before output preparation and model work. Empty
|
||||
protection and an empty pipeline are valid. A completed run can contain
|
||||
recoverable model errors while still applying independently validated edits.
|
||||
|
||||
## Required Output Files
|
||||
## Outputs and lifecycle
|
||||
|
||||
Reporting is enabled by default. For file transcript output, `errors.yml`,
|
||||
`recoveries.yml`, and `report.md` go in the transcript output's parent directory
|
||||
unless `--report-dir` is explicitly supplied. For stdout transcript output, they
|
||||
go in the effective `reporting.stdout_dir` (default `.`), including a
|
||||
`--report-dir` override. A config-file or overlay value for
|
||||
`reporting.stdout_dir` alone does not redirect file-output reports. An active
|
||||
selected report directory must be nonblank. `--reporting=false` suppresses all
|
||||
three reports and leaves their destination inactive, so it is not probed merely
|
||||
because `--report-dir` was supplied. It does not suppress the stderr summary or
|
||||
fatal errors. Stdout contains only transcript JSON; file output leaves it empty.
|
||||
Reporting is enabled by default. For transcript file output, `errors.yml`,
|
||||
`recoveries.yml`, and `report.md` are placed beside that file unless `--report-dir`
|
||||
is supplied. For stdout transcript output they use the effective report directory.
|
||||
Disabling reporting leaves its destination inactive and unprobed. Quiet mode
|
||||
requires a transcript file or `/dev/null`; it suppresses optional messages, not
|
||||
failures. Stdout carries transcript JSON when selected.
|
||||
|
||||
Existing report directories and parents for regular transcript files must be
|
||||
writable. Audita prevents report writes from overwriting transcript, glossary,
|
||||
loaded application configuration, host YAML discovered under `profiles_dir`,
|
||||
or host message files referenced by effective PromptKit `content_file` fields,
|
||||
and rejects conflicting output destinations. Source paths come from the
|
||||
runtime's captured configuration; unrelated files in the selected directory
|
||||
remain eligible output destinations. The final transcript may intentionally
|
||||
replace its own input after successful processing; other input/configuration/
|
||||
profile sources remain protected.
|
||||
Preparation never truncates an existing transcript. Device and pipe destinations
|
||||
(such as `/dev/null` and named pipes) receive ordinary writes at publication time;
|
||||
they are not opened during preflight or replaced with regular files. A pipe write
|
||||
can wait for its reader, and stream write errors are fatal. With reporting enabled,
|
||||
reports still use the output path's parent unless an explicit `--report-dir`
|
||||
selects another directory; disable reporting when that placement is unsuitable
|
||||
for a stream destination.
|
||||
Before processing, Audita checks destination collisions against the transcript,
|
||||
consumed application and PromptKit sources, and effective host message files. An
|
||||
intentional in-place transcript replacement is allowed. A regular transcript
|
||||
output is prepared in a sibling temporary file, leaving the final destination
|
||||
untouched until successful publication. Pipes and devices are opened only for
|
||||
delivery; a pipe can wait for its reader. Parent directories for required outputs
|
||||
must already exist.
|
||||
|
||||
With reporting enabled, all three reports are initialized as `in_progress` before
|
||||
input processing. Output is encoded fully before publication. Each regular output
|
||||
file is replaced through a temporary file on its destination filesystem:
|
||||
transcript first, then `errors.yml`,
|
||||
`recoveries.yml`, and `report.md`. This is per-file replacement, not a
|
||||
multi-file transaction. New files honor the process umask; replacing an existing
|
||||
file preserves its permission bits. A late report error may leave the transcript
|
||||
published; the process exits nonzero and attempts failed-status reports. On fatal processing
|
||||
or cancellation, no completed transcript is published, but partial reports are
|
||||
finalized where possible. An uncatchable termination can leave `in_progress`.
|
||||
Enabled reports are initialized as `in_progress` before processing. The
|
||||
transcript is published before final reports; the files do not form a single
|
||||
transaction. A late report failure may therefore leave a published transcript.
|
||||
Fatal errors and cancellation clean unpublished temporary resources and attempt
|
||||
failed-status final reports. An uncatchable termination can leave `in_progress`
|
||||
evidence. Read exit status and reports together rather than treating a file's
|
||||
existence as proof of success.
|
||||
|
||||
## Diagnostics Directory
|
||||
## Debug evidence
|
||||
|
||||
`--debug` or `diagnostics.debug` enables detailed evidence independently of
|
||||
reporting. One directory named `run-<id>` is created under
|
||||
`diagnostics.work_dir` (default `/tmp/audita`), which `--work-dir` can override,
|
||||
and retained on success or failure. Selecting `--work-dir` alone does not enable
|
||||
capture. Without debug, no directory or payload files are created. The parent
|
||||
is created when needed only for debug. Failure to
|
||||
write optional debug evidence prints a redacted stderr diagnostic but does not
|
||||
change correction decisions or the exit status.
|
||||
The shared `--debug` flag enables retained artifacts in a run directory beneath
|
||||
`diagnostics.work_dir`; selecting a work directory alone does not enable capture.
|
||||
Optional debug preparation or writes can produce redacted warnings without
|
||||
changing correction decisions or the required output result. Debug material can
|
||||
include transcript content, prompts, responses, and correction decisions. It is
|
||||
not anonymized; protect it according to the input's sensitivity. Configured
|
||||
credential values are redacted at the diagnostic boundary.
|
||||
|
||||
Debug evidence includes invocation and effective settings, configuration
|
||||
origins, selected profiles, redacted source and normalized snapshots,
|
||||
normalization and initial chunk summaries, all captured proposed candidates
|
||||
and validator decisions in `correction-ledger.json`, execution/utilization
|
||||
records, and model interaction files. Individual artifact writes can fail; a
|
||||
directory's existence does not prove every file was written. These files can
|
||||
contain transcript text and prompt/response payloads. Configured backend
|
||||
credentials are redacted, but the transcript itself is not anonymized. Protect
|
||||
and retain the directory according to the sensitivity of the input.
|
||||
## Recovery
|
||||
|
||||
## Failure Inspection and Recovery
|
||||
|
||||
Exit `0` means processing and every enabled required output completed. Exit
|
||||
`1` means cancellation, fatal processing, or required-output failure. Exit `2`
|
||||
identifies usage or configuration/destination selection errors. Recoverable
|
||||
model-operation failures appear in `errors.yml` as one record per affected
|
||||
work unit; they do not automatically make the run fail. `recoveries.yml`
|
||||
records successful operations that needed retry, repair, or proposal filtering.
|
||||
`report.md` shows every initial chunk and configured module instance, including
|
||||
interrupted and unattempted work. Ordinary reports omit transcript excerpts and
|
||||
full decision ledgers; enable debug when those are needed.
|
||||
|
||||
Read stderr and report status first. If a required output failed after a
|
||||
transcript was published, do not treat the file alone as proof of a completed
|
||||
run. Fix the input, service, or destination problem and rerun. Audita has no
|
||||
resume/checkpoint API.
|
||||
Exit `0` indicates completion of processing and required outputs; nonzero status
|
||||
indicates an input, processing, cancellation, or required-output failure.
|
||||
Recoverable model-operation failures and successful repairs or retries are
|
||||
recorded separately in the ordinary YAML reports. The Markdown matrices identify
|
||||
pipeline position, module, and optional operator ID. Fix the input, service, or
|
||||
destination and rerun when necessary; Audita has no resume or checkpoint API.
|
||||
|
||||
@@ -29,7 +29,7 @@ License and notices: [CC BY-SA 4.0](licenses/CC-BY-SA-4.0.txt) |
|
||||
|
||||
## ConfigKit
|
||||
|
||||
Selected module: `gitea.maximumdirect.net/eric/ConfigKit` at `v0.7.1`.
|
||||
Selected module: `go.maximumdirect.net/configkit` at `v1.2.0`.
|
||||
|
||||
Copyright (C) 2026 Eric Rakestraw
|
||||
|
||||
@@ -37,14 +37,17 @@ SPDX-License-Identifier: LGPL-3.0-or-later
|
||||
|
||||
License and notices: [LGPL v3](licenses/LGPL-3.0.txt) |
|
||||
[GPL v3](licenses/GPL-3.0.txt) |
|
||||
[COPYRIGHT](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/6fffde4af8f31f99d3ee295c426eb3234cb0fb77/COPYRIGHT).
|
||||
[COPYRIGHT](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/COPYRIGHT).
|
||||
|
||||
[Source](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/6fffde4af8f31f99d3ee295c426eb3234cb0fb77).
|
||||
[Source](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e).
|
||||
|
||||
## ReleaseKit
|
||||
|
||||
Selected module: `go.maximumdirect.net/releasekit` at `v0.9.0`.
|
||||
|
||||
Material: ReleaseKit runtime and adapted launcher and tag-delivery recipes in
|
||||
`scripts/release.sh` and `.woodpecker/release.yml`.
|
||||
|
||||
Copyright (C) 2026 Eric Rakestraw
|
||||
|
||||
SPDX-License-Identifier: LGPL-3.0-or-later
|
||||
@@ -55,6 +58,48 @@ License and notices: [LGPL v3](licenses/LGPL-3.0.txt) |
|
||||
|
||||
[Source](https://forge.maximumdirect.net/base-layer/releasekit/src/commit/2180aa868ec2caa95fc8279fd59d5f0ebac41bd4).
|
||||
|
||||
## CommandKit
|
||||
|
||||
Selected module: `go.maximumdirect.net/commandkit` at `v0.6.0`.
|
||||
|
||||
Copyright (C) 2026 Eric Rakestraw
|
||||
|
||||
SPDX-License-Identifier: LGPL-3.0-or-later
|
||||
|
||||
License and notices: [LGPL v3](licenses/LGPL-3.0.txt) |
|
||||
[GPL v3](licenses/GPL-3.0.txt) |
|
||||
[COPYRIGHT](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/1613ff95cab60581de5d86f6e9e2a3cc091a44a0/COPYRIGHT).
|
||||
|
||||
[Source](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/1613ff95cab60581de5d86f6e9e2a3cc091a44a0).
|
||||
|
||||
## CommandKit ckconfig
|
||||
|
||||
Selected module: `go.maximumdirect.net/ckconfig` at `v0.7.0`.
|
||||
|
||||
Copyright (C) 2026 Eric Rakestraw
|
||||
|
||||
SPDX-License-Identifier: LGPL-3.0-or-later
|
||||
|
||||
License and notices: [LGPL v3](licenses/LGPL-3.0.txt) |
|
||||
[GPL v3](licenses/GPL-3.0.txt) |
|
||||
[COPYRIGHT](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/64f6453b8453f914f272d427bb4ae6f59deacd2b/integrations/ckconfig/COPYRIGHT).
|
||||
|
||||
[Source](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/64f6453b8453f914f272d427bb4ae6f59deacd2b).
|
||||
|
||||
## CommandKit ckprompt
|
||||
|
||||
Selected module: `go.maximumdirect.net/ckprompt` at `v0.7.0`.
|
||||
|
||||
Copyright (C) 2026 Eric Rakestraw
|
||||
|
||||
SPDX-License-Identifier: LGPL-3.0-or-later
|
||||
|
||||
License and notices: [LGPL v3](licenses/LGPL-3.0.txt) |
|
||||
[GPL v3](licenses/GPL-3.0.txt) |
|
||||
[COPYRIGHT](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/f369d6a78253b6e6fcb1e023fb2e9c3af1d20ff0/integrations/ckprompt/COPYRIGHT).
|
||||
|
||||
[Source](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/f369d6a78253b6e6fcb1e023fb2e9c3af1d20ff0).
|
||||
|
||||
## Go runtime and standard library
|
||||
|
||||
Selected toolchain: Go 1.27.3 for the distributed Linux binaries.
|
||||
@@ -64,10 +109,10 @@ Copyright 2009 The Go Authors.
|
||||
SPDX-License-Identifier: BSD-3-Clause
|
||||
|
||||
License and notices: [BSD 3-Clause](licenses/BSD-3-Clause-Go.txt) |
|
||||
[LICENSE](https://github.com/golang/go/blob/go1.27.3/LICENSE) |
|
||||
[PATENTS](https://github.com/golang/go/blob/go1.27.3/PATENTS).
|
||||
[LICENSE](https://github.com/golang/go/blob/2aae1000ea1d4b28923b96db28acf4e3b99d6aff/LICENSE) |
|
||||
[PATENTS](https://github.com/golang/go/blob/2aae1000ea1d4b28923b96db28acf4e3b99d6aff/PATENTS).
|
||||
|
||||
[Source](https://github.com/golang/go/commit/go1.27.3).
|
||||
[Source](https://github.com/golang/go/commit/2aae1000ea1d4b28923b96db28acf4e3b99d6aff).
|
||||
|
||||
## Go vendored golang.org/x/crypto
|
||||
|
||||
@@ -78,10 +123,10 @@ Copyright 2009 The Go Authors.
|
||||
SPDX-License-Identifier: BSD-3-Clause
|
||||
|
||||
License and notices: [BSD 3-Clause](licenses/BSD-3-Clause-Go.txt) |
|
||||
[LICENSE](https://github.com/golang/go/blob/go1.27.3/src/vendor/golang.org/x/crypto/LICENSE) |
|
||||
[PATENTS](https://github.com/golang/go/blob/go1.27.3/src/vendor/golang.org/x/crypto/PATENTS).
|
||||
[LICENSE](https://github.com/golang/go/blob/2aae1000ea1d4b28923b96db28acf4e3b99d6aff/src/vendor/golang.org/x/crypto/LICENSE) |
|
||||
[PATENTS](https://github.com/golang/go/blob/2aae1000ea1d4b28923b96db28acf4e3b99d6aff/src/vendor/golang.org/x/crypto/PATENTS).
|
||||
|
||||
[Source](https://github.com/golang/go/commit/go1.27.3).
|
||||
[Source](https://github.com/golang/go/commit/2aae1000ea1d4b28923b96db28acf4e3b99d6aff).
|
||||
|
||||
## Go vendored golang.org/x/net
|
||||
|
||||
@@ -92,10 +137,10 @@ Copyright 2009 The Go Authors.
|
||||
SPDX-License-Identifier: BSD-3-Clause
|
||||
|
||||
License and notices: [BSD 3-Clause](licenses/BSD-3-Clause-Go.txt) |
|
||||
[LICENSE](https://github.com/golang/go/blob/go1.27.3/src/vendor/golang.org/x/net/LICENSE) |
|
||||
[PATENTS](https://github.com/golang/go/blob/go1.27.3/src/vendor/golang.org/x/net/PATENTS).
|
||||
[LICENSE](https://github.com/golang/go/blob/2aae1000ea1d4b28923b96db28acf4e3b99d6aff/src/vendor/golang.org/x/net/LICENSE) |
|
||||
[PATENTS](https://github.com/golang/go/blob/2aae1000ea1d4b28923b96db28acf4e3b99d6aff/src/vendor/golang.org/x/net/PATENTS).
|
||||
|
||||
[Source](https://github.com/golang/go/commit/go1.27.3).
|
||||
[Source](https://github.com/golang/go/commit/2aae1000ea1d4b28923b96db28acf4e3b99d6aff).
|
||||
|
||||
## Go vendored golang.org/x/sys
|
||||
|
||||
@@ -106,10 +151,10 @@ Copyright 2009 The Go Authors.
|
||||
SPDX-License-Identifier: BSD-3-Clause
|
||||
|
||||
License and notices: [BSD 3-Clause](licenses/BSD-3-Clause-Go.txt) |
|
||||
[LICENSE](https://github.com/golang/go/blob/go1.27.3/src/vendor/golang.org/x/sys/LICENSE) |
|
||||
[PATENTS](https://github.com/golang/go/blob/go1.27.3/src/vendor/golang.org/x/sys/PATENTS).
|
||||
[LICENSE](https://github.com/golang/go/blob/2aae1000ea1d4b28923b96db28acf4e3b99d6aff/src/vendor/golang.org/x/sys/LICENSE) |
|
||||
[PATENTS](https://github.com/golang/go/blob/2aae1000ea1d4b28923b96db28acf4e3b99d6aff/src/vendor/golang.org/x/sys/PATENTS).
|
||||
|
||||
[Source](https://github.com/golang/go/commit/go1.27.3).
|
||||
[Source](https://github.com/golang/go/commit/2aae1000ea1d4b28923b96db28acf4e3b99d6aff).
|
||||
|
||||
## Go vendored golang.org/x/text
|
||||
|
||||
@@ -120,10 +165,10 @@ Copyright 2009 The Go Authors.
|
||||
SPDX-License-Identifier: BSD-3-Clause
|
||||
|
||||
License and notices: [BSD 3-Clause](licenses/BSD-3-Clause-Go.txt) |
|
||||
[LICENSE](https://github.com/golang/go/blob/go1.27.3/src/vendor/golang.org/x/text/LICENSE) |
|
||||
[PATENTS](https://github.com/golang/go/blob/go1.27.3/src/vendor/golang.org/x/text/PATENTS).
|
||||
[LICENSE](https://github.com/golang/go/blob/2aae1000ea1d4b28923b96db28acf4e3b99d6aff/src/vendor/golang.org/x/text/LICENSE) |
|
||||
[PATENTS](https://github.com/golang/go/blob/2aae1000ea1d4b28923b96db28acf4e3b99d6aff/src/vendor/golang.org/x/text/PATENTS).
|
||||
|
||||
[Source](https://github.com/golang/go/commit/go1.27.3).
|
||||
[Source](https://github.com/golang/go/commit/2aae1000ea1d4b28923b96db28acf4e3b99d6aff).
|
||||
|
||||
## go-yaml v4
|
||||
|
||||
@@ -169,7 +214,7 @@ License and notices: [Apache 2.0](licenses/Apache-2.0.txt).
|
||||
|
||||
## PromptKit
|
||||
|
||||
Selected module: `gitea.maximumdirect.net/eric/promptkit` at `v0.14.0`.
|
||||
Selected module: `go.maximumdirect.net/promptkit` at `v0.21.0`.
|
||||
|
||||
Copyright (C) 2026 Eric Rakestraw
|
||||
|
||||
@@ -177,37 +222,9 @@ SPDX-License-Identifier: LGPL-3.0-or-later
|
||||
|
||||
License and notices: [LGPL v3](licenses/LGPL-3.0.txt) |
|
||||
[GPL v3](licenses/GPL-3.0.txt) |
|
||||
[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit/src/commit/143c6e9b3a675872117b0471e2ff8bcb25055a54/COPYRIGHT).
|
||||
[COPYRIGHT](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/171fb11f4e967a1da3af57843c6467954b14ba09/COPYRIGHT).
|
||||
|
||||
[Source](https://gitea.maximumdirect.net/eric/promptkit/src/commit/143c6e9b3a675872117b0471e2ff8bcb25055a54).
|
||||
|
||||
## Promptkit OpenRouter Catalog
|
||||
|
||||
Selected module: `gitea.maximumdirect.net/eric/promptkit-backend-openrouter/v3` at `v3.20260930.0`.
|
||||
|
||||
Copyright (C) 2026 Eric Rakestraw
|
||||
|
||||
SPDX-License-Identifier: MIT
|
||||
|
||||
License and notices: [MIT](licenses/MIT-Catalogs.txt) |
|
||||
[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/commit/f17d208cb7ad1fb043cda2266cd3011d11b40aa0/COPYRIGHT) |
|
||||
[LICENSE](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/commit/f17d208cb7ad1fb043cda2266cd3011d11b40aa0/LICENSE).
|
||||
|
||||
[Source](https://gitea.maximumdirect.net/eric/promptkit-backend-openrouter/src/commit/f17d208cb7ad1fb043cda2266cd3011d11b40aa0).
|
||||
|
||||
## Promptkit Rakestrawhome Catalog
|
||||
|
||||
Selected module: `gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/v3` at `v3.20260930.0`.
|
||||
|
||||
Copyright (C) 2026 Eric Rakestraw
|
||||
|
||||
SPDX-License-Identifier: MIT
|
||||
|
||||
License and notices: [MIT](licenses/MIT-Catalogs.txt) |
|
||||
[COPYRIGHT](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/commit/993443d88c451901f4c917fa00c204fd3b54d28d/COPYRIGHT) |
|
||||
[LICENSE](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/commit/993443d88c451901f4c917fa00c204fd3b54d28d/LICENSE).
|
||||
|
||||
[Source](https://gitea.maximumdirect.net/eric/promptkit-backend-rakestrawhome/src/commit/993443d88c451901f4c917fa00c204fd3b54d28d).
|
||||
[Source](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/171fb11f4e967a1da3af57843c6467954b14ba09).
|
||||
|
||||
## yaml.v3
|
||||
|
||||
@@ -229,3 +246,31 @@ License and notices: [Apache 2.0](licenses/Apache-2.0.txt) |
|
||||
[NOTICE](https://github.com/go-yaml/yaml/blob/f6f7691b1fdeb513f56608cd2c32c51f8194bf51/NOTICE).
|
||||
|
||||
[Source](https://github.com/go-yaml/yaml/commit/f6f7691b1fdeb513f56608cd2c32c51f8194bf51).
|
||||
|
||||
## Cobra
|
||||
|
||||
Selected module: `github.com/spf13/cobra` at `v1.10.1`.
|
||||
|
||||
Copyright 2013-2023 The Cobra Authors
|
||||
|
||||
SPDX-License-Identifier: Apache-2.0
|
||||
|
||||
License and notices: [Apache 2.0](licenses/Apache-2.0.txt) |
|
||||
[LICENSE](https://github.com/spf13/cobra/blob/7da941c3547e93b8c9f70bbd3befca79c6335388/LICENSE.txt).
|
||||
|
||||
[Source](https://github.com/spf13/cobra/tree/7da941c3547e93b8c9f70bbd3befca79c6335388).
|
||||
|
||||
## pflag
|
||||
|
||||
Selected module: `github.com/spf13/pflag` at `v1.0.9`.
|
||||
|
||||
Copyright (c) 2012 Alex Ogier. All rights reserved.
|
||||
|
||||
Copyright (c) 2012 The Go Authors. All rights reserved.
|
||||
|
||||
SPDX-License-Identifier: BSD-3-Clause
|
||||
|
||||
License and notices: [BSD 3-Clause](licenses/BSD-3-Clause-Go.txt) |
|
||||
[LICENSE](https://github.com/spf13/pflag/blob/10438578954bba2527fe5cae3684d4532b064bbe/LICENSE).
|
||||
|
||||
[Source](https://github.com/spf13/pflag/tree/10438578954bba2527fe5cae3684d4532b064bbe).
|
||||
|
||||
@@ -1,18 +0,0 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Eric Rakestraw
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and
|
||||
associated documentation files (the "Software"), to deal in the Software without restriction, including
|
||||
without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the
|
||||
following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all copies or substantial
|
||||
portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT
|
||||
LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO
|
||||
EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER
|
||||
IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
|
||||
USE OR OTHER DEALINGS IN THE SOFTWARE.
|
||||
@@ -77,8 +77,8 @@ hidden global state.
|
||||
- Validate corrections before final mutation, including local proposal checks,
|
||||
decision cardinality, and apply-time safety. Provider-side schema enforcement
|
||||
does not replace these checks.
|
||||
- Keep LLM concurrency bounded and context-aware through PromptKit's engine-local
|
||||
per-backend admission. Proposal and validation calls selecting the same backend
|
||||
- Use PromptKit's context-aware, engine-local per-backend admission for
|
||||
LLM concurrency. Proposal and validation calls selecting the same backend
|
||||
share that capacity, including retries and repairs. Preserve deterministic
|
||||
collection order. Recoverable model-operation failures stay local to their
|
||||
proposal response or validation batch; fatal invariant/application failure
|
||||
@@ -120,13 +120,19 @@ responsibilities and should remain separate. Implementation details belong in
|
||||
|
||||
## Dependencies
|
||||
|
||||
Keep Audita dependency-light and prefer the Go standard library for CLI parsing,
|
||||
filesystems, synchronization, and tests. ConfigKit owns configuration loading,
|
||||
presence, provenance, and secret resolution; PromptKit owns prompt/profile
|
||||
loading, OpenAI-compatible transport, retries, repairs, and backend admission.
|
||||
Audita owns domain meaning, role selection, and deterministic processing.
|
||||
Keep Audita dependency-light and prefer the Go standard library where no adopted
|
||||
library owns the boundary. ConfigKit owns configuration composition, presence,
|
||||
recorded resolution, and provenance. PromptKit owns prompt, profile, backend,
|
||||
credential, transport, retry, repair, and admission behavior. CommandKit and its
|
||||
integrations own command routing and their standard inspection and request
|
||||
surfaces. Audita owns domain meaning, role selection, transcript processing,
|
||||
and presentation of its processing results.
|
||||
|
||||
Do not introduce broad CLI, dependency-injection, workflow, logging, or plugin
|
||||
Pass resolved typed configuration to preflight and processing. Preflight checks
|
||||
execution readiness before output preparation or model work; the output session
|
||||
owns resources, cleanup, and publication. The glossary and readiness decision is
|
||||
recorded in [ADR 0002](../adr/0002-glossary-configuration-and-preflight.md).
|
||||
Do not introduce broad dependency-injection, workflow, logging, or plugin
|
||||
frameworks without a concrete implemented need and focused tests.
|
||||
|
||||
## Testing and Documentation
|
||||
|
||||
@@ -184,6 +184,15 @@ only through an explicit project-specific policy decision explaining the
|
||||
concrete risk and why ordinary change review or automation is insufficient.
|
||||
Do not add routine confirmations for checks the command can perform itself.
|
||||
|
||||
## Audita's Selected Candidate Checks
|
||||
|
||||
Audita uses ReleaseKit v0.9.0 with race-enabled offline tests, full Go vet,
|
||||
formatting, Markdown links, module distribution checks, and the required target
|
||||
builds. Race checks address shared processing and client state; vet and format
|
||||
checks catch common defects and mechanical drift; link and module checks protect
|
||||
the distributed source tree. The [runbook](../release.md) specifies the selected
|
||||
toolchain, native commands, and asynchronous recovery.
|
||||
|
||||
## Publication Guards
|
||||
|
||||
Automate the checks needed to publish the intended source under a fresh version:
|
||||
|
||||
@@ -80,7 +80,9 @@ bash scripts/release.sh publish VERSION
|
||||
```
|
||||
|
||||
The command runs the offline Go test suite with race detection, full Go vet,
|
||||
formatting, Markdown link, and module distribution checks. It builds the
|
||||
formatting, Markdown link, and module distribution checks. Race detection covers
|
||||
shared processing and client state; vet and formatting catch common Go defects and
|
||||
mechanical drift; the link and module checks protect the distributed source tree. It builds the
|
||||
profile's Linux amd64 and arm64 targets and checks the native version output. It creates an annotated tag whose
|
||||
message comes from the committed note, then pushes the exact tag object. A
|
||||
successful publication completes the source release without waiting for CI or
|
||||
|
||||
@@ -38,8 +38,8 @@ profile files require migration from the v2 release format.
|
||||
3. Select a configuration root whose final definitions are valid. PromptKit now
|
||||
validates unused definitions too, including backend references, and conflicting
|
||||
values within a directory source group are errors. The maintained examples
|
||||
use separate [local](../../examples/profiles/local/) and
|
||||
[service](../../examples/profiles/service/) roots. Runtime source edits take
|
||||
use separate [local](https://gitea.maximumdirect.net/eric/audita/src/tag/v3.0.0/examples/profiles/local/) and
|
||||
[service](https://gitea.maximumdirect.net/eric/audita/src/tag/v3.0.0/examples/profiles/service/) roots. Runtime source edits take
|
||||
effect on the next invocation.
|
||||
4. Rename any application backend literally named `extends` and update its
|
||||
profile references. ConfigKit now reserves that key for inheritance.
|
||||
@@ -49,9 +49,9 @@ profile files require migration from the v2 release format.
|
||||
|
||||
The [configuration reference](../config.md#promptkit-profiles-and-operator-prompts)
|
||||
contains the migrated format and presence rules. Start from the maintained
|
||||
[local](../../examples/minimal-config.yml),
|
||||
[service](../../examples/production-config.yml), or
|
||||
[application inheritance](../../examples/configkit-inheritance.yml) examples.
|
||||
[local](https://gitea.maximumdirect.net/eric/audita/src/tag/v3.0.0/examples/minimal-config.yml),
|
||||
[service](https://gitea.maximumdirect.net/eric/audita/src/tag/v3.0.0/examples/production-config.yml), or
|
||||
[application inheritance](https://gitea.maximumdirect.net/eric/audita/src/tag/v3.0.0/examples/configkit-inheritance.yml) examples.
|
||||
The [CLI reference](../cli.md) describes report selection, and the
|
||||
[release procedure](../release.md) describes source publication and asynchronous
|
||||
Linux binary delivery.
|
||||
|
||||
@@ -17,7 +17,7 @@ implementation is pending. The
|
||||
The current [CLI](../cli.md) accepts one glossary input for a processing run.
|
||||
The [pipeline configuration](../config.md) supports repeated glossary stages,
|
||||
but does not select different glossary inputs for each occurrence. Glossary
|
||||
files currently have a separate [YAML input contract](../integrations/transcript-glossary-files.md#glossary-yaml-contract).
|
||||
files currently have a separate [YAML input contract](../integrations/transcript-glossary-files.md#glossary-collections).
|
||||
|
||||
Make glossaries and glossary collections reusable, composable configuration
|
||||
assets loaded and composed by ConfigKit. Allow ordered stages to select different resolved
|
||||
|
||||
@@ -573,6 +573,11 @@ Verify existing partial-success and unavailable-candidate evidence survives.
|
||||
|
||||
## Stage 14: Finish the cutover documentation, examples, and attribution
|
||||
|
||||
**Completed.** Canonical guidance, the decision record, migration guide,
|
||||
maintained projects, release procedure, and dependency attribution reflect the
|
||||
cutover. Controlled-provider project checks and documented offline inspections
|
||||
passed; historical example links point to the unchanged v3.0.0 tag.
|
||||
|
||||
Update canonical owners as behavior lands; use this stage to reconcile them.
|
||||
Follow the [documentation policy](../policy/documentation.md) and glossary
|
||||
[durable record](glossary-stages.md#durable-decision-record).
|
||||
|
||||
74
examples/README.md
Normal file
74
examples/README.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# Maintained examples
|
||||
|
||||
Build Audita from the repository root with Go 1.27 or newer. The commands below
|
||||
assume the executable is on `PATH`. Each project selects its main file from its
|
||||
own directory, which also determines companion resources and overlay context.
|
||||
All models, credentials, names, and endpoints are illustrative.
|
||||
|
||||
## Local processing
|
||||
|
||||
From `examples/local`, edit `prompts.yml` to match a reachable OpenAI-compatible
|
||||
local service and its model. Then run:
|
||||
|
||||
```sh
|
||||
audita config validate --config config.yml
|
||||
audita process --config config.yml --output corrected.json transcript.json
|
||||
```
|
||||
|
||||
Processing makes model requests. Reports are written beside the selected output
|
||||
unless disabled. For an offline validator preview, from the same directory:
|
||||
|
||||
```sh
|
||||
audita promptkit show prepared audita/validators.meaning_reversal_review \
|
||||
--config config.yml --profile local \
|
||||
--var "PayloadJSON=$(cat validation-candidate.json)" \
|
||||
--var 'TranscriptDescriptionBlock='
|
||||
```
|
||||
|
||||
This renders the supplied candidate without inference. The native command help
|
||||
and [CLI reference](../docs/cli.md) describe additional inspection and validation.
|
||||
|
||||
## Service and overlay
|
||||
|
||||
From `examples/service`, edit the synthetic endpoint and models in `prompts.yml`.
|
||||
Set `AUDITA_EXAMPLE_KEY` to your service credential before a real request. Inspect
|
||||
without a provider call:
|
||||
|
||||
```sh
|
||||
audita config validate --config config.yml --overlay testing
|
||||
audita promptkit show profile service-test --config config.yml
|
||||
```
|
||||
|
||||
The example does not promise a provider's model availability, price, or quality.
|
||||
|
||||
## Edit a prompt and run again
|
||||
|
||||
From `examples/prompt-edit`, inspect the operator replacement:
|
||||
|
||||
```sh
|
||||
audita promptkit show prompt audita/modules.grammar.proposal --config config.yml
|
||||
```
|
||||
|
||||
Edit its message in `prompts.yml`, repeat the inspection, and run `audita process
|
||||
--config config.yml --output corrected.json transcript.json` against a configured
|
||||
local service. Each invocation captures the files anew.
|
||||
|
||||
## Shared glossary library
|
||||
|
||||
From `examples/campaigns`, `first.yml` and `second.yml` each consume the common
|
||||
`shared.yml` through the companion `glossary.yml`. Inspect each configuration:
|
||||
|
||||
```sh
|
||||
audita config validate --config first.yml
|
||||
audita config validate --config second.yml
|
||||
```
|
||||
|
||||
With a local service configured, process either campaign:
|
||||
|
||||
```sh
|
||||
audita process --config first.yml --output first-output.json transcript.json
|
||||
audita process --config second.yml --output second-output.json transcript.json
|
||||
```
|
||||
|
||||
Edit `prompts.yml` for a local service before processing either campaign. The
|
||||
first uses separate party and spell corrections; the second selects only party.
|
||||
8
examples/campaigns/first.yml
Normal file
8
examples/campaigns/first.yml
Normal file
@@ -0,0 +1,8 @@
|
||||
version: 3
|
||||
llm: {proposal_profile: local}
|
||||
glossaries: [party, spells]
|
||||
pipeline:
|
||||
stages:
|
||||
- {id: names, module: glossary, glossaries: [party]}
|
||||
- {id: magic, module: glossary, glossaries: [spells]}
|
||||
- {module: grammar}
|
||||
4
examples/campaigns/glossary.yml
Normal file
4
examples/campaigns/glossary.yml
Normal file
@@ -0,0 +1,4 @@
|
||||
config:
|
||||
imports:
|
||||
- path: shared.yml
|
||||
required: true
|
||||
5
examples/campaigns/prompts.yml
Normal file
5
examples/campaigns/prompts.yml
Normal file
@@ -0,0 +1,5 @@
|
||||
profiles:
|
||||
- id: local
|
||||
endpoint: http://localhost:11434/v1
|
||||
model: example-model
|
||||
api_key_required: false
|
||||
7
examples/campaigns/second.yml
Normal file
7
examples/campaigns/second.yml
Normal file
@@ -0,0 +1,7 @@
|
||||
version: 3
|
||||
llm: {proposal_profile: local}
|
||||
glossaries: [party]
|
||||
pipeline:
|
||||
stages:
|
||||
- {id: names, module: glossary}
|
||||
- {module: grammar}
|
||||
12
examples/campaigns/shared.yml
Normal file
12
examples/campaigns/shared.yml
Normal file
@@ -0,0 +1,12 @@
|
||||
party:
|
||||
entries:
|
||||
- id: lyra
|
||||
name: Lyra
|
||||
category: character
|
||||
description: A fictional adventurer.
|
||||
spells:
|
||||
entries:
|
||||
- id: shield
|
||||
name: Shield
|
||||
category: spell
|
||||
aliases: [Ward]
|
||||
9
examples/campaigns/transcript.json
Normal file
9
examples/campaigns/transcript.json
Normal file
@@ -0,0 +1,9 @@
|
||||
[
|
||||
{
|
||||
"id": 1,
|
||||
"speaker": "Narrator",
|
||||
"start": 0,
|
||||
"end": 2,
|
||||
"text": "Lyra raises a ward before the journey."
|
||||
}
|
||||
]
|
||||
@@ -1,15 +0,0 @@
|
||||
version: 2
|
||||
profiles_dir: profiles/local
|
||||
backends:
|
||||
local-base:
|
||||
endpoint: http://127.0.0.1:11434/v1
|
||||
concurrency: 4
|
||||
queue_capacity: 0
|
||||
local:
|
||||
extends: local-base
|
||||
concurrency: 2
|
||||
llm:
|
||||
proposal_profile: local-proposal
|
||||
validation_profile: local-validation
|
||||
pipeline:
|
||||
modules: [grammar]
|
||||
8
examples/local/config.yml
Normal file
8
examples/local/config.yml
Normal file
@@ -0,0 +1,8 @@
|
||||
version: 3
|
||||
llm:
|
||||
proposal_profile: local
|
||||
pipeline:
|
||||
stages:
|
||||
- module: glossary
|
||||
- module: grammar
|
||||
glossaries: [party]
|
||||
7
examples/local/glossary.yml
Normal file
7
examples/local/glossary.yml
Normal file
@@ -0,0 +1,7 @@
|
||||
party:
|
||||
entries:
|
||||
- id: jesters
|
||||
name: Jesters
|
||||
category: faction
|
||||
description: A traveling group of performers.
|
||||
aliases: [Jester]
|
||||
5
examples/local/prompts.yml
Normal file
5
examples/local/prompts.yml
Normal file
@@ -0,0 +1,5 @@
|
||||
profiles:
|
||||
- id: local
|
||||
endpoint: http://localhost:11434/v1
|
||||
model: example-model
|
||||
api_key_required: false
|
||||
9
examples/local/transcript.json
Normal file
9
examples/local/transcript.json
Normal file
@@ -0,0 +1,9 @@
|
||||
[
|
||||
{
|
||||
"id": 1,
|
||||
"speaker": "Narrator",
|
||||
"start": 0,
|
||||
"end": 2,
|
||||
"text": "The gestures gather beside the stage."
|
||||
}
|
||||
]
|
||||
1
examples/local/validation-candidate.json
Normal file
1
examples/local/validation-candidate.json
Normal file
@@ -0,0 +1 @@
|
||||
[{"correction_index":1,"original_text":"not open","corrected_text":"open","original_segment_text":"The door is not open.","corrected_segment_text":"The door is open."}]
|
||||
@@ -1,13 +0,0 @@
|
||||
version: 2
|
||||
profiles_dir: profiles/local
|
||||
backends:
|
||||
local:
|
||||
endpoint: http://127.0.0.1:11434/v1
|
||||
concurrency: 1
|
||||
llm:
|
||||
proposal_profile: local-proposal
|
||||
validation_profile: local-validation
|
||||
pipeline:
|
||||
modules: [grammar]
|
||||
output:
|
||||
schema: bare-segments
|
||||
@@ -1,3 +0,0 @@
|
||||
llm:
|
||||
proposal_profile: service-proposal-testing
|
||||
validation_profile: service-validation-testing
|
||||
@@ -1,41 +0,0 @@
|
||||
version: 2
|
||||
profiles_dir: profiles/service
|
||||
backends:
|
||||
service:
|
||||
endpoint: https://openrouter.ai/api/v1
|
||||
credential: AUDITA_SERVICE_KEY
|
||||
concurrency: 2
|
||||
queue_capacity: 1024
|
||||
llm:
|
||||
proposal_profile: service-proposal
|
||||
validation_profile: service-validation
|
||||
transient_retries: 2
|
||||
repair_attempts: 2
|
||||
operation_timeout: 15m
|
||||
pipeline:
|
||||
modules: [glossary, homophones, glossary, spoken_word, grammar]
|
||||
output:
|
||||
schema: audita-v1
|
||||
chunking:
|
||||
# Requires at least this many normalized segments; omit for automatic chunking.
|
||||
target_sections: 8
|
||||
max_section_tokens: 8192
|
||||
min_section_tokens: 2048
|
||||
normalization:
|
||||
max_segment_gap: 4.0
|
||||
ellipsis_gap: 3.5
|
||||
max_segment_duration: 60.0
|
||||
max_segment_tokens: 2048
|
||||
thresholds:
|
||||
glossary: 0.8
|
||||
homophones: 0.8
|
||||
spoken_word: 0.8
|
||||
grammar: 0.8
|
||||
context:
|
||||
description: "General context for domain vocabulary and speaker names."
|
||||
reporting:
|
||||
enabled: true
|
||||
stdout_dir: .
|
||||
diagnostics:
|
||||
debug: false
|
||||
work_dir: /tmp/audita
|
||||
@@ -1,8 +0,0 @@
|
||||
profiles:
|
||||
local-proposal:
|
||||
backend: local
|
||||
model: replace-with-local-model
|
||||
temperature: 0
|
||||
max_tokens: 2048
|
||||
top_p: 1
|
||||
timeout_seconds: 120
|
||||
@@ -1,8 +0,0 @@
|
||||
profiles:
|
||||
local-validation:
|
||||
backend: local
|
||||
model: replace-with-local-model
|
||||
temperature: 0
|
||||
max_tokens: 1024
|
||||
top_p: 1
|
||||
timeout_seconds: 120
|
||||
@@ -1,8 +0,0 @@
|
||||
profiles:
|
||||
service-proposal-testing:
|
||||
backend: service
|
||||
model: replace-with-testing-model
|
||||
temperature: 0
|
||||
max_tokens: 2048
|
||||
top_p: 1
|
||||
timeout_seconds: 120
|
||||
@@ -1,8 +0,0 @@
|
||||
profiles:
|
||||
service-proposal:
|
||||
backend: service
|
||||
model: replace-with-provider-model
|
||||
temperature: 0
|
||||
max_tokens: 2048
|
||||
top_p: 1
|
||||
timeout_seconds: 120
|
||||
@@ -1,8 +0,0 @@
|
||||
profiles:
|
||||
service-validation-testing:
|
||||
backend: service
|
||||
model: replace-with-testing-model
|
||||
temperature: 0
|
||||
max_tokens: 1024
|
||||
top_p: 1
|
||||
timeout_seconds: 120
|
||||
@@ -1,8 +0,0 @@
|
||||
profiles:
|
||||
service-validation:
|
||||
backend: service
|
||||
model: replace-with-provider-model
|
||||
temperature: 0
|
||||
max_tokens: 1024
|
||||
top_p: 1
|
||||
timeout_seconds: 120
|
||||
8
examples/prompt-edit/config.yml
Normal file
8
examples/prompt-edit/config.yml
Normal file
@@ -0,0 +1,8 @@
|
||||
version: 3
|
||||
llm:
|
||||
proposal_profile: local
|
||||
pipeline:
|
||||
stages:
|
||||
- module: glossary
|
||||
- module: grammar
|
||||
glossaries: [party]
|
||||
7
examples/prompt-edit/glossary.yml
Normal file
7
examples/prompt-edit/glossary.yml
Normal file
@@ -0,0 +1,7 @@
|
||||
party:
|
||||
entries:
|
||||
- id: jesters
|
||||
name: Jesters
|
||||
category: faction
|
||||
description: A traveling group of performers.
|
||||
aliases: [Jester]
|
||||
23
examples/prompt-edit/prompts.yml
Normal file
23
examples/prompt-edit/prompts.yml
Normal file
@@ -0,0 +1,23 @@
|
||||
profiles:
|
||||
- id: local
|
||||
endpoint: http://localhost:11434/v1
|
||||
model: example-model
|
||||
api_key_required: false
|
||||
collections:
|
||||
audita:
|
||||
prompts:
|
||||
- id: modules.grammar.proposal
|
||||
messages:
|
||||
- role: system
|
||||
content: Correct only clearly supported grammatical errors.
|
||||
- role: user
|
||||
content: |
|
||||
{{ .ProposalContract }}
|
||||
{{ .HardeningPolicy }}
|
||||
{{ .TranscriptDescriptionBlock }}
|
||||
Glossary: {{ .GlossaryJSON }}
|
||||
Transcript: {{ .SectionJSON }}
|
||||
output:
|
||||
format: json
|
||||
validation_mode: json_schema
|
||||
schema_path: correction_set.v1.json
|
||||
9
examples/prompt-edit/transcript.json
Normal file
9
examples/prompt-edit/transcript.json
Normal file
@@ -0,0 +1,9 @@
|
||||
[
|
||||
{
|
||||
"id": 1,
|
||||
"speaker": "Narrator",
|
||||
"start": 0,
|
||||
"end": 2,
|
||||
"text": "The gestures gather beside the stage."
|
||||
}
|
||||
]
|
||||
7
examples/service/config.yml
Normal file
7
examples/service/config.yml
Normal file
@@ -0,0 +1,7 @@
|
||||
version: 3
|
||||
llm:
|
||||
proposal_profile: service-proposal
|
||||
validation_profile: service-review
|
||||
pipeline:
|
||||
stages:
|
||||
- module: grammar
|
||||
3
examples/service/overlays/testing.yml
Normal file
3
examples/service/overlays/testing.yml
Normal file
@@ -0,0 +1,3 @@
|
||||
llm:
|
||||
proposal_profile: service-test
|
||||
validation_profile: service-test
|
||||
16
examples/service/prompts.yml
Normal file
16
examples/service/prompts.yml
Normal file
@@ -0,0 +1,16 @@
|
||||
backends:
|
||||
- id: service
|
||||
endpoint: https://example.invalid/v1
|
||||
api_key_env: AUDITA_EXAMPLE_KEY
|
||||
concurrency_limit: 2
|
||||
queue_capacity: 8
|
||||
profiles:
|
||||
- id: service-proposal
|
||||
backend: service
|
||||
model: example-proposal-model
|
||||
- id: service-review
|
||||
backend: service
|
||||
model: example-review-model
|
||||
- id: service-test
|
||||
extends: {id: service-proposal}
|
||||
model: example-testing-model
|
||||
9
examples/service/transcript.json
Normal file
9
examples/service/transcript.json
Normal file
@@ -0,0 +1,9 @@
|
||||
[
|
||||
{
|
||||
"id": 1,
|
||||
"speaker": "Narrator",
|
||||
"start": 0,
|
||||
"end": 2,
|
||||
"text": "The gestures gather beside the stage."
|
||||
}
|
||||
]
|
||||
@@ -1,6 +0,0 @@
|
||||
glossary:
|
||||
- name: Audita
|
||||
aliases:
|
||||
- audita
|
||||
category: product
|
||||
summary: The Audita transcript correction CLI.
|
||||
156
internal/cli/examples_integration_test.go
Normal file
156
internal/cli/examples_integration_test.go
Normal file
@@ -0,0 +1,156 @@
|
||||
package cli
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
|
||||
"gitea.maximumdirect.net/eric/audita/v3/internal/core/config"
|
||||
"gitea.maximumdirect.net/eric/audita/v3/internal/core/schema"
|
||||
"gitea.maximumdirect.net/eric/audita/v3/internal/framework/llm"
|
||||
ck "go.maximumdirect.net/configkit"
|
||||
)
|
||||
|
||||
func TestMaintainedProjectsWithControlledProvider(t *testing.T) {
|
||||
_, file, _, ok := runtime.Caller(0)
|
||||
if !ok {
|
||||
t.Fatal("cannot locate examples")
|
||||
}
|
||||
root := filepath.Join(filepath.Dir(file), "..", "..", "examples")
|
||||
t.Setenv("AUDITA_EXAMPLE_KEY", "synthetic-example-key")
|
||||
for _, tc := range []struct{ dir, main, overlay string }{
|
||||
{"local", "config.yml", ""}, {"service", "config.yml", "testing"},
|
||||
{"prompt-edit", "config.yml", ""}, {"campaigns", "first.yml", ""}, {"campaigns", "second.yml", ""},
|
||||
} {
|
||||
t.Run(tc.dir+"/"+tc.main, func(t *testing.T) {
|
||||
main := filepath.Join(root, tc.dir, tc.main)
|
||||
loaded, err := config.Load(ck.Input{Source: ck.File(main)}, tc.overlay)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
loaded.Config.Reporting.Enabled = false
|
||||
var calls atomic.Int32
|
||||
var mu sync.Mutex
|
||||
var models, prompts []string
|
||||
transport := controlledTransport(func(r *http.Request) (*http.Response, error) {
|
||||
calls.Add(1)
|
||||
var request struct {
|
||||
Model string `json:"model"`
|
||||
Messages []struct {
|
||||
Content string `json:"content"`
|
||||
} `json:"messages"`
|
||||
}
|
||||
if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var content strings.Builder
|
||||
for _, message := range request.Messages {
|
||||
content.WriteString(message.Content)
|
||||
}
|
||||
mu.Lock()
|
||||
models = append(models, request.Model)
|
||||
prompts = append(prompts, content.String())
|
||||
mu.Unlock()
|
||||
return &http.Response{StatusCode: 200, Header: make(http.Header), Body: io.NopCloser(strings.NewReader(`{"model":"fixture","choices":[{"message":{"role":"assistant","content":"{\"corrections\":[]}"}}]}`)), Request: r}, nil
|
||||
})
|
||||
engine, err := llm.NewRuntime(context.Background(), loaded, &http.Client{Transport: transport})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tc.dir == "prompt-edit" {
|
||||
prompt, err := engine.Engine.InspectPrompt(context.Background(), "audita/modules.grammar.proposal")
|
||||
if err != nil || !strings.Contains(prompt.Messages[0].Content, "clearly supported grammatical errors") {
|
||||
t.Fatalf("operator prompt: %+v %v", prompt, err)
|
||||
}
|
||||
}
|
||||
raw, err := os.ReadFile(filepath.Join(root, tc.dir, "transcript.json"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
source, err := schema.ParseSourceTranscriptJSON(raw)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
inputs := &processInputs{Configuration: loaded, Runtime: engine, Source: source}
|
||||
prepared, err := prepareProcess(context.Background(), processInput{Transcript: filepath.Join(root, tc.dir, "transcript.json")}, inputs, &bytes.Buffer{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer prepared.Prepared.Output.Close()
|
||||
if _, err := executePipeline(context.Background(), inputs, prepared, sourceToCanonicalTranscript(source), func(error) {}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if int(calls.Load()) != len(loaded.Config.Pipeline.Stages) {
|
||||
t.Fatalf("provider calls=%d", calls.Load())
|
||||
}
|
||||
if tc.dir == "service" && (len(models) != 1 || models[0] != "example-testing-model") {
|
||||
t.Fatalf("overlay models=%v", models)
|
||||
}
|
||||
if tc.dir == "campaigns" && tc.main == "first.yml" {
|
||||
if len(prompts) != 3 || !strings.Contains(prompts[0], `"name": "Lyra"`) || strings.Contains(prompts[0], `"name": "Shield"`) || !strings.Contains(prompts[1], `"name": "Shield"`) || strings.Contains(prompts[1], `"name": "Lyra"`) {
|
||||
t.Fatalf("campaign glossary selections: %v", prompts)
|
||||
}
|
||||
}
|
||||
if tc.dir == "campaigns" && tc.main == "second.yml" {
|
||||
if len(prompts) != 2 || !strings.Contains(prompts[0], `"name": "Lyra"`) || strings.Contains(prompts[0], `"name": "Shield"`) {
|
||||
t.Fatalf("second campaign glossary selection: %v", prompts)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestExamplePromptEditIsCapturedOnNextLoad(t *testing.T) {
|
||||
_, file, _, ok := runtime.Caller(0)
|
||||
if !ok {
|
||||
t.Fatal("cannot locate examples")
|
||||
}
|
||||
source := filepath.Join(filepath.Dir(file), "..", "..", "examples", "prompt-edit")
|
||||
dir := t.TempDir()
|
||||
for _, name := range []string{"config.yml", "glossary.yml", "prompts.yml"} {
|
||||
raw, err := os.ReadFile(filepath.Join(source, name))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(dir, name), raw, 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
inspect := func() string {
|
||||
loaded, err := config.Load(ck.Input{Source: ck.File(filepath.Join(dir, "config.yml"))}, "")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
engine, err := llm.NewRuntime(context.Background(), loaded, nil)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
prompt, err := engine.Engine.InspectPrompt(context.Background(), "audita/modules.grammar.proposal")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return prompt.Messages[0].Content
|
||||
}
|
||||
before := inspect()
|
||||
filePath := filepath.Join(dir, "prompts.yml")
|
||||
raw, err := os.ReadFile(filePath)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filePath, []byte(strings.Replace(string(raw), "clearly supported grammatical errors", "clearly supported punctuation errors", 1)), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
after := inspect()
|
||||
if !strings.Contains(before, "grammatical errors") || !strings.Contains(after, "punctuation errors") {
|
||||
t.Fatalf("before=%q after=%q", before, after)
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user