Document the new configuration and processing workflow
This commit is contained in:
@@ -9,7 +9,7 @@ CLI wiring, and maintained examples. Definitions of behavior belong to the
|
||||
1. Update fields, defaults, loading, and validation in `internal/core/config`.
|
||||
2. Update file, selected overlay, credential reference, and CLI mapping as
|
||||
applicable. Preserve the precedence defined in the
|
||||
[configuration contract](../config.md#loading-and-precedence), including the
|
||||
[configuration contract](../config.md#loading-and-composition), including the
|
||||
distinct validation and effective-config commands.
|
||||
3. Update redaction when secrets are involved.
|
||||
4. Review configuration tests and related CLI tests for precedence, validation,
|
||||
@@ -29,33 +29,18 @@ CLI wiring, and maintained examples. Definitions of behavior belong to the
|
||||
|
||||
The `version` route uses ReleaseKit's public `buildinfo.Version()` accessor and
|
||||
stays independent of application runtime assembly. Preserve its behavior from
|
||||
the [CLI contract](../cli.md#commands-and-help) when changing the command; the
|
||||
the [CLI contract](../cli.md) when changing the command; the
|
||||
adoption decision is recorded in [ADR 0001](../adr/0001-releasekit.md).
|
||||
|
||||
## Validating Examples
|
||||
|
||||
Keep complete copyable files in [examples](../../examples/) and link to them from
|
||||
their canonical contract documentation. Keep them valid and secret-free.
|
||||
From the repository root, validate the maintained configuration examples:
|
||||
|
||||
```sh
|
||||
go run ./cmd/audita config validate --config examples/minimal-config.yml
|
||||
go run ./cmd/audita config validate --config examples/configkit-inheritance.yml
|
||||
AUDITA_SERVICE_KEY=synthetic-example go run ./cmd/audita config validate --config examples/production-config.yml
|
||||
AUDITA_SERVICE_KEY=synthetic-example go run ./cmd/audita config validate --config examples/production-config.yml --overlay testing
|
||||
go run ./cmd/audita config print-effective --config examples/minimal-config.yml --view profiles
|
||||
go run ./cmd/audita config print-effective --config examples/configkit-inheritance.yml --view explain
|
||||
```
|
||||
|
||||
The local and ConfigKit inheritance files select `profiles/local`; the service
|
||||
file selects `profiles/service`, including the service testing profiles. Keep
|
||||
these roots separate because PromptKit validates every final definition in a
|
||||
selected root, including definitions that are not selected for a role.
|
||||
|
||||
Only referenced credentials read environment values; use a controlled synthetic
|
||||
value when validating the service example. Check affected CLI examples against the
|
||||
parser and use schema tests for maintained transcript/glossary inputs. Example
|
||||
validation should not invoke a live LLM merely to check documentation.
|
||||
Run the commands in the [example guide](../../examples/README.md) from each
|
||||
project's directory. `internal/cli/examples_integration_test.go` exercises the
|
||||
maintained projects with a controlled provider, and the preview fixture can be
|
||||
prepared without inference. Review automatic companion sources and overlays in
|
||||
their actual project directories. Keep service credentials synthetic in tests.
|
||||
|
||||
## Runtime Change Checks
|
||||
|
||||
|
||||
@@ -24,36 +24,27 @@ retain their execution counts and never produce a recovery record.
|
||||
`internal/core/diagnostics.OutputSession` owns active destination checks,
|
||||
transcript preflight, report initialization, regular-file replacement, and direct
|
||||
stream publication. Transcript input is protected from early report writes but
|
||||
may be the final transcript destination. The LLM runtime records successful reads
|
||||
through PromptKit's profile filesystem and supplies those source paths to CLI;
|
||||
may be the final transcript destination. The LLM runtime records consumed PromptKit sources and effective host message
|
||||
files and supplies those source paths to CLI;
|
||||
no second filename-based discovery rule is used.
|
||||
CLI composes it after configuration/profile inspection and before reading
|
||||
inputs or dispatching model work. It does not create ordinary report parents.
|
||||
CLI composes it after configuration, transcript acquisition, and preflight
|
||||
readiness and before dispatching model work. It does not create ordinary report parents.
|
||||
On a final write error it attempts other enabled reports and one failed-status
|
||||
pass; a published transcript is never erased by a later report failure.
|
||||
|
||||
Debug is independent. Only `diagnostics.debug` creates a unique retained child
|
||||
Debug is independent. The shared `--debug` flag enables a unique retained child
|
||||
under `diagnostics.work_dir`. CLI passes empty diagnostics paths and nil writers
|
||||
when debug is off. With debug on, source/normalized snapshots, invocation,
|
||||
effective settings, origins, profiles, initial chunk summary, correction ledger,
|
||||
indexed execution/utilization, and model interactions are written. Runner
|
||||
candidate capture includes proposals later rejected or interrupted. Ledger
|
||||
mapping uses optional string pointers so an empty proposed replacement is
|
||||
serialized for every candidate and an empty applied replacement appears only
|
||||
when that candidate applied; see the [public artifact contract](../integrations/output-files.md#debug-only-files).
|
||||
Debug write failures reach the CLI through an explicit concurrency-safe diagnostic
|
||||
callback and produce redacted stderr messages without aborting processing.
|
||||
Completed validator decisions returned with a fatal batch error remain available
|
||||
to the debug ledger; they never authorize applying an incompletely validated edit.
|
||||
Runner decision records carry the validator's explicit approved, rejected, or
|
||||
unavailable disposition into the ledger. Unavailable candidates have a distinct
|
||||
top-level ledger disposition, while fatal interruptions remain interrupted when
|
||||
no rejection or unavailable decision was returned. Successful validation
|
||||
interaction payloads contain the full decoded response, final content, and
|
||||
reconciliation outcomes; ignored indices do not become ledger candidates.
|
||||
Every debug writer uses all configured backend secrets for redaction, including
|
||||
unselected backend credentials and source snapshots. Debug can still contain
|
||||
sensitive transcript and prompt content.
|
||||
when debug is off. With debug enabled, it captures source and normalized
|
||||
snapshots, invocation, effective settings, correction ledger, LLM execution
|
||||
observations, and model interactions. Candidate capture includes proposals later
|
||||
rejected or interrupted. Ledger mapping preserves empty proposed and applied
|
||||
replacements and distinguishes unavailable candidates from interrupted work.
|
||||
|
||||
A concurrency-safe diagnostic callback reports optional write failures without
|
||||
aborting processing. Diagnostic redaction uses credential source metadata; debug
|
||||
artifacts can still contain sensitive transcript and prompt content. Execution
|
||||
position and optional operator ID are carried separately from generated safe
|
||||
artifact path coordinates.
|
||||
|
||||
## Key Tests
|
||||
|
||||
|
||||
@@ -1,60 +1,22 @@
|
||||
# Audita LLM Runtime
|
||||
|
||||
## Client Boundary
|
||||
`llm.NewRuntime` assembles one captured PromptKit engine using its default
|
||||
catalog, Audita's embedded prompts and schemas, and optional operator sources
|
||||
anchored to the recorded application main context. Captured host sources and
|
||||
message files are passed to output collision checks. An embedded-only diagnostic
|
||||
construction distinguishes shipped asset failures from operator-source failures;
|
||||
it never replaces the requested engine.
|
||||
|
||||
`llm.NewRuntime` assembles one PromptKit engine for both roles. PromptKit's
|
||||
catalogs and defaults are followed by Audita's embedded prompt and schema
|
||||
inputs, operator documents from `profiles_dir`, and the highest-priority
|
||||
application backend mapping. Audita owns backend endpoints, capacity settings,
|
||||
and resolved credentials; native PromptKit layering owns profile inheritance,
|
||||
prompt overrides, and provider-specific fields. Imports are disabled. The
|
||||
runtime inspects both selected roles offline, requiring complete model and
|
||||
endpoint values and rejecting competing profile credential lookups. A
|
||||
role-bound `llm.RoleClient` implements `structuredoutput.Client`; proposal and
|
||||
validator contracts alias that interface. Both roles share engine-local backend
|
||||
capacity.
|
||||
Each resolved pipeline entry receives role-bound clients with its selected
|
||||
profiles and effective budgets. Clients share engine-local backend admission.
|
||||
The adapter uses `PrepareExecution` and `RunPrepared` on the same handle, passes
|
||||
Audita's per-call schema and repair budget, and decodes only validated results.
|
||||
PromptKit owns backend and credential selection. The adapter uses captured
|
||||
credential source names for artifact redaction without inspecting unrelated
|
||||
environment values. See the [integration contract](../integrations/openai-compatible-llm.md)
|
||||
for failure behavior and the [configuration reference](../config.md) for sources.
|
||||
|
||||
Successful construction captures the final definitions and referenced message
|
||||
contents. The runtime does not reload operator documents or message files during
|
||||
calls. If final engine construction fails, an embedded-only diagnostic
|
||||
construction distinguishes shipped asset failures from invalid operator input.
|
||||
An asset failure remains an asset error; when embedded construction succeeds,
|
||||
the original failure remains a configuration error. The diagnostic engine
|
||||
never replaces or executes the requested configuration, and caller cancellation
|
||||
skips this path. The captured inspection also supplies an internal source
|
||||
ledger: all loaded host YAML plus effective, source-relative host
|
||||
`content_file` resources are tracked for output-collision checks.
|
||||
|
||||
For each logical call, the adapter starts an operation context and a diagnostic
|
||||
record before preparation. It supplies the reserved hardening text, the selected
|
||||
profile and resolved backend credential, and the complete embedded schema/repair
|
||||
contract. `PrepareExecution` produces the exact messages and hashes retained in
|
||||
an Audita snapshot. The adapter checks the prepared backend and endpoint before
|
||||
`RunPrepared`. It records available final content, usage, model, and profile
|
||||
metadata and decodes only a validated result. Call hooks expose logical
|
||||
start/finish records and PromptKit execution events for debug utilization.
|
||||
|
||||
The [integration contract](../integrations/openai-compatible-llm.md) describes
|
||||
wire ownership and failure behavior. `structuredoutput.Failure` has a safe
|
||||
message, fixed reason code, typed cause, and recoverable/fatal/canceled class.
|
||||
Counts distinguish provider attempts, transient retries, and structural
|
||||
repairs, including on errors. Secrets from all configured backends, including unselected
|
||||
ones, are available to the diagnostics redaction boundary.
|
||||
|
||||
CLI explain output and debug configuration origins share a value-free projection
|
||||
of effective field origins. The [CLI contract](../cli.md) defines its optional
|
||||
inheritance evidence fields.
|
||||
|
||||
## Orchestration
|
||||
|
||||
CLI processing supplies two role clients from the same engine to the runner.
|
||||
PromptKit's per-backend admission applies to both roles and every generation,
|
||||
retry, and repair. The runner still processes module instances in order,
|
||||
collects concurrent section work deterministically, and cancels remaining
|
||||
section work only on fatal failures. `llm.DiagnosticsWriter` remains the protected
|
||||
request/response artifact writer. `llm.ObservationCollector` captures metadata
|
||||
events under a short mutex and freezes them after runner work joins. Its logical
|
||||
records include preparation failures; summary usage comes from terminal events
|
||||
once per call and retains knownness and completeness for each field. Runner wall
|
||||
and validator timings remain separate from provider attempts. Ordinary reports
|
||||
derive compact runner outcomes; full observations are debug-only.
|
||||
The observation collector tracks logical calls and execution events. Ordinary
|
||||
reports use compact safe outcomes; request and response evidence is optional
|
||||
debug material. The runner preserves ordered pipeline traversal and deterministic
|
||||
section collection while canceling and joining peers on fatal errors.
|
||||
|
||||
@@ -88,4 +88,4 @@ outcomes are defined by the [report contract](../integrations/output-files.md).
|
||||
- `internal/modules/*/module_test.go`
|
||||
- `internal/framework/modules/registry_test.go`
|
||||
- `internal/framework/proposal_generation/*_test.go`
|
||||
- `internal/cli/run_test.go` (pipeline/report integration)
|
||||
- `internal/cli/process_ordered_integration_test.go` (pipeline/report integration)
|
||||
|
||||
@@ -12,7 +12,7 @@ selectable values and defaults belong to
|
||||
|
||||
Configuration validation checks registry support. CLI orchestration resolves the
|
||||
selected encoder, passes the final transcript to it, and routes the encoded bytes
|
||||
to the destination defined by the [CLI contract](../cli.md#streams-and-exit-status).
|
||||
to the destination defined by the [CLI contract](../cli.md#inspection-and-outcomes).
|
||||
An unsupported key fails validation or resolution rather than selecting an
|
||||
implicit fallback.
|
||||
|
||||
|
||||
@@ -14,6 +14,7 @@ Executable and command orchestration:
|
||||
|
||||
Core deterministic components:
|
||||
- `internal/core/config`
|
||||
- `internal/core/glossary`
|
||||
- `internal/core/schema`
|
||||
- `internal/core/normalization`
|
||||
- `internal/core/chunking`
|
||||
@@ -24,6 +25,7 @@ Core deterministic components:
|
||||
|
||||
Framework orchestration and contracts:
|
||||
- `internal/framework/contracts`
|
||||
- `internal/framework/preflight`
|
||||
- `internal/framework/modules`
|
||||
- `internal/framework/proposal_generation`
|
||||
- `internal/framework/proposals`
|
||||
@@ -66,7 +68,7 @@ artifact lifecycle.
|
||||
## Component Responsibilities
|
||||
|
||||
Core components implement configuration defaults/loading/validation/redaction,
|
||||
transcript and glossary schemas, normalization and chunking, output encoding,
|
||||
transcript schemas and derived glossary views, normalization and chunking, output encoding,
|
||||
diagnostics artifact helpers, and public report structures.
|
||||
|
||||
Framework components implement module and validator contracts, proposal
|
||||
|
||||
@@ -4,8 +4,9 @@ The [input contract](../integrations/transcript-glossary-files.md) owns input
|
||||
formats; [configuration](../config.md) owns settings; the
|
||||
[output contract](../integrations/output-files.md) owns serialized results.
|
||||
|
||||
1. CLI filesystem adapters read transcript and glossary files. Schema parsing
|
||||
produces source objects; transcript segments enter canonical normalization.
|
||||
1. CLI loads resolved configuration and transcript input. Preflight checks all
|
||||
selected glossary views and roles before the output session prepares resources.
|
||||
Transcript segments then enter canonical normalization.
|
||||
2. Normalization sorts and merges segments using configured time and token
|
||||
limits, reassigning IDs where needed.
|
||||
3. The runner partitions the normalized transcript exactly once before its
|
||||
|
||||
@@ -116,19 +116,20 @@ is covered by [report assembly](diagnostics-reporting.md).
|
||||
- `internal/validators/*_test.go`
|
||||
- `internal/framework/validators/*_test.go`
|
||||
- `internal/framework/processreport/correction_ledger_test.go`
|
||||
- `internal/cli/run_test.go`
|
||||
- `internal/cli/process_ordered_integration_test.go`
|
||||
|
||||
## Supplemental Validation Context
|
||||
|
||||
Request construction prepares one segment-position lookup and protected vocabulary
|
||||
for all candidates. Glossary selection remains candidate-specific, including the
|
||||
proposed replacement text.
|
||||
Request construction prepares one segment-position lookup and a matcher from the
|
||||
selected context entries for all candidates. Context selection remains candidate-specific,
|
||||
including the proposed replacement text. The independent protection view is used
|
||||
by the deterministic guard and application check.
|
||||
|
||||
Each candidate includes the immediately preceding and following source segments
|
||||
available within its fixed chunk, labeled `preceding` or `following` with speaker,
|
||||
text, and categories, plus glossary entries whose names, aliases, or
|
||||
recognized plurals appear in its original/resulting text or those neighbors.
|
||||
Glossary matching reuses the protected vocabulary; entries retain glossary order.
|
||||
Glossary matching uses the same recognition rules; entries retain selection order.
|
||||
Model-facing candidate and neighboring context omit internal segment IDs.
|
||||
This supplies evidence such as a troupe definition for a proposed `Jesters`
|
||||
correction or an adjacent speaker turn explaining deliberate repetition. A correction
|
||||
|
||||
Reference in New Issue
Block a user