Files
audita/docs/roadmap/glossary-stages.md

329 lines
19 KiB
Markdown
Raw Normal View History

# Stage-Specific Glossary Inputs
**Updated:** 2026-11-08
**Status:** Accepted scope and configuration model for the next Audita major
release. Remaining design decisions and implementation are pending. The
[master roadmap](library-adoption.md) owns sequencing and release completion.
## Baseline and Target
The current [CLI](../cli.md) accepts one glossary input for a processing run.
The [pipeline configuration](../config.md) supports repeated glossary stages,
but does not select a different glossary input for each occurrence.
Allow operators to run ordered glossary stages against different glossary inputs
within one processing run. For a D&D transcript, the first glossary stage can
use `party.yml` and the second can use `spells.yml`. Each stage uses its selected
glossary and processes the transcript resulting from earlier stages. Operators
need not combine those inputs into one glossary or run Audita separately for
each pass.
The major-release policy permits a hard cutover; the current single-glossary CLI
and pipeline representation do not constrain the target design.
The final representation belongs to the agreed
[version-3 application schema](configkit.md#application-schema-version--resolved),
with one hard cutover covering library adoption and this feature.
## Root Defaults and Stage Overrides — Resolved
Define global stage settings at the configuration root. Represent the pipeline
as an ordered `pipeline.stages` list of objects, each selecting a `module` and
optionally overriding stage settings. Use typed configuration fields, not a
separate variable-substitution language. The selected glossary configuration is:
```yaml
version: 3
glossary: party.yml
pipeline:
stages:
- module: glossary
- module: glossary
glossary: spells.yml
- module: grammar
```
Resolve every stage independently against the global settings. Omitted stage
settings inherit their global values; explicit stage values override them.
Preserve meaningful false, zero, and empty values under each setting's validation
rules rather than treating them as omission. A stage override does not alter
the root settings or the defaults seen by later stages.
In this example, the first glossary stage uses `party.yml`, the second uses
`spells.yml`, and grammar inherits `party.yml` wherever it or its validators
consume the glossary. Transcript changes carry forward between stages;
configuration overrides do not. A stage's module and associated validators
receive its effective stage settings consistently. Whether protected vocabulary
needs a broader selection remains an open domain decision below.
Apply this model to settings governing stage processing. Invocation-wide
concerns such as configuration discovery, output destinations, and reporting
mode remain global. The selected scope inventory follows below.
Preserve stage order, repeated modules, and explicit empty-pipeline behavior.
Use native ConfigKit loading, presence, and provenance facilities; Audita owns
the meaning of stage scope and passes the effective settings to its consumers.
## Configuration Scope — Resolved
Settings governing an individual stage's requests, acceptance, or model-operation
budget support root defaults and stage overrides. Settings governing preprocessing,
the invocation's shared resources, or its final outputs remain global. Classify
by the setting's meaning and consumer lifecycle, not by where current code stores
it. The names below identify current settings; version-3 field representation
must preserve these scope decisions.
| Setting | Scope and rationale |
| --- | --- |
| Glossary input | Stage: vocabulary supplied to the module and its validators. |
| Applicable confidence threshold | Stage: acceptance of that stage's proposals. |
| Proposal and validation profile selection | Stage: models and generation settings appropriate to each role. |
| `context.description` | Stage: prompt context shared by proposals and validators. |
| `validation.max_prompt_tokens` | Stage: batching budget for that stage's validators. |
| `llm.operation_timeout` | Stage: deadline for each logical model operation. |
| `llm.repair_attempts` | Stage: structural-repair budget for its requests. |
| `llm.transient_retries` | Stage: native transient-retry budget for its requests. |
| All `normalization.*` settings | Global: establish transcript segments before stage execution. |
| All `chunking.*` settings | Global: establish common section boundaries and identities across stages. |
| Transcript input, output destination, and `output.schema` | Global: input and final product of the whole run. |
| Reporting, debug capture, and artifact directories | Global: invocation reporting and retained evidence. |
| Configuration discovery, imports, and resource-source locations | Global: construct the configuration and catalog shared by stages. |
| Backend definitions, credentials, concurrency, and queue capacity | Global native PromptKit resources and shared backend admission. |
| Pipeline ordering and application schema version | Global: define the overall processing configuration. |
Stages may select different profiles and thereby different configured models or
backends. Model parameters remain in native PromptKit profiles rather than a
duplicated Audita settings system. Apply stage request budgets through native
request controls and operation contexts while retaining the shared engine and
backend admission. Do not rebuild or mutate shared backend configuration for
each stage. Per-stage validation batching does not change transcript chunking;
per-stage rechunking would be a separate feature decision.
### Validation-Profile Fallback — Resolved
Combine root and stage selections, including explicit clearing, before deriving
the validation-profile fallback. If no validation selection remains, use the
stage's effective proposal profile. An explicit root validation profile is
inherited unless the stage replaces or clears it. Preserve presence so a value
previously derived at the root is not mistaken for an explicit choice.
| Root selection | Stage selection | Effective validation profile |
| --- | --- | --- |
| Proposal A; validation omitted | Proposal B; validation omitted | B |
| Proposal A; validation V | Proposal B; validation omitted | V |
| Proposal A; validation V | Proposal B; validation null | B |
| Proposal A; validation null | Proposal B; validation omitted | B |
| Proposal A; validation V or omitted | Validation W | W |
| Proposal A; validation omitted | Both omitted | A |
Clearing the validation-profile selection does not disable validation. This rule
does not invent a proposal selection when none is available; a stage requiring
model execution must have a valid effective proposal profile.
### Compound Values and Explicit Clearing — Resolved
Resolve eligible settings field by field. A partial settings group overrides
only its supplied children; an empty mapping such as `llm: {}` contributes no
overrides. For example, a stage setting `llm.repair_attempts: 0` inherits the
root proposal profile and operation timeout while explicitly selecting no repairs.
Omitted fields inherit root settings and then receive any applicable fallback.
False, zero, and empty values remain explicit and follow per-field validation:
zero repairs is valid, while a zero operation timeout is invalid.
| Explicit input | Meaning |
| --- | --- |
| `context.description: ""` | Clear the inherited description. |
| `glossary: null` | Clear the inherited glossary selection. |
| `glossary: ""` | Error: an empty string is not a valid glossary path. |
| `llm.validation_profile: null` | Clear the inherited selection, then use the effective proposal profile. |
| `llm.validation_profile: ""` | Error: use null to clear the selection deliberately. |
| Empty or null proposal profile | Invalid for a stage requiring model execution. |
| `llm: null` | Error: a settings group cannot be cleared as a whole. |
Declare genuinely optional selectors nullable and preserve omission versus
explicit null through resolution. Null is not a universal reset operation;
settings groups are not nullable. A glossary-correction stage requires a glossary
and rejects a missing selection before execution. Other stages may operate
without glossary context where their operation supports it. The separate
protected-vocabulary decision below still governs its consumers.
Lists replace the entire inherited list where list-valued stage settings apply;
`[]` explicitly selects an empty list. Do not append or union implicitly.
Preserve the owning library's semantics for library-owned resources and
identified collections. These scope rules do not introduce a custom merge
language or change ConfigKit source composition.
### CLI Overrides and Stage Precedence — Resolved
Ordinary CLI setting flags override the corresponding root settings through
ConfigKit's native CLI override API. Resolve stage inheritance afterward:
omitted stage fields inherit the effective root values, while explicit stage
values and clearing remain authoritative for that stage. Apply validation-profile
fallback after combining these selections. This preserves deliberate differences
between stages while allowing invocation-specific changes to their defaults.
For the pipeline example above, `--glossary revised-party.yml` changes the
effective glossary for the first glossary stage and the grammar stage to
`revised-party.yml`; the second glossary stage still uses `spells.yml`. A stage
with `glossary: null` remains cleared. Apply this precedence consistently to all
stage-capable settings. Submit only explicitly supplied CLI values to ConfigKit;
preserve false, zero, and empty values under their per-field rules rather than
allowing parser defaults to override configuration.
Keep CLI functionality within ConfigKit's native capabilities. ConfigKit v1.1.0
does not support sequence-index paths in its CLI override API, so targeted-stage
CLI overrides are not part of this plan. Operators supply stage-specific values
in configuration files. Do not implement a parallel override system, custom
merge machinery, or a workaround for addressing stage-list entries. Any future
targeted-stage CLI support should depend on suitable native ConfigKit support.
The open stage-identity decision concerns reporting and diagnostics; stage IDs
do not imply CLI targeting. Audita's domain-specific inheritance and fallback
rules remain within ConfigKit's supported application-resolution lifecycle.
### ConfigKit Support
Use the selected ConfigKit v1.1.0 public lifecycle for this work. Its
[typed decoding contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9/docs/config.md#defaults-and-typed-decoding)
already preserves explicit false, zero, empty, and permitted null values,
recurses through partial mappings, and replaces ordinary sequences. Represent
nullable selectors with nullable typed fields and settings groups with
non-nullable structs. Use native decoding and validation rather than a parallel
parser or generic deep-merge implementation.
Root-to-stage inheritance remains an Audita application rule: ConfigKit does not
automatically copy root fields into new sequence elements. Implement that rule
in `Definition.Resolve`, using `Resolution.IsExplicit` for presence and
`Resolution.EntryInfo` for the fixed pre-resolution metadata. Use
`configkit.Default` for omitted stage fields and record root-field dependencies.
Do not treat a derived root validation fallback as an explicit selection.
An explicitly null validation selection needs deliberate application fallback:
`configkit.Default` correctly leaves explicit null alone. Apply the chosen
fallback and use `Resolution.RecordDerivedDefault` to record the assignment and
its effective proposal-profile dependency, retaining the original explicitness
and history. Follow the native
[recorded-resolution contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/191015e289d7f2b91abb9fdd9c4f2f5cf75b4de9/docs/config.md#recorded-application-resolution)
for all derived mutations, and use `Definition.Validate` for domain checks on
effective settings. Keep stage values independent when copying compound values.
No upstream feature request is presently needed for these rules.
### Durable Decision Record
Completion requires recording the scope model in durable documentation before
retiring this roadmap. Put the enduring classification rule and its rationale
in the [architecture policy](../policy/architecture.md); keep the implemented
per-field scope, compound-value, clearing, precedence, and fallback contracts in the
[configuration reference](../config.md).
These configuration design decisions do not require an ADR. Reserve ADRs for
major architectural decisions under the
[documentation policy](../policy/documentation.md#architecture-decision-records).
The roadmap continues to own implementation status until the feature lands.
Update the [configuration change guide](../internal/cli-config.md) to require
classifying each new setting by consumer lifecycle, specifying its inheritance
and presence behavior, and documenting its scope in the configuration reference.
New settings should follow the established stage/global distinction; deliberate
exceptions need a domain rationale. Link to the durable policy rather than copy
the field inventory into multiple developer documents. This documentation work
is part of feature acceptance, not an optional follow-up.
## Sequencing and Work Outline
Implement after ConfigKit, PromptKit, CommandKit, and ReleaseKit adoption, and
before final release validation and preparation. This lets the feature use the
adopted configuration and CLI conventions and avoids adapting it twice. Keep the
requirement visible during library design without expanding those migrations
into feature implementation.
1. Apply the selected scope, compound-value, clearing, CLI precedence, and
validation-profile fallback rules. Finalize path resolution. Incorporate
these into the version-3 schema rather than retain the old module-name list.
2. Resolve effective stage settings and carry them through stage execution,
proposal generation, and validation. Carry selected glossary inputs through
loading and settle the protected-vocabulary behavior for all consumers.
3. Make repeated stages and their selected inputs understandable in inspection,
diagnostics, and reports. Account for all consumed glossary files in input
handling and output-collision protection.
4. Verify the selected behavior and update configuration, CLI, input contracts,
pipeline documentation, maintained examples, and hard-cutover guidance as the
implementation lands. Complete the durable decision record and future-setting
classification guidance above.
## Acceptance Direction
Use distinct party and spell fixtures with controlled model responses to show
that each stage receives the intended glossary, that one stage's input does not
leak into another, and that the second stage sees the first stage's accepted
transcript changes. Verify meaningful errors for invalid selections or inputs,
and retain the selected domain validation and partial-success behavior. Exact
test cases follow the design; live provider calls are unnecessary.
Verify root fallback, per-stage override isolation, and explicit-value presence
for the selected stage settings. Include the grammar stage after the overridden
glossary stage to establish that it inherits the root glossary. Verify that each
stage's module and validators receive consistent effective settings, including
their recorded source origins. Cover repeated modules and an empty stage list.
Exercise the selected request and validation overrides through stage consumers,
including independent budgets without changing shared backend admission or the
initial transcript sections. Cover all validation-profile fallback cases above
and preserve the distinction between explicit and derived values in inspection.
Cover partial and empty settings groups, explicit zero and false, nullable
selectors, empty-list replacement, and invalid empty strings or group nulls.
Verify that clearing a validation selection follows the effective proposal
profile without disabling validation, and that clearing a required glossary
fails before execution. Check recorded dependencies and retained clear-input
history as well as effective values. Verify that explicit CLI flags change root
defaults and inherited stage values while preserving explicit stage settings
and clearing. Cover profile fallback after CLI assignment, explicit false/zero/
empty CLI values where valid, and the absence of overrides from parser defaults.
Confirm that the durable policy, configuration contracts, and contributor
guidance are in place before declaring this feature complete.
## Open Decisions
The recommendations below are proposals, not selected contracts.
### Relative Paths and Source Context
**Recommendation:** use ConfigKit's supported source context to resolve a
configuration path relative to the document supplying that value, and resolve
CLI paths relative to the working directory. A stage inheriting a root glossary
also inherits that value's source context. This supports reusable imported
configuration; verify the native facilities before selecting implementation.
**Alternative:** resolve all relative glossary paths from the working directory.
This has one simple base and suits invocation-local inputs, but makes reusable
configuration dependent on where Audita is launched. Decide the base explicitly
for no-main-file operation as well.
### Glossary Selection and Protected Vocabulary
**Recommendation:** start from each stage's effective glossary for its module
and validators, with no implicit carryover or union of other stage inputs. This
matches the selected scope model and keeps each stage understandable in isolation.
Review whether protecting terms corrected by earlier stages requires a separate,
explicit protected-vocabulary selection; overlapping terms need deliberate rules.
**Alternative:** protect vocabulary from all configured glossary inputs while
using the effective stage glossary for proposal generation. This could preserve
party names during later spell or grammar work, but makes stage behavior depend
on other stages' inputs and requires conflict and overlap semantics. Ordinary
input selection alone does not decide this protection policy.
### Stage Identity and Attribution
**Recommendation:** support optional operator-chosen stage IDs alongside execution
position and module identity. Use them for readable reports and diagnostics
when supplied, and show each stage's effective inputs and source origins. This
helps distinguish repeated modules and survives reordering, but introduces ID
validation rules that must be specified. CLI targeting is outside this decision
under the resolved precedence and native-support boundary above.
**Alternative:** identify stages only by position and module. This avoids another
configuration field, but positional references in reports and diagnostics change
when stages are reordered. In either case, retain ordered
execution and repeated modules rather than treating the pipeline as a set.
The capability and its inclusion in the next major release are settled. These
questions refine the design; they do not reopen whether to deliver the feature.