audita/docs/roadmap/glossary-stages.md

Stage-Specific Glossary Inputs

Updated: 2026-11-08

Status: Accepted scope and configuration model for the next Audita major release. Remaining design decisions and implementation are pending. The master roadmap owns sequencing and release completion.

Baseline and Target

The current CLI accepts one glossary input for a processing run. The pipeline configuration supports repeated glossary stages, but does not select a different glossary input for each occurrence.

Allow operators to run ordered glossary stages against different glossary inputs within one processing run. For a D&D transcript, the first glossary stage can use party.yml and the second can use spells.yml. Each stage uses its selected glossary and processes the transcript resulting from earlier stages. Operators need not combine those inputs into one glossary or run Audita separately for each pass.

The major-release policy permits a hard cutover; the current single-glossary CLI and pipeline representation do not constrain the target design. The final representation belongs to the agreed version-3 application schema, with one hard cutover covering library adoption and this feature.

Root Defaults and Stage Overrides — Resolved

Define global stage settings at the configuration root. Represent the pipeline as an ordered pipeline.stages list of objects, each selecting a module and optionally overriding stage settings. Use typed configuration fields, not a separate variable-substitution language. The selected glossary configuration is:

version: 3
glossary: party.yml
pipeline:
  stages:
    - module: glossary
    - module: glossary
      glossary: spells.yml
    - module: grammar

Resolve every stage independently against the global settings. Omitted stage settings inherit their global values; explicit stage values override them. Preserve meaningful false, zero, and empty values under each setting's validation rules rather than treating them as omission. A stage override does not alter the root settings or the defaults seen by later stages.

In this example, the first glossary stage uses party.yml, the second uses spells.yml, and grammar inherits party.yml wherever it or its validators consume the glossary. Transcript changes carry forward between stages; configuration overrides do not. A stage's module and associated validators receive its effective stage settings consistently. Whether protected vocabulary needs a broader selection remains an open domain decision below.

Apply this model to settings governing stage processing. Invocation-wide concerns such as configuration discovery, output destinations, and reporting mode remain global. The exact set of stage settings is still to be selected. Preserve stage order, repeated modules, and explicit empty-pipeline behavior. Use native ConfigKit loading, presence, and provenance facilities; Audita owns the meaning of stage scope and passes the effective settings to its consumers.

Sequencing and Work Outline

Implement after ConfigKit, PromptKit, CommandKit, and ReleaseKit adoption, and before final release validation and preparation. This lets the feature use the adopted configuration and CLI conventions and avoids adapting it twice. Keep the requirement visible during library design without expanding those migrations into feature implementation.

1. Finalize the stage-setting inventory, compound-value and clearing rules, path resolution, and CLI precedence under the selected scope model. Incorporate these into the version-3 schema rather than retain the old module-name list.
2. Resolve effective stage settings and carry them through stage execution, proposal generation, and validation. Carry selected glossary inputs through loading and settle the protected-vocabulary behavior for all consumers.
3. Make repeated stages and their selected inputs understandable in inspection, diagnostics, and reports. Account for all consumed glossary files in input handling and output-collision protection.
4. Verify the selected behavior and update configuration, CLI, input contracts, pipeline documentation, maintained examples, and hard-cutover guidance as the implementation lands.

Acceptance Direction

Use distinct party and spell fixtures with controlled model responses to show that each stage receives the intended glossary, that one stage's input does not leak into another, and that the second stage sees the first stage's accepted transcript changes. Verify meaningful errors for invalid selections or inputs, and retain the selected domain validation and partial-success behavior. Exact test cases follow the design; live provider calls are unnecessary.

Verify root fallback, per-stage override isolation, and explicit-value presence for the selected stage settings. Include the grammar stage after the overridden glossary stage to establish that it inherits the root glossary. Verify that each stage's module and validators receive consistent effective settings, including their recorded source origins. Cover repeated modules and an empty stage list.

Open Decisions

The recommendations below are proposals, not selected contracts.

Settings Eligible for Stage Scope

Recommendation: include glossary input, applicable confidence thresholds, proposal/validation profile selection, and other settings that govern an individual stage's work. Inventory current settings by the lifecycle of their consumer before finalizing the schema. Keep discovery, transcript destinations, reporting, and other invocation-wide settings global. This provides useful flexibility without implying that a stage can reconfigure the entire run.

Alternative: initially expose only glossary overrides and add other stage settings later. This reduces the first implementation's scope, but leaves an incomplete application of the selected model and may require further schema changes for settings already known to vary by stage.

Compound Values and Explicit Clearing

Recommendation: resolve eligible settings by field, preserving explicit presence and following the owning library's semantics for compound values. Define omission, null, and empty behavior for each selected field; for glossary input, distinguish inheriting a default from an intentional absence if that is supported. Reuse native mechanisms rather than introducing a custom generic deep-merge engine. This keeps partial stage settings useful and defaults visible.

Alternative: replace an entire settings group whenever that group appears on a stage. This gives simple replacement semantics, but changing one value may require repeating unrelated global settings. The choice must respect upstream contracts for any library-owned resource rather than redefine their merging.

CLI Overrides and Stage Precedence

Recommendation: an ordinary setting flag changes the root default; explicit stage settings remain more specific. Add a supported targeted selection for changing one stage when needed, coordinated with stage identity below. This preserves the meaning of scope, but a global flag will not force the same value onto every stage. Preserve explicit false, zero, and empty CLI values.

Alternative: apply ordinary CLI setting flags to every stage after resolving stage settings. This makes a CLI override uniformly authoritative, but can erase deliberate stage differences, including different glossary files. Choose and document the rule before wiring flags; do not infer it from parser order.

Relative Paths and Source Context

Recommendation: use ConfigKit's supported source context to resolve a configuration path relative to the document supplying that value, and resolve CLI paths relative to the working directory. A stage inheriting a root glossary also inherits that value's source context. This supports reusable imported configuration; verify the native facilities before selecting implementation.

Alternative: resolve all relative glossary paths from the working directory. This has one simple base and suits invocation-local inputs, but makes reusable configuration dependent on where Audita is launched. Decide the base explicitly for no-main-file operation as well.

Glossary Selection and Protected Vocabulary

Recommendation: start from each stage's effective glossary for its module and validators, with no implicit carryover or union of other stage inputs. This matches the selected scope model and keeps each stage understandable in isolation. Review whether protecting terms corrected by earlier stages requires a separate, explicit protected-vocabulary selection; overlapping terms need deliberate rules.

Alternative: protect vocabulary from all configured glossary inputs while using the effective stage glossary for proposal generation. This could preserve party names during later spell or grammar work, but makes stage behavior depend on other stages' inputs and requires conflict and overlap semantics. Ordinary input selection alone does not decide this protection policy.

Stage Identity and Attribution

Recommendation: support optional operator-chosen stage IDs alongside execution position and module identity. Use them for readable reports and targeted overrides when supplied, and show each stage's effective inputs and source origins. This helps distinguish repeated modules and survives reordering, but introduces ID validation and selection rules that must be specified.

Alternative: identify stages only by position and module. This avoids another configuration field, but positional references change when stages are reordered and are less convenient for targeted overrides. In either case, retain ordered execution and repeated modules rather than treating the pipeline as a set.

The capability and its inclusion in the next major release are settled. These questions refine the design; they do not reopen whether to deliver the feature.