Files
audita/docs/roadmap/glossary-stages.md

963 lines
56 KiB
Markdown
Raw Normal View History

# Stage-Specific Glossary Inputs
**Updated:** 2026-11-10
**Status:** Accepted feature scope, ownership, and stage-selection semantics for
the next Audita major release, including named collections and recursive import
placement, optional default sources, collection-scoped entry IDs, extensible
entry content fields, and optional stage IDs. Active glossary vocabulary takes
precedence over overlapping protection. Empty collections are permitted, with
Audita preflight rejecting empty glossary stages. Design decisions are resolved;
implementation is pending. The
2026-11-09 16:03:16 +00:00
[master roadmap](library-adoption.md) owns release completion; the
[implementation plan](implementation.md) owns sequencing.
## Baseline and Target
The current [CLI](../cli.md) accepts one glossary input for a processing run.
The [pipeline configuration](../config.md) supports repeated glossary stages,
but does not select different glossary inputs for each occurrence. Glossary
files currently have a separate [YAML input contract](../integrations/transcript-glossary-files.md#glossary-yaml-contract).
Make glossaries and glossary collections reusable, composable configuration
assets loaded and composed by ConfigKit. Allow ordered stages to select different resolved
vocabulary within one processing run: for example, a focused party pass, a
focused spell pass, and grammar cleanup using both. Each stage processes the
transcript resulting from earlier stages. Operators need not maintain one
manually combined glossary or run Audita separately for each pass.
The major-release policy permits a hard cutover. The current single-glossary
CLI, standalone parser, and file schema do not constrain the target design.
Finalize the glossary representation with the agreed
[version-3 application schema](configkit.md#application-schema-version--resolved)
so operators migrate once. Collection placement and named selection are selected;
default import sources, their tier, entry identity, and content fields are also
selected. Runtime ordering, deduplication, and active-vocabulary exemptions are
selected below; other validation business logic is retained. List clearing and
the configuration handoff boundary are resolved, as are optional stage identity
and attribution. Empty collections and preflight readiness checks are resolved
below.
## Glossary Configuration Ownership — Resolved
Glossaries are configuration. ConfigKit owns their source loading, parsing,
composition, inheritance, collection operations, strict typed decoding, and
configuration provenance. Audita supplies the Go schema, collection identity
registrations where applicable, and configuration-contract validation through
supported public APIs. Use native imports and layering rather than an Audita YAML
loader or private glossary merge engine.
ConfigKit owns the entire configuration-resolution lifecycle up to handoff,
including explicit CLI assignment, presence tracking, defaults, and stage
inheritance. Audita supplies its domain-specific resolution rules through
ConfigKit's supported application lifecycle. Complete those rules before handing
off fully resolved configuration; do not add a second resolution pass in Audita's
processing code.
Before transcript processing, hand off fully resolved, typed Go glossary
configuration and stage selections to Audita. Schema and configuration-consistency
checks use the supported ConfigKit validation lifecycle. Audita then runs an
explicit preflight phase for execution-readiness business logic, including the
nonempty glossary-stage check below. A valid configuration does not by itself
establish that a processing run has useful work. Audita's processing code must not
reopen glossary documents or parse arbitrary YAML after this handoff. Remove the separate
glossary decoding/composition path when the new contract lands.
All configuration-source path and resolution behavior remains upstream-owned,
including glossary documents, imports, overlays, and source resource contexts.
Retain native source attribution through application resolution. A glossary
filename is a configuration source, not an application-data file for a second
post-handoff loader. Choose the source arrangement through ConfigKit's public
composition APIs; a string in an application field does not automatically cause
ConfigKit to load another document. Use the selected import placement and
application-supplied underlay below.
### Named Collections and Recursive Imports — Resolved
Define `glossary_collections` as a typed mapping of collection names to collection
objects. Each object contains an `entries` collection. Root and stage `glossaries`
and `protected_terms` are ordered lists selecting those names, not filenames.
An omitted stage selection inherits the root selection; an explicit list replaces
it, and `[]` selects none. Report unknown collection names during configuration
validation. Selection order determines runtime traversal, not source precedence.
Mount glossary inputs at `glossary_collections`, with each imported file
declaring its collection name. The default underlay below supplies native file
and recursive directory imports, keeping the main configuration focused on
selection and stages:
```yaml
version: 3
glossaries: [party, spells]
pipeline:
stages:
- module: glossary
glossaries: [party]
- module: glossary
glossaries: [spells]
- module: grammar
```
An imported `party.yml` declares the collection directly, without repeating the
application's `glossary_collections` wrapper:
```yaml
party:
entries:
- id: merrin
name: Merrin
category: character
description: The party's wizard.
player: Alex
class: wizard
level: 5
```
The entry follows the [selected content contract](#entry-content-fields--resolved).
Its additional campaign fields remain available for other consumers.
Keep collection identity in document keys. ConfigKit does not infer keys from
filenames or subdirectories; all matches of the declaration receive the same
destination. Moving a file within the imported tree therefore need not change
stage selections. A file can define multiple collections, and multiple sources
can contribute to one collection under native composition and identity rules.
Repeated membership and runtime entry ordering follow the resolved runtime
vocabulary rules below.
Use native mapping placement, discovery, composition, and provenance. Do not add
a directory walker, filename-to-key conversion, or YAML preprocessing in Audita.
Imports in the same tier are strict peers; their discovery order does not create
an override priority. Intentional overrides use upstream layering semantics.
### Entry Identity — Resolved
Represent each collection's `entries` as an ordered identified list. Each entry
has an explicit, stable, nonempty string `id`, unique within that collection.
Its configuration identity is the pair of collection name and entry ID. Keep
identity independent of the displayed term, category, and descriptive text so
those fields can change without changing the record being addressed.
Register `id` through ConfigKit's application sequence-identity API at every
`glossary_collections.<collection>.entries` destination, using a native mapping
wildcard registration. Authors do not need to repeat identity declarations in
every file. ConfigKit owns duplicate-identity checks, inheritance lookup, and
cross-layer matching, including final typed-result identity validation. Audita's
domain validation requires nonempty string IDs; comparisons use ConfigKit's
native exact matching, without case folding or a custom normalization scheme.
Recommend readable IDs such as `shield` or `mordenkainens-magnificent-mansion`;
do not require generated UUIDs or impose a slug syntax. The same ID may occur in
different collections and identifies distinct configuration records there. Two
entries with different IDs may also share a displayed name. Neither matching
IDs across collections nor matching text implicitly establishes semantic
identity. Runtime protection may still deduplicate their recognized spellings.
A higher-priority definition targets an existing record by retaining its ID in
the same collection. Apply native collection operations: duplicate identities
among strict peers fail, while upsert at a precedence boundary replaces the
whole matching record. Do not treat an identified list as a mapping that merges
individual fields of matching records. The selected ordinary stage-selection
lists remain separate from these definition collections.
### Entry Content Fields — Resolved
Require `id`, `name`, and `category` as nonempty strings. The collection-scoped
identity contract above governs `id`; `name` is the canonical term, and `category`
is free-form text rather than an Audita-defined enumeration.
Support these optional fields:
| Field | Contract |
| --- | --- |
| `description` | String providing contextual guidance; omission or an empty string requires no filler text. |
| `aliases` | List of nonempty strings representing alternate names or spellings. |
| `plural` | Nonempty string when supplied, providing an explicit plural form. |
Allow and preserve additional ordinary entry fields, including nested mappings,
lists, and scalar values. A campaign's party roster can therefore also carry
player names, classes, levels, affiliations, or data for other applications.
Preservation applies to the effective result of native composition; it does not
change whole-record replacement semantics.
Use typed Go fields for the Audita contract and ConfigKit's native inline
string-keyed map of dynamic values for unmatched fields. Its
[schema binding rules](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/config.md#advanced-type-and-field-rules)
cover decoding and inspection without a custom decoder. Keep additional data in
the resolved Go configuration and native inspection. ConfigKit still owns its
reserved syntax and supported value shapes; extensibility does not mean
accepting arbitrary YAML. Audita validates its known fields without imposing
campaign-specific rules on additional fields. A misspelled optional field may
therefore be retained as additional data; do not add a separate typo-detection
mechanism.
Build the rich LLM projection deliberately from Audita's known glossary fields,
including `description` when present. Do not automatically forward additional
fields to the LLM. Derive protected vocabulary from names, aliases, and plurals;
unrelated metadata does not contribute vocabulary. These consumer projections
leave the complete resolved entry intact.
### Default Sources and Explicit Configuration — Resolved
Audita supplies one glossary underlay with two optional native imports:
```yaml
config:
imports:
- path: glossary.yml
required: false
destination: [glossary_collections]
- dir: glossary
required: false
recursive: true
pattern: ["*.yml", "*.yaml"]
destination: [glossary_collections]
```
Both locations follow the selected main application configuration file's
resolved directory under the shared [resource-base rule](configkit.md#companion-resource-base-and-overlays--resolved).
Supply the declarations in a non-file underlay with imports enabled and
`MainRelative: configkit.MainRelative()`. ConfigKit binds the context during the
application load. With no usable main context, skip these optional imports and
retain independent defaults; do not supply a working-directory fallback. Keep
embedded contexts independent and do not add a local discovery pass or walker.
Load both sources when present; this is composition, not first-match discovery.
Missing optional sources contribute nothing. Present sources retain ConfigKit's
normal parse, access, and composition errors; absence does not remove the domain
requirement for a glossary-correction stage to select usable vocabulary.
The file and directory contributions compose as strict peers in the same
underlay tier. Neither overrides the other by declaration or discovery order.
Main configuration and its imports have higher priority and can supplement or
replace definitions according to native mapping and collection policies. An
identified-list upsert replaces the whole matched record; details depend on the
selected entry identity and native collection policy.
Operators can put small inline collections in `glossary.yml`, organize larger
libraries below `glossary/`, or use both. Nested imports retain native relative
path and destination inheritance. A nested import from mounted `glossary.yml`
inherits the `glossary_collections` destination unless it explicitly extends it.
Do not repeat that destination unnecessarily in a manifest.
With both defaults enabled, use the file for inline collections or imports of
other sources, rather than a manifest that also imports the files discovered
under `glossary/`. Repeated inclusion within the tier retains ConfigKit's native
error; Audita does not silently deduplicate the import graph.
Explicit main-file imports supplement the underlay's sources; they do not cancel
its import declarations. In particular, main-file `config.imports: []` does not
disable default loading. An underlay does not set a universal destination for
unrelated explicit imports. Changing or disabling default discovery, if added
later, would require a load-time choice before ConfigKit processes the underlay;
ordinary resolved application values cannot retroactively change those imports.
### Shared Libraries and Campaign Vocabulary — Resolved
As a campaign maintainer, I can compose a reusable, versioned D&D glossary
library with campaign-specific vocabulary without editing the shared definitions.
I can select different combinations for focused correction passes while
protecting the selected vocabulary throughout the pipeline.
Use stable, descriptive collection names independent of filenames. For example:
| Shared collections | Campaign collections |
| --- | --- |
| `dnd5e.spells` | `campaign.party` |
| `forgotten_realms.places` | `campaign.npcs` |
| `dnd5e.creatures` | `campaign.places` |
These are literal mapping keys. Dots are a naming convention, not nested lookup
or another reference language; this convention does not restrict other valid
collection names. A shared document declares the same key wherever it is used.
One practical project layout is:
```text
campaign/
audita.yml
glossary.yml
glossary/
shared/
dnd5e/
spells.yml
creatures.yml
forgotten-realms/
places.yml
campaign/
party.yml
npcs.yml
places.yml
```
Run from the campaign directory to use the default sources. `glossary.yml` can
contain small inline collections; the default recursive directory import discovers
the shared and campaign subtrees through ConfigKit. A vendored library release
or pinned checkout inside the configuration tree lets each
campaign adopt shared updates deliberately and independently. Library distribution
and revision management remain outside Audita; it consumes local configuration
sources through native imports. This layout fits the native import-root model
without adding cross-directory discovery conventions.
Importing collections makes them available; selections determine their use.
For example, the campaign can select the following vocabulary and focused passes:
```yaml
glossaries:
- dnd5e.spells
- forgotten_realms.places
- campaign.party
- campaign.npcs
- campaign.places
pipeline:
stages:
- module: glossary
glossaries: [campaign.party, campaign.npcs]
- module: glossary
glossaries: [dnd5e.spells]
- module: glossary
glossaries: [forgotten_realms.places, campaign.places]
- module: grammar
```
With `protected_terms` omitted, root protection derives from that selection.
Each glossary stage exempts its active vocabulary; grammar protects the complete
selection. An available but unselected collection such as `dnd5e.creatures` does
not automatically enter prompt inputs or protected vocabulary. Imported sources
still undergo normal ConfigKit composition and validation.
Add new campaign vocabulary in campaign collections and select it alongside the
shared collections. Intentional adaptations of shared definitions use native
ConfigKit inheritance or explicit layering, without changing the shared source.
Use the selected collection-scoped entry identity when adapting definitions;
directory order must never silently choose which definition wins.
### Derived Runtime Representations — Resolved
Keep the resolved configuration intact. Audita derives separate runtime views
for its consumers, preserving traceability to configured entries and selections:
| Consumer | Derived representation |
| --- | --- |
| Glossary module LLM | Rich entries projected from known glossary fields, including names, categories, descriptions, aliases, and plurals when present. |
| Protected-term validator | Deduplicated recognized vocabulary prepared for deterministic matching. |
| Glossary-stage protection | Selected protected vocabulary minus the active glossary vocabulary. |
| Diagnostics and inspection | Resolved definitions and selections with their configuration origins, distinguished from derived runtime views. |
ConfigKit's identity and composition rules govern which configuration records
coexist or replace one another. Runtime vocabulary deduplication governs whether
multiple valid records contribute the same recognized spelling. Do not use the
latter to suppress configuration conflicts or rewrite the resolved definitions.
Rich prompt data need not lose distinct definitions merely because their text
matches in the protection view. Audita owns matching, transcript-specific
validation, and the selected stage-specific transformations. Apply the runtime
vocabulary rules below, including the active-vocabulary exemption, while retaining
other validation business logic.
### Runtime Vocabulary and Ordering — Resolved
Retain distinct resolved entries in rich LLM input, even when names or other
content match. Two entries named Shield may describe a spell and an item; both
can supply useful context. Matching text does not merge configuration identities.
Include each selected entry once, identified by `(collection name, id)`. Traverse
collections in selection order and entries in their resolved ConfigKit order.
Keep the first occurrence when selection repeats an entry. Separate records
remain separate even when their content is identical. These rules govern runtime
projections without rewriting the configured selections or definitions.
Deduplicate recognized spellings in protected vocabulary using the existing
matcher rules. Repeated membership must not multiply protection requirements.
Retain the association between each spelling and all contributing entries so
vocabulary deduplication does not discard identity information.
For a glossary stage, active glossary vocabulary takes precedence over protection
from every selected collection. Remove each active recognized spelling from the
stage's effective protected vocabulary even when a distinct, non-active entry
also contributes that spelling. Apply the existing matching rules to names,
aliases, and supported plurals. The glossary stage must be free to manipulate its
active vocabulary without protected-term rules blocking those changes.
For example, if the active spell collection and a protected item collection both
contain Shield, that spelling is exempt during the spell pass. Other protected
spellings remain protected; sharing one spelling does not exempt every spelling
of the non-active entry. Configured entries and their associations remain intact.
The exemption is local to that stage, and other validators continue to apply.
Preserve other existing matching and identity-dependent validation business
logic, including protected-name repetition handling. Integrate the reduced
vocabulary without reintroducing protection for exempt active spellings through
another contributing entry or a downstream protected-term check.
## Empty Collections and Audita Preflight — Resolved
Permit glossary collections with no entries. They remain valid configuration
assets, whether imported for reuse, inspected, or selected alongside populated
collections. A nonempty list of selected collection names does not establish
that a stage has entries to process.
Use an explicit Audita preflight phase after ConfigKit hands off the fully
resolved configuration and before pipeline execution begins. Preflight evaluates
execution-readiness business rules against that resolved data. It does not load
sources, resolve configuration, restore defaults, or change stage inheritance.
Keep the nonempty glossary-stage check in Audita preflight rather than ConfigKit's
configuration-validation callbacks.
2026-11-09 16:03:16 +00:00
The shared `internal/framework/preflight` package owns this phase.
Accept fully resolved configuration and the domain inputs needed for
readiness checks through ordinary Go functions and concrete inputs. Return
errors identifying the affected stage and reason; CLI presentation remains with
the caller. Keep the package small and independent of CLI and provider transport.
It must not reload or modify configuration or make provider calls. ConfigKit
retains configuration resolution, and correction validators retain evaluation of
proposed transcript changes. Future execution-readiness checks belong here when
they fit this responsibility; no generic check-registration framework is needed.
Complete invocation and domain-readiness checks before preparing output
resources, then begin execution after required preparation succeeds. Preflight
coordinates preparation through the output-session component, which owns file
handles, cleanup, and final publication. The CommandKit outline owns the
[output preparation contract](commandkit.md#output-resource-preparation-during-preflight--resolved)
and [output-failure rules](commandkit.md#output-failure-behavior--resolved).
This extends readiness preparation after handoff without adding configuration
resolution or duplicating filesystem machinery in the glossary checks.
Preflight checks every configured glossary-correction stage. Its combined selected
collections must supply at least one entry. Empty selections, null selections,
and nonempty selections whose collections collectively contain no entries fail
this check. A mixture of empty and populated collections passes when the combined
input contains an entry. An empty protection selection remains valid and supplies
no protected vocabulary.
Complete preflight for the entire pipeline before executing any stage or making
processing model calls, so an empty later glossary stage cannot fail only after
earlier stages have incurred work. Reject it before output resource preparation.
Report a clear readiness error identifying the stage by position, module, and
optional ID, with its selected collections.
Do not silently skip the stage. This is a processing preflight phase, not another
operator-configured pipeline module; ordinary configuration/resource inspection
can still inspect valid empty collections.
## Root Defaults and Stage Overrides — Resolved
Define global stage defaults at the configuration root. Represent the pipeline
as an ordered `pipeline.stages` list of objects, each selecting a `module` and
optionally overriding stage settings. Resolve every stage independently against
the root settings. Omitted stage settings inherit; explicit stage values override.
Preserve meaningful false, zero, and empty values under each setting's validation
rules rather than treating them as omission. A stage override does not alter
root settings or defaults seen by later stages.
`glossaries` and `protected_terms` select named definitions from the resolved
`glossary_collections` mapping. Keep file organization in the import declarations
and collection names in stage settings. This is domain selection over typed data,
not a general variable-substitution language.
For the party/spells use case, the root selects both collections, the first
stage selects party vocabulary, the second selects spell vocabulary, and grammar
inherits both. All stages inherit the root protection selection by default;
glossary stages exempt their active vocabulary as described below. Transcript
changes carry forward; configuration overrides do not. Each consumer receives
the appropriate view of its stage's effective settings.
Invocation-wide concerns such as configuration discovery, output destinations,
and reporting mode remain global. Preserve stage order, repeated modules, and
explicit empty-pipeline behavior. Use native ConfigKit presence and provenance;
Audita owns the domain meaning of stage scope.
### Stage Identity and Attribution — Resolved
Support an optional operator-chosen `id` on each stage for reports and diagnostics:
```yaml
pipeline:
stages:
- id: party-pass
module: glossary
glossaries: [party]
- id: spell-pass
module: glossary
glossaries: [spells]
- module: grammar
```
When supplied, an ID must be a nonempty string unique within the pipeline.
Compare IDs exactly and case-sensitively; no slug syntax or generated identifier
is required. Unnamed stages remain valid. Validate supplied IDs through ConfigKit's
supported lifecycle before handoff so runtime consumers receive resolved IDs
without generating or resolving configuration themselves.
Retain execution position and module identity alongside the optional ID. Reports
and diagnostics use the supplied ID when present and otherwise identify the stage
by position and module. An operator-chosen ID remains meaningful when stages are
reordered; position continues to describe execution order within a run. Preserve
ordered execution and repeated modules.
Inspection exposes effective glossary and protection selections and their
ConfigKit source origins for both named and unnamed stages. Stage naming does
not replace configuration provenance. These IDs serve reporting and diagnostics;
they do not introduce targeted-stage CLI overrides or change pipeline composition
semantics.
## Configuration Scope — Resolved
Settings governing an individual stage's requests, acceptance, or model-operation
budget support root defaults and stage overrides. Settings governing preprocessing,
the invocation's shared resources, or its final outputs remain global. Classify
by the setting's meaning and consumer lifecycle, not by where current code stores
it. The inventory below records the selected scopes; version-3 field
representation must preserve these decisions.
| Setting | Scope and rationale |
| --- | --- |
| `glossaries` selection | Stage: resolved glossary collections supplying correction and context vocabulary. |
| `protected_terms` selection | Stage: independently selected glossary collections supplying protected vocabulary; defaults from root `glossaries` before stage inheritance. |
| Glossary definitions and collection composition | Global: typed definitions composed through ConfigKit; runtime views are derived per consumer. |
| Applicable confidence threshold | Stage: acceptance of that stage's proposals. |
| Proposal and validation profile selection | Stage: models and generation settings appropriate to each role. |
| `context.description` | Stage: prompt context shared by proposals and validators. |
| `validation.max_prompt_tokens` | Stage: batching budget for that stage's validators. |
| `llm.operation_timeout` | Stage: deadline for each logical model operation. |
| `llm.repair_attempts` | Stage: structural-repair budget for its requests. |
| `llm.transient_retries` | Stage: native transient-retry budget for its requests. |
| All `normalization.*` settings | Global: establish transcript segments before stage execution. |
| All `chunking.*` settings | Global: establish common section boundaries and identities across stages. |
| Transcript input, output destination, and `output.schema` | Global: input and final product of the whole run. |
| Reporting, debug capture, and artifact directories | Global: invocation reporting and retained evidence. |
| Configuration discovery, imports, and resource-source locations | Global: construct the configuration and catalog shared by stages. |
| Backend definitions, credentials, concurrency, and queue capacity | Global native PromptKit resources and shared backend admission. |
| Pipeline ordering and application schema version | Global: define the overall processing configuration. |
Stages may select different profiles and thereby different configured models or
backends. Model parameters remain in native PromptKit profiles rather than a
duplicated Audita settings system. Apply stage request budgets through native
request controls and operation contexts while retaining the shared engine and
backend admission. Do not rebuild or mutate shared backend configuration for
each stage. Per-stage validation batching does not change transcript chunking;
per-stage rechunking would be a separate feature decision.
### Glossary Selection and Protected Vocabulary — Resolved
Select correction/context vocabulary and protection vocabulary independently
at root and stage scopes. A glossary stage without an override uses the entire
root glossary selection; an explicit stage selection supports a focused pass.
An explicit protection selection is complete, not an additive set of extra
vocabulary. Do not implicitly union it with correction inputs or collections
selected by other stages.
The named mapping, ordered name selections, entry identity, repeated membership,
and rich-input order are selected above. Configuration composition
must follow ConfigKit's contracts rather than a local rule to concatenate
independently parsed files.
Resolve these settings through ConfigKit before deriving runtime views:
1. Load and compose glossary configuration and apply explicit CLI assignments
through ConfigKit.
2. If root `protected_terms` is omitted, derive it from a copy of the effective
root `glossaries`. Preserve any explicit root protection selection, including
`[]` or `null`.
3. Independently inherit the effective root `glossaries` and `protected_terms`
into each stage wherever that stage omits the corresponding setting. A stage
override replaces only its own selected list; explicit `[]` or `null` suppresses
inheritance for that setting.
4. After the configuration handoff, derive runtime views. For a glossary module
stage, construct its effective protected vocabulary by subtracting its active
glossary terms from its selected protected terms.
Non-glossary stages use their selected protection vocabulary without this
subtraction.
Protection therefore defaults at the root before stage inheritance. It does not
fall back to a focused stage's glossary list. In the party/spells scenario, no
operator protection configuration is needed:
| Stage | Glossary input | Protection selection | Effective protected vocabulary |
| --- | --- | --- | --- |
| Party glossary pass | Party | Party + spells | Spells |
| Spell glossary pass | Spells | Party + spells | Party |
| Grammar | Party + spells | Party + spells | Party + spells |
The table assumes distinct party and spell vocabulary. The subtraction applies
after resolving both stage selections, including explicit protection overrides.
It operates on recognized terms using the protection matcher's rules, including
names, aliases, and supported plural forms, rather than subtracting configuration
source files or collection names. Active spellings are exempt even when they
also belong to non-active protected entries, as resolved in the runtime vocabulary
contract above. Explicit protection selections do not override this exemption.
Keep the configured protection selection intact and construct the reduced
vocabulary separately for that stage. Do not mutate root settings, loaded
glossary entries, or another stage's protection. A glossary stage selecting all
of its protected vocabulary as active input has no glossary-derived protection
from those terms; its other validators still assess proposed corrections.
A stage can supply a narrower or different `protected_terms` list. Explicit
`protected_terms: []` disables glossary-derived protection for that stage without
clearing its correction/context inputs or disabling other validators. Explicit
`glossaries: []` clears correction/context inputs without changing the protection
selection.
Audita preflight requires a glossary-correction stage's effective selection to
supply at least one entry; other stages may operate without glossary context
where supported. A glossary selected only by a stage is not automatically added
to root protection.
Use the active-term exemption for glossary correction and retain the existing
module-appropriate allowance for spoken-word cleanup. Carry both selections
through execution and validation rather than passing a single combined glossary
to every consumer.
### Validation-Profile Fallback — Resolved
Combine root and stage selections, including explicit clearing, before deriving
the validation-profile fallback. If no validation selection remains, use the
stage's effective proposal profile. An explicit root validation profile is
inherited unless the stage replaces or clears it. Preserve presence so a value
previously derived at the root is not mistaken for an explicit choice.
| Root selection | Stage selection | Effective validation profile |
| --- | --- | --- |
| Proposal A; validation omitted | Proposal B; validation omitted | B |
| Proposal A; validation V | Proposal B; validation omitted | V |
| Proposal A; validation V | Proposal B; validation null | B |
| Proposal A; validation null | Proposal B; validation omitted | B |
| Proposal A; validation V or omitted | Validation W | W |
| Proposal A; validation omitted | Both omitted | A |
Clearing the validation-profile selection does not disable validation. This rule
does not invent a proposal selection when none is available; a stage requiring
model execution must have a valid effective proposal profile.
### Compound Values and Explicit Clearing — Resolved
Resolve eligible settings field by field. A partial settings group overrides
only its supplied children; an empty mapping such as `llm: {}` contributes no
overrides. For example, a stage setting `llm.repair_attempts: 0` inherits the
root proposal profile and operation timeout while explicitly selecting no repairs.
Omitted fields inherit root settings, with derived defaults applied at the scope
specified for each setting. Root protection defaulting precedes stage inheritance;
validation-profile fallback follows the combination of root and stage selections.
False, zero, and empty values remain explicit and follow per-field validation:
zero repairs is valid, while a zero operation timeout is invalid.
| Explicit input | Meaning |
| --- | --- |
| `context.description: ""` | Clear the inherited description. |
| `glossaries: []` or `glossaries: null` | Clear correction/context inputs; a glossary-correction stage then lacks its required input. |
| `protected_terms: []` or `protected_terms: null` | Disable glossary-derived protection without changing correction/context inputs. |
| `llm.validation_profile: null` | Clear the inherited selection, then use the effective proposal profile. |
| `llm.validation_profile: ""` | Error: use null to clear the selection deliberately. |
| Empty or null proposal profile | Invalid for a stage requiring model execution. |
| `llm: null` | Error: a settings group cannot be cleared as a whole. |
Declare genuinely optional selectors nullable and preserve omission versus
explicit null through resolution. Null is not a universal reset operation;
settings groups are not nullable. For glossary and protection selection lists,
apply the resolved null-as-empty contract below. This concerns stage selections,
not ConfigKit-owned definition collections.
Reject a glossary-correction stage's missing or empty effective input in Audita
preflight after configuration handoff and before execution. Other stages may
operate without glossary context where their operation supports it, independently
of their protection selection.
Lists replace the entire inherited list where list-valued stage settings apply;
`[]` and `null` explicitly select no entries. Do not append or union implicitly.
Preserve the owning library's semantics for library-owned resources and
identified collections. These scope rules do not introduce a custom merge
language or change ConfigKit source composition.
### List Inputs and Configuration Handoff — Resolved
Use ConfigKit's native nullable-list decoding and explicit-presence tracking for
`glossaries` and `protected_terms`. Omission can trigger the agreed root defaults
or stage inheritance within ConfigKit's application-resolution lifecycle;
explicit `null` and `[]` both suppress that defaulting or inheritance. Resolve
all effective selections before handoff.
Once Audita receives the fully resolved configuration, treat a null selection
as equivalent to an empty set. Do not restore defaults, inherit another value,
or run a second configuration-resolution pass because a received list is nil.
Nonempty selections retain their agreed ordering. ConfigKit may preserve the
input distinction between `null` and `[]` in provenance and inspection without
that distinction affecting processing; no normalization of the resolved
configuration is required solely to erase it.
An empty effective protection selection contributes no protected vocabulary.
An empty effective glossary input makes a glossary-correction stage unready to
execute, whether it originated from null, an empty list, absent configuration,
or selected collections with no entries. Enforce that business rule in Audita
preflight after handoff; ConfigKit may successfully resolve those configurations.
Use standard CommandKit/parser conventions for CLI list inputs and ConfigKit's
native override API for explicit root assignments. Pass the completed parser
selection as one override for its root field; let ConfigKit apply it and resolve
the resulting configuration. Exact flag spelling is an implementation detail
within that boundary. Do not introduce custom clearing syntax or merge machinery.
Explicit stage selections remain authoritative under the resolved precedence
rules. Use `[]` as the usual configuration example for clearing while accepting
native null values.
### CLI Overrides and Stage Precedence — Resolved
Ordinary CLI setting flags override the corresponding root settings through
ConfigKit's native CLI override API. Resolve stage inheritance afterward:
omitted stage fields inherit the effective root values, while explicit stage
values and clearing remain authoritative for that stage. Apply validation-profile
fallback after combining these selections. This preserves deliberate differences
between stages while allowing invocation-specific changes to their defaults.
For the party/spells scenario, a CLI assignment changing the root glossary
selection changes grammar's inherited input and, with root `protected_terms`
omitted, the derived protection inherited by all three stages. The explicit party
and spell correction selections stay unchanged.
An explicit root protection list is not changed by a root glossary assignment;
an explicit stage protection list, `[]`, or `null` remains authoritative for that
stage.
Apply this precedence consistently to all stage-capable settings. Submit only
explicitly supplied CLI values to ConfigKit; preserve false, zero, and empty
values under their per-field rules rather than allowing parser defaults to
override configuration.
Keep CLI functionality within ConfigKit's native capabilities. ConfigKit v1.2.0
does not support sequence-index paths in its CLI override API, so targeted-stage
CLI overrides are not part of this plan. Operators supply stage-specific values
in configuration files. Do not implement a parallel override system, custom
merge machinery, or a workaround for addressing stage-list entries. Any future
targeted-stage CLI support should depend on suitable native ConfigKit support.
The resolved stage-identity contract concerns reporting and diagnostics; stage IDs
do not imply CLI targeting. Audita's domain-specific inheritance and fallback
rules remain within ConfigKit's supported application-resolution lifecycle.
### ConfigKit Support
Use the selected ConfigKit v1.2.0 public lifecycle for this work. Its
[typed decoding contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/config.md#defaults-and-typed-decoding)
already preserves explicit false, zero, empty, and permitted null values,
recurses through partial mappings, and replaces ordinary sequences. Represent
nullable selectors with nullable typed fields and settings groups with
non-nullable structs. Use native decoding and validation rather than a parallel
parser or generic deep-merge implementation.
Use the native [collection and inheritance contracts](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/config.md#strict-document-composition)
for glossary definitions. For identified lists, register identity constraints in
Audita's ConfigKit definition so native final validation also covers values from
defaults, CLI, and recorded resolution. YAML-only declarations do not provide
that final-result guarantee. Apply the selected collection-scoped `id` identity
through `SequenceIdentityMatching` with literal mapping keys and an
any-mapping-key segment for the collection name.
Compose sources in the scope where cross-source identity and inheritance are
intended to apply. Loading files separately and concatenating afterward does not
provide those upstream guarantees. Strict peers reject duplicate identities,
including identical records; higher layers follow native collection operations,
and upsert replaces a whole matching record. Empty identified collections follow
their native mode (`[]` does not clear an append/upsert collection). These rules
are distinct from Audita's whole-list replacement of ordinary stage selections.
Do not extend native inheritance beyond its supported sibling scope or treat
inherited entries as abstract templates exempt from normal validation.
Root-to-stage inheritance remains an Audita application rule: ConfigKit does not
automatically copy root fields into new sequence elements. Implement that rule
in `Definition.Resolve`, using `Resolution.IsExplicit` for presence and
`Resolution.EntryInfo` for the fixed pre-resolution metadata. Use
`configkit.Default` for omitted stage fields and record root-field dependencies.
Do not treat a derived root validation fallback as an explicit selection.
Implement root protection defaulting in the same `Definition.Resolve` callback:
use `configkit.Default` for root `protected_terms`, copying effective root
`glossaries` and recording that path as its dependency. Then default each omitted
stage protection list from the effective root protection list, even when that
root value was derived. Preserve source context through both steps and avoid
shared mutable list storage. Explicit empty lists and nulls must suppress
defaulting.
Leave `protected_terms` absent from the bundled configuration underlay. ConfigKit
does not express this live list-to-list dependency as a declarative underlay
reference: native `extends` concerns sibling mappings and runs before CLI
assignment. Use the public recorded-resolution lifecycle, which sees final CLI
assignments, rather than static duplicated lists or custom reference machinery.
An explicitly null validation selection needs deliberate application fallback:
`configkit.Default` correctly leaves explicit null alone. Apply the chosen
fallback and use `Resolution.RecordDerivedDefault` to record the assignment and
its effective proposal-profile dependency, retaining the original explicitness
and history. Follow the native
[recorded-resolution contract](https://forge.maximumdirect.net/base-layer/configkit/src/commit/c1c7e87f2cdf77a0cbf3327831ac05578e756b9e/docs/config.md#recorded-application-resolution)
for all derived mutations, and use `Definition.Validate` for configuration-contract
checks on effective settings. The execution-readiness check for an empty glossary
stage belongs to Audita preflight after handoff. Keep stage values independent
when copying compound values.
The selected defaulting rules use existing public APIs. Verify the final glossary
composition and selection design against the native API before implementation;
do not compensate for a mismatch with a private configuration system.
### Durable Decision Record
Completion requires durable documentation before retiring this roadmap. Record
the glossary-configuration ownership boundary, typed handoff, and Audita preflight
responsibility, including its coordination with the output-session lifecycle, in
an ADR under the [documentation policy](../policy/documentation.md#architecture-decision-records).
This is an architectural ownership decision; routine scope and clearing choices
do not each require an ADR. The roadmap owns implementation status and sequencing.
Put enduring scope classification and ownership rules in the
[architecture policy](../policy/architecture.md) and the implemented collection,
selection, defaulting, and clearing contracts in the
[configuration reference](../config.md). Update the glossary input contract and
consumer internals as the redesign lands, removing obsolete standalone-parser
instructions. Document the distinction between configuration identity and derived
runtime vocabulary without copying the native ConfigKit reference into Audita.
When the package lands, add `internal/framework/preflight` to the
[internal overview](../internal/overview.md) and describe its invocation boundary
in the [pipeline internals](../internal/pipeline.md).
Update the [configuration change guide](../internal/cli-config.md) to require
classifying each new setting by consumer lifecycle, specifying its inheritance
and presence behavior, and documenting its scope in the configuration reference.
New settings should follow the established stage/global distinction; deliberate
exceptions need a domain rationale. Link to the durable policy rather than copy
the field inventory into multiple developer documents. This documentation work
is part of feature acceptance, not an optional follow-up.
2026-11-09 16:03:16 +00:00
## Integrated Target
The glossary feature uses the adopted ConfigKit and CommandKit conventions and
the shared preflight/output-session boundary. It replaces the standalone glossary
input and parsing path with typed, composed configuration. Runtime consumers
receive derived per-stage views from that handoff; reports identify the stage
and configuration inspection explains its effective selections and origins.
Consumed glossary sources participate in output-collision protection. Maintained
examples and hard-cutover guidance describe the new configuration contract.
## Acceptance Direction
Include a shared-library scenario with two campaigns consuming the same reusable
library and maintaining different local party, NPC, and place collections.
Verify independent selections and focused stages, inherited protection and
active-term exemptions, and provenance identifying shared versus campaign
definitions. Demonstrate an intentional adaptation through the selected native
inheritance/layering contract without modifying the shared source or the other
campaign's configuration. Cover an imported but unselected collection remaining
absent from prompt/protection views. Maintained examples should show deliberate
library revision adoption without introducing package-management code in Audita.
Use distinct party and spell fixtures with controlled model responses to show
that each stage receives its intended correction and protection selections,
that overrides do not leak into other stages, and that the second stage sees
the first stage's accepted transcript changes. Verify meaningful errors for
invalid selections or inputs, and retain the selected domain validation and
partial-success behavior. Exact test cases follow the design; live provider
calls are unnecessary.
Verify root fallback, per-stage override isolation, and explicit-value presence
for the selected stage settings. Include the grammar stage after the overridden
glossary stage to establish that it inherits both root glossary inputs. Verify
that all three stages inherit root protection derived from those inputs despite
the focused correction selections. Cover explicit root and stage protection
lists, independent empty-list clearing, a glossary stage with no override using
both inputs, and a collection selected only on a stage not entering root protection.
Verify that modules and validators receive their respective effective selections
and recorded source origins. Cover repeated modules and an empty stage list.
Verify named, unnamed, and mixed pipelines, rejecting empty or non-string supplied
IDs and duplicate IDs through configuration validation. Cover exact case-sensitive
comparison without imposing slug syntax. Verify IDs remain associated with their
stages after reordering while reported positions reflect the new order. Reports
and diagnostics retain module and position, with optional IDs when supplied;
inspection exposes effective selections and source origins for every stage.
Confirm stage IDs do not change pipeline composition or add CLI targeting.
Cover default-source loading with only `glossary.yml`, only `glossary/`, both,
and neither present. Verify resolution beside explicit and discovered main files
outside the working directory. With no main context, verify optional imports are
skipped without a working-directory fallback. Preserve normal errors for invalid
present sources and preflight rejection of unusable glossary stages. Verify
strict-peer conflicts within the underlay, higher-priority main
configuration additions/replacements under the selected native policies, nested
import destination inheritance, and repeated inclusion within the default tier.
Confirm that an empty main-file import list does not cancel the underlay imports.
Exercise recursive directory imports with nested files, both YAML extensions,
explicit collection names, and native placement/source attribution. Verify that
file relocation within the imported tree preserves collection selection, and
unknown collection names fail configuration validation. Cover multiple collections
in one file and contributions from multiple files under the selected identity
rules. Do not derive collection keys from filenames or discovery order.
Verify that missing, null, or empty entry IDs fail, and duplicate IDs in a
collection's strict peer sources retain native errors. Cover native matching of
IDs across layers, including whole-record upsert, and final-result validation.
Allow the same ID in different collections and the same displayed name on distinct
records. Verify that changing a term or description while retaining its ID targets
the same collection record, and that runtime spelling deduplication does not
collapse distinct configuration identities. Cover exact case-sensitive ID matching.
Verify required name/category validation and the optional description, aliases,
and plural contracts, including invalid types and empty values where prohibited.
Cover additional scalar, nested mapping, and list fields surviving native loading
and inspection; extension data must not mask invalid known fields. Verify that
unrelated extra fields affect neither the LLM projection nor protected vocabulary.
Exercise native glossary composition, inheritance, collection identity checks,
and supported explicit layering with reusable party/spells fixtures. Cover source
origins and configuration validation failures during loading, then business-rule
failures during Audita preflight before processing.
Test configured discovery/no-main-file behavior through the supported load path;
do not add a secondary glossary-file resolver. Verify that inheritance and
application derivations retain traceability to their configuration inputs.
Verify that distinct valid definitions can contribute to a deduplicated runtime
vocabulary without rewriting rich configured entries. Check that rich LLM input
contains the selected descriptive fields, and that deriving or using any runtime
view leaves configuration and other consumers' views unchanged. Verify the
selected rules for repeated membership and entry ordering: first occurrence of
each `(collection name, id)` wins, while separate records with matching names or
content remain distinct. Verify that deduplicated spellings retain all contributing
entry associations and do not multiply protection requirements. Preserve existing
ambiguous-alias and protected-name repetition behavior outside the selected
active-vocabulary exemption when adapting consumers to these views.
Exercise the selected request and validation overrides through stage consumers,
including independent budgets without changing shared backend admission or the
initial transcript sections. Cover all validation-profile fallback cases above
and preserve the distinction between explicit and derived values in inspection.
Cover partial and empty settings groups, explicit zero and false, nullable
selectors, empty-list replacement, and invalid empty strings or group nulls.
For glossary and protection selections, cover omitted, null, empty, and nonempty
root and stage inputs through ConfigKit resolution. Confirm explicit null and
empty lists suppress defaulting and inheritance while inspection preserves native
provenance. Verify equivalent null/empty behavior in runtime consumers, with no
post-handoff default restoration or configuration resolution. Both forms must
fail Audita preflight when they leave a glossary-correction stage without input.
Verify ConfigKit can hand off empty collections and empty effective selections
and that ordinary inspection remains available. Cover a selected empty collection,
several collectively empty collections, and mixed empty/populated collections.
Include an empty glossary stage late in the pipeline and verify preflight fails
before output resources are created, any stage executes, or processing model
calls occur. Check useful stage and collection attribution in the readiness
error, and no silent stage skipping.
Exercise the preflight package directly with resolved inputs and through the
processing entry point. Verify it leaves configuration unchanged and returns
readiness errors without provider calls or CLI presentation responsibilities.
Exercise CLI list assignment through the standard parser and ConfigKit override
path, including explicit empty selections and unchanged explicit stage settings.
Verify that clearing a validation selection follows the effective proposal
profile without disabling validation, and that clearing a required glossary
fails preflight before execution. Check recorded dependencies and retained
clear-input history as well as effective values. Verify that explicit CLI flags change root
defaults and inherited stage values while preserving explicit stage settings
and clearing. Cover profile fallback after CLI assignment, explicit false/zero/
empty CLI values where valid, and the absence of overrides from parser defaults.
Cover root protection defaulting after CLI assignment, explicit root protection
surviving glossary overrides, and recorded root-to-stage protection dependencies.
Verify module-appropriate protection using terminology from both collections, including
legitimate corrections and cleanup; retain regression coverage for existing
validation behavior outside the selected active-vocabulary exemption.
For glossary stages, verify that active terms are exempt while remaining selected
terms stay protected, including when protection was explicitly configured.
Cover names, aliases, supported plurals, and vocabulary shared across collections.
Verify a spelling contributed by both an active entry and a distinct non-active
protected entry remains exempt regardless of collection order or explicit
protection selection. Check that protected-term validation does not block changes
to that active spelling, while unrelated spellings of the non-active entry remain
protected. Preserve other validators and later stages' independent protection.
Verify the all-active case produces an empty effective protected vocabulary
without disabling other validators. Confirm that a later non-glossary stage
still protects its full selection and that inspection distinguishes configured
protection inputs from the glossary stage's effective protected vocabulary.
Confirm that the durable policy, configuration contracts, and contributor
guidance are in place before declaring this feature complete.