From 42930d492114003d3f3f6bf4db0795ee8f49967f Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 8 Nov 2026 21:07:15 +0000 Subject: [PATCH] Select named glossary collections with recursive directory imports --- docs/roadmap/configkit.md | 4 +- docs/roadmap/glossary-stages.md | 164 +++++++++++++++++++++----------- 2 files changed, 114 insertions(+), 54 deletions(-) diff --git a/docs/roadmap/configkit.md b/docs/roadmap/configkit.md index d8f25f1..d5b61e3 100644 --- a/docs/roadmap/configkit.md +++ b/docs/roadmap/configkit.md @@ -130,8 +130,10 @@ file remain terminal. glossary assets use native loading, composition, inheritance, collection identity, and typed decoding before handoff. Audita derives consumer-specific runtime views from that result and removes its separate glossary YAML loader. + Use its [named collections and recursive import placement](glossary-stages.md#named-collections-and-recursive-imports--resolved), + retaining native discovery, source context, and tier semantics. Complete its required durable decision record. The glossary roadmap owns - the remaining collection schema, composition scope, identity, runtime-view, + the remaining entry schema, import provisioning, identity, runtime-view, selection-input, and attribution decisions. This application scope model does not change native source composition or sibling-inheritance semantics. diff --git a/docs/roadmap/glossary-stages.md b/docs/roadmap/glossary-stages.md index 7eacddd..c7d2202 100644 --- a/docs/roadmap/glossary-stages.md +++ b/docs/roadmap/glossary-stages.md @@ -3,8 +3,9 @@ **Updated:** 2026-11-08 **Status:** Accepted feature scope, ownership, and stage-selection semantics for -the next Audita major release. Collection schema, remaining design decisions, -and implementation are pending. The +the next Audita major release, including named collections and recursive import +placement. Entry schema, import provisioning, remaining design decisions, and +implementation are pending. The [master roadmap](library-adoption.md) owns sequencing and release completion. ## Baseline and Target @@ -25,8 +26,8 @@ The major-release policy permits a hard cutover. The current single-glossary CLI, standalone parser, and file schema do not constrain the target design. Finalize the glossary representation with the agreed [version-3 application schema](configkit.md#application-schema-version--resolved) -so operators migrate once. Exact collection, entry, and selection syntax remains -open below; the ownership and domain semantics in this roadmap are selected. +so operators migrate once. Collection placement and named selection are selected; +entry identity, rich fields, and remaining import/selection details are open below. ## Glossary Configuration Ownership — Resolved @@ -50,8 +51,77 @@ Retain native source attribution through application resolution. A glossary filename is a configuration source, not an application-data file for a second post-handoff loader. Choose the source arrangement through ConfigKit's public composition APIs; a string in an application field does not automatically cause -ConfigKit to load another document. The exact collection layout and load scope -must be settled before implementation. +ConfigKit to load another document. Use the selected import placement below; +the provision of the import declaration and its source tier remain open. + +### Named Collections and Recursive Imports — Resolved + +Define `glossary_collections` as a typed mapping of collection names to collection +objects. Each object contains an `entries` collection. Root and stage `glossaries` +and `protected_terms` are ordered lists selecting those names, not filenames. +An omitted stage selection inherits the root selection; an explicit list replaces +it, and `[]` selects none. Report unknown collection names during configuration +validation. Selection order determines runtime traversal, not source precedence. + +Use ConfigKit's recursive directory imports mounted at `glossary_collections`, +with each imported file declaring its collection name. The following main-file +example illustrates the native import declaration and stage selections; automatic +provision of that declaration is a separate open decision: + +```yaml +version: 3 +config: + imports: + - dir: glossaries + recursive: true + pattern: ["*.yml", "*.yaml"] + destination: [glossary_collections] + +glossaries: [party, spells] +pipeline: + stages: + - module: glossary + glossaries: [party] + - module: glossary + glossaries: [spells] + - module: grammar +``` + +An imported `party.yml` declares the collection directly, without repeating the +application's `glossary_collections` wrapper: + +```yaml +party: + entries: + - id: merrin + name: Merrin + category: character + summary: The party's wizard. +``` + +The `party`/`entries` nesting is selected. Entry identity and rich fields in this +illustration remain provisional pending the entry-schema decision below. The +`glossaries` directory is an example declaration, not a selected implicit default. + +Keep collection identity in document keys. ConfigKit does not infer keys from +filenames or subdirectories; all matches of the declaration receive the same +destination. Moving a file within the imported tree therefore need not change +stage selections. A file can define multiple collections, and multiple sources +can contribute to one collection under native composition and identity rules. +Repeated membership and runtime entry ordering remain domain decisions below. + +Use native mapping placement, discovery, composition, and provenance. Do not add +a directory walker, filename-to-key conversion, or YAML preprocessing in Audita. +Imports in the same tier are strict peers; their discovery order does not create +an override priority. Intentional overrides use upstream layering semantics. + +An application-supplied declaration may provide this same import behavior, but +an underlay does not set a universal destination for imports in other documents. +Import defaults can supply recursive/pattern policies; they neither discover +sources themselves nor supply destinations. A supplied declaration must have the +appropriate filesystem context, and an underlay places its contributions in a +lower-priority tier. Resolve that provisioning choice below without changing +ConfigKit's import semantics. ### Derived Runtime Representations — Resolved @@ -84,12 +154,10 @@ Preserve meaningful false, zero, and empty values under each setting's validatio rules rather than treating them as omission. A stage override does not alter root settings or defaults seen by later stages. -The names `glossaries` and `protected_terms` below identify the agreed correction/ -context and protection selection roles. Their final representation must select -resolved glossary configuration; direct file-path lists are not a settled schema -for this redesign. Collection definitions, collection references, and their field -names are part of the native configuration-shape decision below. Do not introduce -a general variable-substitution language. +`glossaries` and `protected_terms` select named definitions from the resolved +`glossary_collections` mapping. Keep file organization in the import declarations +and collection names in stage settings. This is domain selection over typed data, +not a general variable-substitution language. For the party/spells use case, the root selects both collections, the first stage selects party vocabulary, the second selects spell vocabulary, and grammar @@ -149,10 +217,10 @@ An explicit protection selection is complete, not an additive set of extra vocabulary. Do not implicitly union it with correction inputs or collections selected by other stages. -These domain rules survive the configuration redesign. The native collection -shape, entry order, and exact selection representation are open; configuration -composition must follow ConfigKit's own contracts rather than a blanket local -rule to concatenate independently parsed files. +The named mapping and ordered name selections are selected above. Entry identity, +repeated membership, and rich-input order remain open. Configuration composition +must follow ConfigKit's contracts rather than a local rule to concatenate +independently parsed files. Resolve these settings in this order: @@ -387,8 +455,9 @@ adopted configuration and CLI conventions and avoids adapting it twice. Keep the requirement visible during library design without expanding those migrations into feature implementation. -1. Settle the native collection/entry schema, source composition scope, stage - selection syntax, and remaining domain decisions below. Preserve the selected +1. Apply named collection selection and recursive directory placement. Settle + entry schema, import provisioning, CLI selection syntax, and remaining domain + decisions below. Preserve the selected scope, clearing, CLI precedence, and fallback semantics in version 3. 2. Adopt ConfigKit loading for glossary configuration. Register the selected collection identities, supply domain validation, and deliver final typed @@ -426,6 +495,12 @@ lists, independent empty-list clearing, a glossary stage with no override using both inputs, and a collection selected only on a stage not entering root protection. Verify that modules and validators receive their respective effective selections and recorded source origins. Cover repeated modules and an empty stage list. +Exercise recursive directory imports with nested files, both YAML extensions, +explicit collection names, and native placement/source attribution. Verify that +file relocation within the imported tree preserves collection selection, and +unknown collection names fail configuration validation. Cover multiple collections +in one file and contributions from multiple files under the selected identity +rules. Do not derive collection keys from filenames or discovery order. Exercise native glossary composition, inheritance, collection identity checks, and supported explicit layering with reusable party/spells fixtures. Cover source origins, strict decoding failures, and domain validation failures before processing. @@ -470,41 +545,24 @@ guidance are in place before declaring this feature complete. The recommendations below are proposals, not selected contracts. -### Native Collection Shape and Stage Selection +### Import Provisioning and Default Directory -**Recommendation:** define named glossary collections in a typed mapping and -have root/stage selections identify those resolved collections. Use native -ConfigKit composition to populate the mapping, with minimal Audita lookup and -selection validation after structural resolution. This decouples reusable assets -from file layout and keeps focused stages concise. It requires choosing the -collection/selection field layout, membership representation, ordering, and -behavior when the same entry is selected through several collections. The exact -YAML syntax is not yet selected. +**Recommendation:** initially express the recursive directory import in the main +configuration through native `config.imports`, as illustrated above. This keeps +its location, filesystem context, and strict-peer relationship visible and avoids +new automatic discovery conventions. It costs a small import declaration in the +main file. The directory path remains operator-selected rather than a fixed +implicit location. -**Alternative:** embed complete glossary entry collections directly in root and -stage settings, using native inheritance and source placement for reuse. This -avoids collection-reference lookup but makes stage settings larger and reuse -subject to native sibling scopes. It can duplicate definitions and complicate -independent correction/protection selections. Neither option adds general -variable interpolation or a custom merge language. - -### Composition Scope and Reuse - -**Recommendation:** compose glossary definitions within the application's native -configuration graph before the final handoff. Treat independent sources as strict -peers and reserve intentional replacement for explicit upstream layering. This -makes cross-source identity checks, inheritance, and provenance available in the -same scope. Verify a practical import/placement layout for standalone reusable -assets; neither file-list order nor application selection should secretly create -override tiers. Standard inspection integration should follow the resulting -ConfigKit configuration rather than a second private catalog. - -**Alternative:** load isolated glossary libraries through separate ConfigKit -definitions and assemble a typed application view before processing. This can -provide useful library boundaries and independent validation, but inheritance and -identity guarantees stop at each load. Cross-library selection validation and -inspection integration would need additional application work. Do not claim that -concatenating separate results gives native cross-load composition guarantees. +**Alternative:** have Audita supply a native configuration input that declares +the directory import, potentially as an underlay. This removes repeated main-file +boilerplate and can offer a standard directory convention, but requires choosing +how its directory is configured, its filesystem context, and its precedence tier. +An underlay is lower priority, not merely a hidden main-file import. Define its +behavior when no main file exists and preserve native optional/required import +semantics. Do not scan a host directory through an unrelated embedded filesystem +or build an application-owned import resolver. This choice remains open; the +recursive mapping destination and file-declared collection names are settled. ### Entry Identity and Rich Fields @@ -548,8 +606,8 @@ already selected and is not reopened by this decision. ConfigKit slice decoding and presence without an additional rejection rule. Keep `[]` as the primary documented clearing form. This requires documenting a second clearing form and ensuring null suppresses protection defaulting just as `[]` -does. After choosing the collection schema, select CLI syntax for whole root -selections using native ConfigKit overrides. These are ordinary selection lists, +does. Select CLI syntax for whole root name selections using native ConfigKit +overrides. These are ordinary selection lists, not entry collections governed by native identity/operation policies. **Alternative:** reject null for the two selection lists and require `[]` for