From d2c0e22a9a43b085657138a1845a9d35ca0e5aa9 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Sun, 8 Nov 2026 15:07:13 +0000 Subject: [PATCH] Resolve stage setting scope and validation profile fallback --- docs/roadmap/configkit.md | 8 ++- docs/roadmap/glossary-stages.md | 101 ++++++++++++++++++++++++++------ 2 files changed, 88 insertions(+), 21 deletions(-) diff --git a/docs/roadmap/configkit.md b/docs/roadmap/configkit.md index a337359..c7e2cc5 100644 --- a/docs/roadmap/configkit.md +++ b/docs/roadmap/configkit.md @@ -121,9 +121,11 @@ file remain terminal. in version-3 schema design. Implement it during the glossary feature stage after library adoption. Each stage resolves independently with explicit-value presence and source provenance, and its module and validators receive consistent - effective settings. The glossary roadmap owns the remaining scope, precedence, - path, and attribution decisions. This application scope model does not change - native source composition or sibling-inheritance semantics. + effective settings. Apply its [selected scope and fallback rules](glossary-stages.md#configuration-scope--resolved) + and complete its required durable decision record. The glossary roadmap owns + the remaining clearing, precedence, path, and attribution decisions. This + application scope model does not change native source composition or + sibling-inheritance semantics. ## Documentation and Acceptance diff --git a/docs/roadmap/glossary-stages.md b/docs/roadmap/glossary-stages.md index 9eea68c..5d7f0cd 100644 --- a/docs/roadmap/glossary-stages.md +++ b/docs/roadmap/glossary-stages.md @@ -58,11 +58,82 @@ needs a broader selection remains an open domain decision below. Apply this model to settings governing stage processing. Invocation-wide concerns such as configuration discovery, output destinations, and reporting -mode remain global. The exact set of stage settings is still to be selected. +mode remain global. The selected scope inventory follows below. Preserve stage order, repeated modules, and explicit empty-pipeline behavior. Use native ConfigKit loading, presence, and provenance facilities; Audita owns the meaning of stage scope and passes the effective settings to its consumers. +## Configuration Scope — Resolved + +Settings governing an individual stage's requests, acceptance, or model-operation +budget support root defaults and stage overrides. Settings governing preprocessing, +the invocation's shared resources, or its final outputs remain global. Classify +by the setting's meaning and consumer lifecycle, not by where current code stores +it. The names below identify current settings; version-3 field representation +must preserve these scope decisions. + +| Setting | Scope and rationale | +| --- | --- | +| Glossary input | Stage: vocabulary supplied to the module and its validators. | +| Applicable confidence threshold | Stage: acceptance of that stage's proposals. | +| Proposal and validation profile selection | Stage: models and generation settings appropriate to each role. | +| `context.description` | Stage: prompt context shared by proposals and validators. | +| `validation.max_prompt_tokens` | Stage: batching budget for that stage's validators. | +| `llm.operation_timeout` | Stage: deadline for each logical model operation. | +| `llm.repair_attempts` | Stage: structural-repair budget for its requests. | +| `llm.transient_retries` | Stage: native transient-retry budget for its requests. | +| All `normalization.*` settings | Global: establish transcript segments before stage execution. | +| All `chunking.*` settings | Global: establish common section boundaries and identities across stages. | +| Transcript input, output destination, and `output.schema` | Global: input and final product of the whole run. | +| Reporting, debug capture, and artifact directories | Global: invocation reporting and retained evidence. | +| Configuration discovery, imports, and resource-source locations | Global: construct the configuration and catalog shared by stages. | +| Backend definitions, credentials, concurrency, and queue capacity | Global native PromptKit resources and shared backend admission. | +| Pipeline ordering and application schema version | Global: define the overall processing configuration. | + +Stages may select different profiles and thereby different configured models or +backends. Model parameters remain in native PromptKit profiles rather than a +duplicated Audita settings system. Apply stage request budgets through native +request controls and operation contexts while retaining the shared engine and +backend admission. Do not rebuild or mutate shared backend configuration for +each stage. Per-stage validation batching does not change transcript chunking; +per-stage rechunking would be a separate feature decision. + +### Validation-Profile Fallback — Resolved + +Combine explicit root and stage settings before deriving the validation-profile +fallback. If no validation profile was explicitly selected at either scope, use +the stage's effective proposal profile. An explicit root validation profile is +inherited unless the stage supplies its own validation profile. Preserve presence +so a value previously derived at the root is not mistaken for an explicit choice. + +| Root selection | Stage selection | Effective validation profile | +| --- | --- | --- | +| Proposal A; validation omitted | Proposal B; validation omitted | B | +| Proposal A; validation V | Proposal B; validation omitted | V | +| Proposal A; validation V or omitted | Validation W | W | +| Proposal A; validation omitted | Both omitted | A | + +This rule does not invent a proposal selection when none is available. Explicit +empty/null values must retain their presence; their clearing or validation +semantics remain part of the open decision below. + +### Durable Decision Record + +Completion requires recording the scope model and its rationale in an accepted +[ADR](../adr/README.md) before retiring this roadmap. Put the enduring +classification rule in the [architecture policy](../policy/architecture.md), +linked to that ADR; keep the implemented per-field scope and fallback contracts +in the [configuration reference](../config.md). The roadmap continues to own +implementation status until the feature lands. + +Update the [configuration change guide](../internal/cli-config.md) to require +classifying each new setting by consumer lifecycle, specifying its inheritance +and presence behavior, and documenting its scope in the configuration reference. +New settings should follow the established stage/global distinction; deliberate +exceptions need a domain rationale. Link to the durable policy rather than copy +the field inventory into multiple developer documents. This documentation work +is part of feature acceptance, not an optional follow-up. + ## Sequencing and Work Outline Implement after ConfigKit, PromptKit, CommandKit, and ReleaseKit adoption, and @@ -71,8 +142,8 @@ adopted configuration and CLI conventions and avoids adapting it twice. Keep the requirement visible during library design without expanding those migrations into feature implementation. -1. Finalize the stage-setting inventory, compound-value and clearing rules, path - resolution, and CLI precedence under the selected scope model. Incorporate +1. Apply the selected scope inventory and validation-profile fallback. Finalize + compound-value and clearing rules, path resolution, and CLI precedence. Incorporate these into the version-3 schema rather than retain the old module-name list. 2. Resolve effective stage settings and carry them through stage execution, proposal generation, and validation. Carry selected glossary inputs through @@ -82,7 +153,8 @@ into feature implementation. handling and output-collision protection. 4. Verify the selected behavior and update configuration, CLI, input contracts, pipeline documentation, maintained examples, and hard-cutover guidance as the - implementation lands. + implementation lands. Complete the durable decision record and future-setting + classification guidance above. ## Acceptance Direction @@ -99,24 +171,17 @@ glossary stage to establish that it inherits the root glossary. Verify that each stage's module and validators receive consistent effective settings, including their recorded source origins. Cover repeated modules and an empty stage list. +Exercise the selected request and validation overrides through stage consumers, +including independent budgets without changing shared backend admission or the +initial transcript sections. Cover all validation-profile fallback cases above +and preserve the distinction between explicit and derived values in inspection. +Confirm that the durable policy, ADR, configuration contracts, and contributor +guidance are in place before declaring this feature complete. + ## Open Decisions The recommendations below are proposals, not selected contracts. -### Settings Eligible for Stage Scope - -**Recommendation:** include glossary input, applicable confidence thresholds, -proposal/validation profile selection, and other settings that govern an -individual stage's work. Inventory current settings by the lifecycle of their -consumer before finalizing the schema. Keep discovery, transcript destinations, -reporting, and other invocation-wide settings global. This provides useful -flexibility without implying that a stage can reconfigure the entire run. - -**Alternative:** initially expose only glossary overrides and add other stage -settings later. This reduces the first implementation's scope, but leaves an -incomplete application of the selected model and may require further schema -changes for settings already known to vary by stage. - ### Compound Values and Explicit Clearing **Recommendation:** resolve eligible settings by field, preserving explicit