Validate correction repair through the processing command
This commit is contained in:
@@ -77,11 +77,29 @@ configuration.
|
||||
| `reporting.enabled`, `reporting.stdout_dir` | `true`, `.` | Ordinary report capture and default directory for stdout output. |
|
||||
| `diagnostics.work_dir` | `/tmp/audita` | Parent directory for optional debug capture. |
|
||||
|
||||
For example, an entry can disable correction repair or select another profile:
|
||||
|
||||
```yaml
|
||||
llm:
|
||||
proposal_profile: local
|
||||
correction_repair_attempts: 2
|
||||
pipeline:
|
||||
stages:
|
||||
- module: grammar
|
||||
llm:
|
||||
correction_repair_profile: review
|
||||
- module: spoken_word
|
||||
llm:
|
||||
correction_repair_attempts: 0
|
||||
```
|
||||
|
||||
Processing profiles are selected explicitly; no profile or glossary is selected
|
||||
automatically. The validation selection falls back after root and per-entry
|
||||
settings are combined. An explicit root correction repair profile is inherited
|
||||
by each entry unless overridden or cleared with null; otherwise the entry uses
|
||||
its effective proposal profile. The repair profile is checked only when its
|
||||
attempt budget is positive. Null does not disable validation. Numeric zero and false
|
||||
attempt budget is positive. Each correction repair attempt is a logical call
|
||||
that can also use the configured transient retries and structural repairs. Null
|
||||
does not disable validation. Numeric zero and false
|
||||
retain their field meaning, subject to validation. Inspect recorded origins and
|
||||
default dependencies using the [native commands](cli.md).
|
||||
|
||||
@@ -16,7 +16,13 @@ credential source names for artifact redaction without inspecting unrelated
|
||||
environment values. See the [integration contract](../integrations/openai-compatible-llm.md)
|
||||
for failure behavior and the [configuration reference](../config.md) for sources.
|
||||
|
||||
Proposal calls can retain the original prompt, effective variables, prepared messages, and final response for section-local repair replay. The adapter renders the repair definition with the selected profile, appends branch history to the original prompt, and checks the prepared prefix before execution. The requested repair definition is retained in call attribution alongside the native base-prompt metadata.
|
||||
When correction repair is enabled, the proposal call retains its prompt,
|
||||
effective variables, prepared messages, and full final response for section-local
|
||||
replay. The adapter renders the repair definition with the selected profile,
|
||||
appends only that branch’s history, and checks the original prepared prefix
|
||||
before execution. The complete conversation is sent to the selected provider;
|
||||
provider rejection ends that branch as unavailable. The requested repair
|
||||
definition is retained in call attribution alongside native base-prompt metadata.
|
||||
|
||||
The observation collector tracks logical calls and execution events. Ordinary
|
||||
reports use compact safe outcomes; request and response evidence is optional
|
||||
|
||||
@@ -13,6 +13,9 @@ Modules implement `contracts.TranscriptModule`:
|
||||
- `Validators()`
|
||||
- `Propose(ctx, req)`
|
||||
|
||||
Built-in modules also expose `RepairPromptID()` for the runner to select their
|
||||
repair definitions; proposal-only callers retain the existing interface.
|
||||
|
||||
Runner resolves configured module specs to module instances through `internal/framework/modules`.
|
||||
|
||||
## Built-In Modules
|
||||
@@ -61,6 +64,10 @@ Module packages call the built-in chain resolver at construction.
|
||||
|
||||
## Failure and Outcome Behavior
|
||||
|
||||
Each actionable rejection can be revised or withdrawn within its original
|
||||
section. A revision retains the original proposal identity and must pass the
|
||||
full validator chain before application.
|
||||
|
||||
Module construction failures are fatal. Recoverable proposal model failures
|
||||
record an unsuccessful operation and yield no candidates for that section;
|
||||
other sections and module instances continue. Fatal adapter or application
|
||||
|
||||
@@ -11,9 +11,10 @@ and `audita/modules.grammar.proposal`; the corresponding repair IDs replace
|
||||
|
||||
PromptKit composes embedded defaults with operator prompt sources. Shared
|
||||
instructions come from the embedded fragments in `assets/shared`; the request
|
||||
adapter fills absent `HardeningPolicy`, `ProposalContract`, `RepairContract`, and
|
||||
`ValidationContract` variables without replacing values explicitly supplied by
|
||||
the caller. Processing supplies Audita's response-schema contract separately.
|
||||
shared-variable helper fills absent `HardeningPolicy`, `ProposalContract`,
|
||||
`RepairContract`, and `ValidationContract` variables. The processing adapter
|
||||
supplies the embedded instruction fragments for those names. Processing supplies
|
||||
Audita's response-schema contract separately.
|
||||
The operator's prompt definition controls which variables its template uses.
|
||||
|
||||
For a standard request preview or validation, select a native PromptKit profile
|
||||
@@ -23,7 +24,10 @@ proposal definitions use `TranscriptDescriptionBlock`, `GlossaryJSON`,
|
||||
and the separate spelling list defines protection. Preview defaults the
|
||||
spelling list to `[]` when omitted. The validator definitions use `TranscriptDescriptionBlock` and
|
||||
`PayloadJSON`. The latter is supplied candidate JSON and does not require a
|
||||
proposal inference call. An operator replacement may use different variables.
|
||||
proposal inference call. Repair definitions use `RepairContract` and
|
||||
`RepairRequestJSON`, containing the original proposal reference, current
|
||||
correction, and ordered findings. An operator replacement may use different
|
||||
variables.
|
||||
See the [upstream CLI reference](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/f369d6a78253b6e6fcb1e023fb2e9c3af1d20ff0/docs/cli.md)
|
||||
for the standard preview, validation, and smoke commands.
|
||||
|
||||
|
||||
@@ -49,7 +49,10 @@ For each module section:
|
||||
When correction repair is enabled, the runner also gathers independent findings
|
||||
for rejected candidates when a validator has usable evidence. Unavailable
|
||||
assessments remain unavailable, and only candidates that pass every applicable
|
||||
check can advance. A zero repair budget retains the original shortlist flow.
|
||||
check can advance. A known rejection can enter repair even when another
|
||||
assessment is unavailable; an unavailable assessment alone does not. Revised
|
||||
candidates run the applicable checks again. A zero repair budget retains the
|
||||
original shortlist flow.
|
||||
|
||||
Decision cardinality is enforced inside Audita: each candidate proposal must
|
||||
receive exactly one outcome per validator it reaches. The runner still rejects
|
||||
|
||||
@@ -12,6 +12,10 @@ show complete projects. Processing preflight checks every configured glossary
|
||||
correction and selected profile before output preparation and model work. Empty
|
||||
protection and an empty pipeline are valid. A completed run can contain
|
||||
recoverable model errors while still applying independently validated edits.
|
||||
Correction repair sends the complete original proposal exchange and branch
|
||||
history to the selected repair provider. A provider rejection ends that branch
|
||||
as unavailable; other approved edits can still be applied. See
|
||||
[configuration](config.md#settings) for the attempt budget and profile selection.
|
||||
|
||||
## Outputs and lifecycle
|
||||
|
||||
@@ -48,7 +52,9 @@ Optional debug preparation or writes can produce redacted warnings without
|
||||
changing correction decisions or the required output result. Debug material can
|
||||
include transcript content, prompts, responses, and correction decisions. It is
|
||||
not anonymized; protect it according to the input's sensitivity. Configured
|
||||
credential values are redacted at the diagnostic boundary.
|
||||
credential values are redacted at the diagnostic boundary. Repair interactions
|
||||
and revalidation artifacts include proposal and attempt coordinates; the
|
||||
correction ledger connects them to the original edit.
|
||||
|
||||
## Recovery
|
||||
|
||||
|
||||
@@ -2,8 +2,8 @@
|
||||
|
||||
**Updated:** 2026-11-11
|
||||
|
||||
**Status:** Feature direction and all six decisions accepted. Implementation
|
||||
is pending. See the [implementation plan](implementation.md).
|
||||
**Status:** Implemented. See the [implementation plan](implementation.md)
|
||||
and the [pipeline guide](../internal/pipeline.md).
|
||||
|
||||
## Objective
|
||||
|
||||
|
||||
@@ -11,9 +11,9 @@ implementing every capability.
|
||||
|
||||
The implemented [partial-success decision reconciliation](../internal/validators.md#llm-validator-batching)
|
||||
preserves usable decisions without another model call. Revision of rejected
|
||||
correction proposals is now an agreed feature direction in the
|
||||
[correction-repair roadmap](correction-repair.md), with its feature decisions accepted.
|
||||
That work includes substantive validator feedback and explicit proposal withdrawal.
|
||||
correction proposals is implemented with substantive
|
||||
validator feedback and explicit proposal withdrawal; see the
|
||||
[pipeline guide](../internal/pipeline.md).
|
||||
It is distinct from recovering an unavailable validator decision.
|
||||
|
||||
Consider bounded feedback and regeneration for model responses that satisfy the
|
||||
@@ -32,7 +32,7 @@ shared library capabilities where appropriate; do not introduce an uncoordinated
|
||||
outer retry loop that multiplies existing retry/repair budgets.
|
||||
|
||||
Editorial rejection remains a valid decision about the evaluated proposal. The
|
||||
correction-repair roadmap owns any subsequent revision of that proposal and full
|
||||
implemented correction repair loop handles subsequent revision and full
|
||||
revalidation; this deferred entry concerns unavailable decisions. Decision recovery
|
||||
must preserve required validation, candidate identity, partial-success safety,
|
||||
and truthful error/recovery reporting. Validate with controlled responses that
|
||||
|
||||
@@ -352,6 +352,8 @@ validator responses remains [deferred](future.md#domain-aware-response-recovery)
|
||||
|
||||
## Stage 9 — Validate the assembled command and update canonical documentation
|
||||
|
||||
**Completed.**
|
||||
|
||||
1. Exercise the normal processing command with controlled provider responses,
|
||||
real configuration and prompt loading, and the output session. Cover a
|
||||
mixed proposal set with two independent repair branches, a second turn
|
||||
|
||||
Reference in New Issue
Block a user