Remove confidence scoring and refine correction prompts

This commit is contained in:
2026-11-11 17:05:52 +00:00
parent bbf89bb4f5
commit d155bf4cb3
92 changed files with 448 additions and 786 deletions

View File

@@ -44,7 +44,7 @@ Each stage prepares the encoded glossary context and protected spellings once.
Proposal generation adds the selected transcript section and trimmed description
for each request. PromptKit renders the selected definition and validates the
correction-set schema before Audita maps proposals. Domain validators still
check confidence, applicability, protected glossary terms, and replacement
check applicability, protected glossary terms, and replacement
policy. Model-facing segment IDs are one-based positions within the proposal
request. Generation maps returned references back to the current chunk's actual
segment IDs, preserving response order and multiple proposals per segment.
@@ -76,6 +76,11 @@ outcomes are defined by the [report contract](../integrations/output-files.md).
## Changing Modules
Frequently reevaluate the default module combination and sequence against what
has been observed to work best for most workloads. Prior empirical results inform
the current choice without fixing a permanent number or order of passes. Maintain
the selected default in the [configuration contract](../config.md#pipeline-and-glossary-selections).
1. Add or update the narrowly scoped package under `internal/modules/<module_key>`.
2. Keep module prompt selection in the module and its definition in
`internal/prompts`; see [prompt assets](prompts.md).

View File

@@ -59,12 +59,14 @@ pass these values through the role-bound adapter. See the
[configuration](../config.md) for operator-selected profiles.
Proposal instructions describe literal matching, each module's replacement policy,
source-based proposal identity, and confidence in whether an edit is warranted.
and source-based proposal identity.
Proposal `id` and validator `correction_index` values are short, one-based
references scoped to each request, not internal transcript/proposal identities.
Mapping belongs to [proposal generation](modules.md) and
[validator reconciliation](validators.md). Validator instructions distinguish
confidence in the verdict from the verdict itself. Compact output examples
[validator reconciliation](validators.md). Validator decisions use `approved` for the verdict. Approval examples use an empty
`reason`; rejection guidance requests a specific defect and supplied evidence
within the validator's remit. The model explains defects; repair owns revision or
withdrawal. This is prompt guidance, not a runtime nonempty-reason check. Compact output examples
illustrate the supplied schema without requiring fixed JSON formatting or key
order. The grammar article exception and glossary mishearing exception apply
consistently across their instructions.
@@ -72,3 +74,19 @@ consistently across their instructions.
Only the three active validator definitions are retained. The unused internal
`grammar_review` and `spoken_word_review` definitions and type selections were
removed; built-in chains and operator configuration are unchanged.
Repair guidance reassesses the proposal against the original transcript, context,
and findings. Unsupported, unnecessary, or unresolved uncertain corrections may
be withdrawn, preserving the original text. Otherwise acceptable minor editorial
cleanup remains permissible during repair.
The meaning-reversal validator has a deliberately narrow remit: reversal of the
full segment's assertion, not every change of meaning. Its prompt distinguishes
changes of referent from reversal and asks rejection reasons to identify the
opposing assertions. Preserve its carefully tuned core instructions when editing;
other validators assess other criteria.
Prompt tests protect rendering with supplied data, resource selection, schemas,
and integration behavior. They do not freeze bundled prose with word/phrase
checks or snapshots. Review editorial intent directly; offline tests cannot
establish live-model editorial quality.

View File

@@ -21,7 +21,6 @@ Execution-class metadata:
Deterministic:
- `proposal_shape`
- `confidence_threshold`
- `original_text_presence`
- `non_empty_resulting_segment` — rejects edits that would leave the segment
blank, regardless of replacement-span length.
@@ -80,9 +79,7 @@ makes all requested candidates unavailable. Decisions and unavailable indices
map back to internal proposal identities; results are ordered by request identity
and satisfy strict internal cardinality before the runner advances
approved candidates. An unavailable candidate never advances as approved.
Invalid confidence in a decoded test double also makes only its candidate
unavailable; production schema validation still rejects out-of-range confidence
before reconciliation. A partially unavailable batch records one failed
A partially unavailable batch records one failed
operation with only those requested indices, while a successful batch with
ignored extras records a decision-filtering recovery. Fatal errors stop the
chain without returning unchecked candidates as approved. A local result builder