Remove confidence scoring and refine correction prompts
This commit is contained in:
@@ -44,7 +44,7 @@ Each stage prepares the encoded glossary context and protected spellings once.
|
||||
Proposal generation adds the selected transcript section and trimmed description
|
||||
for each request. PromptKit renders the selected definition and validates the
|
||||
correction-set schema before Audita maps proposals. Domain validators still
|
||||
check confidence, applicability, protected glossary terms, and replacement
|
||||
check applicability, protected glossary terms, and replacement
|
||||
policy. Model-facing segment IDs are one-based positions within the proposal
|
||||
request. Generation maps returned references back to the current chunk's actual
|
||||
segment IDs, preserving response order and multiple proposals per segment.
|
||||
@@ -76,6 +76,11 @@ outcomes are defined by the [report contract](../integrations/output-files.md).
|
||||
|
||||
## Changing Modules
|
||||
|
||||
Frequently reevaluate the default module combination and sequence against what
|
||||
has been observed to work best for most workloads. Prior empirical results inform
|
||||
the current choice without fixing a permanent number or order of passes. Maintain
|
||||
the selected default in the [configuration contract](../config.md#pipeline-and-glossary-selections).
|
||||
|
||||
1. Add or update the narrowly scoped package under `internal/modules/<module_key>`.
|
||||
2. Keep module prompt selection in the module and its definition in
|
||||
`internal/prompts`; see [prompt assets](prompts.md).
|
||||
|
||||
@@ -59,12 +59,14 @@ pass these values through the role-bound adapter. See the
|
||||
[configuration](../config.md) for operator-selected profiles.
|
||||
|
||||
Proposal instructions describe literal matching, each module's replacement policy,
|
||||
source-based proposal identity, and confidence in whether an edit is warranted.
|
||||
and source-based proposal identity.
|
||||
Proposal `id` and validator `correction_index` values are short, one-based
|
||||
references scoped to each request, not internal transcript/proposal identities.
|
||||
Mapping belongs to [proposal generation](modules.md) and
|
||||
[validator reconciliation](validators.md). Validator instructions distinguish
|
||||
confidence in the verdict from the verdict itself. Compact output examples
|
||||
[validator reconciliation](validators.md). Validator decisions use `approved` for the verdict. Approval examples use an empty
|
||||
`reason`; rejection guidance requests a specific defect and supplied evidence
|
||||
within the validator's remit. The model explains defects; repair owns revision or
|
||||
withdrawal. This is prompt guidance, not a runtime nonempty-reason check. Compact output examples
|
||||
illustrate the supplied schema without requiring fixed JSON formatting or key
|
||||
order. The grammar article exception and glossary mishearing exception apply
|
||||
consistently across their instructions.
|
||||
@@ -72,3 +74,19 @@ consistently across their instructions.
|
||||
Only the three active validator definitions are retained. The unused internal
|
||||
`grammar_review` and `spoken_word_review` definitions and type selections were
|
||||
removed; built-in chains and operator configuration are unchanged.
|
||||
|
||||
Repair guidance reassesses the proposal against the original transcript, context,
|
||||
and findings. Unsupported, unnecessary, or unresolved uncertain corrections may
|
||||
be withdrawn, preserving the original text. Otherwise acceptable minor editorial
|
||||
cleanup remains permissible during repair.
|
||||
|
||||
The meaning-reversal validator has a deliberately narrow remit: reversal of the
|
||||
full segment's assertion, not every change of meaning. Its prompt distinguishes
|
||||
changes of referent from reversal and asks rejection reasons to identify the
|
||||
opposing assertions. Preserve its carefully tuned core instructions when editing;
|
||||
other validators assess other criteria.
|
||||
|
||||
Prompt tests protect rendering with supplied data, resource selection, schemas,
|
||||
and integration behavior. They do not freeze bundled prose with word/phrase
|
||||
checks or snapshots. Review editorial intent directly; offline tests cannot
|
||||
establish live-model editorial quality.
|
||||
|
||||
@@ -21,7 +21,6 @@ Execution-class metadata:
|
||||
|
||||
Deterministic:
|
||||
- `proposal_shape`
|
||||
- `confidence_threshold`
|
||||
- `original_text_presence`
|
||||
- `non_empty_resulting_segment` — rejects edits that would leave the segment
|
||||
blank, regardless of replacement-span length.
|
||||
@@ -80,9 +79,7 @@ makes all requested candidates unavailable. Decisions and unavailable indices
|
||||
map back to internal proposal identities; results are ordered by request identity
|
||||
and satisfy strict internal cardinality before the runner advances
|
||||
approved candidates. An unavailable candidate never advances as approved.
|
||||
Invalid confidence in a decoded test double also makes only its candidate
|
||||
unavailable; production schema validation still rejects out-of-range confidence
|
||||
before reconciliation. A partially unavailable batch records one failed
|
||||
A partially unavailable batch records one failed
|
||||
operation with only those requested indices, while a successful batch with
|
||||
ignored extras records a decision-filtering recovery. Fatal errors stop the
|
||||
chain without returning unchecked candidates as approved. A local result builder
|
||||
|
||||
Reference in New Issue
Block a user