Preserve narrowly tuned meaning reversal validation
This commit is contained in:
@@ -2,12 +2,9 @@
|
||||
|
||||
**Updated:** 2026-11-11
|
||||
|
||||
**Status:** Topic 1 resolved in ADR 0003; topics 2–4 decided, with implementation
|
||||
pending; topic 5 deferred; grammar naming/scope, glossary/homophone overlap,
|
||||
and language/examples in topic 6 decided, with description and example updates
|
||||
pending; meaning-reversal policy awaits discussion.
|
||||
Work through the remaining topics in order. Their recommendations are starting
|
||||
points for discussion, not newly adopted restrictions or guarantees.
|
||||
**Status:** Topic 1 resolved in ADR 0003; topics 2–4 and 6 decided, with
|
||||
implementation pending; topic 5 deferred. No open decisions remain in the
|
||||
current scope. Topic 5 retains recommendations for future discussion.
|
||||
|
||||
## Baseline and Intended Direction
|
||||
|
||||
@@ -50,7 +47,8 @@ little guidance about producing actionable reasons.
|
||||
**Decision:** retain the existing `reason` field. Instruct validators to explain
|
||||
rejections with a concise description of the specific problematic edit, the
|
||||
applicable criterion, and relevant supplied evidence, within their own remit.
|
||||
For example: removing "might" turns a tentative statement into a definite one.
|
||||
For editorial review, for example: removing "might" turns a tentative statement
|
||||
into a definite one.
|
||||
Validators identify the defect; the repair module owns revision or withdrawal.
|
||||
Respect the accepted minor editorial allowance, distinguish insufficient
|
||||
evidence from an established defect, and include distinct material issues when
|
||||
@@ -125,6 +123,11 @@ contradictions and unnecessary repetition, and keep each rendered prompt
|
||||
understandable on its own. Retain repetition across separate model calls when
|
||||
each call needs the instruction.
|
||||
|
||||
Apply the meaning-reversal decision in topic 6 as an exception to broad prompt
|
||||
simplification: preserve that validator's tuned core instructions and make only
|
||||
small, targeted changes to feedback and examples, alongside the agreed confidence
|
||||
removal and approval-reason convention.
|
||||
|
||||
Use short contextual contrasts to clarify difficult judgments, such as an
|
||||
accidental restart versus deliberate emphasis, or a glossary match supported by
|
||||
context versus a merely similar spelling. Examples should illustrate how context
|
||||
@@ -167,8 +170,8 @@ test suite; no numerical quality gate is selected here.
|
||||
|
||||
### 6. Revisit Broader Product Choices
|
||||
|
||||
Use the preceding decisions and any available evidence to discuss these in order,
|
||||
without waiting for the deferred evaluation work:
|
||||
The following decisions guide this pass without waiting for the deferred
|
||||
evaluation work:
|
||||
|
||||
1. **Grammar naming and scope — Decided:** retain the `grammar` name, module key,
|
||||
and corresponding prompt IDs. Clarify its description as conservative
|
||||
@@ -197,9 +200,20 @@ without waiting for the deferred evaluation work:
|
||||
during the prompt editing pass. Use broadly applicable wording for names,
|
||||
terminology, and glossary entries, without asserting D&D specificity or
|
||||
adding a disclaimer about it. Broader language support is outside this pass.
|
||||
4. **Meaning-reversal policy:** the guard can also exclude a genuine correction
|
||||
of a mistranscription. Initially retain the conservative default while
|
||||
explicitly evaluating that tradeoff.
|
||||
|
||||
The remaining meaning-reversal item is a product discussion, not a selected
|
||||
change to validation policy.
|
||||
4. **Meaning-reversal policy — Decided:** retain the narrow guard against reversal
|
||||
of the full segment's assertion, rather than rejection of any change in
|
||||
meaning. The maintainer has observed that overly broad instructions cause this
|
||||
validator to reject virtually all corrections; its wording requires unusually
|
||||
precise tuning. Preserve the existing core instructions and make small,
|
||||
targeted changes, principally improving rejection feedback and adding a
|
||||
contrasting example. Changes to names, referents, or other information are
|
||||
not grounds for rejection by this validator unless they reverse the assertion:
|
||||
"We met Alice" becoming "We met Alex" changes meaning but is not a reversal.
|
||||
Assess the full before/after segment; the presence of an antonym in a
|
||||
replacement does not by itself establish reversal. Instruct rejection feedback
|
||||
to identify the specific opposition in the assertions; "this changes the
|
||||
meaning" is insufficient. Approve corrections outside this narrow rejection
|
||||
criterion, leaving other validators to assess their own criteria. These are
|
||||
prompt instructions, not new runtime checks on explanation text. Preserve the
|
||||
safeguard even though it can exclude a genuine mistranscription correction
|
||||
that reverses the assertion; no contextual exception is selected in this pass.
|
||||
|
||||
Reference in New Issue
Block a user