Preserve narrowly tuned meaning reversal validation

This commit is contained in:
2026-11-11 16:42:28 +00:00
parent 2712a3887f
commit d15d80c79f

View File

@@ -2,12 +2,9 @@
**Updated:** 2026-11-11
**Status:** Topic 1 resolved in ADR 0003; topics 2–4 decided, with implementation
pending; topic 5 deferred; grammar naming/scope, glossary/homophone overlap,
and language/examples in topic 6 decided, with description and example updates
pending; meaning-reversal policy awaits discussion.
Work through the remaining topics in order. Their recommendations are starting
points for discussion, not newly adopted restrictions or guarantees.
**Status:** Topic 1 resolved in ADR 0003; topics 2–4 and 6 decided, with
implementation pending; topic 5 deferred. No open decisions remain in the
current scope. Topic 5 retains recommendations for future discussion.
## Baseline and Intended Direction
@@ -50,7 +47,8 @@ little guidance about producing actionable reasons.
**Decision:** retain the existing `reason` field. Instruct validators to explain
rejections with a concise description of the specific problematic edit, the
applicable criterion, and relevant supplied evidence, within their own remit.
For example: removing "might" turns a tentative statement into a definite one.
For editorial review, for example: removing "might" turns a tentative statement
into a definite one.
Validators identify the defect; the repair module owns revision or withdrawal.
Respect the accepted minor editorial allowance, distinguish insufficient
evidence from an established defect, and include distinct material issues when
@@ -125,6 +123,11 @@ contradictions and unnecessary repetition, and keep each rendered prompt
understandable on its own. Retain repetition across separate model calls when
each call needs the instruction.
Apply the meaning-reversal decision in topic 6 as an exception to broad prompt
simplification: preserve that validator's tuned core instructions and make only
small, targeted changes to feedback and examples, alongside the agreed confidence
removal and approval-reason convention.
Use short contextual contrasts to clarify difficult judgments, such as an
accidental restart versus deliberate emphasis, or a glossary match supported by
context versus a merely similar spelling. Examples should illustrate how context
@@ -167,8 +170,8 @@ test suite; no numerical quality gate is selected here.
### 6. Revisit Broader Product Choices
Use the preceding decisions and any available evidence to discuss these in order,
without waiting for the deferred evaluation work:
The following decisions guide this pass without waiting for the deferred
evaluation work:
1. **Grammar naming and scope — Decided:** retain the `grammar` name, module key,
and corresponding prompt IDs. Clarify its description as conservative
@@ -197,9 +200,20 @@ without waiting for the deferred evaluation work:
during the prompt editing pass. Use broadly applicable wording for names,
terminology, and glossary entries, without asserting D&D specificity or
adding a disclaimer about it. Broader language support is outside this pass.
4. **Meaning-reversal policy:** the guard can also exclude a genuine correction
of a mistranscription. Initially retain the conservative default while
explicitly evaluating that tradeoff.
The remaining meaning-reversal item is a product discussion, not a selected
change to validation policy.
4. **Meaning-reversal policy — Decided:** retain the narrow guard against reversal
of the full segment's assertion, rather than rejection of any change in
meaning. The maintainer has observed that overly broad instructions cause this
validator to reject virtually all corrections; its wording requires unusually
precise tuning. Preserve the existing core instructions and make small,
targeted changes, principally improving rejection feedback and adding a
contrasting example. Changes to names, referents, or other information are
not grounds for rejection by this validator unless they reverse the assertion:
"We met Alice" becoming "We met Alex" changes meaning but is not a reversal.
Assess the full before/after segment; the presence of an antonym in a
replacement does not by itself establish reversal. Instruct rejection feedback
to identify the specific opposition in the assertions; "this changes the
meaning" is insufficient. Approve corrections outside this narrow rejection
criterion, leaving other validators to assess their own criteria. These are
prompt instructions, not new runtime checks on explanation text. Preserve the
safeguard even though it can exclude a genuine mistranscription correction
that reverses the assertion; no contextual exception is selected in this pass.