Defer editorial quality evaluation
This commit is contained in:
@@ -3,7 +3,7 @@
|
||||
**Updated:** 2026-11-11
|
||||
|
||||
**Status:** Topic 1 resolved in ADR 0003; topics 2–4 decided, with implementation
|
||||
pending; topics 5–6 remain for discussion.
|
||||
pending; topic 5 deferred; topic 6 remains for discussion.
|
||||
Work through the remaining topics in order. Their recommendations are starting
|
||||
points for discussion, not newly adopted restrictions or guarantees.
|
||||
|
||||
@@ -97,8 +97,8 @@ the existing repair lifecycle and revalidation.
|
||||
and revalidation continue to operate as intended.
|
||||
|
||||
Removing the proposal threshold can admit corrections it previously rejected.
|
||||
Use the evaluation work in topic 5 to assess that effect; removal is the selected
|
||||
direction, not conditional on a further confidence-policy decision.
|
||||
The deferred evaluation work in topic 5 can assess that effect when resumed;
|
||||
removal is the selected direction and does not depend on that work.
|
||||
|
||||
### 4. Simplify Instructions and Improve Examples — Decided
|
||||
|
||||
@@ -125,15 +125,21 @@ minor editorial cleanup. Shorter prompts alone are not the objective.
|
||||
fragments consistently with topics 1–3, retaining the existing module structure.
|
||||
Check rendering, response-schema compatibility, and integration behavior as
|
||||
appropriate. Proceed with these revisions before waiting for live-model
|
||||
evaluation evidence or a captured baseline. Topic 5 provides subsequent quality
|
||||
assessment and refinement; it is not a prerequisite for this editing pass.
|
||||
evaluation evidence or a captured baseline. Topic 5 is deferred and is not a
|
||||
prerequisite for this editing pass.
|
||||
|
||||
### 5. Evaluate Editorial Quality
|
||||
### 5. Evaluate Editorial Quality — Deferred
|
||||
|
||||
**Decision:** defer this work for now. Evaluation corpus preparation, live-model
|
||||
comparisons, and an evaluation workflow are outside the current implementation
|
||||
scope. This deferral does not block topics 2–4 or the remaining product
|
||||
discussions; ordinary implementation checks still apply. The following direction
|
||||
is retained for reconsideration when evaluation work resumes.
|
||||
|
||||
Rendering, schema, and controlled-response tests establish integration behavior,
|
||||
but do not establish the quality of model editorial judgments.
|
||||
|
||||
**Recommendation:** assemble a modest evaluation set containing both needed edits
|
||||
**Future recommendation:** assemble a modest evaluation set containing both needed edits
|
||||
and text that should remain unchanged. Include ambiguous glossary matches,
|
||||
intentional repetition, meaningful uses of apparent filler, punctuation that
|
||||
changes meaning, and repairs that should be withdrawn. Compare unnecessary edits,
|
||||
@@ -142,16 +148,16 @@ editorial assessment; validator approval alone is insufficient evidence.
|
||||
Include a comparison against the prior confidence-threshold behavior to identify
|
||||
changes in both useful corrections and erroneous edits after confidence removal.
|
||||
|
||||
Capture candidate examples during topics 1–4 where useful, without delaying the
|
||||
agreed prompt revisions. Discuss representative models, expected outcomes, and
|
||||
how to run evaluations as follow-up work. Use the prior repository revisions for
|
||||
When this work resumes, discuss representative models, expected outcomes, and
|
||||
how to run evaluations. Use the prior repository revisions for
|
||||
before/after comparisons when that work proceeds; a pre-change evaluation run is
|
||||
not required. Keep model-dependent evaluation separate from the default offline
|
||||
test suite; no numerical quality gate is selected here.
|
||||
|
||||
### 6. Revisit Broader Product Choices
|
||||
|
||||
Use the preceding decisions and evaluation evidence to discuss these in order:
|
||||
Use the preceding decisions and any available evidence to discuss these in order,
|
||||
without waiting for the deferred evaluation work:
|
||||
|
||||
1. **Grammar naming and scope:** the current module is close to copy editing,
|
||||
with an isolated article exception. Decide whether its name and scope convey
|
||||
|
||||
Reference in New Issue
Block a user