From d2eb5a5662815f85b2cd7b5840458906f1e68bf1 Mon Sep 17 00:00:00 2001 From: Eric Rakestraw Date: Wed, 11 Nov 2026 14:59:10 +0000 Subject: [PATCH] Record validator rejection feedback decision --- docs/roadmap/prompt-quality.md | 30 +++++++++++++++++++++++------- 1 file changed, 23 insertions(+), 7 deletions(-) diff --git a/docs/roadmap/prompt-quality.md b/docs/roadmap/prompt-quality.md index f3ef44b..a6b7d2b 100644 --- a/docs/roadmap/prompt-quality.md +++ b/docs/roadmap/prompt-quality.md @@ -2,7 +2,8 @@ **Updated:** 2026-11-11 -**Status:** Topic 1 resolved in ADR 0003; topics 2–6 remain for discussion. +**Status:** Topic 1 resolved in ADR 0003; topic 2 decided, with prompt updates +pending; topics 3–6 remain for discussion. Work through the remaining topics in order. Their recommendations are starting points for discussion, not newly adopted restrictions or guarantees. @@ -29,16 +30,31 @@ minor editorial cleanup is recorded in testing found that strict scope instructions also suppress desirable edits. Apply the accepted distinction throughout the remaining prompt work. -### 2. Make Rejection Reasons Useful for Repair +### 2. Make Rejection Reasons Useful for Repair — Decided Validator explanations now become repair feedback, but current instructions give little guidance about producing actionable reasons. -**Recommendation:** ask for a concise description of the specific change, the -applicable criterion, and relevant evidence, within the validator's remit. -For example: removing "might" turns a tentative statement into a definite one; -preserve the speaker's uncertainty. Start with the existing `reason` field. -Discuss examples and sufficient detail before considering schema expansion. +**Decision:** retain the existing `reason` field. Instruct validators to explain +rejections with a concise description of the specific problematic edit, the +applicable criterion, and relevant supplied evidence, within their own remit. +For example: removing "might" turns a tentative statement into a definite one. +Validators identify the defect; the repair module owns revision or withdrawal. +Respect the accepted minor editorial allowance, distinguish insufficient +evidence from an established defect, and include distinct material issues when +found without requiring an exhaustive analysis. + +Approvals require no explanation: instruct the model to return `reason: ""`. +The existing schema requires the field but permits an empty string, which the +response handler already accepts. This convention needs no schema or +response-handling changes. The rejection guidance is a prompt instruction; +this decision adds no runtime rule rejecting blank explanations. + +**Remaining work:** update the shared validation guidance and all three validator +definitions with representative rejection examples and the empty-reason approval +convention. Check rendered prompts and representative repair payloads to confirm +that rejection feedback identifies defects without prescribing replacement +corrections. Consider schema expansion only if a concrete need emerges. ### 3. Make Repair a Reconsideration of the Edit's Merits