Remove confidence scoring and refine correction prompts

This commit is contained in:
2026-11-11 17:05:52 +00:00
parent bbf89bb4f5
commit d155bf4cb3
92 changed files with 448 additions and 786 deletions

View File

@@ -2,7 +2,7 @@
**Updated:** 2026-11-11
**Status:** Decisions settled; implementation pending. Editorial evaluation is
**Status:** All six implementation stages completed. Editorial evaluation remains
deferred. No open decisions remain in the current scope.
## Target State and Constraints
@@ -215,6 +215,36 @@ are not a substitute for that review or for the deferred live-model evaluation.
Summarize the tests removed, revised, and retained with their behavioral rationale,
and run the checks appropriate to any test changes made in this final stage.
## Completion and Test Review
Confidence has been removed from schemas, models, validators, configuration,
CLI flags, feedback, and fixtures. All eleven prompt definitions and their shared
contracts have been revised. Current behavior and cutover instructions are
recorded in the canonical documentation linked above and the
[v3 migration guide](../migrations/v3.md#confidence-removal).
The final test-policy review made the following changes:
- Removed assertions about bundled instruction phrases, module-guidance wording,
and exact shared-instruction repetition. These detected prose changes without
demonstrating model behavior and obstructed prompt experimentation.
- Removed confidence-specific tests with the retired contract. Repair and report
scenarios now use substantive rejection cases, preserving their coverage of
revision, withdrawal, exhaustion, partial success, and failure handling.
- Revised subprocess model fixtures to identify structured validation input
without depending on a prose heading; removed an incidental schema-formatting
assertion.
- Retained supplied-data interpolation, operator prompt replacement, schema and
response handling, repair history, protected-term behavior, and end-to-end
outcomes. Test-controlled strings and machine-contract comparisons remain where
they detect meaningful defects.
The full offline Go test suite and application build passed. Prompt rendering,
response examples, documentation links, and whitespace were checked. Prompt
language was reviewed directly for the selected responsibilities and narrow
meaning-reversal criterion; these checks do not establish live-model editorial
quality.
## Deferred Editorial Evaluation
Evaluation corpus preparation, live-model comparisons, and an evaluation workflow