Improve prompt roles and correction repair context
This commit is contained in:
@@ -23,6 +23,11 @@ appends only that branch’s history, and checks the original prepared prefix
|
||||
before execution. The complete conversation is sent to the selected provider;
|
||||
provider rejection ends that branch as unavailable. The requested repair
|
||||
definition is retained in call attribution alongside native base-prompt metadata.
|
||||
The runner supplements each repair instruction with its current proposal preview
|
||||
as described in the [module guide](modules.md#failure-and-outcome-behavior).
|
||||
The bundled prompts separate durable system principles from the initial user
|
||||
assignment so later repair instructions can allow acceptable minor cleanup;
|
||||
see [prompt instruction ownership](prompts.md).
|
||||
|
||||
The observation collector tracks logical calls and execution events. Ordinary
|
||||
reports use compact safe outcomes; request and response evidence is optional
|
||||
|
||||
@@ -68,6 +68,13 @@ Each actionable rejection can be revised or withdrawn within its original
|
||||
section. A revision retains the original proposal identity and must pass the
|
||||
full validator chain before application.
|
||||
|
||||
Before each repair attempt, the runner derives the current proposal's full
|
||||
before/after text with the same domain preview used by validation. It refreshes
|
||||
the view after revisions, including changes of target segment. A failed preview
|
||||
retains available original text and findings without supplying a resulting
|
||||
segment. This context supplements the original exchange and does not mutate the
|
||||
transcript. The model-facing fields are described in the [prompt guide](prompts.md).
|
||||
|
||||
Module construction failures are fatal. Recoverable proposal model failures
|
||||
record an unsuccessful operation and yield no candidates for that section;
|
||||
other sections and module instances continue. Fatal adapter or application
|
||||
|
||||
@@ -24,6 +24,13 @@ when maintaining proposal, validation, and repair instructions; the difference
|
||||
alone is not a reason to tighten validation. The testing rationale is recorded in
|
||||
[ADR 0003](../adr/0003-module-focus-and-editorial-acceptance.md).
|
||||
|
||||
Bundled proposal system messages carry durable correction principles and the
|
||||
editorial allowance during repair. The initial user message carries each
|
||||
module's focused proposal assignment and exclusions. Repair can therefore give
|
||||
a subsequent reassessment assignment without inheriting a system prohibition
|
||||
on acceptable incidental cleanup. Review complete conversations when editing
|
||||
these roles, including the transition from a correction set to one repair result.
|
||||
|
||||
For a standard request preview or validation, select a native PromptKit profile
|
||||
and supply the template variables used by the selected prompt. The shipped
|
||||
proposal definitions use `TranscriptDescriptionBlock`, `GlossaryJSON`,
|
||||
@@ -33,8 +40,12 @@ spelling list to `[]` when omitted. The validator definitions use `TranscriptDes
|
||||
`PayloadJSON`. The latter is supplied candidate JSON and does not require a
|
||||
proposal inference call. Repair definitions use `RepairContract` and
|
||||
`RepairRequestJSON`, containing the original proposal reference, current
|
||||
correction, and ordered findings. An operator replacement may use different
|
||||
variables.
|
||||
correction, and ordered findings. When the target exists, the request also
|
||||
includes `original_segment_text`; when the correction is applicable, it includes
|
||||
`corrected_segment_text`. These fields describe a preview of the current version,
|
||||
not an applied change. An inapplicable correction omits the resulting text and
|
||||
retains its findings and available source text. An operator replacement may use
|
||||
different variables.
|
||||
See the [upstream CLI reference](https://forge.maximumdirect.net/base-layer/commandkit/src/commit/f369d6a78253b6e6fcb1e023fb2e9c3af1d20ff0/docs/cli.md)
|
||||
for the standard preview, validation, and smoke commands.
|
||||
|
||||
@@ -60,6 +71,10 @@ pass these values through the role-bound adapter. See the
|
||||
|
||||
Proposal instructions describe literal matching, each module's replacement policy,
|
||||
and source-based proposal identity.
|
||||
They request independently useful edits and a combined proposal when edits
|
||||
overlap or depend on one another, while allowing independent edits to share a
|
||||
segment. This is generation guidance; application still uses the existing
|
||||
ordering and applicability checks.
|
||||
Proposal `id` and validator `correction_index` values are short, one-based
|
||||
references scoped to each request, not internal transcript/proposal identities.
|
||||
Mapping belongs to [proposal generation](modules.md) and
|
||||
@@ -80,6 +95,10 @@ and findings. Unsupported, unnecessary, or unresolved uncertain corrections may
|
||||
be withdrawn, preserving the original text. Otherwise acceptable minor editorial
|
||||
cleanup remains permissible during repair.
|
||||
|
||||
Examples contrast meaningful apparent filler with removable hesitation and show
|
||||
both revision and withdrawal. Treat their words as contextual illustrations,
|
||||
not unconditional triggers for deletion or replacement.
|
||||
|
||||
The meaning-reversal validator has a deliberately narrow remit: reversal of the
|
||||
full segment's assertion, not every change of meaning. Its prompt distinguishes
|
||||
changes of referent from reversal and asks rejection reasons to identify the
|
||||
|
||||
@@ -131,8 +131,10 @@ matched entries in selection order for each candidate, including matches in the
|
||||
replacement text. The deterministic guard and application check share the
|
||||
stage's separate protection matcher.
|
||||
|
||||
Each candidate includes the immediately preceding and following source segments
|
||||
available within its fixed chunk, labeled `preceding` or `following` with speaker,
|
||||
Each candidate carries its target segment's supplied `speaker`, including an
|
||||
empty value when no speaker is provided. It also includes the immediately
|
||||
preceding and following source segments available within its fixed chunk,
|
||||
labeled `preceding` or `following` with speaker,
|
||||
text, and categories, plus glossary entries whose names, aliases, or
|
||||
recognized plurals appear in its original/resulting text or those neighbors.
|
||||
Glossary matching uses the same recognition rules; entries retain selection order.
|
||||
@@ -143,8 +145,9 @@ already clear from its own segment does not require such evidence. No audio,
|
||||
out-of-chunk context, or complete transcript is implied.
|
||||
|
||||
Context is serialized inside the validation item and included in the existing
|
||||
payload token estimate. If one enriched item exceeds the configured limit, optional
|
||||
neighbors are removed from the end first, then glossary entries from the end,
|
||||
payload token estimate, including the target speaker. If one enriched item exceeds
|
||||
the configured limit, optional neighbors are removed from the end first, then
|
||||
glossary entries from the end,
|
||||
until it fits or only required candidate data remains. Required before/after text
|
||||
is never truncated. Existing oversized-candidate handling still applies if that
|
||||
required data does not fit. The built-in estimator reuses encoded fragment
|
||||
|
||||
@@ -2,11 +2,11 @@
|
||||
|
||||
**Updated:** 2026-11-11
|
||||
|
||||
**Status:** Accepted; implementation has not begun. This roadmap replaces the
|
||||
completed prompt-quality plan with the recommendations from the fresh review of
|
||||
revision `d155bf4`. The review covered all eleven prompt definitions, shared
|
||||
instructions, response schemas, and request assembly. It identified design
|
||||
concerns, not measured live-model regressions.
|
||||
**Status:** All six stages implemented and verified. Broader context changes and
|
||||
live-model editorial evaluation remain deferred as described below. The plan
|
||||
followed the review of revision `d155bf4`, which covered all eleven prompt
|
||||
definitions, shared instructions, response schemas, and request assembly.
|
||||
That review identified design concerns, not measured live-model regressions.
|
||||
|
||||
## Target State and Boundaries
|
||||
|
||||
@@ -173,6 +173,20 @@ establish live-model editorial quality.
|
||||
|
||||
## Deferred Evidence and Follow-Up
|
||||
|
||||
The evidence review examined the maintained local, campaign, and prompt-edit
|
||||
transcript examples, adjacent-context validation fixtures, and proposal
|
||||
application fixtures. The glossary example is a single-segment correction;
|
||||
the validation fixture supplies the troupe definition and adjacent evidence
|
||||
needed to assess its proposed name change. These sources contain no recorded
|
||||
live-model rejection demonstrating a need for more distant context. The target
|
||||
speaker is now included; broader selection and context fitting remain unchanged.
|
||||
Revisit the window when a captured case shows material evidence in the proposal
|
||||
request that was absent from validation.
|
||||
|
||||
The existing stale-span application fixture demonstrates the mechanical overlap
|
||||
addressed by the new guidance. It does not establish a need for new combined
|
||||
semantic validation, so application ordering and rejection rules remain unchanged.
|
||||
|
||||
Formal evaluation corpus preparation and live-model comparisons remain deferred
|
||||
and are not prerequisites for the agreed changes. Broader validation context and
|
||||
additional handling of combined semantic effects should follow concrete evidence
|
||||
@@ -182,3 +196,34 @@ repair outcomes using independent editorial judgment. Keep live calls opt-in
|
||||
and separate from the default offline suite.
|
||||
|
||||
No blocking product decisions remain for the implementation scope above.
|
||||
|
||||
## Completion and Verification
|
||||
|
||||
- The four proposal definitions now separate durable system principles from
|
||||
their initial user assignments. The shared proposal guidance covers overlapping
|
||||
and dependent edits; the grammar prompt illustrates a combined edit.
|
||||
- Every repair attempt receives the current target's original text when
|
||||
available and its previewed result when applicable, using the same domain
|
||||
replacement function as validation. Revision and withdrawal examples accompany
|
||||
the shared repair guidance.
|
||||
- Validation includes the target speaker in rendered input and token accounting.
|
||||
Spoken-word proposal and editorial review examples distinguish meaningful
|
||||
apparent filler from removable hesitation. The meaning-reversal definition and
|
||||
all response schemas retain their existing contracts.
|
||||
- Behavioral tests cover repair deletion and replace-all previews, ambiguous or
|
||||
missing spans, blank results, missing targets, and refresh after retargeting.
|
||||
Existing validation tests now cover target-speaker propagation, absence, and
|
||||
budget fitting, including parity with serialized payload estimates.
|
||||
- The affected prompt, module, request-assembly, and conversation tests were
|
||||
reviewed for wording checks. No further bundled-prose change detectors were
|
||||
found to remove; the earlier cleanup remains effective. Retained assertions
|
||||
protect supplied-data rendering, schema selection, operator replacement,
|
||||
conversation history, revalidation, withdrawal, and outcomes. New assertions
|
||||
check request data and behavior rather than prompt phrases.
|
||||
- All eleven definitions prepared through the CLI. Four illustrative complete
|
||||
conversations, including subsequent repair requests, were reviewed, and the
|
||||
sixteen embedded JSON response examples passed schema checks. The application
|
||||
build and full offline Go suite passed. Canonical documentation and maintained
|
||||
preview examples were updated; links and whitespace were checked.
|
||||
|
||||
These checks verify implementation and editorial intent, not live-model quality.
|
||||
|
||||
Reference in New Issue
Block a user