Record native migration integration gaps

This commit is contained in:
2026-11-22 09:00:55 +00:00
parent fd134e85e5
commit fdaf39162b

View File

@@ -6,12 +6,18 @@ Review date: 2026-11-22.
Reviewed commit: `75444b8ef300ca8efee57d7a2271d21ce258b49c`. Reviewed commit: `75444b8ef300ca8efee57d7a2271d21ce258b49c`.
Integration follow-up baseline: `fd134e85e54c6e0f74aab5c77df5bada7c1b1e6a`.
The follow-up baseline adds only the initial review findings; application code
is unchanged between these revisions.
This review followed the 46 stages in [the implementation plan](implementation.md) This review followed the 46 stages in [the implementation plan](implementation.md)
against the [native migration target](promptkit.md), current owning code, against the [native migration target](promptkit.md), current owning code,
focused tests, maintained examples, and canonical documentation. The migration focused tests, maintained examples, and canonical documentation. The migration
is substantially complete. The required follow-up is limited to the two findings is substantially complete. Findings 1–2 come from the initial review. Findings
below; neither calls for another compatibility layer or application execution 3–5 come from a second, stage-ordered pass through the interaction of the native
abstraction. This document records findings, not implemented fixes. migration with existing domain, validation, and diagnostic behavior. None calls
for another compatibility layer or application execution abstraction. This
document records findings, not implemented fixes.
## Finding 1: Retain Genuine Native Timeouts During Concurrent Error Selection ## Finding 1: Retain Genuine Native Timeouts During Concurrent Error Selection
@@ -127,6 +133,154 @@ retained.
prose-snapshot or mandatory-source-validation test is needed for this prose-snapshot or mandatory-source-validation test is needed for this
documentation correction. documentation correction.
## Finding 3: Rebind Cached Scene Identity To The Current Chunk Plan
Priority: P1. Relevant stages: 11, 23, 27–29, and 37.
### Evidence And Impact
The new message-content cache interacts incorrectly with the existing
position-derived scene identity:
- [`MaterializeChunkPlan`](../../internal/core/source/chunk_plan.go), lines
169–179, renders only source units into chunk content, while assigning
`chunk-000001`, etc. from the range's position in the plan.
- [Scene extraction](../../internal/modules/dnd/extract/scenedescriptions/extractor.go)
deterministically attaches the current chunk ID and range in `mapResponse`.
The model does not supply this identity, and the ID is not rendered into the
initial scene-description messages.
- [`artifactProducerCache`](../../internal/framework/pipeline/runner_cache.go),
lines 21–37, decodes a hit and checks cited source units, but does not rebind
its scene identity. The hit bypasses extraction interpretation and assessment.
Changing an earlier scene boundary can therefore renumber an unchanged later
scene without changing that later scene's prompt hash. Its old accepted artifact
is a legitimate content-key hit, but its stored scene ID belongs to the old plan.
A temporary native-CLI integration probe used real scene planning, extraction,
merging, normalization, default validators, and a filesystem cache. The chunk
binding used `cache: refresh`; scene extraction used ordinary `auto`. The fake
provider returned these two valid plans for the same three-unit transcript:
1. `[1–2], [3]`, followed by `[1], [2], [3]`: the second run failed normalization
with `scene ID "chunk-000002" has conflicting records`.
2. The reverse change: the second run succeeded but published IDs
`chunk-000001` and `chunk-000003` for a two-chunk plan. The last ID should have
been `chunk-000002`.
Both cases pass with invocation-wide cache bypass. The unchanged suffix was
reused in the failing cases, as confirmed by provider-call counts. This is not
an invalid generated plan or intentionally skipped semantic revalidation.
The stale ID also conflicts with the existing [scene registry's exact match](../../internal/modules/dnd/scenedescriptions/registry/registry.go):
combat/enemy consumers require both the current chunk ID and range, so a stale
scene can become unavailable classification and suppress downstream extraction.
### Required Change
Add the missing current-operation adaptation at the scene cache-hit boundary.
For a usable single-scene artifact describing the exact current range,
deterministically reattach the current chunk identity while retaining its accepted
kind/title/summary. If a cached artifact cannot be safely adapted, treat it as an
unusable miss before returning it as accepted work.
Keep domain-specific adaptation with the scene module, using the supplied typed
cache collaboration or a small extension to that boundary if needed. Preserve
the content-only native cache key and no-semantic-reassessment policy. Do not
render an internal scene ID into the prompt, add a partition fingerprint/key
dimension, restore canonical source-only chunking, or ask the model to copy IDs.
Review other current cached artifacts for equivalent operation-derived fields;
do not invent a generic identity framework for families that do not need one.
### Acceptance
- Repartitioning earlier units can move an unchanged later scene to a different
chunk ordinal without duplicate or stale scene IDs in the published artifact.
- Current scene IDs/ranges match the chunk map and remain usable by the existing
combat/enemy eligibility resolver.
- The unchanged scene can reuse its accepted semantic content without another
model call or validator assessment when deterministic adaptation is safe.
- Wrong-range, multi-scene, or otherwise unusable entries are not silently
rebound into valid-looking scenes; they miss normally.
- Add assembled two-run expansion/collapse coverage using the existing native
fake-provider/cache setup, and retain ordinary same-plan hit coverage.
## Finding 4: Bound Framework-Generated Fallback Diagnostic Messages
Priority: P2. Relevant stages: 9, 11, 24, 29, and 34.
### Evidence And Impact
[`FinishProducerFailure`](../../internal/framework/pipeline/producer_outcome.go)
retains the complete failure text in `ProducerOutcome.FallbackReason` after a
supported permissive fallback is successfully constructed.
[`operationDiagnosticGroups`](../../internal/framework/pipeline/runner_operation.go),
lines 51–58, copies that text unchanged into a framework-generated warning sample.
The existing [diagnostic contract](../../internal/framework/contracts/diagnostics.go)
requires valid UTF-8 and a maximum sample-message size of 4096 bytes.
Native errors and joined assessment errors are not bounded diagnostic samples.
Even individually bounded domain validator messages can exceed the sample limit
after joining several findings and adding validator context. The new fallback
bridge thus creates invalid diagnostics from otherwise valid operation outcomes.
A temporary offline native-CLI probe ran a minimal spell pipeline in permissive
mode with a fake provider returning a long ordinary error. The valid empty spell
fallback was constructed, but the run then failed with
`producer diagnostic 0: diagnostic sample 0: diagnostic sample message exceeds maximum length`.
The failure occurs even with ordinary visibility hiding optional warnings.
### Required Change
Render the framework's fallback warning into a bounded, valid-UTF-8 sample before
promoting it. Reuse or factor the small bounding behavior already present in
[`producerCacheObserver`](../../internal/framework/pipeline/runner_cache.go)
rather than adding a second diagnostic system.
Bound only the generated diagnostic sample, leaving the original error/outcome
text and requested evidence unchanged. This is operator diagnostic presentation,
not model correction guidance. Preserve validation of module-supplied diagnostics;
do not raise/remove the existing cap or suppress genuine malformed-module
contracts to work around the framework's own unbounded sample.
### Acceptance
- A supported permissive fallback still publishes qualified output when its
native or aggregated-assessment cause exceeds the diagnostic sample limit.
- The generated warning is nonblank, bounded, and valid UTF-8, including at a
multibyte truncation boundary.
- Strict handling, caller cancellation, fallback counting/reuse exclusion,
full requested evidence, and module-diagnostic validation remain unchanged.
- Add focused fallback-to-diagnostic integration coverage, including a real
typed producer path with a fake long native error. No model retry is added.
## Finding 5: Remove The Disconnected Legacy LLM Error Contract
Priority: P3. Relevant stages: 8, 24, 35, and 44.
### Evidence And Impact
[`contracts/errors.go`](../../internal/framework/contracts/errors.go) still
defines the pre-migration `ErrInvalidStructuredOutput`, `ErrLLMCapacityExceeded`,
`ErrLLMGeneration`, `LLMGenerationError`, constructors, and copied failure/attempt
metadata. Current producers and validators retain native PromptKit errors instead.
Graph/code searches find this old taxonomy only in that file and its
[self-test](../../internal/framework/contracts/errors_test.go), not in an active
application consumer.
This is leftover contract surface from the removed adapter, not a second active
execution path. It nevertheless presents obsolete sentinels and metadata accessors
as available framework conventions, although matching actual native failures
against them no longer works. Stage 35's hard-cutover cleanup is incomplete here.
### Required Change And Acceptance
Remove the unused legacy error definitions and their self-test. Keep native
`errors.Is`/`errors.As` behavior and the separate, actively used application
assessment/capture error types. Do not bridge native errors into these retired
sentinels or add aliases for compatibility. Verify no current callers remain and
run contracts, pipeline, CLI, and module integration tests after removal.
## Review Coverage And Verification ## Review Coverage And Verification
Owning boundaries were inspected in implementation order: Owning boundaries were inspected in implementation order:
@@ -158,6 +312,25 @@ Checks completed during review:
repository code or tests. Their deliberately failing assertions are review repository code or tests. Their deliberately failing assertions are review
evidence, separate from the passing default suite. evidence, separate from the passing default suite.
The integration follow-up inspected the stage boundaries above against the
retained source/chunk identity, domain codecs/validators, generated references,
diagnostic limits, physical publication, and subprocess contracts. It reused
the earlier full-suite/vet/build/race results only after confirming that the
application code and dependencies were unchanged. Additional checks were:
- `GOWORK=off GOPROXY=off go test ./internal/core/config
./internal/framework/pipeline ./internal/framework/semanticreconcile
./internal/modules/integration ./internal/cli` passed.
- Real native-CLI/fake-provider Go-overlay probes reproduced both directions of
scene repartitioning in Finding 3. Both corresponding bypass controls passed.
- A native-CLI/fake-provider overlay probe reproduced the long-error permissive
fallback failure in Finding 4.
- Graph/code searches confirmed that Finding 5's legacy taxonomy has no active
callers outside its own definition/self-test.
These probes live only in a temporary review directory. No production code,
test, configuration, schema, or existing canonical documentation was changed.
No live provider inference was performed. This review does not establish No live provider inference was performed. This review does not establish
production model quality or performance. Intentional changes such as the hard production model quality or performance. Intentional changes such as the hard
configuration/CLI cutover, native retry ownership, content-only cache identity, configuration/CLI cutover, native retry ownership, content-only cache identity,