Implement [Contextual Entity Grounding](contextual-entity-grounding.md) so D&D
LLM prompts return evidence-grounded contextual selectors while Notarius owns
canonical entity IDs and referential integrity. Preserve every durable D&D
artifact contract and remove opaque IDs only from model-visible inputs and
private model responses.
This plan is written for a gpt-5.6-terra coding agent. Implement the stages in
numeric order. Each stage is intentionally scoped to one implementation prompt
and must leave the repository buildable and its focused tests passing before
the next stage begins.
## Plan-Wide Decisions
Apply these decisions throughout every stage:
- Read `docs/development.md`, all files under `docs/policy/`, the feature
roadmap, and the focused implementation/tests named by the stage before
editing.
- Preserve the fixed pipeline, typed artifact boundaries, root `assets`
content-only rule, module ownership, PromptKit boundary, and evidence rules.
- Do not change the durable NPC-, item-, location-registry, or occurrence Go
types, JSON schemas, schema IDs, schema versions, media types, reference-slot
contracts, categories, or generated-reference compatibility.
- Keep the affected prompt and private response-schema identities at `v1`.
They are private pre-release transport contracts; their changed content
hashes provide the required compatibility boundary.
- Advance semantic policy identifiers exactly as directed in each stage. Do
not bump unrelated policy identifiers.
- A contextual name match uses the entity family's existing comparison policy,
never fuzzy matching. A model selection must resolve to exactly one supplied
record before a durable ID is attached.
- An invalid NPC, item, or location selection invalidates the complete
extraction operation. Do not silently drop one response record, accept a
partial artifact, or defer a known mapping failure to a later validator.
- Registry provenance remains grounding only. Only the current extraction
chunk's `source_refs` become occurrence evidence.
- Preserve prompt message order and cache controls unless a stage explicitly
directs otherwise. Edit only the selected module or shared assets; do not
rewrite unrelated shared prompt bytes.
- Preserve internal opaque IDs where deterministic code needs them. The rule
applies to material shown to the model or requested from it, not to maps,
fingerprints, checkpoints, diagnostics, or durable artifacts.
- Follow `docs/policy/testing.md`: test package-level behavior and meaningful
failure modes, not prompt prose, exact message counts, private helper
structure, or a repository-wide string-scanning change detector. All tests
remain deterministic, offline, and credential-free.
- Use `apply_patch` for edits, `gofmt` changed Go files, and preserve unrelated
worktree changes.
## Final Private Selector Contracts
These shapes are implementation requirements, not public artifact schemas.
### NPC occurrence response
Each response record contains exactly the required fields `name`, `kind`, and
`source_refs`. It does not contain `npc_id`. Notarius resolves `name` through
the normalized NPC registry and writes the matched registry record's `ID` and
canonical `Name` into the durable occurrence.
### Item occurrence response
Each response record contains the existing required `name`, `kind`,
`quantity`, `from`, `to`, and `source_refs` fields. It does not contain
`item_id`. Retain the current nullable representation and kind-specific
semantics. Notarius resolves `name` through the normalized item registry and
adds the matched `ID` and canonical `Name`.
### Location occurrence response
Each response record contains exactly the required fields `name`,
`registry_refs`, `kind`, and `source_refs`. `registry_refs` is always an array
of strict objects containing required integer `start_unit_id` and
`end_unit_id`; it may be empty.
- When `name` has one comparison-identity match in the supplied registry,
`registry_refs` must be empty and name resolution selects that record.
- When multiple registry records share the comparison identity,
`registry_refs` must equal one record's complete canonically ordered source
ranges with `source_id` removed.
- The model-facing registry projection uses the same `name` plus
`registry_refs` selector and adds a required `context` array. Unique-name
records project empty `registry_refs` and `context` arrays. The private
response does not reproduce `context`.
- Same-name records receive bounded identity context consisting of the ordered
source units covered by their registry ranges. Each context element is a
strict object with exactly the required fields `unit_id` (integer) and
`text` (string); do not expose the durable location ID, source ID, digest,
or a replacement token.
- Building same-name grounding validates that every registry range belongs to
and resolves against the current source. If two records still produce the
same contextual selector, grounding construction fails before the LLM call.
-`registry_refs` never flow into the durable occurrence's `source_refs`.
### Entity-reconciliation response
The shared response remains an object with required `duplicate_groups`.
Every group has required `members` and `canonical`. A member and the canonical
selection are strict contextual objects containing:
```json
{
"name": "Mira Thorn",
"source_refs": [
{"start_unit_id": 12, "end_unit_id": 12}
]
}
```
The candidate prompt input uses the same descriptor and contains no `key`.
`source_refs` is required and non-empty for every eligible candidate. The
shared helper may retain its existing `candidate-000001`-style keys strictly
inside Go state to preserve input-position mapping; those keys must never be
serialized into prompt input or accepted in the private response.
If two candidates produce an identical contextual descriptor, neither is
eligible for model-assisted reconciliation because the model cannot identify
them independently. Otherwise the helper converts returned descriptors to its
internal candidate keys before applying all existing unknown-member,
ineligible-member, duplicate-member, canonical-membership, overlap, retry, and
fallback rules.
## Stage 1: Convert NPC Occurrences To Name-Based Resolution
### Goal
Remove durable NPC IDs from the NPC-occurrence prompt and private response,
then resolve the model's contextual name deterministically without weakening
checkpoint identity or downstream validation.
### Work
1. Inspect:
-`assets/dnd/npc-occurrences/`;
-`internal/modules/dnd/extract/npcoccurrences/`;
-`internal/modules/dnd/npcs/registry/`;
- NPC-occurrence normalizer and validator checkpoint fingerprints; and
- their focused tests.
2. Change `dnd_npc_occurrences_llm.v1.json` so every occurrence requires only
`name`, `kind`, and `source_refs`, continues to reject unknown fields, and
no longer declares `npc_id`.
3. Revise the NPC-occurrence instructions to require a supplied canonical NPC
name and current-chunk evidence, with no instruction to copy or invent an
ID. Continue using the existing shared names-only NPC registry fragment and
preserve manifest order/cache controls.
4. Remove `NPCID` from the private `occurrenceResponse`. After canonicalizing
response evidence, resolve every response name with the existing
`npcregistry.Registry.Lookup` comparison-key lookup. On the first unknown
or non-unique selection, return an extractor-scoped mapping error and no
value. For a match, construct the durable occurrence with the registry
record's exact `ID` and canonical `Name`.
5. Change `mappingPolicy` to
`dnd.npc_occurrences.extract_mapping.v3`.
6. Stop passing `IdentityPromptInput()` to the LLM; use the existing
names-only `PromptInput()`.
7. Replace the misleading exported model-input API used only for identity
fingerprints: retain the unexported ordered `{id,name}` projection and its
digest, expose that value as `IdentityDigest() string`, remove
`IdentityPromptInput()`, and update NPC-occurrence extractor, normalizer,
invariant-validator, and registry-validator fingerprints to use
`IdentityDigest()`. The digest must still distinguish ID/name identity from
the names-only prompt projection.
8. Rewrite existing focused tests around observable behavior: rendered NPC
registry input is names-only; the private schema rejects `npc_id`; valid
names acquire the registry ID; comparison-equivalent names canonicalize;
unknown names fail the whole extraction; registry identity fingerprints
remain distinct and defensive; empty registries accept only empty model
results. Remove tests whose only purpose was requiring the model to return
exact ID/name pairs.
### Acceptance Criteria
- No NPC-occurrence LLM input or private response contains a durable NPC ID.
- Durable NPC occurrences still contain the exact registry ID/name pair.
- Mapping failures remain extractor failures eligible for the configured
pipeline retry behavior.
- Deterministic consumers still fingerprint the ordered registry identity,
while spells, combat turns, enemy events, and NPC occurrences share the
names-only model projection.
### Validation
```sh
go fmt ./internal/modules/dnd/npcs/registry ./internal/modules/dnd/extract/npcoccurrences ./internal/modules/dnd/normalize/npcoccurrences ./internal/modules/dnd/validate/npcoccurrences/...
go test ./internal/modules/dnd/npcs/registry ./internal/modules/dnd/extract/npcoccurrences ./internal/modules/dnd/normalize/npcoccurrences ./internal/modules/dnd/validate/npcoccurrences/...
```
This stage is suitable for one gpt-5.6-terra prompt.
## Stage 2: Convert Item Occurrences To Name-Based Resolution
### Goal
Give item occurrences the same contextual-name/deterministic-ID boundary while
preserving item-specific nullable fields and kind rules.
### Work
1. Inspect `assets/dnd/item-occurrences/`, the item occurrence extractor, the
item registry, the item occurrence normalizer and registry validator, and
their focused tests.
2. Change `dnd_item_occurrences_llm.v1.json` to remove `item_id` from required
fields and properties. Preserve required `name`, `kind`, `quantity`, `from`,
`to`, and `source_refs`, all current enums/nullability, and strict unknown
field rejection.
3. Rewrite the item registry fragment and module instructions to require the
supplied canonical name and current-chunk evidence without mentioning an
ID. Preserve prompt order and cache controls.
4. Change the item registry's model projection from ordered `{id,name}` pairs
to ordered names-only objects, add a comparison-key index, and expose a
defensive `Lookup(name) (dnd.Item, bool)` analogous to the NPC registry.
Retain exact `LookupID` for durable normalizers and validators. Because item
IDs are derived from the item comparison identity, the names-only
`ProjectionDigest` remains sufficient for model input and existing
checkpoint consumers.
5. Remove `ItemID` from the private response. During response canonicalization,
resolve every contextual name, replace it with the registry record's
canonical name, and attach its durable ID when constructing the final
`dnd.ItemOccurrence`. Unknown selections fail the complete extraction; do
not alter evidence or nullable-field validation ownership.
6. Change `mappingPolicy` to
`dnd.item_occurrences.extract_mapping.v2`.
7. Update focused tests to cover names-only projection, defensive comparison
lookup, schema rejection of `item_id`, deterministic durable mapping,
unknown-name failure after an otherwise valid record, empty registry/result
behavior, and preservation of nullable/kind-specific fields.
### Acceptance Criteria
- Model-visible item registry and response content contain no item hash.
- Every accepted durable occurrence has the matched registry ID and canonical
name.
- Invalid selection remains all-or-nothing, and existing normalizer/validator
defense in depth remains unchanged.
### Validation
```sh
go fmt ./internal/modules/dnd/items/registry ./internal/modules/dnd/extract/itemoccurrences
go test ./internal/modules/dnd/items/registry ./internal/modules/dnd/extract/itemoccurrences ./internal/modules/dnd/normalize/itemoccurrences ./internal/modules/dnd/validate/itemoccurrences/...
```
This stage is suitable for one gpt-5.6-terra prompt.
## Stage 3: Add Contextual Location Grounding
### Goal
Replace the location registry's ID/name prompt projection with an immutable,
source-aware grounding object that can represent same-name locations safely.
Introduce the new path alongside the old occurrence input so this stage remains
buildable; Stage 4 performs the atomic extractor cutover and removes the old
path.
### Work
1. Inspect the location registry, location identity and source-reference
helpers, the generic source document index, occurrence checkpoint consumers,
and their focused tests.
2. In `internal/modules/dnd/locations/registry`, define the private-model
types needed by both grounding and the location occurrence extractor:
- a returned selector with exactly `name` and `registry_refs`;
- a registry projection entry with exactly `name`, `registry_refs`, and
`context`;
- a source-free range with exactly `start_unit_id` and `end_unit_id`; and
- a context unit with exactly `unit_id` and `text`.
All fields are required in their private JSON shapes, and constructors and
accessors must make defensive copies.
3. Add an operation-scoped immutable grounding type constructed from a resolved
registry and the current `*source.SourceDocument`. Its API must provide:
- a cloned `contracts.LLMInputMaterial` for the `location_registry` slot;
- deterministic resolution of a returned selector to one cloned
collisions, empty registries, and identity fingerprint stability. Do not
assert large rendered prompt strings; decode the JSON projection and assert
its semantic shape.
### Acceptance Criteria
- The location package can build and resolve contextual selectors without
exposing `location_id`, `source_id`, digests, or replacement labels.
- Same-name locations remain distinct and receive meaningful bounded context.
- Deterministic checkpoint consumers retain an ID-sensitive fingerprint.
- Only the existing occurrence extractor remains wired to the legacy
ID-bearing prompt path until Stage 4; the repository compiles and tests pass.
### Validation
```sh
go fmt ./internal/modules/dnd/locations/registry ./internal/modules/dnd/normalize/locationoccurrences ./internal/modules/dnd/validate/locationoccurrences/...
go test ./internal/modules/dnd/locations/registry ./internal/modules/dnd/normalize/locationoccurrences ./internal/modules/dnd/validate/locationoccurrences/...
```
This stage is suitable for one gpt-5.6-terra prompt.
## Stage 4: Convert Location Occurrences To Contextual Resolution
### Goal
Wire the Stage 3 grounding object into location occurrence extraction and
remove durable location IDs from the prompt and private response.
### Work
1. Inspect `assets/dnd/location-occurrences/`, the location occurrence model,
schema loader, extractor, canonicalization, prompt tests, and Stage 3
grounding tests.
2. Change `dnd_location_occurrences_llm.v1.json` so each occurrence requires
exactly `name`, `registry_refs`, `kind`, and `source_refs`; remove
`location_id`. Keep all four occurrence kinds. Make `registry_refs` a
required array, including an empty array, of strict required positive
integer ranges. Keep occurrence `source_refs` separate and unchanged.
3. Rewrite `location-registry.md` and module instructions to explain the two
same-name selection, failure for an unsupported ambiguous mention,
partial/reordered ranges, no registry-to-occurrence evidence leakage,
all-or-nothing failure, empty registry/result behavior, and unchanged
durable ordering/deduplication.
### Acceptance Criteria
- The location prompt and private response contain no durable location ID.
- Unique and same-name records resolve according to the final selector
contract.
- Accepted durable output is unchanged in shape and still contains an exact
location ID/name pair.
- Registry context cannot become durable occurrence evidence.
### Validation
```sh
go fmt ./internal/modules/dnd/extract/locationoccurrences ./internal/modules/dnd/locations/registry
go test ./internal/modules/dnd/locations/registry ./internal/modules/dnd/extract/locationoccurrences ./internal/modules/dnd/normalize/locationoccurrences ./internal/modules/dnd/validate/locationoccurrences/...
```
This stage is suitable for one gpt-5.6-terra prompt.
## Stage 5: Replace Reconciliation Keys With Contextual Descriptors
### Goal
Change the shared NPC/item/location registry-normalization proposal contract so
opaque candidate keys remain internal and the model sees and returns only
membership, invalid structured-output fallback, currency safety, and
preservation of every non-applied deterministic candidate. Update prompt
asset fixtures to the new selector schema; do not snapshot prompt prose.
### Acceptance Criteria
- No registry normalization prompt input or private response contains a
`candidate-*` key.
- Internal keys remain inaccessible to the model but may still support safe
deterministic position mapping.
- All existing normalizer safety, retry, fallback, warning, currency, and
same-name-location policies remain intact.
- Identical contextual descriptors cannot be arbitrarily reconciled.
### Validation
```sh
go fmt ./internal/modules/dnd/shared/entityreconcile ./internal/modules/dnd/normalize/npcregistry ./internal/modules/dnd/normalize/itemregistry ./internal/modules/dnd/normalize/locationregistry
go test ./internal/modules/dnd/shared/entityreconcile ./internal/modules/dnd/normalize/npcregistry ./internal/modules/dnd/normalize/itemregistry ./internal/modules/dnd/normalize/locationregistry
```
This is the largest stage, but it is one cohesive shared-contract migration
and is suitable for one gpt-5.6-terra prompt when implemented exactly within
the listed packages. Do not combine it with occurrence or documentation work.
## Stage 6: Record The Decision And Update Canonical Documentation
### Goal
Document the implemented policy in its durable architectural, internal, and
integration homes without duplicating volatile details or presenting roadmap
