1625 lines
98 KiB
Markdown
1625 lines
98 KiB
Markdown
# Native Migration Implementation Plan
|
||
|
||
## Purpose And Execution Rules
|
||
|
||
Implement the complete target in [Native PromptKit And CommandKit Migration](promptkit.md).
|
||
That feature roadmap owns product choices, defaults, ownership, and acceptance
|
||
semantics. This document owns execution order and stage completion. Completion
|
||
markers below record finished stages. The inspected application baseline is Notarius commit
|
||
`b3664d587914b1d7b0377f25fa377b57fa97c815`; its recent roadmap commits have not
|
||
implemented the migration.
|
||
|
||
Each numbered stage is one implementation prompt for a Luna coding agent.
|
||
Execute stages in integer order as one end-to-end sprint. Intermediate stages
|
||
may leave callers, tests, or packages unbuildable. Do not spend prompts preserving
|
||
intermediate compilation with compatibility adapters, duplicate loaders, aliases
|
||
for retired inputs, or replacement completion abstractions. Final compilation,
|
||
behavior, and documentation must satisfy the entire target. The native `run`
|
||
alias is a selected new interface, not a legacy forwarding implementation.
|
||
|
||
Before each stage, read `docs/development.md`, follow its applicable policy links,
|
||
and inspect the stage's named source and focused tests. Prefer the codebase
|
||
knowledge graph for code discovery. Use the selected upstream public APIs and
|
||
maintained examples, not old dependency APIs inferred from current Notarius code.
|
||
Keep code and tests scoped to that stage. Commit each completed nonempty unit
|
||
under the agent policy, accurately recording any checks blocked by not-yet-migrated
|
||
callers. Do not claim a blocked test passed. Later stages own the corresponding
|
||
integration; do not omit their work because an earlier package check was blocked.
|
||
|
||
The validation commands below are focused checks, not additional permanent
|
||
release gates. Run them when their dependencies can compile; otherwise inspect
|
||
the affected code, record the concrete dependency blocker, and continue the
|
||
sprint. Stage 46 requires the complete assembled repository to pass. Test with
|
||
real kits and a fake `promptkit.LLMClient` or local fake HTTP provider, as appropriate.
|
||
No ordinary test requires real credentials, a production transcript, paid inference,
|
||
or an external service. Dependency acquisition is separate from offline testing.
|
||
|
||
No stage authorizes a release, push, tag, deployment, rewrite of published history,
|
||
or changes to sibling repositories. Do not delete legacy cache data on disk.
|
||
Do not add security restrictions, filesystem permission policy, schema changes,
|
||
backend limits, retry caps, or public reporting guarantees beyond the feature
|
||
roadmap. Retain deterministic domain correctness and the existing intrinsic
|
||
artifact schemas; retire only machinery and contracts the roadmap replaces.
|
||
|
||
## Settled Design And Implementation Boundaries
|
||
|
||
- Use PromptKit `v0.24.0`, ConfigKit `v1.4.0`, CommandKit `v0.7.0`, ckconfig
|
||
`v0.10.0`, and ckprompt `v0.11.0` from their `go.maximumdirect.net` paths.
|
||
Keep the Go consumer minimum `1.27.0`; use stable go-yaml v4 and selected
|
||
transitive versions from the feature roadmap.
|
||
- Version 6 is the sole application configuration version. One application-owned
|
||
ConfigKit result supplies native PromptKit settings, engine construction, and
|
||
both inspection scopes. Composition-root loaders live in `internal/cli`.
|
||
- Use ordinary `assets` embedding, native source contexts, and
|
||
`promptkit.collections.notarius`. Prompt declarations own default output
|
||
contracts. PromptKit owns schema preparation and structural repair.
|
||
- Keep typed stage interfaces. Model-executing code receives the shared native
|
||
engine. A small producer helper in `internal/framework/pipeline` accepts ordinary
|
||
module request/interpretation/correction/fallback functions and the pipeline's
|
||
typed assessment callback. No new LLM service interface or execution-plan language.
|
||
- The pipeline assesses each initial/revised candidate once through the whole
|
||
applicable validator chain. Native coordination owns growing conversation
|
||
history and turn iteration. `max_additional_turns` resolves binding, pipeline,
|
||
then `2`; explicit zero disables semantic correction. Native transient and
|
||
structural-repair defaults remain unspecified by Notarius.
|
||
- `mode` resolves producer binding, pipeline, then `strict`. A permissive fallback
|
||
is qualified output, not ordinary acceptance. Cancellation stops without fallback.
|
||
Fallback and affected dependent work skip cache lookup and storage.
|
||
- `concurrency.workers`, default `64`, bounds one shared ready-job pool for
|
||
extraction and lane completion. Native backend admission is independent; no
|
||
injected backend limits or replacement global LLM ceiling.
|
||
- Cache only final accepted artifacts/plans under the initial prepared
|
||
`MessageContentHash`. The key has no extra dimensions. Hits get ordinary decode/
|
||
usability checks and no new semantic assessment. `auto`, `bypass`, and `refresh`
|
||
use the roadmap's exact precedence; cache I/O is best effort.
|
||
- Publish domain artifacts at their existing run-specific paths. Report one small
|
||
native completion value, including a positive `fallback_count` when applicable.
|
||
Retire required manifests, receipts, index files, and reporting sidecars.
|
||
- Use one invocation `ExecutionGroup`, native observations, and optional convenient
|
||
debug evidence. Detailed debug is not a public serialization contract. Preserve
|
||
native credential omissions and ConfigKit secret-aware inspection; do not add
|
||
heuristic content rewriting.
|
||
|
||
Implementation details such as private helper names and debug filenames may be
|
||
refined without changing these choices. Put application outcome/callback types
|
||
where typed stage contracts can use them without a `contracts -> pipeline` import
|
||
cycle. Do not pass pipeline collaborators through hidden process globals. Modules
|
||
still receive no physical roots or CommandKit reporters.
|
||
|
||
### Domain Fallback Construction
|
||
|
||
Use this construction guidance in the corresponding module stages, under U6:
|
||
|
||
| Producer | Supported fallback construction |
|
||
| --- | --- |
|
||
| D&D scene chunker | One plan entry covering the complete source in document order; use normal materialization and source-addressing rules. |
|
||
| Spell, registry, occurrence, combat-turn, and enemy-event extraction | The family's codec-valid empty collection, retaining required top-level shape and any deterministically required context. No invented entity or event. |
|
||
| Scene-description extraction | Use the final usable candidate only when it satisfies intrinsic shape, source identity, and downstream consumption invariants. Its artifact requires at least one scene, so an empty list is not a valid fallback. If no such final candidate exists, fail fallback construction; do not fabricate kind/title/summary or change the schema. |
|
||
| Semantic registry normalization | Apply compatible safe groups from the final usable proposal afresh to fixed originals, preserving other originals separately. With no usable final proposal, use the deterministic merged input. |
|
||
| Deterministic merge/normalization | Reuse suitable incoming/merged data only when the module can construct an invariant-preserving result. An operation error or inability to construct that result fails normally; there is no generic retry. |
|
||
|
||
Necessary fallback checks belong to the module; they do not rerun the configured
|
||
ordinary validator chain. This table does not promise that every operation can
|
||
recover from every failure. Insufficient reconciliation candidates can remain
|
||
an ordinary deterministic success; exceeding a required semantic preparation
|
||
limit is not silently presented as normally validated model work.
|
||
|
||
## Stage 1: Select Dependencies And Update Their Notices
|
||
|
||
**Status:** Complete. Selected releases resolve together with `GOWORK=off go list
|
||
-m all`; Go remains `1.27.0`, with no replacements. Updated commit-pinned notices
|
||
for the selected kits and changed runtime dependencies. `git diff --check` passed.
|
||
Application API migration remains assigned to the subsequent stages; no application
|
||
compilation claim is made here.
|
||
|
||
**Goal:** make the reviewed upstream APIs available without a compatibility patch.
|
||
|
||
**Inspect/change:** `go.mod`, `go.sum`, `docs/oss/attribution.md`; selected kit
|
||
`go.mod`, `COPYRIGHT`, `LICENSE`, and release notes. Read the roadmap's dependency
|
||
and evidence sections. Consult the composed CLI example under
|
||
`../commandkit/integrations/ckprompt/examples/composed/` as a reference, not a new
|
||
Notarius dependency.
|
||
|
||
**Work:** add the five selected vanity-path dependencies, retain Go `1.27.0`,
|
||
and let Go resolve the reviewed stable YAML/jsonschema versions. Convert import
|
||
paths as their callers migrate; do not patch upstream or keep permanent `replace`
|
||
directives. Old-path dependencies may remain temporarily while old callers exist;
|
||
Stage 35 removes them. Add/update factual attribution with exact selected upstream
|
||
commit links and required license material under the OSS policy.
|
||
|
||
**Acceptance:** all five selected modules resolve together using declared
|
||
versions; no sibling workspace substitutes for a release. Notices describe actual
|
||
incorporated dependencies. API incompatibilities are identified as migration work,
|
||
not hidden by shims.
|
||
|
||
**Validation:** `GOWORK=off go list -m all`, inspect `git diff -- go.mod go.sum
|
||
docs/oss/attribution.md`, and `git diff --check`. Do not require the old application
|
||
to compile against the new APIs yet.
|
||
|
||
## Stage 2: Introduce The Native Version 6 Configuration Definition
|
||
|
||
**Status:** Complete. Native settings and composed rules retain the original
|
||
application result for direct engine handoff. Version 6 numeric/string loads,
|
||
version 5 rejection, application origins, native inheritance, imported message
|
||
origins, and secret-aware inspection are covered by the passing
|
||
`GOWORK=off go test ./internal/core/config`. `git diff --check` passed.
|
||
The old CLI compile check is blocked by old-path ConfigKit importing the removed
|
||
`go.yaml.in/yaml/v4/plugin/limit`; later caller migration and dependency removal
|
||
own that integration. Source assembly remains assigned to Stage 5.
|
||
|
||
**Goal:** preserve one typed ConfigKit load and its original result evidence.
|
||
|
||
**Inspect/change:** `internal/core/config/document.go`, `load.go`,
|
||
`loaded_config.go`, `document_normalizer.go`, `operational_document.go`, and their
|
||
definition/loading tests; PromptKit `Configure`, `Settings`, and `NewFromConfig`.
|
||
|
||
**Work:** replace `PromptKitDocument` with native `promptkit.Settings` at
|
||
`yaml:"promptkit"`, compose the application definition with `promptkit.Configure`,
|
||
and select application version 6 only. Retain the native `configkit.Result` for
|
||
handoff and inspection rather than reserializing settings. Preserve application
|
||
pipeline normalization, ConfigKit discovery/import/version semantics, and
|
||
source-relative application paths. Remove `profile_file`, `profile_dir`, and
|
||
`local_backend` consumption; add no separate PromptKit configuration file setting.
|
||
Runtime-only client options belong in native `RuntimeConfig`, and invocation
|
||
transient-retry overrides use ckprompt's native request override. Do not recreate
|
||
the removed configuration wrapper to hold library behavior.
|
||
|
||
**Acceptance:** `version: 6` and its native string equivalent follow ConfigKit's
|
||
version handling; version 5 has no compatibility translation. PromptKit rules,
|
||
including resource identity and secret marking, participate in the same definition.
|
||
The whole result can be supplied to `NewFromConfig` without another load.
|
||
|
||
**Validation:** focused `go test ./internal/core/config`; test typed `/promptkit`
|
||
handoff and unchanged source-origin semantics. Source assembly is completed in
|
||
Stage 5; no fake result or YAML round-trip is an acceptable substitute.
|
||
|
||
## Stage 3: Resolve The New Application Controls
|
||
|
||
**Status:** Complete. Added omission-aware producer controls and final resolution,
|
||
invocation-wide cache precedence, independent positive workers, source-relative
|
||
state roots, and explicit-empty debug behavior. Removed the superseded application
|
||
retry/repair/failure-policy and nested cache configuration surfaces. Updated
|
||
focused loading, inheritance, directory, and resolution coverage.
|
||
The isolated cache-root test and `git diff --check` pass. Full focused package
|
||
tests are blocked by not-yet-migrated pipeline preparation, attempts, and debug
|
||
callers of removed fields/types (for example `ValidationPolicy` and `Retries`),
|
||
plus old-path ConfigKit's removed YAML `plugin/limit` import. Their scheduled
|
||
caller migration and final whole-tree validation remain required; these tests
|
||
are not claimed as passing.
|
||
|
||
**Goal:** define the selected budgets, modes, workers, and state paths once.
|
||
|
||
**Inspect/change:** configuration document/runtime conversion and validation,
|
||
`internal/framework/pipeline/profile.go`, `options.go`, `validation_policy.go`,
|
||
and policy-resolution tests.
|
||
|
||
**Work:** add omission-aware pipeline/binding `mode`, `max_additional_turns`, and
|
||
`cache`; root `cache_directory`; and positive `concurrency.workers`. Resolve the
|
||
roadmap's precedence/defaults, including explicit zero turns, strict normalization,
|
||
and the CLI cache override taking precedence over every binding. Remove producer/
|
||
validator `retries`, application structural-repair fields, failure-category
|
||
policies, nested legacy cache settings, `total_llm`, and extraction-worker coupling.
|
||
Use native source origins for nonempty config paths. Preserve explicit empty debug
|
||
directory as capture-disabled; omitted debug root is `/tmp/notarius-debug`.
|
||
Omitted/empty cache root uses `os.UserCacheDir()/notarius`; a configured root is exact.
|
||
|
||
**Acceptance:** resolved operations contain final controls rather than repeating
|
||
inheritance at execution. Tests distinguish omission, zero, and explicit values.
|
||
There are no module-declared permissive defaults, new maximum worker/turn caps,
|
||
backend settings derived from workers, or replacement non-LLM retry options.
|
||
|
||
**Validation:** focused configuration and pipeline resolution tests, including
|
||
binding-over-pipeline mode/turn/cache selection, invocation cache selection,
|
||
worker positivity, and explicit-empty path behavior.
|
||
|
||
## Stage 4: Convert Embedded Initial Resources And Schema Declarations
|
||
|
||
**Status:** Complete. Converted all 16 initial/validator/reconciliation prompt
|
||
declarations and the extraction profile to native identified lists. The embedded
|
||
`assets/notarius.yml` imports them into the `notarius` collection with its own
|
||
filesystem-rooted schemas. Real shared-file paths replace virtual mounts; schema
|
||
bytes, prose, message order, and cache hints are unchanged. Native offline loading
|
||
and preparation of every declaration pass in `GOWORK=off go test ./assets`,
|
||
including exact loaded-schema comparisons and selected profile defaults.
|
||
`git diff --check` passed. Module caller/manifest migration remains in its assigned
|
||
later stages; no intermediate whole-tree compilation claim is made.
|
||
|
||
**Goal:** native declarations become the only default prompt/schema assembly.
|
||
|
||
**Inspect/change:** existing `assets/**/prompts/prompt.yaml`,
|
||
`assets/dnd/profiles/dnd-extraction.yaml`, `assets/package.go`, and old prompt/schema
|
||
manifest tests. Inspect all current content-file paths before changing them.
|
||
|
||
**Work:** convert initial producer, validator, and generic reconciliation prompts
|
||
to native identified lists in the `notarius` collection, retaining local IDs and
|
||
using qualified prompt/profile references. Add one ordinary embedded native source
|
||
that imports the resources and declares the collection schema root from the assets
|
||
filesystem context. Use real relative paths to shared files, not `sharedassets`
|
||
mounts. Set each prompt's response format, validation mode, and schema selection;
|
||
remove explicit `repair_attempts`. Keep the default extraction profile's current
|
||
model, timeout, service tier, unset reasoning, and unqualified `openrouter` backend.
|
||
Remove old resource-version selectors; no new PromptKit version header is needed.
|
||
|
||
**Acceptance:** every existing LLM-facing schema is still self-contained and
|
||
selected from its actual asset location. Intrinsic artifact schemas remain
|
||
unchanged. Shared prose/order/cache controls are preserved except for deliberate
|
||
conversation wording changes made with each correction pair later. The assets
|
||
Go package remains a content-only leaf, without a behavioral registry.
|
||
|
||
**Validation:** native ConfigKit load plus PromptKit construction/preparation with
|
||
fake request inputs; cover real shared-file paths and the embedded schema root.
|
||
Do not add prose snapshots or fixed shared-prefix-length tests.
|
||
|
||
## Stage 5: Assemble One Application Load And Native Engine
|
||
|
||
**Status:** Complete. Added invocation-owned selection/result/engine reuse and
|
||
one native layered application load with bundled, embedded, and strict peer
|
||
operator underlays. Native main-relative imports anchor optional companions to
|
||
the selected main source; explicit-empty configuration paths fail. Added real
|
||
temporary-file coverage for selection differences, companions/conflicts,
|
||
inheritance/imports, captured messages, selective resource replacement, and native
|
||
schema contexts. `git diff --check` and the existing native asset tests pass.
|
||
CLI/config test execution remains blocked by the recorded legacy YAML import
|
||
and pipeline callers of retired policy/retry types. Old command-handler tests
|
||
also await the command replacement in Stage 6; no blocked test is claimed passing.
|
||
Native standard-command and run wiring remain in their assigned later stages.
|
||
|
||
**Goal:** runtime and standard commands consume the same selected resources.
|
||
|
||
**Inspect/change:** `internal/cli/configkit_load.go`,
|
||
`internal/core/config/loaded_config.go`, new focused CLI loader/engine files, and
|
||
old `internal/framework/llm` source assembly. Read upstream shared-loading guidance.
|
||
|
||
**Work:** select the actual application main source using the existing explicit/
|
||
environment/native discovery policy. Reject explicitly empty `--config` rather
|
||
than discovering another file. Assemble bundled OpenRouter, embedded Notarius,
|
||
and operator companions as ordered underlays of the one application load.
|
||
Discover optional `prompts.yml` and recursive `.yml`/`.yaml` files in `prompts/`
|
||
beside that selected main source, in one strict operator peer group. Preserve
|
||
native imports/destinations/source contexts; do not fall back to the working
|
||
directory or hide malformed operator sources. Main imports, overlay, and explicit
|
||
overrides retain native precedence. Construct `NewFromConfig` from the successful
|
||
whole result, with runtime-only options, without callbacks/loading happening again.
|
||
|
||
**Acceptance:** invocation-owned reuse distinguishes complete selections,
|
||
including overlay/overrides and the two sides of `config diff`. Changes to source
|
||
files after capture do not mutate an engine. Shared loaders create no run storage.
|
||
No duplicate asset registry, settings reparse, or persistent loader cache remains
|
||
on the new path.
|
||
|
||
**Validation:** focused CLI/config tests using real temporary files: absent
|
||
companions, file+directory conflicts, selective replacement, native inheritance,
|
||
imports, source-relative messages, embedded/external schema roots, inline/
|
||
source-relative schemas, and missing-file behavior after root replacement.
|
||
|
||
## Stage 6: Install CommandKit And The Standard Configuration Suite
|
||
|
||
**Status:** Complete. Added the lazy native application constructor, generated
|
||
help/version and Bash/Zsh/Fish completion, and shared `app`/focused `promptkit`
|
||
ckconfig scopes with native overrides and comparison. The executable supplies
|
||
the release-linked build identity directly. Removed the manual configuration
|
||
handlers, dispatcher, and assembled help. Added focused built-in, scope/report,
|
||
explicit-empty selection, and diff-status tests.
|
||
Build-info/asset tests and `git diff --check` pass. CLI tests and executable
|
||
subprocess checks remain blocked by the previously recorded legacy YAML import
|
||
and not-yet-migrated pipeline consumers; final assembled verification must run
|
||
them. PromptKit-suite convenience aliases are installed with that suite in Stage 7.
|
||
|
||
**Goal:** one native application constructor replaces manual command plumbing.
|
||
|
||
**Inspect/change:** `cmd/notarius`, `internal/cli/run.go`, `configkit_commands.go`,
|
||
`configkit_load.go`, `internal/buildinfo`, and new focused command-composition files.
|
||
|
||
**Work:** declare application identity, native help/version, Bash/Zsh/Fish completion,
|
||
and presentation through CommandKit. Install ckconfig scopes named `app` and
|
||
`promptkit`, using the shared result and native focused-subtree adapter where
|
||
appropriate. Install native validation/show/sources/explain/diff and convenience
|
||
aliases. Preserve native causes/outcome classification and supplied-value inputs.
|
||
Construction itself performs no configuration load, engine build, or storage I/O.
|
||
Retire the old configuration command handlers rather than keeping two surfaces.
|
||
|
||
**Acceptance:** help/version/completion work without any config or API key;
|
||
configuration commands use native report values and the same load evidence.
|
||
`config diff` does not accidentally compare a memoized selection to itself.
|
||
Release-linked version identity remains available at the real executable boundary.
|
||
|
||
**Validation:** focused CLI tests and representative subprocess invocations for
|
||
help/version/completion, one app-scope explanation, one focused PromptKit report,
|
||
and diff status `3`. Test semantics, not entire upstream help text.
|
||
|
||
## Stage 7: Install ckprompt And Reusable Run Overrides
|
||
|
||
**Status:** Complete. Installed native resource listings/inspection, requirements,
|
||
request validation/preparation, namespace selection, and explicit smoke operations
|
||
with ckprompt's builders and scoped convenience aliases. Added reusable native
|
||
profile/session/execution flag input and request application, including omitted
|
||
versus empty reasoning and native transient-retry overrides. Removed the copied
|
||
CLI profile-inspection path and replaced its tests with native command coverage.
|
||
The isolated real-kit override test, build-info/asset tests, and
|
||
`git diff --check` pass. Assembled CLI inspection/preview/fake-provider smoke tests
|
||
remain blocked by the recorded legacy dependency/caller compilation blockers;
|
||
their results are not claimed passing. Full pipeline binding remains Stage 33.
|
||
|
||
**Goal:** adopt the full selected native diagnostics and execution input suite.
|
||
|
||
**Inspect/change:** CLI composition from Stage 6, old `promptkit_profiles.go`, and
|
||
ckprompt's maintained composed example and public integration declarations.
|
||
|
||
**Work:** use `FromEngine`, request validation/preparation, smoke integration, and
|
||
the shared loaders to install all selected resource listings/inspection,
|
||
requirements, preview, and explicit smoke checks. Use upstream request builders
|
||
unless an application callback is genuinely needed; preserve supplied inputs.
|
||
Expose ckprompt's reusable profile/session/execution flags for pipeline run-wide
|
||
overrides, carrying omission and explicit empty reasoning distinctly. Use native
|
||
`--llm-profile` and `--llm-session-id`; remove copied profile inspectors and
|
||
effective-parameter reports. Do not build new native-command wrappers or probes.
|
||
|
||
**Acceptance:** standard inspection needs no transcript/provider call. Concrete
|
||
request validation/preview performs native request preparation; smoke work is
|
||
explicit only and preserves supported/applicable/result distinctions. The reusable
|
||
override value can reach every producer, validator, and correction without
|
||
mutating captured configuration. Native transient retries replace the former
|
||
application-specific override machinery.
|
||
|
||
**Validation:** focused CLI tests with a fake provider: listings, partial profile,
|
||
explicit preview inputs, no implicit smoke, supplied false/zero/empty values, and
|
||
one explicit smoke outcome. Full pipeline binding is completed in Stage 33.
|
||
|
||
## Stage 8: Define Typed Producer Collaboration And Resource Dependencies
|
||
|
||
**Status:** Complete. Construction now passes the shared native engine. Typed
|
||
producer requests carry explicit assessment callbacks, resolved controls, native
|
||
execution options, and incoming reuse eligibility; results carry accepted/fallback
|
||
disposition, reason, and eligibility without a completion facade. Validators receive
|
||
native execution options only. Static prompt-ID dependencies replace correction
|
||
transport capabilities and are copied through registration, resolution, and
|
||
preparation. Exact artifact type, codec/evidence, option, and reference checks remain.
|
||
Added registry callback/native-option, incompatible-type, dependency-isolation, and
|
||
preparation coverage. Contracts tests/vet and `git diff --check` pass. Pipeline
|
||
tests remain blocked by old ConfigKit's removed YAML `plugin/limit` import; its
|
||
production build also encounters unmigrated policy/normalize-retry consumers.
|
||
Their scheduled migration remains required; blocked tests are not claimed passing.
|
||
|
||
**Goal:** expose only the application contracts required by native execution.
|
||
|
||
**Inspect/change:** `internal/framework/contracts/{contracts,typed_pipeline}.go`,
|
||
pipeline construction/registries/preparation, module specifications, and typed
|
||
registration tests.
|
||
|
||
**Work:** replace `ModuleDependencies.LLM` with the shared native engine for
|
||
model-executing modules. Add small typed operation collaboration/outcome types
|
||
for the pipeline assessment callback, accepted/fallback disposition, fallback
|
||
reason, and reuse eligibility. Pass native execution overrides/observers without
|
||
creating another completion request/result facade. Adapt typed method arguments
|
||
or call options to carry the assessment callback explicitly without an import
|
||
cycle. Add static initial/correction prompt dependencies using the same IDs as
|
||
request builders; keep dynamic selection runtime-checked. Validators remain
|
||
assessment modules, not producer-loop participants.
|
||
|
||
**Acceptance:** artifact exact-type checks, codec/evidence compatibility, and
|
||
module option/reference validation remain intact. New dependency declarations
|
||
are small ID selections, not asset manifests, schema registries, or fingerprints.
|
||
Pure domain types and deterministic helpers acquire no PromptKit dependency merely
|
||
because surrounding producers do.
|
||
|
||
**Validation:** focused contracts/registry/preparation tests, including incompatible
|
||
typed registrations and statically declared correction dependencies. This stage
|
||
need not migrate every module caller.
|
||
|
||
## Stage 9: Centralize Candidate Assessment And Terminal Disposition
|
||
|
||
**Status:** Complete. Replaced retrying validator execution and the producer
|
||
attempt state machine with one ordered assessment path and strict/permissive
|
||
terminal handling. Typed and materialized-chunk assessors give each validator
|
||
detached candidate/context views and preserve rejection guidance separately from
|
||
failed/unavailable assessments and their native causes. Guidance aggregation has
|
||
no hard-coded correction assignment. Fallback construction runs once without
|
||
ordinary revalidation, never on caller cancellation, and cannot claim acceptance
|
||
or reuse eligibility. Added complete-chain, revised/unchanged candidate, isolation,
|
||
guidance-contract, unavailable-assessment, timeout, and cancellation coverage.
|
||
Contracts tests/vet, isolated terminal-mode tests, and `git diff --check` pass.
|
||
Full pipeline tests remain blocked by the recorded old ConfigKit YAML import;
|
||
production compilation also awaits migration of runner/debug/diagnostic consumers
|
||
of removed attempt/report types. No blocked test is claimed passing; runner and
|
||
native continuation integration remain assigned to Stages 10–11.
|
||
|
||
**Goal:** use one complete assessment path for initial and revised candidates.
|
||
|
||
**Inspect/change:** `validation_executor.go`, `validation_policy.go`,
|
||
`producer_attempts.go`, typed validator execution, and validation-isolation tests.
|
||
|
||
**Work:** remove the validator invocation retry loop. Invoke each applicable
|
||
validator once per candidate through its public boundary, in configured order,
|
||
with immutable candidate views. Collect all rejections/guidance despite earlier
|
||
rejections; distinguish reject, fail, and unavailable/skip. Preserve actionable
|
||
`CorrectionGuidance` separately from reason codes and operator prose. Expose one
|
||
typed assessor to the producer helper. Apply resolved strict/permissive handling
|
||
to terminal nonacceptance; unavailable assessment is not a semantic defect.
|
||
Fallback construction checks remain module-owned, outside ordinary revalidation.
|
||
|
||
**Acceptance:** repeated/unchanged corrected candidates still run the complete
|
||
chain. A producer does not acquire acceptance from native execution success alone.
|
||
No reason-code text becomes correction instructions, no validator rerun extends
|
||
native structural repair, and no final duplicate assessment occurs.
|
||
|
||
**Validation:** focused pipeline tests for two independent rejections, earlier
|
||
approval followed by a changed candidate, isolated typed validator views,
|
||
unavailable assessment, guidance contract errors, and cancellation.
|
||
|
||
## Stage 10: Implement The Small Native Producer Helper
|
||
|
||
**Status:** Complete. Added one typed helper using native Run/Coordinate and
|
||
module-owned interpretation/correction/fallback functions. It checks native
|
||
structural status before interpretation, reuses the initial assessment on the
|
||
first coordination callback, and assesses every delivered revision once,
|
||
including the final allowed turn. Native coordination owns history and budget.
|
||
Execution/profile/session selections and existing observers/preparation callbacks
|
||
are preserved without duplicate enrollment. Failed continuations cannot recover
|
||
earlier candidates; caller cancellation and undelivered responses never become
|
||
acceptance or fallback. Real-engine offline fake-provider tests cover acceptance,
|
||
zero turns, final-turn acceptance, exhaustion, nil-error structural failure,
|
||
operational/unavailable failures, interpretation guidance, cancellation, and
|
||
typed native operation timeouts. File-scoped helper tests/vet/race checks and
|
||
contracts tests pass. Full pipeline tests remain blocked by the recorded legacy
|
||
ConfigKit YAML import and unmigrated consumers; no whole-package success is
|
||
claimed. Runner integration, caching, and capture remain in their assigned stages.
|
||
|
||
**Goal:** delegate turn iteration while retaining typed Notarius acceptance.
|
||
|
||
**Inspect/change:** a focused helper alongside pipeline assessment; upstream
|
||
`Coordinate`, `RunRequest`, prepared execution, and native validation status.
|
||
|
||
**Work:** accept the module's initial native request, interpretation, correction
|
||
builder, supported fallback functions, and the pipeline assessor. Use `Engine.Run`
|
||
for uncached work with exchange capture when correction/debug needs it. Check
|
||
native errors and `ValidationFailed` before interpretation, including nil-Go-error
|
||
failure results. Assess the initial usable candidate, then coordinate actionable
|
||
correction from its captured exchange with native `AdditionalTurns`. On the
|
||
initial coordination callback, use existing findings; do not reassess the initial
|
||
response. Assess each newly delivered result exactly once, including the final
|
||
allowed turn. Return typed acceptance/fallback state, not a second retry protocol.
|
||
If a continuation fails without a usable response, do not retain an earlier
|
||
proposal as the terminal fallback source; use the module's no-usable-response
|
||
fallback direction.
|
||
|
||
**Acceptance:** native coordination alone counts turns and carries history.
|
||
Acceptance stops early; exhausted/stopped are not acceptance. Structural terminal
|
||
failure and unavailable assessment do not generate invented semantic guidance.
|
||
Cancellation and `Undelivered` do not produce acceptance, fallback, or cache writes.
|
||
Distinguish caller cancellation from native execution failures using caller context
|
||
state and native typed causes rather than flattening every timeout into cancellation.
|
||
No earlier-candidate recovery, cross-turn plan union, or manual assistant replay.
|
||
|
||
**Validation:** real PromptKit with a fake provider: initial acceptance, zero turns,
|
||
correction acceptance, final-turn acceptance, exhaustion, validation failure with
|
||
nil error, operational failure, and cancellation before callback delivery.
|
||
Stage 27 adds cache selection to this same helper, not a parallel executor.
|
||
|
||
## Stage 11: Connect Typed Runner Operations And Eligibility Propagation
|
||
|
||
**Status:** Complete. Runner operations now carry resolved controls, native
|
||
execution options, and operation-scoped assessors. The runner consumes native
|
||
helper outcomes without outer retries or revalidation; deterministic modules
|
||
produce once and are assessed directly, with an optional module-owned fallback
|
||
function. Final codec/source checks remain mandatory. Fallback eligibility follows
|
||
chunk ancestry, lane results, and actual generated-reference consumers without
|
||
tainting unrelated work; emitting operations contribute once to fallback_count.
|
||
Removed superseded attempt/correction hooks and their debug/policy projections.
|
||
The pipeline production package builds. All production files plus the focused
|
||
typed registry, codec, assessment, native-helper, and runner tests pass, including
|
||
focused vet/race checks; contracts tests and `git diff --check` also pass.
|
||
The complete pipeline test package still imports retired dependencies/fixtures
|
||
and is not claimed passing. Production module migration, worker-pool replacement,
|
||
legacy-store removal, publication, and full capture remain in their assigned stages.
|
||
|
||
**Goal:** the runner supplies assessment once and consumes the returned outcome.
|
||
|
||
**Inspect/change:** `runner_typed.go`, `typed_execution.go`, `runner.go`,
|
||
`runner_chunk_plan.go`, `handoff.go`, and the old state/correction hooks.
|
||
|
||
**Work:** bind each resolved producer's assessor and controls to its operation,
|
||
route LLM producers through the shared helper, and retain a direct production/
|
||
assessment path for deterministic stages. Remove outer assessment/retry calls
|
||
around helper-owned assessment. Carry fallback and cache-eligibility facts through
|
||
extract -> merge -> normalize and generated reference handoffs. A chunk fallback
|
||
affects all derived lane work; a lane fallback affects that lane and actual
|
||
consumers, not unrelated siblings. Keep only the small facts needed for outcomes,
|
||
fallback counting, and reuse suppression, without dependency fingerprints.
|
||
|
||
**Acceptance:** generated references may carry valid fallback artifacts with
|
||
explicit reuse ineligibility. Stable ordering, exact codec handoff, required
|
||
references, and fixed pipeline steps remain intact. Each actual fallback operation
|
||
contributes once; affected downstream operations are not themselves counted unless
|
||
they also emit a fallback.
|
||
|
||
**Validation:** focused runner/handoff tests using small typed modules: full-chain
|
||
assessment once, deterministic single execution, chunk/lane/generated-reference
|
||
eligibility propagation, unrelated predecessors unaffected, and cancellation.
|
||
Native production modules are migrated in Stages 12–23.
|
||
|
||
## Stage 12: Migrate Spell Extraction As The Representative Producer
|
||
|
||
**Status:** Complete. Spell extraction now prepares fixed transcript/reference,
|
||
catalog, and NPC projections once and uses the shared native producer helper.
|
||
Added the same-schema/default-profile correction prompt, enduring system
|
||
principles, explicit run/profile/session propagation, static dependencies, and
|
||
the codec-valid empty fallback. Removed spell completion/asset/schema registries
|
||
and manifest/fingerprint metadata; retained private-response interpretation and
|
||
source-order canonicalization. Migrated focused tests to a native provider fake
|
||
and added a leaf-assembled generic-chunker spell contract covering complete ordered
|
||
assessment, final correction, strict exhaustion, permissive output, and exact
|
||
continuation history without repeated references.
|
||
Native asset, catalog/shape, isolated schema/input-helper, and focused pipeline
|
||
tests pass; runnable asset/catalog/shape vet/race checks and production pipeline/
|
||
generic/Seriatim builds pass. Full spell/shared/source-validator tests, including
|
||
the new assembled contract, are blocked at setup by shared legacy asset helpers
|
||
importing old-path ConfigKit's removed YAML `plugin/limit`. Those tests are not
|
||
claimed passing. Central registration and remaining legacy fixtures/dependencies
|
||
remain assigned to Stages 24 and 35, with final assembled verification required.
|
||
|
||
**Goal:** prove native extraction with the real typed module and validators.
|
||
|
||
**Inspect/change:** `internal/modules/dnd/extract/spells`, shared extraction input
|
||
helpers, spell catalog projection, spell validators, and `assets/dnd/spells`.
|
||
|
||
**Work:** prepare transcript, catalog, references, and NPC resolution once; build
|
||
native initial requests and interpret native results through the helper. Preserve
|
||
source-reference ordering/swapping, contextual entity resolution, spell identity,
|
||
catalog semantics, and exact artifact encoding. Add `notarius/dnd.spells.repair`
|
||
using the same schema and default profile as the initial prompt. Put enduring
|
||
principles in the initial system context and only findings/new correction
|
||
assignment in the follow-up. Keep explicit profile/run overrides on both requests.
|
||
Implement the valid empty spell fallback and static dependencies.
|
||
|
||
**Acceptance:** one generic-chunker spell pipeline exercises native production,
|
||
ordered validation, correction, strict failure, and permissive empty output.
|
||
Source/ref input bytes are not regenerated during continuation. The helper owns
|
||
the operation and the runner does not validate it again.
|
||
|
||
**Validation:** `go test ./internal/modules/dnd/extract/spells
|
||
./internal/modules/dnd/validate/spells/...`; adapt the assembled spell contract
|
||
test to a native provider fake, including catalog rejection and final correction.
|
||
|
||
## Stage 13: Refactor The Shared Reconciliation Core
|
||
|
||
**Status:** Complete. Removed the reconciliation execution service and its
|
||
private completion/correction transport, retry classifications, asset/schema
|
||
registration, metadata, and fingerprints. The core now exposes fixed owned
|
||
preparation bytes, native initial/follow-up request builders, proposal decoding,
|
||
and the existing independent safety assessment/application functions. Candidate
|
||
handles, declared ranges, bounded source windows, group-safety rules, typed
|
||
application, and actionable issue prose remain; the follow-up asset owns the
|
||
replacement assignment rather than an application retry message builder.
|
||
Added the generic native correction pair with the unchanged response schema and
|
||
matching unset profile defaults, moving enduring protocol/policy into initial
|
||
system context. Retained domain-neutral regressions and added native provider/
|
||
preparation, structural-failure, fixed-material, and successive-proposal tests.
|
||
The complete reconciliation and asset tests, vet, race checks, and
|
||
`git diff --check` pass. Domain normalizer and central registration consumers of
|
||
the removed APIs remain assigned to Stages 14–15 and 24; no whole-tree compilation
|
||
claim is made at this intermediate boundary.
|
||
|
||
**Goal:** separate one-time evidence preparation from proposal interpretation.
|
||
|
||
**Inspect/change:** `internal/framework/semanticreconcile/{engine,preparation,
|
||
proposal,application,schema}.go` and relevant tests; generic normalization prompt assets.
|
||
|
||
**Work:** expose reusable preparation, native request construction, and proposal
|
||
decoding/assessment as ordinary functions. Keep fixed candidate-number mappings,
|
||
range declarations, namespace guidance, bounded source windows, proposal safety,
|
||
and model-facing issue prose. Remove the core's old completion client/correction
|
||
transport and response-schema registration/identity machinery. The module's
|
||
producer helper invocation owns native execution; do not create a replacement
|
||
reconciliation service/exchange wrapper. Convert the generic default prompt and
|
||
add its correction pair, reusing the same response schema and native defaults.
|
||
|
||
**Acceptance:** each operation prepares candidates/windows once and applies every
|
||
complete proposal afresh to those originals. Unknown/overlapping handles and
|
||
unsafe groups remain rejected. No source evidence or opaque identity is copied by
|
||
the model. Pure proposal assessment/application remains independently testable.
|
||
|
||
**Validation:** `go test ./internal/framework/semanticreconcile`; retain the
|
||
candidate-number, group-safety, deterministic ordering, limits, and source-window
|
||
behavioral tests. Native structural failures terminate at the kit boundary, not
|
||
as a request to restart the core.
|
||
|
||
## Stage 14: Migrate Item Registry Normalization
|
||
|
||
**Status:** Complete. Item normalization now uses one fixed preprocessing and
|
||
reconciliation preparation, native initial/correction requests, and the shared
|
||
producer helper. Each response applies its own safe proposal to original records
|
||
before operation-scoped assessment. Currency compatibility, identity derivation,
|
||
evidence ordering, and domain diagnostics remain. Added the same-schema/default-
|
||
profile correction declaration and static dependencies; removed module retries,
|
||
completion/asset registration, and manifest/fingerprint metadata.
|
||
Permissive output uses only the final safe candidate or the supplied merged
|
||
artifact after a result-less failure. Fallback construction checks codec, registry
|
||
identity, and source usability without ordinary revalidation. Preparation limits
|
||
honor resolved mode; insufficient candidates use the existing assessment path
|
||
without native requests. Exposed/factored the existing terminal/no-model assessment
|
||
functions for these real callers rather than duplicating acceptance policy.
|
||
Retained domain regressions and added native-fake correction, final safe partial,
|
||
unchanged-candidate, failed-continuation/no-earlier-recovery, structural-failure,
|
||
mode/limit, no-model, fallback-usability, override, and cancellation tests.
|
||
Focused pipeline tests/vet/race, terminal tests, complete reconciliation/asset and
|
||
item-shape tests/vet/race, production framework builds, and `git diff --check`
|
||
pass. Item-normalizer/codec/identity/source-validator tests remain blocked at
|
||
setup by shared legacy asset helpers importing old ConfigKit's removed YAML
|
||
`plugin/limit`; the new module tests are not claimed passing. Their full execution
|
||
remains required after scheduled dependency cleanup and in final verification.
|
||
|
||
**Goal:** prove a complete native normalization operation and safe fallback.
|
||
|
||
**Inspect/change:** `internal/modules/dnd/normalize/itemregistry`, its validators/
|
||
codec/identity helpers, and `assets/dnd/item-registry/normalize`.
|
||
|
||
**Work:** connect the core functions to the typed producer helper. Preprocess
|
||
fixed originals once, use initial/correction requests with fixed handles, and
|
||
apply each proposal afresh before the complete normalization assessment. Add the
|
||
domain correction prompt and dependencies. Treat preparation-limit inability and
|
||
unusable native output through resolved strict/permissive handling, without
|
||
automatic permissive defaults. On permissive exhaustion apply only safe groups
|
||
from the final usable proposal; otherwise retain the deterministic merged input.
|
||
Reuse already available merged data, never rerun deterministic stages to get it.
|
||
|
||
**Acceptance:** a failed correction cannot mutate originals, union earlier groups,
|
||
or silently claim complete semantic success. Insufficient candidates avoid a model
|
||
call as before. Reuse eligibility distinguishes ordinary accepted normalization
|
||
from final safe partial/deterministic fallback.
|
||
|
||
**Validation:** focused item-normalizer tests with a native fake provider: initial
|
||
success, repaired proposal, unsafe final groups, strict failure, permissive
|
||
preservation, unchanged-candidate reassessment, and cancellation.
|
||
|
||
## Stage 15: Migrate NPC And Location Registry Normalization
|
||
|
||
**Status:** Complete. Both normalizers now use the shared native producer helper
|
||
and fixed one-time reconciliation preparation. Each response applies its own safe
|
||
groups afresh to the original records, before typed operation assessment. Added
|
||
same-schema/default-profile correction prompts and static dependencies; removed
|
||
retry transport, schema/policy fingerprints, and manifest-only asset registration.
|
||
Name-based NPC identity, evidence-anchored location identity, canonical naming,
|
||
source ordering, and domain application rules remain unchanged. Strict handling
|
||
does not silently fall back; permissive output uses only the final safe candidate
|
||
or the supplied merged artifact, with codec/identity/source construction checks.
|
||
Retained domain regressions and added native provider coverage for correction,
|
||
unchanged reassessment, final-only safe groups, failed continuation without prior
|
||
proposal recovery, structural failure, overrides/history, cancellation, and
|
||
unusable fallbacks. Native prompt-pair checks run in the content package without
|
||
depending on legacy module helpers.
|
||
Asset/reconciliation tests, vet/race checks, focused pipeline producer/runner
|
||
tests, production framework builds, and `git diff --check` pass. Both normalizer
|
||
package tests remain blocked at setup by shared legacy asset helpers importing
|
||
old ConfigKit's removed YAML `plugin/limit`; their new tests are not claimed
|
||
passing. Scheduled dependency cleanup and final whole-tree validation must run
|
||
them and close that blocker.
|
||
|
||
**Goal:** apply the proven normalizer path without domain-policy drift.
|
||
|
||
**Inspect/change:** `internal/modules/dnd/normalize/{npcregistry,locationregistry}`
|
||
and their normalization assets, identity/grounding helpers, and focused tests.
|
||
|
||
**Work:** migrate these two parallel consumers using Stages 13–14's shared core
|
||
and producer helper. Preserve each family's canonical naming, proper-name scope,
|
||
source grounding, identity computation, compatible-group checks, and output order.
|
||
Add each domain correction prompt with the same initial schema/default profile;
|
||
retain generic protocol/candidate presentation as ordinary shared files. Remove
|
||
their retry directives, schema/policy fingerprints, and manifest-only metadata.
|
||
|
||
**Acceptance:** all three registry normalizers share execution and preparation
|
||
mechanics but retain their typed application rules. Every turn operates on the
|
||
same candidate namespace. Final safe-group/deterministic fallbacks are explicit,
|
||
strict remains default, and model errors do not trigger an outer restart.
|
||
|
||
**Validation:** `go test ./internal/modules/dnd/normalize/npcregistry
|
||
./internal/modules/dnd/normalize/locationregistry`; adapt boundary tests to native
|
||
provider fakes and retain domain identity/reconciliation regression coverage.
|
||
|
||
## Stage 16: Migrate D&D Scene Planning And Chunk Assessment
|
||
|
||
**Status:** Complete. Scene planning now prepares the full transcript/reference
|
||
inputs once and uses the shared native producer helper with initial/correction
|
||
prompts, matching schema/profile defaults, and static dependencies. Enduring
|
||
scene principles remain in initial system context. Removed the private completion,
|
||
response-schema/asset registration, and manifest metadata paths. Existing
|
||
source-order, coverage, gap/overlap, and boundary interpretation rules remain.
|
||
Unresolvable boundaries supply actionable correction guidance; unusable native
|
||
structural results never become semantic candidates. Permissive terminal handling
|
||
constructs one unannotated whole-source plan using normal materialization; invalid
|
||
input, cancellation, and fallback-construction failures do not publish a fallback.
|
||
The runner already binds materialization and complete chunk assessment to the
|
||
native operation without an outer attempt/replay loop. Added a runnable runner
|
||
contract proving ordered full-chain assessment through final-turn acceptance,
|
||
strict/permissive exhaustion, no fallback revalidation, and derived reuse exclusion.
|
||
Migrated scene regressions to a native provider fake and added correction/history,
|
||
override, structural-failure, whole-source fallback, and cancellation cases.
|
||
Scene prompt-pair/schema checks run in the asset package.
|
||
Asset/source tests and vet/race checks, focused pipeline tests and vet/race checks,
|
||
and `git diff --check` pass. Scene-chunker package tests remain blocked at setup
|
||
by shared legacy helpers importing old ConfigKit's removed YAML `plugin/limit`;
|
||
their runtime results are not claimed passing. Scheduled dependency cleanup and
|
||
final verification must execute them. Chunk-cache replacement remains Stage 28.
|
||
|
||
**Goal:** one native chunk producer with deterministic materialization.
|
||
|
||
**Inspect/change:** `internal/modules/dnd/chunk/scenes`, chunk-plan interpretation/
|
||
materialization in the runner, chunk validation, and `assets/dnd/scenes`.
|
||
|
||
**Work:** prepare the full transcript once, produce/interpret a complete native
|
||
plan, materialize it through existing source-addressing checks, and assess it
|
||
through the pipeline's chunk validator chain. Add the scene-plan correction
|
||
prompt using its initial schema/profile. Implement the permissive one-whole-source
|
||
plan and materialize it normally; do not fabricate scene descriptions/tags or add
|
||
deterministic slicing. Retain source order for nonconsecutive unit IDs. Remove
|
||
outer attempt/replay logic around plan generation. Stage 28 replaces storage.
|
||
|
||
**Acceptance:** malformed or unusable plans never become ordinary acceptance.
|
||
Every revised usable plan gets complete chunk assessment, including the final
|
||
turn. A fallback preserves the full source and marks all derived work ineligible
|
||
for cache reuse. Empty/invalid input and fallback-construction errors fail normally.
|
||
|
||
**Validation:** focused scene chunker, chunk materialization, and chunk-validator
|
||
tests with a fake native provider, including reversed/resolvable ranges, invalid
|
||
units, full-source fallback, final-turn acceptance, and cancellation.
|
||
|
||
## Stage 17: Migrate The Combat-Semantics LLM Validator
|
||
|
||
**Status:** Complete. The validator now makes one native request on the shared
|
||
engine, preserving the validator profile/default, session, execution overrides,
|
||
transient-retry override, observer, and preparation callback. Static prompt
|
||
dependencies replace asset/schema registration and manifest/fingerprint metadata.
|
||
Native errors and failed validation status are checked before decoding; execution
|
||
and interpretation failures produce an unavailable assessment with inspectable
|
||
causes, not semantic rejection or invented correction guidance. Cancellation
|
||
does not deliver a judgment. Successful classification retains the original
|
||
combat policy, scene-local transcript boundary, explanation checks, and actionable
|
||
producer guidance. Interpretation and its existing cases are independently
|
||
testable; evaluation fixtures and schema/policy assets are unchanged.
|
||
Added native provider tests for judgments, override/input isolation, nil-error
|
||
failed validation, unavailable provider/capacity, interpretation failure,
|
||
cancellation, and nested same-backend admission. A runnable framework contract
|
||
proves complete ordered native assessment on a backend limited to one active call,
|
||
without holding producer admission or repeating the final chain.
|
||
Asset tests/vet/race, independent interpretation/evaluation tests/vet/race,
|
||
focused pipeline tests/vet/race, and `git diff --check` pass. The concrete
|
||
validator package remains blocked at setup by shared legacy asset helpers
|
||
importing old ConfigKit's removed YAML `plugin/limit`; its native runtime tests
|
||
are not claimed passing. Scheduled dependency cleanup and final verification
|
||
must execute them. No validator-owned semantic continuation loop was added.
|
||
|
||
**Goal:** validators use native execution without another semantic retry loop.
|
||
|
||
**Inspect/change:** `internal/modules/dnd/validate/scenedescriptions/combat_semantics`,
|
||
its prompt/schema assets, registration, and evaluator tests.
|
||
|
||
**Work:** build a native request using the shared engine and run-wide overrides.
|
||
Use the configured validator profile/default and native output contract. Check
|
||
errors and failed structural validation before decoding. Return domain rejection
|
||
with actionable correction guidance, or an unavailable assessment when execution/
|
||
interpretation cannot complete. Preserve scene-local contextual inputs and the
|
||
combat policy. Remove validator retry/repair fields and private schema/client
|
||
adapters; do not give the validator its own producer coordination loop.
|
||
|
||
**Acceptance:** native structural repair may occur internally, but Notarius
|
||
invokes the validator only once for each producer candidate. Capacity/provider
|
||
failures are unavailable assessment, not semantic rejection. A successful rejection
|
||
supplies useful prose to the scene producer and no opaque diagnostic code.
|
||
|
||
**Validation:** focused validator and ordered-chain tests for approve, reject,
|
||
malformed/failed validation, nil-error failure status, unavailable provider, and
|
||
nested same-backend admission with a limit of one.
|
||
|
||
## Stage 18: Migrate The Three Registry Extractors
|
||
|
||
**Goal:** canonical noun extraction follows the representative native path.
|
||
|
||
**Inspect/change:** `internal/modules/dnd/extract/{npcregistry,itemregistry,
|
||
locationregistry}`, their extraction assets and shared citation/identity helpers.
|
||
|
||
**Work:** migrate the three parallel list producers to native builders,
|
||
interpretation, and the shared helper. Add domain correction prompts, static
|
||
dependencies, and codec-valid empty fallbacks. Prepare transcript/reference
|
||
material once per chunk. Keep proper-name scope, item/currency identity rules,
|
||
source evidence resolution, deterministic IDs, and required validator chains.
|
||
Remove per-module prompt/schema manifests and completion metadata, without
|
||
consolidating unlike domain interpretation rules.
|
||
|
||
**Acceptance:** each produces its existing typed durable artifact and preserves
|
||
source refs/identity across correction. Empty fallback is distinguished from a
|
||
normally accepted empty extraction. No module implements its own turn loop or
|
||
pins structural repair.
|
||
|
||
**Validation:** focused tests for all three extractor packages and one
|
||
representative correction/fallback case each, using the shared native fake pattern
|
||
from Stage 12. Retain meaningful per-family grounding and identity regressions.
|
||
|
||
## Stage 19: Migrate NPC And Location Occurrence Extraction
|
||
|
||
**Goal:** registry-consuming occurrence lanes retain contextual resolution.
|
||
|
||
**Inspect/change:** `internal/modules/dnd/extract/{npcoccurrences,locationoccurrences}`,
|
||
their assets, registry resolvers, and source/registry validators.
|
||
|
||
**Work:** build initial native requests from fixed chunk/registry material;
|
||
interpret contextual selections and attach durable IDs deterministically. Add
|
||
the two correction pairs and empty occurrence fallbacks. Keep occurrence kinds,
|
||
unknown/ambiguous reference handling, accepted source citations, and compatible
|
||
generated-reference expectations. Pass profiles/session/overrides unchanged.
|
||
|
||
**Acceptance:** model responses never reproduce opaque application IDs. Source
|
||
context and registry resolution remain fixed throughout each correction exchange.
|
||
Fallback effects propagate to actual later consumers without invalidating
|
||
unrelated registry/predecessor artifacts.
|
||
|
||
**Validation:** focused NPC/location occurrence tests covering contextual matches,
|
||
ambiguous selection, invalid source refs, corrected final candidate, and fallback.
|
||
Do not relax existing domain invariants merely to make the migration tests pass.
|
||
|
||
## Stage 20: Migrate Item Occurrence Extraction
|
||
|
||
**Goal:** retain the smaller-model holder reliability fixes in the native path.
|
||
|
||
**Inspect/change:** `internal/modules/dnd/extract/itemoccurrences`, associated
|
||
holder/registry/source validators and assets, and holder regression tests.
|
||
|
||
**Work:** use native initial/correction requests and fixed item registry context.
|
||
Preserve `discovered`, `acquired`, `lost`, `consumed`, and `transferred` semantics,
|
||
the private response-to-durable holder mapping, party-member handling, and
|
||
source-grounded deterministic entity resolution. Add the same-schema correction
|
||
pair and valid empty fallback; keep module interpretation guidance separate from
|
||
operator messages. Remove only execution/asset registration machinery.
|
||
|
||
**Acceptance:** existing acquired/from/to reliability behavior and event ordering
|
||
survive. No additional optional model-facing schema fields, synthesized holders,
|
||
opaque model IDs, or durable artifact changes are introduced.
|
||
|
||
**Validation:** focused item occurrence extractor/validator tests with native
|
||
provider fakes, including the previously reported incompatible-holder regression,
|
||
correction acceptance, empty fallback, and final semantic rejection.
|
||
|
||
## Stage 21: Migrate Combat-Turn Extraction
|
||
|
||
**Goal:** native combat production preserves scene eligibility and evidence rules.
|
||
|
||
**Inspect/change:** `internal/modules/dnd/extract/combatturns`, combat validators,
|
||
scene-description reference selection, and combat prompt assets.
|
||
|
||
**Work:** migrate initial native requests, result interpretation, correction pair,
|
||
and empty fallback. Preserve combat-only scene eligibility, actor/turn-kind rules,
|
||
source-order endpoint normalization, and source/registry context. Skipped
|
||
ineligible chunks remain deterministic no-model outcomes, not invented native
|
||
requests or synthetic cache entries.
|
||
|
||
**Acceptance:** no combat model call is made for an ineligible scene; correction
|
||
uses the same fixed operation inputs and schema. The prior resolvable reversed-
|
||
endpoint reliability behavior remains without new semantic retry cost.
|
||
|
||
**Validation:** focused combat extractor/validator tests and assembled combat
|
||
eligibility coverage with a native fake provider. Check no-call cases by provider
|
||
invocations, not by obsolete completion-interface mocks.
|
||
|
||
## Stage 22: Migrate Enemy-Event Extraction
|
||
|
||
**Goal:** preserve multi-artifact grounding while changing execution ownership.
|
||
|
||
**Inspect/change:** `internal/modules/dnd/extract/enemyevents`, grounding/mapping
|
||
helpers, its validators, and enemy-event assets.
|
||
|
||
**Work:** move the module to the shared helper and native initial/correction pair.
|
||
Keep NPC/scene/combat-turn/NPC-occurrence references, evidence intersection,
|
||
engagement checks, contextual names, and durable event order. Prepare reference
|
||
projections once per operation; use the codec-valid empty event fallback. Remove
|
||
its private completion and manifest/fingerprint metadata.
|
||
|
||
**Acceptance:** an operator override applies to initial, correction, and nested
|
||
validation without changing domain reference compatibility. Failed/fallback
|
||
predecessors affect reuse eligibility but valid fallback references remain usable
|
||
in the current run. No fabricated enemy event or expanded extraction scope.
|
||
|
||
**Validation:** focused enemy-event extraction/validation tests plus the assembled
|
||
enemy-event codec contract, using a native provider fake and current typed codecs.
|
||
|
||
## Stage 23: Migrate Scene-Description Extraction
|
||
|
||
**Goal:** retain the required scene artifact while making fallback limits explicit.
|
||
|
||
**Inspect/change:** `internal/modules/dnd/extract/scenedescriptions`, its codec,
|
||
shape/source/invariant validators, semantic validator integration, and assets.
|
||
|
||
**Work:** use native initial/correction requests with one description for the
|
||
operation's source range as currently required. Preserve deterministic scene IDs
|
||
and source ranges; keep the kind/title/summary contract unchanged. Add the
|
||
domain correction prompt and same-schema/profile dependency pair. Implement the
|
||
fallback rule in this plan's table: only a final usable candidate meeting the
|
||
intrinsic artifact/downstream invariants can be qualified fallback output. If
|
||
the final execution yields no usable scene, fail fallback construction rather
|
||
than emitting an empty list, inventing a classification, or recovering an earlier
|
||
candidate. Ordinary semantic revalidation is not a fallback prerequisite.
|
||
|
||
**Acceptance:** strict rejection fails; permissive handling never publishes an
|
||
invalid scene artifact or claims semantic approval. A rejected combat judgment's
|
||
guidance reaches correction, and every revised scene gets the full chain.
|
||
|
||
**Validation:** focused scene-description/semantic-validator tests with native
|
||
fakes: correction, valid-but-semantically-rejected final fallback, structurally
|
||
unusable final response, no prior-candidate recovery, and source identity.
|
||
|
||
## Stage 24: Finish Deterministic Stage Paths And Production Registration
|
||
|
||
**Goal:** all non-LLM work runs directly, and production registration is asset-free.
|
||
|
||
**Inspect/change:** generic unit chunker/append-order merger/noop normalizer,
|
||
deterministic D&D normalizers, Seriatim adapter, D&D/generic registrars, typed runner,
|
||
and module integration fixtures.
|
||
|
||
**Work:** remove outer retries and obsolete correction/repair arguments from
|
||
deterministic operations. Preserve typed merge/normalize rules, ordering, domain
|
||
validators, generated references, evidence projectors, and raw artifact codecs.
|
||
Use resolved mode only for rejected candidates with a supported module fallback;
|
||
ordinary parse/materialization/application/encoding errors propagate normally.
|
||
Register modules, validator defaults, and static dependencies without registering
|
||
prompt files, schemas, or fallback profiles. Complete migration of every LLM role
|
||
identified in Stages 12–23, not just the examples.
|
||
|
||
**Acceptance:** there is no synthetic PromptKit request for deterministic work
|
||
or generic retry setting. Durable artifact-schema validation remains; removing
|
||
duplicate model-response schema validation does not remove this application
|
||
boundary. Pipeline default-chain replacement, including explicit empty chains,
|
||
and exact typed registration still work.
|
||
|
||
**Validation:** focused generic/D&D deterministic normalizer, registrar, Seriatim,
|
||
and typed integration tests; retain data-integrity tests and remove tests whose
|
||
only purpose was the retired generic retry/asset API.
|
||
|
||
## Stage 25: Replace Worker Pools And Remove The Global LLM Scheduler
|
||
|
||
**Goal:** one ready-job limit and native backend admission govern concurrent work.
|
||
|
||
**Inspect/change:** `runner_concurrent.go`, synchronized collaborators,
|
||
CLI runtime construction, old `llm/scheduler.go`, and concurrency/cancellation tests.
|
||
|
||
**Work:** use the existing coordinator to dispatch ready extraction and ready
|
||
merge/normalize jobs to one worker pool controlled by `concurrency.workers`.
|
||
Lane completion becomes ready only after all of that lane's extract jobs finish.
|
||
Never occupy a worker while waiting for prerequisites, or acquire another worker
|
||
for nested validation/correction. Keep sequential input/chunking/publication and
|
||
ordered steps. Remove scheduled-client/global permit plumbing and worker coupling;
|
||
share the same native engine across every role. Handle native capacity failures
|
||
through the producing operation's mode without another queue or backoff.
|
||
|
||
**Acceptance:** one worker can complete a multi-lane pipeline without deadlock.
|
||
The limit bounds both job types together, independent of backend settings.
|
||
Completion timing cannot choose public order or the terminal scoped error.
|
||
Cancellation stops dispatch, drains started work, and releases native admission.
|
||
|
||
**Validation:** focused runner concurrency tests with test-controlled worker and
|
||
backend limits, including overlap, serial work, ready-lane interleaving, nested
|
||
same-backend validation, capacity failure, and cancellation. Use focused race
|
||
testing when supported; it is a migration check, not a new release requirement.
|
||
|
||
## Stage 26: Implement The Small Accepted-Artifact Filesystem Cache
|
||
|
||
**Goal:** one best-effort cache facility replaces complex persistent records.
|
||
|
||
**Inspect/change:** existing `internal/core/fileio` and cache permission/path tests;
|
||
add a focused accepted-artifact store under `internal/framework` or the existing
|
||
state area, with a narrow lookup/write collaborator for pipeline use.
|
||
|
||
**Work:** read/write accepted artifact bytes at `<cache_directory>/<hash>.json`,
|
||
using the native opaque hash unchanged. Use ordinary temporary-file/rename
|
||
publication and existing proportionate path/permission correctness helpers.
|
||
Keep roots at the CLI/state boundary. The caller owns typed codec/usability
|
||
checks. Missing entries are normal misses; unreadable/malformed/unusable entries
|
||
are misses with diagnostics. Failed writes warn without failing successful work.
|
||
Implement `auto`, `bypass`, and `refresh` I/O semantics; bypass creates no directory.
|
||
Refresh cannot remove a previous entry before a newly accepted replacement exists.
|
||
Do not add envelopes, manifests, key prefixes, expiry, migration, or collection.
|
||
|
||
**Acceptance:** records contain only the accepted artifact/plan; no execution,
|
||
configuration, transcript copy, acceptance ledger, or historical usage is required.
|
||
Creation respects umask/setgid/default ACLs under ADR-0017, without permission
|
||
convergence or added access restrictions. Legacy stores are left untouched.
|
||
|
||
**Validation:** focused real-temp-filesystem tests for hit/miss, unusable data,
|
||
read/write failure diagnostics, atomic replacement, modes, narrow paths, and
|
||
ordinary creation permissions. Reuse existing safety coverage where applicable.
|
||
|
||
## Stage 27: Cache Final Typed Producer Artifacts Under The Initial Request
|
||
|
||
**Goal:** integrate caching into the same native producer helper.
|
||
|
||
**Inspect/change:** the helper, typed artifact codecs, operation eligibility,
|
||
resolved cache controls, and cache store from Stage 26.
|
||
|
||
**Work:** for eligible cacheable LLM work in auto/refresh, prepare the initial
|
||
request with `PrepareExecution`; obtain `Details().MessageContentHash`, retain
|
||
the prepared details, and keep that key for the complete operation. In auto,
|
||
decode/usability-check a found artifact and discard the prepared handle on a hit.
|
||
On a miss execute that exact prepared handle using `RunPrepared`, then use the
|
||
same interpretation/assessment/coordination path. After final ordinary acceptance,
|
||
encode the final typed artifact and write it under the initial key. Normalizers
|
||
store the applied artifact, not the reconciliation proposal. Dispose of unused
|
||
handles on every exit. Bypass/ineligible work uses ordinary native execution and
|
||
performs no cache I/O. Capture must not cause another preparation.
|
||
|
||
**Acceptance:** a hit makes no model calls, semantic validation, exchange replay,
|
||
or historical usage claims. Models/settings/session/cache hints/schema metadata
|
||
add no key dimensions; only the upstream ordered message-content value selects
|
||
reuse. Changed policy can deliberately reuse old artifacts until refresh/bypass.
|
||
Ordinary usability includes compatibility with the expected codec and current
|
||
source addressing; it does not rerun the configured domain/semantic validator chain.
|
||
Failed/rejected/incomplete/fallback and affected dependent operations neither
|
||
load nor store entries. Corrected accepted output uses the initial key.
|
||
|
||
**Validation:** focused helper/store tests: content changes miss, settings-only
|
||
changes can hit, ordinary usability failure misses, late accepted write, no write
|
||
after terminal failure/cancellation/fallback, eligible predecessor reuse, and
|
||
native prepared details retained on a hit without an execution event.
|
||
|
||
## Stage 28: Unify Chunk-Plan Reuse And Remove Checkpoint Execution Paths
|
||
|
||
**Goal:** LLM chunk plans use the same accepted-result cache and no second store.
|
||
|
||
**Inspect/change:** `runner_chunk_plan.go`, `chunk_cache.go`, `chunk_plan_store.go`,
|
||
checkpoint hooks in runner/preparation/handoff, CLI cache composition, and old
|
||
`internal/framework/{chunkplan,checkpoint}` consumers.
|
||
|
||
**Work:** cache a generated plan only after materialization and complete chunk
|
||
assessment/continuation acceptance, under its initial request's native hash.
|
||
On a hit decode/rematerialize it against the current source with ordinary
|
||
structural/source-reference usability checks, without semantic revalidation.
|
||
An unusable cached plan is a miss, not a defective assistant response. Keep
|
||
deterministic chunkers uncached without synthetic requests. Remove source-digest
|
||
plan lookup/publication, checkpoint event/state transitions, stage/dependency
|
||
fingerprints, resume barriers, and recomputation bookkeeping from execution.
|
||
Old records are unread and unmodified, even below the selected new cache root.
|
||
|
||
**Acceptance:** changed chunk request messages can select a different plan;
|
||
source identity alone no longer pins a partition. Default reruns reuse accepted
|
||
predecessors without `--resume`. Chunk fallback and dependent work skip reuse.
|
||
No checkpoint identity is retained solely to fill diagnostics.
|
||
|
||
**Validation:** focused chunk/runner/cache tests with fake providers: matching
|
||
plan reuse, changed references/prompt text, settings-only reuse, invalid cached
|
||
source ranges, strict/permissive outcomes, bypass/refresh, and legacy files left
|
||
unchanged. Remove old canonical-partition/resume-policy tests as superseded.
|
||
|
||
## Stage 29: Publish Only Domain Artifacts At Existing Paths
|
||
|
||
**Goal:** simplify the logical output boundary without changing payload contracts.
|
||
|
||
**Inspect/change:** `internal/modules/generic/output/json/encoder.go`,
|
||
`internal/core/artifacts`, output contracts/runner result assembly, evidence output,
|
||
and chunk-map publication. Inspect the existing lane filename mapping.
|
||
|
||
**Work:** keep run-specific `lanes/<safe-lane-id>.json`, optional `chunk-map.json`,
|
||
and optional `evidence-context.json`. Preserve domain schemas, exact lane mapping,
|
||
and union-of-cited-source-segments evidence behavior. Remove mandatory index,
|
||
manifest, rejected/warnings/diagnostics sidecar generation and reporting-only
|
||
models/metadata. Return the small application outcome and logical deliverable
|
||
files needed for publication, fallback disclosure, and optional capture.
|
||
Intrinsic domain metadata remains intrinsic, not discarded as reporting data.
|
||
|
||
**Acceptance:** codecs and normal downstream use need no metadata bundle or
|
||
persisted receipt. Artifact count includes actual published domain deliverables,
|
||
including reused artifacts and enabled auxiliaries, not intermediates/cache/debug.
|
||
Fallback artifacts retain required downstream shape without normal-acceptance claims.
|
||
|
||
**Validation:** focused encoder/evidence/chunk-map/codec tests verifying preserved
|
||
payloads and paths, no mandatory sidecars, optional artifact inclusion, accurate
|
||
counting, and evidence source order/window behavior.
|
||
|
||
## Stage 30: Build The Debug Recorder And Session Lifecycle
|
||
|
||
**Goal:** requested capture has one simple recorder and explicit failure semantics.
|
||
|
||
**Inspect/change:** `internal/core/debugbundle`, `internal/framework/debug`,
|
||
pipeline debug collaborators, run IDs, and CLI debug/state composition.
|
||
|
||
**Work:** replace the old summary/trace hierarchy with an application recorder
|
||
for run context, operation evidence, and one final outcome. Implement
|
||
`<debug-root>/<run-id>/` and start-time relative `latest`, without completion-time
|
||
retargeting. Resolve the selected native debug boolean and CLI/config/empty/default
|
||
directory semantics. Allocate only on the run path after config/run identity and
|
||
before engine construction/preflight. Persist incrementally and finalize once.
|
||
Allocation/link failure prevents model work; recording failure surfaces at normal
|
||
boundaries; finalization failure fails the invocation. Preserve an existing primary
|
||
failure and available data/publication paths; never retry a model for capture failure.
|
||
|
||
**Acceptance:** a configured directory alone never enables capture; explicit empty
|
||
disables it; a nonempty CLI override can re-enable it. Disabled runs leave `latest`
|
||
untouched. Quiet mode and capture selection are independent. No automatic bundle
|
||
deletion, fallback directory, capture-depth setting, or new filesystem framework.
|
||
|
||
**Validation:** focused temp-filesystem lifecycle tests: absent/empty/nonempty
|
||
selection, start failure, failed run retention, concurrent start/link semantics,
|
||
incremental recording failure, finalization failure, and no completion retarget.
|
||
|
||
## Stage 31: Attach Native Execution Facts And Complete Debug Evidence
|
||
|
||
**Goal:** use native observations/snapshots instead of parallel execution accounting.
|
||
|
||
**Inspect/change:** helper/validator request observers, runner operation boundaries,
|
||
new recorder, old `pipeline/debug.go`, and PromptKit observation/exchange APIs.
|
||
|
||
**Work:** create one invocation `ExecutionGroup`. Enroll the initial request once,
|
||
continuations once through `CoordinateRequest.Observer`, and nested validators
|
||
once on their own requests; do not also attach the same group to continuation
|
||
requests. Compose scope-aware diagnostic/capture observers without replacing
|
||
existing ones. Use native RunID/ParentRunID and totals, preserving unknown usage
|
||
and terminal coverage. Keep `OnPrepared` lightweight; retain its detached details
|
||
for normal boundary writes. For split prepared/cache paths use existing details.
|
||
|
||
Record the roadmap's full available inventory: effective secret-aware config and
|
||
composition; parsed source; self-contained source/reference/normalization inputs;
|
||
prepared requests/output contracts; native events/results/raw response text;
|
||
all visible exchanges/readable messages; typed candidates, findings/guidance,
|
||
correction inputs, fallbacks, deterministic results, and cache decisions. Label
|
||
unassessed `Undelivered` responses and failures lacking prepared/result evidence.
|
||
Use convenient indented JSON/text without truncation, fabricated provider bodies,
|
||
heuristic scrubbing, or whole-run content accumulation. Exact filenames stay private.
|
||
|
||
**Acceptance:** native totals count executions rather than producer attempts;
|
||
cache hits add none. Debug needs no second prepare or model call. Shared callbacks
|
||
are concurrency-safe; observer write failures reach the next boundary. ConfigKit
|
||
secrets/native credentials remain omitted without rewriting ordinary application
|
||
content. No duplicate usage accumulator, synthetic call-ID model, or profile ledger.
|
||
|
||
**Validation:** focused helper/recorder tests for initial+continuation+nested calls,
|
||
result-less terminal failures, cache hit details, canceled/undelivered turns,
|
||
full untruncated selected content, native credential omissions, and concurrent
|
||
operations. Check meaningful evidence, not the complete native JSON shape.
|
||
|
||
## Stage 32: Add Offline Preflight And Pipeline Inspection Commands
|
||
|
||
**Goal:** execution and inspection share domain resolution and known dependencies.
|
||
|
||
**Inspect/change:** new `internal/preflight`, CLI pipeline declarations, catalog,
|
||
profile/reference/module resolution, and existing explicit-profile preflight tests.
|
||
|
||
**Work:** implement `pipeline list`, `show <pipeline-id>`, and
|
||
`validate <pipeline-id>`. List only enumerates configuration. Show reports resolved
|
||
steps/lanes/modules/validators/references without building an engine merely to
|
||
expand every profile. Validation reuses resolver checks and, when LLM dependencies
|
||
are selected, the same captured engine and static module declarations as runtime.
|
||
Check initial/correction/validator prompts, explicit/applicable/default profiles,
|
||
and known dependencies with pipeline/stage/lane context. Remove the old pipeline
|
||
mode of `config validate`; that standard command validates resolved configuration.
|
||
|
||
**Acceptance:** offline preflight needs no input transcript, reference file bytes,
|
||
credential values, provider call, output/cache/debug root, or synthetic request.
|
||
Valid partial profiles pass. Dynamic request selections are checked at runtime.
|
||
Preflight does not load configuration, register modules, construct engines, or
|
||
render CLI reports; it accepts those collaborators/data from composition.
|
||
|
||
**Validation:** focused preflight/CLI tests for known missing prompt/profile,
|
||
validator and correction dependencies, defaults with no explicit profile, valid
|
||
partial profile, no-credential/no-transcript operation, and useful scope context.
|
||
|
||
## Stage 33: Wire Native Pipeline Run Inputs And Runtime Composition
|
||
|
||
**Goal:** the executable runs the migrated pipeline through one native handler.
|
||
|
||
**Inspect/change:** CLI application/pipeline commands, `run.go`, catalog,
|
||
reference selectors, `session.go`, run IDs, configuration/engine/state loaders,
|
||
and `cmd/notarius`.
|
||
|
||
**Work:** install `pipeline run <pipeline-id> --input-file PATH` and
|
||
`commandkit.Alias("run", "pipeline run")`. Use typed supplied values for `--lane`,
|
||
`--output-dir`, `--cache`, references/unbinds, debug directory, and ckprompt
|
||
overrides. Keep repeated lanes literal (commas are not separators); validate every
|
||
singleton occurrence and retain the last successfully parsed value. Expand
|
||
reference selectors before framework resolution using the existing specificity,
|
||
required-slot, external/generated-conflict, and file-origin policies. Preserve
|
||
deterministic input-based session generation, independent of references, and
|
||
native explicit session override behavior. Apply run-wide profile/execution
|
||
overrides to producers, validators, and continuation requests.
|
||
|
||
Reuse the one loaded selection and one engine, resolve/preflight before source
|
||
execution, then materialize references and prepare typed modules. Allocate requested
|
||
debug before engine/preflight as specified; keep all physical roots at composition.
|
||
Remove old dispatch, assembled help, parsers, comma `--only`, bespoke `--json`,
|
||
reasoning-clearing flag, resume/recompute/chunk-cache flags, and legacy aliases.
|
||
|
||
**Acceptance:** native alias and canonical route invoke the same declarations/
|
||
handler; `--`, operands, repeated flags, modes, and supplied empty/false/zero retain
|
||
upstream behavior. No forwarding command, recursive command invocation, fallback
|
||
to another config for explicit empty input, or request override mutation of config.
|
||
|
||
**Validation:** focused command/reference/session tests and subprocess cases for
|
||
the two run routes, multi-reference input, literal comma lane, required input,
|
||
native overrides reaching all roles, explicit empty reasoning/debug path, and
|
||
input-hash session stability across changed reference bundles.
|
||
|
||
## Stage 34: Finish Publication, Native Completion Reports, And Failure Outcomes
|
||
|
||
**Goal:** subprocesses receive artifacts plus the selected small native report.
|
||
|
||
**Inspect/change:** `run_result.go`, `run_terminal.go`, physical output placement,
|
||
CommandKit reporter/outcomes, debug finalization, and subprocess receipt tests.
|
||
|
||
**Work:** publish domain files in the existing run-specific destination, finish
|
||
requested capture, then call `Reporter.Report` with pipeline, artifact_count,
|
||
absolute output_directory, optional debug_directory, and positive-only
|
||
fallback_count. Supply the selected human text template from that same value.
|
||
Remove separate encoders, receipt/error envelopes, automatic report persistence,
|
||
warning-count summaries, and durable execution ledgers. Send native diagnostics
|
||
to stderr and results to stdout, preserving quiet/mode/progress semantics and
|
||
fallback qualification even when warnings are hidden.
|
||
|
||
Map native outcomes: completion `0`, operational/cancellation/publication/
|
||
presentation failure `1`, identifiable invalid invocation/configuration `2`, native
|
||
diff differences `3`. Preserve typed underlying causes and distinguish operation
|
||
failure from report/diagnostic delivery failure. A post-publication capture/report
|
||
failure identifies available artifact/debug paths, returns nonzero, and does not
|
||
rerun or remove valid artifacts. Failed stdout may be partial; consumers check status.
|
||
|
||
**Acceptance:** `--format json --verbose` has report-only valid JSON on stdout
|
||
for success. Positive fallback_count counts emitting operations once, not affected
|
||
artifacts, attempts, or validators. Quiet suppresses stdout but not useful failure
|
||
explanations. Native standard commands have no Notarius-wide envelope.
|
||
|
||
**Validation:** focused subprocess tests for text/JSON/YAML success, fallback and
|
||
zero omission, verbose stderr, quiet/debug order, render/delivery failure,
|
||
publication then capture failure, useful exit categories, and no reporting sidecars.
|
||
|
||
## Stage 35: Remove Superseded Machinery And Restore Whole-Tree Compilation
|
||
|
||
**Goal:** delete the old architecture rather than leave dormant alternatives.
|
||
|
||
**Inspect/change:** all production/test callers of the feature roadmap's removal
|
||
inventory; `internal/framework/llm`, `promptfs`, `readonlyfs` consumers,
|
||
checkpoint/chunkplan stores, private completion/correction/schema types,
|
||
manifest/receipt models, and old CLI tests/dependencies.
|
||
|
||
**Work:** remove now-unused adapters, schedulers, asset/schema registries,
|
||
flattening/mount helpers, effective-YAML walking, fingerprints, manual correction
|
||
transport/capability checks, generic retries, profile/usage conversions, debug
|
||
wrappers, state ledgers, and old command surfaces. Retain useful pure helpers only
|
||
where a current consumer needs them, in the actual owning package. Do not keep an
|
||
empty `internal/framework/llm` for construction. Adapt remaining tests to native
|
||
provider boundaries and final contracts; delete tests solely asserting retired
|
||
behavior, not tests protecting unchanged domain correctness. Run tidy and remove
|
||
old-path kits/catalogs after their last imports disappear; reconcile notices.
|
||
|
||
**Acceptance:** every package and test compiles with the declared dependencies.
|
||
Production code has one native path and no legacy compatibility scaffold. Removal
|
||
does not delete output/cache/debug directories or rewrite accepted ADR decisions.
|
||
Pure domain/codecs remain free of unnecessary library dependencies.
|
||
|
||
**Validation:** `GOWORK=off go mod tidy`, `GOWORK=off go test ./... -run '^$'`,
|
||
`go list -m all`, and scoped code searches for the removal inventory. Classify
|
||
historical release/ADR/archive mentions rather than blindly deleting text matches.
|
||
|
||
## Stage 36: Migrate The Maintained Minimal And Complete Examples
|
||
|
||
**Goal:** two useful D&D configurations demonstrate the final native workflow.
|
||
|
||
**Inspect/change:** `examples/dnd-{minimal,complete}.config.yml`, pipeline imports,
|
||
profiles/overlay files, transcript/reference fixtures, and
|
||
`internal/cli/example_contract_test.go`.
|
||
|
||
**Work:** organize the two examples as `examples/dnd-minimal/config.yml` and
|
||
`examples/dnd-complete/config.yml`, moving their required imports/fixtures and
|
||
fixing paths. Separate directories prevent the complete example's companion
|
||
resources from changing the minimal example. Minimal uses embedded defaults and
|
||
a simple pipeline. Complete uses every configured D&D family and ordered generated
|
||
references, with optional chunk-map/evidence output; demonstrate native companion
|
||
profiles, inheritance/operator override, and a limited backend without overriding
|
||
the native queue default unnecessarily. Use version 6, qualified IDs, new worker/
|
||
cache/mode/turn controls, and no old retry/structural-repair fields. Replace the old
|
||
development variant with a native profile/prompt customization or semantic-budget
|
||
example, not a compatibility translation. Keep all files secret-free.
|
||
|
||
**Acceptance:** both actual config paths load, resolve, and pass offline pipeline
|
||
preflight without credential values. Imported paths are declaration-relative.
|
||
The minimal example has no accidental companion dependency. Retire superseded
|
||
copyable configurations, retaining useful shared test/reference data as needed.
|
||
|
||
**Validation:** maintained example-loading tests plus native configuration and
|
||
pipeline validate commands for both paths. Fake-provider integration can exercise
|
||
them; live OpenRouter/local services are not default validation requirements.
|
||
|
||
## Stage 37: Verify Assembled Recovery And Native Command Workflows
|
||
|
||
**Goal:** prove the cross-boundary behavior most likely to regress in this cutover.
|
||
|
||
**Inspect/change:** CLI assembled/command/cache contracts, module integration
|
||
tests, and new provider-boundary fixtures. Consolidate existing coverage rather
|
||
than creating a second comprehensive fixture framework.
|
||
|
||
**Work:** exercise a representative ordered multi-step pipeline with typed
|
||
generated references and one native engine. Include the representative extractor,
|
||
normalizer, LLM validator, and deterministic paths. Add focused assembled cases
|
||
for downstream failure/fix/rerun reuse, correction stored under the initial key,
|
||
fallback taint across handoffs versus unaffected predecessors, chunk-plan reuse,
|
||
refresh failure preserving an old entry, native limited-backend admission and
|
||
serial workers, debug on hits/failure, and report/stream outcomes. Ensure no
|
||
unexpected external calls in help/inspection/preflight, or duplicate preparation
|
||
for debug. Cover custom operator schemas/contracts through real preparation;
|
||
where an actual request-level override is needed, use a complete native
|
||
`OutputContractInput` with `RepairAttempts` unset unless the operator supplied it.
|
||
|
||
**Acceptance:** behavior follows the feature roadmap's full acceptance matrix,
|
||
including result-less failures, unknown usage, native status distinctions, final
|
||
turn assessment, all-validators guidance, source/session/reference preservation,
|
||
and unchanged intrinsic payloads. Avoid duplicating upstream parser/renderer/
|
||
retry tests, pinning upstream default attempt counts, or snapshotting full help/
|
||
native debug objects. No production model-quality claim is inferred from fakes.
|
||
|
||
**Validation:** `GOWORK=off go test ./internal/cli ./internal/modules/integration
|
||
./internal/framework/pipeline ./internal/framework/semanticreconcile`; run focused
|
||
race checks for shared observers/coordinator when practical. Stage 46 owns the
|
||
complete final suite after documentation and remaining corrections.
|
||
|
||
## Stage 38: Record Successor Decisions And Align Architecture Policy
|
||
|
||
**Goal:** document deliberate boundary changes without rewriting their history.
|
||
|
||
**Inspect/change:** `docs/policy/architecture.md`, ADRs 0005–0007, 0011, 0014,
|
||
0016, 0019, and the retained 0012/0013/0015/0017/0020 decisions. Inspect the current
|
||
next ADR number before adding sequential records.
|
||
|
||
**Work:** write concise successor ADRs for (1) native kit composition/execution/
|
||
admission and prompt-declared assets, (2) native continuation with application
|
||
assessment/modes, (3) unified accepted-artifact caching, and (4) artifact-first
|
||
publication/native reporting and debug ownership. Use the next available numbers,
|
||
starting at 0021 if none have since been added. Change earlier statuses/links to
|
||
identify full or partial supersession, preserving accepted decision bodies and
|
||
retained content-only asset, contextual-ID, process-warning, deployment-permission,
|
||
and ConfigKit ownership decisions. Architecture now describes native library
|
||
types as an intentional execution boundary, one worker pool/native admission,
|
||
new retry/fallback/cache semantics, optional diagnostic records, and current
|
||
output/debug responsibilities.
|
||
|
||
**Acceptance:** no current policy still mandates the retired private LLM API,
|
||
global scheduler, latest-only replay, source-only plan partition, checkpoint
|
||
resume, manifests, or heuristic sanitization. Retained source/domain/type and
|
||
filesystem correctness boundaries remain, with no new access policy, storage
|
||
guarantee, release gate, or public debug format. Keep policy provenance headers
|
||
accurate; this is a local architecture update, not a new upstream policy adoption.
|
||
|
||
**Validation:** review each changed invariant against the feature roadmap and
|
||
implemented behavior. Check ADR status links and accepted-body preservation;
|
||
`git diff --check`. Do not rewrite historical release notes or archived roadmaps.
|
||
|
||
## Stage 39: Rewrite Canonical Configuration And CLI References
|
||
|
||
**Goal:** operators can configure and invoke only the implemented native surface.
|
||
|
||
**Inspect/change:** `docs/config.md`, `docs/cli.md`, final command declarations,
|
||
ConfigKit definition, resolver tests, and the two maintained examples.
|
||
|
||
**Work:** document native discovery and companions under one result; version 6;
|
||
qualified resources, replacement/inheritance/null semantics and schema roots;
|
||
native profile/backend controls; worker and semantic-turn controls; strict/
|
||
permissive precedence; and cache settings. Explain no backend concurrency default
|
||
is injected and no independent Notarius structural-repair count is pinned.
|
||
Document new pipeline commands/alias, required operands, supplied values, repeated
|
||
lanes/references, run-wide ckprompt flags, native formats/modes/progress, stdout/
|
||
stderr, and outcome codes. Link upstream-owned standard command semantics where
|
||
appropriate, with short Notarius-specific selection examples rather than a
|
||
copied upstream manual. Link full copyable files in `examples/`.
|
||
|
||
**Acceptance:** current references describe implemented behavior only. Retired
|
||
flags/fields occur solely in clearly identified migration material, not usable
|
||
syntax. Config fields/defaults belong in the configuration owner, invocation
|
||
syntax in the CLI owner; neither duplicates physical state or artifact schemas.
|
||
|
||
**Validation:** compare documented commands with native help and focused parser/
|
||
loading tests. Check relative links/anchors and example paths. Test omitted and
|
||
explicit values through declarations, not prose-based change detectors.
|
||
|
||
## Stage 40: Update Operations And Downstream Consumer Contracts
|
||
|
||
**Goal:** explain the intentional persistence and subprocess breaking changes.
|
||
|
||
**Inspect/change:** `docs/operations.md`, `docs/consumers/{subprocess,dnd-pipeline}.md`,
|
||
`docs/integrations/run-result.md`, JSON/chunk-map/evidence integration contracts,
|
||
and any release-runbook command references affected by the CLI cutover.
|
||
|
||
**Work:** describe the independent output/cache/debug roots, per-user cache default
|
||
and `/var/cache/notarius` system convention, reuse modes, best-effort I/O, unchanged
|
||
legacy data, and explicit refresh for policy/model experiments. Explain fallback
|
||
qualification and reuse exclusion, whole-source chunk fallback, debug selection/
|
||
empty paths/start-time `latest`, retained evidence, and capture/publication failure.
|
||
Delete the superseded persisted receipt contract and replace consumer links with
|
||
the CLI report contract plus documented domain paths; do not create a new manifest
|
||
contract. Consumers check status, optionally decode the small report, then read
|
||
configured lane/auxiliary paths. Note partial stdout on delivery failure and valid
|
||
artifacts after post-publication failure.
|
||
|
||
Add `docs/consumers/native-migration.md` covering old-to-new commands,
|
||
configuration/resources/catalog IDs, native retry intent, removed transient/repair
|
||
wrapper controls, workers/admission, cache reuse, and receipt traversal replacement.
|
||
Link it from canonical entry points. Exact old provider-attempt parity and automatic
|
||
cache conversion are not promised. Keep tagged release notes unchanged; preparing
|
||
or publishing a new release is outside this sprint.
|
||
|
||
**Acceptance:** the complete D&D subprocess workflow uses `--input-file`,
|
||
`--output-dir`, and pipeline-level references as first-class inputs. Documentation
|
||
does not require removed sidecars, depend on debug layout as an API, claim cache
|
||
revalidation, or add new provisioning/retention restrictions. Artifact contracts
|
||
preserve their payload schemas and logical paths.
|
||
|
||
**Validation:** exercise maintained subprocess examples with a fake provider and
|
||
compare resulting files/report. Check all references to the removed receipt,
|
||
old cache families, and legacy flags; retain necessary migration/history mentions.
|
||
|
||
## Stage 41: Refresh Configuration, CLI, And LLM Internal Guides
|
||
|
||
**Goal:** explain the actual native composition and execution boundary.
|
||
|
||
**Inspect/change:** `docs/internal/{configuration,cli,llm}.md`; the final
|
||
configuration definition, composition-root loaders, command declarations, engine
|
||
construction, and native producer/validator call sites.
|
||
|
||
**Work:** document one result handoff, source assembly, invocation reuse, standard
|
||
suite wiring, request overrides, native execution/continuation, and application
|
||
interpretation. Keep `llm.md` as an integration-focused explanation even though
|
||
its old package is removed. Link canonical operator and upstream contracts rather
|
||
than reproducing their fields, flags, or library mechanics. Remove old schemas/
|
||
asset registration, provider-neutral adapters, scheduler, profile recording, and
|
||
inspection-conversion descriptions.
|
||
|
||
**Acceptance:** a contributor can locate the current owner without being sent to
|
||
a deleted wrapper. Native execution types are explicit; pure domain types remain
|
||
separate. Volatile public details have one canonical owner.
|
||
|
||
**Validation:** inspect the named packages and focused tests for every flow;
|
||
check cross-links and stale package references.
|
||
|
||
## Stage 42: Refresh Pipeline And State Internal Guides
|
||
|
||
**Goal:** explain acceptance, concurrency, recovery, and capture as implemented.
|
||
|
||
**Inspect/change:** `docs/internal/{pipeline,state}.md`; typed helper/assessment,
|
||
worker coordinator, eligibility propagation, cache/store, recorder, and publication.
|
||
|
||
**Work:** document initial/revised assessment, native semantic continuation,
|
||
strict/permissive terminal handling, supported fallback, ready-job workers,
|
||
generated-reference eligibility, accepted-artifact/plan reuse, and output/debug
|
||
ownership. Remove checkpoint fingerprints/resume, old chunk-plan canonicalization,
|
||
duplicate usage/debug wrappers, required manifest/receipt, and generic retry flows.
|
||
Link public state controls, artifact paths, and output contract definitions.
|
||
|
||
**Acceptance:** current-flow descriptions distinguish native execution success,
|
||
domain acceptance, qualified fallback, reuse, and publication. No internal guide
|
||
promises a public debug layout or cache revalidation policy that does not exist.
|
||
|
||
**Validation:** compare diagrams/prose with runner/cache/recorder and focused
|
||
tests; verify no stale state collaborator or operation ordering is carried forward.
|
||
|
||
## Stage 43: Refresh Module And D&D Contributor Guides
|
||
|
||
**Goal:** future modules can follow the native architecture without copying drift.
|
||
|
||
**Inspect/change:** `docs/internal/{modules,dnd}.md`; production registration,
|
||
representative spell producer, registry normalizers, scene chunker/validator,
|
||
native initial/correction assets, codecs, and shared domain helpers.
|
||
|
||
**Work:** show how a module prepares fixed inputs, builds initial/correction native
|
||
requests, interprets results, supplies fallback, and receives pipeline assessment.
|
||
Document static dependency declarations and native prompt/schema ownership instead
|
||
of manifests/registration. Explain one-time normalization handles, fresh proposal
|
||
application, domain guidance, unchanged contextual-ID/source-reference rules, and
|
||
deterministic paths without synthetic requests. Link durable schemas and public
|
||
module configuration rather than repeating them.
|
||
|
||
**Acceptance:** the extension recipe works for the actual typed interfaces and
|
||
retains artifact-family/domain boundaries. It requires no private completion
|
||
interface, schema registry, fingerprint, or module-owned retry engine.
|
||
|
||
**Validation:** walk the recipe against one extraction and one normalization
|
||
implementation; inspect native asset paths/IDs and retained domain regression tests.
|
||
|
||
## Stage 44: Align The Kit Integration Contracts
|
||
|
||
**Goal:** describe Notarius's actual use of all three native libraries.
|
||
|
||
**Inspect/change:** `docs/integrations/{pkg-configkit,pkg-promptkit}.md`; create
|
||
`docs/integrations/pkg-commandkit.md` for the implemented CLI integration. Inspect
|
||
selected public APIs, final module versions, source/engine loaders, standard suite
|
||
wiring, request execution, and reporting.
|
||
|
||
**Work:** update supported integration/version references and Notarius's supplied
|
||
configuration, source contexts, runtime/request values, outcomes, and ownership.
|
||
Document the single application result/focused views, native prompt/schema
|
||
selection, continuation/admission, and standard CLI composition at the consumer
|
||
boundary. Link tagged upstream guides for native mechanics and canonical local
|
||
docs for product policy. Describe no unused future API, full copied upstream
|
||
manual, compatibility adapter, or private implementation as a public protocol.
|
||
|
||
**Acceptance:** the integration docs agree with declared dependencies and current
|
||
calls; CommandKit/ckconfig/ckprompt adoption is complete, not deferred. Notices and
|
||
integration source links resolve to the intended components/revisions.
|
||
|
||
**Validation:** inspect imports/call sites and local/upstream link destinations;
|
||
compare claims with Stage 37's assembled boundary tests.
|
||
|
||
## Stage 45: Refresh Orientation And Reconcile Planning Status
|
||
|
||
**Goal:** entry points lead to current docs and preserve separate future scope.
|
||
|
||
**Inspect/change:** `docs/internal/overview.md`, `docs/development.md`, `README.md`,
|
||
and `docs/roadmap/future.md`, plus status in this plan and the feature roadmap.
|
||
|
||
**Work:** update package inventory/navigation for the actual composition root,
|
||
producer helper, cache, and recorder. Remove obsolete receipt/LLM-package reading
|
||
directions. Give README the shortest valid native `notarius run` quickstart and
|
||
links to the two examples, current reference docs, and migration guide. Link the
|
||
new kit integration contract where useful. Reconcile backlog items implemented
|
||
by this cutover without adding deferred features. Mark stages complete only from
|
||
actual evidence; leave final verification pending until Stage 46. Do not delete
|
||
the active plan/feature roadmap or prepare a release in this stage.
|
||
|
||
**Acceptance:** orientation describes implemented behavior and routes to canonical
|
||
owners. Future scope stays in the backlog; historical ADR/release/archive records
|
||
are not rewritten to match new syntax.
|
||
|
||
**Validation:** check quickstart/example paths and navigation, then search current
|
||
docs/examples for stale names while retaining meaningful migration/history mentions.
|
||
|
||
## Stage 46: Verify The Complete Native Cutover
|
||
|
||
**Goal:** finish only when the full end state is supported by current evidence.
|
||
|
||
**Inspect/change:** the assembled repository, feature roadmap acceptance matrix
|
||
and removal inventory, stage evidence, maintained examples, dependencies/notices,
|
||
and documentation links. Make final focused corrections where evidence exposes
|
||
an actual gap; do not invent another feature or compatibility layer.
|
||
|
||
**Work:** verify every acceptance row against tests or direct inspection of its
|
||
owning boundary. Close compilation blockers recorded during intermediate stages.
|
||
Confirm all production LLM roles use the one engine/native path, domain artifacts
|
||
are unchanged, new standard commands/loaders share configuration, and no completed
|
||
decision remains deferred to a later CommandKit adoption. Confirm old adapters,
|
||
retry loops, global limits, schema registration, both legacy caches, required
|
||
reporting sidecars, and copied accounting have no current consumers. Review
|
||
dependency attribution and absence of permanent replacement modules.
|
||
|
||
Run final development checks with declared dependencies and no sibling workspaces:
|
||
|
||
```sh
|
||
GOWORK=off go test ./...
|
||
GOWORK=off go vet ./...
|
||
GOWORK=off go build ./cmd/notarius
|
||
git diff --check
|
||
```
|
||
|
||
Use a temporary binary destination if an ordinary build would leave an untracked
|
||
binary. Also run maintained example/preflight and representative native help/
|
||
version/completion/format checks not already covered by the default suite.
|
||
Inspect removal matches in current Go/config/example files, rather than requiring
|
||
historical docs to contain none. Check local links with a small read-only checker
|
||
or manual verification if the repository has no link-check facility.
|
||
|
||
**Acceptance:** final tests/vet/build pass; retained domain regressions and all
|
||
meaningful cross-boundary cases have coverage; examples and canonical docs agree
|
||
with the executable. Native timeout/cancellation, structural-failure status,
|
||
final-turn assessment, fallback disclosure/eligibility, cache hits, worker liveness,
|
||
and post-publication errors are verified. No live-model run, production key,
|
||
release publication, permanent new gate, or comprehensive performance benchmark
|
||
is required. Report any genuinely unverified result instead of declaring success.
|
||
|
||
## Coverage Map
|
||
|
||
| Feature-roadmap target | Primary stages |
|
||
| --- | --- |
|
||
| U1 dependency baseline and native handoff | 1–7, 35 |
|
||
| U2 embedded/native prompt and schema ownership | 4–5, 12–24, 35 |
|
||
| U3 typed modules and shared producer assessment | 8–12, 24 |
|
||
| U4 prompt pairs, guidance, complete revalidation | 9–23, 37 |
|
||
| U5 native retries and semantic-turn allowance | 3, 9–10, 13–24 |
|
||
| U6 strict/permissive outcomes and supported fallbacks | 3, 9–24, 27–28, 34, 37 |
|
||
| U7 workers and native backend admission | 3, 17, 25, 37 |
|
||
| U8 accepted-artifact and chunk-plan cache | 3, 11, 26–28, 37 |
|
||
| U9 native execution facts, artifact delivery, capture | 11, 29–31, 34, 37 |
|
||
| U10 complete native CLI and shared offline preflight | 5–7, 32–34, 36–37 |
|
||
| Successor decisions, canonical docs, migration, backlog | 38–45 |
|
||
| Entire acceptance matrix and removal inventory | 35, 37, 46 |
|
||
|
||
No architectural decisions remain open for this implementation. Private API names,
|
||
debug filenames, and routine serialization choices stay within the selected
|
||
boundaries; they do not require another decision cycle or stronger contracts.
|