Plan the native PromptKit and CommandKit migration
This commit is contained in:
1369
docs/roadmap/implementation.md
Normal file
1369
docs/roadmap/implementation.md
Normal file
File diff suppressed because it is too large
Load Diff
@@ -8,13 +8,13 @@ presentation directly. Retain the application work of producing, assessing,
|
||||
publishing, and reusing typed artifacts. Historical duplication of library
|
||||
behavior is not a requirement to preserve it.
|
||||
|
||||
This is the single migration plan for PromptKit, ConfigKit, CommandKit,
|
||||
This is the feature roadmap for PromptKit, ConfigKit, CommandKit,
|
||||
accepted-artifact caching, and artifact-first output. It owns the selected target,
|
||||
decision status, implementation sequence, removal inventory, acceptance
|
||||
criteria, and supporting evidence. The earlier discussion documents are retired;
|
||||
Git history preserves the discussion. All implementation phases are pending.
|
||||
Recording these decisions has not changed application behavior. Unrelated future
|
||||
work remains in the [separate backlog](future.md).
|
||||
decision status, removal inventory, acceptance criteria, and supporting evidence.
|
||||
The earlier discussion documents are retired; Git history preserves the discussion.
|
||||
The [implementation plan](implementation.md) owns the ordered, prompt-sized stages.
|
||||
The migration is pending; recording these decisions has not changed application
|
||||
behavior. Unrelated future work remains in the [separate backlog](future.md).
|
||||
|
||||
Notarius predates many modern PromptKit capabilities: the library grew out of
|
||||
mechanics shared by Notarius and related applications. Historical duplication
|
||||
@@ -721,8 +721,8 @@ Record why fallback was used and expose that qualification in the ordinary
|
||||
completion report, even when diagnostic verbosity hides individual warnings.
|
||||
Use the same application outcome for reporting and optional debug detail. Keep
|
||||
conversation state in process, outside ordinary artifact/cache payloads;
|
||||
explicitly requested detailed debug is separate. U6 is resolved; implementation
|
||||
and module-specific construction details remain in Phases 3–4.
|
||||
explicitly requested detailed debug is separate. U6 is resolved; module-specific
|
||||
construction remains implementation work.
|
||||
|
||||
### Admission And Pipeline Workers (U7 Resolved)
|
||||
|
||||
@@ -809,7 +809,7 @@ new shared limit, but its scope now includes lane completion. Do not automatical
|
||||
translate a cross-backend ceiling into per-backend limits or preserve worker
|
||||
inheritance/caps from that field. Provide a native limited-backend example with
|
||||
usable queue capacity, without requiring operators to override the upstream
|
||||
queue default. U7 is resolved; implementation remains in Phase 4.
|
||||
queue default. U7 is resolved; implementation remains pending.
|
||||
|
||||
### Accepted-Artifact Cache (U8 Resolved)
|
||||
|
||||
@@ -955,8 +955,8 @@ source partition to request-content-based plan reuse.
|
||||
Remove checkpoint identities, dependency fingerprints, invocation-wide barriers,
|
||||
recovery bookkeeping, and the separate canonical-plan lookup/publication policy.
|
||||
Retain domain plan interpretation and materialization without retaining a second
|
||||
cache mechanism. U8 is resolved; implementation remains pending in Phases 3–5
|
||||
using U1's selected upstream hash capability.
|
||||
cache mechanism. U8 is resolved; implementation remains pending using U1's
|
||||
selected upstream hash capability.
|
||||
|
||||
### Inspection And Execution Bookkeeping
|
||||
|
||||
@@ -1221,36 +1221,36 @@ support, applicability, and outcome distinctions.
|
||||
U1–U10 are resolved. There are no remaining open architectural or dependency
|
||||
choices in this roadmap. Observer wiring, debug filenames, routine API placement,
|
||||
and serialization details remain implementation work within the selected designs.
|
||||
IDs retain continuity with the earlier roadmap; all implementation phases remain
|
||||
pending.
|
||||
IDs retain continuity with the earlier roadmap. Implementation status and ordered
|
||||
stages are tracked in the [implementation plan](implementation.md).
|
||||
|
||||
**U1 resolved:** The [reviewed dependency set](#baseline-and-dependency-target)
|
||||
selects PromptKit v0.24.0, ConfigKit v1.4.0, ckconfig v0.10.0, ckprompt v0.11.0,
|
||||
and CommandKit v0.7.0. Native `MessageContentHash` satisfies the cache requirement;
|
||||
typed result handoff and focused inspection enable one application-owned load.
|
||||
Adopt the native resource layout, schema selections, and `OutputContractInput`
|
||||
API in Phase 2 and complete cache use in Phase 5. Upstream combination checks
|
||||
The native resource layout, schema selections, and `OutputContractInput`
|
||||
API are part of the selected target. Upstream combination checks
|
||||
passed; assembled Notarius acceptance remains implementation work.
|
||||
|
||||
**U2 resolved:** Ordinary embedded assets, prompt-declared output contracts,
|
||||
native schema-source selection, and removal of duplicate schema/filesystem
|
||||
machinery are selected in [the schema design](#prompt-assets-and-response-schemas-u2-resolved).
|
||||
Implementation remains in Phase 2; U4 selects prompt roles and contracts, and U5
|
||||
Implementation remains pending; U4 selects prompt roles and contracts, and U5
|
||||
selects native repair delegation and the application correction allowance.
|
||||
|
||||
**U3 resolved:** Typed stage boundaries, composition-root engine construction,
|
||||
and the shared pipeline producer helper with module functions and a pipeline
|
||||
assessment callback are selected in [the execution design](#module-execution-and-assessment-u3-resolved).
|
||||
Implement in Phase 3 and refine signatures during Phase 4 without reopening
|
||||
ownership. U4/U5 select prompt/guidance and retry behavior; U6 selects terminal
|
||||
Concrete signatures remain implementation details within that ownership.
|
||||
U4/U5 select prompt/guidance and retry behavior; U6 selects terminal
|
||||
handling, configuration precedence, and module-owned fallbacks.
|
||||
|
||||
**U4 resolved:** Domain-specific initial/correction prompt pairs, shared response
|
||||
schemas, default profile behavior, validator-owned guidance, and complete
|
||||
applicable-chain assessment are selected in
|
||||
[the prompt and guidance design](#prompt-roles-and-correction-guidance-u4-resolved).
|
||||
Implement templates and operation-specific inputs in Phases 3–4; exact prose does
|
||||
not reopen ownership. U5 selects budgets; U6 selects strict/permissive handling.
|
||||
Exact template prose and operation-specific inputs remain implementation work
|
||||
within that ownership. U5 selects budgets; U6 selects strict/permissive handling.
|
||||
|
||||
**U5 resolved:** `max_additional_turns` defaults to `2`, with binding over
|
||||
pipeline precedence and explicit zero disabling correction. Native transient and
|
||||
@@ -1258,7 +1258,7 @@ structural-repair defaults remain upstream-owned; bundled prompts and applicatio
|
||||
requests do not pin structural repair. The [retry-control design](#retry-controls-and-correction-budget-u5-resolved)
|
||||
also removes generic non-LLM retries, all producer/validator `retries` fields,
|
||||
and the generic outer attempt loop. Any demonstrated transient I/O recovery stays
|
||||
with its specific operation. Implement in Phases 3–4; U6 selects terminal handling.
|
||||
with its specific operation. U6 selects terminal handling.
|
||||
|
||||
**U6 resolved:** [Strict/permissive handling and module-owned fallbacks](#terminal-outcomes-and-module-fallbacks-u6-resolved)
|
||||
replace failure-category policies. Fallbacks satisfy artifact/downstream invariants
|
||||
@@ -1269,7 +1269,7 @@ application default, including normalization; there are no module-declared
|
||||
defaults. `mode` resolves module binding, then pipeline, then that default.
|
||||
Chunking falls back to one whole-transcript chunk; spell extraction uses its empty
|
||||
artifact, and normalization uses safe final-response groups or merged input.
|
||||
Implement module-specific construction details in Phases 3–4.
|
||||
Module-specific construction details remain implementation work.
|
||||
|
||||
**U7 resolved:** [One pipeline worker pool and native admission](#admission-and-pipeline-workers-u7-resolved)
|
||||
replace the separate worker pools and application LLM scheduler.
|
||||
@@ -1277,7 +1277,7 @@ replace the separate worker pools and application LLM scheduler.
|
||||
jobs together, independently of CPU count and backend settings. PromptKit owns
|
||||
backend capacity with no injected application defaults; admission failures use
|
||||
U6's mode without extra retries or queues. Retire `total_llm` and the old extraction
|
||||
worker field with explicit migration guidance. Implement in Phase 4.
|
||||
worker field with explicit migration guidance.
|
||||
|
||||
**U8 resolved:** [The unified accepted-result cache](#accepted-artifact-cache-u8-resolved)
|
||||
includes LLM chunk plans and replaces both legacy caches. Root-level
|
||||
@@ -1286,7 +1286,7 @@ includes LLM chunk plans and replaces both legacy caches. Root-level
|
||||
explicit `--cache`, module-binding `cache`, pipeline `cache`, then `auto`.
|
||||
`auto` reads/writes, `bypass` does neither, and `refresh` writes newly accepted
|
||||
results without reading. Cache I/O is best effort; legacy records remain untouched
|
||||
and are not imported. Implement in Phases 3–5 using U1's selected native hash API.
|
||||
and are not imported. U1 supplies the selected native hash API.
|
||||
|
||||
**U9 resolved:** [Execution bookkeeping](#inspection-and-execution-bookkeeping)
|
||||
uses one native execution group per invocation, with all actual initial,
|
||||
@@ -1297,150 +1297,21 @@ fallbacks. Preserve run-specific domain artifact paths and schemas; retire index
|
||||
manifest, and reporting-only sidecars and migrate consumers to documented paths.
|
||||
Observer wiring and [debug filenames/layout](#inventory-and-depth) are explicitly
|
||||
implementation choices, not unresolved architecture or stable debug contracts.
|
||||
Implement in Phases 3–5.
|
||||
Implementation remains pending.
|
||||
|
||||
**U10 resolved:** CommandKit versions, capabilities, commands, loaders/preflight,
|
||||
output, debug policy, and coordinated cutover sequencing are selected above.
|
||||
The former later CommandKit phase is retired. CLI compatibility is a non-goal;
|
||||
CommandKit belongs to the coordinated cutover. CLI compatibility is a non-goal;
|
||||
existing artifact schemas remain domain contracts. Any actual artifact-format
|
||||
change discovered during implementation needs an explicit compatibility decision,
|
||||
not an assumption that native CLI adoption authorizes it.
|
||||
|
||||
## Implementation Sequence
|
||||
## Implementation Planning
|
||||
|
||||
All six phases are pending. Each delivers a reviewable outcome, not a separate
|
||||
release or compatibility layer. Tests and canonical documentation accompany
|
||||
behavior changes; the last phase verifies integration and completeness. Record
|
||||
major boundary changes in successor ADRs without rewriting accepted decisions.
|
||||
|
||||
### Phase 1: Establish The Migration Baseline
|
||||
|
||||
- Use U1's reviewed dependency set and release/API evidence as the implementation
|
||||
baseline; the upstream cache dependency is available and resolved.
|
||||
- Map affected ADRs, current callers, contracts, and removal inventory. Carry
|
||||
forward Audita evidence without adopting its application wrappers.
|
||||
- Carry resolved U1–U10 designs into their dependent phases without reopening
|
||||
ownership. Leave observer wiring and debug filenames to implementation.
|
||||
|
||||
**Completion evidence:** reviewed dependency baseline, affected callers/ADRs,
|
||||
scope, and removals are mapped to implementation work. The upstream review is
|
||||
complete; applying those dependencies to Notarius and verifying the migration
|
||||
remain separate work.
|
||||
|
||||
### Phase 2: Build Native Composition, Sources, And Preflight
|
||||
|
||||
- Apply U1's dependency set and implement U2: introduce configuration version 6,
|
||||
mount native embedded/companion resources beneath `promptkit`, and replace the
|
||||
old schema constructor options with native prompt selections and schema roots.
|
||||
Use `OutputContractInput` for necessary runtime replacements. Omit bundled
|
||||
structural-repair settings and remove their application override fields.
|
||||
- Build the small CommandKit application with one application-owned ConfigKit
|
||||
load, composed PromptKit rules, explicit bundled underlay, and `NewFromConfig`
|
||||
engine construction. Share that result with native inspection and standard
|
||||
suites; add pipeline declarations/native alias and `internal/preflight`.
|
||||
- Add maintained embedded-only and operator-override examples. Remove duplicated
|
||||
loading and asset assembly as callers migrate.
|
||||
|
||||
**Completion evidence:** runtime/inspection/preflight share the same loaded result
|
||||
and intended definitions without a second PromptKit load. Source contexts, shared
|
||||
message paths, prompt-declared output contracts, selected schema roots,
|
||||
source-relative schemas, strict peer conflicts, and replacement work offline. Help,
|
||||
version, and completion need no configuration. Preflight checks known dependencies
|
||||
without credentials or synthetic requests. `config diff` handles two selections.
|
||||
|
||||
### Phase 3: Prove A Representative Native Pipeline
|
||||
|
||||
- Implement the resolved U3–U5 collaboration and retry policy and U6's
|
||||
strict-by-default model, `mode` precedence, and representative module fallbacks.
|
||||
Connect module request/interpretation functions to the pipeline assessment
|
||||
callback through the shared producer helper; refine concrete signatures here.
|
||||
Implement domain-specific initial/correction prompts using the same schema and
|
||||
default profile, supplied overrides, and validator-owned correction guidance.
|
||||
Resolve `max_additional_turns` with omission/zero distinctions and pass it to
|
||||
native coordination; inherit native transport and structural-repair defaults.
|
||||
Treat a native failed-validation result as a terminal failure even when its
|
||||
Go error is nil, including in continuation and validator paths.
|
||||
- Exercise an LLM validator on the same engine and normalization with fixed
|
||||
handles and fresh plan application. Wire U9's single invocation execution group
|
||||
with application scope for initial, continuation, and validator executions.
|
||||
- Run through the new command surface and alias, using the small completion
|
||||
report, native outcomes/streams, and useful debug capture from the outset.
|
||||
Begin the resolved U8 cache integration and U9 reporting/capture implementation
|
||||
without a provisional local message hash.
|
||||
|
||||
**Completion evidence:** fake-provider workflows exercise accumulated history,
|
||||
collection of independent validator findings without rejection short-circuiting,
|
||||
full candidate revalidation through the same assessor, final-turn acceptance
|
||||
without duplicate final validation, strict failure, permissive fallback, and useful
|
||||
failures. Native failed-validation results never become accepted artifacts or
|
||||
trigger an application extension of structural repair. Unavailable assessments
|
||||
remain distinct from actionable defects. The executable publishes artifacts and
|
||||
reports them through native presentation
|
||||
without the private completion/correction API.
|
||||
|
||||
### Phase 4: Migrate All Roles And Remove Superseded Machinery
|
||||
|
||||
- Migrate every LLM-backed chunker, extractor, normalizer, and validator; preserve
|
||||
deterministic paths and ordered validation. Apply U4's prompt/guidance design
|
||||
and U5's retry policy across roles; implement U6's module fallbacks and U7's
|
||||
shared worker pool and native admission handling.
|
||||
- Remove the adapter, copied completion contracts, manual replay/capability checks,
|
||||
generic outer attempt loop, all producer/validator `retries` fields, global
|
||||
scheduler/ceiling, and coupled worker settings.
|
||||
- Replace failure-category policy fields with the selected mode configuration
|
||||
and module-owned fallback behavior.
|
||||
- Finish native command/inspection migration and remove custom dispatch, flags,
|
||||
encoders, verbosity handling, and obsolete asset registration. Retire the
|
||||
emptied `internal/framework/llm` package after moving engine assembly to the
|
||||
composition root and producer collaboration to the pipeline.
|
||||
|
||||
**Completion evidence:** every role uses the shared native engine and selected
|
||||
admission policy. One worker limit bounds extraction and lane completion together,
|
||||
including serial operation without dependency or nested-validator deadlock.
|
||||
No production path depends on the private completion API,
|
||||
latest-response-only correction, generic stage retries, global ceiling, or legacy
|
||||
CLI infrastructure. Obsolete tests retire with the machinery; surviving domain
|
||||
requirements remain.
|
||||
|
||||
### Phase 5: Complete Cache, Output, And Capture Integration
|
||||
|
||||
- Implement U8's unified lookup using the native prepared `MessageContentHash`,
|
||||
accepted-artifact/plan storage under that initial hash, directory and mode
|
||||
precedence, and best-effort cache I/O. Retire both legacy cache mechanisms.
|
||||
- Complete U9's artifact-first publication/report contract. Preserve domain paths
|
||||
and schemas, add `fallback_count`, and retire index/reporting sidecars. Remove
|
||||
checkpoint fingerprint graphs, mandatory manifests/receipts, durable execution
|
||||
ledgers, and duplicate accounting; do not retain them to populate summaries.
|
||||
- Complete capture inventory, per-operation evidence, directory/empty-value
|
||||
behavior, `latest`, and recording/finalization failure handling.
|
||||
|
||||
**Completion evidence:** downstream failure/fix/retry reuses accepted predecessors;
|
||||
message changes miss and settings-only changes can hit; corrected answers use the
|
||||
initial key; hits make no model calls or fresh validation claims. Fallbacks and
|
||||
affected downstream work skip cache lookup/storage; unaffected accepted work
|
||||
remains reusable. Chunk plans use the same cache, rematerialize against current
|
||||
source, and no longer pin one partition across changed messages. Mode precedence,
|
||||
cache I/O failures, and leaving both legacy stores untouched are verified.
|
||||
Artifacts need no metadata bundle and consumers use documented domain paths.
|
||||
Machine output and debug capture obey the selected contracts, including omitted
|
||||
zero/accurate positive fallback counts, failures after publication, and quiet-mode
|
||||
capture. Observer wiring and debug organization satisfy the selected scope and
|
||||
inventory without becoming separate public contracts.
|
||||
|
||||
### Phase 6: Verify And Document The Coordinated Cutover
|
||||
|
||||
- Complete the acceptance checks and removal inventory, including dependency
|
||||
cleanup, maintained examples, and the assembled command surface.
|
||||
- Finish release migration guidance, attribution, successor ADRs, and affected
|
||||
canonical documentation. Update this roadmap's status and reconcile the
|
||||
separate future-work backlog.
|
||||
- Confirm no compatibility scaffold, deferred CommandKit adoption, local hash
|
||||
policy, or obsolete metadata requirement remains in the finished migration.
|
||||
|
||||
**Completion evidence:** PromptKit, ConfigKit, and CommandKit adoption, accepted-
|
||||
artifact reuse, artifact delivery, diagnostics, and maintained offline workflows
|
||||
are verified together. Remaining work is explicitly separate future scope.
|
||||
Release preparation/publication is governed separately by release policy.
|
||||
The [implementation plan](implementation.md) owns execution order, repository
|
||||
touchpoints, stage acceptance criteria, and validation commands. This roadmap
|
||||
defines the complete target and acceptance criteria. Release preparation and
|
||||
publication remain separate work.
|
||||
|
||||
## Removal Inventory
|
||||
|
||||
@@ -1490,7 +1361,7 @@ retry tests or freeze complete help text and native debug snapshots.
|
||||
| Execution observations | One native invocation group enrolls actual initial/continuation/LLM validator executions once, with application scope; native totals preserve unknown-versus-zero usage without application accumulators or overlapping-group aggregation. Result-less failures remain observable; cache hits add no executions/history; native execution success remains distinct from acceptance. |
|
||||
| Debug controls/lifecycle | Default/CLI/config/explicit-empty paths and native mode ordering; quiet capture; allocation before engine/preflight; start-time relative `latest`, no completion retarget; disabled runs leave it untouched; capture failures preserve primary causes and publication facts. |
|
||||
| Debug evidence | Available input/source/native snapshots, readable text, all visible turns, assessments, normalization and deterministic outputs, cache hits, and failed-run evidence. Cache hits retain prepared details without an execution event, duplicate preparation, or model call; no invented provider transcript. ConfigKit secrets/native credentials omitted, no heuristic content scrubbing. |
|
||||
| End-to-end migration | Maintained minimal and complete D&D workflows, session routing, reference behavior, raw artifacts, reports, and migration examples work through the new executable. No postponed CommandKit phase. |
|
||||
| End-to-end migration | Maintained minimal and complete D&D workflows, session routing, reference behavior, raw artifacts, reports, and migration examples work through the new executable. No postponed CommandKit adoption. |
|
||||
|
||||
Run focused checks during implementation and the repository-wide checks for the
|
||||
completed cutover under the [testing policy](../policy/testing.md):
|
||||
@@ -1562,7 +1433,8 @@ Record significant changes in successor ADRs, including
|
||||
the [canonical chunk-plan policy](../adr/0005-cache-canonical-chunk-plans-by-source.md),
|
||||
admission, caching, and CLI/reporting ownership. Preserve accepted decision text
|
||||
and update supersession status under [documentation policy](../policy/documentation.md#architecture-decision-records).
|
||||
This roadmap owns implementation status; Git history retains earlier discussions.
|
||||
This roadmap owns the target and decision rationale; the implementation plan
|
||||
owns execution stages and their status. Git history retains earlier discussions.
|
||||
|
||||
## Evidence And Reading
|
||||
|
||||
|
||||
Reference in New Issue
Block a user