Record CommandKit verification and migration plan

This commit is contained in:
2026-11-20 00:48:29 +00:00
parent d9e1d1e4a7
commit 3c73ccfad0

View File

@@ -26,7 +26,9 @@ selects native composition, shared loaders, and application preflight. Item 4
selects native reporting, machine output, and outcomes. Item 5 selects conditional selects native reporting, machine output, and outcomes. Item 5 selects conditional
debug capture through the shared `--debug` flag, native diagnostics, and a run/ debug capture through the shared `--debug` flag, native diagnostics, and a run/
operation bundle with a `latest` symlink. Capture depth, exact run-report fields, operation bundle with a `latest` symlink. Capture depth, exact run-report fields,
and verification remain open below. and other concrete record details remain open below. Item 6 selects verification,
migration guidance, and incorporation into the master roadmap; that incorporation
is still pending.
The PromptKit architecture and checkpoint decisions remain in force. This The PromptKit architecture and checkpoint decisions remain in force. This
discussion develops the CommandKit portion of that migration and its interactions discussion develops the CommandKit portion of that migration and its interactions
@@ -85,8 +87,9 @@ spending migration effort recreating existing CLI conventions.
Adopt the standard capability suite selected in item 1, including upstream Adopt the standard capability suite selected in item 1, including upstream
commands and recommendations beyond the existing Notarius surface. Items 2–4 commands and recommendations beyond the existing Notarius surface. Items 2–4
select application inputs, composition, and reporting; the remaining discussion select application inputs, composition, and reporting; items 5–6 select debug
determines debug capture, verification, and concrete record details. capture principles, verification, and migration. Concrete record details remain
open as identified below.
Pipeline and domain code should remain independent of CommandKit. Native Pipeline and domain code should remain independent of CommandKit. Native
PromptKit observations supply execution facts; Notarius attaches application PromptKit observations supply execution facts; Notarius attaches application
@@ -114,8 +117,8 @@ Two application needs already identified for deliberate treatment are:
4. Complete command and reporting migration before final end-to-end verification 4. Complete command and reporting migration before final end-to-end verification
and migration documentation. and migration documentation.
This is the agreed sequencing direction, not a replacement phase plan. Concrete Item 6 selects the replacement phase sequence for incorporation into the master
work units and their placement in the master roadmap follow the discussion below. roadmap. The master roadmap has not yet been updated.
## 1. Adoption Scope And Dependency Baseline ## 1. Adoption Scope And Dependency Baseline
@@ -886,28 +889,136 @@ The current [debug lifecycle](../internal/state.md#debug-bundles) and
## 6. Verification, Migration, And Roadmap Incorporation ## 6. Verification, Migration, And Roadmap Incorporation
**Discussion / decision:** Pending. **Discussion / decision:** Agreed. Complete one coordinated native cutover, with
CommandKit composition introduced early, verification focused on Notarius's
integration and domain responsibilities, and practical release migration guidance.
Incorporate these decisions into the existing master roadmap; another replacement
document is unnecessary. Implementation and master-roadmap incorporation are pending.
- Which focused command and subprocess tests protect the new native command, ### Focused Verification Of Application Behavior
output, outcome, receipt, and debug behavior? Which tests of superseded CLI
contracts and parser/formatter implementation can retire?
- Which representative assembled workflows prove that execution, inspection,
and preflight use the same intended configuration without duplicating lower-level
coverage? Keep the default suite offline and independent of real credentials.
- How should CommandKit steps replace the master roadmap's deferred Phase 7,
and which migration examples, documentation, and ADRs need coordinated updates?
- What release-documentation guidance will show operators and subprocess callers
how to migrate their workflows and integrations to the new conventions?
- Reconcile older debug-sensitivity language, special-handling requirements, and
heuristic redaction behavior/tests with item 5's clarified boundary. Do not
reintroduce them as implicit requirements while migrating documentation or code.
After discussion, reconcile the master roadmap's U1/U9/U10 entries, sequencing, Use real CommandKit, ConfigKit, and PromptKit components in representative
removals, and acceptance criteria, including its treatment of existing inspection assembled workflows, with a fake at the provider boundary. Keep the default
commands and machine-output compatibility. Carry forward the explicit non-goal suite deterministic, offline, and independent of real credentials. Following the
of CLI backward compatibility and release migration guidance. Preserve other [testing policy](../policy/testing.md), assign each behavior an appropriate test
unresolved choices explicitly. Then mark this document as the discussion record owner and avoid duplicating upstream parser, renderer, or retry coverage.
and keep implementation status in the master roadmap.
Select a small set of workflows that collectively establish:
- Execution, inspection, and preflight resolve the same intended configuration
and prompt definitions, including application overrides and companion sources.
- Help, version, and completion work without application configuration; offline
preflight makes no provider calls.
- `run` and `pipeline run` reach the same application behavior, with the selected
flags, supplied-value distinctions, and overrides mapped correctly.
- Reports, diagnostics, and exit outcomes follow item 4, distinguishing domain
rejection, command failure, and presentation failure after publication.
- A downstream failure followed by resume reuses successful predecessors under
the simpler checkpoint policy, with explicit recomputation and consumed-input
changes handled as selected in the master roadmap.
- Debug capture follows item 5's directory selection, reporting-mode independence,
`latest` update timing, useful evidence, and allocation/write/finalization
failure behavior.
Use subprocess tests where the actual executable boundary adds protection:
stdout/stderr separation, process exit status, and representative machine output.
Test domain interpretation, validation, continuation, and recovery at their
appropriate package boundaries. Preserve meaningful native-execution acceptance
checks from the master roadmap without repeating them throughout CLI tests.
### Retire Tests With Superseded Machinery
Review tests of the private adapter, global scheduler, manual continuation
protocol, broad checkpoint fingerprints, and superseded CLI contracts for removal
as those implementations retire. Where an application requirement survives, test
that requirement through its new boundary. Do not mechanically reproduce old
parser, formatter, or wrapper tests against the new implementation.
Avoid freezing complete upstream help output, incidental diagnostic wording, or
every field of native debug snapshots. Debug verification should establish useful
evidence and correct failure handling while allowing representations to evolve
for convenience. Do not require exhaustive provider-attempt reconstruction or
strict serialization deduplication.
Reconcile older debug-sensitivity language, special-handling requirements, and
heuristic redaction behavior/tests with item 5. Avoid directly disclosing values
marked as secrets by ConfigKit and preserve native omissions. Do not carry forward
heuristic scrubbing of prompts, source content, or model output, or special
debug-data handling requirements. Operators secure debug artifacts as they do
ordinary application inputs and outputs.
### Migration Guidance And Documentation
Provide release guidance for operators and subprocess callers, with a practical
old-to-new migration map covering:
- Commands and flags, including the native `run` alias, configuration inspection,
and `--format`.
- Application and PromptKit configuration changes, source discovery, profiles,
and companion files.
- Native backend admission, independent worker settings, and revised retry controls.
- Checkpoint reuse, explicit recomputation, and the selected treatment of existing
checkpoint files.
- Machine reports, exit outcomes, and the distinction between successful command
execution and accepted domain results.
- Debug capture under `/tmp/notarius-debug`, `latest`, and the interaction between
capture and reporting modes.
CLI backward compatibility remains a non-goal; migration guidance does not imply
compatibility shims or preservation of old command behavior. Existing checkpoint
and published-artifact compatibility remain explicit application decisions about
stored work and consumer data, separate from CLI syntax.
Maintain runnable examples for embedded defaults and operator overrides. Update
canonical CLI, configuration, operations, integration/consumer, and internal
documentation as the corresponding behavior lands. Record major architectural
changes in successor ADRs, preserving accepted decision text under the
[documentation policy](../policy/documentation.md#architecture-decision-records).
Keep future behavior in roadmap documents until implemented.
### Incorporation Into The Master Roadmap
Replace the deferred CommandKit Phase 7 with adoption throughout the existing
master roadmap. Use this phase sequence:
1. Establish the agreed dependency baseline and remaining contract decisions.
2. Build native application composition, shared loaders, standard commands, and
the dedicated preflight package alongside native source assembly.
3. Prove a representative native pipeline through the actual command surface,
including reporting and useful debug capture.
4. Migrate remaining roles and remove the adapter, scheduler, retry machinery,
and custom CLI infrastructure.
5. Complete checkpoint migration, durable records, and capture behavior.
6. Verify the assembled application and finish migration documentation.
Each phase delivers a reviewable result with appropriate verification. Tests and
documentation proceed throughout; the final phase checks integration and
completeness. The phases do not require separate releases or an intermediate
compatibility framework.
Resolve U1's dependency baseline and U10's CommandKit scope from items 1–4. Narrow
U9 to remaining durable-field, accounting, and capture details within the selected
boundaries. Reconcile removal inventory and acceptance criteria, including old
inspection-command and machine-output compatibility assumptions. Preserve
unresolved schema handling, role-specific retry/fallback policy, worker defaults,
checkpoint transition choices, and other concrete choices until actually settled.
Once incorporated, the master roadmap owns implementation sequencing and status;
both migration notepads remain discussion records. Do not maintain competing
implementation checklists in the notepads.
### Completion Checks
Use focused checks while implementing and the existing repository-wide
`go test ./...`, `go vet ./...`, and `go build ./cmd/notarius` checks for the
completed cutover. Check the affected maintained examples, documentation links,
dependency attribution, and removal inventory as part of this migration.
Live provider smoke tests remain explicitly optional and outside the deterministic
default suite. These change-specific migration checks do not create additional
permanent release gates; publication continues to follow the
[release policy](../policy/release.md). Recording these decisions does not claim
that implementation acceptance checks have already passed.
## Evidence And Reading ## Evidence And Reading