Record CommandKit verification and migration plan
This commit is contained in:
@@ -26,7 +26,9 @@ selects native composition, shared loaders, and application preflight. Item 4
|
|||||||
selects native reporting, machine output, and outcomes. Item 5 selects conditional
|
selects native reporting, machine output, and outcomes. Item 5 selects conditional
|
||||||
debug capture through the shared `--debug` flag, native diagnostics, and a run/
|
debug capture through the shared `--debug` flag, native diagnostics, and a run/
|
||||||
operation bundle with a `latest` symlink. Capture depth, exact run-report fields,
|
operation bundle with a `latest` symlink. Capture depth, exact run-report fields,
|
||||||
and verification remain open below.
|
and other concrete record details remain open below. Item 6 selects verification,
|
||||||
|
migration guidance, and incorporation into the master roadmap; that incorporation
|
||||||
|
is still pending.
|
||||||
|
|
||||||
The PromptKit architecture and checkpoint decisions remain in force. This
|
The PromptKit architecture and checkpoint decisions remain in force. This
|
||||||
discussion develops the CommandKit portion of that migration and its interactions
|
discussion develops the CommandKit portion of that migration and its interactions
|
||||||
@@ -85,8 +87,9 @@ spending migration effort recreating existing CLI conventions.
|
|||||||
|
|
||||||
Adopt the standard capability suite selected in item 1, including upstream
|
Adopt the standard capability suite selected in item 1, including upstream
|
||||||
commands and recommendations beyond the existing Notarius surface. Items 2–4
|
commands and recommendations beyond the existing Notarius surface. Items 2–4
|
||||||
select application inputs, composition, and reporting; the remaining discussion
|
select application inputs, composition, and reporting; items 5–6 select debug
|
||||||
determines debug capture, verification, and concrete record details.
|
capture principles, verification, and migration. Concrete record details remain
|
||||||
|
open as identified below.
|
||||||
|
|
||||||
Pipeline and domain code should remain independent of CommandKit. Native
|
Pipeline and domain code should remain independent of CommandKit. Native
|
||||||
PromptKit observations supply execution facts; Notarius attaches application
|
PromptKit observations supply execution facts; Notarius attaches application
|
||||||
@@ -114,8 +117,8 @@ Two application needs already identified for deliberate treatment are:
|
|||||||
4. Complete command and reporting migration before final end-to-end verification
|
4. Complete command and reporting migration before final end-to-end verification
|
||||||
and migration documentation.
|
and migration documentation.
|
||||||
|
|
||||||
This is the agreed sequencing direction, not a replacement phase plan. Concrete
|
Item 6 selects the replacement phase sequence for incorporation into the master
|
||||||
work units and their placement in the master roadmap follow the discussion below.
|
roadmap. The master roadmap has not yet been updated.
|
||||||
|
|
||||||
## 1. Adoption Scope And Dependency Baseline
|
## 1. Adoption Scope And Dependency Baseline
|
||||||
|
|
||||||
@@ -886,28 +889,136 @@ The current [debug lifecycle](../internal/state.md#debug-bundles) and
|
|||||||
|
|
||||||
## 6. Verification, Migration, And Roadmap Incorporation
|
## 6. Verification, Migration, And Roadmap Incorporation
|
||||||
|
|
||||||
**Discussion / decision:** Pending.
|
**Discussion / decision:** Agreed. Complete one coordinated native cutover, with
|
||||||
|
CommandKit composition introduced early, verification focused on Notarius's
|
||||||
|
integration and domain responsibilities, and practical release migration guidance.
|
||||||
|
Incorporate these decisions into the existing master roadmap; another replacement
|
||||||
|
document is unnecessary. Implementation and master-roadmap incorporation are pending.
|
||||||
|
|
||||||
- Which focused command and subprocess tests protect the new native command,
|
### Focused Verification Of Application Behavior
|
||||||
output, outcome, receipt, and debug behavior? Which tests of superseded CLI
|
|
||||||
contracts and parser/formatter implementation can retire?
|
|
||||||
- Which representative assembled workflows prove that execution, inspection,
|
|
||||||
and preflight use the same intended configuration without duplicating lower-level
|
|
||||||
coverage? Keep the default suite offline and independent of real credentials.
|
|
||||||
- How should CommandKit steps replace the master roadmap's deferred Phase 7,
|
|
||||||
and which migration examples, documentation, and ADRs need coordinated updates?
|
|
||||||
- What release-documentation guidance will show operators and subprocess callers
|
|
||||||
how to migrate their workflows and integrations to the new conventions?
|
|
||||||
- Reconcile older debug-sensitivity language, special-handling requirements, and
|
|
||||||
heuristic redaction behavior/tests with item 5's clarified boundary. Do not
|
|
||||||
reintroduce them as implicit requirements while migrating documentation or code.
|
|
||||||
|
|
||||||
After discussion, reconcile the master roadmap's U1/U9/U10 entries, sequencing,
|
Use real CommandKit, ConfigKit, and PromptKit components in representative
|
||||||
removals, and acceptance criteria, including its treatment of existing inspection
|
assembled workflows, with a fake at the provider boundary. Keep the default
|
||||||
commands and machine-output compatibility. Carry forward the explicit non-goal
|
suite deterministic, offline, and independent of real credentials. Following the
|
||||||
of CLI backward compatibility and release migration guidance. Preserve other
|
[testing policy](../policy/testing.md), assign each behavior an appropriate test
|
||||||
unresolved choices explicitly. Then mark this document as the discussion record
|
owner and avoid duplicating upstream parser, renderer, or retry coverage.
|
||||||
and keep implementation status in the master roadmap.
|
|
||||||
|
Select a small set of workflows that collectively establish:
|
||||||
|
|
||||||
|
- Execution, inspection, and preflight resolve the same intended configuration
|
||||||
|
and prompt definitions, including application overrides and companion sources.
|
||||||
|
- Help, version, and completion work without application configuration; offline
|
||||||
|
preflight makes no provider calls.
|
||||||
|
- `run` and `pipeline run` reach the same application behavior, with the selected
|
||||||
|
flags, supplied-value distinctions, and overrides mapped correctly.
|
||||||
|
- Reports, diagnostics, and exit outcomes follow item 4, distinguishing domain
|
||||||
|
rejection, command failure, and presentation failure after publication.
|
||||||
|
- A downstream failure followed by resume reuses successful predecessors under
|
||||||
|
the simpler checkpoint policy, with explicit recomputation and consumed-input
|
||||||
|
changes handled as selected in the master roadmap.
|
||||||
|
- Debug capture follows item 5's directory selection, reporting-mode independence,
|
||||||
|
`latest` update timing, useful evidence, and allocation/write/finalization
|
||||||
|
failure behavior.
|
||||||
|
|
||||||
|
Use subprocess tests where the actual executable boundary adds protection:
|
||||||
|
stdout/stderr separation, process exit status, and representative machine output.
|
||||||
|
Test domain interpretation, validation, continuation, and recovery at their
|
||||||
|
appropriate package boundaries. Preserve meaningful native-execution acceptance
|
||||||
|
checks from the master roadmap without repeating them throughout CLI tests.
|
||||||
|
|
||||||
|
### Retire Tests With Superseded Machinery
|
||||||
|
|
||||||
|
Review tests of the private adapter, global scheduler, manual continuation
|
||||||
|
protocol, broad checkpoint fingerprints, and superseded CLI contracts for removal
|
||||||
|
as those implementations retire. Where an application requirement survives, test
|
||||||
|
that requirement through its new boundary. Do not mechanically reproduce old
|
||||||
|
parser, formatter, or wrapper tests against the new implementation.
|
||||||
|
|
||||||
|
Avoid freezing complete upstream help output, incidental diagnostic wording, or
|
||||||
|
every field of native debug snapshots. Debug verification should establish useful
|
||||||
|
evidence and correct failure handling while allowing representations to evolve
|
||||||
|
for convenience. Do not require exhaustive provider-attempt reconstruction or
|
||||||
|
strict serialization deduplication.
|
||||||
|
|
||||||
|
Reconcile older debug-sensitivity language, special-handling requirements, and
|
||||||
|
heuristic redaction behavior/tests with item 5. Avoid directly disclosing values
|
||||||
|
marked as secrets by ConfigKit and preserve native omissions. Do not carry forward
|
||||||
|
heuristic scrubbing of prompts, source content, or model output, or special
|
||||||
|
debug-data handling requirements. Operators secure debug artifacts as they do
|
||||||
|
ordinary application inputs and outputs.
|
||||||
|
|
||||||
|
### Migration Guidance And Documentation
|
||||||
|
|
||||||
|
Provide release guidance for operators and subprocess callers, with a practical
|
||||||
|
old-to-new migration map covering:
|
||||||
|
|
||||||
|
- Commands and flags, including the native `run` alias, configuration inspection,
|
||||||
|
and `--format`.
|
||||||
|
- Application and PromptKit configuration changes, source discovery, profiles,
|
||||||
|
and companion files.
|
||||||
|
- Native backend admission, independent worker settings, and revised retry controls.
|
||||||
|
- Checkpoint reuse, explicit recomputation, and the selected treatment of existing
|
||||||
|
checkpoint files.
|
||||||
|
- Machine reports, exit outcomes, and the distinction between successful command
|
||||||
|
execution and accepted domain results.
|
||||||
|
- Debug capture under `/tmp/notarius-debug`, `latest`, and the interaction between
|
||||||
|
capture and reporting modes.
|
||||||
|
|
||||||
|
CLI backward compatibility remains a non-goal; migration guidance does not imply
|
||||||
|
compatibility shims or preservation of old command behavior. Existing checkpoint
|
||||||
|
and published-artifact compatibility remain explicit application decisions about
|
||||||
|
stored work and consumer data, separate from CLI syntax.
|
||||||
|
|
||||||
|
Maintain runnable examples for embedded defaults and operator overrides. Update
|
||||||
|
canonical CLI, configuration, operations, integration/consumer, and internal
|
||||||
|
documentation as the corresponding behavior lands. Record major architectural
|
||||||
|
changes in successor ADRs, preserving accepted decision text under the
|
||||||
|
[documentation policy](../policy/documentation.md#architecture-decision-records).
|
||||||
|
Keep future behavior in roadmap documents until implemented.
|
||||||
|
|
||||||
|
### Incorporation Into The Master Roadmap
|
||||||
|
|
||||||
|
Replace the deferred CommandKit Phase 7 with adoption throughout the existing
|
||||||
|
master roadmap. Use this phase sequence:
|
||||||
|
|
||||||
|
1. Establish the agreed dependency baseline and remaining contract decisions.
|
||||||
|
2. Build native application composition, shared loaders, standard commands, and
|
||||||
|
the dedicated preflight package alongside native source assembly.
|
||||||
|
3. Prove a representative native pipeline through the actual command surface,
|
||||||
|
including reporting and useful debug capture.
|
||||||
|
4. Migrate remaining roles and remove the adapter, scheduler, retry machinery,
|
||||||
|
and custom CLI infrastructure.
|
||||||
|
5. Complete checkpoint migration, durable records, and capture behavior.
|
||||||
|
6. Verify the assembled application and finish migration documentation.
|
||||||
|
|
||||||
|
Each phase delivers a reviewable result with appropriate verification. Tests and
|
||||||
|
documentation proceed throughout; the final phase checks integration and
|
||||||
|
completeness. The phases do not require separate releases or an intermediate
|
||||||
|
compatibility framework.
|
||||||
|
|
||||||
|
Resolve U1's dependency baseline and U10's CommandKit scope from items 1–4. Narrow
|
||||||
|
U9 to remaining durable-field, accounting, and capture details within the selected
|
||||||
|
boundaries. Reconcile removal inventory and acceptance criteria, including old
|
||||||
|
inspection-command and machine-output compatibility assumptions. Preserve
|
||||||
|
unresolved schema handling, role-specific retry/fallback policy, worker defaults,
|
||||||
|
checkpoint transition choices, and other concrete choices until actually settled.
|
||||||
|
|
||||||
|
Once incorporated, the master roadmap owns implementation sequencing and status;
|
||||||
|
both migration notepads remain discussion records. Do not maintain competing
|
||||||
|
implementation checklists in the notepads.
|
||||||
|
|
||||||
|
### Completion Checks
|
||||||
|
|
||||||
|
Use focused checks while implementing and the existing repository-wide
|
||||||
|
`go test ./...`, `go vet ./...`, and `go build ./cmd/notarius` checks for the
|
||||||
|
completed cutover. Check the affected maintained examples, documentation links,
|
||||||
|
dependency attribution, and removal inventory as part of this migration.
|
||||||
|
|
||||||
|
Live provider smoke tests remain explicitly optional and outside the deterministic
|
||||||
|
default suite. These change-specific migration checks do not create additional
|
||||||
|
permanent release gates; publication continues to follow the
|
||||||
|
[release policy](../policy/release.md). Recording these decisions does not claim
|
||||||
|
that implementation acceptance checks have already passed.
|
||||||
|
|
||||||
## Evidence And Reading
|
## Evidence And Reading
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user