Files
notarius/docs/roadmap/commandkit-migration-discussion.md

421 lines
23 KiB
Markdown
Raw Normal View History

# CommandKit Migration Discussion
## Purpose And Status
Working notepad for the CommandKit decisions that will be incorporated into the
[master PromptKit-native migration roadmap](promptkit.md). Record the proposal,
open questions, evidence, and decisions here as we discuss them.
**Direction agreed:** Make a full cutover to a CommandKit-native application as
part of the same migration as modern PromptKit, in separate, reviewable
implementation steps. Adopt CommandKit's public-facing conventions and
recommendations to the fullest extent. Bring its design decisions forward and
complete adoption before the overall cutover.
The master roadmap currently places CommandKit in a subsequent Phase 7 and leaves
its details in U10. That sequencing has not yet been revised. Once the relevant
details below are settled, fold them into the master roadmap, including its
dependency baseline, implementation phases, removal inventory, and acceptance
criteria. No application or dependency changes have been implemented through
this discussion.
Item 1 selects the dependency versions and standard capabilities, including
PromptKit v0.23.3 in place of the master roadmap's initial v0.23.2 target.
Item 2 selects the application command structure and input conventions. Loader
wiring and detailed reporting choices remain open below.
The PromptKit architecture and checkpoint decisions remain in force. This
discussion develops the CommandKit portion of that migration and its interactions
with U1 (dependencies), U9 (records/reporting), and U10 (CommandKit adoption).
## Agreed Proposal
### Guiding Rule: Full Native Cutover
Backward compatibility with the existing CLI conventions is a non-goal. Use
CommandKit's native command, input, help/version, reporting, error, and outcome
conventions as the design baseline. This mirrors the PromptKit-native direction.
Existing command names, flags, aliases, parsing behavior, report shapes, and exit
conventions are migration evidence, not requirements to preserve. Do not add
legacy aliases, compatibility shims, parallel parsers, or reporting wrappers
solely to reproduce the old CLI. Any application-specific behavior should serve
an actual Notarius need and fit the native conventions.
Provide migration guidance in release documentation, explaining the new commands
and conventions and the actions operators or subprocess callers must take.
Documenting the transition replaces a requirement to keep the old CLI working.
Update canonical CLI and consumer documentation when the behavior lands.
Application responsibilities such as producing artifacts, reporting publication
outcomes, and managing checkpoint/debug storage remain meaningful. Their existing
CLI expression is not protected by an implicit compatibility exception. This
guiding rule does not itself choose stored artifact or checkpoint formats; those
remain separate application decisions in the master roadmap.
### Why Adopt Together
- Source assembly and preflight are already changing. CommandKit integrations
can consume the same application and native engine loaders.
- ckconfig and ckprompt can replace standard inspection commands directly,
avoiding an intermediate adaptation of commands that would soon be replaced.
- Native execution facts and application outcomes can feed the intended reporting
facilities while duplicate LLM bookkeeping is removed.
- Configuration and command migration can be documented and verified as one
coherent transition.
The added work is principally CLI composition and contract migration: dispatch,
flag parsing, configuration selection, reporting, and process-status mapping.
Choose their native forms deliberately and document the transition, rather than
spending migration effort recreating existing CLI conventions.
### Ownership And Scope
| Concern | Selected owner |
| --- | --- |
| Command hosting, standard input/help mechanics, common report formatting and visibility | CommandKit. |
| Standard configuration and PromptKit inspection commands | ckconfig and ckprompt, supplied with application loaders and request builders. |
| Configuration composition and model execution | ConfigKit and PromptKit, using the native boundaries already selected. |
| Pipeline execution and listing, pipeline-specific preflight, artifact publication, checkpoints, and domain outcomes | Notarius application handlers and framework. |
| Receipt content, debug artifact lifecycle, and application outcome meaning | Notarius, expressed through CommandKit's reporting and outcome conventions. |
Adopt the standard capability suite selected in item 1, including upstream
commands and recommendations beyond the existing Notarius surface. The remaining
discussion determines application-specific inputs, wiring, and result content.
Pipeline and domain code should remain independent of CommandKit. Native
PromptKit observations supply execution facts; Notarius attaches application
meaning and selects report content. CommandKit supplies presentation mechanics.
Two application needs already identified for deliberate treatment are:
- The current `run --json` publishes a receipt after output publication. Determine
the needed result information and express it through an appropriate native
output facility. The old flag, envelope, formatting, and stream conventions are
not compatibility requirements; reports must still describe the actual outcome.
- The current `--debug` requests durable artifacts. Retain the distinction between
artifact capture and reporting verbosity, while choosing native controls and
meanings rather than requiring that flag to retain its old behavior.
### Intended Sequencing
1. Incorporate items 1–2's selected dependency baseline, standard capabilities,
and native application command design, bringing U10 forward.
2. Introduce CommandKit composition alongside native source assembly. Connect
shared loaders and standard inspection while retaining application
orchestration responsibilities.
3. Develop native execution and accounting against the intended CLI boundary,
with pipeline and domain tests independent of CommandKit.
4. Complete command and reporting migration before final end-to-end verification
and migration documentation.
This is the agreed sequencing direction, not a replacement phase plan. Concrete
work units and their placement in the master roadmap follow the discussion below.
## 1. Adoption Scope And Dependency Baseline
**Discussion / decision:** Versions and standard capability scope agreed. Adopt
the set below, including the move to PromptKit v0.23.3. Application-specific
command design and reporting details remain with the subsequent topics. These
are selected migration targets; the application dependencies have not changed.
### Selected Versions
| Module | Version | Role |
| --- | --- | --- |
| `go.maximumdirect.net/commandkit` | `v0.7.0` | Native command declarations, inputs, help/version, reporting, and completion. |
| `go.maximumdirect.net/ckconfig` | `v0.9.0` | Standard configuration commands and ConfigKit diagnostic classification. |
| `go.maximumdirect.net/ckprompt` | `v0.10.0` | Standard PromptKit inspection, validation, preparation, and smoke diagnostics. |
| `go.maximumdirect.net/configkit` | `v1.2.1` | Shared configuration dependency for the application and integrations. |
| `go.maximumdirect.net/promptkit` | `v0.23.3` | Native model execution, including corrected timeout/cancellation reporting. |
The CommandKit integration releases form a documented compatible set using
PromptKit v0.23.2. Select v0.23.3 explicitly for its timeout attribution and
cancellation/error identity fixes. It retains the same dependencies and requires
no configuration or request API migration from v0.23.2. The Go consumer minimum
remains 1.27. Verify the assembled dependency set through the migration's normal
integration checks and update attribution when the dependency changes land.
The version review used locally available release tags and tagged dependency
manifests. Remote release lookup was unavailable, so it did not establish whether
newer tags existed remotely. These are deliberate selected versions, not a claim
that every module is the latest published release.
Evidence: [integration version guidance](../../../commandkit/docs/consumers/guide.md#module-selection)
and [PromptKit v0.23.3 release notes](../../../promptkit/docs/releases/v0.23.3.md).
### Selected Standard Capabilities
| Area | Adopt in the cutover |
| --- | --- |
| Core CLI | Native help, `version`, typed flags and operands, supplied-value semantics, standard outcomes, and command declaration/dispatch. |
| Reporting | Native `--format` with text/YAML/JSON, `--quiet`, `--verbose`, `--debug`, and `--progress`, including upstream visibility and failure behavior. |
| Configuration | `config scopes`, `validate`, `show`, `sources`, `explain`, and `diff`, for application and PromptKit configuration. |
| Resource discovery | PromptKit listings for prompts, profiles, backends, namespaces, and collections, with native filters and selectors. |
| Resource inspection | Native prompt, profile, and backend reports, plus requirements discovery. |
| Concrete request checks | Requirements validation and prepared-request preview using native inputs, variables, profiles, and execution overrides. |
| Provider diagnostics | Reachability, authentication, text-inference, structured-output, and aggregate smoke tests. |
| Convenience features | Bash, Zsh, and Fish completion, useful declaration hints, and upstream PromptKit convenience aliases. |
Use the [upstream CLI conventions](../../../commandkit/docs/cli.md) directly.
Providing the standard capability suite means supplying application loaders and
callbacks, not recreating those commands locally. Upstream convenience aliases
are selected native features; this does not reintroduce legacy Notarius aliases.
### Wiring Boundaries And Follow-Up
- Keep configuration validation, requirements discovery, and concrete request
validation distinct. Pipeline preflight remains application-owned. Concrete
preparation may read inputs and schemas and check credentials even though it
makes no model call.
- Install smoke commands for explicit invocation. Their availability does not
introduce provider calls during startup, ordinary validation, or offline tests.
- Enable application configuration overrides through ConfigKit. The selected
PromptKit configuration scope does not support arbitrary `--set` overrides;
use supported request-level controls rather than implementing a local equivalent.
- Retain application handlers for pipeline execution and discovery, with native
commands, inputs, and result presentation to be designed in items 2–5.
No upstream feature request has been identified as a blocker for this scope.
Evaluate any later demonstrated gap on its actual application need. On roadmap
reconciliation, incorporate these versions into U1, replace U10's open standard
capability selection with this decision, and retain the unresolved application
wiring and presentation details explicitly.
## 2. Native Command Surface And Inputs
**Discussion / decision:** Command structure and input conventions agreed. Use a
small application-owned `pipeline` family alongside the standard `config` and
`promptkit` families. These are migration targets, not implemented commands.
### Command Structure
```text
notarius pipeline list
notarius pipeline show <pipeline-id>
notarius pipeline validate <pipeline-id>
notarius pipeline run <pipeline-id> --input-file <path>
notarius config scopes
notarius config validate|show|sources|explain|diff ...
notarius promptkit list ...
notarius promptkit show ...
notarius promptkit requirements show|validate ...
notarius promptkit test <kind>
notarius version
notarius completion bash|zsh|fish
notarius help [command...]
```
The alternatives separated by `|` above summarize the installed operations;
they are not literal shell commands.
| Application command | Responsibility |
| --- | --- |
| `pipeline list` | Discover configured pipelines. |
| `pipeline show` | Describe the resolved ordered steps, lanes, modules, reference slots, and configured selections. |
| `pipeline validate` | Perform offline application preflight using the intended selections and overrides. |
| `pipeline run` | Execute the selected pipeline and publish artifacts. |
Move pipeline-specific validation out of `config validate --pipeline ...`.
Configuration validation, pipeline composition checks, and concrete PromptKit
request validation remain distinct. Pipeline preflight does not synthesize stage
requests or require provider credentials. PromptKit's concrete validation may
read inputs and schemas and check credentials without making a model call.
**Naming clarification:** CommandKit supports both a root `run` command and a
nested `pipeline run` command. Neither is more native. The selected grouping
keeps pipeline discovery, inspection, validation, and execution together; it is
an application usability choice, not an upstream naming requirement. A shorter
root command could be selected on usability grounds without changing the native
architecture. Do not introduce a second route solely for legacy compatibility.
### Input Conventions
Use a required pipeline-ID operand and a required `--input-file PATH` for runs.
Use native typed declarations and supplied-value semantics, including explicit
empty, false, and zero values. Apply defaults only when the relevant value is
omitted; application validation still decides whether a supplied value is valid.
| Concern | Selected convention |
| --- | --- |
| Configuration | Native `--config` and `--overlay`; name the application scope `app`. |
| Lane selection | Repeatable `--lane ID`; omission selects the normal pipeline lane set. Replace comma-separated `--only`. |
| Output placement | Application-owned `--output-dir PATH`. |
| Chunk-plan caching | `--chunk-cache auto\|bypass\|refresh`. |
| Checkpoint reuse | Keep `--resume` and `--recompute-step ID` with the agreed recovery policy. |
| Presentation | Native `--format`, `--quiet`, `--verbose`, `--debug`, and `--progress`. |
| Configuration-valued tuning | Supported application `--set POINTER=JSON` overrides, including worker and semantic-budget settings once their configuration design is settled. |
Adopt native parsing rules: flags follow the complete command path and may be
interspersed with operands; `--` ends option parsing. Singleton flags use the
last successfully parsed occurrence, with every occurrence checked. Repeatable
flags preserve occurrence order and treat commas as ordinary value content.
Use filesystem and choice completion hints where applicable.
Keep worker counts and semantic continuation budgets in configuration rather
than introducing additional dedicated CLI controls now. Their field names,
defaults, and domain meaning remain master-roadmap decisions. Item 3 will settle
shared loading and override wiring across application and standard commands.
### Native LLM Controls
Use ckprompt's reusable flag declarations for applicable profile, session, and
execution controls on `pipeline run`, including model, temperature, timeout,
reasoning effort, and transient retries. Run-wide overrides reach all LLM-backed
producers and validators, including continuation calls. Without an override,
normal binding/pipeline/prompt selection applies.
| Reasoning input | Meaning |
| --- | --- |
| Omitted | Inherit the selected settings. |
| `--llm-reasoning-effort high` | Replace the inherited setting. |
| `--llm-reasoning-effort ""` | Clear the inherited setting. |
Remove the separate `--clear-reasoning-effort` control. Native transient retries
remain distinct from application semantic continuation budgets. Individual
prompt selection and named prompt input injection belong on standard PromptKit
commands; pipeline modules construct their own requests and response contracts.
Use native `--llm-profile` and `--llm-session-id` spellings, without inventing
another LLM input parser.
### Application Reference Inputs
Retain repeatable `--reference SELECTOR=PATH` and `--without-reference SELECTOR`.
Keep useful pipeline-wide, lane, chunk, and stage-specific selectors, such as
`party=party.txt` and `npc-registry.extract.party=npc-party.txt`. CommandKit owns
repeatable input parsing and typed conversion; Notarius owns selector expansion,
specificity, required slots, and external/generated-reference conflict checks.
These selectors describe pipeline relationships and are not interchangeable
with PromptKit's individual-request `--llm-input-file` bindings.
### Preparing Pipeline Inspection Content
**Follow-up clarification:** Leverage ConfigKit for the composed, decoded
configuration, defaults, imports, overlay, inheritance, overrides, and native
provenance/redacted inspection. Feed that loaded configuration through the same
application resolution used by validation and execution to obtain selected
lanes, registered modules, validator chains, and reference relationships.
CommandKit renders a small application report of that resolved composition.
ConfigKit does not know Notarius's module catalog or pipeline semantics. Reuse
the existing resolver rather than writing a second resolution path for `show`.
Generic effective configuration and source explanations remain available through
`config show` and `config explain`. ConfigKit's safe inspection facilities do not
automatically redact an arbitrary application report; select appropriate report
content rather than dumping the raw typed configuration. Detailed report content
and loader wiring remain with items 3–4.
### Removals And Remaining Work
Retire custom dispatch, argument-count handling, assembled help, version-command
parsing, standard configuration/LLM flag parsing, comma-separated lane parsing,
and the separate reasoning-clearing machinery. Replace `config effective` with
native `config show`, `config profile` with `promptkit show profile`, and bespoke
`--json` switches with `--format json`.
Application handlers still validate meaningful input combinations and translate
typed inputs into pipeline selections. Native `--debug` supplies reporting
behavior; a distinct artifact-capture control remains to be selected in item 5.
Items 3–4 own loader behavior, exact report content, and outcome presentation.
Evidence: [CommandKit command/input conventions](../../../commandkit/docs/cli.md#registered-command-syntax),
[reusable PromptKit flags](../../../commandkit/docs/consumers/guide.md#request-builders-and-loading),
[ConfigKit loading and inspection](../../../ConfigKit/README.md), and the current
[Notarius configuration boundaries](../internal/configuration.md).
Use the current [CLI reference](../cli.md) for migration guidance, not as a
compatibility constraint.
## 3. Application Composition, Loaders, And Preflight
**Discussion / decision:** Pending.
- How should command declarations, application handlers, and invocation state be
composed without introducing another general CLI or LLM wrapper?
- How should native command selections and overrides reach shared loaders while
retaining the selected ConfigKit source composition and main-source-relative
companion behavior across execution and inspection?
- How should `pipeline show`, `pipeline validate`, and `pipeline run` share
application resolution alongside standard configuration validation, and what
state should be acquired only when a command needs it?
Retain the selected offline preflight boundary: it checks application composition
and known native resources without synthesizing a stage request or requiring
provider credentials. Help/version should not require runtime configuration or
provider work. Model-executing roles share one native engine within an execution;
the CLI wiring must preserve that ownership.
## 4. Reports, Machine Output, And Outcomes
**Discussion / decision:** Pending.
- How should native reports and output modes express application results? Where
does an application data requirement justify raw output rather than a standard
report, independently of the old CLI's output conventions?
- How should syntax, configuration, pipeline, and publication outcomes map to
native outcome/status conventions and user-facing explanations? How should
presentation failures remain distinguishable from the operation outcome?
- Which application scope and minimal projections connect native observations
to progress, terminal summaries, manifests, and receipts without duplicate
execution accounting?
Review the current [run-result contract](../integrations/run-result.md) and
[CLI failure mapping](../internal/cli.md#failure-mapping-and-terminal-reporting)
to understand application needs and caller migration impact. Their existing CLI
shapes and status mappings do not override the native-cutover direction.
Item 6 of the [PromptKit discussion record](promptkit-architecture-discussion.md#6-inspection-and-execution-bookkeeping)
provides the bookkeeping boundary; U9 still owns concrete durable record choices.
## 5. Debug Capture And Optional Diagnostics
**Discussion / decision:** Pending.
- How should artifact-capture controls fit alongside native quiet/verbose/debug
reporting selections without requiring the existing `--debug` behavior?
- How should requested debug artifact writes, failures, disclosure, and retention
connect to command completion and native callbacks?
- For selected preview, request-validation, or smoke capabilities, what native
request builders and explicit execution choices should the application supply?
Keep debug creation and physical storage in Notarius. Ordinary records exclude
sensitive response/history content, and provider smoke diagnostics remain
explicitly opt-in. Native inspection alone must not become a model call.
## 6. Verification, Migration, And Roadmap Incorporation
**Discussion / decision:** Pending.
- Which focused command and subprocess tests protect the new native command,
output, outcome, receipt, and debug behavior? Which tests of superseded CLI
contracts and parser/formatter implementation can retire?
- Which representative assembled workflows prove that execution, inspection,
and preflight use the same intended configuration without duplicating lower-level
coverage? Keep the default suite offline and independent of real credentials.
- How should CommandKit steps replace the master roadmap's deferred Phase 7,
and which migration examples, documentation, and ADRs need coordinated updates?
- What release-documentation guidance will show operators and subprocess callers
how to migrate their workflows and integrations to the new conventions?
After discussion, reconcile the master roadmap's U1/U9/U10 entries, sequencing,
removals, and acceptance criteria, including its treatment of existing inspection
commands and machine-output compatibility. Carry forward the explicit non-goal
of CLI backward compatibility and release migration guidance. Preserve other
unresolved choices explicitly. Then mark this document as the discussion record
and keep implementation status in the master roadmap.
## Evidence And Reading
These are starting points for discussion, not a completed CommandKit migration
audit. Sibling links assume the local repositories remain available.
- [Notarius CLI implementation overview](../internal/cli.md) and
[configuration internals](../internal/configuration.md).
- [CommandKit consumer guide](../../../commandkit/docs/consumers/guide.md), including
module selection, typed inputs, raw/formatted output, reporting, and loaders.
- [ckconfig integration](../../../commandkit/integrations/ckconfig/README.md) and
[ckprompt integration](../../../commandkit/integrations/ckprompt/README.md).
- [Audita command composition](../../../audita/internal/cli/app_wiring.go), showing
application handlers and upstream integrations in one CommandKit application.