commandkit/docs/roadmap/registration-backend.md

Registration and execution internals

Status: finalized internal design; no implementation has begun. Runnable-command children, scope-error attachment, and the unused-binding validation resolution are incorporated into the registration API. Both composition questions below are resolved; the staged implementation plan owns the ordered work and completion evidence.

Purpose and authority

Provide the registration API through one coherent internal system. Application commands, optional integration families, and built-in core commands should use the same declaration, installation, routing, and execution machinery. Command-specific handlers remain different where their responsibilities are different.

ADR 0011 owns the accepted capture, installation, namespace, and input/state decisions. ADR 0010 owns module boundaries, and the architecture policy governs implementation. The module roadmap owns the broader extraction and release work. This document defines selected internal mechanics and records the resolved composition decisions. Public declarations and consumer examples belong in the registration API document; these changes remain unimplemented.

The prototype has no compatibility priority. Replace its parallel paths and bookkeeping where they obstruct the selected design. Preserve deliberately selected grammar, presence, outcomes, reports, and callback lifetimes. Private names below describe responsibilities rather than required exported symbols or a mandated file layout. This is design work, not a staged implementation plan.

Baseline findings

The following findings come from the current implementation. They identify reuse opportunities and structural risks, not measured performance regressions.

Evidence	Pain point	Proposed solution
New in app.go inserts help/version placeholders; installBuiltins in builtins.go replaces them directly	Built-ins bypass ordinary registry installation and have separate reserved-node bookkeeping	Construct real built-in declarations and install them with caller declarations in one batch
Register in registry.go already stages roots before committing	Useful atomic installation exists, but privileged writers can bypass it	Make this the sole registry mutation boundary for every command source
RegisterConfig in config.go checks the root namespace and App.configRegistered before calling Register	The configuration installer duplicates collision policy and carries integration state in the core	Integration factories return root-only declarations; core installation decides collisions
RegisterPromptKit in promptkit.go returns early when both inspection callbacks are nil	An absence shortcut can hide construction or metadata errors	Retain errors before representing absence; installation checks errors before skipping a no-op
Group, validateCommandDeclaration, and cloneCommands in command.go recursively validate/copy child trees; registration repeats this	Nested assembly can revisit a subtree at each ancestor and again at installation	Capture immutable child handles and local metadata once; perform contextual installation checks in one traversal
normalizeConfigSuite and normalizePromptKitSuite repeat flag-name, short-name, duplicate, and wiring rules	Moving integrations into modules could create several competing implementations of core binding policy	Validate generated command bindings in the core; integrations validate only their own suite semantics
adaptContextSelector and contextsForScopes in selectors.go apply selectors to mixed context storage, then copy/project it again	Selector storage and shared state have tangled lifetimes; pointer contexts can alias	Capture converted values once; initialize each scope input independently; create shared state after all input checks
Configuration CheckInput calls preflight, while runConfigShow and runConfigValidation select scopes again	Metadata selection and scope setup can be repeated between validation and execution	Carry an invocation-local prepared selection from CheckInput into Run
Execute in execute.go identifies help by its path and asserts its private input type	A built-in name controls a separate execution path	Let the help declaration's typed runner receive a private execution context through the common compiler
bridgeTree in parser.go caches neutral metadata; internal/cobra/parse.go creates a session per invocation	The useful metadata/session split should survive the refactor	Share captured metadata, refresh it after registry changes, and keep mutable backend values invocation-local
commandAt traverses names again after the adapter has already resolved a command	Duplicate route lookup can become another source of dispatch disagreement	Return a private command key from the adapter and resolve the captured runner directly

Selected internal ownership

Keep most cohesive core machinery private in the root package. A facade does not require forwarding-only files or duplicate representations for every public type. Use an internal package where it protects an actual dependency boundary.

Owner	Internal responsibility	Does not own
Core declarations	Captured command records, child handles, local declaration errors, root placement, explicit absence	Registry occupancy, provider acquisition, Cobra objects
Core bindings	Binding identity, typed conversion, presence/defaults, converted-value application, binding metadata validation	Scope names, ConfigKit/PromptKit types, application object cloning
Core registry	Validate/stage/commit installation batches and invalidate derived metadata	Integration-specific installed flags, command factories, report rendering
Core execution	Invoke the selected compiled runner, share the current presentation context, normalize/present outcomes	Application startup, provider schemas, process exit
internal/commandmeta (planned)	Narrow backend-neutral routing/help DTOs, private command keys, parse result shapes	Public contracts, typed handlers, registry policy, upstream library types
internal/cobra	Translate neutral metadata into one invocation session; route, collect raw occurrences, render help	Typed value conversion, declaration validation, family ownership, acquisitions
ckconfig private implementation	Capture scopes, compile applicable selectors, prepare selected inputs, acquire views, perform each configuration operation	Core registry state or private binding access
ckprompt private implementation	Capture definition providers, adapt engine inspection, prepare definition input/state, render definition reports	A second dispatcher/parser or configuration-suite installation

The neutral metadata package is justified by the existing core/adapter exchange: today its neutral structures live inside internal/cobra. Move those structures as a unit rather than introduce an additional copy of the same DTOs. Only the core execution coordinator needs to call the concrete adapter. Declaration and binding logic can depend on the neutral shapes without depending on adapter operations. This is not a backend-selection interface or a new public package.

Integrations use supported public core contracts. Do not solve extraction by letting them import core private assembly helpers. Small local selection-flag or report-formatting helpers are acceptable; a fourth utility module or a universal provider framework is not justified merely by a few similar declarations.

One installation path for every command

flowchart TD
    A[Application command factories] --> D[Captured core declarations]
    B[Integration family factories] --> D
    C[Core help and version factories] --> D
    D --> V[Shared batch validation and staging]
    V --> R[Instance registry commit]
    R --> M[Cached neutral routing and help metadata]
    M --> S[Fresh invocation adapter session]
    S --> E[Selected typed runner and common outcome delivery]

Construction and later registration

Recommend one private installBatch operation used by both New and App.Register:

1. New validates application identity and creates an unpublished empty app.
2. Core factories construct real help/version declarations, using the same typed command compiler as application declarations.
3. New submits built-ins and caller declarations together to installBatch.
4. App.Register submits its supplied declarations to that same operation.

There are no placeholder commands, direct built-in registry writes, privileged installation switches, or separate family installers. The app exists before its built-in callbacks are captured, but those callbacks do not execute during construction. A failed initial batch exposes no usable app.

Help/version names are occupied by real built-in nodes. Optional family names are occupied only by installed real roots. The ordinary collision check consequently implements reservation without a parallel list of known integration names. Root-only placement is captured metadata, not a check for a particular family name or the package declaring it.

Batch validation and commit

The shared installer should:

• Inspect retained errors and zero/absence state. Report attached errors before skipping an absent declaration, including absence nested in a group.
• Traverse real declarations with their installation path; enforce root placement and report contextual construction/structural errors. Read captured binding validation results rather than invoking parsers or factories.
• Check real root names against the existing registry and other roots in the pending batch. Sibling collisions inside a declaration belong to the same core structural-validation rules.
• Stage additions in a local pending map. Commit only after the entire batch is valid. Existing captured nodes need not be cloned to stage new roots.
• Invalidate derived routing/help metadata once after a successful nonempty commit. A failed batch or an entirely absent batch leaves registry and cached metadata intact.

All mutation is confined to the commit portion. Do not implement atomicity through temporary insertion followed by rollback or through copying the whole existing registry for every call. This preserves the selected per-call registry guarantee; it adds no transaction for callback effects, allocation failures, or concurrency.

Preserve retained causes with ordinary wrapping/joining where needed. Do not flatten them into strings. The implementation need not promise exhaustive error enumeration, stable error text, or a public aggregate-error type.

Capture once; keep installation state separate

Immutable declaration records

Recommend an opaque Command handle referring to a private captured record. That record contains descriptions, child handles, an optional compiled typed payload, retained errors, root-only placement, and explicit absence state.

Copy caller-owned metadata slices at their ownership boundary, including binding lists, child-handle lists, scope operations, and selector applicability. Once captured, those records are read-only. Group copies its child-handle slice; it does not recursively clone already captured child records. WithError returns a new wrapper/record retaining the existing content and added cause without mutating the original handle. RootGroup reuses ordinary group capture and adds placement metadata. A zero handle remains distinguishable from explicit absence.

This permits declaration reuse without copying entire trees at every assembly level. It does not clone application values, callbacks, or their object graphs. Do not write registry paths, app pointers, converted values, or invocation caches into reusable declarations. Core built-ins capture their own app-specific context when each app is constructed.

Keep payload and children independent in the private model. Do not introduce a new prohibition on runnable commands with children merely because separate leaf and group constructors are convenient.

The selected public addition is children on runnable commands through CommandSpec.Children. It exposes the behavior described by ADR 0006. Capture the child-handle slice using the same ownership and immediate composition checks as Group, and enforce the same placement rules at installation. Group remains a convenience for non-runnable parents. This addition requires no separate registration mechanism.

Divide validation by responsibility and lifetime

Use one implementation of each core metadata rule:

• Command/binding constructors capture local metadata and retain local errors.
• Group capture checks its immediate composition without recursively rerunning every child's local binding validator.
• Installation checks contextual placement and registry/batch collisions while reporting retained errors through one tree traversal.
• Integrations check scope identifiers, applicability references, declared operations, adapter issues, and enabled provider wiring. They hand generated flags and operands to core command construction for generic binding checks.

This prevents ancestor-by-ancestor validation and copying from growing with both tree size and nesting depth. Exact allocations remain implementation choices; do not add a global intern table, reflection-driven compiler, or mutable "validated" cache shared between applications.

Core name/short-name/help-reservation rules also check the combined standard and selector flags on generated commands. Integration code should not reproduce those rules with handwritten reserved-name lists. Validate bindings used by generated commands; unused selector bindings in an absent family are ignored, as clarified in the OQ-1 resolution below. Retained adapter-construction errors still reach installation.

One typed conversion and application mechanism

Prepared typed values

Retain the useful separation in the prototype's prepareInput: convert raw occurrences before initializing input. Refactor its prepared setter closures into one private typed prepared-value mechanism usable by ordinary bindings and CaptureFlags/FlagValues.

A prepared entry needs original binding identity, supplied state, and a typed application closure. The closure captures the converted/default value and applies it to initialized input storage. A repeatable binding copies its core-owned slice when applying it to another input; it does not clone arbitrary elements.

Scalar, boolean, repeated-flag, and operand constructors remain distinct where their parsing and cardinality differ. Their execution glue should feed the same typed preparation/application machinery, rather than create another conversion system for integrations.

CaptureFlags adapts each original binding to an outer command binding that stores the prepared typed entry in FlagValues. It does not apply selectors to a shared template, initialize suite input, or invoke a parser a second time. FlagValues.ApplyTo selects the original identities and applies their prepared values to the supplied destination.

All supplied occurrences undergo conversion, including selectors inapplicable to the selected scope and values supplied alongside help. Scope filtering governs value application and required-value checks, not conversion. An earlier malformed singleton occurrence remains an error even if a later occurrence is valid.

Recommend a private identity token per original binding and an invocation-local lookup from token to prepared entry. Handle copies preserve identity; lookalike bindings get a different token. A private nonzero-sized token object avoids relying on addresses of zero-sized Go objects. No global registry or public ID is needed. The lookup belongs to the invocation store, not the reusable binding.

Required values and errors

Separate required metadata used for help from the point where execution enforces it. Ordinary flags enforce their requirement in ordinary input preparation. Captured inner flags enforce it when ApplyTo selects applicable bindings. Do not erase required metadata or reimplement missing-value policy in each integration.

Apply the API's existing ErrFlagInput/ErrFlagWiring categories through one core classification helper. The command's input-validation path should recognize wrapped wiring failures as operational; ordinary validation and missing input use usage outcomes. An integration may wrap those causes with scope/family context without parsing error messages or inventing another outcome protocol.

Help still converts supplied occurrences and rejects malformed values through the same preparation logic, but does not materialize the target command's application or suite input, invoke its validation or execution callbacks, or enforce its execution-required fields. Suite state factories and providers also remain uninvoked. The built-in help command may prepare its own private typed input through the common compiler. Presence, singleton repetition, and repeated-value semantics remain those of the finalized API and selected grammar.

One compiled execution path, including help

Recommend that the private typed command compiler produce a prepared invocation closure. After raw conversion, this closure materializes I, applies values, runs CheckInput, and invokes the typed handler. Keep I inside that closure rather than passing an any input through separate factory/check/run calls and repeated type assertions. A heterogeneous registry can store the resulting non-generic private runner interface/function.

The private runner receives the current execution context, containing streams, cancellation, the app, and access to the current help/presentation session. For an application or integration handler, the compiler passes only the public Invocation and typed input. Built-in help uses a narrow private typed runner adapter to render targets through that same current execution context.

That private adapter shares declaration capture, metadata validation, binding preparation, registry installation, and outcome presentation with every other command. It changes only the final callback's access to core services. Version can use ordinary typed handler wiring with captured identity. Do not add a second built-in compiler or a dispatcher branch keyed on path == "help".

The adapter's local --help mechanism is still common syntax machinery, not a separately registered application command. Bare root/group help and explicit help commands use the same presentation service. Cobra must not add hidden command families independently of captured registrations.

Metadata and session lifetime

Preserve the existing cache/session split:

• Registry changes invalidate one derived neutral routing/help snapshot. Sorting and backend metadata projection occur when that snapshot is rebuilt, not on every parse or help operation.
• A fresh mutable adapter session is built for each execution and reused for its parsing, help, and usage presentation. Mutable Cobra values are not cached across invocations.
• Give each snapshot a private key-to-runner table. The adapter returns the selected neutral command key along with its path/raw values. Core execution uses the table associated with that session rather than resolving the same path again through commandAt.
• Command keys are meaningful only within their snapshot. Do not expose them, depend on persistent numbering, or use a newly rebuilt table to interpret a result from an older session.
• If a callback registers additional commands and subsequent presentation needs current metadata, refresh the presentation session on demand. Do not reparse argv or rerun factories, validators, or handlers merely to refresh help.

This removes duplicated route resolution and preserves useful existing behavior without promising concurrent dispatch, registration during another goroutine's execution, or a general snapshot-isolation contract.

Integration preparation without parallel execution frameworks

Configuration suite

At family construction, capture a private prepared suite containing ordered scope records, lookup by scope name, supported operations, discovery metadata, and the applicable original selector bindings for each scope. Those lists are a natural output of composing scopes and selectors; no separate mutable membership cache or public capability framework is needed.

Each inspection command uses an invocation-local outer input containing standard selectors, FlagValues[I], and its prepared operation selection. Recommend a pointer-shaped private outer input initialized by the command's own NewInput. Its CheckInput can populate the prepared selection and Run receives that same object. This matters because the public CheckInput receives its generic input by value: assigning a plan to a field in a copied value struct would lose it. This outer allocation is distinct from the suite's input and shared-state factories and is not performed on help paths.

The shared configuration preparation routine should:

1. Resolve scopes, operation availability, formats, and applicable operand rules from metadata before suite factories or acquisition.
2. For each selected scope, obtain a fresh I through NewInput or its zero value, apply only applicable captured bindings, and call the scope-aware CheckInput with scope identity and selection.
3. Retain the successfully prepared scope/input records in selected order on the outer invocation input. On failure, return the supported error and stop; later inputs need not be initialized.
4. In Run, after input validation succeeds and execution reaches acquisition, obtain shared S once through NewState or its zero value. Pass that state and each prepared input explicitly to the selected provider.

Do not recompute selection in every operation's Run, put the plan in a shared declaration closure, project a shallow copy of one input pointer, or clear non-applicable fields after initialization. Non-applicable bindings should leave the scope input's own initialized fields untouched.

Share this preparation across validate/show/sources/explain. Keep acquisition and report loops operation-owned: show acquires all selected views before report output, while other operations retain their selected timing. A universal "acquire everything then render" wrapper would alter behavior unnecessarily. Discovery reads captured metadata without initializing suite input or state.

AppCommand should build the ordinary scope and suite internally and call the same family factory. It is convenience assembly, not a second config installer, handler set, or execution cache.

PromptKit suite and scope adapter

Definition commands use the same core captured bindings, typed runner, and outcome delivery. A small private definition-input helper applies the entire selected flag set, validates the typed input, and retains it for the runner. It needs no configuration-scope projection or inherited mixed-context selector model.

Within ckprompt, share engine loading/inspection and diagnostic adaptation between scope and named-definition adapters where their upstream contract is the same. Inspect the original loader before wrapping it. Provider lookup failures still retain diagnostics and error identity. Keep report models and native configuration views with their proper owners.

ConfigScope constructs a public ckconfig.Scope and delegates configuration command generation to ckconfig. It must not construct another config family or reach into ckconfig's private prepared suite. Its construction-error handoff is the selected ckconfig.WithScopeError operation described below.

Share mechanics where ownership supports them

Core owns outcome classification and diagnostic delivery once; integration handlers return public outcomes and diagnostic envelopes. Within an integration, share mundane writing/encoding and provider-error helpers where appropriate, while preserving each report's actual timing and incomplete-delivery semantics.

Some small helpers may remain similar between independent modules. Do not expose generic encoders, upstream severity converters, standard-flag getters, or a generic pipeline merely to eliminate a few lines. Important duplication to remove is competing registration, validation, conversion, acquisition-lifetime, and presentation policy—not every structurally similar field assignment.

Efficiency and implementation review evidence

The intended cost profile is straightforward:

• Capture local metadata once and share read-only subtrees; avoid recursively copying/revalidating a child at every ancestor.
• Stage only added roots, with one contextual tree walk and ordinary name lookups; avoid cloning the existing registry or installing commands one by one inside a supposedly atomic batch.
• Derive routing/help metadata once per changed registry state, and use the adapter's selected command key rather than repeating route traversal.
• Convert each supplied occurrence once per invocation. Apply prepared values only to selected inputs, copying only core-owned repeated collections.
• Prepare scope selection once per invocation and shared state once when acquisition is reached. No process-global provider cache is introduced.

These are algorithmic recommendations, not latency/allocation guarantees. Existing benchmarks can inform the implementation sprint; this audit has not run them or measured performance. Profile before adding pools, segmented trees, bitsets, concurrent registries, or mutable backend reuse.

Focused implementation evidence should demonstrate the common path with all three command sources: built-ins, application declarations, and integrations. Include invalid bindings, conflicting roots in either order, an invalid final member of a batch, nested root-only declarations, absence with retained errors, source metadata edits after capture, and declaration reuse across apps. Include runnable parents with children through CommandSpec.Children, applying the same capture, collision, and placement checks as ordinary groups.

For execution, check wrapped wiring versus input failures, fresh repeatable collections, pointer-shaped scope inputs with shared state, scope-aware validation, help without target application/suite factories or providers, and metadata refresh without callback replay. Check that supplied but inapplicable selectors still undergo conversion, including on help paths, while the built-in help command can prepare its own private input through the common compiler. Evidence for the internal path itself can inspect the shared installer/compiler; do not turn a private function name or exact allocation count into a public contract. These are review targets for the future implementation, not a request to add or run tests during this planning pass.

Resolved scope-error handoff (OQ-2)

The selected Scope[I, S] includes private adapter-construction issues owned by ckconfig. Before the selected addition below, ckprompt.ConfigScope could not attach its original nil-loader error through supported public contracts. An Acquire == nil check alone is insufficient: narrowing Operations to empty makes a scope metadata-only, while the original invalid loader must remain an error. Wrapping the nil loader in a non-nil acquisition closure would incorrectly defer it to invocation. A core WithError only accepts a Command, not a scope.

Selected: use the integration-owned retained scope-error operation, ckconfig.WithScopeError, declared in the registration API document. It returns a scope retaining the original issue, preserving supported error identity. Repeated attachments preserve earlier causes and leave the original scope unchanged; a nil error preserves the scope. Retained causes survive scope copies and narrowing of Operations. The family factory collects those issues into its command declaration before capability selection or absence can discard them. ConfigScope uses this for its original loader validation and otherwise fills the ordinary public scope fields.

This mirrors the accepted core retained-error model at the boundary that needs it. It does not add scope installation, expose private representations, or move PromptKit types into the core. Scope capture still occurs at the existing family factory; this operation is evidence attachment rather than a new lifecycle.

Alternative considered: give the public scope declaration an explicit construction-error field, consumed by the family factory alongside provider metadata. This is simple and permits application-created adapters to supply errors directly. It expands mutable public representation and requires defining preservation/combination of multiple errors; the opaque attachment operation keeps those details private.

Resolved unused-binding validation (OQ-1)

Selected: empty definition suites contribute no commands, and unused selector bindings are ignored. Generate commands for the supplied capabilities and validate the bindings needed by those commands. When neither definition-inspection capability is enabled, the family contributes no namespace, help entry, or handler. There is no need to construct validation-only commands or a temporary application for selector bindings that will not be installed.

This follows the capability-driven assembly model: selecting no capabilities naturally produces no commands. Rejecting unused bindings would report defects that cannot affect routing, parsing, or execution. This clarifies the validation boundary while preserving ADR 0011's selected empty-suite behavior; it does not promise exhaustive validation of unused metadata.

An explicitly invalid adapter construction remains distinct from an omitted capability. Retain errors such as a nil loader supplied to an adapter even when its capabilities are subsequently cleared. Return ordinary NoCommand() for an empty definition suite, attaching any retained construction errors through the existing WithError. Installation checks those errors before skipping absence. No validation-bearing NoCommand overload or public declaration-validation helper is needed.

The binding rule also applies to configuration selectors unused by any generated inspection command. Existing configuration-suite semantics remain unchanged: zero named scopes is an error, while named metadata-only scopes supply the real config scopes discovery command and remain valid. Integration scope identifiers, applicability references, and enabled-provider wiring retain their own validation; ignoring unused bindings does not erase retained adapter errors or bypass checks on commands that are generated.

This construction-time rule does not skip invocation-time conversion of a supplied flag on an installed command merely because it is inapplicable to the selected scope. Conversion and scope-specific application remain separate as described above.

Alternatives considered: rejecting empty definition suites could catch an accidental omission, but would make an empty capability selection an error without a demonstrated need. Validating all unused bindings, through either an extended absence factory or an explicit validation helper, would catch latent defects but add public API and validation machinery for commands that do not exist. Neither alternative is selected.