238 lines
13 KiB
Markdown
238 lines
13 KiB
Markdown
# Invalid configuration outcomes
|
|
|
|
Status — proposed implementation roadmap. The user has accepted the targeted
|
|
integration behavior. The decisions at the end identify remaining policy and
|
|
API choices for the implementation plan.
|
|
|
|
## Purpose
|
|
|
|
Make standard ckconfig and ckprompt commands return usage status `2` for
|
|
identifiable invalid configuration input, while returning operational status `1`
|
|
for cancellation and other operational failures. Preserve actionable upstream
|
|
diagnostics and supported typed error identity through the integration boundary.
|
|
|
|
CommandKit owns this CLI classification. ConfigKit and PromptKit own loading,
|
|
validation, diagnostic meaning, and their public error contracts. Application
|
|
commands own their domain outcomes. These responsibilities follow the
|
|
[architecture policy](../policy/architecture.md#ownership-boundaries).
|
|
|
|
## Current baseline
|
|
|
|
The reviewed baseline is CommandKit commit
|
|
`1e92a87706e44684ee34395245e5b633474677e0`, following the publication of
|
|
ckconfig v0.7.0 and ckprompt v0.7.0. Both integration modules select core
|
|
v0.6.0 and ConfigKit v1.2.0. ckprompt selects ckconfig v0.7.0 and PromptKit
|
|
v0.21.0. Their manifests own the dependency selections.
|
|
|
|
The consumer report reproduced malformed operator YAML in Audita, including
|
|
its automatic `prompts.yml` companion and an explicitly selected application
|
|
configuration. Delegated inspection returned `yaml.invalid` with status `1`,
|
|
while Audita's processing command returned the same diagnostic with status `2`.
|
|
The report's reproduction is retained in Audita's
|
|
[upstream handoff](https://forge.maximumdirect.net/eric/audita/src/commit/a355f05a273d6ac9d577eab8d30caa60cb74eee8/docs/roadmap/implementation.md).
|
|
|
|
The review confirmed the behavior through Audita's existing regression tests and
|
|
a temporary public-API reproduction against the aligned CommandKit workspace.
|
|
The latter covered catalog listing and all five configuration operations in both
|
|
the application and PromptKit scopes, including wrapped ConfigKit loading errors
|
|
and already-canceled contexts.
|
|
|
|
- [FromConfigKit](../../integrations/ckconfig/configkit_adapter.go) and
|
|
[the PromptKit adapters](../../integrations/ckprompt/adapters.go) preserve the
|
|
ConfigKit loading error and adapt its diagnostic evidence.
|
|
- [configFailure](../../integrations/ckconfig/operations.go) and
|
|
[definitionFailure](../../integrations/ckprompt/command.go) construct operational
|
|
outcomes for the affected failures. Configuration comparison and validation
|
|
also reach the former helper.
|
|
- ConfigKit's public `LoadError` represents several kinds of loading failure,
|
|
including malformed YAML, acquisition errors, and schema or callback problems.
|
|
Its public diagnostic codes provide the classification evidence. Its error
|
|
contract intentionally exposes safe diagnostics and preserves its own typed
|
|
identity across CommandKit adaptation.
|
|
- `config sources` can report discovered source evidence after failed loading,
|
|
and `config validate` can report failed validation. Cancellation observed
|
|
before acquisition leaves stdout empty in the reviewed command paths.
|
|
|
|
The current [CLI contract](../cli.md#outcome-status) assigns loading failures
|
|
status `1`, with more specific statements in the configuration comparison,
|
|
requirements, validation, and smoke sections. This roadmap proposes a deliberate
|
|
change to that published status behavior.
|
|
|
|
## Targeted end state
|
|
|
|
| Evidence available to the integration | Selected outcome |
|
|
| --- | --- |
|
|
| ConfigKit error diagnostics that identify invalid configuration input under the selected mapping | Usage, status `2` |
|
|
| Caller cancellation or deadline observed at an existing operation checkpoint, or present in the returned error chain | Operational, status `1` |
|
|
| I/O, schema or wiring defects, callback execution failures, and other recognized operational failures | Operational, status `1` |
|
|
| An unrecognized failure, including a loading error with insufficient classification evidence | Operational, status `1` |
|
|
| Successful acquisition with warnings | Follow the command's successful-result behavior |
|
|
|
|
Classification uses upstream structured evidence. Diagnostic text, filenames,
|
|
and source labels serve presentation and provenance. The exact input-error code
|
|
mapping is proposed under Open Decisions.
|
|
|
|
The selected outcome retains the original returned cause, including ordinary
|
|
wrapping and joined errors, so supported `errors.Is` and `errors.As` queries
|
|
continue to work. Existing explanations, source context, diagnostic visibility,
|
|
and operation-specific reporting lifecycles apply to the newly classified
|
|
failures. Shared rendering and delivery failures follow the
|
|
[core output-failure rules](../cli.md#output-failures), including their effect on
|
|
usage outcomes.
|
|
|
|
Apply the classification to configuration acquisition failures throughout the
|
|
standard suites, including failures returned by application callbacks when they
|
|
carry the supported ConfigKit evidence. CLI syntax, resource lookup, request
|
|
execution, and application-specific error semantics retain their owning
|
|
contracts. The work concerns interpreting returned failure evidence at the CLI
|
|
boundary.
|
|
|
|
## Work needed
|
|
|
|
### 1. Define the configuration-failure classification
|
|
|
|
- Inventory ConfigKit v1.2.0's public diagnostic codes and their documented
|
|
meaning. Record the selected input-error cases explicitly using upstream
|
|
constants. Review all code families, including composition, imports,
|
|
inheritance, versions, overrides, secrets, and configured limits.
|
|
- Use error-severity diagnostics as failure evidence. Warning records can
|
|
accompany a failure or a successful result.
|
|
- Handle wrapped `LoadError` values and standard joined error branches. A failure
|
|
selects usage only when its failing evidence is classified as invalid input.
|
|
An operational or unclassified failing branch makes the combined result
|
|
operational. Preserve the original cause tree.
|
|
- Account for cancellation and deadlines before choosing usage. When an existing
|
|
checkpoint observes cancellation alongside an acquisition failure, preserve
|
|
both causes using the established error-joining approach.
|
|
- Keep classification within integration ownership, using the sharing approach
|
|
selected below. Existing core outcome constructors and reporting provide the
|
|
required mechanics.
|
|
|
|
### 2. Apply it throughout ckconfig
|
|
|
|
Wire the classification into acquisition failure handling for `config show`,
|
|
`config sources`, `config validate`, `config explain`, and `config diff`.
|
|
|
|
Follow each operation's current preparation, aggregation, and reporting flow.
|
|
In particular, source and validation reports can accompany a failed load.
|
|
Detailed validation filters control displayed findings, while the complete
|
|
underlying failure supplies classification evidence. Comparison must classify a
|
|
failure on either loading side and retain the side-specific diagnostic context.
|
|
|
|
Include native ConfigKit scopes, PromptKit configuration scopes, and custom
|
|
scope callbacks returning the supported evidence. Scope selection and input
|
|
preflight continue through their existing contracts.
|
|
|
|
### 3. Apply it throughout ckprompt
|
|
|
|
Cover engine-acquisition failures in catalogs, named prompt/profile/backend
|
|
inspection, requirements discovery, concrete request validation, prepared-request
|
|
preview, individual smoke operations, and aggregate smoke testing.
|
|
|
|
Acquisition evidence may arrive through loaders or application request builders.
|
|
Use the same classification policy wherever it reaches the standard command's
|
|
failure boundary. For joined aggregate smoke errors, an operational or
|
|
unclassified failure takes precedence over input failures. Follow the existing
|
|
aggregate execution and report lifecycle when selecting the final outcome.
|
|
|
|
### 4. Verify observable integration behavior
|
|
|
|
Extend focused public-command tests where they provide regression protection.
|
|
Use upstream errors as fixtures to establish CommandKit's classification and
|
|
preservation behavior, following the
|
|
[integration testing policy](../policy/testing.md#upstream-integration-testing-ownership).
|
|
|
|
Cover these distinct risks with a proportionate set of cases.
|
|
|
|
- Malformed main and companion configuration produce status `2` with
|
|
`yaml.invalid` and the typed ConfigKit cause available through wrapping.
|
|
- Representative decoding and selected semantic-validation errors receive the
|
|
intended classification. Operational and unrecognized loading diagnostics
|
|
receive status `1`, including mixed failing evidence and warnings alongside
|
|
errors.
|
|
- The five configuration operations and affected PromptKit command families use
|
|
the policy, including comparison-side failures and aggregate smoke outcomes.
|
|
- Cancellation and deadlines receive status `1` and preserve their supported
|
|
identity. Already-canceled invocations retain the reviewed empty-output
|
|
behavior. A failed acquisition combined with cancellation preserves both causes.
|
|
- Source and validation reports retain their operation-specific failure evidence.
|
|
Detailed display filters do not determine failure classification.
|
|
- Output failures continue to follow the shared reporting contract when the
|
|
command has selected usage status `2`.
|
|
|
|
Run the complete deterministic suites independently for each changed integration
|
|
and with the aligned local workspace as described in the
|
|
[developer guide](../development.md#documentation-validation). Exercise the
|
|
maintained examples where their existing assertions cover affected outcomes.
|
|
The Audita reproduction supplies supporting consumer evidence. CommandKit-owned
|
|
tests provide the maintained regression coverage.
|
|
|
|
### 5. Document and prepare the integration releases
|
|
|
|
Update the CLI status table and affected operation sections with the implemented
|
|
classification. Update any affected exported GoDoc and consumer guidance under
|
|
their canonical owners. Explain the classification boundary and fallback through
|
|
the chosen API's documentation.
|
|
|
|
Prepare ckconfig v0.8.0 followed by ckprompt v0.8.0, with ckprompt selecting the
|
|
published ckconfig version. The status-contract change warrants minor releases
|
|
under the [compatibility policy](../policy/release.md#versioning-and-compatibility).
|
|
The release notes should explain that identifiable invalid configuration now
|
|
selects status `2`, alongside the operational classification and preserved error
|
|
evidence. Follow the [release procedure](../release.md) when publication is
|
|
authorized. Implementation completion and publication are tracked separately.
|
|
|
|
## Completion evidence
|
|
|
|
Implementation is complete when the accepted classification is exercised through
|
|
the public integration commands, the focused regression cases and selected
|
|
integration suites pass, and the canonical documentation describes the
|
|
implemented behavior. Review the final diff for the agreed scope and retire this
|
|
roadmap under the normal documentation lifecycle after recording any unfinished
|
|
release work.
|
|
|
|
## Open Decisions
|
|
|
|
### Exact diagnostic-code mapping
|
|
|
|
**Recommendation.** Use an explicit mapping based on ConfigKit v1.2.0's public
|
|
code semantics. Include YAML-content and decoding errors, explicit
|
|
`version.error`, `resolve.error`, and `validate.error` findings, and other codes
|
|
whose documented meaning clearly identifies invalid configuration content.
|
|
Classify schema/options defects, acquisition failures, hook execution failures,
|
|
invalid callback paths or records, and validation mutation as operational.
|
|
Keep ambiguous limit, secret, and source-selection cases operational until their
|
|
individual meaning supports input classification. Review and record those cases
|
|
when preparing the implementation plan.
|
|
|
|
This uses the evidence upstream already supplies and distinguishes reported
|
|
configuration rejection from callback machinery failing. Source evidence
|
|
alone may be insufficient to establish whether application-supplied data came
|
|
from an operator, so the rule follows the meaning recorded in upstream
|
|
diagnostics.
|
|
|
|
**Alternative.** Add an application-supplied classification hook for cases that
|
|
depend on source ownership or application policy, alongside the built-in mapping.
|
|
Applications can then identify input failures using knowledge their loaders
|
|
possess. This supports genuinely contextual cases, but expands the public
|
|
integration contracts and creates additional configuration and precedence rules.
|
|
The current reproduction is addressable through existing typed diagnostics, so
|
|
the additional hook has limited demonstrated value in this scope.
|
|
|
|
### Shared classification interface
|
|
|
|
**Recommendation.** Place one narrowly documented classification function in
|
|
ckconfig and have ckprompt call it through its existing dependency. Keep command
|
|
explanation construction and reporting in their owning handlers. Select its
|
|
exact signature during implementation planning based on the needed cause and
|
|
cancellation inputs. A shared implementation reduces drift across the two
|
|
standard families and fits ckconfig's ownership of ConfigKit adaptation. The
|
|
public function adds an API maintenance obligation.
|
|
|
|
**Alternative.** Maintain small private classifiers in both integration modules,
|
|
with matching behavioral cases. This keeps the helper private and supports
|
|
package-local evolution, but duplicates the diagnostic mapping and joined-error
|
|
rules. Future upstream code changes would require coordinated edits in both
|
|
modules. Prefer this approach if the shared signature would expose more policy
|
|
or machinery than the demonstrated integration need warrants.
|