Files
commandkit/docs/roadmap/promptkit-companion-yaml-exit-status.md

92 lines
4.9 KiB
Markdown
Raw Normal View History

2026-11-17 00:11:00 +00:00
# PromptKit companion YAML still exits with status 1 in v0.8.0
2026-11-17 00:53:48 +00:00
Status — resolved in the prepared ckprompt v0.9.0 candidate with PromptKit
v0.23.1. `TestPromptConstructorConfigurationOutcomes` exercises real malformed
companion construction through both integrations, including source and validation
reports and both comparison sides. `TestPromptConstructorFailurePrecedence`
checks operational and cancellation precedence and preserved causes.
2026-11-17 00:11:00 +00:00
The original Audita report covered invalid configuration passing through both
`ckprompt` and `ckconfig` v0.7.0. Retesting with both integrations at v0.8.0
confirmed that malformed main configuration now exits with status `2`. A
malformed `prompts.yml` companion still exits with status `1` through both
integrations, even though stderr includes the ConfigKit `yaml.invalid` diagnostic.
[Original Audita reproduction](https://forge.maximumdirect.net/eric/audita/src/commit/a355f05a273d6ac9d577eab8d30caa60cb74eee8/docs/roadmap/implementation.md#stage-1--establish-the-delegated-inspection-dependency)
## Reproduction
From the Audita repository root, build from a scratch copy of the reported
revision. The two integration pins are the only selected module versions that
change. The build uses CommandKit v0.6.0, ConfigKit v1.2.0, and PromptKit v0.23.0.
```sh
scratch=$(mktemp -d)
git archive a355f05a273d6ac9d577eab8d30caa60cb74eee8 | tar -x -C "$scratch"
(
cd "$scratch" || exit
export GOWORK=off
go get go.maximumdirect.net/ckprompt@v0.8.0 go.maximumdirect.net/ckconfig@v0.8.0 &&
go build -buildvcs=false -o "$scratch/audita" ./cmd/audita
)
mkdir "$scratch/repro"
cd "$scratch/repro"
printf '%s\n' 'version: 3' > config.yml
printf '%s\n' 'profiles: [' > prompts.yml
printf '%s\n' '[]' > transcript.json
printf '%s\n' 'version: [' > bad.yml
```
Run the following commands using `"$scratch/audita"` from the fixture
directory. They fail during configuration loading, before any provider calls.
| Command arguments | v0.7.0 | v0.8.0 | Expected | Diagnostic |
| --- | ---: | ---: | ---: | --- |
| `promptkit list prompts --config config.yml` | 1 | 1 | 2 | `yaml.invalid` in `prompts.yml` |
| `config show --scope promptkit --config config.yml` | 1 | 1 | 2 | `yaml.invalid` in `prompts.yml` |
| `process transcript.json --config config.yml` | 2 | 2 | 2 | `yaml.invalid` in `prompts.yml` |
| `config show --scope app --config bad.yml` | 1 | 2 | 2 | `yaml.invalid` in `bad.yml` |
`config sources`, `config validate`, `config explain --list`, and
`config diff --against-config config.yml` match the `config show` result in
each scope. The source and validation commands still deliver their available
reports on stdout. With `--config bad.yml`, both `promptkit list prompts`
and `config show --scope promptkit` also return `2` in v0.8.0 because the
main configuration load fails before construction of the PromptKit engine.
## Why the companion case returns status 1
PromptKit's `NewEngine` wraps engine construction errors with
`fmt.Errorf("%w: %w", ErrInvalidConfig, err)`. In the reproduction the two
branches are the `promptkit.ErrInvalidConfig` category sentinel and a wrapped
`*configkit.LoadError` containing the `yaml.invalid` diagnostic.
[PromptKit v0.23.0 engine source](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fcbff80ab227397a03935ab3bf8114481c0736a7/engine.go)
Both integrations can recover the typed load error through `errors.As`, so
the source and diagnostic reach stderr. `ckconfig.ClassifyConfigurationError`
classifies the typed load branch as usage and the standalone sentinel branch
as operational. The full error therefore becomes operational because the
classifier requires every branch to identify invalid input. Both
`ckprompt.definitionFailure` and `ckconfig.configFailure` use that shared
classification.
[ckconfig v0.8.0 classifier](../../integrations/ckconfig/errors.go) ·
[ckconfig outcome handling](../../integrations/ckconfig/operations.go) ·
[ckprompt outcome handling](../../integrations/ckprompt/command.go)
## Expected behavior
- Return status `2` when the typed diagnostic identifies malformed companion
YAML. Keep the diagnostic, source information, available reports, and typed
cause. Preserve recognition of `promptkit.ErrInvalidConfig` through
`errors.Is`.
- Return status `1` for cancellation and genuine operational failures,
including mixed errors with an independent operational cause. Audita's
already-canceled context test passed against v0.8.0 for catalog listing,
configuration show, and processing. It checks status `1`,
`context.Canceled` in the cause chain, and empty stdout.
One possible upstream fix is for PromptKit to expose `ErrInvalidConfig` via
`Is` on a wrapper that unwraps to the underlying cause. This would let the
typed load branch determine the outcome for malformed YAML while preserving
the rule for independent operational branches.
[CommandKit configuration failure contract](../cli.md#configuration-failure-classification)