92 lines
4.9 KiB
Markdown
92 lines
4.9 KiB
Markdown
# PromptKit companion YAML still exits with status 1 in v0.8.0
|
|
|
|
Status — resolved in the prepared ckprompt v0.9.0 candidate with PromptKit
|
|
v0.23.1. `TestPromptConstructorConfigurationOutcomes` exercises real malformed
|
|
companion construction through both integrations, including source and validation
|
|
reports and both comparison sides. `TestPromptConstructorFailurePrecedence`
|
|
checks operational and cancellation precedence and preserved causes.
|
|
|
|
The original Audita report covered invalid configuration passing through both
|
|
`ckprompt` and `ckconfig` v0.7.0. Retesting with both integrations at v0.8.0
|
|
confirmed that malformed main configuration now exits with status `2`. A
|
|
malformed `prompts.yml` companion still exits with status `1` through both
|
|
integrations, even though stderr includes the ConfigKit `yaml.invalid` diagnostic.
|
|
[Original Audita reproduction](https://forge.maximumdirect.net/eric/audita/src/commit/a355f05a273d6ac9d577eab8d30caa60cb74eee8/docs/roadmap/implementation.md#stage-1--establish-the-delegated-inspection-dependency)
|
|
|
|
## Reproduction
|
|
|
|
From the Audita repository root, build from a scratch copy of the reported
|
|
revision. The two integration pins are the only selected module versions that
|
|
change. The build uses CommandKit v0.6.0, ConfigKit v1.2.0, and PromptKit v0.23.0.
|
|
|
|
```sh
|
|
scratch=$(mktemp -d)
|
|
git archive a355f05a273d6ac9d577eab8d30caa60cb74eee8 | tar -x -C "$scratch"
|
|
(
|
|
cd "$scratch" || exit
|
|
export GOWORK=off
|
|
go get go.maximumdirect.net/ckprompt@v0.8.0 go.maximumdirect.net/ckconfig@v0.8.0 &&
|
|
go build -buildvcs=false -o "$scratch/audita" ./cmd/audita
|
|
)
|
|
mkdir "$scratch/repro"
|
|
cd "$scratch/repro"
|
|
printf '%s\n' 'version: 3' > config.yml
|
|
printf '%s\n' 'profiles: [' > prompts.yml
|
|
printf '%s\n' '[]' > transcript.json
|
|
printf '%s\n' 'version: [' > bad.yml
|
|
```
|
|
|
|
Run the following commands using `"$scratch/audita"` from the fixture
|
|
directory. They fail during configuration loading, before any provider calls.
|
|
|
|
| Command arguments | v0.7.0 | v0.8.0 | Expected | Diagnostic |
|
|
| --- | ---: | ---: | ---: | --- |
|
|
| `promptkit list prompts --config config.yml` | 1 | 1 | 2 | `yaml.invalid` in `prompts.yml` |
|
|
| `config show --scope promptkit --config config.yml` | 1 | 1 | 2 | `yaml.invalid` in `prompts.yml` |
|
|
| `process transcript.json --config config.yml` | 2 | 2 | 2 | `yaml.invalid` in `prompts.yml` |
|
|
| `config show --scope app --config bad.yml` | 1 | 2 | 2 | `yaml.invalid` in `bad.yml` |
|
|
|
|
`config sources`, `config validate`, `config explain --list`, and
|
|
`config diff --against-config config.yml` match the `config show` result in
|
|
each scope. The source and validation commands still deliver their available
|
|
reports on stdout. With `--config bad.yml`, both `promptkit list prompts`
|
|
and `config show --scope promptkit` also return `2` in v0.8.0 because the
|
|
main configuration load fails before construction of the PromptKit engine.
|
|
|
|
## Why the companion case returns status 1
|
|
|
|
PromptKit's `NewEngine` wraps engine construction errors with
|
|
`fmt.Errorf("%w: %w", ErrInvalidConfig, err)`. In the reproduction the two
|
|
branches are the `promptkit.ErrInvalidConfig` category sentinel and a wrapped
|
|
`*configkit.LoadError` containing the `yaml.invalid` diagnostic.
|
|
[PromptKit v0.23.0 engine source](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/fcbff80ab227397a03935ab3bf8114481c0736a7/engine.go)
|
|
|
|
Both integrations can recover the typed load error through `errors.As`, so
|
|
the source and diagnostic reach stderr. `ckconfig.ClassifyConfigurationError`
|
|
classifies the typed load branch as usage and the standalone sentinel branch
|
|
as operational. The full error therefore becomes operational because the
|
|
classifier requires every branch to identify invalid input. Both
|
|
`ckprompt.definitionFailure` and `ckconfig.configFailure` use that shared
|
|
classification.
|
|
[ckconfig v0.8.0 classifier](../../integrations/ckconfig/errors.go) ·
|
|
[ckconfig outcome handling](../../integrations/ckconfig/operations.go) ·
|
|
[ckprompt outcome handling](../../integrations/ckprompt/command.go)
|
|
|
|
## Expected behavior
|
|
|
|
- Return status `2` when the typed diagnostic identifies malformed companion
|
|
YAML. Keep the diagnostic, source information, available reports, and typed
|
|
cause. Preserve recognition of `promptkit.ErrInvalidConfig` through
|
|
`errors.Is`.
|
|
- Return status `1` for cancellation and genuine operational failures,
|
|
including mixed errors with an independent operational cause. Audita's
|
|
already-canceled context test passed against v0.8.0 for catalog listing,
|
|
configuration show, and processing. It checks status `1`,
|
|
`context.Canceled` in the cause chain, and empty stdout.
|
|
|
|
One possible upstream fix is for PromptKit to expose `ErrInvalidConfig` via
|
|
`Is` on a wrapper that unwraps to the underlying cause. This would let the
|
|
typed load branch determine the outcome for malformed YAML while preserving
|
|
the rule for independent operational branches.
|
|
[CommandKit configuration failure contract](../cli.md#configuration-failure-classification)
|