Files
commandkit/docs/roadmap/promptkit-companion-yaml-exit-status.md

4.9 KiB

PromptKit companion YAML still exits with status 1 in v0.8.0

Status — resolved in the prepared ckprompt v0.9.0 candidate with PromptKit v0.23.1. TestPromptConstructorConfigurationOutcomes exercises real malformed companion construction through both integrations, including source and validation reports and both comparison sides. TestPromptConstructorFailurePrecedence checks operational and cancellation precedence and preserved causes.

The original Audita report covered invalid configuration passing through both ckprompt and ckconfig v0.7.0. Retesting with both integrations at v0.8.0 confirmed that malformed main configuration now exits with status 2. A malformed prompts.yml companion still exits with status 1 through both integrations, even though stderr includes the ConfigKit yaml.invalid diagnostic. Original Audita reproduction

Reproduction

From the Audita repository root, build from a scratch copy of the reported revision. The two integration pins are the only selected module versions that change. The build uses CommandKit v0.6.0, ConfigKit v1.2.0, and PromptKit v0.23.0.

scratch=$(mktemp -d)
git archive a355f05a273d6ac9d577eab8d30caa60cb74eee8 | tar -x -C "$scratch"
(
  cd "$scratch" || exit
  export GOWORK=off
  go get go.maximumdirect.net/ckprompt@v0.8.0 go.maximumdirect.net/ckconfig@v0.8.0 &&
    go build -buildvcs=false -o "$scratch/audita" ./cmd/audita
)
mkdir "$scratch/repro"
cd "$scratch/repro"
printf '%s\n' 'version: 3' > config.yml
printf '%s\n' 'profiles: [' > prompts.yml
printf '%s\n' '[]' > transcript.json
printf '%s\n' 'version: [' > bad.yml

Run the following commands using "$scratch/audita" from the fixture directory. They fail during configuration loading, before any provider calls.

Command arguments v0.7.0 v0.8.0 Expected Diagnostic
promptkit list prompts --config config.yml 1 1 2 yaml.invalid in prompts.yml
config show --scope promptkit --config config.yml 1 1 2 yaml.invalid in prompts.yml
process transcript.json --config config.yml 2 2 2 yaml.invalid in prompts.yml
config show --scope app --config bad.yml 1 2 2 yaml.invalid in bad.yml

config sources, config validate, config explain --list, and config diff --against-config config.yml match the config show result in each scope. The source and validation commands still deliver their available reports on stdout. With --config bad.yml, both promptkit list prompts and config show --scope promptkit also return 2 in v0.8.0 because the main configuration load fails before construction of the PromptKit engine.

Why the companion case returns status 1

PromptKit's NewEngine wraps engine construction errors with fmt.Errorf("%w: %w", ErrInvalidConfig, err). In the reproduction the two branches are the promptkit.ErrInvalidConfig category sentinel and a wrapped *configkit.LoadError containing the yaml.invalid diagnostic. PromptKit v0.23.0 engine source

Both integrations can recover the typed load error through errors.As, so the source and diagnostic reach stderr. ckconfig.ClassifyConfigurationError classifies the typed load branch as usage and the standalone sentinel branch as operational. The full error therefore becomes operational because the classifier requires every branch to identify invalid input. Both ckprompt.definitionFailure and ckconfig.configFailure use that shared classification. ckconfig v0.8.0 classifier · ckconfig outcome handling · ckprompt outcome handling

Expected behavior

  • Return status 2 when the typed diagnostic identifies malformed companion YAML. Keep the diagnostic, source information, available reports, and typed cause. Preserve recognition of promptkit.ErrInvalidConfig through errors.Is.
  • Return status 1 for cancellation and genuine operational failures, including mixed errors with an independent operational cause. Audita's already-canceled context test passed against v0.8.0 for catalog listing, configuration show, and processing. It checks status 1, context.Canceled in the cause chain, and empty stdout.

One possible upstream fix is for PromptKit to expose ErrInvalidConfig via Is on a wrapper that unwraps to the underlying cause. This would let the typed load branch determine the outcome for malformed YAML while preserving the rule for independent operational branches. CommandKit configuration failure contract