Add PromptKit definition inspection
This commit is contained in:
@@ -8,7 +8,8 @@ and operand bindings, Cobra-backed routing, invocation outcomes, generated help,
|
||||
application or CommandKit version reporting, configuration scope discovery,
|
||||
ConfigKit-backed validation, native effective YAML display, source inventory,
|
||||
and path explanation reports. Applications can register a PromptKit engine
|
||||
configuration scope independently of PromptKit definition commands.
|
||||
configuration scope independently of PromptKit definition commands, and can
|
||||
install optional prompt and profile inspection commands.
|
||||
|
||||
## Documentation
|
||||
|
||||
|
||||
70
docs/cli.md
70
docs/cli.md
@@ -225,6 +225,76 @@ environment-variable name; effective and historical secret fragments remain
|
||||
redacted. Empty location, inheritance, history, reason, and dependency lists are
|
||||
serialized as `[]`.
|
||||
|
||||
## PromptKit definition inspection
|
||||
|
||||
Applications can install prompt and profile inspection independently with
|
||||
`RegisterPromptKit`. Each installed handler constructs the selected engine once
|
||||
and asks PromptKit to inspect the named definition. Inspection reads the
|
||||
engine's captured configuration; it does not generate a prompt, contact a
|
||||
provider, or look up credentials.
|
||||
Prompt definitions and profile resolution remain owned by
|
||||
[PromptKit](https://pkg.go.dev/gitea.maximumdirect.net/eric/promptkit).
|
||||
|
||||
```text
|
||||
example promptkit show prompt customer/summary --version 2 --format yaml
|
||||
example promptkit show prompt customer/summary
|
||||
example promptkit show profile customer-support --format yaml
|
||||
```
|
||||
|
||||
Both commands require one ID operand and accept `--config`, `--overlay`, and
|
||||
`--format text|yaml` (default `text`). Prompt inspection also accepts singleton
|
||||
`--version`, defaulting to the empty string. PromptKit owns version selection:
|
||||
an omitted version works for a single-version ID, while an ambiguous ID is an
|
||||
operational error. These commands do not accept `--scope`; their namespace is
|
||||
`promptkit`.
|
||||
|
||||
Prompt YAML has these fields. Input order is preserved and an empty input list is
|
||||
`[]`. The prompt hash is an opaque owning-library equality value.
|
||||
|
||||
```yaml
|
||||
prompt_id: customer/summary
|
||||
prompt_version: "2"
|
||||
prompt_hash: opaque-value
|
||||
default_profile_id: customer-support
|
||||
inputs:
|
||||
- name: topic
|
||||
required: true
|
||||
content_type: text/plain
|
||||
description: Topic to summarize
|
||||
output_contract:
|
||||
format: json
|
||||
validation_mode: basic
|
||||
schema_path: ""
|
||||
repair_attempts: 2
|
||||
```
|
||||
|
||||
Profile YAML preserves the effective values reported by PromptKit, including
|
||||
zero values and nested provider parameters. `api_key_env` is only an
|
||||
environment-variable name. `api_key_required` reports a direct credential
|
||||
requirement; neither field reports credential availability. A nil provider
|
||||
parameter map is displayed as `{}`.
|
||||
|
||||
```yaml
|
||||
profile_id: customer-support
|
||||
effective_model_params:
|
||||
backend_id: local
|
||||
endpoint: https://example.invalid/v1
|
||||
model: support-model
|
||||
temperature: 0
|
||||
max_tokens: 0
|
||||
top_p: 0
|
||||
timeout_seconds: 600
|
||||
service_tier: ""
|
||||
reasoning_effort: ""
|
||||
api_key_env: ""
|
||||
extra_params: {}
|
||||
api_key_required: true
|
||||
```
|
||||
|
||||
Text uses labeled fields and includes nested `extra_params` values. A missing ID
|
||||
is a usage error. Blank, unknown, ambiguous, and canceled selections are
|
||||
operational inspection results from PromptKit and retain its error identity.
|
||||
|
||||
## Outcome status
|
||||
|
||||
CommandKit usage failures use status `2`; operational failures use status `1`.
|
||||
|
||||
@@ -6,8 +6,8 @@ declarations, an instance-owned registry, a Cobra-backed routing and parsing
|
||||
adapter, public `App.Execute` coordination, and generated help/version commands.
|
||||
Configuration suite registration, scope discovery, ConfigKit integration,
|
||||
validation, native effective YAML display, source inventory, and path
|
||||
explanation reports, and PromptKit engine configuration inspection are
|
||||
implemented; named prompt/profile inspection remains pending. The
|
||||
explanation reports, PromptKit engine configuration inspection, and optional
|
||||
PromptKit prompt/profile inspection commands are implemented. The
|
||||
[README](../README.md) provides project orientation; the
|
||||
[documentation policy](policy/documentation.md#canonical-ownership) assigns
|
||||
canonical documentation owners.
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Initial CommandKit implementation plan
|
||||
|
||||
Status: implementation underway; Stages 1-12 complete and Stages 13-14 pending.
|
||||
Status: implementation underway; Stages 1-13 complete and Stage 14 pending.
|
||||
|
||||
Date: 2026-10-07.
|
||||
|
||||
@@ -750,7 +750,7 @@ PromptKit/ConfigKit error identities. `go test ./...` passes.
|
||||
|
||||
## Stage 13: implement named prompt/profile inspection
|
||||
|
||||
**Status:** pending. **Prerequisite:** Stage 12.
|
||||
**Status:** complete. **Prerequisite:** Stage 12.
|
||||
|
||||
```go
|
||||
type PromptKitSuite[C any] struct {
|
||||
@@ -811,6 +811,11 @@ formats, nil/empty parameters, independently installed prompt/profile capability
|
||||
definition-only registration, and help without engine construction. Assert no
|
||||
handler enters generation rather than exercising paid/network execution paths.
|
||||
|
||||
**Completion evidence:** `promptkit_test.go` exercises real captured prompt and
|
||||
profile definitions, both report formats, inheritance and provider parameters,
|
||||
selection errors, optional capabilities, and help without engine construction.
|
||||
`go test ./...` passes.
|
||||
|
||||
## Stage 14: substantiate consumer fit and complete documentation
|
||||
|
||||
**Status:** pending. **Prerequisite:** Stage 13.
|
||||
|
||||
463
promptkit.go
Normal file
463
promptkit.go
Normal file
@@ -0,0 +1,463 @@
|
||||
package commandkit
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
"gitea.maximumdirect.net/eric/promptkit"
|
||||
)
|
||||
|
||||
// PromptKitSuite declares optional PromptKit prompt and profile inspection
|
||||
// commands for an App.
|
||||
type PromptKitSuite[C any] struct {
|
||||
// NewContext creates command-local application context when an inspection runs.
|
||||
NewContext func() C
|
||||
// Selectors declares context flags accepted by the installed commands. Each
|
||||
// explicit scope list must include "promptkit"; an empty list applies here.
|
||||
Selectors []Selector[C]
|
||||
// CheckInput validates standard configuration selections and selector values.
|
||||
CheckInput func(Selection, C) error
|
||||
// InspectPrompt inspects an ID and exact or omitted version.
|
||||
InspectPrompt func(context.Context, Selection, C, string, string) (Acquired[*promptkit.PromptInspection], error)
|
||||
// InspectProfile inspects one profile ID.
|
||||
InspectProfile func(context.Context, Selection, C, string) (Acquired[*promptkit.ProfileInspection], error)
|
||||
}
|
||||
|
||||
// NewPromptKitSuite creates prompt and profile inspection callbacks backed by
|
||||
// a PromptKit Engine loader. The loader runs once per selected command. Each
|
||||
// callback captures configuration diagnostics before calling the corresponding
|
||||
// PromptKit inspection method, and returns those diagnostics with lookup errors.
|
||||
func NewPromptKitSuite[C any](
|
||||
load func(context.Context, Selection, C) (*promptkit.Engine, error),
|
||||
) (PromptKitSuite[C], error) {
|
||||
if load == nil {
|
||||
return PromptKitSuite[C]{}, fmt.Errorf("PromptKit engine loader is nil")
|
||||
}
|
||||
return PromptKitSuite[C]{
|
||||
InspectPrompt: func(ctx context.Context, selection Selection, appContext C, id, version string) (Acquired[*promptkit.PromptInspection], error) {
|
||||
engine, _, diagnostics, err := loadPromptKitInspection(ctx, selection, appContext, load)
|
||||
if err != nil {
|
||||
return Acquired[*promptkit.PromptInspection]{Diagnostics: diagnostics}, err
|
||||
}
|
||||
inspection, err := engine.InspectPrompt(ctx, id, version)
|
||||
return Acquired[*promptkit.PromptInspection]{Value: inspection, Diagnostics: diagnostics}, err
|
||||
},
|
||||
InspectProfile: func(ctx context.Context, selection Selection, appContext C, id string) (Acquired[*promptkit.ProfileInspection], error) {
|
||||
engine, _, diagnostics, err := loadPromptKitInspection(ctx, selection, appContext, load)
|
||||
if err != nil {
|
||||
return Acquired[*promptkit.ProfileInspection]{Diagnostics: diagnostics}, err
|
||||
}
|
||||
inspection, err := engine.InspectProfile(ctx, id)
|
||||
return Acquired[*promptkit.ProfileInspection]{Value: inspection, Diagnostics: diagnostics}, err
|
||||
},
|
||||
}, nil
|
||||
}
|
||||
|
||||
func loadPromptKitInspection[C any](ctx context.Context, selection Selection, appContext C, load func(context.Context, Selection, C) (*promptkit.Engine, error)) (*promptkit.Engine, promptkit.ConfigurationInspection, []Diagnostic, error) {
|
||||
engine, err := load(ctx, selection, appContext)
|
||||
if err != nil {
|
||||
var loadErr *configkit.LoadError
|
||||
if errors.As(err, &loadErr) {
|
||||
return nil, promptkit.ConfigurationInspection{}, ConfigKitDiagnostics("promptkit", loadErr.Sources(), loadErr.Diagnostics()), err
|
||||
}
|
||||
return nil, promptkit.ConfigurationInspection{}, nil, err
|
||||
}
|
||||
if engine == nil {
|
||||
return nil, promptkit.ConfigurationInspection{}, nil, fmt.Errorf("PromptKit configuration loader returned a nil engine")
|
||||
}
|
||||
inspection, err := engine.InspectConfiguration()
|
||||
if err != nil {
|
||||
return nil, promptkit.ConfigurationInspection{}, nil, err
|
||||
}
|
||||
sources := inspection.Sources()
|
||||
diagnostics := ConfigKitDiagnostics("promptkit", sources, inspection.Diagnostics())
|
||||
return engine, inspection, diagnostics, nil
|
||||
}
|
||||
|
||||
// RegisterPromptKit installs the supplied PromptKit inspection commands under
|
||||
// promptkit show. Each nonnil callback installs only its matching command. A
|
||||
// suite with no inspection callbacks installs no PromptKit namespace.
|
||||
func RegisterPromptKit[C any](app *App, suite PromptKitSuite[C]) error {
|
||||
if suite.InspectPrompt == nil && suite.InspectProfile == nil {
|
||||
return nil
|
||||
}
|
||||
if app == nil {
|
||||
return fmt.Errorf("cannot register PromptKit commands on a nil app")
|
||||
}
|
||||
normalized, err := normalizePromptKitSuite(suite)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
showCommands := make([]Command, 0, 2)
|
||||
if normalized.InspectPrompt != nil {
|
||||
command, err := newPromptKitPromptCommand(normalized)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
showCommands = append(showCommands, command)
|
||||
}
|
||||
if normalized.InspectProfile != nil {
|
||||
command, err := newPromptKitProfileCommand(normalized)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
showCommands = append(showCommands, command)
|
||||
}
|
||||
show, err := Group("show", "Inspect PromptKit definitions.", showCommands...)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
root, err := Group("promptkit", "Inspect PromptKit definitions.", show)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return app.Register(root)
|
||||
}
|
||||
|
||||
func normalizePromptKitSuite[C any](suite PromptKitSuite[C]) (PromptKitSuite[C], error) {
|
||||
normalized := suite
|
||||
normalized.Selectors = append([]Selector[C](nil), suite.Selectors...)
|
||||
seenNames := make(map[string]struct{}, len(normalized.Selectors))
|
||||
seenShorts := make(map[string]struct{}, len(normalized.Selectors))
|
||||
for i, selector := range normalized.Selectors {
|
||||
binding := selector.Binding.binding
|
||||
meta := binding.meta
|
||||
if !binding.valid || binding.clear == nil || meta.Name == "" {
|
||||
return PromptKitSuite[C]{}, fmt.Errorf("PromptKit selector %d has an invalid flag binding", i)
|
||||
}
|
||||
if !validFlagName(meta.Name) {
|
||||
return PromptKitSuite[C]{}, fmt.Errorf("PromptKit selector flag name %q is invalid", meta.Name)
|
||||
}
|
||||
switch meta.Name {
|
||||
case "config", "overlay", "scope", "format", "version", "help":
|
||||
return PromptKitSuite[C]{}, fmt.Errorf("PromptKit selector flag --%s conflicts with a standard flag", meta.Name)
|
||||
}
|
||||
if meta.Short == "h" {
|
||||
return PromptKitSuite[C]{}, fmt.Errorf("PromptKit selector short flag -%s conflicts with help", meta.Short)
|
||||
}
|
||||
if meta.Short != "" && (len(meta.Short) != 1 || meta.Short == "-") {
|
||||
return PromptKitSuite[C]{}, fmt.Errorf("PromptKit selector short spelling %q must be one byte", meta.Short)
|
||||
}
|
||||
if _, exists := seenNames[meta.Name]; exists {
|
||||
return PromptKitSuite[C]{}, fmt.Errorf("duplicate PromptKit selector flag --%s", meta.Name)
|
||||
}
|
||||
seenNames[meta.Name] = struct{}{}
|
||||
if meta.Short != "" {
|
||||
if _, exists := seenShorts[meta.Short]; exists {
|
||||
return PromptKitSuite[C]{}, fmt.Errorf("duplicate PromptKit selector short flag -%s", meta.Short)
|
||||
}
|
||||
seenShorts[meta.Short] = struct{}{}
|
||||
}
|
||||
applies := len(selector.Scopes) == 0
|
||||
for _, scope := range selector.Scopes {
|
||||
if scope == "promptkit" {
|
||||
applies = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !applies {
|
||||
return PromptKitSuite[C]{}, fmt.Errorf("PromptKit selector --%s must apply to scope %q", meta.Name, "promptkit")
|
||||
}
|
||||
normalized.Selectors[i].Scopes = append([]string(nil), selector.Scopes...)
|
||||
}
|
||||
return normalized, nil
|
||||
}
|
||||
|
||||
type promptKitSelectorState[C any] struct {
|
||||
scopes []string
|
||||
apply func(*C)
|
||||
clear func(*C)
|
||||
}
|
||||
|
||||
type promptKitInput[C any] struct {
|
||||
Selection Selection
|
||||
Format string
|
||||
ID string
|
||||
Version string
|
||||
SelectorContext C
|
||||
selectors []promptKitSelectorState[C]
|
||||
}
|
||||
|
||||
func promptKitInputFlags[C any](suite PromptKitSuite[C], prompt bool) []Flag[promptKitInput[C]] {
|
||||
formatDefault := "text"
|
||||
flags := []Flag[promptKitInput[C]]{
|
||||
ValueFlag[promptKitInput[C]](FlagMeta{Name: "format", Usage: "Output format."}, &formatDefault, parseString, func(input *promptKitInput[C], value Supplied[string]) {
|
||||
input.Format = value.Value
|
||||
}),
|
||||
ValueFlag[promptKitInput[C]](FlagMeta{Name: "config", Usage: "Configuration source."}, nil, parseString, func(input *promptKitInput[C], value Supplied[string]) {
|
||||
input.Selection.Config = value
|
||||
}),
|
||||
ValueFlag[promptKitInput[C]](FlagMeta{Name: "overlay", Usage: "Overlay source."}, nil, parseString, func(input *promptKitInput[C], value Supplied[string]) {
|
||||
input.Selection.Overlay = value
|
||||
}),
|
||||
}
|
||||
if prompt {
|
||||
versionDefault := ""
|
||||
flags = append(flags, ValueFlag[promptKitInput[C]](FlagMeta{Name: "version", Usage: "Exact prompt version; empty selects the owning default."}, &versionDefault, parseString, func(input *promptKitInput[C], value Supplied[string]) {
|
||||
input.Version = value.Value
|
||||
}))
|
||||
}
|
||||
for _, selector := range suite.Selectors {
|
||||
flags = append(flags, adaptPromptKitSelector(selector))
|
||||
}
|
||||
return flags
|
||||
}
|
||||
|
||||
func adaptPromptKitSelector[C any](selector Selector[C]) Flag[promptKitInput[C]] {
|
||||
binding := selector.Binding.binding
|
||||
meta := binding.meta
|
||||
return Flag[promptKitInput[C]]{binding: flagBinding[promptKitInput[C]]{
|
||||
meta: meta,
|
||||
defaultText: binding.defaultText,
|
||||
repeated: binding.repeated,
|
||||
boolean: binding.boolean,
|
||||
valid: binding.valid,
|
||||
prepare: func(values []string) (func(*promptKitInput[C]), error) {
|
||||
apply, err := binding.prepare(values)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
state := promptKitSelectorState[C]{scopes: append([]string(nil), selector.Scopes...), apply: apply, clear: binding.clear}
|
||||
return func(input *promptKitInput[C]) {
|
||||
apply(&input.SelectorContext)
|
||||
input.selectors = append(input.selectors, state)
|
||||
}, nil
|
||||
},
|
||||
}}
|
||||
}
|
||||
|
||||
func checkPromptKitInput[C any](suite PromptKitSuite[C], input promptKitInput[C]) error {
|
||||
if _, err := parseConfigFormat(input.Format); err != nil {
|
||||
return err
|
||||
}
|
||||
if suite.CheckInput != nil {
|
||||
return suite.CheckInput(input.Selection, input.SelectorContext)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func promptKitContext[C any](suite PromptKitSuite[C], input promptKitInput[C]) C {
|
||||
var result C
|
||||
if suite.NewContext != nil {
|
||||
result = suite.NewContext()
|
||||
}
|
||||
for _, selector := range input.selectors {
|
||||
if selectorApplies(selector.scopes, "promptkit") {
|
||||
selector.apply(&result)
|
||||
} else {
|
||||
selector.clear(&result)
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func newPromptKitPromptCommand[C any](suite PromptKitSuite[C]) (Command, error) {
|
||||
inputSpec := CommandSpec[promptKitInput[C]]{
|
||||
Name: "prompt",
|
||||
Summary: "Inspect one PromptKit prompt definition.",
|
||||
Flags: promptKitInputFlags(suite, true),
|
||||
Arguments: []Argument[promptKitInput[C]]{ValueArgument[promptKitInput[C]](ArgumentMeta{Name: "ID"}, true, parseString, func(input *promptKitInput[C], value Supplied[string]) {
|
||||
input.ID = value.Value
|
||||
})},
|
||||
CheckInput: func(input promptKitInput[C]) error { return checkPromptKitInput(suite, input) },
|
||||
Run: func(invocation Invocation, input promptKitInput[C]) Outcome {
|
||||
return runPromptKitPrompt(invocation, suite, input)
|
||||
},
|
||||
}
|
||||
return NewCommand(inputSpec)
|
||||
}
|
||||
|
||||
func newPromptKitProfileCommand[C any](suite PromptKitSuite[C]) (Command, error) {
|
||||
inputSpec := CommandSpec[promptKitInput[C]]{
|
||||
Name: "profile",
|
||||
Summary: "Inspect one PromptKit profile definition.",
|
||||
Flags: promptKitInputFlags(suite, false),
|
||||
Arguments: []Argument[promptKitInput[C]]{ValueArgument[promptKitInput[C]](ArgumentMeta{Name: "ID"}, true, parseString, func(input *promptKitInput[C], value Supplied[string]) {
|
||||
input.ID = value.Value
|
||||
})},
|
||||
CheckInput: func(input promptKitInput[C]) error { return checkPromptKitInput(suite, input) },
|
||||
Run: func(invocation Invocation, input promptKitInput[C]) Outcome {
|
||||
return runPromptKitProfile(invocation, suite, input)
|
||||
},
|
||||
}
|
||||
return NewCommand(inputSpec)
|
||||
}
|
||||
|
||||
func runPromptKitPrompt[C any](invocation Invocation, suite PromptKitSuite[C], input promptKitInput[C]) Outcome {
|
||||
contextValue := promptKitContext(suite, input)
|
||||
acquired, err := suite.InspectPrompt(invocation.Context, input.Selection, contextValue, input.ID, input.Version)
|
||||
if err != nil {
|
||||
return promptKitFailure(err, acquired.Diagnostics)
|
||||
}
|
||||
if acquired.Value == nil {
|
||||
return promptKitFailure(fmt.Errorf("PromptKit prompt inspection returned no result"), acquired.Diagnostics)
|
||||
}
|
||||
report := promptReportFromInspection(acquired.Value)
|
||||
return writePromptKitReport(invocation, input.Format, report, formatPromptText(report), acquired.Diagnostics)
|
||||
}
|
||||
|
||||
func runPromptKitProfile[C any](invocation Invocation, suite PromptKitSuite[C], input promptKitInput[C]) Outcome {
|
||||
contextValue := promptKitContext(suite, input)
|
||||
acquired, err := suite.InspectProfile(invocation.Context, input.Selection, contextValue, input.ID)
|
||||
if err != nil {
|
||||
return promptKitFailure(err, acquired.Diagnostics)
|
||||
}
|
||||
if acquired.Value == nil {
|
||||
return promptKitFailure(fmt.Errorf("PromptKit profile inspection returned no result"), acquired.Diagnostics)
|
||||
}
|
||||
report := profileReportFromInspection(acquired.Value)
|
||||
textReport, err := formatProfileText(report)
|
||||
if err != nil {
|
||||
return promptKitFailure(err, acquired.Diagnostics)
|
||||
}
|
||||
return writePromptKitReport(invocation, input.Format, report, textReport, acquired.Diagnostics)
|
||||
}
|
||||
|
||||
func promptKitFailure(err error, diagnostics []Diagnostic) Outcome {
|
||||
outcome := Failure(err)
|
||||
outcome.Diagnostics = diagnostics
|
||||
return outcome
|
||||
}
|
||||
|
||||
func writePromptKitReport(invocation Invocation, format string, report any, textReport string, diagnostics []Diagnostic) Outcome {
|
||||
var output strings.Builder
|
||||
if format == "yaml" {
|
||||
if err := encodeYAML(&output, report); err != nil {
|
||||
return promptKitFailure(err, diagnostics)
|
||||
}
|
||||
} else if _, err := io.WriteString(&output, textReport); err != nil {
|
||||
return promptKitFailure(err, diagnostics)
|
||||
}
|
||||
if err := writeReport(invocation.Streams.Out, []byte(output.String())); err != nil {
|
||||
outcome := incompleteReport(err)
|
||||
outcome.Diagnostics = diagnostics
|
||||
return outcome
|
||||
}
|
||||
outcome := Success()
|
||||
outcome.Diagnostics = diagnostics
|
||||
return outcome
|
||||
}
|
||||
|
||||
type promptInputReport struct {
|
||||
Name string `yaml:"name"`
|
||||
Required bool `yaml:"required"`
|
||||
ContentType string `yaml:"content_type"`
|
||||
Description string `yaml:"description"`
|
||||
}
|
||||
|
||||
type promptOutputContractReport struct {
|
||||
Format string `yaml:"format"`
|
||||
ValidationMode string `yaml:"validation_mode"`
|
||||
SchemaPath string `yaml:"schema_path"`
|
||||
RepairAttempts int `yaml:"repair_attempts"`
|
||||
}
|
||||
|
||||
type promptInspectionReport struct {
|
||||
PromptID string `yaml:"prompt_id"`
|
||||
PromptVersion string `yaml:"prompt_version"`
|
||||
PromptHash string `yaml:"prompt_hash"`
|
||||
DefaultProfileID string `yaml:"default_profile_id"`
|
||||
Inputs []promptInputReport `yaml:"inputs"`
|
||||
OutputContract promptOutputContractReport `yaml:"output_contract"`
|
||||
}
|
||||
|
||||
func promptReportFromInspection(inspection *promptkit.PromptInspection) promptInspectionReport {
|
||||
report := promptInspectionReport{
|
||||
PromptID: inspection.PromptID,
|
||||
PromptVersion: inspection.PromptVersion,
|
||||
PromptHash: inspection.PromptHash,
|
||||
DefaultProfileID: inspection.DefaultProfileID,
|
||||
Inputs: make([]promptInputReport, 0, len(inspection.Inputs)),
|
||||
OutputContract: promptOutputContractReport{
|
||||
Format: string(inspection.OutputContract.Format),
|
||||
ValidationMode: string(inspection.OutputContract.ValidationMode),
|
||||
SchemaPath: inspection.OutputContract.SchemaPath,
|
||||
RepairAttempts: inspection.OutputContract.RepairAttempts,
|
||||
},
|
||||
}
|
||||
for _, input := range inspection.Inputs {
|
||||
report.Inputs = append(report.Inputs, promptInputReport{Name: input.Name, Required: input.Required, ContentType: input.ContentType, Description: input.Description})
|
||||
}
|
||||
return report
|
||||
}
|
||||
|
||||
type modelParamsReport struct {
|
||||
BackendID string `yaml:"backend_id"`
|
||||
Endpoint string `yaml:"endpoint"`
|
||||
Model string `yaml:"model"`
|
||||
Temperature float64 `yaml:"temperature"`
|
||||
MaxTokens int `yaml:"max_tokens"`
|
||||
TopP float64 `yaml:"top_p"`
|
||||
TimeoutSeconds int `yaml:"timeout_seconds"`
|
||||
ServiceTier string `yaml:"service_tier"`
|
||||
ReasoningEffort string `yaml:"reasoning_effort"`
|
||||
APIKeyEnv string `yaml:"api_key_env"`
|
||||
ExtraParams map[string]any `yaml:"extra_params"`
|
||||
}
|
||||
|
||||
type profileInspectionReport struct {
|
||||
ProfileID string `yaml:"profile_id"`
|
||||
EffectiveModelParams modelParamsReport `yaml:"effective_model_params"`
|
||||
APIKeyRequired bool `yaml:"api_key_required"`
|
||||
}
|
||||
|
||||
func profileReportFromInspection(inspection *promptkit.ProfileInspection) profileInspectionReport {
|
||||
params := inspection.EffectiveModelParams
|
||||
extra := params.ExtraParams
|
||||
if extra == nil {
|
||||
extra = map[string]any{}
|
||||
}
|
||||
return profileInspectionReport{
|
||||
ProfileID: inspection.ProfileID,
|
||||
EffectiveModelParams: modelParamsReport{
|
||||
BackendID: params.BackendID, Endpoint: params.Endpoint, Model: params.Model,
|
||||
Temperature: params.Temperature, MaxTokens: params.MaxTokens, TopP: params.TopP,
|
||||
TimeoutSeconds: params.TimeoutSeconds, ServiceTier: params.ServiceTier,
|
||||
ReasoningEffort: params.ReasoningEffort, APIKeyEnv: params.APIKeyEnv, ExtraParams: extra,
|
||||
},
|
||||
APIKeyRequired: inspection.APIKeyRequired,
|
||||
}
|
||||
}
|
||||
|
||||
func formatPromptText(report promptInspectionReport) string {
|
||||
var output strings.Builder
|
||||
fmt.Fprintf(&output, "Prompt ID: %s\n", strconv.Quote(report.PromptID))
|
||||
fmt.Fprintf(&output, "Prompt version: %s\n", strconv.Quote(report.PromptVersion))
|
||||
fmt.Fprintf(&output, "Prompt hash: %s\n", strconv.Quote(report.PromptHash))
|
||||
fmt.Fprintf(&output, "Default profile ID: %s\n", strconv.Quote(report.DefaultProfileID))
|
||||
output.WriteString("Inputs:\n")
|
||||
for _, input := range report.Inputs {
|
||||
fmt.Fprintf(&output, " - name: %s\n required: %t\n content_type: %s\n description: %s\n", strconv.Quote(input.Name), input.Required, strconv.Quote(input.ContentType), strconv.Quote(input.Description))
|
||||
}
|
||||
if len(report.Inputs) == 0 {
|
||||
output.WriteString(" []\n")
|
||||
}
|
||||
fmt.Fprintf(&output, "Output contract:\n format: %s\n validation_mode: %s\n schema_path: %s\n repair_attempts: %d\n", strconv.Quote(report.OutputContract.Format), strconv.Quote(report.OutputContract.ValidationMode), strconv.Quote(report.OutputContract.SchemaPath), report.OutputContract.RepairAttempts)
|
||||
return output.String()
|
||||
}
|
||||
|
||||
func formatProfileText(report profileInspectionReport) (string, error) {
|
||||
params := report.EffectiveModelParams
|
||||
var output strings.Builder
|
||||
fmt.Fprintf(&output, "Profile ID: %s\n", strconv.Quote(report.ProfileID))
|
||||
output.WriteString("Effective model params:\n")
|
||||
fmt.Fprintf(&output, " backend_id: %s\n endpoint: %s\n model: %s\n temperature: %s\n max_tokens: %d\n top_p: %s\n timeout_seconds: %d\n service_tier: %s\n reasoning_effort: %s\n api_key_env: %s\n", strconv.Quote(params.BackendID), strconv.Quote(params.Endpoint), strconv.Quote(params.Model), strconv.FormatFloat(params.Temperature, 'g', -1, 64), params.MaxTokens, strconv.FormatFloat(params.TopP, 'g', -1, 64), params.TimeoutSeconds, strconv.Quote(params.ServiceTier), strconv.Quote(params.ReasoningEffort), strconv.Quote(params.APIKeyEnv))
|
||||
output.WriteString(" extra_params:\n")
|
||||
var nested strings.Builder
|
||||
if err := encodeYAML(&nested, params.ExtraParams); err != nil {
|
||||
return "", err
|
||||
}
|
||||
for _, line := range strings.Split(strings.TrimSuffix(nested.String(), "\n"), "\n") {
|
||||
output.WriteString(" ")
|
||||
output.WriteString(line)
|
||||
output.WriteByte('\n')
|
||||
}
|
||||
fmt.Fprintf(&output, "API key required: %t\n", report.APIKeyRequired)
|
||||
return output.String(), nil
|
||||
}
|
||||
@@ -2,10 +2,8 @@ package commandkit
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
"gitea.maximumdirect.net/eric/promptkit"
|
||||
)
|
||||
|
||||
@@ -23,24 +21,10 @@ func NewPromptKitScope[C any](
|
||||
Summary: "PromptKit engine configuration.",
|
||||
Operations: []ConfigOperation{ConfigValidate, ConfigShow, ConfigSources, ConfigExplain},
|
||||
Acquire: func(ctx context.Context, request ConfigRequest, appContext C) (Acquired[ConfigView], error) {
|
||||
engine, err := load(ctx, request.Selection, appContext)
|
||||
_, inspection, diagnostics, err := loadPromptKitInspection(ctx, request.Selection, appContext, load)
|
||||
if err != nil {
|
||||
var loadErr *configkit.LoadError
|
||||
if errors.As(err, &loadErr) {
|
||||
return Acquired[ConfigView]{Diagnostics: ConfigKitDiagnostics("promptkit", loadErr.Sources(), loadErr.Diagnostics())}, err
|
||||
}
|
||||
return Acquired[ConfigView]{}, err
|
||||
return Acquired[ConfigView]{Diagnostics: diagnostics}, err
|
||||
}
|
||||
if engine == nil {
|
||||
return Acquired[ConfigView]{}, fmt.Errorf("PromptKit configuration loader returned a nil engine")
|
||||
}
|
||||
|
||||
inspection, err := engine.InspectConfiguration()
|
||||
if err != nil {
|
||||
return Acquired[ConfigView]{}, err
|
||||
}
|
||||
sources := inspection.Sources()
|
||||
diagnostics := ConfigKitDiagnostics("promptkit", sources, inspection.Diagnostics())
|
||||
return Acquired[ConfigView]{
|
||||
Value: ConfigView{
|
||||
Effective: inspection,
|
||||
|
||||
378
promptkit_test.go
Normal file
378
promptkit_test.go
Normal file
@@ -0,0 +1,378 @@
|
||||
package commandkit_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
|
||||
commandkit "gitea.maximumdirect.net/eric/CommandKit"
|
||||
configkit "gitea.maximumdirect.net/eric/ConfigKit"
|
||||
"gitea.maximumdirect.net/eric/promptkit"
|
||||
"go.yaml.in/yaml/v4"
|
||||
)
|
||||
|
||||
const promptKitInspectionFixture = `prompts:
|
||||
"team/summary":
|
||||
"2":
|
||||
default_profile: child
|
||||
inputs:
|
||||
- name: topic
|
||||
required: true
|
||||
content_type: text/plain
|
||||
description: "Topic to summarize"
|
||||
- name: audience
|
||||
required: false
|
||||
content_type: text/plain
|
||||
description: ""
|
||||
messages:
|
||||
- role: user
|
||||
content: "Summarize {{.topic}}"
|
||||
output:
|
||||
format: json
|
||||
validation_mode: basic
|
||||
repair_attempts: 2
|
||||
"3":
|
||||
messages:
|
||||
- role: user
|
||||
content: "Version three"
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
empty:
|
||||
"1":
|
||||
messages:
|
||||
- role: user
|
||||
content: "No inputs"
|
||||
output:
|
||||
format: text
|
||||
validation_mode: none
|
||||
`
|
||||
|
||||
func promptKitInspectionEngine(t *testing.T) *promptkit.Engine {
|
||||
t.Helper()
|
||||
inputs, err := promptkit.FilesystemInputs("prompt fixture", fstest.MapFS{
|
||||
"config/prompts.yml": &fstest.MapFile{Data: []byte(promptKitInspectionFixture)},
|
||||
}, "config")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
engine, err := promptkit.NewEngine(promptkit.Config{Sources: promptkit.ConfigurationSources{
|
||||
Underlays: []configkit.Underlay{
|
||||
{Inputs: inputs},
|
||||
{Inputs: []configkit.Input{{Source: configkit.Mapping("profile fixture", map[string]any{
|
||||
"backends": map[string]any{"local": map[string]any{"endpoint": "https://example.invalid/v1"}},
|
||||
"profiles": map[string]any{
|
||||
"parent": map[string]any{
|
||||
"backend": "local", "model": "base-model", "temperature": 0.7,
|
||||
"max_tokens": 100, "top_p": 0.8,
|
||||
"extra_params": map[string]any{
|
||||
"token": "provider-private-sentinel",
|
||||
"nested": map[string]any{"null_value": nil, "visible": "kept"},
|
||||
"sequence": []any{"first", nil, 3},
|
||||
},
|
||||
},
|
||||
"child": map[string]any{"extends": "parent", "temperature": 0, "max_tokens": 0, "api_key_env": "PROMPTKIT_TEST_KEY"},
|
||||
"requires-key": map[string]any{"backend": "local", "model": "key-model", "api_key_required": true},
|
||||
"no-extra": map[string]any{"backend": "local", "model": "empty-params"},
|
||||
},
|
||||
})}}},
|
||||
},
|
||||
}})
|
||||
if err != nil {
|
||||
t.Fatalf("construct PromptKit fixture engine: %v", err)
|
||||
}
|
||||
return engine
|
||||
}
|
||||
|
||||
func TestPromptKitInspectionCommandsReportRealCapturedDefinitions(t *testing.T) {
|
||||
t.Setenv("PROMPTKIT_TEST_KEY", "credential-private-sentinel")
|
||||
engine := promptKitInspectionEngine(t)
|
||||
loadCalls := 0
|
||||
suite, err := commandkit.NewPromptKitSuite(func(context.Context, commandkit.Selection, struct{}) (*promptkit.Engine, error) {
|
||||
loadCalls++
|
||||
return engine, nil
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
app, err := commandkit.New(commandkit.Identity{Name: "example"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := commandkit.RegisterPromptKit(app, suite); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
var promptYAML bytes.Buffer
|
||||
outcome := app.Execute(context.Background(), []string{"promptkit", "show", "prompt", "team/summary", "--version", "2", "--format", "yaml"}, commandkit.Streams{Out: &promptYAML, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeSuccess || loadCalls != 1 {
|
||||
t.Fatalf("prompt inspection outcome=%#v calls=%d report=%q", outcome, loadCalls, promptYAML.String())
|
||||
}
|
||||
var promptReport map[string]any
|
||||
if err := yaml.Unmarshal(promptYAML.Bytes(), &promptReport); err != nil {
|
||||
t.Fatalf("decode prompt YAML: %v", err)
|
||||
}
|
||||
if !reflect.DeepEqual(sortedKeys(promptReport), []string{"default_profile_id", "inputs", "output_contract", "prompt_hash", "prompt_id", "prompt_version"}) {
|
||||
t.Fatalf("prompt report keys = %v", sortedKeys(promptReport))
|
||||
}
|
||||
if promptReport["prompt_id"] != "team/summary" || promptReport["prompt_version"] != "2" || promptReport["default_profile_id"] != "child" || promptReport["prompt_hash"] == "" {
|
||||
t.Fatalf("prompt report identity = %#v", promptReport)
|
||||
}
|
||||
inputs := promptReport["inputs"].([]any)
|
||||
if len(inputs) != 2 || inputs[0].(map[string]any)["name"] != "topic" || inputs[1].(map[string]any)["name"] != "audience" {
|
||||
t.Fatalf("prompt inputs/order = %#v", inputs)
|
||||
}
|
||||
outputContract := promptReport["output_contract"].(map[string]any)
|
||||
if outputContract["format"] != "json" || outputContract["validation_mode"] != "basic" || outputContract["repair_attempts"] != 2 {
|
||||
t.Fatalf("output contract = %#v", outputContract)
|
||||
}
|
||||
|
||||
var promptText bytes.Buffer
|
||||
outcome = app.Execute(context.Background(), []string{"promptkit", "show", "prompt", "team/summary", "--version", "2"}, commandkit.Streams{Out: &promptText, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeSuccess || !strings.Contains(promptText.String(), `Prompt hash: "`) || !strings.Contains(promptText.String(), `- name: "topic"`) || !strings.Contains(promptText.String(), `validation_mode: "basic"`) {
|
||||
t.Fatalf("prompt text = (%#v, %q)", outcome, promptText.String())
|
||||
}
|
||||
var omittedVersion bytes.Buffer
|
||||
outcome = app.Execute(context.Background(), []string{"promptkit", "show", "prompt", "empty", "--format", "yaml"}, commandkit.Streams{Out: &omittedVersion, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeSuccess || !strings.Contains(omittedVersion.String(), "prompt_version: \"1\"") {
|
||||
t.Fatalf("omitted singleton prompt version = (%#v, %q)", outcome, omittedVersion.String())
|
||||
}
|
||||
|
||||
var profileYAML bytes.Buffer
|
||||
outcome = app.Execute(context.Background(), []string{"promptkit", "show", "profile", "child", "--format", "yaml"}, commandkit.Streams{Out: &profileYAML, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeSuccess {
|
||||
t.Fatalf("profile inspection outcome=%#v report=%q", outcome, profileYAML.String())
|
||||
}
|
||||
var profileReport map[string]any
|
||||
if err := yaml.Unmarshal(profileYAML.Bytes(), &profileReport); err != nil {
|
||||
t.Fatalf("decode profile YAML: %v", err)
|
||||
}
|
||||
if !reflect.DeepEqual(sortedKeys(profileReport), []string{"api_key_required", "effective_model_params", "profile_id"}) || profileReport["profile_id"] != "child" || profileReport["api_key_required"] != false {
|
||||
t.Fatalf("profile report = %#v", profileReport)
|
||||
}
|
||||
params := profileReport["effective_model_params"].(map[string]any)
|
||||
if !reflect.DeepEqual(sortedKeys(params), []string{"api_key_env", "backend_id", "endpoint", "extra_params", "max_tokens", "model", "reasoning_effort", "service_tier", "temperature", "timeout_seconds", "top_p"}) {
|
||||
t.Fatalf("model parameter keys = %v", sortedKeys(params))
|
||||
}
|
||||
if params["temperature"] != 0 || params["max_tokens"] != 0 || params["api_key_env"] != "PROMPTKIT_TEST_KEY" {
|
||||
t.Fatalf("effective owning values = %#v", params)
|
||||
}
|
||||
extra := params["extra_params"].(map[string]any)
|
||||
nested := extra["nested"].(map[string]any)
|
||||
sequence := extra["sequence"].([]any)
|
||||
if nested["null_value"] != nil || nested["visible"] != "kept" || len(sequence) != 3 || sequence[1] != nil || sequence[2] != 3 {
|
||||
t.Fatalf("nested provider parameters changed: %#v", extra)
|
||||
}
|
||||
if extra["token"] != "provider-private-sentinel" || strings.Contains(profileYAML.String(), "credential-private-sentinel") {
|
||||
t.Fatalf("provider parameters or credential reference changed: %q", profileYAML.String())
|
||||
}
|
||||
|
||||
var profileText bytes.Buffer
|
||||
outcome = app.Execute(context.Background(), []string{"promptkit", "show", "profile", "child"}, commandkit.Streams{Out: &profileText, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeSuccess || !strings.Contains(profileText.String(), "temperature: 0\n") || !strings.Contains(profileText.String(), "api_key_env: \"PROMPTKIT_TEST_KEY\"") || !strings.Contains(profileText.String(), "null_value: null") {
|
||||
t.Fatalf("profile text = (%#v, %q)", outcome, profileText.String())
|
||||
}
|
||||
|
||||
var requiredKey bytes.Buffer
|
||||
outcome = app.Execute(context.Background(), []string{"promptkit", "show", "profile", "requires-key", "--format", "yaml"}, commandkit.Streams{Out: &requiredKey, Err: io.Discard})
|
||||
var requiredReport map[string]any
|
||||
if outcome.Kind != commandkit.OutcomeSuccess || yaml.Unmarshal(requiredKey.Bytes(), &requiredReport) != nil || requiredReport["api_key_required"] != true {
|
||||
t.Fatalf("credential requirement report = (%#v, %q)", outcome, requiredKey.String())
|
||||
}
|
||||
|
||||
var emptyPrompt bytes.Buffer
|
||||
outcome = app.Execute(context.Background(), []string{"promptkit", "show", "prompt", "empty", "--version", "1", "--format", "yaml"}, commandkit.Streams{Out: &emptyPrompt, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeSuccess || !strings.Contains(emptyPrompt.String(), "inputs: []") {
|
||||
t.Fatalf("empty inputs report = (%#v, %q)", outcome, emptyPrompt.String())
|
||||
}
|
||||
var zeroExtra bytes.Buffer
|
||||
outcome = app.Execute(context.Background(), []string{"promptkit", "show", "profile", "no-extra", "--format", "yaml"}, commandkit.Streams{Out: &zeroExtra, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeSuccess || !strings.Contains(zeroExtra.String(), "extra_params: {}") {
|
||||
t.Fatalf("nil extra params report = (%#v, %q)", outcome, zeroExtra.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestPromptKitInspectionSelectionErrorsHelpAndOptionalCapabilities(t *testing.T) {
|
||||
engine := promptKitInspectionEngine(t)
|
||||
loaderCalls := 0
|
||||
suite, err := commandkit.NewPromptKitSuite(func(context.Context, commandkit.Selection, struct{}) (*promptkit.Engine, error) {
|
||||
loaderCalls++
|
||||
return engine, nil
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
app, _ := commandkit.New(commandkit.Identity{Name: "example"})
|
||||
if err := commandkit.RegisterPromptKit(app, suite); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
badTarget := suite
|
||||
badTarget.Selectors = []commandkit.Selector[struct{}]{{
|
||||
Binding: commandkit.ValueFlag[struct{}](commandkit.FlagMeta{Name: "campaign"}, nil, func(value string) (string, error) { return value, nil }, func(*struct{}, commandkit.Supplied[string]) {}),
|
||||
Scopes: []string{"app"},
|
||||
}}
|
||||
invalidApp, _ := commandkit.New(commandkit.Identity{Name: "invalid"})
|
||||
if err := commandkit.RegisterPromptKit(invalidApp, badTarget); err == nil {
|
||||
t.Fatal("selector for another scope was accepted")
|
||||
}
|
||||
for _, tc := range []struct {
|
||||
args []string
|
||||
want error
|
||||
}{
|
||||
{args: []string{"promptkit", "show", "prompt", "team/summary"}, want: promptkit.ErrPromptLoad},
|
||||
{args: []string{"promptkit", "show", "prompt", "missing", "--version", "1"}, want: promptkit.ErrPromptNotFound},
|
||||
{args: []string{"promptkit", "show", "profile", "missing"}, want: promptkit.ErrProfileNotFound},
|
||||
{args: []string{"promptkit", "show", "prompt", "", "--version", "2"}, want: promptkit.ErrInvalidRequest},
|
||||
} {
|
||||
outcome := app.Execute(context.Background(), tc.args, commandkit.Streams{Out: io.Discard, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeOperational || !errors.Is(outcome.Cause, tc.want) {
|
||||
t.Errorf("execute %q = (%#v), want operational error matching %v", tc.args, outcome, tc.want)
|
||||
}
|
||||
}
|
||||
cancelled, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
outcome := app.Execute(cancelled, []string{"promptkit", "show", "prompt", "empty", "--version", "1"}, commandkit.Streams{Out: io.Discard, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeOperational || !errors.Is(outcome.Cause, context.Canceled) {
|
||||
t.Fatalf("cancelled prompt inspection = %#v", outcome)
|
||||
}
|
||||
beforeHelp := loaderCalls
|
||||
outcome = app.Execute(context.Background(), []string{"promptkit", "show", "prompt", "--help"}, commandkit.Streams{Out: io.Discard, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeSuccess || loaderCalls != beforeHelp {
|
||||
t.Fatalf("help acquired engine: outcome=%#v calls=%d", outcome, loaderCalls)
|
||||
}
|
||||
beforeMissing := loaderCalls
|
||||
outcome = app.Execute(context.Background(), []string{"promptkit", "show", "prompt"}, commandkit.Streams{Out: io.Discard, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeUsage || loaderCalls != beforeMissing {
|
||||
t.Fatalf("missing ID = (%#v, calls=%d), want usage before acquisition", outcome, loaderCalls)
|
||||
}
|
||||
|
||||
profileOnlyApp, _ := commandkit.New(commandkit.Identity{Name: "profile-only"})
|
||||
if err := commandkit.RegisterPromptKit(profileOnlyApp, commandkit.PromptKitSuite[struct{}]{InspectProfile: func(ctx context.Context, _ commandkit.Selection, _ struct{}, id string) (commandkit.Acquired[*promptkit.ProfileInspection], error) {
|
||||
value, err := engine.InspectProfile(ctx, id)
|
||||
return commandkit.Acquired[*promptkit.ProfileInspection]{Value: value}, err
|
||||
}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
outcome = profileOnlyApp.Execute(context.Background(), []string{"promptkit", "show", "profile", "child"}, commandkit.Streams{Out: io.Discard, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeSuccess {
|
||||
t.Fatalf("profile-only capability was not installed: %#v", outcome)
|
||||
}
|
||||
outcome = profileOnlyApp.Execute(context.Background(), []string{"promptkit", "show", "prompt", "empty"}, commandkit.Streams{Out: io.Discard, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeUsage {
|
||||
t.Fatalf("profile-only suite installed prompt inspection: %#v", outcome)
|
||||
}
|
||||
promptOnlyApp, _ := commandkit.New(commandkit.Identity{Name: "prompt-only"})
|
||||
if err := commandkit.RegisterPromptKit(promptOnlyApp, commandkit.PromptKitSuite[struct{}]{InspectPrompt: func(ctx context.Context, _ commandkit.Selection, _ struct{}, id, version string) (commandkit.Acquired[*promptkit.PromptInspection], error) {
|
||||
value, err := engine.InspectPrompt(ctx, id, version)
|
||||
return commandkit.Acquired[*promptkit.PromptInspection]{Value: value}, err
|
||||
}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
outcome = promptOnlyApp.Execute(context.Background(), []string{"promptkit", "show", "prompt", "empty"}, commandkit.Streams{Out: io.Discard, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeSuccess {
|
||||
t.Fatalf("prompt-only capability was not installed: %#v", outcome)
|
||||
}
|
||||
outcome = promptOnlyApp.Execute(context.Background(), []string{"promptkit", "show", "profile", "child"}, commandkit.Streams{Out: io.Discard, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeUsage {
|
||||
t.Fatalf("prompt-only suite installed profile inspection: %#v", outcome)
|
||||
}
|
||||
|
||||
emptyApp, _ := commandkit.New(commandkit.Identity{Name: "empty"})
|
||||
if err := commandkit.RegisterPromptKit(emptyApp, commandkit.PromptKitSuite[struct{}]{}); err != nil {
|
||||
t.Fatalf("empty PromptKit suite = %v", err)
|
||||
}
|
||||
outcome = emptyApp.Execute(context.Background(), []string{"help"}, commandkit.Streams{Out: io.Discard, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeSuccess {
|
||||
t.Fatalf("empty PromptKit suite changed command family: %#v", outcome)
|
||||
}
|
||||
outcome = emptyApp.Execute(context.Background(), []string{"help", "promptkit"}, commandkit.Streams{Out: io.Discard, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeUsage {
|
||||
t.Fatalf("empty PromptKit suite installed its namespace: %#v", outcome)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPromptKitSuitePassesSelectionAndProjectedContext(t *testing.T) {
|
||||
type appContext struct {
|
||||
Campaign commandkit.Supplied[string]
|
||||
Runtime string
|
||||
}
|
||||
type contextKey struct{}
|
||||
engine := promptKitInspectionEngine(t)
|
||||
ctx := context.WithValue(context.Background(), contextKey{}, "caller")
|
||||
wantSelection := commandkit.Selection{
|
||||
Config: commandkit.Supplied[string]{Value: "main.yml", Set: true},
|
||||
Overlay: commandkit.Supplied[string]{Value: "dev", Set: true},
|
||||
}
|
||||
loaderCalls, contexts, checks := 0, 0, 0
|
||||
var gotSelection commandkit.Selection
|
||||
var gotContext appContext
|
||||
suite, err := commandkit.NewPromptKitSuite(func(gotCtx context.Context, selection commandkit.Selection, input appContext) (*promptkit.Engine, error) {
|
||||
loaderCalls++
|
||||
gotSelection, gotContext = selection, input
|
||||
if gotCtx != ctx || gotCtx.Value(contextKey{}) != "caller" {
|
||||
t.Fatal("engine loader did not receive invocation context")
|
||||
}
|
||||
return engine, nil
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
suite.NewContext = func() appContext { contexts++; return appContext{Runtime: "invocation cache"} }
|
||||
suite.Selectors = []commandkit.Selector[appContext]{{
|
||||
Binding: commandkit.ValueFlag[appContext](commandkit.FlagMeta{Name: "campaign"}, nil, func(value string) (string, error) { return value, nil }, func(input *appContext, value commandkit.Supplied[string]) { input.Campaign = value }),
|
||||
Scopes: []string{"promptkit"},
|
||||
}}
|
||||
suite.CheckInput = func(selection commandkit.Selection, input appContext) error {
|
||||
checks++
|
||||
if selection != wantSelection || input.Campaign != (commandkit.Supplied[string]{Value: "spring", Set: true}) {
|
||||
t.Errorf("preflight inputs = (%+v, %+v)", selection, input)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
app, _ := commandkit.New(commandkit.Identity{Name: "example"})
|
||||
if err := commandkit.RegisterPromptKit(app, suite); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
outcome := app.Execute(ctx, []string{"promptkit", "show", "profile", "child", "--config", "main.yml", "--overlay", "dev", "--campaign", "spring"}, commandkit.Streams{Out: io.Discard, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeSuccess || loaderCalls != 1 || contexts != 1 || checks != 1 {
|
||||
t.Fatalf("inspection = (%#v), loader/context/check calls = %d/%d/%d", outcome, loaderCalls, contexts, checks)
|
||||
}
|
||||
if gotSelection != wantSelection || gotContext.Campaign != (commandkit.Supplied[string]{Value: "spring", Set: true}) || gotContext.Runtime != "invocation cache" {
|
||||
t.Fatalf("loader selection/context = (%+v, %+v)", gotSelection, gotContext)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPromptKitDirectSuiteRetainsDiagnosticsOnNamedLookupFailure(t *testing.T) {
|
||||
warning := commandkit.Diagnostic{Severity: configkit.SeverityWarning, Write: func(w io.Writer) error { _, err := io.WriteString(w, "configuration warning\n"); return err }}
|
||||
app, _ := commandkit.New(commandkit.Identity{Name: "example"})
|
||||
err := commandkit.RegisterPromptKit(app, commandkit.PromptKitSuite[struct{}]{InspectPrompt: func(context.Context, commandkit.Selection, struct{}, string, string) (commandkit.Acquired[*promptkit.PromptInspection], error) {
|
||||
return commandkit.Acquired[*promptkit.PromptInspection]{Diagnostics: []commandkit.Diagnostic{warning}}, promptkit.ErrPromptNotFound
|
||||
}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
outcome := app.Execute(context.Background(), []string{"promptkit", "show", "prompt", "missing"}, commandkit.Streams{Out: io.Discard, Err: io.Discard})
|
||||
if outcome.Kind != commandkit.OutcomeOperational || !errors.Is(outcome.Cause, promptkit.ErrPromptNotFound) || len(outcome.Diagnostics) < 1 || outcome.Diagnostics[0].Severity != configkit.SeverityWarning {
|
||||
t.Fatalf("named lookup outcome did not retain acquisition diagnostic: %#v", outcome)
|
||||
}
|
||||
}
|
||||
|
||||
func sortedKeys(values map[string]any) []string {
|
||||
keys := make([]string, 0, len(values))
|
||||
for key := range values {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
for i := 0; i < len(keys); i++ {
|
||||
for j := i + 1; j < len(keys); j++ {
|
||||
if keys[j] < keys[i] {
|
||||
keys[i], keys[j] = keys[j], keys[i]
|
||||
}
|
||||
}
|
||||
}
|
||||
return keys
|
||||
}
|
||||
Reference in New Issue
Block a user