base-layer/commandkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Adopt native PromptKit resource inspection

Browse Source
This commit is contained in:
Eric Rakestraw
2026-11-07 04:05:51 +00:00
parent 6c35da3ab6
commit 5ff0dd4ea2
12 changed files with 313 additions and 385 deletions
Download Patch File Download Diff File Expand all files Collapse all files

80
docs/cli.md
View File

@@ -528,72 +528,32 @@ output.
## PromptKit definition inspection ## PromptKit definition inspection
Applications can install prompt and profile inspection independently with Applications can install prompt, profile, and backend inspection independently
`ckprompt.Command(suite)`. The `ckprompt.FromEngine(loadEngine)` adapter with `ckprompt.Command(suite)`. `ckprompt.FromEngine(loadEngine)` supplies these
supplies these callbacks alongside native requirements discovery and the five callbacks alongside requirements discovery and the five catalogs. Its loader
catalogs. Its loader receives invocation context, configuration selection, receives invocation context, configuration selection, typed input, and shared
typed input, and shared state. Direct suite callbacks retain their own state. Direct callbacks retain their own lifecycle.
implementation and lifecycle.
Prompt inspection uses an ID only. The prompt format and native selection
behavior remain owned by
[PromptKit v0.18.0](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/d9f6aad30509e538fc73ea16710df3a3ec084502/docs/consumers/pkg-promptkit.md).
```text ```text
example promptkit show prompt customer/summary --format yaml
example promptkit show prompt customer/summary example promptkit show prompt customer/summary
example promptkit show profile customer-support --format yaml example promptkit show profile customer-support --format json
example promptkit show backend local --format yaml
``` ```
Both commands require one ID operand and accept `--config`, `--overlay`, and Each command takes one resource ID and accepts `--config` and `--overlay`.
`--format text|yaml` (default `text`). These commands have no built-in Reports use the native PromptKit inspection value through the shared reporting
`--scope` selection; their namespace is `promptkit`. formats; automatic and explicit text use YAML for these structured values. Native
JSON field names and omission rules apply. Absent optional fields are omitted;
explicit zero values and empty collections remain present. Prompt messages are
captured content, not rendered requests, and the prompt hash is opaque.
Prompt YAML preserves native input order. The prompt hash is an opaque Profile reports describe the selected profile before backend completion or
owning-library equality value. request overrides. Backend reports describe configured backend fields. An
`api_key_env` value names an environment variable; it is not a credential
```yaml value or a claim that one is available. Neither report resolves references or
prompt_id: customer/summary performs a network probe. See the [PromptKit resource inspection contracts](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/8ff9aa3efdc9f16b9d2729e0a29ea083eb3c3eaf/resource_inspection.go).
prompt_hash: opaque-value A missing ID is a usage error; native lookup errors and cancellation retain
default_profile_id: customer-support supported cause identity.
inputs:
- name: topic
required: true
content_type: text/plain
description: Topic to summarize
output_contract:
format: json
validation_mode: basic
schema_path: ""
repair_attempts: 2
```
Profile YAML preserves the effective values reported by PromptKit, including
zero values and nested provider parameters. `api_key_env` is only an
environment-variable name. `api_key_required` reports a direct credential
requirement; neither field reports credential availability. A nil provider
parameter map is displayed as `{}`.
```yaml
profile_id: customer-support
effective_model_params:
backend_id: local
endpoint: https://example.invalid/v1
model: support-model
temperature: 0
max_tokens: 0
top_p: 0
timeout_seconds: 600
service_tier: ""
reasoning_effort: ""
api_key_env: ""
extra_params: {}
api_key_required: true
```
Text uses labeled fields and includes nested `extra_params` values. A missing ID
is a usage error. Definition selection reaches the owning inspection API and
preserves supported error identity and cancellation.
## PromptKit requirements ## PromptKit requirements

9
docs/consumers/guide.md
View File

@@ -258,12 +258,12 @@ engine's captured configuration. Add that scope to a `ckconfig.Suite` to expose
it under `config` with validate/show/sources/explain and native diff operations; it under `config` with validate/show/sources/explain and native diff operations;
it does not install PromptKit catalog, definition, or requirements commands. it does not install PromptKit catalog, definition, or requirements commands.
`ckprompt.FromEngine(loadEngine)` supplies eight callbacks: prompt, profile, `ckprompt.FromEngine(loadEngine)` supplies nine callbacks: prompt, profile,
backend, namespace, and collection catalogs; named prompt and profile backend, namespace, and collection catalogs; named prompt, profile, and backend
inspection; and native requirements discovery. Concrete request validation and inspection; and native requirements discovery. Concrete request validation and
smoke testing are separate capabilities, added explicitly with smoke testing are separate capabilities, added explicitly with
`ckprompt.WithRequestValidation` and `ckprompt.WithSmokeTesting`. The suite has `ckprompt.WithRequestValidation` and `ckprompt.WithSmokeTesting`. The suite has
ten independently optional callbacks in total. `ckprompt.Command(suite)` creates eleven independently optional callbacks in total. `ckprompt.Command(suite)` creates
only operations whose callbacks are present; discovery, validation, and smoke only operations whose callbacks are present; discovery, validation, and smoke
testing can be installed independently. A composed suite can enable both testing can be installed independently. A composed suite can enable both
additional operations: additional operations:
@@ -300,7 +300,7 @@ absent, so explicitly supplied empty values stay meaningful. Applications that
validate and then execute a request are encouraged to share their request validate and then execute a request are encouraged to share their request
construction logic between those paths. construction logic between those paths.
The eight catalog and inspection callbacks and the two explicit additions are The nine catalog and inspection callbacks and the two explicit additions are
independently optional. For catalogs without named or requirements inspection, independently optional. For catalogs without named or requirements inspection,
narrow the suite before command capture: narrow the suite before command capture:
@@ -308,6 +308,7 @@ narrow the suite before command capture:
definitions := ckprompt.FromEngine(loadEngine) definitions := ckprompt.FromEngine(loadEngine)
definitions.InspectPrompt = nil definitions.InspectPrompt = nil
definitions.InspectProfile = nil definitions.InspectProfile = nil
definitions.InspectBackend = nil
definitions.InspectRequirements = nil definitions.InspectRequirements = nil
definitionCommand := ckprompt.Command(definitions) definitionCommand := ckprompt.Command(definitions)
``` ```

10
docs/roadmap/implementation.md
View File

@@ -1,6 +1,6 @@
# Shared reporting and PromptKit v0.20.0 implementation plan # Shared reporting and PromptKit v0.20.0 implementation plan
Status: implementation in progress; Stages 1–7 complete. Status: implementation in progress; Stages 1–8 complete.
## Scope, authority, and execution ## Scope, authority, and execution
@@ -287,6 +287,14 @@ policy wording as behavior changes. Do not add domain comparison tests.
## Stage 8: Adopt PromptKit v0.20.0 and native resource inspection ## Stage 8: Adopt PromptKit v0.20.0 and native resource inspection
**Complete.** ckprompt selects released PromptKit v0.20.0 at commit
`8ff9aa3efdc9f16b9d2729e0a29ea083eb3c3eaf` with verified module sums and
updated attribution. Native prompt, profile, and backend inspection tests,
including format, callback, error, and cancellation cases, passed with Go
1.27.0. Root and ckconfig workspace suites also passed. The full ckprompt suite
still encounters the remaining local format bindings scheduled for Stage 9;
those bindings are not part of the completed inspection migration.
Update ckprompt's PromptKit requirement and authentic sums to v0.20.0; keep Update ckprompt's PromptKit requirement and authentic sums to v0.20.0; keep
ConfigKit v1.1.0 and existing Go minima. Review the released notices and update ConfigKit v1.1.0 and existing Go minima. Review the released notices and update
[module attribution](../../integrations/ckprompt/docs/oss/attribution.md) and [module attribution](../../integrations/ckprompt/docs/oss/attribution.md) and

4
integrations/ckprompt/README.md
View File

@@ -19,8 +19,8 @@ Add `ckprompt.ConfigScope(loadEngine)` to the scopes passed to
`ckconfig.Command` when the application wants PromptKit engine configuration `ckconfig.Command` when the application wants PromptKit engine configuration
and native comparison in the `config` family. Install and native comparison in the `config` family. Install
`ckprompt.Command(ckprompt.FromEngine(loadEngine))` separately for catalogs, `ckprompt.Command(ckprompt.FromEngine(loadEngine))` separately for catalogs,
named inspections, and requirements discovery. `FromEngine` supplies eight named inspections, and requirements discovery. `FromEngine` supplies nine
PromptKit-backed callbacks: three inspections and five catalogs. Applications PromptKit-backed callbacks: four inspections and five catalogs. Applications
may narrow, wrap, or replace them before capture or provide custom callbacks may narrow, wrap, or replace them before capture or provide custom callbacks
with `ckprompt.Suite`. Add concrete request validation and smoke testing with `ckprompt.Suite`. Add concrete request validation and smoke testing
explicitly with `WithRequestValidation` and `WithSmokeTesting`. Inspection, explicitly with `WithRequestValidation` and `WithSmokeTesting`. Inspection,

25
integrations/ckprompt/adapters.go
View File

@@ -29,6 +29,14 @@ type ProfileRequest struct {
ID string ID string
} }
// BackendRequest selects one named backend for inspection.
type BackendRequest struct {
// Selection carries configuration and overlay values supplied to the command.
Selection ckconfig.Selection
// ID is the selected named backend.
ID string
}
// CatalogRequest selects the configuration and native namespace filter for a // CatalogRequest selects the configuration and native namespace filter for a
// prompt, profile, or backend catalog operation. A zero Filter preserves // prompt, profile, or backend catalog operation. A zero Filter preserves
// PromptKit's native default selection. // PromptKit's native default selection.
@@ -96,6 +104,8 @@ type Suite[I, S any] struct {
// preparation. Its acquisition carries independent loader diagnostics and // preparation. Its acquisition carries independent loader diagnostics and
// terminal context alongside any native inspection result. // terminal context alongside any native inspection result.
InspectProfile func(context.Context, ProfileRequest, I, S) (commandkit.Acquired[*promptkit.ProfileInspection], error) InspectProfile func(context.Context, ProfileRequest, I, S) (commandkit.Acquired[*promptkit.ProfileInspection], error)
// InspectBackend independently enables inspection of one named backend.
InspectBackend func(context.Context, BackendRequest, I, S) (commandkit.Acquired[*promptkit.BackendInspection], error)
// InspectRequirements inspects native PromptKit requirements after input and // InspectRequirements inspects native PromptKit requirements after input and
// state preparation. Its acquisition carries independent loader diagnostics // state preparation. Its acquisition carries independent loader diagnostics
// and terminal context alongside any native inspection result. It is // and terminal context alongside any native inspection result. It is
@@ -300,9 +310,9 @@ func promptKitContextError(ctx context.Context) error {
return ctx.Err() return ctx.Err()
} }
// FromEngine supplies eight PromptKit-backed callbacks: prompt, profile, // FromEngine supplies nine PromptKit-backed callbacks: prompt, profile,
// backend, namespace, and collection catalogs, plus named prompt and profile // backend, namespace, and collection catalogs; named prompt, profile, and
// inspection and requirements discovery. // backend inspection; and requirements discovery.
// The requirements callback forwards RequirementsRequest.Inspection to // The requirements callback forwards RequirementsRequest.Inspection to
// Engine.InspectRequirements. Concrete request validation is an independent // Engine.InspectRequirements. Concrete request validation is an independent
// capability and can be composed explicitly with WithRequestValidation. Smoke // capability and can be composed explicitly with WithRequestValidation. Smoke
@@ -384,6 +394,15 @@ func FromEngine[I, S any](load func(context.Context, ckconfig.Selection, I, S) (
messages = messages.withOperationError(err) messages = messages.withOperationError(err)
return commandkit.Acquired[*promptkit.ProfileInspection]{Value: inspection, Diagnostics: messages.diagnostics, Explanation: messages.explanation}, err return commandkit.Acquired[*promptkit.ProfileInspection]{Value: inspection, Diagnostics: messages.diagnostics, Explanation: messages.explanation}, err
}, },
InspectBackend: func(ctx context.Context, request BackendRequest, input I, state S) (commandkit.Acquired[*promptkit.BackendInspection], error) {
engine, _, messages, err := loadPromptKitEngine(ctx, request.Selection, input, state, "", load)
if err != nil {
return commandkit.Acquired[*promptkit.BackendInspection]{Diagnostics: messages.diagnostics, Explanation: messages.explanation}, err
}
inspection, err := engine.InspectBackend(ctx, request.ID)
messages = messages.withOperationError(err)
return commandkit.Acquired[*promptkit.BackendInspection]{Value: inspection, Diagnostics: messages.diagnostics, Explanation: messages.explanation}, err
},
InspectRequirements: func(ctx context.Context, request RequirementsRequest, input I, state S) (commandkit.Acquired[*promptkit.RequirementInspection], error) { InspectRequirements: func(ctx context.Context, request RequirementsRequest, input I, state S) (commandkit.Acquired[*promptkit.RequirementInspection], error) {
engine, _, messages, err := loadPromptKitEngine(ctx, request.Selection, input, state, "", load) engine, _, messages, err := loadPromptKitEngine(ctx, request.Selection, input, state, "", load)
if err != nil { if err != nil {

243
integrations/ckprompt/command.go
View File

@@ -6,7 +6,6 @@ import (
"errors" "errors"
"fmt" "fmt"
"io" "io"
"strconv"
"strings" "strings"
"go.maximumdirect.net/ckconfig" "go.maximumdirect.net/ckconfig"
@@ -18,7 +17,6 @@ import (
type definitionInput[I any] struct { type definitionInput[I any] struct {
selection ckconfig.Selection selection ckconfig.Selection
id string id string
format string
captured commandkit.FlagValues[I] captured commandkit.FlagValues[I]
selected I selected I
} }
@@ -63,11 +61,10 @@ type inputAssignment struct {
// callback-referenced objects and effects remain caller-owned. Suite documents // callback-referenced objects and effects remain caller-owned. Suite documents
// invocation input and state initialization, zero values, and caller ownership // invocation input and state initialization, zero values, and caller ownership
// of returned objects. // of returned objects.
// Catalog commands accept config/overlay selection and suite selectors, and // Named resource inspections return native reports through shared formatting.
// render YAML with `yaml` as the only format and default. Named-show formats are // Catalog commands accept config/overlay selection and suite selectors.
// unchanged.
func Command[I, S any](suite Suite[I, S]) commandkit.Command { func Command[I, S any](suite Suite[I, S]) commandkit.Command {
if suite.InspectPrompt == nil && suite.InspectProfile == nil && suite.InspectRequirements == nil && suite.ValidateRequest == nil && suite.SmokeTest == nil && suite.ListPrompts == nil && suite.ListProfiles == nil && suite.ListBackends == nil && suite.ListNamespaces == nil && suite.ListCollections == nil { if suite.InspectPrompt == nil && suite.InspectProfile == nil && suite.InspectBackend == nil && suite.InspectRequirements == nil && suite.ValidateRequest == nil && suite.SmokeTest == nil && suite.ListPrompts == nil && suite.ListProfiles == nil && suite.ListBackends == nil && suite.ListNamespaces == nil && suite.ListCollections == nil {
command := commandkit.NoCommand() command := commandkit.NoCommand()
for _, issue := range suite.issues { for _, issue := range suite.issues {
command = commandkit.WithError(command, issue) command = commandkit.WithError(command, issue)
@@ -79,10 +76,13 @@ func Command[I, S any](suite Suite[I, S]) commandkit.Command {
rootCommands := make([]commandkit.Command, 0, 3) rootCommands := make([]commandkit.Command, 0, 3)
showCommands := make([]commandkit.Command, 0, 2) showCommands := make([]commandkit.Command, 0, 2)
if suite.InspectPrompt != nil { if suite.InspectPrompt != nil {
showCommands = append(showCommands, newDefinitionCommand(suite, "prompt", true)) showCommands = append(showCommands, newDefinitionCommand(suite, "prompt"))
} }
if suite.InspectProfile != nil { if suite.InspectProfile != nil {
showCommands = append(showCommands, newDefinitionCommand(suite, "profile", false)) showCommands = append(showCommands, newDefinitionCommand(suite, "profile"))
}
if suite.InspectBackend != nil {
showCommands = append(showCommands, newDefinitionCommand(suite, "backend"))
} }
if len(showCommands) != 0 { if len(showCommands) != 0 {
rootCommands = append(rootCommands, commandkit.Group("show", "Show a named PromptKit definition.", showCommands...)) rootCommands = append(rootCommands, commandkit.Group("show", "Show a named PromptKit definition.", showCommands...))
@@ -248,7 +248,7 @@ func prepareCatalogInput[I, S any](suite Suite[I, S], input *catalogInput[I]) (I
return prepareSuiteInput(suite, input.selection, input.captured) return prepareSuiteInput(suite, input.selection, input.captured)
} }
func newDefinitionCommand[I, S any](suite Suite[I, S], name string, prompt bool) commandkit.Command { func newDefinitionCommand[I, S any](suite Suite[I, S], name string) commandkit.Command {
flags := commandkit.CaptureFlags(suite.Selectors, func(input **definitionInput[I]) *commandkit.FlagValues[I] { flags := commandkit.CaptureFlags(suite.Selectors, func(input **definitionInput[I]) *commandkit.FlagValues[I] {
if input == nil || *input == nil { if input == nil || *input == nil {
return nil return nil
@@ -262,8 +262,6 @@ func newDefinitionCommand[I, S any](suite Suite[I, S], name string, prompt bool)
func(input **definitionInput[I], value commandkit.Supplied[string]) { func(input **definitionInput[I], value commandkit.Supplied[string]) {
(*input).selection.Overlay = value (*input).selection.Overlay = value
}), }),
commandkit.ValueFlag(commandkit.FlagMeta{Name: "format", Usage: "Report format: text or yaml.", Completion: completionChoices("text", "yaml")}, stringDefault("text"), parseDefinitionString,
func(input **definitionInput[I], value commandkit.Supplied[string]) { (*input).format = value.Value }),
) )
spec := commandkit.CommandSpec[*definitionInput[I]]{ spec := commandkit.CommandSpec[*definitionInput[I]]{
Name: name, Name: name,
@@ -273,13 +271,10 @@ func newDefinitionCommand[I, S any](suite Suite[I, S], name string, prompt bool)
}, },
Flags: flags, Flags: flags,
Arguments: []commandkit.Argument[*definitionInput[I]]{ Arguments: []commandkit.Argument[*definitionInput[I]]{
commandkit.ValueArgument(commandkit.ArgumentMeta{Name: "id", Usage: "Prompt or profile ID."}, true, parseDefinitionString, commandkit.ValueArgument(commandkit.ArgumentMeta{Name: strings.ToUpper(name) + "_ID", Usage: "Resource ID."}, true, parseDefinitionString,
func(input **definitionInput[I], value commandkit.Supplied[string]) { (*input).id = value.Value }), func(input **definitionInput[I], value commandkit.Supplied[string]) { (*input).id = value.Value }),
}, },
CheckInput: func(input *definitionInput[I]) error { CheckInput: func(input *definitionInput[I]) error {
if err := checkDefinitionFormat(input.format); err != nil {
return err
}
selected, err := prepareDefinitionInput(suite, input) selected, err := prepareDefinitionInput(suite, input)
if err != nil { if err != nil {
return err return err
@@ -288,7 +283,7 @@ func newDefinitionCommand[I, S any](suite Suite[I, S], name string, prompt bool)
return nil return nil
}, },
Run: func(invocation commandkit.Invocation, input *definitionInput[I]) commandkit.Outcome { Run: func(invocation commandkit.Invocation, input *definitionInput[I]) commandkit.Outcome {
return runDefinitionInspection(invocation, suite, input, prompt) return runDefinitionInspection(invocation, suite, input, name)
}, },
} }
return commandkit.NewCommand(spec) return commandkit.NewCommand(spec)
@@ -419,77 +414,56 @@ func runCatalogList[T any, I, S any](
}) })
} }
func runDefinitionInspection[I, S any](invocation commandkit.Invocation, suite Suite[I, S], input *definitionInput[I], prompt bool) commandkit.Outcome { func runDefinitionInspection[I, S any](invocation commandkit.Invocation, suite Suite[I, S], input *definitionInput[I], kind string) commandkit.Outcome {
if invocation.Context != nil { if err := promptKitContextError(invocation.Context); err != nil {
if err := invocation.Context.Err(); err != nil {
return definitionFailure("PromptKit inspection was canceled.", commandkit.Explanation{}, err) return definitionFailure("PromptKit inspection was canceled.", commandkit.Explanation{}, err)
} }
}
var state S var state S
if suite.NewState != nil { if suite.NewState != nil {
state = suite.NewState() state = suite.NewState()
} }
if invocation.Context != nil { if err := promptKitContextError(invocation.Context); err != nil {
if err := invocation.Context.Err(); err != nil {
return definitionFailure("PromptKit inspection was canceled.", commandkit.Explanation{}, err) return definitionFailure("PromptKit inspection was canceled.", commandkit.Explanation{}, err)
} }
var value any
var diagnostics []commandkit.Diagnostic
var explanation commandkit.Explanation
var err error
switch kind {
case "prompt":
var result commandkit.Acquired[*promptkit.PromptInspection]
result, err = suite.InspectPrompt(invocation.Context, PromptRequest{Selection: input.selection, ID: input.id}, input.selected, state)
if result.Value != nil {
value = result.Value
} }
diagnostics, explanation = result.Diagnostics, result.Explanation
if prompt { case "profile":
request := PromptRequest{Selection: input.selection, ID: input.id} var result commandkit.Acquired[*promptkit.ProfileInspection]
acquired, err := suite.InspectPrompt(invocation.Context, request, input.selected, state) result, err = suite.InspectProfile(invocation.Context, ProfileRequest{Selection: input.selection, ID: input.id}, input.selected, state)
queuePromptDiagnostics(invocation.Reporter(), acquired.Diagnostics) if result.Value != nil {
value = result.Value
}
diagnostics, explanation = result.Diagnostics, result.Explanation
case "backend":
var result commandkit.Acquired[*promptkit.BackendInspection]
result, err = suite.InspectBackend(invocation.Context, BackendRequest{Selection: input.selection, ID: input.id}, input.selected, state)
if result.Value != nil {
value = result.Value
}
diagnostics, explanation = result.Diagnostics, result.Explanation
}
queuePromptDiagnostics(invocation.Reporter(), diagnostics)
if err != nil { if err != nil {
return definitionFailure(fmt.Sprintf("Unable to inspect PromptKit prompt %q.", input.id), acquired.Explanation, err) return definitionFailure(fmt.Sprintf("Unable to inspect PromptKit %s %q.", kind, input.id), explanation, err)
} }
if acquired.Value == nil { if value == nil {
return definitionFailure("PromptKit prompt inspection returned no result.", acquired.Explanation, nil) return definitionFailure(fmt.Sprintf("PromptKit %s inspection returned no result.", kind), explanation, nil)
} }
if invocation.Context != nil { if err := promptKitContextError(invocation.Context); err != nil {
if err := invocation.Context.Err(); err != nil { return definitionFailure(fmt.Sprintf("Inspection of PromptKit %s %q was canceled.", kind, input.id), explanation, err)
return definitionFailure(fmt.Sprintf("Inspection of PromptKit prompt %q was canceled.", input.id), acquired.Explanation, err)
} }
} _ = invocation.Reporter().Report(commandkit.Report{Value: value})
return renderPromptReport(invocation, acquired.Value, input.format, acquired.Explanation) return commandkit.Success()
}
request := ProfileRequest{Selection: input.selection, ID: input.id}
acquired, err := suite.InspectProfile(invocation.Context, request, input.selected, state)
queuePromptDiagnostics(invocation.Reporter(), acquired.Diagnostics)
if err != nil {
return definitionFailure(fmt.Sprintf("Unable to inspect PromptKit profile %q.", input.id), acquired.Explanation, err)
}
if acquired.Value == nil {
return definitionFailure("PromptKit profile inspection returned no result.", acquired.Explanation, nil)
}
if invocation.Context != nil {
if err := invocation.Context.Err(); err != nil {
return definitionFailure(fmt.Sprintf("Inspection of PromptKit profile %q was canceled.", input.id), acquired.Explanation, err)
}
}
return renderProfileReport(invocation, acquired.Value, input.format, acquired.Explanation)
}
type promptInputReport struct {
Name string `yaml:"name"`
Required bool `yaml:"required"`
ContentType string `yaml:"content_type"`
Description string `yaml:"description"`
}
type promptOutputContractReport struct {
Format string `yaml:"format"`
ValidationMode string `yaml:"validation_mode"`
SchemaPath string `yaml:"schema_path"`
RepairAttempts int `yaml:"repair_attempts"`
}
type promptInspectionReport struct {
PromptID string `yaml:"prompt_id"`
PromptHash string `yaml:"prompt_hash"`
DefaultProfileID string `yaml:"default_profile_id"`
Inputs []promptInputReport `yaml:"inputs"`
OutputContract promptOutputContractReport `yaml:"output_contract"`
} }
type requirementsInspectionReport struct { type requirementsInspectionReport struct {
@@ -524,45 +498,6 @@ type requirementIssueReport struct {
Description string `yaml:"description"` Description string `yaml:"description"`
} }
type modelParamsReport struct {
BackendID string `yaml:"backend_id"`
Endpoint string `yaml:"endpoint"`
Model string `yaml:"model"`
Temperature float64 `yaml:"temperature"`
MaxTokens int `yaml:"max_tokens"`
TopP float64 `yaml:"top_p"`
TimeoutSeconds int `yaml:"timeout_seconds"`
ServiceTier string `yaml:"service_tier"`
ReasoningEffort string `yaml:"reasoning_effort"`
APIKeyEnv string `yaml:"api_key_env"`
ExtraParams map[string]any `yaml:"extra_params"`
}
type profileInspectionReport struct {
ProfileID string `yaml:"profile_id"`
EffectiveModelParams modelParamsReport `yaml:"effective_model_params"`
APIKeyRequired bool `yaml:"api_key_required"`
}
func promptReportFromInspection(inspection *promptkit.PromptInspection) promptInspectionReport {
report := promptInspectionReport{
PromptID: inspection.PromptID,
PromptHash: inspection.PromptHash,
DefaultProfileID: inspection.DefaultProfileID,
Inputs: make([]promptInputReport, 0, len(inspection.Inputs)),
OutputContract: promptOutputContractReport{
Format: string(inspection.OutputContract.Format),
ValidationMode: string(inspection.OutputContract.ValidationMode),
SchemaPath: inspection.OutputContract.SchemaPath,
RepairAttempts: inspection.OutputContract.RepairAttempts,
},
}
for _, input := range inspection.Inputs {
report.Inputs = append(report.Inputs, promptInputReport{Name: input.Name, Required: input.Required, ContentType: input.ContentType, Description: input.Description})
}
return report
}
func requirementsReportFromInspection(inspection *promptkit.RequirementInspection) requirementsInspectionReport { func requirementsReportFromInspection(inspection *promptkit.RequirementInspection) requirementsInspectionReport {
report := requirementsInspectionReport{ report := requirementsInspectionReport{
PromptID: inspection.PromptID, PromptID: inspection.PromptID,
@@ -601,24 +536,6 @@ func requirementsReportFromInspection(inspection *promptkit.RequirementInspectio
return report return report
} }
func profileReportFromInspection(inspection *promptkit.ProfileInspection) profileInspectionReport {
params := inspection.EffectiveModelParams
extra := params.ExtraParams
if extra == nil {
extra = map[string]any{}
}
return profileInspectionReport{
ProfileID: inspection.ProfileID,
EffectiveModelParams: modelParamsReport{
BackendID: params.BackendID, Endpoint: params.Endpoint, Model: params.Model,
Temperature: params.Temperature, MaxTokens: params.MaxTokens, TopP: params.TopP,
TimeoutSeconds: params.TimeoutSeconds, ServiceTier: params.ServiceTier,
ReasoningEffort: params.ReasoningEffort, APIKeyEnv: params.APIKeyEnv, ExtraParams: extra,
},
APIKeyRequired: inspection.APIKeyRequired,
}
}
func runRequirementsInspection[I, S any](invocation commandkit.Invocation, suite Suite[I, S], input *requirementsInput[I]) commandkit.Outcome { func runRequirementsInspection[I, S any](invocation commandkit.Invocation, suite Suite[I, S], input *requirementsInput[I]) commandkit.Outcome {
if invocation.Context != nil { if invocation.Context != nil {
if err := invocation.Context.Err(); err != nil { if err := invocation.Context.Err(); err != nil {
@@ -654,27 +571,6 @@ func runRequirementsInspection[I, S any](invocation commandkit.Invocation, suite
}) })
} }
func renderPromptReport(invocation commandkit.Invocation, inspection *promptkit.PromptInspection, format string, explanation commandkit.Explanation) commandkit.Outcome {
return renderPreparedPromptReport(invocation, explanation, func() ([]byte, error) {
report := promptReportFromInspection(inspection)
if format == "yaml" {
return encodePromptYAMLBytes(report)
}
return []byte(formatPromptText(report)), nil
})
}
func renderProfileReport(invocation commandkit.Invocation, inspection *promptkit.ProfileInspection, format string, explanation commandkit.Explanation) commandkit.Outcome {
return renderPreparedPromptReport(invocation, explanation, func() ([]byte, error) {
report := profileReportFromInspection(inspection)
if format == "yaml" {
return encodePromptYAMLBytes(report)
}
text, err := formatProfileText(report)
return []byte(text), err
})
}
// These reports were already prepared before delivery. Keep that boundary // These reports were already prepared before delivery. Keep that boundary
// inside managed rendering, with cancellation owned by the command outcome. // inside managed rendering, with cancellation owned by the command outcome.
func renderPreparedPromptReport(invocation commandkit.Invocation, explanation commandkit.Explanation, prepare func() ([]byte, error)) commandkit.Outcome { func renderPreparedPromptReport(invocation commandkit.Invocation, explanation commandkit.Explanation, prepare func() ([]byte, error)) commandkit.Outcome {
@@ -696,42 +592,6 @@ func renderPreparedPromptReport(invocation commandkit.Invocation, explanation co
return commandkit.Success() return commandkit.Success()
} }
func formatPromptText(report promptInspectionReport) string {
var output strings.Builder
fmt.Fprintf(&output, "Prompt ID: %s\n", strconv.Quote(report.PromptID))
fmt.Fprintf(&output, "Prompt hash: %s\n", strconv.Quote(report.PromptHash))
fmt.Fprintf(&output, "Default profile ID: %s\n", strconv.Quote(report.DefaultProfileID))
output.WriteString("Inputs:\n")
for _, input := range report.Inputs {
fmt.Fprintf(&output, " - name: %s\n required: %t\n content_type: %s\n description: %s\n", strconv.Quote(input.Name), input.Required, strconv.Quote(input.ContentType), strconv.Quote(input.Description))
}
if len(report.Inputs) == 0 {
output.WriteString(" []\n")
}
fmt.Fprintf(&output, "Output contract:\n format: %s\n validation_mode: %s\n schema_path: %s\n repair_attempts: %d\n", strconv.Quote(report.OutputContract.Format), strconv.Quote(report.OutputContract.ValidationMode), strconv.Quote(report.OutputContract.SchemaPath), report.OutputContract.RepairAttempts)
return output.String()
}
func formatProfileText(report profileInspectionReport) (string, error) {
params := report.EffectiveModelParams
var output strings.Builder
fmt.Fprintf(&output, "Profile ID: %s\n", strconv.Quote(report.ProfileID))
output.WriteString("Effective model params:\n")
fmt.Fprintf(&output, " backend_id: %s\n endpoint: %s\n model: %s\n temperature: %s\n max_tokens: %d\n top_p: %s\n timeout_seconds: %d\n service_tier: %s\n reasoning_effort: %s\n api_key_env: %s\n", strconv.Quote(params.BackendID), strconv.Quote(params.Endpoint), strconv.Quote(params.Model), strconv.FormatFloat(params.Temperature, 'g', -1, 64), params.MaxTokens, strconv.FormatFloat(params.TopP, 'g', -1, 64), params.TimeoutSeconds, strconv.Quote(params.ServiceTier), strconv.Quote(params.ReasoningEffort), strconv.Quote(params.APIKeyEnv))
output.WriteString(" extra_params:\n")
var nested strings.Builder
if err := encodePromptYAML(&nested, params.ExtraParams); err != nil {
return "", err
}
for _, line := range strings.Split(strings.TrimSuffix(nested.String(), "\n"), "\n") {
output.WriteString(" ")
output.WriteString(line)
output.WriteByte('\n')
}
fmt.Fprintf(&output, "API key required: %t\n", report.APIKeyRequired)
return output.String(), nil
}
func encodePromptYAML(writer io.Writer, value any) error { func encodePromptYAML(writer io.Writer, value any) error {
encoder := yaml.NewEncoder(writer) encoder := yaml.NewEncoder(writer)
encoder.SetIndent(2) encoder.SetIndent(2)
@@ -808,13 +668,6 @@ func definitionFailure(message string, explanation commandkit.Explanation, cause
return outcome return outcome
} }
func checkDefinitionFormat(format string) error {
if format != "text" && format != "yaml" {
return fmt.Errorf("unsupported format %q; expected text or yaml", format)
}
return nil
}
func parseDefinitionString(value string) (string, error) { return value, nil } func parseDefinitionString(value string) (string, error) { return value, nil }
func parseVariableAssignment(raw string) (variableAssignment, error) { func parseVariableAssignment(raw string) (variableAssignment, error) {

110
integrations/ckprompt/command_test.go
View File

@@ -809,11 +809,11 @@ func TestCatalogAndInspectionRespectSuccessfulReportingModes(t *testing.T) {
func TestQuietInspectionStillFindsReportEncodingFailure(t *testing.T) { func TestQuietInspectionStillFindsReportEncodingFailure(t *testing.T) {
called := false called := false
extra := map[string]any{"unsupported": func() {}}
app := newPromptTestApp(t, Command(Suite[struct{}, struct{}]{InspectProfile: func(context.Context, ProfileRequest, struct{}, struct{}) (commandkit.Acquired[*promptkit.ProfileInspection], error) { app := newPromptTestApp(t, Command(Suite[struct{}, struct{}]{InspectProfile: func(context.Context, ProfileRequest, struct{}, struct{}) (commandkit.Acquired[*promptkit.ProfileInspection], error) {
called = true called = true
return commandkit.Acquired[*promptkit.ProfileInspection]{Value: &promptkit.ProfileInspection{ return commandkit.Acquired[*promptkit.ProfileInspection]{Value: &promptkit.ProfileInspection{
ProfileID: "broken", ProfileID: "broken", ExtraParams: &extra,
EffectiveModelParams: promptkit.ExecutionTarget{ExtraParams: map[string]any{"unsupported": func() {}}},
}}, nil }}, nil
}})) }}))
result := &countingWriter{} result := &countingWriter{}
@@ -924,99 +924,6 @@ func TestDefinitionInputAndStateLifetimesFollowChecks(t *testing.T) {
} }
} }
func TestDefinitionReportsRetainNativeFieldsAndSafeCredentialReferences(t *testing.T) {
t.Setenv("PROMPTKIT_ADAPTER_TEST_KEY", "credential-value-must-not-appear")
engine := adapterEngine(t)
loadCalls := 0
suite := FromEngine(func(context.Context, ckconfig.Selection, struct{}, struct{}) (*promptkit.Engine, error) {
loadCalls++
return engine, nil
})
app := newPromptTestApp(t, Command(suite))
var output bytes.Buffer
outcome := app.Execute(context.Background(), []string{"promptkit", "show", "prompt", "briefing", "--format", "yaml"}, commandkit.Streams{Out: &output, Err: io.Discard})
if outcome.Kind != commandkit.OutcomeSuccess || loadCalls != 1 {
t.Fatalf("prompt report outcome=%#v load calls=%d report=%s", outcome, loadCalls, output.String())
}
var prompt map[string]any
if err := yaml.Unmarshal(output.Bytes(), &prompt); err != nil {
t.Fatal(err)
}
if _, exists := prompt["version"]; exists {
t.Fatalf("prompt version emitted: %#v", prompt)
}
inputs := prompt["inputs"].([]any)
if len(inputs) != 2 || inputs[0].(map[string]any)["name"] != "topic" || inputs[1].(map[string]any)["name"] != "audience" || prompt["prompt_hash"] == "" {
t.Fatalf("prompt fields/order = %#v", prompt)
}
contract := prompt["output_contract"].(map[string]any)
if contract["repair_attempts"] != 0 {
t.Fatalf("zero-valued contract field was lost: %#v", contract)
}
output.Reset()
outcome = app.Execute(context.Background(), []string{"promptkit", "show", "prompt", "briefing"}, commandkit.Streams{Out: &output, Err: io.Discard})
if outcome.Kind != commandkit.OutcomeSuccess || loadCalls != 2 || strings.Contains(output.String(), "Prompt version:") || !strings.Contains(output.String(), `- name: "topic"`) {
t.Fatalf("default text prompt report = (%#v, calls=%d, %s)", outcome, loadCalls, output.String())
}
output.Reset()
outcome = app.Execute(context.Background(), []string{"promptkit", "show", "profile", "default", "--format", "yaml"}, commandkit.Streams{Out: &output, Err: io.Discard})
if outcome.Kind != commandkit.OutcomeSuccess || loadCalls != 3 {
t.Fatalf("profile report outcome=%#v calls=%d report=%s", outcome, loadCalls, output.String())
}
var profile map[string]any
if err := yaml.Unmarshal(output.Bytes(), &profile); err != nil {
t.Fatal(err)
}
params := profile["effective_model_params"].(map[string]any)
if params["api_key_env"] != "PROMPTKIT_ADAPTER_TEST_KEY" || params["temperature"] != 0 || params["max_tokens"] != 0 || strings.Contains(output.String(), "credential-value-must-not-appear") {
t.Fatalf("credential or zero model fields = %#v / %s", params, output.String())
}
extra := params["extra_params"].(map[string]any)
nested := extra["nested"].(map[string]any)
sequence := extra["sequence"].([]any)
if extra["token"] != "caller-owned-value" || nested["null_value"] != nil || nested["visible"] != true || len(sequence) != 3 || sequence[1] != nil || sequence[2] != 3 {
t.Fatalf("nested ExtraParams changed: %#v", extra)
}
output.Reset()
outcome = app.Execute(context.Background(), []string{"promptkit", "show", "profile", "default"}, commandkit.Streams{Out: &output, Err: io.Discard})
if outcome.Kind != commandkit.OutcomeSuccess || loadCalls != 4 || !strings.Contains(output.String(), "API key required: false") || !strings.Contains(output.String(), `api_key_env: "PROMPTKIT_ADAPTER_TEST_KEY"`) {
t.Fatalf("default text profile report = (%#v, calls=%d, %s)", outcome, loadCalls, output.String())
}
output.Reset()
outcome = app.Execute(context.Background(), []string{"promptkit", "show", "profile", "requires-key", "--format", "yaml"}, commandkit.Streams{Out: &output, Err: io.Discard})
if outcome.Kind != commandkit.OutcomeSuccess || !strings.Contains(output.String(), "api_key_required: true") {
t.Fatalf("credential requirement report = (%#v, %s)", outcome, output.String())
}
output.Reset()
outcome = app.Execute(context.Background(), []string{"promptkit", "show", "prompt", "empty", "--format", "yaml"}, commandkit.Streams{Out: &output, Err: io.Discard})
if outcome.Kind != commandkit.OutcomeSuccess || !strings.Contains(output.String(), "inputs: []") {
t.Fatalf("empty prompt input list = (%#v, %s)", outcome, output.String())
}
}
func TestPromptReportOmitsRemovedVersion(t *testing.T) {
report := promptReportFromInspection(&promptkit.PromptInspection{PromptID: "briefing", PromptHash: "hash"})
encoded, err := encodePromptYAMLBytes(report)
if err != nil {
t.Fatal(err)
}
var document map[string]any
if err := yaml.Unmarshal(encoded, &document); err != nil {
t.Fatal(err)
}
if _, exists := document["version"]; exists || document["prompt_id"] != "briefing" || document["prompt_hash"] != "hash" {
t.Fatalf("prompt report = %#v", document)
}
if text := formatPromptText(report); strings.Contains(text, "Prompt version:") || !strings.Contains(text, `Prompt hash: "hash"`) {
t.Fatalf("prompt text = %q", text)
}
}
func TestNativeLookupErrorsAndBlankIDsReachCallbacks(t *testing.T) { func TestNativeLookupErrorsAndBlankIDsReachCallbacks(t *testing.T) {
nativeErr := errors.New("native lookup failure") nativeErr := errors.New("native lookup failure")
calls := 0 calls := 0
@@ -1075,7 +982,7 @@ func TestDefinitionFormatPreflightAndReportDeliveryFailure(t *testing.T) {
return commandkit.Acquired[*promptkit.ProfileInspection]{Value: &promptkit.ProfileInspection{ProfileID: "default"}}, nil return commandkit.Acquired[*promptkit.ProfileInspection]{Value: &promptkit.ProfileInspection{ProfileID: "default"}}, nil
}})) }}))
var output bytes.Buffer var output bytes.Buffer
outcome := app.Execute(context.Background(), []string{"promptkit", "show", "profile", "default", "--format", "json"}, commandkit.Streams{Out: &output, Err: io.Discard}) outcome := app.Execute(context.Background(), []string{"promptkit", "show", "profile", "default", "--format", "invalid"}, commandkit.Streams{Out: &output, Err: io.Discard})
if outcome.Kind != commandkit.OutcomeUsage || calls != 0 || output.Len() != 0 { if outcome.Kind != commandkit.OutcomeUsage || calls != 0 || output.Len() != 0 {
t.Fatalf("bad format was not rejected before callback: outcome=%#v calls=%d output=%q", outcome, calls, output.String()) t.Fatalf("bad format was not rejected before callback: outcome=%#v calls=%d output=%q", outcome, calls, output.String())
} }
@@ -1098,17 +1005,6 @@ func (writer partialReportWriter) Write(content []byte) (int, error) {
var _ io.Writer = partialReportWriter{} var _ io.Writer = partialReportWriter{}
func TestProfileTextUsesCredentialNamesAndExplicitEmptyExtraParams(t *testing.T) {
report := profileReportFromInspection(&promptkit.ProfileInspection{ProfileID: "default", EffectiveModelParams: promptkit.ExecutionTarget{APIKeyEnv: "APP_KEY", ExtraParams: nil}})
text, err := formatProfileText(report)
if err != nil || !strings.Contains(text, "api_key_env: \"APP_KEY\"") || !strings.Contains(text, "extra_params:\n {}\n") || !strings.Contains(text, "API key required: false") {
t.Fatalf("profile text = (%q, %v)", text, err)
}
if strings.Contains(text, "APP_KEY_VALUE") {
t.Fatalf("report contains a credential value: %q", text)
}
}
func TestPromptCatalogYAMLPreservesSchemaStringsAndOrder(t *testing.T) { func TestPromptCatalogYAMLPreservesSchemaStringsAndOrder(t *testing.T) {
want := []promptkit.PromptIdentity{ want := []promptkit.PromptIdentity{
{ID: " briefing "}, {ID: " briefing "},

6
integrations/ckprompt/docs/oss/attribution.md
View File

@@ -2,16 +2,16 @@
## PromptKit ## PromptKit
Selected module: `go.maximumdirect.net/promptkit` at `v0.18.0`. Selected module: `go.maximumdirect.net/promptkit` at `v0.20.0`.
Copyright (C) 2026 Eric Rakestraw Copyright (C) 2026 Eric Rakestraw
SPDX-License-Identifier: LGPL-3.0-or-later SPDX-License-Identifier: LGPL-3.0-or-later
License and notices: [LGPL v3.0](../../LICENSE) | License and notices: [LGPL v3.0](../../LICENSE) |
[LICENSE](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/d9f6aad30509e538fc73ea16710df3a3ec084502/LICENSE). [LICENSE](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/8ff9aa3efdc9f16b9d2729e0a29ea083eb3c3eaf/LICENSE).
[Source](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/d9f6aad30509e538fc73ea16710df3a3ec084502). [Source](https://forge.maximumdirect.net/base-layer/promptkit/src/commit/8ff9aa3efdc9f16b9d2729e0a29ea083eb3c3eaf).
## ConfigKit ## ConfigKit

2
integrations/ckprompt/go.mod
View File

@@ -6,7 +6,7 @@ require (
go.maximumdirect.net/ckconfig v0.3.0 go.maximumdirect.net/ckconfig v0.3.0
go.maximumdirect.net/commandkit v0.3.0 go.maximumdirect.net/commandkit v0.3.0
go.maximumdirect.net/configkit v1.1.0 go.maximumdirect.net/configkit v1.1.0
go.maximumdirect.net/promptkit v0.18.0 go.maximumdirect.net/promptkit v0.20.0
go.yaml.in/yaml/v4 v4.0.0-rc.6 go.yaml.in/yaml/v4 v4.0.0-rc.6
) )

2
integrations/ckprompt/go.sum
View File

@@ -18,6 +18,8 @@ go.maximumdirect.net/configkit v1.1.0 h1:UzXN2ZMnEkcL0O4YSLqD9qd9M1gb/NWCHQR/H8A
go.maximumdirect.net/configkit v1.1.0/go.mod h1:s1zLJ9GGB/1Q4a+aOHMLISonJxWFgNWpA6SpPlOafeA= go.maximumdirect.net/configkit v1.1.0/go.mod h1:s1zLJ9GGB/1Q4a+aOHMLISonJxWFgNWpA6SpPlOafeA=
go.maximumdirect.net/promptkit v0.18.0 h1:mrO+mangAPiR1xdtYjTHemH5xA//waddekVnhI5i0yE= go.maximumdirect.net/promptkit v0.18.0 h1:mrO+mangAPiR1xdtYjTHemH5xA//waddekVnhI5i0yE=
go.maximumdirect.net/promptkit v0.18.0/go.mod h1:NTdAFcqISmmgiAGch2+s/mHrZTrkUXJYGKc1UfUMF24= go.maximumdirect.net/promptkit v0.18.0/go.mod h1:NTdAFcqISmmgiAGch2+s/mHrZTrkUXJYGKc1UfUMF24=
go.maximumdirect.net/promptkit v0.20.0 h1:ZpwbjiCZWXdnnZ26RuWKoOco0jFnaAX8m2tQAX7ZlnM=
go.maximumdirect.net/promptkit v0.20.0/go.mod h1:kCoyvI0xSax9K9aRPCEkpkyAP5YLs157oBLFLdzgRbM=
go.yaml.in/yaml/v4 v4.0.0-rc.6 h1:1h7H1ohdUh93/FyE4YaDa1Zh64K6VVbjF4K6WUxMtH4= go.yaml.in/yaml/v4 v4.0.0-rc.6 h1:1h7H1ohdUh93/FyE4YaDa1Zh64K6VVbjF4K6WUxMtH4=
go.yaml.in/yaml/v4 v4.0.0-rc.6/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0= go.yaml.in/yaml/v4 v4.0.0-rc.6/go.mod h1:aZqd9kCMsGL7AuUv/m/PvWLdg5sjJsZ4oHDEnfPPfY0=
golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ= golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=

17
integrations/ckprompt/report_cancellation_test.go
View File

@@ -2,33 +2,30 @@ package ckprompt
import ( import (
"context" "context"
"errors"
"testing" "testing"
commandkit "go.maximumdirect.net/commandkit" commandkit "go.maximumdirect.net/commandkit"
promptkit "go.maximumdirect.net/promptkit" promptkit "go.maximumdirect.net/promptkit"
) )
type cancelDuringYAML struct{ cancel context.CancelFunc } type cancelDuringJSON struct{ cancel context.CancelFunc }
func (value cancelDuringYAML) MarshalYAML() (any, error) { func (value cancelDuringJSON) MarshalJSON() ([]byte, error) {
value.cancel() value.cancel()
return "prepared", nil return []byte(`"prepared"`), nil
} }
func TestInspectionCancellationDuringPreparationPreventsDelivery(t *testing.T) { func TestInspectionRetainsAcquiredReportWhenRenderingCancels(t *testing.T) {
for _, format := range []string{"text", "yaml"} { for _, format := range []string{"text", "yaml"} {
for _, mode := range []string{"ordinary", "quiet"} { for _, mode := range []string{"ordinary", "quiet"} {
t.Run(format+"/"+mode, func(t *testing.T) { t.Run(format+"/"+mode, func(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background()) ctx, cancel := context.WithCancel(context.Background())
defer cancel() defer cancel()
extra := map[string]any{"controlled": cancelDuringJSON{cancel}}
app := newPromptTestApp(t, Command(Suite[struct{}, struct{}]{ app := newPromptTestApp(t, Command(Suite[struct{}, struct{}]{
InspectProfile: func(context.Context, ProfileRequest, struct{}, struct{}) (commandkit.Acquired[*promptkit.ProfileInspection], error) { InspectProfile: func(context.Context, ProfileRequest, struct{}, struct{}) (commandkit.Acquired[*promptkit.ProfileInspection], error) {
return commandkit.Acquired[*promptkit.ProfileInspection]{Value: &promptkit.ProfileInspection{ return commandkit.Acquired[*promptkit.ProfileInspection]{Value: &promptkit.ProfileInspection{
ProfileID: "demo", ProfileID: "demo", ExtraParams: &extra,
EffectiveModelParams: promptkit.ExecutionTarget{ExtraParams: map[string]any{
"controlled": cancelDuringYAML{cancel},
}},
}}, nil }}, nil
}, },
})) }))
@@ -38,7 +35,7 @@ func TestInspectionCancellationDuringPreparationPreventsDelivery(t *testing.T) {
} }
output := &countingWriter{} output := &countingWriter{}
got := app.Execute(ctx, args, commandkit.Streams{Out: output}) got := app.Execute(ctx, args, commandkit.Streams{Out: output})
if got.Kind != commandkit.OutcomeOperational || got.Status != 1 || !errors.Is(got.Cause, context.Canceled) || output.calls != 0 { if got.Kind != commandkit.OutcomeSuccess || got.Status != 0 || ctx.Err() != context.Canceled || (mode == "ordinary" && output.calls == 0) || (mode == "quiet" && output.calls != 0) {
t.Fatalf("outcome=%#v writes=%d", got, output.calls) t.Fatalf("outcome=%#v writes=%d", got, output.calls)
} }
if got.Presentation.Output.RenderError != nil || got.Presentation.Output.WriteError != nil { if got.Presentation.Output.RenderError != nil || got.Presentation.Output.WriteError != nil {

192
integrations/ckprompt/resource_inspection_test.go Normal file
View File

@@ -0,0 +1,192 @@
package ckprompt
import (
"bytes"
"context"
"encoding/json"
"errors"
"io"
"reflect"
"strings"
"testing"
"go.maximumdirect.net/ckconfig"
commandkit "go.maximumdirect.net/commandkit"
promptkit "go.maximumdirect.net/promptkit"
)
func jsonValue(t *testing.T, value any) any {
t.Helper()
data, err := json.Marshal(value)
if err != nil {
t.Fatal(err)
}
decoder := json.NewDecoder(bytes.NewReader(data))
decoder.UseNumber()
var result any
if err := decoder.Decode(&result); err != nil {
t.Fatal(err)
}
return result
}
func TestResourceInspectionsForwardNativeReports(t *testing.T) {
zero := 0
no := false
empty := ""
inputs := []promptkit.PromptInputInspection{{Name: "topic", Required: &no}}
extra := map[string]any{}
prompt := &promptkit.PromptInspection{PromptID: "p", PromptHash: "hash", Description: &empty, Inputs: &inputs, Messages: []promptkit.PromptMessageInspection{{Role: "user", Content: "captured {{.topic}}"}}}
profile := &promptkit.ProfileInspection{ProfileID: "p", Temperature: nil, MaxTokens: &zero, APIKeyRequired: &no, ExtraParams: &extra}
backend := &promptkit.BackendInspection{BackendID: "b", Endpoint: "https://example.invalid", QueueCapacity: &zero, ExtraParams: &extra}
var selections []ckconfig.Selection
calls := 0
suite := Suite[struct{}, struct{}]{
InspectPrompt: func(_ context.Context, r PromptRequest, _, _ struct{}) (commandkit.Acquired[*promptkit.PromptInspection], error) {
calls++
selections = append(selections, r.Selection)
if r.ID != "p" {
t.Errorf("prompt ID %q", r.ID)
}
return commandkit.Acquired[*promptkit.PromptInspection]{Value: prompt}, nil
},
InspectProfile: func(_ context.Context, r ProfileRequest, _, _ struct{}) (commandkit.Acquired[*promptkit.ProfileInspection], error) {
calls++
selections = append(selections, r.Selection)
if r.ID != "p" {
t.Errorf("profile ID %q", r.ID)
}
return commandkit.Acquired[*promptkit.ProfileInspection]{Value: profile}, nil
},
InspectBackend: func(_ context.Context, r BackendRequest, _, _ struct{}) (commandkit.Acquired[*promptkit.BackendInspection], error) {
calls++
selections = append(selections, r.Selection)
if r.ID != "b" {
t.Errorf("backend ID %q", r.ID)
}
return commandkit.Acquired[*promptkit.BackendInspection]{Value: backend}, nil
},
}
app := newPromptTestApp(t, Command(suite), commandkit.Completion("example"))
for _, kind := range []struct {
name, id string
value any
}{{"prompt", "p", prompt}, {"profile", "p", profile}, {"backend", "b", backend}} {
for _, format := range []string{"text", "yaml", "json"} {
t.Run(kind.name+"/"+format, func(t *testing.T) {
var out bytes.Buffer
outcome := app.Execute(t.Context(), []string{"promptkit", "show", kind.name, kind.id, "--config=fixture", "--format=" + format}, commandkit.Streams{Out: &out, Err: io.Discard})
if outcome.Status != 0 {
t.Fatalf("%#v, %s", outcome, out.String())
}
var got any
var err error
if format == "json" {
d := json.NewDecoder(&out)
d.UseNumber()
err = d.Decode(&got)
} else {
got, err = commandkit.DecodeYAMLReport(&out)
}
if err != nil || !reflect.DeepEqual(got, jsonValue(t, kind.value)) {
t.Fatalf("got %#v, %v; want %#v", got, err, jsonValue(t, kind.value))
}
})
}
}
for _, selection := range selections {
if !selection.Config.Set || selection.Config.Value != "fixture" {
t.Fatalf("selection %#v", selection)
}
}
before := calls
for _, argv := range [][]string{{"promptkit", "show", "backend", "--help"}, {"__complete", "promptkit", "show", "backend", ""}} {
var help bytes.Buffer
result := app.Execute(t.Context(), argv, commandkit.Streams{Out: &help, Err: io.Discard})
if result.Status != 0 || calls != before {
t.Fatalf("%v: %#v calls %d", argv, result, calls)
}
if argv[0] == "promptkit" && !strings.Contains(help.String(), "BACKEND_ID") {
t.Fatalf("backend help = %q", help.String())
}
}
}
func TestBackendInspectionOptionalAndNativeErrorIdentity(t *testing.T) {
calls := 0
engine := adapterEngine(t)
suite := FromEngine(func(context.Context, ckconfig.Selection, struct{}, struct{}) (*promptkit.Engine, error) {
calls++
return engine, nil
})
backendOnly := Suite[struct{}, struct{}]{InspectBackend: suite.InspectBackend}
app := newPromptTestApp(t, Command(backendOnly))
result := app.Execute(t.Context(), []string{"promptkit", "show", "backend", "missing"}, commandkit.Streams{Err: io.Discard})
if !errors.Is(result.Cause, promptkit.ErrBackendNotFound) || result.Kind != commandkit.OutcomeOperational || calls != 1 {
t.Fatalf("outcome %#v calls %d", result, calls)
}
without := newPromptTestApp(t, Command(Suite[struct{}, struct{}]{InspectPrompt: suite.InspectPrompt}))
result = without.Execute(t.Context(), []string{"promptkit", "show", "backend", "missing"}, commandkit.Streams{Err: io.Discard})
if result.Kind != commandkit.OutcomeUsage || calls != 1 {
t.Fatalf("outcome %#v calls %d", result, calls)
}
}
func TestEngineInspectionReportsMatchNativeResults(t *testing.T) {
t.Setenv("PROMPTKIT_ADAPTER_TEST_KEY", "private-credential-value")
engine := adapterEngine(t)
calls := 0
native := FromEngine(func(context.Context, ckconfig.Selection, struct{}, struct{}) (*promptkit.Engine, error) {
calls++
return engine, nil
})
suite := Suite[struct{}, struct{}]{InspectPrompt: native.InspectPrompt, InspectProfile: native.InspectProfile, InspectBackend: native.InspectBackend}
app := newPromptTestApp(t, Command(suite))
ctx := t.Context()
prompt, err := engine.InspectPrompt(ctx, "briefing")
if err != nil {
t.Fatal(err)
}
profile, err := engine.InspectProfile(ctx, "default")
if err != nil {
t.Fatal(err)
}
backend, err := engine.InspectBackend(ctx, "local")
if err != nil {
t.Fatal(err)
}
for _, tc := range []struct {
name, id string
want any
}{{"prompt", "briefing", prompt}, {"profile", "default", profile}, {"backend", "local", backend}} {
t.Run(tc.name, func(t *testing.T) {
var out bytes.Buffer
result := app.Execute(ctx, []string{"promptkit", "show", tc.name, tc.id, "--format=json"}, commandkit.Streams{Out: &out, Err: io.Discard})
if bytes.Contains(out.Bytes(), []byte("private-credential-value")) {
t.Fatal("resource report exposed a credential value")
}
var got any
decoder := json.NewDecoder(&out)
decoder.UseNumber()
err := decoder.Decode(&got)
if result.Status != 0 || err != nil || !reflect.DeepEqual(got, jsonValue(t, tc.want)) {
t.Fatalf("result=%#v got=%#v err=%v want=%#v", result, got, err, jsonValue(t, tc.want))
}
})
}
if calls != 3 {
t.Fatalf("loader calls %d", calls)
}
}
func TestBackendInspectionPreservesDiagnosticsAndCause(t *testing.T) {
cause := errors.New("backend lookup failed")
app := newPromptTestApp(t, Command(Suite[struct{}, struct{}]{InspectBackend: func(context.Context, BackendRequest, struct{}, struct{}) (commandkit.Acquired[*promptkit.BackendInspection], error) {
return commandkit.Acquired[*promptkit.BackendInspection]{Diagnostics: []commandkit.Diagnostic{commandkit.MessageDiagnostic(commandkit.SeverityWarning, "loader warning")}}, cause
}}))
var diagnostics bytes.Buffer
result := app.Execute(t.Context(), []string{"promptkit", "show", "backend", "example", "--verbose"}, commandkit.Streams{Out: io.Discard, Err: &diagnostics})
if !errors.Is(result.Cause, cause) || !strings.Contains(diagnostics.String(), "loader warning") {
t.Fatalf("result=%#v diagnostics=%q", result, diagnostics.String())
}
}