base-layer/commandkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

Demonstrate PromptKit catalogs and smoke tests

Browse Source
This commit is contained in:
Eric Rakestraw
2026-10-17 19:06:05 +00:00
parent 079576361b
commit 35e958c210
4 changed files with 127 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

41
integrations/ckprompt/examples/composed/README.md
View File

@@ -12,6 +12,10 @@ go run . config diff --scope app --config fixtures/before.yml --against-config f
go run . config diff --scope promptkit --config fixtures/before.yml --against-config fixtures/after.yml --campaign spring
go run . promptkit list prompts --config fixtures/before.yml --campaign spring
go run . promptkit list profiles --config fixtures/after.yml --campaign autumn --format yaml
go run . promptkit list backends --campaign spring
go run . promptkit list namespaces --campaign spring
go run . promptkit list collections --campaign spring
go run . promptkit list prompts --namespace field-notes --campaign spring
go run . promptkit show prompt briefing --campaign spring --format yaml
go run . promptkit show profile spring --campaign spring
go run . promptkit requirements show briefing --campaign spring
@@ -19,6 +23,8 @@ go run . promptkit requirements show briefing --campaign spring --var mode=full
go run . promptkit requirements validate briefing --campaign spring --input topic=fixtures/topic.txt
go run . promptkit requirements validate briefing --campaign spring --input topic=fixtures/topic.txt --var mode=full --input notes=fixtures/notes.txt --format yaml
go run . promptkit requirements validate briefing --campaign spring
go run . promptkit test text_inference --campaign spring --prompt briefing --profile spring
go run . promptkit test all --campaign spring --prompt briefing --profile spring
go run . session restore session-42 --campaign spring
```
@@ -38,11 +44,36 @@ on both sides and reports the native engine change under the spring profile.
The application input check requires `--limit=0` when `--campaign` is supplied.
Catalogs use the same selection-aware loader: the prompt catalog includes
`briefing` version `1`, and the profile catalog includes the selected `autumn`
profile. The example's shared run cache keys successful results by the complete
supplied configuration/overlay selection and the supplied campaign, including
presence. The campaign override is applied through ConfigKit before loading.
These are choices for this example's loader and do not define a CommandKit cache
policy.
profile. Prompt inspection uses an ID only; its optional version is descriptive
metadata in the report, not a selector. The synthetic `field-notes` collection
provides one backend, profile, and prompt for collection discovery and native
namespace filtering. The example's shared run cache keys successful results by
the complete supplied configuration/overlay selection and the supplied
campaign, including presence. The campaign override is applied through ConfigKit
before loading. These are choices for this example's loader and do not define a
CommandKit cache policy.
Smoke testing is explicitly wired with `ckprompt.WithSmokeTesting` and its
optional request builder is nil, so CommandKit forwards the selected native
request. Applications that need to adjust the request can provide a typed
builder that starts with the seed and returns the chosen native request:
```go
definitions = ckprompt.WithSmokeTesting(definitions, loadEngine,
func(ctx context.Context, seed ckprompt.SmokeRequest, input *inspectionInput, state *runCache) (promptkit.SmokeTestRequest, error) {
request := seed.Request
request.ProfileID = "spring"
return request, nil
})
```
The example injects a small in-memory `promptkit.LLMClient` through its shared
run state. Catalog, inspection, and requirements commands do not call it;
inference smoke commands do. No model service or credentials are required.
Native HTTP checks may retain unsupported or not-applicable observations because
the example does not configure a network probe. An aggregate smoke command can
return an operational outcome for a native diagnostic while still delivering
the complete aggregate report.
The first requirements command reports incomplete discovery: PromptKit records
the conditional `notes` input and a missing `mode` issue. Supplying `mode=full`