Document PromptKit catalogs and smoke testing

README.md

@@ -15,8 +15,8 @@ constructing them, and returns outcomes without exiting the process. Its
 | Module | Package | Provides |
 | --- | --- | --- |
 | `gitea.maximumdirect.net/eric/CommandKit` | `commandkit` | Typed commands, bindings, execution, outcomes, help, and version. |
-| `gitea.maximumdirect.net/eric/CommandKit/integrations/ckconfig` | `ckconfig` | ConfigKit-backed `config` scopes and inspection commands. |
+| `gitea.maximumdirect.net/eric/CommandKit/integrations/ckconfig` | `ckconfig` | ConfigKit-backed `config` scopes, including configuration source evidence. |
-| `gitea.maximumdirect.net/eric/CommandKit/integrations/ckprompt` | `ckprompt` | PromptKit engine configuration/comparison, prompt/profile catalogs, named inspection, requirements discovery, and concrete request validation. |
+| `gitea.maximumdirect.net/eric/CommandKit/integrations/ckprompt` | `ckprompt` | PromptKit engine configuration/comparison, five catalogs, named inspection, requirements discovery, concrete request validation, and native smoke tests. |
 
 Importing an integration does not install commands. Pass its declaration to
 `commandkit.New` or `App.Register` to install it. The integrations depend on the

docs/cli.md

@@ -175,14 +175,36 @@ sources:
     kind: file
     underlay_index: 0
     imported_from: null
+    destination: /server
+    placement: mapping
+    destination_unresolved: false
+    version_check:
+      path: /version
+      version:
+        present: true
+        null: false
+        text: "1"
+      location:
+        source_id: source-1
+        line: 2
+        column: 1
+      redacted: false
 ```
 
 Each source includes `id`, `label`, `path`, `layer`, `kind`, `underlay_index`,
-and `imported_from`. The import location is null when absent; otherwise it has
+and `imported_from`, plus the native destination pointer, placement, unresolved
-`source_id`, `line`, and `column`. Owning values and source order are preserved,
+destination flag, and nullable `version_check`. Placement is `mapping` or
-including empty fields, zero coordinates, and synthetic sources without paths.
+`sequence_item`. The import location is null when absent; otherwise it has
-An empty inventory is `sources: []`. Text output begins with the scope heading
+`source_id`, `line`, and `column`. Version evidence preserves `path`, `present`,
-and then prints the same source fields, quoting labels and paths.
+`null`, `text`, optional source `location`, and `redacted` as returned by
+ConfigKit. Owning values and source order are preserved, including empty fields,
+zero coordinates, false and null observations, and synthetic sources without
+paths. An absent version check is null. An empty inventory is `sources: []`.
+Text output begins with the scope heading and prints the same source fields,
+quoting labels and paths. These fields project the selected ConfigKit source
+records; CommandKit does not reload files or rerun version checks. See the
+[ConfigKit v0.8.0 inspection API](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/137769f3f18a370c68444242fb5d88e620e307e5/inspection.go)
+for native evidence definitions.
 
 ### Explain a configuration path
 
@@ -306,38 +328,35 @@ existing incomplete-report delivery outcome and can leave partial output.
 ## PromptKit definition inspection
 
 Applications can install prompt and profile inspection independently with
-`ckprompt.Command(suite)`, passing the resulting declaration to core. The
+`ckprompt.Command(suite)`. The `ckprompt.FromEngine(loadEngine)` adapter
-`ckprompt.FromEngine(loadEngine)` adapter supplies callbacks that acquire an
+supplies these callbacks alongside native requirements discovery and the five
-engine only after input checks pass and inspection is reached. The loader
+catalogs. Its loader receives invocation context, configuration selection,
-receives invocation context, selection, input, and state; its implementation
+typed input, and shared state. Direct suite callbacks retain their own
-controls whether an engine is constructed or reused. Direct suite callbacks
+implementation and lifecycle.
-retain their own implementation and lifecycle.
 
-CommandKit's inspection path does not execute prompts, contact model providers,
+Prompt inspection uses an ID only. PromptKit returns any descriptive version
-or resolve provider credential values. Application acquisition and direct
+metadata as an optional report field; there is no `--version` flag for selecting
-callback effects remain application-owned. Prompt definitions and profile
+a prompt version. The prompt format and native selection behavior remain owned by
-resolution remain owned by
+[PromptKit v0.16.0](https://gitea.maximumdirect.net/eric/promptkit/src/commit/9178f3eb91c2b2138caa78e22be93c048b4bc992/docs/consumers/pkg-promptkit.md).
-[PromptKit's inspection API](https://gitea.maximumdirect.net/eric/promptkit/src/commit/4b8169da286917fde7875425bb32b799b12c6af3/engine.go).
 
 ```text
-example promptkit show prompt customer/summary --version 2 --format yaml
+example promptkit show prompt customer/summary --format yaml
 example promptkit show prompt customer/summary
 example promptkit show profile customer-support --format yaml
 ```
 
 Both commands require one ID operand and accept `--config`, `--overlay`, and
-`--format text|yaml` (default `text`). Prompt inspection also accepts singleton
+`--format text|yaml` (default `text`). These commands have no built-in
-`--version`, defaulting to the empty string. PromptKit owns version selection:
+`--scope` selection; their namespace is `promptkit`.
-an omitted version works for a single-version ID, while an ambiguous ID is an
-operational error. These commands have no built-in `--scope` selection; their
-namespace is `promptkit`.
 
-Prompt YAML has these fields. Input order is preserved and an empty input list is
+Prompt YAML preserves native input order and uses an optional `version` field.
-`[]`. The prompt hash is an opaque owning-library equality value.
+The key is omitted when PromptKit returns no version and is retained when the
+version is present, including an empty string. The prompt hash is an opaque
+owning-library equality value.
 
 ```yaml
 prompt_id: customer/summary
-prompt_version: "2"
+version: "2"
 prompt_hash: opaque-value
 default_profile_id: customer-support
 inputs:
@@ -376,47 +395,41 @@ api_key_required: true
 ```
 
 Text uses labeled fields and includes nested `extra_params` values. A missing ID
-is a usage error. Blank, unknown, and ambiguous definition selections reach the
+is a usage error. Definition selection reaches the owning inspection API and
-owning inspection API and produce operational results with supported public
+preserves supported error identity and cancellation.
-error identity. Cancellation is also operational: standard handlers check the
-caller's context at inspection boundaries, and PromptKit can return cancellation
-from its own inspection methods. Supported cancellation identity is preserved.
 
 ## PromptKit requirements
 
-When `Suite.InspectRequirements` is present,
+When `Suite.InspectRequirements` is present, `ckprompt.Command(suite)` installs
-`ckprompt.Command(suite)` installs `promptkit requirements show`. It accepts one
+`promptkit requirements show`. It accepts one prompt ID and uses PromptKit's
-required prompt ID and uses PromptKit's native selection and discovery rules:
+native selection and discovery rules:
 
 ```text
 example promptkit requirements show briefing --campaign spring
-example promptkit requirements show briefing --version 2 --profile editorial --var mode=full
+example promptkit requirements show briefing --profile editorial --var mode=full
 example promptkit requirements show briefing --session-id session-42 --var note= --format yaml
 ```
 
-The command accepts these flags:
+The command accepts:
 
 | Flag | Values | Default | Meaning |
 | --- | --- | --- | --- |
 | `--config` | path string | omitted | Select the configuration input; explicit presence is retained. |
 | `--overlay` | path string | omitted | Select one overlay; explicit presence is retained. Repeated occurrences use the last value. |
 | Suite selectors | application-defined | per selector | The flags captured from `Suite.Selectors`. |
-| `--version` | string | empty | Pass the prompt version to PromptKit; empty uses its native version-selection rule. |
+| `--profile` | string | empty | Pass the profile ID to PromptKit; empty uses its native selection rule. |
-| `--profile` | string | empty | Pass the profile ID to PromptKit; empty uses its native profile-selection rule. |
 | `--session-id` | string | empty | Pass session context to PromptKit. |
 | `--var` | repeatable `KEY=VALUE` | none | Supply caller-variable context for native discovery. |
 | `--format` | `yaml` | `yaml` | Select the requirements report format. Other formats are usage errors for this operation. |
 
-The `promptkit` namespace identifies the PromptKit operation, so there is no
+The `promptkit` namespace identifies the operation, so there is no built-in
-built-in `--scope` flag. The version, profile, and session fields use native
+`--scope` flag. An explicitly empty profile or session value retains PromptKit's
-empty-string semantics: an explicit empty string has the same meaning as
+native empty-string meaning. Requirements inspection does not require a profile
-omission. PromptKit owns their interpretation, including default version and
+to be execution-ready. See the
-profile selection, session context, and selection errors. Requirements
+[PromptKit v0.16.0 consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/commit/9178f3eb91c2b2138caa78e22be93c048b4bc992/docs/consumers/pkg-promptkit.md)
-inspection does not require a profile to be execution-ready. See the pinned
+for native configuration, selection, and requirements behavior. The built-in
-[PromptKit requirements API](https://gitea.maximumdirect.net/eric/promptkit/src/commit/143c6e9b3a675872117b0471e2ff8bcb25055a54/requirements.go)
+flags other than `--var` are singleton bindings; when repeated, the last value
-for the owning discovery contract. The built-in flags other than `--var` are
+is used. Suite selectors retain their declared binding behavior.
-singleton bindings; when repeated, the last value is used. Suite selectors retain
-their declared binding behavior.
 
 Each `--var` occurrence must contain `=`. CommandKit splits at the first
 separator and preserves the exact key and remaining value. Empty keys and
@@ -424,20 +437,21 @@ values, whitespace, commas, and additional `=` characters are retained. All
 occurrences are parsed; duplicate keys use the last value, while any malformed
 occurrence is a usage error even if a later assignment is valid.
 
-The successful report is YAML with selected identities, native completeness,
+The YAML report contains selected identities, native completeness, and input,
-and the input, variable, and issue inventories. False values and empty
+variable, and issue inventories. False values and empty inventories are
-inventories are retained. Input metadata preserves null versus an empty string.
+retained. Input metadata preserves null versus an empty string. Variable entries
-Variable entries contain `name`, `kind`, `supplied`, and `explanation`.
+contain `name`, `kind`, `supplied`, and `explanation`. `supplied` reports native
-`supplied` reports native exact-key presence, including an explicitly empty
+exact-key presence, including an explicitly empty value, independently of
-value, independently of `kind` and `complete`. Explanations and issue descriptions
+`kind` and `complete`. Explanations and issue descriptions are upstream-owned
-are upstream-owned human-readable summaries, not machine-readable details.
+human-readable summaries, not machine-readable details. The optional `version`
+key is omitted when absent and retained when supplied by PromptKit.
 
 For a prompt with the template `{{if .mode}}{{input "notes"}}{{end}}` and no
-input declarations, omitting `mode` can produce this partial report:
+input declarations, omitting `mode` can produce a partial report:
 
 ```yaml
 prompt_id: briefing
-prompt_version: "2"
+version: "2"
 profile_id: editorial
 complete: false
 inputs:
@@ -459,28 +473,24 @@ issues:
     description: A missing variable prevents selecting a dependency-bearing branch.
 ```
 
-PromptKit owns the inventory, classifications, explanations, completeness,
+PromptKit owns the inventory, classifications, explanations, completeness, and
-and structured issues. Completeness means discovery covered the relevant
+structured issues. Successfully acquired and delivered partial reports return
-dependencies; it does not mean inputs or variables are supplied, artifacts are
+status `0`; structured issues do not independently fail the command. Provider
-valid, credentials are available, or execution is ready. A complete report can
+errors, native selection errors, cancellation, and report delivery failures use
-contain unsupplied required variables, and an incomplete report can still be
+the operational path (status `1`) with supported cause identity. Syntax,
-useful. Successfully acquired and delivered partial reports return status `0`;
+operand, variable-assignment, and format errors use usage status `2`.
-structured issues remain in YAML and do not independently fail the command.
-Provider errors, native selection errors, cancellation, and report delivery
-failures use the operational path (status `1`) with supported cause identity.
-Syntax, operand, variable-assignment, and format errors use usage status `2`.
 
 ### Concrete request validation
 
 When `Suite.ValidateRequest` is present, `ckprompt.Command(suite)` also installs
 `promptkit requirements validate`. It can be installed without discovery, and
 discovery can remain installed without validation. The command accepts one
-required prompt ID and passes a concrete request to PromptKit's native
+prompt ID and passes a concrete request to PromptKit's native
 `Engine.ValidateRequest` preparation path:
 
 ```text
 example promptkit requirements validate briefing --campaign spring --input topic=topic.txt
-example promptkit requirements validate briefing --version 2 --profile editorial --session-id session-42 --var mode=full --input notes=notes.txt
+example promptkit requirements validate briefing --profile editorial --session-id session-42 --var mode=full --input notes=notes.txt
 ```
 
 The command accepts:
@@ -490,7 +500,6 @@ The command accepts:
 | `--config` | path string | omitted | Select the configuration input; explicit presence is retained. |
 | `--overlay` | path string | omitted | Select one overlay; explicit presence is retained. Repeated occurrences use the last value. |
 | Suite selectors | application-defined | per selector | The flags captured from `Suite.Selectors`. |
-| `--version` | string | empty | Pass the prompt version to PromptKit. |
 | `--profile` | string | empty | Pass the profile ID to PromptKit. |
 | `--session-id` | string | empty | Pass session context to PromptKit. |
 | `--var` | repeatable `KEY=VALUE` | none | Supply caller variables for the concrete request. |
@@ -498,24 +507,17 @@ The command accepts:
 | `--format` | `yaml` | `yaml` | Select the success confirmation format. Other formats are usage errors. |
 
 The built-in flags other than repeatable `--var` and `--input` are singleton
-bindings; when repeated, the last value is used. Suite selectors retain their
+bindings; when repeated, the last value is used. Explicit empty `--config` or
-declared binding behavior. Explicit empty `--config` or `--overlay` values keep
+`--overlay` values keep their supplied-presence information. The profile and
-their supplied-presence information.
+session fields use native empty-string semantics. PromptKit owns their
+interpretation and selection errors.
 
-The version, profile, and session fields use native empty-string semantics:
+Each `--var` and `--input` occurrence must contain `=`. CommandKit splits at the
-an explicit empty string has the same meaning as omission. PromptKit owns their
+first separator and preserves the exact key/name and remaining value/path.
-interpretation and selection errors. The `promptkit` namespace identifies the
+Empty variable keys and values and empty input names and paths are passed to
-operation, so there is no built-in `--scope` flag.
+native validation. File paths become `promptkit.File(PATH)` references without
-
+being read, normalized, expanded, or rebased during argument binding. Duplicate
-Each `--var` and `--input` occurrence must contain `=`. CommandKit splits at
+keys or names use the last value, but every occurrence is parsed.
-the first separator and preserves the exact key/name and remaining value/path.
-Empty variable keys and values are passed to PromptKit. Empty input names and
-paths are also passed to native validation. File paths are translated to
-`promptkit.File(PATH)` references without reading, normalizing, expanding, or
-rebasing them during argument binding. Duplicate keys or names use the last
-value, but all occurrences are parsed, so any malformed occurrence is a usage
-error even if a later assignment is valid. Syntax and path values after shell
-processing are otherwise preserved.
 
 On success, the YAML confirmation contains only the requested prompt ID and
 `valid: true`:
@@ -525,46 +527,64 @@ prompt_id: briefing
 valid: true
 ```
 
-Validation delegates concrete request preparation to PromptKit. Depending on
+Validation delegates concrete request preparation to PromptKit. Native
-the request and configuration, native preparation may resolve prompt/profile
+preparation may resolve prompt/profile resources, load artifact and schema
-resources, load artifact and schema resources, render messages and session
+resources, render messages and session context, and perform applicable
-context, and perform applicable configuration and credential checks. Native
+configuration and credential checks. It does not make a model call or retain a
-errors are returned as encountered rather than as an exhaustive inventory.
+prepared execution snapshot. A successful report describes this preparation
-Validation does not make a model call or retain a prepared execution snapshot.
+invocation; it does not certify provider acceptance or guarantee later
-A successful report means preparation passed for this invocation; it does not
+execution. See PromptKit's
-certify provider acceptance or guarantee that later execution will succeed.
+[`Engine.ValidateRequest` API](https://gitea.maximumdirect.net/eric/promptkit/src/commit/9178f3eb91c2b2138caa78e22be93c048b4bc992/engine.go).
-See PromptKit's [v0.15.0 `Engine.ValidateRequest` API](https://gitea.maximumdirect.net/eric/promptkit/src/commit/635674c066d0b432187f96e72b9c03d085443ac5/engine.go)
-for the native contract.
 
 Builder, engine-loading, and native validation errors use the operational path
 (status `1`) with supported error identity and acquired diagnostics retained.
 Malformed syntax, missing operands, and unsupported formats use usage status
-`2`. Input-check failures keep their existing shared handling. Successful
+`2`. Successful diagnostics and report-delivery failures follow the common
-diagnostics and report-delivery failures follow the common delivery contract.
+delivery contract.
 
 ### PromptKit catalogs
 
-An installed `ckprompt.Command(suite)` exposes the catalog commands supplied by
+An installed `ckprompt.Command(suite)` exposes each catalog whose callback is
-the suite:
+present:
 
 ```text
 example promptkit list prompts
 example promptkit list profiles --config settings.yml --campaign spring
+example promptkit list backends --namespace team
+example promptkit list namespaces --kind prompt --kind profile
+example promptkit list collections
 ```
 
-Each command accepts `--config`, `--overlay`, and applicable selectors from
+All catalogs accept `--config`, `--overlay`, applicable flags from
-`Suite.Selectors`. It takes no ID or version operand and has no `--version` or
+`Suite.Selectors`, and `--format yaml` (the default). Other formats are usage
-`--scope` selector. `--format` defaults to `yaml`; `--format yaml` is accepted,
+errors. Prompt, profile, and backend listings accept namespace selection:
-and other formats are usage errors.
 
-Each invocation writes one bare YAML sequence. Prompt entries have explicit
+| Binding | Effect |
-string `id` and `version` fields; profile entries are strings. Provider order and
+| --- | --- |
-exact identity values are preserved. For example:
+| omitted `--namespace` and `--all-namespaces` | Forward PromptKit's native default namespace filter. |
+| `--namespace team` | Forward the named namespace filter. |
+| `--namespace ""` | Select the unqualified namespace. |
+| `--all-namespaces` | Forward the all-namespaces filter. |
+| `--all-namespaces=false` | Do not select an override; a supplied namespace still applies, otherwise native default is used. |
+
+The namespace binding is a singleton and its last occurrence is used.
+Supplying a namespace while `--all-namespaces=true` is active is a usage error
+before provider work. Namespace inventory accepts repeatable `--kind KIND`;
+omitting `--kind` forwards PromptKit's native default kind selection. Repeated
+values are forwarded in order. Collections have no namespace or kind filter.
+
+Each operation writes one bare YAML sequence, retaining provider order and
+native identities:
+
+- Prompt entries are `{id, version?}`. The optional `version` key is omitted
+  when absent and retained for a present empty or nonempty value.
+- Profiles, backends, and namespaces are strings.
+- Collection entries are `{namespace, version?}`, where `version` projects
+  PromptKit's optional release version.
+- Nil and allocated-empty inventories both render as `[]`.
 
 ```yaml
-- id: briefing
+- id: team/briefing
-  version: "1"
-- id: briefing
   version: "2"
 ```
 
@@ -573,11 +593,155 @@ exact identity values are preserved. For example:
 - spring
 ```
 
-Successful nil and allocated-empty inventories both render as `[]`. Catalog
+A catalog result is an inventory, not a readiness check or execution result.
-membership and ordering follow the selected provider, and a listed definition
+The owning provider controls membership and order. Namespace filters are passed
-is not a readiness check or execution result. PromptKit owns the meaning of its
+to PromptKit without local membership interpretation. See the
-native listing APIs; custom providers own their results. Applications retain
+[PromptKit v0.16.0 consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/commit/9178f3eb91c2b2138caa78e22be93c048b4bc992/docs/consumers/pkg-promptkit.md)
-their own disclosure choices for catalog data.
+for native resource and selection definitions.
+
+### PromptKit smoke testing
+
+When `Suite.SmokeTest` is present, `ckprompt.Command(suite)` installs
+`promptkit test KIND`. The special kind `all` runs the four concrete operations
+sequentially in this order: `reachability`, `authentication`, `text_inference`,
+and `structured_output`. Both forms use the same single-operation callback;
+each aggregate call changes the seed kind before invoking it. The command
+accepts:
+
+```text
+example promptkit test text_inference --campaign spring --prompt briefing --profile spring
+example promptkit test all --campaign spring --prompt briefing --profile spring
+example promptkit test authentication --backend local
+```
+
+| Flag | Values | Default | Meaning |
+| --- | --- | --- | --- |
+| `--config`, `--overlay` | path strings | omitted | Select PromptKit configuration; supplied presence is retained. |
+| Suite selectors | application-defined | per selector | Captured flags from `Suite.Selectors`. |
+| `--backend` | string | empty | Native backend ID. |
+| `--prompt` | string | empty | Native prompt ID. |
+| `--profile` | string | empty | Native profile ID. |
+| `--endpoint` | string | omitted | Override the execution endpoint. |
+| `--model` | string | omitted | Override the execution model. |
+| `--temperature` | float | omitted | Override execution temperature. |
+| `--max-tokens` | integer | omitted | Override execution maximum tokens. |
+| `--top-p` | float | omitted | Override execution top-p. |
+| `--timeout-seconds` | integer | omitted | Override the execution timeout. |
+| `--service-tier` | string | omitted | Override the execution service tier. |
+| `--reasoning-effort` | string | omitted | Override execution reasoning effort. |
+| `--api-key-env` | string | omitted | Supply an API-key environment-variable name to native configuration. |
+| `--transient-retries` | integer | omitted | Set the native inference transient retry count. |
+| `--format` | `yaml` | `yaml` | Select the report format. Other formats are usage errors. |
+
+The execution flags populate PromptKit's native execution override. The
+`--api-key-env` value is a variable name, not a credential value. Application
+selectors, loader policy, credentials, and client wiring remain application
+owned. The `WithSmokeTesting` helper accepts an optional typed builder. A nil
+builder forwards the seed; a supplied builder's complete native request is
+authoritative. The helper retains a native report when PromptKit returns both a
+report and an error.
+
+A single report projects the native smoke report into YAML. It preserves native
+false, unknown/null, and incomplete values and does not add a readiness score.
+The report includes `kind`, `supported`, `applicable`, `completed`, `reason`,
+`target`, `facts`, `timing`, `attempts`, `usage`, and `failure`. Its target
+contains selected IDs, optional `version`, selection source, endpoint, effective
+settings, and settings-presence flags. Attempt entries retain their own facts,
+timing, failure, and token-usage evidence. Durations use Go duration strings;
+token counts preserve `value`, `known`, and `complete`.
+
+Here is a native projection as rendered by the command:
+
+```yaml
+kind: reachability
+supported: false
+applicable: null
+completed: true
+reason: http_diagnostics_unavailable
+target:
+  backend_id: ""
+  profile_id: spring
+  prompt_id: briefing
+  version: "1"
+  default_source_backend_id: ""
+  selection_source: profile
+  endpoint: https://example.invalid/v1
+  effective_settings:
+    endpoint: https://example.invalid/v1
+    model: spring-model
+    temperature: 0
+    max_tokens: 0
+    top_p: 0
+    timeout_seconds: 600
+    service_tier: ""
+    reasoning_effort: ""
+    api_key_env: ""
+    extra_params:
+      campaign: spring
+  settings_presence:
+    temperature: false
+    max_tokens: false
+    top_p: false
+    timeout_seconds: false
+facts:
+  credentials_supplied: null
+  endpoint_reachable: null
+  credentials_accepted: null
+  response_received: null
+  generation_completed: null
+  text_returned: null
+  expected_text_matched: null
+  structured_output_requested: null
+  valid_json: null
+  schema_matched: null
+timing:
+  started_at: 2026-10-17T19:05:36Z
+  duration: 35.685µs
+  capacity_wait: null
+  request_duration: null
+  retry_delay: null
+  evaluation_duration: null
+attempts: []
+usage:
+  prompt_tokens: {value: 0, known: true, complete: true}
+  completion_tokens: {value: 0, known: true, complete: true}
+  total_tokens: {value: 0, known: true, complete: true}
+  cached_tokens: {value: 0, known: true, complete: true}
+  cache_write_tokens: {value: 0, known: true, complete: true}
+failure: ""
+```
+
+Aggregate output is one YAML mapping with ordered `tests` entries. Each entry
+has selected `kind`, nullable `report`, and optional `error` text. The inner
+report keeps its native `kind`. This cancellation example shows two attempted
+operations with no reports; later kinds are absent because cancellation stops
+the sequence:
+
+```yaml
+tests:
+  - kind: reachability
+    report: null
+    error: "reachability operation failed"
+  - kind: authentication
+    report: null
+    error: "context canceled"
+```
+
+A successful delivered single operation returns status `0`. For `all`, all
+four callbacks must return without error for status `0`. Native false, unknown,
+unsupported, or incomplete observations do not independently change the
+outcome. Operation errors use operational status `1`; aggregate runs continue
+after individual errors, retain their reports and ordered error text, and join
+causes for supported `errors.Is`/`errors.As` access. Cancellation stops later
+operations and preserves results already acquired. Report encoding or delivery
+failures use the common incomplete-report delivery outcome and retain operation
+causes where applicable. Syntax, operand, unsupported format, and conflicting
+namespace selection are usage status `2`.
+
+Native configuration, selection, and diagnostic meanings remain owned by
+PromptKit. See the [v0.16.0 consumer guide](https://gitea.maximumdirect.net/eric/promptkit/src/commit/9178f3eb91c2b2138caa78e22be93c048b4bc992/docs/consumers/pkg-promptkit.md),
+[smoke API](https://gitea.maximumdirect.net/eric/promptkit/src/commit/9178f3eb91c2b2138caa78e22be93c048b4bc992/smoke_diagnostics.go),
+and [engine API](https://gitea.maximumdirect.net/eric/promptkit/src/commit/9178f3eb91c2b2138caa78e22be93c048b4bc992/engine.go).
 
 ## Outcome status
 

docs/consumers/guide.md

@@ -11,7 +11,7 @@ through `commandkit.New` or `App.Register`.
 | --- | --- |
 | `gitea.maximumdirect.net/eric/CommandKit` | The application needs typed command declarations and common CLI behavior. |
 | `gitea.maximumdirect.net/eric/CommandKit/integrations/ckconfig` | The application wants ConfigKit-backed configuration scopes and reports. |
-| `gitea.maximumdirect.net/eric/CommandKit/integrations/ckprompt` | The application wants PromptKit engine configuration/comparison, prompt/profile catalogs, named inspection, requirements discovery, or concrete request validation. |
+| `gitea.maximumdirect.net/eric/CommandKit/integrations/ckprompt` | The application wants PromptKit engine configuration/comparison, five catalogs, named inspection, requirements discovery, request validation, or native smoke testing. |
 
 The two integrations are independent optional imports. `ckprompt` uses the
 public scope and error contracts from `ckconfig`; importing it does not add the
@@ -22,7 +22,7 @@ core-only, ConfigKit-only, and composed applications.
 ## Go compatibility
 
 All three modules declare a consumer minimum of Go 1.26.0. The root module also
-selects ReleaseKit v0.4.0 as a Go tool, which makes that tool's module graph part
+selects ReleaseKit v0.4.2 as a Go tool, which makes that tool's module graph part
 of the root consumer module requirements even though production packages do not
 import ReleaseKit. The [release policy](../policy/release.md#consumer-minimum-and-release-toolchain)
 defines the support assumption. Maintainers use Go 1.26.8 for validation and
@@ -126,17 +126,20 @@ engine's captured configuration. Add that scope to a `ckconfig.Suite` to expose
 it under `config` with validate/show/sources/explain and native diff operations;
 it does not install PromptKit catalog, definition, or requirements commands.
 
-`ckprompt.FromEngine(loadEngine)` supplies five callbacks: prompt and profile
+`ckprompt.FromEngine(loadEngine)` supplies eight callbacks: prompt, profile,
-catalogs, named prompt and profile inspection, and native requirements
+backend, namespace, and collection catalogs; named prompt and profile
-discovery. Concrete request validation is a sixth, independently optional
+inspection; and native requirements discovery. Concrete request validation and
-capability and is added explicitly with `ckprompt.WithRequestValidation`.
+smoke testing are separate capabilities, added explicitly with
-`ckprompt.Command(suite)` creates only operations whose callbacks are present;
+`ckprompt.WithRequestValidation` and `ckprompt.WithSmokeTesting`. The suite has
-discovery and validation share `promptkit requirements` but can be installed
+ten independently optional callbacks in total. `ckprompt.Command(suite)` creates
-independently. A composed suite can keep the standard five and add validation:
+only operations whose callbacks are present; discovery, validation, and smoke
+testing can be installed independently. A composed suite can enable both
+additional operations:
 
 ```go
 suite := ckprompt.FromEngine(loadEngine)
 suite = ckprompt.WithRequestValidation(suite, loadEngine, buildRequest)
+suite = ckprompt.WithSmokeTesting(suite, loadEngine, nil)
 suite.NewInput = newInput
 suite.NewState = newState
 suite.Selectors = selectors
@@ -165,8 +168,9 @@ absent, so explicitly supplied empty values stay meaningful. Applications that
 validate and then execute a request are encouraged to share their request
 construction logic between those paths.
 
-All six suite capabilities are independently optional. For catalogs without
+The eight catalog and inspection callbacks and the two explicit additions are
-named or requirements inspection, narrow the suite before command capture:
+independently optional. For catalogs without named or requirements inspection,
+narrow the suite before command capture:
 
 ```go
 definitions := ckprompt.FromEngine(loadEngine)
@@ -176,7 +180,7 @@ definitions.InspectRequirements = nil
 definitionCommand := ckprompt.Command(definitions)
 ```
 
-Each selected inspection, requirements, validation, or catalog operation
+Each selected inspection, requirements, validation, smoke, or catalog operation
 prepares one input, applies its captured selectors, checks that input, and
 initializes one state value when those steps are reached. The bundled validation
 adapter delegates concrete request preparation to PromptKit and carries its
@@ -195,13 +199,19 @@ PromptKit preparation, which can resolve prompt/profile resources, load
 artifacts and schemas, render messages, and perform applicable configuration
 and credential checks. It does not make a model call or certify that a later
 execution will succeed. Application loaders and direct callbacks retain their
-own effects. The [CLI reference](../cli.md#promptkit-definition-inspection)
+own effects. Smoke testing explicitly invokes one native PromptKit diagnostic
-and [requirements contract](../cli.md#promptkit-requirements) own operator
+or all four in order. Native inference checks use the configured public
-syntax and reports.
+`promptkit.LLMClient`; applications retain ownership of client and probe
+configuration. A builder may customize the seeded native request, and its
+complete return value is authoritative. The [CLI reference](../cli.md#promptkit-smoke-testing),
+[definition inspection](../cli.md#promptkit-definition-inspection), and
+[requirements contract](../cli.md#promptkit-requirements) own operator syntax
+and reports.
 
 The [composed example](../../integrations/ckprompt/examples/composed/README.md)
 combines app and PromptKit config scopes, comparison and catalogs, named and
-requirements inspection, and an application-owned `session restore` command.
+requirements inspection, request validation, smoke testing, and an
+application-owned `session restore` command.
 
 ## Outcomes and ownership
 
@@ -214,7 +224,8 @@ process.
 Go declarations and GoDoc own exact exported contracts. ConfigKit owns
 configuration decoding, validation, and provenance. PromptKit owns prompt
 formats, profile resolution, and inspection details. See the
-[ConfigKit inspection API](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/dc09959cca4e105f49ec316c6d4ba585bd2340bd/inspection.go),
+[ConfigKit v0.8.0 inspection API](https://gitea.maximumdirect.net/eric/ConfigKit/src/commit/137769f3f18a370c68444242fb5d88e620e307e5/inspection.go),
-[PromptKit engine methods](https://gitea.maximumdirect.net/eric/promptkit/src/commit/4b8169da286917fde7875425bb32b799b12c6af3/engine.go),
+[PromptKit engine methods](https://gitea.maximumdirect.net/eric/promptkit/src/commit/9178f3eb91c2b2138caa78e22be93c048b4bc992/engine.go),
-[configuration inspection](https://gitea.maximumdirect.net/eric/promptkit/src/commit/4b8169da286917fde7875425bb32b799b12c6af3/configuration_inspection.go),
+[configuration inspection](https://gitea.maximumdirect.net/eric/promptkit/src/commit/9178f3eb91c2b2138caa78e22be93c048b4bc992/configuration_inspection.go),
-and [inspection types](https://gitea.maximumdirect.net/eric/promptkit/src/commit/4b8169da286917fde7875425bb32b799b12c6af3/types.go).
+[smoke diagnostics](https://gitea.maximumdirect.net/eric/promptkit/src/commit/9178f3eb91c2b2138caa78e22be93c048b4bc992/smoke_diagnostics.go),
+and [PromptKit v0.16.0 consumer guidance](https://gitea.maximumdirect.net/eric/promptkit/src/commit/9178f3eb91c2b2138caa78e22be93c048b4bc992/docs/consumers/pkg-promptkit.md).

docs/development.md

@@ -131,7 +131,8 @@ or publishing.
 ## Current public examples
 
 The maintained consumers compile with their owning module suites and run without
-external services:
+external services. The PromptKit smoke examples use local fixtures and the
+example's configured client:
 
 ```sh
 go run ./examples/core workspace greet --count=0 --tag=first --tag second
@@ -142,11 +143,16 @@ go run ./examples/core workspace greet --count=0 --tag=first --tag second
 (cd integrations/ckprompt/examples/composed && go run . config diff --scope promptkit --config fixtures/before.yml --against-config fixtures/after.yml --campaign spring)
 (cd integrations/ckprompt/examples/composed && go run . promptkit list prompts --config fixtures/before.yml --campaign spring)
 (cd integrations/ckprompt/examples/composed && go run . promptkit list profiles --config fixtures/after.yml --campaign autumn --format yaml)
+(cd integrations/ckprompt/examples/composed && go run . promptkit list backends --namespace team)
+(cd integrations/ckprompt/examples/composed && go run . promptkit list namespaces --kind prompt --kind profile)
+(cd integrations/ckprompt/examples/composed && go run . promptkit list collections)
 (cd integrations/ckprompt/examples/composed && go run . promptkit show prompt briefing --campaign spring --format yaml)
 (cd integrations/ckprompt/examples/composed && go run . promptkit requirements show briefing --campaign spring)
 (cd integrations/ckprompt/examples/composed && go run . promptkit requirements show briefing --campaign spring --var mode=full --format yaml)
 (cd integrations/ckprompt/examples/composed && go run . promptkit requirements validate briefing --campaign spring --input topic=fixtures/topic.txt)
 (cd integrations/ckprompt/examples/composed && go run . promptkit requirements validate briefing --campaign spring --input topic=fixtures/topic.txt --var mode=full --input notes=fixtures/notes.txt --format yaml)
+(cd integrations/ckprompt/examples/composed && go run . promptkit test reachability --campaign spring --prompt briefing --profile spring)
+(cd integrations/ckprompt/examples/composed && go run . promptkit test all --campaign spring --prompt briefing --profile spring)
 (cd integrations/ckprompt/examples/composed && go run . session restore session-42 --campaign spring)
 ```
 

docs/internal/overview.md

@@ -47,10 +47,12 @@ initialized input, while one suite-owned state value is shared by an invocation'
 acquisitions or comparison callback.
 
 `integrations/ckprompt` owns PromptKit engine loading, configuration-scope
-adaptation and native comparison, plus optional prompt/profile catalogs, named
+adaptation and native comparison, plus optional prompt/profile/backend/namespace/
-inspection, native requirements discovery, and concrete request validation.
+collection catalogs, named inspection, native requirements discovery,
-Validation delegates request preparation to PromptKit and is independently
+concrete request validation, and native smoke testing. Validation and smoke
-opted into a suite. It uses public core and ckconfig contracts plus PromptKit's
+testing delegate to PromptKit and are independently opted into a suite. Smoke
+operations use the application's native engine and its configured client or
+probes. The integration uses public core and ckconfig contracts plus PromptKit's
 public APIs. It does not add a configuration family when its declaration is
 installed; callers compose `ckprompt.ConfigScope` with `ckconfig.Command`
 separately when they want configuration operations alongside PromptKit
@@ -70,8 +72,10 @@ graph isolation.
 ConfigKit owns configuration decoding, validation, source history, and path
 explanations. PromptKit owns its captured engine configuration, prompt
 definitions, profile inheritance, named inspection, requirements discovery,
-and concrete request validation semantics. CommandKit adapts those public
+and concrete request validation, and native smoke-test semantics. CommandKit
-contracts without maintaining parallel schemas or invoking model generation.
+adapts those public contracts without maintaining parallel schemas. Explicit
+native smoke operations may use the configured client or probes; other
+inspection, catalog, and request-validation paths do not make model calls.
 Native request validation may load resources and artifacts, render messages,
 and perform applicable configuration or credential checks through PromptKit.
 Application loaders and domain behavior remain application owned.

docs/roadmap/deferred.md

@@ -33,7 +33,8 @@ operator or consumer need and an appropriate ownership/behavior decision:
 - JSON reports, shell completion, and output-file management or reloadable export.
 - Configuration mutation, generic value assignment, and exhaustive field-origin
   inventories.
-- Credential diagnostics and live provider probes.
+- Credential diagnostics beyond PromptKit's native smoke checks and additional
+  live provider probes.
 - Additional component identity registration and domain artifact catalogs.
 - Additional library families, including possible FeedKit integration; families
   can define suitable verbs without requiring comparison or configuration scopes.

docs/roadmap/implementation.md

@@ -1,6 +1,6 @@
 # Dependency upgrades, discovery, and smoke testing implementation plan
 
-Status: Stages 1–16 complete; Stage 17 is next.
+Status: Stages 1–17 complete; Stage 18 is next.
 
 ## Authority and baseline
 
@@ -829,6 +829,16 @@ is required.
 **Complete when:** current contracts describe implemented behavior under their
 canonical owners and historical references remain correctly scoped.
 
+**Completed:** 2026-10-17. GoDoc, module summaries, the README, CLI reference,
+consumer guide, development commands, and internal inventory now describe the
+adopted source evidence, eight `FromEngine` callbacks, five catalogs, optional
+request validation, and native smoke behavior. Smoke examples and catalog
+commands were run against the maintained composed consumer; `go doc` confirmed
+the exported helper and request documentation. Local Markdown links/fragments,
+stale upstream revision references, and whitespace checks passed. Deferred-work
+scope and historical implementation-plan links were reconciled without changing
+historical API examples or release notes.
+
 ## Stage 18: Verify the complete target and finish
 
 1. Format all changed Go files and inspect the full diff for unrelated changes,

docs/roadmap/promptkit-validation.md

@@ -3,7 +3,8 @@
 Status: accepted adoption and feature roadmap; implementation complete.
 Command placement, adapter composition, request construction, artifact syntax,
 and initial presentation are settled. This document defines the target and
-required work; the [implementation plan](implementation.md) sequences the work.
+required work; the [implementation plan](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/38b49f29fb3a8f22ff02528d2a0ed413e5c6ab1e/docs/roadmap/implementation.md)
+sequences the historical implementation work.
 
 ## Purpose
 

docs/roadmap/remediation.md

@@ -8,7 +8,8 @@ architectural redesign was identified as necessary.
 Reviewed implementation commits after planning baseline `3218222`, through
 `420483b2d4659862626422b0577e39bc0a4c1a2d`, against the
 [accepted feature roadmap](promptkit-validation.md),
-[implementation plan](implementation.md), applicable ADRs, and repository policy.
+[implementation plan](https://gitea.maximumdirect.net/eric/CommandKit/src/commit/38b49f29fb3a8f22ff02528d2a0ed413e5c6ab1e/docs/roadmap/implementation.md),
+applicable ADRs, and repository policy.
 The review covered production changes, new tests, the maintained consumer,
 dependency metadata/attribution, and updated documentation.
 

integrations/ckconfig/README.md

@@ -27,8 +27,10 @@ scopes, selectors, input checks, and state shared by the selected acquisitions
 or comparison in a config invocation. `ckconfig.FromConfigKit` adapts a typed
 ConfigKit loader. The `config` command family exposes scope discovery and the
 validate, show, sources, explain, and diff operations supported by the declared
-scopes. Comparison returns ConfigKit's native records to the YAML report path;
+scopes. Source reports include native destination, placement, unresolved
-the [CLI reference](https://gitea.maximumdirect.net/eric/CommandKit/src/branch/main/docs/cli.md#compare-configuration-selections)
+destination, and source version-check evidence. Comparison returns ConfigKit's
+native records to the YAML report path; the
+[CLI reference](https://gitea.maximumdirect.net/eric/CommandKit/src/branch/main/docs/cli.md#compare-configuration-selections)
 owns its operator details.
 
 The maintained [simple example](examples/simple) shows a complete consumer.

integrations/ckconfig/doc.go

@@ -1,4 +1,6 @@
 // Package ckconfig provides CommandKit's optional ConfigKit integration. Its
 // declarations install the standard config command family through the core
-// commandkit package; importing ckconfig alone installs no commands.
+// commandkit package. Source reports preserve ConfigKit's destination,
+// placement, and version-check evidence; importing ckconfig alone installs no
+// commands.
 package ckconfig

integrations/ckprompt/README.md

@@ -1,9 +1,9 @@
 # CommandKit ckprompt
 
 `ckprompt` is the optional PromptKit integration for CommandKit. It adapts
-PromptKit engine configuration, native comparison, prompt/profile catalogs,
+PromptKit engine configuration, native comparison, five catalogs, named
-named inspection, and prompt requirements discovery. Importing it does not
+inspection, requirements discovery and validation, and native smoke tests.
-install either command family.
+Importing it does not install either command family.
 
 ## Module
 
@@ -19,14 +19,17 @@ for callback, input, state, and acquisition contracts.
 Add `ckprompt.ConfigScope(loadEngine)` to the scopes passed to
 `ckconfig.Command` when the application wants PromptKit engine configuration
 and native comparison in the `config` family. Install
-`ckprompt.Command(ckprompt.FromEngine(loadEngine))` separately for prompt/profile
+`ckprompt.Command(ckprompt.FromEngine(loadEngine))` separately for catalogs,
-catalogs, named inspections, and requirements discovery. `FromEngine` supplies
+named inspections, and requirements discovery. `FromEngine` supplies eight
-all five PromptKit-backed callbacks; applications may narrow, wrap, or replace
+PromptKit-backed callbacks: three inspections and five catalogs. Applications
-them before capture or provide custom callbacks with `ckprompt.Suite`.
+may narrow, wrap, or replace them before capture or provide custom callbacks
-Requirements is independently optional. CommandKit-owned inspection and
+with `ckprompt.Suite`. Add concrete request validation and smoke testing
-listing code does not execute prompts or contact model providers; application
+explicitly with `WithRequestValidation` and `WithSmokeTesting`. Inspection,
-loaders and direct callbacks retain their own effects. See the
+requirements, and catalog operations do not execute prompts or contact model
-[CLI reference](https://gitea.maximumdirect.net/eric/CommandKit/src/branch/main/docs/cli.md#promptkit-requirements)
+providers; application loaders and callbacks retain their own effects. Smoke
+testing invokes the selected native PromptKit operation; the
+application retains ownership of client and configuration choices. See the
+[CLI reference](https://gitea.maximumdirect.net/eric/CommandKit/src/branch/main/docs/cli.md#promptkit-smoke-testing)
 for operator commands and report semantics. The maintained
 [composed example](examples/composed) shows complete application wiring.
 

integrations/ckprompt/adapters.go

@@ -73,20 +73,20 @@ type RequirementsRequest struct {
 // Factories are not invoked during construction, installation, help, or version,
 // and need not run after failure.
 type Suite[I, S any] struct {
-	// NewInput is called once when definition, requirements, validation, or
+	// NewInput is called once when inspection, requirements, validation, smoke,
-	// catalog input preparation is reached, before applying selectors and
+	// or catalog input preparation is reached, before applying selectors and
 	// calling CheckInput.
 	// If nil, input is the zero value of I, including nil for pointer types.
 	NewInput func() I
-	// NewState is called once when an inspection, validation, or catalog
+	// NewState is called once when an inspection, validation, smoke, or catalog
 	// callback is reached, after input checks pass. If nil, state is the zero
 	// value of S, including nil for pointer types.
 	NewState func() S
-	// Selectors declares the original typed flags accepted by definition,
+	// Selectors declares the original typed flags accepted by inspection,
-	// requirements, validation, and catalog commands.
+	// requirements, validation, smoke, and catalog commands.
 	Selectors []commandkit.Flag[I]
-	// CheckInput validates selected definition, requirements, validation, or catalog input
+	// CheckInput validates selected inspection, requirements, validation, smoke,
-	// before state creation.
+	// or catalog input before state creation.
 	CheckInput func(ckconfig.Selection, I) error
 	// InspectPrompt inspects the selected prompt after input and state preparation.
 	InspectPrompt func(context.Context, PromptRequest, I, S) (commandkit.Acquired[*promptkit.PromptInspection], error)
@@ -106,7 +106,8 @@ type Suite[I, S any] struct {
 	// SmokeTest runs one concrete native smoke-test operation after input and
 	// state preparation. It is optional; nil omits only the smoke-test command.
 	// The acquisition retains a native report even when the callback also
-	// returns an error.
+	// returns an error. For `test all`, CommandKit calls the captured callback
+	// sequentially for each selected kind after preparing input and state once.
 	SmokeTest func(context.Context, SmokeRequest, I, S) (commandkit.Acquired[*promptkit.SmokeTestReport], error)
 	// ListPrompts lists prompt identities after input and state preparation.
 	// It is optional independently of the other catalogs; nil omits only this
@@ -273,13 +274,14 @@ func promptKitContextError(ctx context.Context) error {
 // The requirements callback forwards RequirementsRequest.Inspection to
 // Engine.InspectRequirements. Concrete request validation is an independent
 // capability and can be composed explicitly with WithRequestValidation. Smoke
-// testing is also an explicit capability installed by its helper.
+// testing is also an explicit capability installed with WithSmokeTesting.
 // FromEngine does not call the loader during construction. Each selected
 // operation passes its context, supplied selection, initialized input, and
 // shared invocation state to the loader before calling PromptKit's public
 // listing or inspection method. Applications can narrow, wrap, or replace these
 // callbacks before Command captures them. A nil loader is retained as an
-// adapter issue.
+// adapter issue. Each callback returns loader diagnostics with the native
+// operation's value and error.
 func FromEngine[I, S any](load func(context.Context, ckconfig.Selection, I, S) (*promptkit.Engine, error)) Suite[I, S] {
 	if load == nil {
 		return Suite[I, S]{issues: []error{errors.New("PromptKit engine loader is nil")}}

integrations/ckprompt/doc.go

@@ -1,5 +1,5 @@
 // Package ckprompt provides optional PromptKit configuration comparison,
-// definition catalogs, named inspections, requirements discovery, and concrete
+// prompt catalogs, named inspections, requirements discovery and validation,
-// request validation for CommandKit. Importing ckprompt installs neither
+// and native smoke testing for CommandKit. Importing ckprompt installs neither
 // family; callers compose its declarations with core and ckconfig explicitly.
 package ckprompt

integrations/ckprompt/smoke.go

@@ -10,8 +10,9 @@ import (
 	promptkit "gitea.maximumdirect.net/eric/promptkit"
 )
 
-// SmokeRequest carries the configuration selection and seed request for one
+// SmokeRequest pairs the configuration selection with the seed request for one
-// concrete PromptKit smoke-test operation.
+// concrete PromptKit smoke-test operation. It is passed to the optional
+// builder and the Suite.SmokeTest callback.
 type SmokeRequest struct {
 	// Selection identifies the configuration sources used to load the engine.
 	Selection ckconfig.Selection
@@ -25,7 +26,8 @@ type SmokeRequest struct {
 // invocation context. Its complete returned request is passed to PromptKit;
 // without a builder, the seed is forwarded unchanged. Reports and errors from
 // PromptKit are both retained. A nil loader is kept as a suite wiring issue and
-// does not install a smoke-test callback.
+// does not install a smoke-test callback. The callback also retains engine
+// loading diagnostics with the report and error.
 func WithSmokeTesting[I, S any](
 	suite Suite[I, S],
 	load func(context.Context, ckconfig.Selection, I, S) (*promptkit.Engine, error),