promptkit/docs/roadmap/inspect-backend.md

Backend Inspection Roadmap

Purpose And Status

Add Engine.InspectBackend so applications can retrieve one backend's effective definition by its runtime identity. The direction is accepted for planning; implementation has not begun. The open decisions below remain to be resolved.

The consumer benefit is backend-detail reporting without selecting an arbitrary profile or reconstructing domain settings from ConfigKit paths and list indexes. For example, Scriptorium could show a backend's endpoint, credential source, capacity settings, provider defaults, and diagnostic configuration. Promptkit owns this reusable interpretation; downstream applications own presentation, disclosure, CLI commands, and exit codes.

API symmetry alone does not justify the addition. The scope is a small typed view over captured information that Promptkit already owns, with no new source composition, validation framework, or execution lifecycle.

Relationship To Existing APIs

API	Responsibility
ListBackends	Discover captured backend identities.
Proposed InspectBackend	Describe one backend's effective settings by ID.
InspectProfile	Resolve a profile's backend-completed execution settings.
InspectConfiguration	Inspect effective configuration and source provenance.
SmokeTest	Execute an explicitly requested diagnostic and report evidence.

See the consumer guide for the implemented inspection and discovery boundaries, and the architecture policy for library responsibilities.

Proposed Public Contract

func (e *Engine) InspectBackend(
    ctx context.Context,
    backendID string,
) (*BackendInspection, error)

Use a dedicated public result rather than returning BackendConfig. BackendConfig is an input type whose pointers express omitted and explicit values; inspection describes the normalized effective definition after composition and defaulting. Do not expose internal domain types.

The proposed result includes:

Information	Meaning
BackendID	Full selected runtime identity.
Endpoint	Captured backend base endpoint.
APIKeyEnv	Configured environment-variable name, never its resolved value.
ExtraParams	Backend provider defaults, before profile or request overrides.
ConcurrencyLimit	Configured generation concurrency limit.
QueueCapacity	Effective queue capacity, including construction-time defaults.
SmokeTests	Optional diagnostic settings, including the inference default profile reference.

Scalar fields describe effective values rather than reproducing source-field presence. Diagnostic optionality must remain expressible; its representation depends on open decision 1. Exact nested type names are implementation choices. Follow existing typed inspection conventions without adding a stable JSON serialization contract for this feature. Applications can define their own output formats.

Selection And Errors

Follow the existing namespace selection rules: a scoped engine view qualifies slashless IDs with its backend namespace; qualified IDs pass through unchanged. An unscoped view looks up the supplied ID directly. Return the full selected ID. Do not add fallback search across namespaces, implicit backend selection, or special handling for bundled backends. Use the existing ID rules, including case sensitivity, rather than introducing new normalization.

A nil or zero-value engine should match ErrInvalidConfig; a blank or otherwise invalid selection should match ErrInvalidRequest. Cancellation should retain the context error, and failures should return no partial result. The public classification for a missing backend is open decision 2.

Captured State And Ownership

Inspect the engine's captured backend definition without reloading sources, reading credential values, preparing a prompt, acquiring capacity, or making provider calls. Engine construction already validates effective definitions; inspection should not repeat that work or create an additional acceptance path.

Capacity fields describe configuration, not current occupancy or queue length. Diagnostic settings describe configuration, not provider availability, key acceptance, or inference readiness. In particular, report a smoke-test default profile reference without resolving or validating that profile merely to inspect the backend. Its selection-time behavior remains owned by SmokeTest.

Return independently owned mutable values, including nested provider maps and diagnostic settings. Mutating a result must not affect the engine or another inspection. Reuse existing ownership operations and avoid consecutive copies whose only purpose is to cross private helper boundaries.

Provider defaults should retain their values, consistent with typed target inspection. They are application data and may contain sensitive content; the result is not automatically safe for logging or disclosure. Do not resolve credentials or silently substitute ConfigKit's redacted reporting values for the typed settings. Operator provenance and redacted configuration reporting remain available through InspectConfiguration.

Open Decisions

1. Declared Diagnostic Settings Or Derived Destinations?

Recommendation: Return normalized captured diagnostic settings, preserving omitted endpoint overrides and the default profile reference. An absent override continues to mean that the diagnostic applies its existing default when run. Use dedicated result types where needed to express this without treating the whole inspection result as configuration input.

This keeps inspection independent of provider-specific destination derivation and execution selection. It also avoids a second place that must reproduce authentication-adapter rules. Construction-time defaults already captured in the backend, such as queue capacity, still appear as effective values.

Alternative: Also expose derived reachability and authentication URLs. This would help applications display the destination before invoking a diagnostic, but would require sharing the adapter's derivation logic and clearly separating declared settings from derived results. It must not imply that the selected client supports HTTP diagnostics or that the destination is reachable. Add this only if consumers need that extra information.

2. How Should An Unknown Backend Be Classified?

Recommendation: Add public ErrBackendNotFound for this explicit lookup, matching the role of ErrProfileNotFound and ErrPromptNotFound. The registry already has an internal backend-not-found sentinel. Map it at the facade so applications can distinguish a missing backend from malformed input without inspecting text or internal packages.

Limit this addition to the new inspection operation unless existing callers justify a broader change; do not automatically reclassify smoke-test or engine construction errors.

Alternative: Classify an absent ID as ErrInvalidRequest. This avoids another exported sentinel but combines a well-formed missing identity with an invalid selection. Consumers needing different presentation would then require an inventory lookup or additional error detail. A sentinel is the smaller consumer-facing solution when that distinction matters.

Implementation Scope And Verification

1. Resolve the two public-contract decisions and define the result and GoDoc.
2. Connect namespace-aware engine selection to the existing captured backend lookup and project the result at the public facade. Introduce only the private collaborators needed for this path; no new backend cache or loader.
3. Add focused public tests for root and namespaced identities, bundled and caller-defined backends, effective capacity defaults, provider maps, optional diagnostic settings, errors, cancellation, and independent mutable results. Cover a missing smoke default profile and absent credential environment variable to establish that inspection does not perform execution readiness checks. Use controlled local fixtures; no live provider calls are needed.
4. Update the consumer guide and relevant internal documentation when the API lands, with a short ListBackends followed by InspectBackend example. Record the additive API and selected error behavior in release notes.

Reuse meaningful existing namespace, configuration, and ownership coverage; avoid tests that merely mirror private projection helpers. Run checks appropriate to the implementation under the testing policy.

Completion means consumers can inspect a backend without selecting a profile or navigating raw configuration, while existing execution, inspection, diagnostic, and configuration-composition behavior remains intact. No separate provider capability registry, live capacity monitor, or backend health API is included.