promptkit/docs/roadmap/inspect-backend.md

# Backend Inspection And Configuration API Roadmap

## Purpose And Status

Add `Engine.InspectBackend` so applications can retrieve one backend's effective
definition by its runtime identity, and expose existing backend inheritance
through `BackendConfig` and `BackendSource`. These additions are selected for
planning; implementation has not begun. The open decisions below remain to be
resolved. Typed prompt construction is a separate discussion item, and listing
return-type consistency is a possible later cleanup.

The consumer benefit is backend-detail reporting without selecting an arbitrary
profile or reconstructing domain settings from ConfigKit paths and list indexes.
For example, Scriptorium could show a backend's endpoint, credential source,
capacity settings, provider defaults, and diagnostic configuration. Promptkit
owns this reusable interpretation; downstream applications own presentation,
disclosure, CLI commands, and exit codes.

API symmetry alone does not justify additions. Prioritize backend inspection
with the recommended missing-backend error, followed by typed backend
inheritance. Both expose behavior or information Promptkit already owns, with
no new source composition, validation framework, or execution lifecycle.

## Relationship To Existing APIs

| API | Responsibility |
| --- | --- |
| `ListBackends` | Discover captured backend identities. |
| Proposed `InspectBackend` | Describe one backend's effective settings by ID. |
| `InspectProfile` | Resolve a profile's backend-completed execution settings. |
| `InspectConfiguration` | Inspect effective configuration and source provenance. |
| `SmokeTest` | Execute an explicitly requested diagnostic and report evidence. |

See the [consumer guide](../consumers/pkg-promptkit.md#discover-available-resources-and-collections)
for the implemented inspection and discovery boundaries, and the
[architecture policy](../policy/architecture.md) for library responsibilities.

## Proposed Backend Inspection Contract

```go
func (e *Engine) InspectBackend(
    ctx context.Context,
    backendID string,
) (*BackendInspection, error)
```

Use a dedicated public result rather than returning `BackendConfig`.
`BackendConfig` is an input type whose pointers express omitted and explicit
values; inspection describes the normalized effective definition after
composition and defaulting. Do not expose internal domain types.

The proposed result includes:

| Information | Meaning |
| --- | --- |
| `BackendID` | Full selected runtime identity. |
| `Endpoint` | Captured backend base endpoint. |
| `APIKeyEnv` | Configured environment-variable name, never its resolved value. |
| `ExtraParams` | Backend provider defaults, before profile or request overrides. |
| `ConcurrencyLimit` | Configured generation concurrency limit. |
| `QueueCapacity` | Effective queue capacity, including construction-time defaults. |
| `SmokeTests` | Optional diagnostic settings, including the inference default profile reference. |

Scalar fields describe effective values rather than reproducing source-field
presence. Diagnostic optionality must remain expressible; its representation
depends on open decision 1. Exact nested type names are implementation choices.
Follow existing typed inspection conventions without adding a stable JSON
serialization contract for this feature. Applications can define their own
output formats.

### Selection And Errors

Follow the existing namespace selection rules: a scoped engine view qualifies
slashless IDs with its backend namespace; qualified IDs pass through unchanged.
An unscoped view looks up the supplied ID directly. Return the full selected ID.
Do not add fallback search across namespaces, implicit backend selection, or
special handling for bundled backends. Use the existing ID rules, including
case sensitivity, rather than introducing new normalization.

A nil or zero-value engine should match `ErrInvalidConfig`; a blank or otherwise
invalid selection should match `ErrInvalidRequest`. Cancellation should retain
the context error, and failures should return no partial result. The public
classification for a missing backend is open decision 2.

### Captured State And Ownership

Inspect the engine's captured backend definition without reloading sources,
reading credential values, preparing a prompt, acquiring capacity, or making
provider calls. Engine construction already validates effective definitions;
inspection should not repeat that work or create an additional acceptance path.

Capacity fields describe configuration, not current occupancy or queue length.
Diagnostic settings describe configuration, not provider availability, key
acceptance, or inference readiness. In particular, report a smoke-test default
profile reference without resolving or validating that profile merely to inspect
the backend. Its selection-time behavior remains owned by `SmokeTest`.

Return independently owned mutable values, including nested provider maps and
diagnostic settings. Mutating a result must not affect the engine or another
inspection. Reuse existing ownership operations and avoid consecutive copies
whose only purpose is to cross private helper boundaries.

Provider defaults should retain their values, consistent with typed target
inspection. They are application data and may contain sensitive content;
the result is not automatically safe for logging or disclosure. Do not resolve
credentials or silently substitute ConfigKit's redacted reporting values for
the typed settings. Operator provenance and redacted configuration reporting
remain available through `InspectConfiguration`.

## Typed Backend Inheritance

Add `Extends *string` to `BackendConfig`, matching `ProfileConfig.Extends`.
When non-nil, `BackendSource` should encode it as `extends: {id: value}` in the
backend record. Nil omits the field. Preserve the supplied value and let
ConfigKit resolve and validate inheritance through the existing source-loading
path, just as it does for profiles.

The purpose is to let programmatic callers derive a backend variant without
switching from `BackendSource` to a raw mapping. The helper must not resolve a
parent, fill inherited fields, or become a second inheritance implementation.
It remains a source constructor rather than an engine registry mutation API.

This exposes existing sibling inheritance, not a new configuration feature.
Parent IDs select siblings in the same root or collection resource list;
namespace views do not qualify these configuration references. Existing
same-ID replacement, omitted-field, and explicit-null behavior remain owned by
ConfigKit composition. Raw mappings remain available when callers need explicit
null or other generic configuration controls that the typed helper cannot express.
See the [inheritance reference](../formats.md#resource-inheritance) and the
existing [profile source helper](../../profiles.go).

## Scope Of API Consistency

Backends, profiles, and prompts already share source layering, collection
membership, namespace selection, sibling inheritance, and construction-time
definition validation. The gaps addressed here are in public convenience and
inspection APIs, not in those underlying configuration capabilities.

Inspection remains purpose-specific: `InspectPrompt` reports prompt interface
metadata, `InspectProfile` reports backend-completed execution settings, and
`InspectBackend` will describe an effective backend. This work does not turn
them into uniform complete-definition retrieval APIs. Expanding prompt inspection
to return descriptions, message templates, or session templates should follow a
concrete consumer need rather than the new backend result's shape.

`ListPrompts` currently returns `[]PromptIdentity`, whose only field is `ID`;
`ListProfiles` and `ListBackends` return `[]string`. At a suitable breaking-change
window, prefer considering `[]string` for all three over adding thin identity
wrappers for backends and profiles. That would simplify consumer code, but
requires a deliberate compatibility decision and migration instructions. Keep
the existing return types in this work; listing consistency does not block the
backend additions.

Collections, schemas, and engine configuration remain distinct concepts rather
than additional peers requiring identical resource APIs. Prompt requirement
inspection and backend capacity/diagnostic settings retain their specialized
roles. Profile/backend hashes are not added for symmetry; the existing
[deferred fingerprint proposal](deferred.md#semantic-execution-target-fingerprints)
owns that separate question.

## Open Decisions

### 1. Declared Diagnostic Settings Or Derived Destinations?

**Recommendation:** Return normalized captured diagnostic settings, preserving
omitted endpoint overrides and the default profile reference. An absent override
continues to mean that the diagnostic applies its existing default when run.
Use dedicated result types where needed to express this without treating the
whole inspection result as configuration input.

This keeps inspection independent of provider-specific destination derivation
and execution selection. It also avoids a second place that must reproduce
authentication-adapter rules. Construction-time defaults already captured in
the backend, such as queue capacity, still appear as effective values.

**Alternative:** Also expose derived reachability and authentication URLs. This
would help applications display the destination before invoking a diagnostic,
but would require sharing the adapter's derivation logic and clearly separating
declared settings from derived results. It must not imply that the selected
client supports HTTP diagnostics or that the destination is reachable. Add this
only if consumers need that extra information.

### 2. How Should An Unknown Backend Be Classified?

**Recommendation:** Add public `ErrBackendNotFound` for this explicit lookup,
matching the role of `ErrProfileNotFound` and `ErrPromptNotFound`. The registry
already has an internal backend-not-found sentinel. Map it at the facade so
applications can distinguish a missing backend from malformed input without
inspecting text or internal packages.

Limit this addition to the new inspection operation unless existing callers
justify a broader change; do not automatically reclassify smoke-test or engine
construction errors.

**Alternative:** Classify an absent ID as `ErrInvalidRequest`. This avoids
another exported sentinel but combines a well-formed missing identity with an
invalid selection. Consumers needing different presentation would then require
an inventory lookup or additional error detail. A sentinel is the smaller
consumer-facing solution when that distinction matters.

### 3. Should Typed Prompt Construction Be Added In This Round?

**Recommendation:** Keep `PromptConfig` and `PromptSource` as a separate
discussion item until a programmatic prompt-authoring use case establishes
their value. File sources and ConfigKit mappings already support prompt
definitions. The absence of a helper is a Go convenience gap, not a missing
configuration capability, and does not block the backend additions.

**Alternative:** Add a typed prompt source alongside the existing backend and
profile helpers. It would improve discoverability, compiler assistance, and
consistency for applications constructing prompts in Go. Its cost is a larger
public input model covering messages, inline/file content, inputs, output
contracts, session templates, and omission semantics. If selected, define that
scope separately, including sibling inheritance and source context for relative
content files, while retaining ConfigKit's ownership of composition and raw
mapping support for explicit null. Do not implement a second prompt parser or
validation path merely to provide the helper.

## Implementation Scope And Verification

1. Resolve backend inspection decisions 1–2 and define the result and GoDoc.
   Decide separately whether to expand this round for typed prompt construction;
   the recommended backend scope does not depend on that expansion.
2. Connect namespace-aware engine selection to the existing captured backend
   lookup and project the result at the public facade. Introduce only the
   private collaborators needed for this path; no new backend cache or loader.
3. Add focused public tests for root and namespaced identities, bundled and
   caller-defined backends, effective capacity defaults, provider maps, optional
   diagnostic settings, errors, cancellation, and independent mutable results.
   Cover a missing smoke default profile and absent credential environment
   variable to establish that inspection does not perform execution readiness
   checks. Use controlled local fixtures; no live provider calls are needed.
4. Add `BackendConfig.Extends` and encode it through `BackendSource`. Verify
   an inherited backend through real ConfigKit composition, including an
   overridden field and a retained parent field. Reuse existing inheritance
   coverage for generic missing-parent, cycle, and replacement semantics rather
   than retesting ConfigKit's entire feature set.
5. Update the consumer guide and relevant internal documentation when the API
   lands, with a short `ListBackends` followed by `InspectBackend` example.
   Include typed backend inheritance guidance. Record the additive APIs and
   selected error behavior in release notes.

Reuse meaningful existing namespace, configuration, and ownership coverage;
avoid tests that merely mirror private projection helpers. Run checks appropriate
to the implementation under the [testing policy](../policy/testing.md).

Completion means consumers can inspect a backend without selecting a profile or
navigating raw configuration, and derive a backend through the typed source
helper without constructing a raw mapping. Existing execution, inspection,
diagnostic, and configuration-composition behavior remains intact. No separate
provider capability registry, live capacity monitor, or backend health API is
included. Typed prompt construction and listing return-type changes are not
completion requirements for the selected backend scope.