Select provider conventions through backend configuration

This commit is contained in:
2026-11-22 15:27:12 +00:00
parent 0564c8086c
commit a10d1a2245

View File

@@ -126,6 +126,81 @@ must not inflate its totals. Terminal usage is an aggregate, not an additional
contribution to be added to attempt usage. These rules apply to cost as well as
token counts.
## Backend Provider Selection
Add an optional backend `provider` field identifying the API implementation
whose conventions PromptKit should use. It identifies the service contacted,
not the organization that trained the selected model. Backend IDs remain resource
identities; names, namespaces, and endpoint hostnames do not select provider
behavior.
```yaml
promptkit:
backends:
- id: research-router
provider: openrouter
endpoint: https://openrouter.ai/api/v1
api_key_env: OPENROUTER_API_KEY
```
An omitted or null provider retains ordinary OpenAI-compatible behavior without
provider-specific defaults or metadata interpretation. Initially, `openrouter`
selects the OpenRouter conventions described here. A vLLM-specific selection can
follow when its relevant response contract and representative fixtures have been
reviewed; this work does not claim vLLM-specific support.
Resolve the field through ordinary ConfigKit composition and inheritance. Carry
the selected value through configuration capture and prepared execution, expose
it in backend inspection, and declare it in the bundled OpenRouter backend.
It is PromptKit metadata, not an outbound `extra_params` entry. Selecting a
provider does not choose a model or alter the configured endpoint. An endpoint
override retains the provider selection.
### Smoke-Test Defaults And Explicit Overrides
When an authentication smoke test is explicitly requested, the selected provider
supplies its default probe convention. `provider: openrouter` supplies the
existing `openrouter_key` convention. Preserve explicit
`smoke_tests.authentication.kind` and `smoke_tests.authentication.endpoint`
settings as overrides:
- A supplied kind selects the diagnostic adapter instead of the provider default.
`kind: none` continues to declare authentication testing inapplicable.
- An omitted/null kind uses the provider default, if available. This permits an
endpoint override without repeating the provider's default kind.
- A supplied endpoint overrides the selected adapter's default destination.
An omitted/null endpoint uses that adapter's existing endpoint fallback.
An absent/null authentication block likewise leaves selection to the provider
default. Clearing the block removes its explicit overrides; `kind: none` is the
explicit way to declare authentication inapplicable despite a provider default.
Without a provider default or explicit adapter, retain existing unsupported
diagnostic behavior. Existing explicit adapters continue to work without a
provider declaration. Reachability and inference-default settings keep their
existing behavior.
These are conventions for explicitly requested diagnostics. Loading a backend,
constructing an engine, and ordinary generation do not initiate probes.
### Provider Behavior Ownership
Centralize provider-specific behavior within `internal/llm`, which already owns
OpenAI-compatible transport and OpenRouter authentication-response recognition.
Keep common request construction, credentials, HTTP execution, framing, and
common response fields shared. Provider-specific code owns additional response
metadata interpretation and authentication-probe conventions.
Use one provider-selection mechanism inside that package. Generation,
observation, and smoke-test coordination consume provider-neutral facts or invoke
generic diagnostic operations rather than repeating provider dispatch and
response interpretation. Explicit diagnostic adapter selection remains supported
at this boundary. Update the current OpenRouter-specific diagnostic collaborator
method to a generic operation as part of this consolidation.
Provider-specific files and straightforward dispatch are sufficient initially.
No new `internal/providers` package, registration framework, or broad interface
requiring every provider to implement every capability is needed.
## Optional Provider-Reported Cost
### Initial Provider Support
@@ -137,18 +212,11 @@ and [USD cost explanation](https://openrouter.ai/blog/insights/cost-vs-quality-t
Do not add `cost_details` breakdowns or combine upstream inference cost with the
top-level charge; those describe different facts and could double-count cost.
For the built-in client, select this interpretation for the resolved
`openrouter` backend ID. Do not infer USD semantics from an arbitrary backend's
field named `cost`, or add endpoint discovery or verification. An endpoint
override on that backend continues to select its OpenRouter response contract.
Other built-in backend selections report unknown cost initially. Additional
provider adapters or cost-capability configuration are not needed for this scope.
This baseline selects by exact backend identity: a separately named or
collection-qualified backend pointing to OpenRouter would report unknown cost.
Backend configuration as an alternative to identity-based selection is now an
open discussion; settle that selection before implementing provider cost support.
The contextual-observation stage does not depend on that decision.
For the built-in client, select this interpretation through `provider: openrouter`,
regardless of backend identity or namespace. Do not infer USD semantics from an
arbitrary backend's field named `cost`, or add endpoint discovery or verification.
Without an OpenRouter provider selection, built-in cost is unknown initially.
No separate cost-reporting selector is needed.
Use the cost included in the normal completion response. No additional usage
lookup, pricing request, or opt-in provider request parameter is needed.
@@ -245,6 +313,17 @@ logging infrastructure are outside this scope.
- Preserve existing explicit selection and coordinator behavior while applying
the composition and lifetime rules above. Preserve existing observer selection
when no contextual observer is selected.
- Add `provider` to the backend configuration model, typed source helpers,
capture, inspection, and execution plumbing. Update the bundled OpenRouter
backend to select it explicitly. Retain the selection through prepared runs.
- Consolidate provider-specific response and authentication-probe behavior in
`internal/llm`; keep coordination and accounting provider-neutral. Cover
provider-default probes, explicit kind and endpoint overrides, `kind: none`,
and existing explicit probes without a provider selection.
- Verify provider selection independently of backend names and namespaces,
including renamed and inherited backends, generic behavior when omitted/null,
and retention through endpoint overrides. Confirm provider metadata is not
serialized into the completion request body.
- Carry optional OpenRouter cost through response decoding, internal/public
conversions, successful run results, attempt and terminal events, and existing
group summaries. Keep result cost available without registering an observer.
@@ -263,7 +342,7 @@ logging infrastructure are outside this scope.
failures or cancellation, and known zero versus unknown or incomplete counts.
Use offline fixtures and injected clients; live inference is unnecessary.
- Cover fractional USD values, explicit zero, omitted/null and unusable cost,
non-OpenRouter built-in selection, optional injected-client cost, mixed known
generic built-in selection, optional injected-client cost, mixed known
and unknown totals, repair/retry aggregation, and duplicate group registration.
Use tolerant comparisons where floating-point addition requires them; do not
turn telemetry into an exact-decimal guarantee.
@@ -272,6 +351,9 @@ logging infrastructure are outside this scope.
availability, units, and completeness. Document OpenRouter cost decoding in
the [integration contract](../integrations/openai-compatible-chat.md#response-handling),
and update internal observation and model-client documentation for changed flows.
Document `provider`, its smoke-test defaults and override semantics in the
framework format reference and relevant public GoDoc; update inspection and
diagnostic documentation with the implemented behavior.
## Implementation Planning Guidance
@@ -279,12 +361,14 @@ Plan two separately reviewable stages:
1. Contextual observer selection, including ordinary, prepared, coordinated,
and nested execution boundaries and the existing group collector.
2. Provider-reported cost, carried through decoding, accounting, public results,
events, and group summaries, with the corresponding documentation.
2. Backend provider selection and consolidation within `internal/llm`, including
smoke-test defaults and explicit overrides, followed by provider-reported cost
through decoding, accounting, public results, events, and group summaries.
Include the corresponding configuration and API documentation.
The second stage is more extensive than the context helper. Resolve the backend
selection discussion before finalizing that stage; avoid coupling delivery of
contextual observation to provider cost support.
The second stage is more extensive than the context helper and may be divided
into focused implementation units. Backend provider selection is decided above;
avoid coupling delivery of contextual observation to provider cost support.
Today, successful results accumulate generation responses in the runner, while
optional observation recorders account for dispatched attempts. Adding cost to