diff --git a/docs/roadmap/invocation-scoped-token-usage.md b/docs/roadmap/invocation-scoped-token-usage.md index 3d07af0..463ca94 100644 --- a/docs/roadmap/invocation-scoped-token-usage.md +++ b/docs/roadmap/invocation-scoped-token-usage.md @@ -1,7 +1,6 @@ # Invocation-Scoped Execution Observation -Status: agreed direction; default-provider and vLLM scope under discussion; -implementation pending. +Status: agreed direction; vLLM scope under discussion; implementation pending. ## Purpose @@ -147,15 +146,19 @@ promptkit: Normalize the final effective provider value by trimming whitespace and lowercasing. Omitted, null, empty, and whitespace-only values are equivalent. Reject an unsupported nonempty normalized value as an ordinary configuration -error rather than silently using generic behavior. `openrouter` selects the -OpenRouter conventions described here. +error rather than silently using generic behavior. The agreed provider selections +are `openai-compatible` and `openrouter`; the possible `vllm` addition remains +under discussion. -The current generic fallback retains ordinary OpenAI-compatible behavior without -provider-specific defaults or metadata interpretation. Before finalizing this -roadmap, discuss making this fallback an explicit lowest-common-denominator -provider and adding a `vllm` selection. Their names, supported conventions, and -final scope remain to be settled; this document does not yet select a vLLM -contract. +Omitted/null/blank provider values select the same effective behavior as explicit +`provider: openai-compatible`. This is a runtime fallback, not an explicitly +sourced configuration value. Backend inspection retains the distinction between +an explicit provider declaration and an omitted/null/blank declaration. + +The `openrouter` provider reuses the shared OpenAI-compatible foundation and adds +the OpenRouter response-metadata and diagnostic conventions below. Before +finalizing this roadmap, discuss the conventions and scope of a `vllm` selection; +this document does not yet select a vLLM contract. Resolve the field through ordinary ConfigKit composition and inheritance. Carry the selected value through configuration capture and prepared execution, expose @@ -164,6 +167,35 @@ It is PromptKit metadata, not an outbound `extra_params` entry. Selecting a provider does not choose a model or alter the configured endpoint. An endpoint override retains the provider selection. +### Default OpenAI-Compatible Provider + +`openai-compatible` names PromptKit's current supported text Chat Completions +conventions. Preserve the existing shared behavior documented in the +[integration contract](../integrations/openai-compatible-chat.md), including: + +- JSON POST requests to `/chat/completions` relative to the configured base; +- existing optional bearer-credential handling, including explicitly required + credential settings when selected; +- text messages, explicitly selected generation controls and structured-output + requests, and configured extra parameters; +- first-choice text extraction, existing supported token counts and their + presence-aware observation; and +- existing response validation, error handling, timeouts, cancellation, and + transient retries. + +This selection supplies no default endpoint, model, API-key environment variable, +or sampling settings. It does not interpret monetary cost or provide a default +authentication-probe adapter. Without an explicit authentication adapter, report +the existing unsupported diagnostic outcome rather than assuming authentication +is inapplicable. Existing reachability and inference diagnostics and explicit +authentication adapters remain available. + +"Lowest common denominator" identifies the shared protocol foundation; it does +not remove existing optional features or promise every backend supports them. +Continue sending caller-selected supported settings and return backend errors +normally, without capability negotiation or fallback requests. This is the +existing text Chat Completions scope, not a claim to support the full OpenAI API. + ### Smoke-Test Defaults And Explicit Overrides When an authentication smoke test is explicitly requested, the selected provider @@ -240,8 +272,10 @@ top-level charge; those describe different facts and could double-count cost. For the built-in client, select this interpretation through `provider: openrouter`, regardless of backend identity or namespace. Do not infer USD semantics from an arbitrary backend's field named `cost`, or add endpoint discovery or verification. -Without an OpenRouter provider selection, built-in cost is unknown initially. -No separate cost-reporting selector is needed. +Built-in cost interpretation is an extension unique to `openrouter` initially; +`openai-compatible` leaves cost unknown even if a response contains `usage.cost`. +No separate cost-reporting selector is needed. Aggregation remains +provider-neutral after the model-client boundary supplies interpreted USD facts. Use the cost included in the normal completion response. No additional usage lookup, pricing request, or opt-in provider request parameter is needed. @@ -349,8 +383,9 @@ logging infrastructure are outside this scope. including renamed and inherited backends, equivalent omitted/null/blank values, trimming and case normalization, rejection of unsupported nonempty values, and retention through endpoint overrides. Confirm provider metadata is not - serialized into the completion request body. Use the generic fallback selected - when the remaining provider discussion is complete. + serialized into the completion request body. Verify that explicit + `openai-compatible` and omitted/null/blank selection preserve the same existing + shared behavior without manufacturing explicit configuration provenance. - Verify that provider diagnostic defaults do not become explicit values in the loaded configuration or replace source provenance in inspection. Cover the changed clearing behavior and endpoint-only authentication overrides. @@ -423,14 +458,11 @@ the detailed work sequence. ## Topics For Further Discussion -Before finalizing the roadmap, settle: +Before finalizing the roadmap, discuss a `vllm` provider with vLLM-specific +conventions, identifying the response and diagnostic behavior to support from +reviewed upstream contracts and fixtures. -1. A default, lowest-common-denominator OpenAI API-compatible provider used when - the backend does not specify a provider, including whether it has an explicit - selectable identifier and which common conventions it owns. -2. A `vllm` provider with vLLM-specific conventions, identifying the response and - diagnostic behavior to support from reviewed upstream contracts and fixtures. - -The normalization, unsupported-value handling, override precedence, and -configuration-inspection boundaries above remain agreed decisions. These topics -do not yet add further provider behavior or an implementation commitment. +The `openai-compatible` default, normalization, unsupported-value handling, +override precedence, and configuration-inspection boundaries above are agreed +decisions. The vLLM discussion does not yet add provider behavior or an +implementation commitment.