diff --git a/docs/roadmap/invocation-scoped-token-usage.md b/docs/roadmap/invocation-scoped-token-usage.md index d386ec6..3d07af0 100644 --- a/docs/roadmap/invocation-scoped-token-usage.md +++ b/docs/roadmap/invocation-scoped-token-usage.md @@ -1,6 +1,7 @@ # Invocation-Scoped Execution Observation -Status: accepted scope; implementation pending. +Status: agreed direction; default-provider and vLLM scope under discussion; +implementation pending. ## Purpose @@ -143,11 +144,18 @@ promptkit: api_key_env: OPENROUTER_API_KEY ``` -An omitted or null provider retains ordinary OpenAI-compatible behavior without -provider-specific defaults or metadata interpretation. Initially, `openrouter` -selects the OpenRouter conventions described here. A vLLM-specific selection can -follow when its relevant response contract and representative fixtures have been -reviewed; this work does not claim vLLM-specific support. +Normalize the final effective provider value by trimming whitespace and +lowercasing. Omitted, null, empty, and whitespace-only values are equivalent. +Reject an unsupported nonempty normalized value as an ordinary configuration +error rather than silently using generic behavior. `openrouter` selects the +OpenRouter conventions described here. + +The current generic fallback retains ordinary OpenAI-compatible behavior without +provider-specific defaults or metadata interpretation. Before finalizing this +roadmap, discuss making this fallback an explicit lowest-common-denominator +provider and adding a `vllm` selection. Their names, supported conventions, and +final scope remain to be settled; this document does not yet select a vLLM +contract. Resolve the field through ordinary ConfigKit composition and inheritance. Carry the selected value through configuration capture and prepared execution, expose @@ -182,6 +190,21 @@ existing behavior. These are conventions for explicitly requested diagnostics. Loading a backend, constructing an engine, and ordinary generation do not initiate probes. +Resolve provider-supplied diagnostic defaults at the provider boundary without +writing them into the loaded configuration or recording them as explicit source +values. Configuration inspection preserves the distinction between configured +overrides and provider-supplied fallbacks, including their existing source +provenance. + +### Diagnostic Migration + +This changes the current authentication metadata contract: an authentication +object currently requires `kind`, and clearing the object removes its configured +adapter. Under this roadmap, endpoint-only overrides become valid, and clearing +the object restores any selected provider's default. `kind: none` suppresses +that default. Explain these changes in the framework format documentation and +release notes, alongside the new provider field. + ### Provider Behavior Ownership Centralize provider-specific behavior within `internal/llm`, which already owns @@ -200,6 +223,8 @@ method to a generic operation as part of this consolidation. Provider-specific files and straightforward dispatch are sufficient initially. No new `internal/providers` package, registration framework, or broad interface requiring every provider to implement every capability is needed. +This is configurable selection among PromptKit-supported provider implementations, +not a public API for downstream registration of additional implementations. ## Optional Provider-Reported Cost @@ -321,9 +346,14 @@ logging infrastructure are outside this scope. provider-default probes, explicit kind and endpoint overrides, `kind: none`, and existing explicit probes without a provider selection. - Verify provider selection independently of backend names and namespaces, - including renamed and inherited backends, generic behavior when omitted/null, + including renamed and inherited backends, equivalent omitted/null/blank values, + trimming and case normalization, rejection of unsupported nonempty values, and retention through endpoint overrides. Confirm provider metadata is not - serialized into the completion request body. + serialized into the completion request body. Use the generic fallback selected + when the remaining provider discussion is complete. +- Verify that provider diagnostic defaults do not become explicit values in the + loaded configuration or replace source provenance in inspection. Cover the + changed clearing behavior and endpoint-only authentication overrides. - Carry optional OpenRouter cost through response decoding, internal/public conversions, successful run results, attempt and terminal events, and existing group summaries. Keep result cost available without registering an observer. @@ -353,11 +383,12 @@ logging infrastructure are outside this scope. and update internal observation and model-client documentation for changed flows. Document `provider`, its smoke-test defaults and override semantics in the framework format reference and relevant public GoDoc; update inspection and - diagnostic documentation with the implemented behavior. + diagnostic documentation with the implemented behavior. Include the diagnostic + migration in release notes. ## Implementation Planning Guidance -Plan two separately reviewable stages: +Organize the eventual implementation plan around two workstreams: 1. Contextual observer selection, including ordinary, prepared, coordinated, and nested execution boundaries and the existing group collector. @@ -366,9 +397,11 @@ Plan two separately reviewable stages: through decoding, accounting, public results, events, and group summaries. Include the corresponding configuration and API documentation. -The second stage is more extensive than the context helper and may be divided -into focused implementation units. Backend provider selection is decided above; -avoid coupling delivery of contextual observation to provider cost support. +The second area is more extensive than the context helper. Keep backend provider +configuration, diagnostic consolidation, and cost accounting in separately +reviewable implementation units rather than one large implementation prompt. +Complete the remaining provider discussion before preparing that plan; avoid +coupling delivery of contextual observation to provider cost support. Today, successful results accumulate generation responses in the runner, while optional observation recorders account for dispatched attempts. Adding cost to @@ -385,5 +418,19 @@ field separately so an unusable cost becomes unknown without changing existing framing or content validation. This needs no general numeric-conversion layer. Use the completion criteria above to select focused offline checks for each -stage. This roadmap owns target behavior; a separate implementation plan owns +unit. This roadmap owns target behavior; a separate implementation plan owns the detailed work sequence. + +## Topics For Further Discussion + +Before finalizing the roadmap, settle: + +1. A default, lowest-common-denominator OpenAI API-compatible provider used when + the backend does not specify a provider, including whether it has an explicit + selectable identifier and which common conventions it owns. +2. A `vllm` provider with vLLM-specific conventions, identifying the response and + diagnostic behavior to support from reviewed upstream contracts and fixtures. + +The normalization, unsupported-value handling, override precedence, and +configuration-inspection boundaries above remain agreed decisions. These topics +do not yet add further provider behavior or an implementation commitment.