5.5 KiB
5.5 KiB
Promptkit v0.25.0
This minor release adds invocation-scoped execution observation, provider-specific reasoning and cache usage, and optional USD cost reporting. Backend definitions can now select API conventions independently of their names and endpoints. Consumers upgrading from v0.24.x should review the configuration and accounting changes below.
Added And Changed
WithExecutionObserverselects observation through a caller-owned context across nested, concurrent, coordinated, and multi-engine calls. Request observers run before contextual observers. Prepared handles retain their explicit request observer but select contextual observers when executed, not when prepared. Each execution retains independent accounting; groups count each run's first terminal once. See the observation guide and offline example.- Backend
providerselectsopenai-compatible,openrouter, orvllmconventions. Omitted, null, or blank selects the generic OpenAI-compatible behavior. The bundled OpenRouter backend explicitly selectsopenrouter.InspectBackendexposes declared provider andzero_costsettings without manufacturing runtime defaults. See the backend reference. - Usage values, presence indicators, observations, and group summaries include
ReasoningTokens. OpenRouter and vLLM decoding includes their respective reasoning, cache-read, and cache-write fields. Reasoning is a subset of completion tokens; cache activity does not add to reported token totals.ObservedTokenUsage.CacheHitRatioderives a ratio from complete, consistent input and cache-read totals. See the response contract. RunResult.CostUSD, execution events, and group summaries report a USD subtotal with availability, completeness, and source metadata. OpenRouter supplies optionalusage.cost; injected clients can supplyGenerateResponse.CostUSD. Explicit backendzero_cost: truesupplies a zero fallback when no usable amount is reported. A reported amount, including zero, takes precedence. See the cost GoDoc.- Authentication diagnostics can use provider defaults:
openrouter_keyfor OpenRouter and the newvllm_modelsadapter for vLLM. The latter probes base-relative/modelsand reports recognized response facts without asserting authentication enforcement or inference authorization. See the diagnostic integration contract.
Compatibility And Upgrade
- For application-defined OpenRouter backends, set
provider: openrouterto enable OpenRouter-specific usage and cost decoding and its authentication default. Backend IDs and endpoint hostnames do not select conventions. Generic backends retain their existing request and token conventions and do not decode monetary cost. Unsupported nonblank provider names are configuration errors; supplied names are trimmed and lowercased. - OpenRouter cache-write reporting now reads
usage.prompt_tokens_details.cache_write_tokens, not the generic top-levelusage.cache_write_tokensfield. vLLM usesusage.prompt_tokens_details.created_cache_tokens. Missing or unusable new details remain unknown rather than invalidating otherwise usable content. Availability depends on the provider and server settings; consult the response contract. - A supplied authentication object no longer requires
kind: an omitted or null kind uses the provider default. Clearing the authentication block also restores that default. Setkind: noneto declare authentication testing inapplicable. The generic provider has no authentication default; explicit adapters remain supported independently of provider selection. Probes still run only when explicitly requested. See the backend reference. - Cost includes all dispatched attempts, including transient retries and
structural repairs, and is available on successful results without an
observer. On operational failure, observations can retain cost even though
no partial result is returned. Existing
RunResult.Usagecontinues to sum successful generation responses; presence-aware observations cover all dispatched attempts. Do not add terminal totals to attempt totals. - Unknown cost is distinct from reported or configured zero. No backend is automatically declared zero-cost, and no token-rate estimation is performed. Group cost completeness describes included terminals, independently of terminal coverage. Consumers own presentation and interpretation; complete reporting does not verify billing.
- Result JSON adds
cost_usd, and token JSON addsreasoning_tokens. Update strict decoders if necessary. Older result JSON without cost decodes to unknown cost. Consumers using positional literals for expanded public structs may need to update them. See the public types.
The Go consumer minimum remains 1.27.0; dependencies are unchanged from v0.24.0.
go get go.maximumdirect.net/promptkit@v0.25.0