base-layer/promptkit
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 2 Wiki Activity

Set bundled OpenRouter smoke-test defaults

Browse Source
This commit is contained in:
Eric Rakestraw
2026-11-03 14:40:58 +00:00
parent 75204d6134
commit e0cd63923d
1 changed files with 39 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files

78
docs/roadmap/bundled-openrouter.md
View File

@@ -2,7 +2,7 @@
## Status And Purpose
Proposed; no implementation changes yet.
Design decisions settled; implementation has not started.
Ship a small OpenRouter configuration bundle with PromptKit so consumers
can begin using named profiles without installing a separate catalog. Include
@@ -78,9 +78,36 @@ Include the root `openrouter` backend with OpenRouter's API base endpoint,
bundle. Reuse the existing OpenAI-compatible client and credential resolution.
Keep the backend minimal. Leave concurrency limits, queue policy, and provider
request parameters to existing defaults or application configuration. Smoke-test
metadata is a separate decision below; bundling does not automatically run
diagnostics or select an inference smoke-test profile.
request parameters to existing defaults or application configuration.
### Smoke-Test Defaults
Include the following metadata on the root `openrouter` backend:
```yaml
smoke_tests:
default_profile: gpt-luna-latest
authentication:
kind: openrouter_key
```
Omit the reachability endpoint to use the existing backend-endpoint fallback.
Omit the authentication endpoint to use the adapter's backend-relative `/key`
destination. OpenRouter documents the authenticated
[current-key endpoint](https://openrouter.ai/docs/api/api-reference/api-keys/get-current-api-key).
Use `gpt-luna-latest` as the inference diagnostic default when the caller supplies
neither a prompt nor a profile. Explicit prompt/profile selection continues to
follow existing smoke-test selection rules. The default names the effective
root profile, so an application replacement of that profile also affects this
fallback. Applications can change the backend metadata through ordinary
configuration composition.
These defaults make explicit diagnostics usable without additional setup; they
do not run tests automatically or establish a default for ordinary prompt
execution. Results describe the selected diagnostic path, not backend-wide
readiness. The inference smoke test makes a small request with potentially
non-zero cost; no fixed cost or model-availability guarantee is introduced.
### Small Profile Set
@@ -146,8 +173,8 @@ does not identify or pin the concrete model selected upstream.
## Implementation Work
1. Resolve the diagnostic-metadata decision below. Verify the provider endpoint
and exact model values for the twelve selected root profiles.
1. Verify the provider endpoint and exact model values for the twelve selected
root profiles.
2. Add the embedded YAML backend and profiles, and integrate their dedicated
underlay with engine construction. Add the disable option using the existing
facade option pattern. Apply the ordinary load/normalize/validate lifecycle.
@@ -159,42 +186,15 @@ does not identify or pin the concrete model selected upstream.
source evidence. Exercise request preparation with a fake client or local
server as appropriate; construction must not depend on provider reachability
or a live API key. Reuse existing credential tests and add coverage only for
new integration behavior.
new integration behavior. Cover default reachability, the authentication
adapter, inference fallback to `gpt-luna-latest`, explicit prompt/profile
selection, and application replacement of the fallback profile or metadata.
5. Update the format reference, consumer guide, relevant GoDoc, and maintained
examples when implementation lands. Explain how to disable, replace, or derive
from bundled profiles, and distinguish routers/latest aliases from fixed
models. Include the inventory/inspection change in release notes.
Keep current-state documentation unchanged until implementation lands. Retire
this roadmap after the behavior and remaining decisions reach their durable
owners. No new configuration syntax or collection specification version is
needed solely to distribute this data.
## Open Decisions
### Smoke-Test Metadata In The Bundled Backend
**Recommendation:** Include `smoke_tests.authentication.kind: openrouter_key`
using the existing adapter's backend-relative `/key` endpoint. Leave a separate
reachability endpoint and inference `default_profile` unspecified. The adapter
already exists, so this is a small configuration addition that makes explicit
API-key diagnostics useful out of the box without choosing an inference model.
OpenRouter documents the authenticated
[current-key endpoint](https://openrouter.ai/docs/api/api-reference/api-keys/get-current-api-key).
Its acceptance of a key is evidence about that endpoint, not a guarantee that
an arbitrary inference request will succeed.
This revises the initial recommendation to omit diagnostic metadata: given the
existing adapter and the bundle's convenience purpose, the added utility seems
worth maintaining the small declaration. The decision remains open pending
maintainer agreement.
**Alternative:** Include only the backend endpoint and credential-variable name.
This keeps the shipped configuration smaller and leaves diagnostic declarations
entirely with the consumer, but applications wanting authentication diagnostics
must configure the existing adapter themselves. Ordinary inference is unaffected.
Either approach uses existing diagnostic APIs and execution rules. Neither
implies automatic diagnostics during construction or execution. Selecting a
built-in inference smoke-test default would be a separate choice; this proposal
leaves callers able to select a profile explicitly without prescribing one.
this roadmap after the implemented behavior reaches its durable owners. No new
configuration syntax or collection specification version is needed solely to
distribute this data.