Use resolved targets for client routing
This commit is contained in:
@@ -16,14 +16,13 @@ Generation sends an HTTP `POST` with `Content-Type: application/json`.
|
|||||||
Before the client is called, the engine resolves framework, backend, profile,
|
Before the client is called, the engine resolves framework, backend, profile,
|
||||||
and request values into one execution target. Endpoint configuration is trimmed
|
and request values into one execution target. Endpoint configuration is trimmed
|
||||||
and must be an absolute HTTP or HTTPS URL with a host and without user
|
and must be an absolute HTTP or HTTPS URL with a host and without user
|
||||||
information, a query, or a fragment. A non-empty endpoint from the target
|
information, a query, or a fragment. The target's endpoint is validated again
|
||||||
overrides the client's configured base URL. The final selected endpoint is
|
before transport.
|
||||||
validated again before transport.
|
|
||||||
|
|
||||||
The completion URL is composed through parsed URL path operations. Nested base
|
The completion URL is composed through parsed URL path operations. Nested base
|
||||||
paths are retained, repeated trailing slashes are normalized, and the result
|
paths are retained, repeated trailing slashes are normalized, and the result
|
||||||
has exactly one appended `/chat/completions` suffix. Generation fails before
|
has exactly one appended `/chat/completions` suffix. Generation fails before
|
||||||
sending when neither source supplies a valid endpoint.
|
sending when the target does not supply a valid endpoint.
|
||||||
|
|
||||||
The target's backend ID is routing metadata for prepared values, results, and
|
The target's backend ID is routing metadata for prepared values, results, and
|
||||||
injected clients. The built-in client does not derive the URL from that ID and
|
injected clients. The built-in client does not derive the URL from that ID and
|
||||||
@@ -89,8 +88,7 @@ transport for HTTP-only diagnostics.
|
|||||||
## Request Body
|
## Request Body
|
||||||
|
|
||||||
The request body always contains `model` and `messages`. The execution
|
The request body always contains `model` and `messages`. The execution
|
||||||
target's model takes precedence over the client's configured model, and one
|
target's model must be available.
|
||||||
must be available.
|
|
||||||
|
|
||||||
Each ordinary message contains its `role` and string `content`. A
|
Each ordinary message contains its `role` and string `content`. A
|
||||||
cache-controlled message instead uses a text content block containing `type`,
|
cache-controlled message instead uses a text content block containing `type`,
|
||||||
|
|||||||
@@ -26,10 +26,9 @@ client uses its endpoint, credential metadata, generation fields, and extra
|
|||||||
parameters. `BackendID` remains routing metadata
|
parameters. `BackendID` remains routing metadata
|
||||||
for the generation boundary and is not mapped into the provider payload.
|
for the generation boundary and is not mapped into the provider payload.
|
||||||
|
|
||||||
Construction trims and validates a nonempty configured base URL and clones any
|
Construction clones any supplied `http.Client` so Promptkit can apply its
|
||||||
supplied `http.Client` so Promptkit can apply its timeout default without
|
timeout default without mutating the caller's client. Generation preparation
|
||||||
mutating the caller's client. An empty configured base remains valid because a
|
then:
|
||||||
resolved request target may supply the endpoint. Generation preparation then:
|
|
||||||
|
|
||||||
1. validates shared execution-setting invariants and the final selected base
|
1. validates shared execution-setting invariants and the final selected base
|
||||||
endpoint;
|
endpoint;
|
||||||
@@ -106,14 +105,13 @@ public ownership and redaction semantics belong to the
|
|||||||
|
|
||||||
## Failure Categories
|
## Failure Categories
|
||||||
|
|
||||||
The package preserves distinct error identities for invalid client
|
The package preserves distinct error identities for invalid generation
|
||||||
configuration, invalid generation requests, request execution failures,
|
requests, request execution failures, non-success provider statuses, and
|
||||||
non-success provider statuses, and malformed successful responses. Provider
|
malformed successful responses. Provider response bodies are never exposed in
|
||||||
response bodies are never exposed in raw form through non-success errors.
|
raw form through non-success errors.
|
||||||
|
|
||||||
Invalid nonempty configured endpoints are configuration failures. A missing or
|
A missing or invalid final selected endpoint is an invalid generation request
|
||||||
invalid final selected endpoint is an invalid generation request and is
|
and is rejected before transport.
|
||||||
rejected before transport.
|
|
||||||
|
|
||||||
Authentication resolves a trimmed direct key before a trimmed configured
|
Authentication resolves a trimmed direct key before a trimmed configured
|
||||||
environment value. Optional missing, empty, or whitespace-only sources do not
|
environment value. Optional missing, empty, or whitespace-only sources do not
|
||||||
|
|||||||
@@ -71,6 +71,11 @@ introduce a new ownership subsystem.
|
|||||||
|
|
||||||
## Stage 1: Remove Client-Level Routing Fallbacks
|
## Stage 1: Remove Client-Level Routing Fallbacks
|
||||||
|
|
||||||
|
**Completed:** Client routing now uses the resolved target; constructor fixtures
|
||||||
|
and integration guidance were migrated. With Go 1.27.3, `go test ./internal/llm
|
||||||
|
./internal/generation ./internal/configuration ./internal/usecase ./internal/engine`
|
||||||
|
passed, including loaded-target coverage.
|
||||||
|
|
||||||
**Finding:** 1. **Files:** `internal/llm/openai_compatible_client.go`, client
|
**Finding:** 1. **Files:** `internal/llm/openai_compatible_client.go`, client
|
||||||
fixtures/tests, `internal/engine/engine.go` caller verification, `docs/internal/llm.md`.
|
fixtures/tests, `internal/engine/engine.go` caller verification, `docs/internal/llm.md`.
|
||||||
|
|
||||||
|
|||||||
@@ -219,10 +219,7 @@ func TestLoadedZeroControlsReachBuiltInRequest(t *testing.T) {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
client, err := llm.NewOpenAICompatibleClient(llm.OpenAICompatibleConfig{})
|
client := llm.NewOpenAICompatibleClient(llm.OpenAICompatibleConfig{})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hello"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hello"}}},
|
||||||
Target: *profile.ResolvedTarget, TargetPresence: profile.TargetPresence,
|
Target: *profile.ResolvedTarget, TargetPresence: profile.TargetPresence,
|
||||||
|
|||||||
@@ -157,10 +157,7 @@ func New(cfg Config, options Options) (*Engine, error) {
|
|||||||
var client llm.Client
|
var client llm.Client
|
||||||
var diagnosticHTTPClient *llm.OpenAICompatibleClient
|
var diagnosticHTTPClient *llm.OpenAICompatibleClient
|
||||||
if options.llmClient == nil {
|
if options.llmClient == nil {
|
||||||
builtIn, newErr := llm.NewOpenAICompatibleClient(llm.OpenAICompatibleConfig{Timeout: cfg.Timeout, HTTPClient: cfg.HTTPClient})
|
builtIn := llm.NewOpenAICompatibleClient(llm.OpenAICompatibleConfig{Timeout: cfg.Timeout, HTTPClient: cfg.HTTPClient})
|
||||||
if newErr != nil {
|
|
||||||
return nil, newErr
|
|
||||||
}
|
|
||||||
diagnosticHTTPClient = builtIn
|
diagnosticHTTPClient = builtIn
|
||||||
client = generation.NewBuiltIn(manager, builtIn)
|
client = generation.NewBuiltIn(manager, builtIn)
|
||||||
} else {
|
} else {
|
||||||
|
|||||||
@@ -40,10 +40,7 @@ func testCoordinator(t *testing.T, transport transportFunc) *Coordinator {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
client, err := llm.NewOpenAICompatibleClient(llm.OpenAICompatibleConfig{HTTPClient: &http.Client{Transport: transport}})
|
client := llm.NewOpenAICompatibleClient(llm.OpenAICompatibleConfig{HTTPClient: &http.Client{Transport: transport}})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
coordinator := NewBuiltIn(manager, client)
|
coordinator := NewBuiltIn(manager, client)
|
||||||
coordinator.jitter = func(time.Duration) time.Duration { return 0 }
|
coordinator.jitter = func(time.Duration) time.Duration { return 0 }
|
||||||
coordinator.wait = func(ctx context.Context, _ time.Duration) error { return ctx.Err() }
|
coordinator.wait = func(ctx context.Context, _ time.Duration) error { return ctx.Err() }
|
||||||
|
|||||||
@@ -25,10 +25,7 @@ func TestGetWithEvidenceCapturesResponseAndOmitsURLUserInfo(t *testing.T) {
|
|||||||
}))
|
}))
|
||||||
defer server.Close()
|
defer server.Close()
|
||||||
|
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{Timeout: time.Second})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{Timeout: time.Second})
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("construct client: %v", err)
|
|
||||||
}
|
|
||||||
endpoint := strings.Replace(server.URL, "http://", "http://user:secret@", 1) + "/health"
|
endpoint := strings.Replace(server.URL, "http://", "http://user:secret@", 1) + "/health"
|
||||||
response, evidence, err := client.GetWithEvidence(context.Background(), endpoint, 0)
|
response, evidence, err := client.GetWithEvidence(context.Background(), endpoint, 0)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -88,7 +85,7 @@ func TestGetWithAuthenticationEvidenceResolvesOptionalCredentials(t *testing.T)
|
|||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Setenv("PROMPTKIT_DIAGNOSTIC_KEY", tc.environment)
|
t.Setenv("PROMPTKIT_DIAGNOSTIC_KEY", tc.environment)
|
||||||
var authorization string
|
var authorization string
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(request *http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(request *http.Request) (*http.Response, error) {
|
||||||
authorization = request.Header.Get("Authorization")
|
authorization = request.Header.Get("Authorization")
|
||||||
return &http.Response{
|
return &http.Response{
|
||||||
@@ -98,9 +95,6 @@ func TestGetWithAuthenticationEvidenceResolvesOptionalCredentials(t *testing.T)
|
|||||||
}, nil
|
}, nil
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
response, evidence, err := client.GetWithAuthenticationEvidence(
|
response, evidence, err := client.GetWithAuthenticationEvidence(
|
||||||
context.Background(), "https://example.invalid/key", 0, tc.target,
|
context.Background(), "https://example.invalid/key", 0, tc.target,
|
||||||
@@ -269,10 +263,7 @@ func TestGetWithEvidencePreservesClientAndIsolatesConcurrentCalls(t *testing.T)
|
|||||||
|
|
||||||
transport := http.DefaultTransport
|
transport := http.DefaultTransport
|
||||||
supplied := &http.Client{Transport: transport, Timeout: 7 * time.Second, CheckRedirect: redirect, Jar: jar}
|
supplied := &http.Client{Transport: transport, Timeout: 7 * time.Second, CheckRedirect: redirect, Jar: jar}
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{HTTPClient: supplied})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{HTTPClient: supplied})
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("construct client: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
const calls = 8
|
const calls = 8
|
||||||
evidences := make([]HTTPExecutionEvidence, calls)
|
evidences := make([]HTTPExecutionEvidence, calls)
|
||||||
@@ -307,15 +298,12 @@ func TestGetWithEvidencePreservesClientAndIsolatesConcurrentCalls(t *testing.T)
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestGetWithEvidenceAppliesTargetTimeout(t *testing.T) {
|
func TestGetWithEvidenceAppliesTargetTimeout(t *testing.T) {
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{Timeout: 5 * time.Second, HTTPClient: &http.Client{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{Timeout: 5 * time.Second, HTTPClient: &http.Client{
|
||||||
Transport: roundTripFunc(func(request *http.Request) (*http.Response, error) {
|
Transport: roundTripFunc(func(request *http.Request) (*http.Response, error) {
|
||||||
<-request.Context().Done()
|
<-request.Context().Done()
|
||||||
return nil, request.Context().Err()
|
return nil, request.Context().Err()
|
||||||
}),
|
}),
|
||||||
}})
|
}})
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("construct client: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
response, evidence, err := client.GetWithEvidence(context.Background(), "https://diagnostic.example/slow", 1)
|
response, evidence, err := client.GetWithEvidence(context.Background(), "https://diagnostic.example/slow", 1)
|
||||||
if response != nil || !errors.Is(err, context.DeadlineExceeded) {
|
if response != nil || !errors.Is(err, context.DeadlineExceeded) {
|
||||||
|
|||||||
@@ -20,7 +20,6 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
var (
|
var (
|
||||||
ErrInvalidConfig = errors.New("invalid llm client configuration")
|
|
||||||
ErrInvalidRequest = errors.New("invalid generate request")
|
ErrInvalidRequest = errors.New("invalid generate request")
|
||||||
ErrRequestFailed = errors.New("llm request failed")
|
ErrRequestFailed = errors.New("llm request failed")
|
||||||
ErrTransport = errors.New("llm transport failed")
|
ErrTransport = errors.New("llm transport failed")
|
||||||
@@ -43,28 +42,15 @@ func (e *requestFailedError) Unwrap() []error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
type OpenAICompatibleConfig struct {
|
type OpenAICompatibleConfig struct {
|
||||||
BaseURL string
|
|
||||||
Model string
|
|
||||||
Timeout time.Duration
|
Timeout time.Duration
|
||||||
HTTPClient *http.Client
|
HTTPClient *http.Client
|
||||||
}
|
}
|
||||||
|
|
||||||
type OpenAICompatibleClient struct {
|
type OpenAICompatibleClient struct {
|
||||||
baseURL string
|
httpClient *http.Client
|
||||||
defaultModel string
|
|
||||||
httpClient *http.Client
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func NewOpenAICompatibleClient(cfg OpenAICompatibleConfig) (*OpenAICompatibleClient, error) {
|
func NewOpenAICompatibleClient(cfg OpenAICompatibleConfig) *OpenAICompatibleClient {
|
||||||
baseURL := ""
|
|
||||||
if strings.TrimSpace(cfg.BaseURL) != "" {
|
|
||||||
var err error
|
|
||||||
baseURL, err = domain.NormalizeOpenAICompatibleBaseEndpoint(cfg.BaseURL)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("%w: invalid base URL: %v", ErrInvalidConfig, err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
timeout := cfg.Timeout
|
timeout := cfg.Timeout
|
||||||
if timeout <= 0 {
|
if timeout <= 0 {
|
||||||
timeout = defaults.LLMRequestTimeoutDefault
|
timeout = defaults.LLMRequestTimeoutDefault
|
||||||
@@ -82,10 +68,8 @@ func NewOpenAICompatibleClient(cfg OpenAICompatibleConfig) (*OpenAICompatibleCli
|
|||||||
}
|
}
|
||||||
|
|
||||||
return &OpenAICompatibleClient{
|
return &OpenAICompatibleClient{
|
||||||
baseURL: baseURL,
|
httpClient: client,
|
||||||
defaultModel: cfg.Model,
|
}
|
||||||
httpClient: client,
|
|
||||||
}, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (c *OpenAICompatibleClient) Generate(ctx context.Context, req domain.GenerateRequest) (*domain.GenerateResponse, error) {
|
func (c *OpenAICompatibleClient) Generate(ctx context.Context, req domain.GenerateRequest) (*domain.GenerateResponse, error) {
|
||||||
@@ -132,11 +116,7 @@ func (c *OpenAICompatibleClient) PrepareGeneration(req domain.GenerateRequest) (
|
|||||||
return nil, fmt.Errorf("%w: %v", ErrInvalidRequest, err)
|
return nil, fmt.Errorf("%w: %v", ErrInvalidRequest, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
selectedEndpoint := req.Target.Endpoint
|
endpoint, err := domain.NormalizeOpenAICompatibleBaseEndpoint(req.Target.Endpoint)
|
||||||
if strings.TrimSpace(selectedEndpoint) == "" {
|
|
||||||
selectedEndpoint = c.baseURL
|
|
||||||
}
|
|
||||||
endpoint, err := domain.NormalizeOpenAICompatibleBaseEndpoint(selectedEndpoint)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("%w: invalid endpoint: %v", ErrInvalidRequest, err)
|
return nil, fmt.Errorf("%w: invalid endpoint: %v", ErrInvalidRequest, err)
|
||||||
}
|
}
|
||||||
@@ -145,7 +125,7 @@ func (c *OpenAICompatibleClient) PrepareGeneration(req domain.GenerateRequest) (
|
|||||||
return nil, fmt.Errorf("%w: invalid endpoint path: %v", ErrInvalidRequest, err)
|
return nil, fmt.Errorf("%w: invalid endpoint path: %v", ErrInvalidRequest, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
wireReq, err := openAIChatRequestFromGenerateRequest(req, c.defaultModel)
|
wireReq, err := openAIChatRequestFromGenerateRequest(req)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("%w: %v", ErrInvalidRequest, err)
|
return nil, fmt.Errorf("%w: %v", ErrInvalidRequest, err)
|
||||||
}
|
}
|
||||||
@@ -445,11 +425,8 @@ func openAIChatResponseTooLargeError() error {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
func openAIChatRequestFromGenerateRequest(req domain.GenerateRequest, defaultModel string) (openAIChatRequest, error) {
|
func openAIChatRequestFromGenerateRequest(req domain.GenerateRequest) (openAIChatRequest, error) {
|
||||||
model := strings.TrimSpace(req.Target.Model)
|
model := strings.TrimSpace(req.Target.Model)
|
||||||
if model == "" {
|
|
||||||
model = strings.TrimSpace(defaultModel)
|
|
||||||
}
|
|
||||||
if model == "" {
|
if model == "" {
|
||||||
return openAIChatRequest{}, errors.New("model is required")
|
return openAIChatRequest{}, errors.New("model is required")
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -123,26 +123,19 @@ func (p *recordingProvider) lastRequest(t *testing.T) recordedProviderRequest {
|
|||||||
return p.requests[len(p.requests)-1]
|
return p.requests[len(p.requests)-1]
|
||||||
}
|
}
|
||||||
|
|
||||||
func newProviderClient(t *testing.T, provider *recordingProvider, config OpenAICompatibleConfig) *OpenAICompatibleClient {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
if config.BaseURL == "" {
|
|
||||||
config.BaseURL = provider.endpoint("/v1")
|
|
||||||
}
|
|
||||||
client, err := NewOpenAICompatibleClient(config)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("construct client: %v", err)
|
|
||||||
}
|
|
||||||
return client
|
|
||||||
}
|
|
||||||
|
|
||||||
func ordinaryGenerateRequest() domain.GenerateRequest {
|
func ordinaryGenerateRequest() domain.GenerateRequest {
|
||||||
return domain.GenerateRequest{
|
return domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
Target: domain.ExecutionTarget{Model: "model"},
|
Target: domain.ExecutionTarget{Endpoint: "https://provider.example/v1", Model: "model"},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (p *recordingProvider) generateRequest() domain.GenerateRequest {
|
||||||
|
req := ordinaryGenerateRequest()
|
||||||
|
req.Target.Endpoint = p.endpoint("/v1")
|
||||||
|
return req
|
||||||
|
}
|
||||||
|
|
||||||
type deadlineCapturingTransport struct {
|
type deadlineCapturingTransport struct {
|
||||||
deadline time.Time
|
deadline time.Time
|
||||||
hasDeadline bool
|
hasDeadline bool
|
||||||
@@ -253,35 +246,13 @@ func assertDeadlineNear(t *testing.T, deadline, before, after time.Time, duratio
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestNewOpenAICompatibleClientRejectsInvalidBaseURL(t *testing.T) {
|
|
||||||
for _, endpoint := range []string{
|
|
||||||
"://invalid",
|
|
||||||
"/v1",
|
|
||||||
"https:///v1",
|
|
||||||
"ftp://provider.example/v1",
|
|
||||||
"https://user@provider.example/v1",
|
|
||||||
"https://provider.example/v1?mode=chat",
|
|
||||||
"https://provider.example/v1#chat",
|
|
||||||
} {
|
|
||||||
t.Run(endpoint, func(t *testing.T) {
|
|
||||||
_, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{BaseURL: endpoint})
|
|
||||||
if !errors.Is(err, ErrInvalidConfig) {
|
|
||||||
t.Fatalf("expected ErrInvalidConfig, got %v", err)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestNewOpenAICompatibleClientDoesNotMutateSuppliedZeroTimeoutClient(t *testing.T) {
|
func TestNewOpenAICompatibleClientDoesNotMutateSuppliedZeroTimeoutClient(t *testing.T) {
|
||||||
transport := http.DefaultTransport
|
transport := http.DefaultTransport
|
||||||
supplied := &http.Client{Transport: transport}
|
supplied := &http.Client{Transport: transport}
|
||||||
|
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
HTTPClient: supplied,
|
HTTPClient: supplied,
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("unexpected constructor error: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if supplied.Timeout != 0 {
|
if supplied.Timeout != 0 {
|
||||||
t.Fatalf("expected supplied client timeout to remain zero, got %v", supplied.Timeout)
|
t.Fatalf("expected supplied client timeout to remain zero, got %v", supplied.Timeout)
|
||||||
@@ -305,13 +276,10 @@ func TestNewOpenAICompatibleClientDoesNotMutateSuppliedNonzeroTimeoutClient(t *t
|
|||||||
Transport: transport,
|
Transport: transport,
|
||||||
}
|
}
|
||||||
|
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
Timeout: 2 * time.Second,
|
Timeout: 2 * time.Second,
|
||||||
HTTPClient: supplied,
|
HTTPClient: supplied,
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("unexpected constructor error: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if supplied.Timeout != suppliedTimeout {
|
if supplied.Timeout != suppliedTimeout {
|
||||||
t.Fatalf("expected supplied client timeout to remain %v, got %v", suppliedTimeout, supplied.Timeout)
|
t.Fatalf("expected supplied client timeout to remain %v, got %v", suppliedTimeout, supplied.Timeout)
|
||||||
@@ -335,13 +303,10 @@ func TestNewOpenAICompatibleClientTreatsSuppliedNegativeTimeoutAsUnset(t *testin
|
|||||||
}
|
}
|
||||||
configuredTimeout := 23 * time.Second
|
configuredTimeout := 23 * time.Second
|
||||||
|
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
Timeout: configuredTimeout,
|
Timeout: configuredTimeout,
|
||||||
HTTPClient: supplied,
|
HTTPClient: supplied,
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("unexpected constructor error: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if supplied.Timeout != -time.Second {
|
if supplied.Timeout != -time.Second {
|
||||||
t.Fatalf("expected supplied client timeout to remain negative, got %v", supplied.Timeout)
|
t.Fatalf("expected supplied client timeout to remain negative, got %v", supplied.Timeout)
|
||||||
@@ -370,7 +335,6 @@ func TestOpenAICompatibleClientRequestMapping(t *testing.T) {
|
|||||||
{name: "optional response fields", run: checkOptionalResponseFieldOmission},
|
{name: "optional response fields", run: checkOptionalResponseFieldOmission},
|
||||||
{name: "reasoning and extra parameters", run: checkReasoningAndExtraParameterMapping},
|
{name: "reasoning and extra parameters", run: checkReasoningAndExtraParameterMapping},
|
||||||
{name: "optional request field presence", run: checkRequestFieldPresence},
|
{name: "optional request field presence", run: checkRequestFieldPresence},
|
||||||
{name: "configured model fallback", run: checkConfiguredModelFallback},
|
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, tc := range tests {
|
for _, tc := range tests {
|
||||||
@@ -380,14 +344,14 @@ func TestOpenAICompatibleClientRequestMapping(t *testing.T) {
|
|||||||
|
|
||||||
func checkCanonicalRoleAndExactContentMapping(t *testing.T) {
|
func checkCanonicalRoleAndExactContentMapping(t *testing.T) {
|
||||||
provider := newRecordingProvider(t)
|
provider := newRecordingProvider(t)
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
const content = " exact content\nwith whitespace "
|
const content = " exact content\nwith whitespace "
|
||||||
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{
|
||||||
Role: domain.RoleDeveloper,
|
Role: domain.RoleDeveloper,
|
||||||
Content: content,
|
Content: content,
|
||||||
}}},
|
}}},
|
||||||
Target: domain.ExecutionTarget{Model: "model"},
|
Target: domain.ExecutionTarget{Endpoint: provider.endpoint("/v1"), Model: "model"},
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("Generate() error = %v", err)
|
t.Fatalf("Generate() error = %v", err)
|
||||||
@@ -413,7 +377,7 @@ func checkCompleteRequestAndResponseMapping(t *testing.T) {
|
|||||||
"usage": {"prompt_tokens": 11, "completion_tokens": 22, "total_tokens": 33}
|
"usage": {"prompt_tokens": 11, "completion_tokens": 22, "total_tokens": 33}
|
||||||
}`)
|
}`)
|
||||||
|
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
Timeout: 2 * time.Second,
|
Timeout: 2 * time.Second,
|
||||||
})
|
})
|
||||||
t.Setenv("PROMPTKIT_TEST_API_KEY", "secret-key")
|
t.Setenv("PROMPTKIT_TEST_API_KEY", "secret-key")
|
||||||
@@ -424,6 +388,7 @@ func checkCompleteRequestAndResponseMapping(t *testing.T) {
|
|||||||
{Role: "user", Content: "Say hello"},
|
{Role: "user", Content: "Say hello"},
|
||||||
}},
|
}},
|
||||||
Target: domain.ExecutionTarget{
|
Target: domain.ExecutionTarget{
|
||||||
|
Endpoint: provider.endpoint("/v1"),
|
||||||
Model: "gpt-test",
|
Model: "gpt-test",
|
||||||
Temperature: 0.4,
|
Temperature: 0.4,
|
||||||
MaxTokens: 123,
|
MaxTokens: 123,
|
||||||
@@ -594,9 +559,11 @@ func TestOpenAICompatibleClientAuthentication(t *testing.T) {
|
|||||||
tc.configureEnv(t)
|
tc.configureEnv(t)
|
||||||
}
|
}
|
||||||
provider := newRecordingProvider(t)
|
provider := newRecordingProvider(t)
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{Model: "model"})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
request := ordinaryGenerateRequest()
|
request := provider.generateRequest()
|
||||||
request.Target = tc.target
|
request.Target = tc.target
|
||||||
|
request.Target.Endpoint = provider.endpoint("/v1")
|
||||||
|
request.Target.Model = "model"
|
||||||
|
|
||||||
_, err := client.Generate(context.Background(), request)
|
_, err := client.Generate(context.Background(), request)
|
||||||
if tc.wantError != nil {
|
if tc.wantError != nil {
|
||||||
@@ -625,7 +592,7 @@ func TestOpenAICompatibleClientAuthentication(t *testing.T) {
|
|||||||
|
|
||||||
func checkCacheControlledMessageMapping(t *testing.T) {
|
func checkCacheControlledMessageMapping(t *testing.T) {
|
||||||
provider := newRecordingProvider(t)
|
provider := newRecordingProvider(t)
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
|
|
||||||
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{
|
||||||
@@ -639,7 +606,7 @@ func checkCacheControlledMessageMapping(t *testing.T) {
|
|||||||
},
|
},
|
||||||
{Role: "user", Content: "Dynamic request."},
|
{Role: "user", Content: "Dynamic request."},
|
||||||
}},
|
}},
|
||||||
Target: domain.ExecutionTarget{Model: "model"},
|
Target: domain.ExecutionTarget{Endpoint: provider.endpoint("/v1"), Model: "model"},
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("expected no error, got %v", err)
|
t.Fatalf("expected no error, got %v", err)
|
||||||
@@ -691,7 +658,7 @@ func checkCacheControlledMessageMapping(t *testing.T) {
|
|||||||
|
|
||||||
func checkEmptyCacheControlTTLOmission(t *testing.T) {
|
func checkEmptyCacheControlTTLOmission(t *testing.T) {
|
||||||
provider := newRecordingProvider(t)
|
provider := newRecordingProvider(t)
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
|
|
||||||
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{
|
||||||
@@ -703,7 +670,7 @@ func checkEmptyCacheControlTTLOmission(t *testing.T) {
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
}},
|
}},
|
||||||
Target: domain.ExecutionTarget{Model: "model"},
|
Target: domain.ExecutionTarget{Endpoint: provider.endpoint("/v1"), Model: "model"},
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("expected no error, got %v", err)
|
t.Fatalf("expected no error, got %v", err)
|
||||||
@@ -745,8 +712,8 @@ func checkSessionIDMapping(t *testing.T) {
|
|||||||
for _, tc := range tests {
|
for _, tc := range tests {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
provider := newRecordingProvider(t)
|
provider := newRecordingProvider(t)
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
request := ordinaryGenerateRequest()
|
request := provider.generateRequest()
|
||||||
request.Prompt.SessionID = tc.sessionID
|
request.Prompt.SessionID = tc.sessionID
|
||||||
|
|
||||||
if _, err := client.Generate(context.Background(), request); err != nil {
|
if _, err := client.Generate(context.Background(), request); err != nil {
|
||||||
@@ -766,19 +733,14 @@ func checkSessionIDMapping(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestOpenAICompatibleClientRejectsTooLongSessionID(t *testing.T) {
|
func TestOpenAICompatibleClientRejectsTooLongSessionID(t *testing.T) {
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
BaseURL: "http://example.com/v1",
|
|
||||||
Model: "model",
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{
|
Prompt: domain.RenderedPrompt{
|
||||||
SessionID: strings.Repeat("x", domain.SessionIDMaxLength+1),
|
SessionID: strings.Repeat("x", domain.SessionIDMaxLength+1),
|
||||||
Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}},
|
Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}},
|
||||||
},
|
},
|
||||||
|
Target: domain.ExecutionTarget{Endpoint: "http://example.com/v1", Model: "model"},
|
||||||
})
|
})
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal("expected invalid request error")
|
t.Fatal("expected invalid request error")
|
||||||
@@ -828,9 +790,9 @@ func checkContentPresence(t *testing.T) {
|
|||||||
"cache_write_tokens": 5
|
"cache_write_tokens": 5
|
||||||
}
|
}
|
||||||
}`)
|
}`)
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{Model: "model"})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
|
|
||||||
response, err := client.Generate(context.Background(), ordinaryGenerateRequest())
|
response, err := client.Generate(context.Background(), provider.generateRequest())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("generate: %v", err)
|
t.Fatalf("generate: %v", err)
|
||||||
}
|
}
|
||||||
@@ -865,10 +827,11 @@ func checkCacheUsageMapping(t *testing.T) {
|
|||||||
"cache_write_tokens": 60
|
"cache_write_tokens": 60
|
||||||
}
|
}
|
||||||
}`)
|
}`)
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{Model: "model"})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
|
|
||||||
resp, err := client.Generate(context.Background(), domain.GenerateRequest{
|
resp, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
|
Target: domain.ExecutionTarget{Endpoint: provider.endpoint("/v1"), Model: "model"},
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("expected no error, got %v", err)
|
t.Fatalf("expected no error, got %v", err)
|
||||||
@@ -901,8 +864,8 @@ func TestOpenAICompatibleUsagePresence(t *testing.T) {
|
|||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
provider := newRecordingProvider(t)
|
provider := newRecordingProvider(t)
|
||||||
provider.respond(http.StatusOK, `{`+tc.usage+`"choices":[{"message":{"content":"ok"}}]}`)
|
provider.respond(http.StatusOK, `{`+tc.usage+`"choices":[{"message":{"content":"ok"}}]}`)
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{Model: "model"})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
resp, err := client.Generate(context.Background(), ordinaryGenerateRequest())
|
resp, err := client.Generate(context.Background(), provider.generateRequest())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
@@ -913,17 +876,17 @@ func TestOpenAICompatibleUsagePresence(t *testing.T) {
|
|||||||
}
|
}
|
||||||
provider := newRecordingProvider(t)
|
provider := newRecordingProvider(t)
|
||||||
provider.respond(http.StatusOK, `{"usage":{"prompt_tokens":"seven"},"choices":[{"message":{"content":"ok"}}]}`)
|
provider.respond(http.StatusOK, `{"usage":{"prompt_tokens":"seven"},"choices":[{"message":{"content":"ok"}}]}`)
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{Model: "model"})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
if resp, err := client.Generate(context.Background(), ordinaryGenerateRequest()); resp != nil || !errors.Is(err, ErrMalformedResponse) {
|
if resp, err := client.Generate(context.Background(), provider.generateRequest()); resp != nil || !errors.Is(err, ErrMalformedResponse) {
|
||||||
t.Fatalf("invalid numeric usage response=%+v, error=%v", resp, err)
|
t.Fatalf("invalid numeric usage response=%+v, error=%v", resp, err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func checkOptionalResponseFieldOmission(t *testing.T) {
|
func checkOptionalResponseFieldOmission(t *testing.T) {
|
||||||
provider := newRecordingProvider(t)
|
provider := newRecordingProvider(t)
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
|
|
||||||
_, err := client.Generate(context.Background(), ordinaryGenerateRequest())
|
_, err := client.Generate(context.Background(), provider.generateRequest())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("expected no error, got %v", err)
|
t.Fatalf("expected no error, got %v", err)
|
||||||
}
|
}
|
||||||
@@ -938,11 +901,12 @@ func checkOptionalResponseFieldOmission(t *testing.T) {
|
|||||||
|
|
||||||
func checkReasoningAndExtraParameterMapping(t *testing.T) {
|
func checkReasoningAndExtraParameterMapping(t *testing.T) {
|
||||||
provider := newRecordingProvider(t)
|
provider := newRecordingProvider(t)
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
|
|
||||||
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
Target: domain.ExecutionTarget{
|
Target: domain.ExecutionTarget{
|
||||||
|
Endpoint: provider.endpoint("/v1"),
|
||||||
Model: "model",
|
Model: "model",
|
||||||
ReasoningEffort: "high",
|
ReasoningEffort: "high",
|
||||||
ExtraParams: map[string]any{
|
ExtraParams: map[string]any{
|
||||||
@@ -1020,8 +984,8 @@ func checkRequestFieldPresence(t *testing.T) {
|
|||||||
for _, tc := range tests {
|
for _, tc := range tests {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
provider := newRecordingProvider(t)
|
provider := newRecordingProvider(t)
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
request := ordinaryGenerateRequest()
|
request := provider.generateRequest()
|
||||||
request.TargetPresence = tc.presence
|
request.TargetPresence = tc.presence
|
||||||
|
|
||||||
if _, err := client.Generate(context.Background(), request); err != nil {
|
if _, err := client.Generate(context.Background(), request); err != nil {
|
||||||
@@ -1073,21 +1037,17 @@ func checkOmittedGenerationTimeout(t *testing.T) {
|
|||||||
transport := &deadlineCapturingTransport{}
|
transport := &deadlineCapturingTransport{}
|
||||||
clientTimeout := 5 * time.Second
|
clientTimeout := 5 * time.Second
|
||||||
|
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "http://example.com/v1",
|
|
||||||
Timeout: clientTimeout,
|
Timeout: clientTimeout,
|
||||||
HTTPClient: &http.Client{
|
HTTPClient: &http.Client{
|
||||||
Transport: transport,
|
Transport: transport,
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
before := time.Now()
|
before := time.Now()
|
||||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
Target: domain.ExecutionTarget{Model: "model", TimeoutSeconds: 0},
|
Target: domain.ExecutionTarget{Endpoint: "http://example.com/v1", Model: "model", TimeoutSeconds: 0},
|
||||||
})
|
})
|
||||||
after := time.Now()
|
after := time.Now()
|
||||||
if err == nil {
|
if err == nil {
|
||||||
@@ -1146,11 +1106,11 @@ func TestOpenAICompatibleClientRejectsInvalidExtraParamsBeforeProviderCall(t *te
|
|||||||
for _, tc := range tests {
|
for _, tc := range tests {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
provider := newRecordingProvider(t)
|
provider := newRecordingProvider(t)
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
|
|
||||||
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
Target: domain.ExecutionTarget{Model: "model", ExtraParams: tc.extraParams},
|
Target: domain.ExecutionTarget{Endpoint: provider.endpoint("/v1"), Model: "model", ExtraParams: tc.extraParams},
|
||||||
})
|
})
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal("expected invalid request error")
|
t.Fatal("expected invalid request error")
|
||||||
@@ -1192,8 +1152,8 @@ func TestOpenAICompatibleClientClassifiesSchemaEncodingFailures(t *testing.T) {
|
|||||||
for _, tc := range tests {
|
for _, tc := range tests {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
provider := newRecordingProvider(t)
|
provider := newRecordingProvider(t)
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
request := ordinaryGenerateRequest()
|
request := provider.generateRequest()
|
||||||
request.Target.ExtraParams = tc.extraParams
|
request.Target.ExtraParams = tc.extraParams
|
||||||
request.StructuredOutput = &domain.StructuredOutputSpec{
|
request.StructuredOutput = &domain.StructuredOutputSpec{
|
||||||
Type: domain.StructuredOutputJSONSchema,
|
Type: domain.StructuredOutputJSONSchema,
|
||||||
@@ -1218,30 +1178,12 @@ func TestOpenAICompatibleClientClassifiesSchemaEncodingFailures(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func checkConfiguredModelFallback(t *testing.T) {
|
|
||||||
provider := newRecordingProvider(t)
|
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{Model: "default-model"})
|
|
||||||
|
|
||||||
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
|
||||||
Target: domain.ExecutionTarget{},
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("expected no error, got %v", err)
|
|
||||||
}
|
|
||||||
var model string
|
|
||||||
if !provider.lastRequest(t).decodeField(t, "model", &model) || model != "default-model" {
|
|
||||||
t.Fatalf("model = %q, want default-model", model)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestOpenAICompatibleClientEndpointComposition(t *testing.T) {
|
func TestOpenAICompatibleClientEndpointComposition(t *testing.T) {
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
name string
|
name string
|
||||||
run func(*testing.T)
|
run func(*testing.T)
|
||||||
}{
|
}{
|
||||||
{name: "request endpoint overrides configured endpoint", run: checkEndpointOverride},
|
{name: "request endpoint", run: checkSelectedEndpoint},
|
||||||
{name: "request endpoint works without configured endpoint", run: checkEmptyConfiguredEndpoint},
|
|
||||||
{name: "completion URL preserves valid base paths", run: checkCompletionURLComposition},
|
{name: "completion URL preserves valid base paths", run: checkCompletionURLComposition},
|
||||||
{name: "invalid selected endpoint fails before transport", run: checkInvalidSelectedEndpointRejection},
|
{name: "invalid selected endpoint fails before transport", run: checkInvalidSelectedEndpointRejection},
|
||||||
{name: "an endpoint is required", run: checkRequiredEndpoint},
|
{name: "an endpoint is required", run: checkRequiredEndpoint},
|
||||||
@@ -1252,35 +1194,6 @@ func TestOpenAICompatibleClientEndpointComposition(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func checkEndpointOverride(t *testing.T) {
|
|
||||||
configuredProvider := newRecordingProvider(t)
|
|
||||||
configuredProvider.respond(http.StatusOK, `{"choices":[{"message":{"content":"default"}}]}`)
|
|
||||||
selectedProvider := newRecordingProvider(t)
|
|
||||||
selectedProvider.respond(http.StatusOK, `{"choices":[{"message":{"content":"override"}}]}`)
|
|
||||||
|
|
||||||
client := newProviderClient(t, configuredProvider, OpenAICompatibleConfig{Model: "m"})
|
|
||||||
|
|
||||||
resp, err := client.Generate(context.Background(), domain.GenerateRequest{
|
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
|
||||||
Target: domain.ExecutionTarget{Endpoint: selectedProvider.endpoint("/v1")},
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("expected no error, got %v", err)
|
|
||||||
}
|
|
||||||
if resp.Content != "override" {
|
|
||||||
t.Fatalf("expected override response, got %q", resp.Content)
|
|
||||||
}
|
|
||||||
if got := configuredProvider.requestCount(); got != 0 {
|
|
||||||
t.Fatalf("configured provider calls = %d, want 0", got)
|
|
||||||
}
|
|
||||||
if got := selectedProvider.requestCount(); got != 1 {
|
|
||||||
t.Fatalf("selected provider calls = %d, want 1", got)
|
|
||||||
}
|
|
||||||
if got := selectedProvider.lastRequest(t).url; got != "/v1/chat/completions" {
|
|
||||||
t.Fatalf("selected URL = %q, want /v1/chat/completions", got)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func checkCommonResponseFailures(t *testing.T) {
|
func checkCommonResponseFailures(t *testing.T) {
|
||||||
const sensitiveBody = `provider-secret-fragment request_payload_details`
|
const sensitiveBody = `provider-secret-fragment request_payload_details`
|
||||||
tests := []struct {
|
tests := []struct {
|
||||||
@@ -1310,9 +1223,9 @@ func checkCommonResponseFailures(t *testing.T) {
|
|||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
provider := newRecordingProvider(t)
|
provider := newRecordingProvider(t)
|
||||||
provider.respond(tc.statusCode, tc.body)
|
provider.respond(tc.statusCode, tc.body)
|
||||||
client := newProviderClient(t, provider, OpenAICompatibleConfig{Model: "m"})
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
|
|
||||||
response, err := client.Generate(context.Background(), ordinaryGenerateRequest())
|
response, err := client.Generate(context.Background(), provider.generateRequest())
|
||||||
if response != nil {
|
if response != nil {
|
||||||
t.Fatalf("expected no partial response, got %+v", response)
|
t.Fatalf("expected no partial response, got %+v", response)
|
||||||
}
|
}
|
||||||
@@ -1357,9 +1270,7 @@ func checkSuccessfulResponseByteBoundary(t *testing.T) {
|
|||||||
for _, tc := range tests {
|
for _, tc := range tests {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
body := sizedSuccessResponseBody(tc.size)
|
body := sizedSuccessResponseBody(tc.size)
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "https://provider.example/v1",
|
|
||||||
Model: "m",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
||||||
return &http.Response{
|
return &http.Response{
|
||||||
StatusCode: http.StatusOK,
|
StatusCode: http.StatusOK,
|
||||||
@@ -1370,12 +1281,10 @@ func checkSuccessfulResponseByteBoundary(t *testing.T) {
|
|||||||
}, nil
|
}, nil
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("construct client: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
response, err := client.Generate(context.Background(), domain.GenerateRequest{
|
response, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
|
Target: domain.ExecutionTarget{Endpoint: "https://provider.example/v1", Model: "m"},
|
||||||
})
|
})
|
||||||
if tc.wantErr {
|
if tc.wantErr {
|
||||||
if response != nil {
|
if response != nil {
|
||||||
@@ -1425,9 +1334,7 @@ func checkContinuingOversizedResponse(t *testing.T) {
|
|||||||
strings.NewReader(prefix),
|
strings.NewReader(prefix),
|
||||||
continuation,
|
continuation,
|
||||||
)}
|
)}
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "https://provider.example/v1",
|
|
||||||
Model: "m",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
||||||
return &http.Response{
|
return &http.Response{
|
||||||
StatusCode: http.StatusOK,
|
StatusCode: http.StatusOK,
|
||||||
@@ -1438,9 +1345,6 @@ func checkContinuingOversizedResponse(t *testing.T) {
|
|||||||
}, nil
|
}, nil
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("construct client: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
type outcome struct {
|
type outcome struct {
|
||||||
response *domain.GenerateResponse
|
response *domain.GenerateResponse
|
||||||
@@ -1450,6 +1354,7 @@ func checkContinuingOversizedResponse(t *testing.T) {
|
|||||||
go func() {
|
go func() {
|
||||||
response, generateErr := client.Generate(context.Background(), domain.GenerateRequest{
|
response, generateErr := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
|
Target: domain.ExecutionTarget{Endpoint: "https://provider.example/v1", Model: "m"},
|
||||||
})
|
})
|
||||||
done <- outcome{response: response, err: generateErr}
|
done <- outcome{response: response, err: generateErr}
|
||||||
}()
|
}()
|
||||||
@@ -1497,9 +1402,7 @@ func checkSingleResponseDocument(t *testing.T) {
|
|||||||
for _, tc := range tests {
|
for _, tc := range tests {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
body := &countingReadCloser{reader: strings.NewReader(tc.body)}
|
body := &countingReadCloser{reader: strings.NewReader(tc.body)}
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "https://provider.example/v1",
|
|
||||||
Model: "m",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
||||||
return &http.Response{
|
return &http.Response{
|
||||||
StatusCode: http.StatusOK,
|
StatusCode: http.StatusOK,
|
||||||
@@ -1510,12 +1413,10 @@ func checkSingleResponseDocument(t *testing.T) {
|
|||||||
}, nil
|
}, nil
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("construct client: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
response, err := client.Generate(context.Background(), domain.GenerateRequest{
|
response, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
|
Target: domain.ExecutionTarget{Endpoint: "https://provider.example/v1", Model: "m"},
|
||||||
})
|
})
|
||||||
if tc.wantErr {
|
if tc.wantErr {
|
||||||
if response != nil {
|
if response != nil {
|
||||||
@@ -1541,22 +1442,19 @@ func checkGenerationTimeoutPrecedence(t *testing.T) {
|
|||||||
transport := &deadlineCapturingTransport{err: context.DeadlineExceeded}
|
transport := &deadlineCapturingTransport{err: context.DeadlineExceeded}
|
||||||
generationTimeout := 2 * time.Second
|
generationTimeout := 2 * time.Second
|
||||||
|
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "http://example.com/v1",
|
|
||||||
Model: "m",
|
|
||||||
HTTPClient: &http.Client{
|
HTTPClient: &http.Client{
|
||||||
Timeout: 10 * time.Second,
|
Timeout: 10 * time.Second,
|
||||||
Transport: transport,
|
Transport: transport,
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
before := time.Now()
|
before := time.Now()
|
||||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
Target: domain.ExecutionTarget{
|
Target: domain.ExecutionTarget{
|
||||||
|
Endpoint: "http://example.com/v1",
|
||||||
|
Model: "m",
|
||||||
TimeoutSeconds: int(generationTimeout / time.Second),
|
TimeoutSeconds: int(generationTimeout / time.Second),
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
@@ -1578,25 +1476,22 @@ func checkGenerationTimeoutPrecedence(t *testing.T) {
|
|||||||
|
|
||||||
func checkCallerDeadlinePrecedence(t *testing.T) {
|
func checkCallerDeadlinePrecedence(t *testing.T) {
|
||||||
transport := &deadlineCapturingTransport{err: context.DeadlineExceeded}
|
transport := &deadlineCapturingTransport{err: context.DeadlineExceeded}
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "http://example.com/v1",
|
|
||||||
Model: "m",
|
|
||||||
HTTPClient: &http.Client{
|
HTTPClient: &http.Client{
|
||||||
Timeout: 10 * time.Second,
|
Timeout: 10 * time.Second,
|
||||||
Transport: transport,
|
Transport: transport,
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
callerDeadline := time.Now().Add(time.Second)
|
callerDeadline := time.Now().Add(time.Second)
|
||||||
ctx, cancel := context.WithDeadline(context.Background(), callerDeadline)
|
ctx, cancel := context.WithDeadline(context.Background(), callerDeadline)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|
||||||
_, err = client.Generate(ctx, domain.GenerateRequest{
|
_, err := client.Generate(ctx, domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
Target: domain.ExecutionTarget{
|
Target: domain.ExecutionTarget{
|
||||||
|
Endpoint: "http://example.com/v1",
|
||||||
|
Model: "m",
|
||||||
TimeoutSeconds: 2,
|
TimeoutSeconds: 2,
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
@@ -1618,22 +1513,18 @@ func checkCallerDeadlinePrecedence(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func checkCallerCancellationIdentity(t *testing.T) {
|
func checkCallerCancellationIdentity(t *testing.T) {
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "http://example.com/v1",
|
|
||||||
Model: "m",
|
|
||||||
HTTPClient: &http.Client{
|
HTTPClient: &http.Client{
|
||||||
Transport: contextErrorTransport{},
|
Transport: contextErrorTransport{},
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
ctx, cancel := context.WithCancel(context.Background())
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
cancel()
|
cancel()
|
||||||
|
|
||||||
_, err = client.Generate(ctx, domain.GenerateRequest{
|
_, err := client.Generate(ctx, domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
|
Target: domain.ExecutionTarget{Endpoint: "http://example.com/v1", Model: "m"},
|
||||||
})
|
})
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal("expected cancellation error")
|
t.Fatal("expected cancellation error")
|
||||||
@@ -1647,21 +1538,17 @@ func checkCallerCancellationIdentity(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func checkExpiredCallerDeadlineIdentity(t *testing.T) {
|
func checkExpiredCallerDeadlineIdentity(t *testing.T) {
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "http://example.com/v1",
|
|
||||||
Model: "m",
|
|
||||||
HTTPClient: &http.Client{
|
HTTPClient: &http.Client{
|
||||||
Transport: contextErrorTransport{},
|
Transport: contextErrorTransport{},
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
ctx, cancel := context.WithDeadline(context.Background(), time.Unix(1, 0))
|
ctx, cancel := context.WithDeadline(context.Background(), time.Unix(1, 0))
|
||||||
defer cancel()
|
defer cancel()
|
||||||
_, err = client.Generate(ctx, domain.GenerateRequest{
|
_, err := client.Generate(ctx, domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
|
Target: domain.ExecutionTarget{Endpoint: "http://example.com/v1", Model: "m"},
|
||||||
})
|
})
|
||||||
if !errors.Is(err, ErrRequestFailed) || !errors.Is(err, context.DeadlineExceeded) {
|
if !errors.Is(err, ErrRequestFailed) || !errors.Is(err, context.DeadlineExceeded) {
|
||||||
t.Fatalf("expected request failure and caller deadline identities, got %v", err)
|
t.Fatalf("expected request failure and caller deadline identities, got %v", err)
|
||||||
@@ -1669,20 +1556,16 @@ func checkExpiredCallerDeadlineIdentity(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func checkWholeRequestTimeoutIdentity(t *testing.T) {
|
func checkWholeRequestTimeoutIdentity(t *testing.T) {
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "http://example.com/v1",
|
|
||||||
Model: "m",
|
|
||||||
HTTPClient: &http.Client{
|
HTTPClient: &http.Client{
|
||||||
Timeout: time.Millisecond,
|
Timeout: time.Millisecond,
|
||||||
Transport: waitingContextTransport{},
|
Transport: waitingContextTransport{},
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
|
Target: domain.ExecutionTarget{Endpoint: "http://example.com/v1", Model: "m"},
|
||||||
})
|
})
|
||||||
if !errors.Is(err, ErrRequestFailed) || !errors.Is(err, context.DeadlineExceeded) {
|
if !errors.Is(err, ErrRequestFailed) || !errors.Is(err, context.DeadlineExceeded) {
|
||||||
t.Fatalf("expected request failure and client timeout identities, got %v", err)
|
t.Fatalf("expected request failure and client timeout identities, got %v", err)
|
||||||
@@ -1696,20 +1579,15 @@ func checkTransportFailureIdentityAndRedaction(t *testing.T) {
|
|||||||
apiKey = "sensitive-api-key"
|
apiKey = "sensitive-api-key"
|
||||||
content = "sensitive prompt content"
|
content = "sensitive prompt content"
|
||||||
)
|
)
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: endpoint,
|
|
||||||
Model: "m",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
||||||
return nil, transportCause
|
return nil, transportCause
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: content}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: content}}},
|
||||||
Target: domain.ExecutionTarget{APIKey: apiKey},
|
Target: domain.ExecutionTarget{Endpoint: endpoint, Model: "m", APIKey: apiKey},
|
||||||
})
|
})
|
||||||
if !errors.Is(err, ErrRequestFailed) || !errors.Is(err, transportCause) {
|
if !errors.Is(err, ErrRequestFailed) || !errors.Is(err, transportCause) {
|
||||||
t.Fatalf("expected request failure and transport cause identities, got %v", err)
|
t.Fatalf("expected request failure and transport cause identities, got %v", err)
|
||||||
@@ -1726,13 +1604,7 @@ func checkTransportFailureIdentityAndRedaction(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestOpenAICompatibleClientRejectsInvalidExecutionSettings(t *testing.T) {
|
func TestOpenAICompatibleClientRejectsInvalidExecutionSettings(t *testing.T) {
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
BaseURL: "http://example.com/v1",
|
|
||||||
Model: "m",
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
type testCase struct {
|
type testCase struct {
|
||||||
name string
|
name string
|
||||||
@@ -1765,11 +1637,9 @@ func TestOpenAICompatibleClientRejectsInvalidExecutionSettings(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func checkEmptyConfiguredEndpoint(t *testing.T) {
|
func checkSelectedEndpoint(t *testing.T) {
|
||||||
var selectedURL string
|
var selectedURL string
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "",
|
|
||||||
Model: "m",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
||||||
selectedURL = req.URL.String()
|
selectedURL = req.URL.String()
|
||||||
return &http.Response{
|
return &http.Response{
|
||||||
@@ -1782,13 +1652,10 @@ func checkEmptyConfiguredEndpoint(t *testing.T) {
|
|||||||
}, nil
|
}, nil
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("expected empty configured base URL to be allowed, got %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
response, err := client.Generate(context.Background(), domain.GenerateRequest{
|
response, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
Target: domain.ExecutionTarget{Endpoint: "http://request-endpoint.example/v1"},
|
Target: domain.ExecutionTarget{Endpoint: "http://request-endpoint.example/v1", Model: "m"},
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("generate with request endpoint: %v", err)
|
t.Fatalf("generate with request endpoint: %v", err)
|
||||||
@@ -1817,9 +1684,7 @@ func checkCompletionURLComposition(t *testing.T) {
|
|||||||
for _, tc := range tests {
|
for _, tc := range tests {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
var selectedURL string
|
var selectedURL string
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: tc.baseURL,
|
|
||||||
Model: "m",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
||||||
selectedURL = req.URL.String()
|
selectedURL = req.URL.String()
|
||||||
return &http.Response{
|
return &http.Response{
|
||||||
@@ -1830,12 +1695,10 @@ func checkCompletionURLComposition(t *testing.T) {
|
|||||||
}, nil
|
}, nil
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("construct client: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
|
Target: domain.ExecutionTarget{Endpoint: tc.baseURL, Model: "m"},
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("generate: %v", err)
|
t.Fatalf("generate: %v", err)
|
||||||
@@ -1860,21 +1723,16 @@ func checkInvalidSelectedEndpointRejection(t *testing.T) {
|
|||||||
for _, endpoint := range invalidEndpoints {
|
for _, endpoint := range invalidEndpoints {
|
||||||
t.Run(endpoint, func(t *testing.T) {
|
t.Run(endpoint, func(t *testing.T) {
|
||||||
transportCalls := 0
|
transportCalls := 0
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "https://configured.example/v1",
|
|
||||||
Model: "m",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
||||||
transportCalls++
|
transportCalls++
|
||||||
return nil, errors.New("transport must not be called")
|
return nil, errors.New("transport must not be called")
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("construct client: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
Target: domain.ExecutionTarget{Endpoint: endpoint},
|
Target: domain.ExecutionTarget{Endpoint: endpoint, Model: "m"},
|
||||||
})
|
})
|
||||||
if !errors.Is(err, ErrInvalidRequest) {
|
if !errors.Is(err, ErrInvalidRequest) {
|
||||||
t.Fatalf("expected ErrInvalidRequest, got %v", err)
|
t.Fatalf("expected ErrInvalidRequest, got %v", err)
|
||||||
@@ -1890,17 +1748,11 @@ func checkInvalidSelectedEndpointRejection(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func checkRequiredEndpoint(t *testing.T) {
|
func checkRequiredEndpoint(t *testing.T) {
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
BaseURL: "",
|
|
||||||
Model: "m",
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
_, err = client.Generate(context.Background(), domain.GenerateRequest{
|
_, err := client.Generate(context.Background(), domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
Target: domain.ExecutionTarget{},
|
Target: domain.ExecutionTarget{Model: "m"},
|
||||||
})
|
})
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal("expected endpoint-required error")
|
t.Fatal("expected endpoint-required error")
|
||||||
@@ -1909,3 +1761,13 @@ func checkRequiredEndpoint(t *testing.T) {
|
|||||||
t.Fatalf("expected ErrInvalidRequest, got %v", err)
|
t.Fatalf("expected ErrInvalidRequest, got %v", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestOpenAICompatibleClientRequiresModel(t *testing.T) {
|
||||||
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
|
_, err := client.PrepareGeneration(domain.GenerateRequest{
|
||||||
|
Target: domain.ExecutionTarget{Endpoint: "https://provider.example/v1"},
|
||||||
|
})
|
||||||
|
if !errors.Is(err, ErrInvalidRequest) {
|
||||||
|
t.Fatalf("missing model error = %v, want invalid request", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -31,16 +31,13 @@ func TestProbeOpenRouterKey(t *testing.T) {
|
|||||||
} {
|
} {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
var authorization string
|
var authorization string
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
||||||
if req.URL.User != nil {
|
if req.URL.User != nil {
|
||||||
t.Fatal("request retained URL credentials")
|
t.Fatal("request retained URL credentials")
|
||||||
}
|
}
|
||||||
authorization = req.Header.Get("Authorization")
|
authorization = req.Header.Get("Authorization")
|
||||||
return &http.Response{StatusCode: tc.status, Header: make(http.Header), Body: io.NopCloser(strings.NewReader(tc.body))}, nil
|
return &http.Response{StatusCode: tc.status, Header: make(http.Header), Body: io.NopCloser(strings.NewReader(tc.body))}, nil
|
||||||
})}})
|
})}})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
got, err := client.ProbeOpenRouterKey(context.Background(), "https://user:password@example.invalid/key", 0, domain.ExecutionTarget{APIKey: tc.key})
|
got, err := client.ProbeOpenRouterKey(context.Background(), "https://user:password@example.invalid/key", 0, domain.ExecutionTarget{APIKey: tc.key})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
@@ -67,12 +64,9 @@ func (b authenticationFailingBody) Close() error { return nil }
|
|||||||
|
|
||||||
func TestProbeOpenRouterKeyRetainsBodyFailure(t *testing.T) {
|
func TestProbeOpenRouterKeyRetainsBodyFailure(t *testing.T) {
|
||||||
want := errors.New("interrupted")
|
want := errors.New("interrupted")
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
||||||
return &http.Response{StatusCode: 200, Header: make(http.Header), Body: authenticationFailingBody{err: want}}, nil
|
return &http.Response{StatusCode: 200, Header: make(http.Header), Body: authenticationFailingBody{err: want}}, nil
|
||||||
})}})
|
})}})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
got, err := client.ProbeOpenRouterKey(context.Background(), "https://example.invalid/key", 0, domain.ExecutionTarget{})
|
got, err := client.ProbeOpenRouterKey(context.Background(), "https://example.invalid/key", 0, domain.ExecutionTarget{})
|
||||||
if !errors.Is(err, want) || !got.HTTP.ResponseReceived {
|
if !errors.Is(err, want) || !got.HTTP.ResponseReceived {
|
||||||
t.Fatalf("evidence = %+v, error = %v", got, err)
|
t.Fatalf("evidence = %+v, error = %v", got, err)
|
||||||
@@ -108,12 +102,9 @@ func TestProbeOpenRouterKeyBoundsAndClosesBodies(t *testing.T) {
|
|||||||
} {
|
} {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
body := &observedAuthenticationBody{reader: strings.NewReader(strings.Repeat("x", 2*maxAuthenticationResponseBytes))}
|
body := &observedAuthenticationBody{reader: strings.NewReader(strings.Repeat("x", 2*maxAuthenticationResponseBytes))}
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
||||||
return &http.Response{StatusCode: tc.status, ContentLength: 1, Header: make(http.Header), Body: body}, nil
|
return &http.Response{StatusCode: tc.status, ContentLength: 1, Header: make(http.Header), Body: body}, nil
|
||||||
})}})
|
})}})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
got, err := client.ProbeOpenRouterKey(context.Background(), "https://example.invalid/key", 0, domain.ExecutionTarget{})
|
got, err := client.ProbeOpenRouterKey(context.Background(), "https://example.invalid/key", 0, domain.ExecutionTarget{})
|
||||||
if err != nil || body.read != tc.wantRead || !body.closed || got.Overflow != (tc.status == http.StatusOK) {
|
if err != nil || body.read != tc.wantRead || !body.closed || got.Overflow != (tc.status == http.StatusOK) {
|
||||||
t.Fatalf("evidence = %+v, read = %d, closed = %v, error = %v", got, body.read, body.closed, err)
|
t.Fatalf("evidence = %+v, read = %d, closed = %v, error = %v", got, body.read, body.closed, err)
|
||||||
@@ -130,13 +121,10 @@ func TestOpenRouterKeyEndpoint(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestProbeOpenRouterKeyPreservesCancellation(t *testing.T) {
|
func TestProbeOpenRouterKeyPreservesCancellation(t *testing.T) {
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
||||||
<-req.Context().Done()
|
<-req.Context().Done()
|
||||||
return nil, req.Context().Err()
|
return nil, req.Context().Err()
|
||||||
})}})
|
})}})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
ctx, cancel := context.WithCancel(context.Background())
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
cancel()
|
cancel()
|
||||||
got, err := client.ProbeOpenRouterKey(ctx, "https://example.invalid/key", 0, domain.ExecutionTarget{})
|
got, err := client.ProbeOpenRouterKey(ctx, "https://example.invalid/key", 0, domain.ExecutionTarget{})
|
||||||
|
|||||||
@@ -12,13 +12,7 @@ import (
|
|||||||
var prepareGenerationBenchmarkSink *PreparedGeneration
|
var prepareGenerationBenchmarkSink *PreparedGeneration
|
||||||
|
|
||||||
func BenchmarkPrepareGeneration(b *testing.B) {
|
func BenchmarkPrepareGeneration(b *testing.B) {
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{})
|
||||||
BaseURL: "https://api.example.test/v1",
|
|
||||||
Model: "benchmark-model",
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
b.Fatalf("construct benchmark client: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
prompt := domain.RenderedPrompt{
|
prompt := domain.RenderedPrompt{
|
||||||
Messages: []domain.RenderedMessage{{Role: "user", Content: "Summarize this request."}},
|
Messages: []domain.RenderedMessage{{Role: "user", Content: "Summarize this request."}},
|
||||||
@@ -31,14 +25,14 @@ func BenchmarkPrepareGeneration(b *testing.B) {
|
|||||||
name: "no_extras",
|
name: "no_extras",
|
||||||
request: domain.GenerateRequest{
|
request: domain.GenerateRequest{
|
||||||
Prompt: prompt,
|
Prompt: prompt,
|
||||||
Target: domain.ExecutionTarget{Model: "benchmark-model"},
|
Target: domain.ExecutionTarget{Endpoint: "https://api.example.test/v1", Model: "benchmark-model"},
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "small_extras",
|
name: "small_extras",
|
||||||
request: domain.GenerateRequest{
|
request: domain.GenerateRequest{
|
||||||
Prompt: prompt,
|
Prompt: prompt,
|
||||||
Target: domain.ExecutionTarget{Model: "benchmark-model", ExtraParams: map[string]any{
|
Target: domain.ExecutionTarget{Endpoint: "https://api.example.test/v1", Model: "benchmark-model", ExtraParams: map[string]any{
|
||||||
"priority": "low",
|
"priority": "low",
|
||||||
"attempt": 3,
|
"attempt": 3,
|
||||||
"enabled": true,
|
"enabled": true,
|
||||||
@@ -49,7 +43,7 @@ func BenchmarkPrepareGeneration(b *testing.B) {
|
|||||||
name: "1MiB_string_extra",
|
name: "1MiB_string_extra",
|
||||||
request: domain.GenerateRequest{
|
request: domain.GenerateRequest{
|
||||||
Prompt: prompt,
|
Prompt: prompt,
|
||||||
Target: domain.ExecutionTarget{Model: "benchmark-model", ExtraParams: map[string]any{
|
Target: domain.ExecutionTarget{Endpoint: "https://api.example.test/v1", Model: "benchmark-model", ExtraParams: map[string]any{
|
||||||
"payload": strings.Repeat("x", 1<<20),
|
"payload": strings.Repeat("x", 1<<20),
|
||||||
}},
|
}},
|
||||||
},
|
},
|
||||||
@@ -58,7 +52,7 @@ func BenchmarkPrepareGeneration(b *testing.B) {
|
|||||||
name: "nested_json_extras",
|
name: "nested_json_extras",
|
||||||
request: domain.GenerateRequest{
|
request: domain.GenerateRequest{
|
||||||
Prompt: prompt,
|
Prompt: prompt,
|
||||||
Target: domain.ExecutionTarget{Model: "benchmark-model", ExtraParams: map[string]any{
|
Target: domain.ExecutionTarget{Endpoint: "https://api.example.test/v1", Model: "benchmark-model", ExtraParams: map[string]any{
|
||||||
"options": map[string]any{
|
"options": map[string]any{
|
||||||
"enabled": true,
|
"enabled": true,
|
||||||
"limits": []any{2, 4, 8},
|
"limits": []any{2, 4, 8},
|
||||||
@@ -71,7 +65,7 @@ func BenchmarkPrepareGeneration(b *testing.B) {
|
|||||||
name: "structured_output",
|
name: "structured_output",
|
||||||
request: domain.GenerateRequest{
|
request: domain.GenerateRequest{
|
||||||
Prompt: prompt,
|
Prompt: prompt,
|
||||||
Target: domain.ExecutionTarget{Model: "benchmark-model"},
|
Target: domain.ExecutionTarget{Endpoint: "https://api.example.test/v1", Model: "benchmark-model"},
|
||||||
StructuredOutput: &domain.StructuredOutputSpec{
|
StructuredOutput: &domain.StructuredOutputSpec{
|
||||||
Type: domain.StructuredOutputJSONSchema,
|
Type: domain.StructuredOutputJSONSchema,
|
||||||
JSONSchema: &domain.StructuredOutputJSONSpec{
|
JSONSchema: &domain.StructuredOutputJSONSpec{
|
||||||
|
|||||||
@@ -73,9 +73,7 @@ func TestPreparedGenerationEvidenceSeparatesResponseAndGenerationFacts(t *testin
|
|||||||
|
|
||||||
for _, test := range tests {
|
for _, test := range tests {
|
||||||
t.Run(test.name, func(t *testing.T) {
|
t.Run(test.name, func(t *testing.T) {
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "https://provider.example/v1",
|
|
||||||
Model: "model",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(request *http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(request *http.Request) (*http.Response, error) {
|
||||||
response := responseFor(request, test.status, "")
|
response := responseFor(request, test.status, "")
|
||||||
response.Body = io.NopCloser(strings.NewReader(test.body))
|
response.Body = io.NopCloser(strings.NewReader(test.body))
|
||||||
@@ -83,9 +81,6 @@ func TestPreparedGenerationEvidenceSeparatesResponseAndGenerationFacts(t *testin
|
|||||||
return response, nil
|
return response, nil
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
request := ordinaryGenerateRequest()
|
request := ordinaryGenerateRequest()
|
||||||
if test.structured {
|
if test.structured {
|
||||||
request.StructuredOutput = &domain.StructuredOutputSpec{
|
request.StructuredOutput = &domain.StructuredOutputSpec{
|
||||||
@@ -136,9 +131,7 @@ func TestPreparedGenerationEvidenceSeparatesResponseAndGenerationFacts(t *testin
|
|||||||
func TestPreparedGenerationEvidenceRetainsDecodedResponseOnCancellation(t *testing.T) {
|
func TestPreparedGenerationEvidenceRetainsDecodedResponseOnCancellation(t *testing.T) {
|
||||||
ctx, cancel := context.WithCancel(context.Background())
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
body := `{"choices":[{"message":{"content":"ok"}}],"usage":{"prompt_tokens":0}}`
|
body := `{"choices":[{"message":{"content":"ok"}}],"usage":{"prompt_tokens":0}}`
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "https://provider.example/v1",
|
|
||||||
Model: "model",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(request *http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(request *http.Request) (*http.Response, error) {
|
||||||
return &http.Response{
|
return &http.Response{
|
||||||
StatusCode: http.StatusOK,
|
StatusCode: http.StatusOK,
|
||||||
@@ -149,9 +142,6 @@ func TestPreparedGenerationEvidenceRetainsDecodedResponseOnCancellation(t *testi
|
|||||||
}, nil
|
}, nil
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
prepared, err := client.PrepareGeneration(ordinaryGenerateRequest())
|
prepared, err := client.PrepareGeneration(ordinaryGenerateRequest())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
@@ -174,16 +164,11 @@ func TestPreparedGenerationEvidenceRetainsDecodedResponseOnCancellation(t *testi
|
|||||||
|
|
||||||
func TestPreparedGenerationEvidenceRecordsStructuredRequestAtDispatch(t *testing.T) {
|
func TestPreparedGenerationEvidenceRecordsStructuredRequestAtDispatch(t *testing.T) {
|
||||||
transportErr := errors.New("provider unavailable")
|
transportErr := errors.New("provider unavailable")
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "https://provider.example/v1",
|
|
||||||
Model: "model",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
||||||
return nil, transportErr
|
return nil, transportErr
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
request := ordinaryGenerateRequest()
|
request := ordinaryGenerateRequest()
|
||||||
request.StructuredOutput = &domain.StructuredOutputSpec{
|
request.StructuredOutput = &domain.StructuredOutputSpec{
|
||||||
Type: domain.StructuredOutputJSONSchema,
|
Type: domain.StructuredOutputJSONSchema,
|
||||||
|
|||||||
@@ -26,8 +26,7 @@ func TestPreparedGenerationFreezesPayloadAndAuthentication(t *testing.T) {
|
|||||||
const envName = "PROMPTKIT_PREPARED_GENERATION_TEST_KEY"
|
const envName = "PROMPTKIT_PREPARED_GENERATION_TEST_KEY"
|
||||||
t.Setenv(envName, tc.initialKey)
|
t.Setenv(envName, tc.initialKey)
|
||||||
var bodies, auth, urls []string
|
var bodies, auth, urls []string
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "https://example.invalid/v1", Model: "default-model",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
||||||
body, err := io.ReadAll(req.Body)
|
body, err := io.ReadAll(req.Body)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -39,12 +38,9 @@ func TestPreparedGenerationFreezesPayloadAndAuthentication(t *testing.T) {
|
|||||||
return &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(strings.NewReader(`{"choices":[{"message":{"content":"ok"}}]}`))}, nil
|
return &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(strings.NewReader(`{"choices":[{"message":{"content":"ok"}}]}`))}, nil
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
req := domain.GenerateRequest{
|
req := domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "original"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "original"}}},
|
||||||
Target: domain.ExecutionTarget{APIKeyEnv: envName},
|
Target: domain.ExecutionTarget{Endpoint: "https://example.invalid/v1", Model: "default-model", APIKeyEnv: envName},
|
||||||
}
|
}
|
||||||
prepared, err := client.PrepareGeneration(req)
|
prepared, err := client.PrepareGeneration(req)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -68,8 +64,7 @@ func TestPreparedGenerationFreezesPayloadAndAuthentication(t *testing.T) {
|
|||||||
func TestPreparedGenerationAttemptsHaveIndependentContextsAndCloseBodies(t *testing.T) {
|
func TestPreparedGenerationAttemptsHaveIndependentContextsAndCloseBodies(t *testing.T) {
|
||||||
var contexts []context.Context
|
var contexts []context.Context
|
||||||
var closed int
|
var closed int
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "https://example.invalid/v1", Model: "model",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(req *http.Request) (*http.Response, error) {
|
||||||
if err := req.Context().Err(); err != nil {
|
if err := req.Context().Err(); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -81,12 +76,9 @@ func TestPreparedGenerationAttemptsHaveIndependentContextsAndCloseBodies(t *test
|
|||||||
}}, nil
|
}}, nil
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
prepared, err := client.PrepareGeneration(domain.GenerateRequest{
|
prepared, err := client.PrepareGeneration(domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
Target: domain.ExecutionTarget{TimeoutSeconds: 1},
|
Target: domain.ExecutionTarget{Endpoint: "https://example.invalid/v1", Model: "model", TimeoutSeconds: 1},
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
@@ -115,17 +107,14 @@ func TestPreparedGenerationAttemptsHaveIndependentContextsAndCloseBodies(t *test
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestPreparedGenerationRetainsUsageFromMalformedChoice(t *testing.T) {
|
func TestPreparedGenerationRetainsUsageFromMalformedChoice(t *testing.T) {
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "https://example.invalid/v1", Model: "model",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
||||||
return &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(strings.NewReader(`{"usage":{"prompt_tokens":0},"choices":[]}`))}, nil
|
return &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(strings.NewReader(`{"usage":{"prompt_tokens":0},"choices":[]}`))}, nil
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
prepared, err := client.PrepareGeneration(domain.GenerateRequest{
|
prepared, err := client.PrepareGeneration(domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
|
Target: domain.ExecutionTarget{Endpoint: "https://example.invalid/v1", Model: "model"},
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
@@ -137,17 +126,14 @@ func TestPreparedGenerationRetainsUsageFromMalformedChoice(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestPreparedGenerationDoesNotUseInvalidlyFramedUsage(t *testing.T) {
|
func TestPreparedGenerationDoesNotUseInvalidlyFramedUsage(t *testing.T) {
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "https://example.invalid/v1", Model: "model",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
||||||
return &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(strings.NewReader(`{"usage":{"prompt_tokens":9},"choices":[]}{}`))}, nil
|
return &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(strings.NewReader(`{"usage":{"prompt_tokens":9},"choices":[]}{}`))}, nil
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
prepared, err := client.PrepareGeneration(domain.GenerateRequest{
|
prepared, err := client.PrepareGeneration(domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
|
Target: domain.ExecutionTarget{Endpoint: "https://example.invalid/v1", Model: "model"},
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
@@ -160,17 +146,14 @@ func TestPreparedGenerationDoesNotUseInvalidlyFramedUsage(t *testing.T) {
|
|||||||
|
|
||||||
func TestPreparedGenerationRetainsReadCancellation(t *testing.T) {
|
func TestPreparedGenerationRetainsReadCancellation(t *testing.T) {
|
||||||
closed := false
|
closed := false
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "https://example.invalid/v1", Model: "model",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
||||||
return &http.Response{StatusCode: http.StatusOK, Body: &readErrorBody{err: context.Canceled, closed: &closed}}, nil
|
return &http.Response{StatusCode: http.StatusOK, Body: &readErrorBody{err: context.Canceled, closed: &closed}}, nil
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
prepared, err := client.PrepareGeneration(domain.GenerateRequest{
|
prepared, err := client.PrepareGeneration(domain.GenerateRequest{
|
||||||
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
Prompt: domain.RenderedPrompt{Messages: []domain.RenderedMessage{{Role: "user", Content: "hi"}}},
|
||||||
|
Target: domain.ExecutionTarget{Endpoint: "https://example.invalid/v1", Model: "model"},
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
|
|||||||
@@ -127,9 +127,7 @@ func TestOpenAICompatibleClientNonSuccessBodyOwnership(t *testing.T) {
|
|||||||
func newNonSuccessResponseClient(t *testing.T, statusCode int, contentLength int64, body io.ReadCloser) *OpenAICompatibleClient {
|
func newNonSuccessResponseClient(t *testing.T, statusCode int, contentLength int64, body io.ReadCloser) *OpenAICompatibleClient {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
client, err := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
client := NewOpenAICompatibleClient(OpenAICompatibleConfig{
|
||||||
BaseURL: "https://provider.example/v1",
|
|
||||||
Model: "m",
|
|
||||||
HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
HTTPClient: &http.Client{Transport: roundTripFunc(func(*http.Request) (*http.Response, error) {
|
||||||
return &http.Response{
|
return &http.Response{
|
||||||
StatusCode: statusCode,
|
StatusCode: statusCode,
|
||||||
@@ -138,8 +136,5 @@ func newNonSuccessResponseClient(t *testing.T, statusCode int, contentLength int
|
|||||||
}, nil
|
}, nil
|
||||||
})},
|
})},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("construct client: %v", err)
|
|
||||||
}
|
|
||||||
return client
|
return client
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -111,12 +111,9 @@ func TestReachabilityProbeIsUnsupportedWithoutHTTPClient(t *testing.T) {
|
|||||||
|
|
||||||
func reachabilityClient(t *testing.T, roundTrip func(*http.Request) (*http.Response, error)) *llm.OpenAICompatibleClient {
|
func reachabilityClient(t *testing.T, roundTrip func(*http.Request) (*http.Response, error)) *llm.OpenAICompatibleClient {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
client, err := llm.NewOpenAICompatibleClient(llm.OpenAICompatibleConfig{
|
client := llm.NewOpenAICompatibleClient(llm.OpenAICompatibleConfig{
|
||||||
HTTPClient: &http.Client{Transport: reachabilityProbeTransport(roundTrip)},
|
HTTPClient: &http.Client{Transport: reachabilityProbeTransport(roundTrip)},
|
||||||
})
|
})
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
return client
|
return client
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user