Skip to content

05.2 Custom Providers

Jump to bottom
Nikolay Vyahhi edited this page Feb 19, 2026 · 3 revisions

Custom Providers

Relevant source files

The following files were used as context for generating this wiki page:

This page documents how to configure custom LLM API endpoints in ZeroClaw. Custom providers enable you to use any OpenAI-compatible or Anthropic-compatible service as your LLM backend.

For information about built-in providers (OpenRouter, OpenAI, Anthropic, etc.), see Built-in Providers. For provider resilience features (retries, fallbacks), see Provider Resilience.

Overview

ZeroClaw supports two custom provider prefixes:

Prefix API Format Use Case
custom: OpenAI /v1/chat/completions Any OpenAI-compatible service (local LLMs, cloud gateways, custom inference servers)
anthropic-custom: Anthropic /v1/messages Services implementing the Anthropic Messages API format

Custom providers are first-class citizens in the provider ecosystem. They receive the same resilience wrappers (retries, fallbacks, rate limiting) as built-in providers.

Sources: src/providers/mod.rs:744-770, docs/custom-providers.md:1-112

Provider Factory Flow

flowchart TD
    Config["Config: default_provider"] --> ParsePrefix["Parse Provider Name"]
    
    ParsePrefix --> CheckCustom{"Starts with\n'custom:' or\n'anthropic-custom:'?"}
    
    CheckCustom -->|No| BuiltIn["Route to Built-in Provider\n(OpenRouter, OpenAI, etc.)"]
    CheckCustom -->|Yes| ExtractURL["Extract URL Suffix"]
    
    ExtractURL --> ValidateURL["parse_custom_provider_url()"]
    
    ValidateURL --> CheckScheme{"Valid HTTP/HTTPS?"}
    CheckScheme -->|No| Error["anyhow::bail\nInvalid URL"]
    CheckScheme -->|Yes| DetermineType{"Provider Type?"}
    
    DetermineType -->|custom:| CreateCompatible["OpenAiCompatibleProvider::new\nAuthStyle::Bearer"]
    DetermineType -->|anthropic-custom:| CreateAnthropic["AnthropicProvider::with_base_url"]
    
    CreateCompatible --> ResolveCredential["resolve_provider_credential()"]
    CreateAnthropic --> ResolveCredential
    
    ResolveCredential --> WrapReliable["ReliableProvider::new\n(retries + fallbacks)"]
    
    WrapReliable --> Return["Return Box<dyn Provider>"]
    
    style ValidateURL fill:#f9f9f9
    style ResolveCredential fill:#f9f9f9
    style WrapReliable fill:#f9f9f9
Loading

Diagram: Custom Provider Instantiation Flow

The factory function create_provider_with_url in src/providers/mod.rs:592-778 handles custom provider creation. The flow:

  1. Prefix Detection: Lines 744 and 760 check for custom: and anthropic-custom: prefixes
  2. URL Extraction: strip_prefix() removes the prefix to get the base URL
  3. Validation: parse_custom_provider_url() src/providers/mod.rs:549-570 validates URL format and scheme
  4. Provider Construction: Instantiates either OpenAiCompatibleProvider or AnthropicProvider with the custom base URL
  5. Credential Resolution: resolve_provider_credential() src/providers/mod.rs:465-547 finds the API key
  6. Resilience Wrapping: The provider is wrapped in ReliableProvider by create_resilient_provider() src/providers/mod.rs:796-840

Sources: src/providers/mod.rs:592-778, src/providers/mod.rs:549-570, src/providers/mod.rs:796-840

URL Validation

flowchart TD
    Input["Raw URL String\ne.g., 'https://api.example.com'"]
    
    Input --> Trim["Trim Whitespace"]
    Trim --> CheckEmpty{"Empty?"}
    CheckEmpty -->|Yes| ErrEmpty["Error:\nURL required"]
    CheckEmpty -->|No| Parse["reqwest::Url::parse()"]
    
    Parse --> ParseOk{"Parse Success?"}
    ParseOk -->|No| ErrFormat["Error:\nInvalid URL format"]
    ParseOk -->|Yes| CheckScheme["Extract scheme()"]
    
    CheckScheme --> ValidScheme{"http:// or https://?"}
    ValidScheme -->|No| ErrScheme["Error:\nMust be http:// or https://"]
    ValidScheme -->|Yes| Success["Return Validated URL"]
    
    style Parse fill:#f9f9f9
    style Success fill:#e8f5e9
Loading

Diagram: URL Validation Process

The parse_custom_provider_url() function src/providers/mod.rs:549-570 enforces these constraints:

  1. Non-empty: Rejects blank strings
  2. Valid URL syntax: Uses reqwest::Url::parse() to validate structure
  3. HTTP/HTTPS only: Rejects other schemes (file://, ftp://, etc.)

Validation errors include helpful format hints like "custom:https://your-api.com" to guide users.

Sources: src/providers/mod.rs:549-570

Credential Resolution

flowchart TD
    Start["resolve_provider_credential(name, api_key_override)"]
    
    Start --> CheckOverride{"api_key parameter\nprovided?"}
    CheckOverride -->|Yes, non-empty| UseOverride["Return explicit\nAPI key"]
    CheckOverride -->|No or empty| CheckProviderEnv["Check provider-specific\nenv vars"]
    
    CheckProviderEnv --> EnvMapping["Environment Variable Mapping:\n- anthropic → ANTHROPIC_OAUTH_TOKEN,\n  ANTHROPIC_API_KEY\n- openrouter → OPENROUTER_API_KEY\n- openai → OPENAI_API_KEY\n- etc."]
    
    EnvMapping --> ProviderEnvFound{"Found?"}
    ProviderEnvFound -->|Yes| UseProviderEnv["Return provider-specific\nenv var value"]
    ProviderEnvFound -->|No| CheckGenericEnv["Check generic env vars:\n1. ZEROCLAW_API_KEY\n2. API_KEY"]
    
    CheckGenericEnv --> GenericFound{"Found?"}
    GenericFound -->|Yes| UseGeneric["Return generic\nenv var value"]
    GenericFound -->|No| ReturnNone["Return None\n(no credential)"]
    
    style UseOverride fill:#e8f5e9
    style UseProviderEnv fill:#e8f5e9
    style UseGeneric fill:#e8f5e9
    style ReturnNone fill:#ffebee
Loading

Diagram: Credential Resolution Hierarchy

For custom providers (custom: and anthropic-custom:), credential resolution follows this priority order implemented in resolve_provider_credential() src/providers/mod.rs:465-547:

Priority Source Example
1 Explicit api_key parameter in config.toml api_key = "sk-..."
2 Generic ZEROCLAW_API_KEY env var export ZEROCLAW_API_KEY="sk-..."
3 Generic API_KEY env var export API_KEY="sk-..."

Note: Unlike built-in providers (which check provider-specific env vars like OPENAI_API_KEY), custom providers only check the generic fallback variables. This is because the system cannot know which provider-specific variable to check when the provider name is arbitrary (e.g., custom:https://mycompany.ai).

Sources: src/providers/mod.rs:465-547, src/providers/mod.rs:537-545

OpenAI-Compatible Implementation

classDiagram
    class OpenAiCompatibleProvider {
        +String name
        +String base_url
        +Option~String~ credential
        +AuthStyle auth_header
        +bool supports_responses_fallback
        +Option~String~ user_agent
        +new(name, base_url, credential, auth_style)
        +chat_completions_url() String
        +responses_url() String
        +apply_auth_header(req, credential) RequestBuilder
    }
    
    class AuthStyle {
        <<enumeration>>
        Bearer
        XApiKey
        Custom(String)
    }
    
    class Provider {
        <<trait>>
        +chat_with_system()
        +chat_with_history()
        +chat_with_tools()
        +supports_native_tools() bool
    }
    
    OpenAiCompatibleProvider --> AuthStyle
    OpenAiCompatibleProvider ..|> Provider
Loading

Diagram: OpenAiCompatibleProvider Architecture

URL Construction

The OpenAiCompatibleProvider src/providers/compatible.rs:20-201 handles both standard and non-standard endpoint paths:

Standard Endpoint (most providers):

https://api.example.com → https://api.example.com/chat/completions

Full Path Endpoint (e.g., VolcEngine ARK):

https://api.example.com/api/coding/v3/chat/completions → (no modification)

Detection logic in chat_completions_url() src/providers/compatible.rs:128-146:

  • If base_url already ends with /chat/completions, use as-is
  • Otherwise, append /chat/completions

Authentication Styles

The AuthStyle enum src/providers/compatible.rs:32-40 supports three header formats:

Style HTTP Header Used By
Bearer Authorization: Bearer {key} Most OpenAI-compatible services (default for custom:)
XApiKey x-api-key: {key} Some Chinese providers
Custom(String) {custom_header}: {key} Specialized endpoints

Custom providers created with custom: prefix always use AuthStyle::Bearer src/providers/mod.rs:750-754.

Responses API Fallback

Some providers expose both /chat/completions (tool calling) and /v1/responses (simple text) endpoints. The fallback mechanism src/providers/compatible.rs:822-831:

  1. Attempt POST to /chat/completions
  2. If 404 Not Found and supports_responses_fallback = true, retry with /v1/responses
  3. Return combined error if both fail

GLM/Zhipu disables this fallback via new_no_responses_fallback() src/providers/compatible.rs:54-61 because it only supports chat completions.

Sources: src/providers/compatible.rs:20-201, src/providers/compatible.rs:128-146, src/providers/compatible.rs:32-40, src/providers/compatible.rs:822-831, src/providers/mod.rs:750-754

Anthropic-Compatible Implementation

classDiagram
    class AnthropicProvider {
        +Option~String~ credential
        +String base_url
        +new(credential)
        +with_base_url(credential, base_url) AnthropicProvider
        -is_setup_token(token) bool
        -apply_auth(request, credential) RequestBuilder
    }
    
    class Provider {
        <<trait>>
        +chat_with_system()
        +chat()
        +supports_native_tools() bool
    }
    
    AnthropicProvider ..|> Provider
    
    note for AnthropicProvider "Supports OAuth setup tokens\n(sk-ant-oat01-) and regular\nAPI keys (sk-ant-)"
Loading

Diagram: AnthropicProvider with Custom Base URL

Base URL Configuration

The AnthropicProvider::with_base_url() method src/providers/anthropic.rs:152-164 allows custom endpoints:

Default:

base_url = "https://api.anthropic.com"

Custom (via anthropic-custom: prefix):

base_url = "https://proxy.example.com"  // user-provided

All API calls append /v1/messages to the base URL src/providers/anthropic.rs:435.

Authentication Detection

Anthropic supports two credential types, auto-detected by is_setup_token() src/providers/anthropic.rs:166-168:

Token Type Prefix Auth Method Header
Setup Token (OAuth) sk-ant-oat01- Bearer Authorization: Bearer {token} + anthropic-beta: oauth-2025-04-20
Regular API Key sk-ant- Custom x-api-key: {key}

The apply_auth() method src/providers/anthropic.rs:170-182 automatically selects the correct authentication scheme based on token prefix detection.

Sources: src/providers/anthropic.rs:152-164, src/providers/anthropic.rs:166-182, src/providers/anthropic.rs:435

Configuration Examples

Config File (Recommended)

Edit ~/.zeroclaw/config.toml:

OpenAI-compatible endpoint:

default_provider = "custom:https://api.mycompany.com"
api_key = "custom-api-key-123"
default_model = "my-model"

Anthropic-compatible endpoint:

default_provider = "anthropic-custom:https://llm-proxy.corp.example.com"
api_key = "sk-internal-token"
default_model = "claude-3-5-sonnet"

Local LLM server:

default_provider = "custom:http://localhost:8080"
default_model = "llama-3.1-8b"
api_key = "optional-key"  # Optional for localhost

Environment Variables

Custom providers use the generic credential environment variables:

export ZEROCLAW_API_KEY="your-api-key"
# or
export API_KEY="your-api-key"

zeroclaw agent -m "test message"

Command-Line Override

Specify provider and model at runtime:

# Interactive mode
zeroclaw agent --provider "custom:https://api.example.com" --model "gpt-4"

# Single message
zeroclaw agent --provider "anthropic-custom:https://proxy.local" \
                --model "claude-3-opus" \
                -m "Hello"

Sources: docs/custom-providers.md:29-59, src/providers/mod.rs:537-545

Resilience Integration

flowchart LR
    CustomProvider["Custom Provider\n(OpenAiCompatibleProvider\nor AnthropicProvider)"] --> Reliable["ReliableProvider Wrapper"]
    
    Reliable --> Features["Resilience Features:\n• Exponential backoff\n• Max retries (configurable)\n• API key rotation\n• Model fallbacks\n• Rate limit handling"]
    
    Features --> Return["Box<dyn Provider>"]
    
    style Reliable fill:#f9f9f9
Loading

Diagram: Custom Provider Resilience Wrapping

Custom providers receive the same resilience features as built-in providers via create_resilient_provider() src/providers/mod.rs:796-840. Configuration in config.toml:

[reliability]
provider_retries = 3
provider_backoff_ms = 1000
fallback_providers = ["openrouter", "anthropic"]  # Fallback if custom fails
model_fallbacks = { "my-model" = ["backup-model-1", "backup-model-2"] }

The ReliableProvider wrapper src/providers/reliable.rs:182-653 provides:

  1. Retry Logic: Exponential backoff with configurable base delay src/providers/reliable.rs:241-248
  2. Error Classification: Distinguishes retryable (network, 5xx) from non-retryable (auth, 4xx) errors src/providers/reliable.rs:10-56
  3. Rate Limit Handling: Parses Retry-After headers src/providers/reliable.rs:118-147
  4. Key Rotation: Cycles through multiple API keys on rate limit src/providers/reliable.rs:232-238
  5. Model Fallbacks: Tries alternative models if primary fails src/providers/reliable.rs:223-229

Sources: src/providers/mod.rs:796-840, src/providers/reliable.rs:182-653, src/providers/reliable.rs:10-56, src/providers/reliable.rs:118-147

Troubleshooting

URL Validation Errors

Error: "Custom provider requires a valid URL"

Cause: URL failed reqwest::Url::parse() validation

Solution: Ensure URL includes scheme (http:// or https://) and valid domain:

# ✗ Invalid
default_provider = "custom:api.example.com"

# ✓ Valid
default_provider = "custom:https://api.example.com"

Authentication Failures

Error: "API key not set" or 401 Unauthorized

Resolution steps:

  1. Check credential presence src/providers/compatible.rs:780-785:

    zeroclaw agent  # Error message shows which env vars to set

  2. Verify credential resolution:

    export ZEROCLAW_API_KEY="your-key"
echo $ZEROCLAW_API_KEY  # Confirm it's set

  3. Test with explicit config:

    api_key = "your-key"  # Explicit beats env vars

  4. Check auth header format: Custom providers use Bearer tokens. If your endpoint requires a different format, you may need to request a built-in provider addition or submit a PR to add custom auth style support.

Model Availability

Error: "Model not found" or 400 Bad Request

Diagnosis:

  1. List available models (if endpoint supports /models):

    curl -H "Authorization: Bearer $ZEROCLAW_API_KEY" \
     https://your-api.com/v1/models

  2. Check model name format: Some providers require namespace prefixes (e.g., models/gemini-pro vs gemini-pro)

  3. Verify endpoint compatibility: Ensure the service actually implements OpenAI or Anthropic API format. Test with a known working model first:

    zeroclaw agent --provider "custom:https://your-api.com" \
               --model "gpt-3.5-turbo" \
               -m "test"

Connection Issues

Error: reqwest::Error with connection timeout or refused

Checklist:

  1. Test raw connectivity:

    curl -I https://your-api.com

  2. Check proxy settings src/config/mod.rs. ZeroClaw respects HTTP_PROXY/HTTPS_PROXY environment variables.

  3. Verify TLS certificate: Some corporate proxies use self-signed certificates. Check logs for TLS errors.

  4. Local endpoints: For http://localhost:* or http://127.0.0.1:*, ensure the local server is running:

    # Example: Ollama
ollama serve

# Then test
zeroclaw agent --provider "custom:http://localhost:11434" --model "llama3"

Sources: src/providers/compatible.rs:780-834, docs/custom-providers.md:62-88

Implementation Reference

Key Functions

Function Location Purpose
create_provider_with_url src/providers/mod.rs:592-778 Main factory for all providers including custom
parse_custom_provider_url src/providers/mod.rs:549-570 Validates and normalizes custom URLs
resolve_provider_credential src/providers/mod.rs:465-547 Hierarchical credential resolution
create_resilient_provider src/providers/mod.rs:796-840 Wraps providers with retry/fallback logic

Core Types

Type Location Description
OpenAiCompatibleProvider src/providers/compatible.rs:20-202 Generic OpenAI-format provider
AnthropicProvider src/providers/anthropic.rs:10-164 Anthropic Messages API provider
AuthStyle src/providers/compatible.rs:32-40 Authentication header format enum
ReliableProvider src/providers/reliable.rs:183-193 Resilience wrapper with retries

Configuration Keys

Key Type Description Default
default_provider String Provider name with optional custom: or anthropic-custom: prefix "openrouter"
api_key String API credential (or use env vars) None
default_model String Model identifier Provider-specific
api_url String Alternative to prefix syntax (applies only to primary provider) None

Sources: src/providers/mod.rs:592-778, src/providers/compatible.rs:20-202, src/providers/anthropic.rs:10-164, src/providers/reliable.rs:183-193

Home

  1. Overview

  2. Getting Started

  3. Core Architecture

  4. Configuration

  5. Providers

  6. Channels

  7. Agent Core

  8. Tools

  9. Memory System

  10. Gateway

  11. Scheduler

  12. Development Guide

  13. Deployment

  14. Maturity Framework

Clone this wiki locally
⚡