Platforms & Managed Endpoints

This page covers providers that route through a platform-specific gateway, enterprise cloud integration, managed inference fabric, or regional model marketplace rather than a single first-party vendor API.

Included Providers

• Replicate
• NVIDIA NIM
• DeepInfra
• Azure OpenAI
• GitHub Models (Azure AI Inference)
• Qwen (DashScope)
• MiniMax
• Qianfan (Baidu)
• VolcEngine (Doubao)
• BytePlus ModelArk
• Zhipu (GLM)
• Zhipu Coding (CodeGeex)
• Z.ai
• Vertex AI

Note on Bedrock: AWS Bedrock is documented in Hosted APIs since the current driver authenticates via a long-lived bearer token (AWS_BEARER_TOKEN_BEDROCK) rather than per-request SigV4 signing. The older SigV4 entry that used to live on this page has been removed.

Replicate

Available Models:

• replicate/meta-llama-3.3-70b-instruct (Balanced)

Setup:

1. Sign up at replicate.com
2. Go to Account > API Tokens
3. export REPLICATE_API_TOKEN="r8_..."

NVIDIA NIM

Setup:

1. Sign up at build.nvidia.com
2. Create an API key
3. export NVIDIA_API_KEY="nvapi-..."

DeepInfra

Setup:

1. Sign up at deepinfra.com
2. Create an API key
3. export DEEPINFRA_API_KEY="..."

Azure OpenAI

Setup:

1. Create an Azure OpenAI resource in the Azure Portal
2. Deploy a model in Azure OpenAI Studio
3. Set environment variables:

   export AZURE_OPENAI_API_KEY="..."
   export AZURE_OPENAI_ENDPOINT="https://<your-resource>.openai.azure.com"
   

   CopyCopied!

GitHub Models (Azure AI Inference)

Setup:

1. Sign in to github.com/marketplace/models
2. Generate a fine-grained Personal Access Token (no repository scopes are required for the Models endpoint)
3. export GITHUB_MODELS_TOKEN="ghp_..." (the same PAT works as GITHUB_TOKEN for github-copilot, but the env vars are kept separate so configuring one product doesn't auto-activate the other)

Minimal config.toml:

[default_model]
provider = "microsoft"
model = "phi-4"

Notes: Despite the microsoft provider id, this is the GitHub Models / Azure AI Inference endpoint, distinct from azure-openai (which addresses your own Azure OpenAI deployments via AZURE_OPENAI_ENDPOINT) and from github-copilot (which is the IDE-side Copilot subscription). Models are catalog-driven and include third-party hosts like meta-llama-3-70b-instruct, mistral-large, phi-4, etc.

Qwen (DashScope)

Regions:

Setup:

1. Sign up at DashScope Console
2. Create an API key
3. export DASHSCOPE_API_KEY="sk-..."
4. Optionally select a region in config.toml:

   [provider_regions]
   qwen = "intl"    # or "us"
   

   CopyCopied!

Notes: Qwen uses Alibaba Cloud's DashScope API. The default endpoint serves mainland China; use the intl or us region for lower latency outside China. Models are defined in the registry TOML and loaded at boot.

MiniMax

Regions:

Setup:

1. Sign up at minimax.io (international) or minimaxi.com (China)
2. Create an API key
3. export MINIMAX_API_KEY="..."
4. For China region:

   [provider_regions]
   minimax = "china"
   

   CopyCopied!

   export MINIMAX_CN_API_KEY="..."
   

   CopyCopied!

Media Generation: In addition to LLM chat models, MiniMax provides media generation capabilities when used with the Creator Hand or media API endpoints:

These are accessed via /api/media/* endpoints or through the image_generate, video_generate, music_generate, text_to_speech tools.

Notes: MiniMax international (minimax.io) and China (minimaxi.com) use separate API keys. When selecting the china region, LibreFang automatically reads from MINIMAX_CN_API_KEY instead of MINIMAX_API_KEY.

Qianfan (Baidu)

Setup:

1. Sign up at the Baidu Qianfan console
2. Create an API key under the IAM panel
3. export QIANFAN_API_KEY="..."

Minimal config.toml:

[default_model]
provider = "qianfan"
model = "ernie-4.0-8k"

Notes: Qianfan is Baidu's LLM platform; the ernie-4.0, ernie-4.0-turbo, and ernie-speed families are exposed via the OpenAI-compatible v2 endpoint. Tools and function calling are supported on ernie-4.0+.

VolcEngine (Doubao)

Setup:

1. Sign up at the VolcEngine console
2. Create an inference endpoint in Ark > Endpoints and note its model ID (ep-...)
3. Generate an API key under Ark > API Keys
4. export VOLCENGINE_API_KEY="..."

Minimal config.toml:

[default_model]
provider = "volcengine"
model = "ep-20240101120000-abcde"   # endpoint ID, not the model name

Notes: Doubao is ByteDance's LLM product, exposed via VolcEngine's Ark inference platform. Unlike most providers, the model field must be the endpoint ID (ep-...), not the model family name. Vision and tools are supported on Doubao 1.5 / 1.6 endpoints. The Coding Plan endpoint (https://ark.cn-beijing.volces.com/api/coding/v3) is registered as the separate provider volcengine_coding, which uses its own VOLCENGINE_CODING_API_KEY — set the same key value under both env vars if you use both endpoints.

BytePlus ModelArk

International edition of VolcEngine. Same Ark platform, different region (ap-southeast) and pricing in USD. The two endpoints below have separate env vars (BYTEPLUS_API_KEY for byteplus, BYTEPLUS_CODING_API_KEY for byteplus_coding) — the same BytePlus account key works for both, you just export it under each name you actually use.

byteplus — standard /api/v3 endpoint

[default_model]
provider = "byteplus"
model = "seed-2-0-pro-260328"       # versioned snapshot, see byteplus.toml

byteplus_coding — Anthropic-compatible coding endpoint

[default_model]
provider = "byteplus_coding"
model = "ark-code-latest"           # auto-routed friendly alias

Setup:

1. Sign up at the BytePlus console
2. Activate the models you want under ModelArk > Model Inventory
3. Generate an API key under ModelArk > API Keys
4. export BYTEPLUS_API_KEY="..." (for the standard byteplus endpoint)
5. export BYTEPLUS_CODING_API_KEY="..." (for the byteplus_coding endpoint — same key value, separate env)

Notes: byteplus uses versioned snapshot IDs like seed-2-0-pro-260328. byteplus_coding uses friendly aliases like ark-code-latest (auto-router), dola-seed-2.0-pro, kimi-k2.5, glm-4.7, gpt-oss-120b — these stay stable across model upgrades.

Zhipu (GLM)

Setup:

1. Sign up at the Zhipu open platform
2. Generate an API key under Account > API Keys
3. export ZHIPU_API_KEY="..."

Minimal config.toml:

[default_model]
provider = "zhipu"
model = "glm-4-plus"

Notes: Zhipu develops the GLM (General Language Model) family. Tools, vision, and embeddings supported. z.ai (below) is the international-region front-end of the same backend but is registered as its own provider with its own ZAI_API_KEY. The Coding Plan endpoints (zhipu_coding, zai_coding) similarly use their own ZHIPU_CODING_API_KEY and ZAI_CODING_API_KEY. Register once at Zhipu and re-export the same key value under each env var you actually use.

Zhipu Coding (CodeGeex)

Setup:

1. Use the same Zhipu API key from above (register once on the Zhipu open platform)
2. export ZHIPU_CODING_API_KEY="..." (same value as ZHIPU_API_KEY, separate env so configuring Zhipu chat doesn't auto-activate this endpoint)

Minimal config.toml:

[default_model]
provider = "zhipu-coding"
model = "codegeex-4"

Notes: Coding-specialized endpoint serving CodeGeex models. Same auth as Zhipu/GLM but different base URL — use this provider when you want fill-in-the-middle / repo-aware coding completions instead of generic chat.

Z.ai

Setup:

1. Use the same Zhipu API key (register once on the Zhipu open platform)
2. export ZAI_API_KEY="..." (same value as ZHIPU_API_KEY, separate env so configuring Zhipu chat doesn't auto-activate this endpoint)

Minimal config.toml:

[default_model]
provider = "z.ai"
model = "glm-4-plus"

Notes: Branded Z.ai endpoint — same Zhipu backend, different routing for international traffic. The coding variant zai_coding (base URL https://api.z.ai/api/coding/paas/v4) is registered as its own provider; it uses ZAI_CODING_API_KEY (re-export the same Zhipu key value).

Vertex AI

Setup:

1. Enable Vertex AI API in Google Cloud Console
2. Either create a service account key file, or authenticate with gcloud auth application-default login
3. Set environment variables:

   # Option A: Service account key file
   export GOOGLE_APPLICATION_CREDENTIALS="/path/to/service-account.json"
   export VERTEX_PROJECT="your-gcp-project"
   export VERTEX_LOCATION="us-central1"
   
   # Option B: gcloud CLI (no key file needed)
   gcloud auth application-default login
   export VERTEX_PROJECT="your-gcp-project"
   export VERTEX_LOCATION="us-central1"
   

   CopyCopied!
4. Configure in config.toml:

   [vertex_ai]
   project = "your-gcp-project"
   location = "us-central1"
   

   CopyCopied!

Notes: Vertex AI uses the same Gemini generateContent API format as the native Gemini driver but authenticates via Google Cloud OAuth2 instead of an API key. Access tokens are cached with a ~50-minute TTL and auto-refreshed before expiry. The endpoint format is https://{location}-aiplatform.googleapis.com/v1/projects/{project}/locations/{location}/publishers/google/models/{model}:generateContent.