🌍 Global Mirror — Visit original CN site →
Skip to main content

OpenAI-Compatible

Overview​

Open WebUI connects to any server or provider that implements the OpenAI-compatible API. This guide covers how to set up connections for popular cloud providers and local servers.

For OpenAI itself (or Azure OpenAI), see the dedicated OpenAI guide.

Protocol-Oriented Design​

Open WebUI is built around Standard Protocols. Instead of building specific modules for every individual AI provider (which leads to inconsistent behavior and configuration bloat), Open WebUI focuses on protocols like the OpenAI Chat Completions Protocol.

This means that while Open WebUI handles the interface and tools, it expects your backend to follow the universal Chat Completions standard.

  • We Support Protocols: Any provider that follows widely adopted API standards is natively supported. We also have experimental support for Open Responses.
  • We Avoid Proprietary APIs: We do not implement provider-specific, non-standard APIs in the core to maintain a universal, maintainable codebase. For unsupported providers, use a pipe or a middleware proxy like LiteLLM or OpenRouter to bridge them.

For a detailed explanation of this architectural decision, see our FAQ on protocol support.

Important: Connection Verification May Fail for Some Providers

When you add a connection, Open WebUI verifies it by calling the provider's /models endpoint using a standard Bearer token. Some providers do not implement the /models endpoint at all or use non-standard authentication for it. In these cases:

  • The connection verification will fail with an error (e.g., 400, 401 or 403).
  • This does not mean the provider is incompatible — chat completions will still work.
  • You just need to manually add model names to the Model IDs (Filter) allowlist in the connection settings.

Providers with known /models issues:

Provider/models works?Action Needed
AnthropicYes — built-in compatibility layerAuto-detection works
GitHub ModelsNo — uses non-standard pathAdd model IDs manually to the whitelist
PerplexityNo — endpoint doesn't existAdd model IDs manually to the whitelist
MiniMaxNo — endpoint doesn't existAdd model IDs manually to the whitelist
OpenRouterYes — but returns thousands of modelsStrongly recommend adding a filtered allowlist
Google GeminiYesAuto-detection works
DeepSeekYesAuto-detection works
MistralYesAuto-detection works
GroqYesAuto-detection works

How to add models manually: In the connection settings, find Model IDs (Filter), type the model ID, and click the + icon, then save. The models will then appear in your model selector even though the connection verification showed an error.

Step 1: Add Your Provider Connection​

  1. Open Open WebUI in your browser.
  2. Go to ⚙️ Admin Settings → Connections → OpenAI.
  3. Click ➕ Add Connection.
  4. Fill in the URL and API Key for your provider (see tabs below). The URL field will suggest common provider endpoints as you type.
  5. If your provider doesn't support /models auto-detection, add your model IDs to the Model IDs (Filter) allowlist.
  6. Click Save.
tip

If running Open WebUI in Docker and your model server is on the host machine, replace localhost with host.docker.internal in the URL.

Enable/Disable Connections

Each connection has a toggle switch that lets you enable or disable it without deleting the connection. This is useful for temporarily deactivating a provider while preserving its configuration.

Cloud Providers​

tip

See the dedicated Anthropic (Claude) guide for a full step-by-step walkthrough.

Anthropic (Claude) offers an OpenAI-compatible endpoint. Open WebUI includes a built-in compatibility layer that automatically detects Anthropic URLs and handles model discovery — just plug in your API key and models are auto-detected. Note that this is intended for testing and comparison — for production use with full Claude features (PDF processing, citations, extended thinking, prompt caching), Anthropic recommends their native API.

SettingValue
URLhttps://api.anthropic.com/v1
API KeyYour Anthropic API key from console.anthropic.com
Model IDsAuto-detected — leave empty or filter to specific models

Local Servers​

Llama.cpp runs efficient, quantized GGUF models locally with an OpenAI-compatible API server. See the dedicated Llama.cpp guide for full setup instructions (installation, model download, server startup).

SettingValue
URLhttp://localhost:10000/v1 (or your configured port)
API KeyLeave blank

Quick start:

./llama-server --model /path/to/model.gguf --port 10000 --ctx-size 1024 --n-gpu-layers 40
Connection Timeout Configuration

If your server is slow to start or you're connecting over a high-latency network, you can adjust the model list fetch timeout:

# Adjust timeout for slower connections (default is 10 seconds)
AIOHTTP_CLIENT_TIMEOUT_MODEL_LIST=5

If you've saved an unreachable URL and the UI becomes unresponsive, see the Model List Loading Issues troubleshooting guide for recovery options.

Required API Endpoints​

To ensure full compatibility with Open WebUI, your server should implement the following OpenAI-standard endpoints:

EndpointMethodRequired?Purpose
/v1/modelsGETRecommendedUsed for model discovery and selecting models in the UI. If not available, add models to the allowlist manually.
/v1/chat/completionsPOSTYesThe core endpoint for chat, supporting streaming and parameters like temperature.
/v1/embeddingsPOSTNoRequired if you want to use this provider for RAG (Retrieval Augmented Generation).
/v1/audio/speechPOSTNoRequired for Text-to-Speech (TTS) functionality.
/v1/audio/transcriptionsPOSTNoRequired for Speech-to-Text (STT/Whisper) functionality.
/v1/images/generationsPOSTNoRequired for Image Generation (DALL-E) functionality.

Supported Parameters​

Open WebUI passes standard OpenAI parameters such as temperature, top_p, max_tokens (or max_completion_tokens), stop, seed, and logit_bias. It also supports Tool Use (Function Calling) if your model and server support the tools and tool_choice parameters.

Step 2: Start Chatting!​

Select your connected provider's model in the chat menu and get started!

That's it! Whether you choose a cloud provider or a local server, you can manage multiple connections — all from within Open WebUI.

🚀 Enjoy building your perfect AI setup!