Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions apiclient/types/model.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ type ModelManifest struct {
Alias string `json:"alias,omitempty"`
Active bool `json:"active"`
Usage ModelUsage `json:"usage"`
Dialect string `json:"dialect,omitempty"`
}

type ModelList List[Model]
Expand Down
3 changes: 3 additions & 0 deletions apiclient/types/modelprovider.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ type CommonProviderMetadata struct {
IconDark string `json:"iconDark,omitempty"`
Description string `json:"description,omitempty"`
Link string `json:"link,omitempty"`
// Dialect specifies the LLM API format used by this provider
// (e.g. "AnthropicMessages", "OpenAIChatCompletions", "OpenAIResponses").
Dialect string `json:"dialect,omitempty"`
}

type CommonProviderStatus struct {
Expand Down
69 changes: 44 additions & 25 deletions docs/docs/configuration/model-providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ Obot supports a variety of model providers, including:
- Google

**Enterprise**
- [Azure OpenAI / Microsoft Foundry](#azure-openai-enterprise-only)
- [Azure OpenAI / Microsoft Foundry](#azure-enterprise-only)
Comment thread
calvinmclean marked this conversation as resolved.
- Amazon Bedrock
- Google Vertex (Gemini models)

Expand Down Expand Up @@ -71,51 +71,70 @@ Setting a default model here does not automatically grant users access to it. Us

### Instructions for configuring specific providers

#### Azure OpenAI (Enterprise only)
#### Azure (Enterprise only)

The Azure OpenAI model provider supports legacy Azure OpenAI resources. Microsoft Foundry works with API key authentication as well.

This model provider supports two forms of authentication: API keys and Microsoft Entra.
Obot supports two Azure providers, each with a different authentication method. These are compatible with both Azure OpenAI deployments and Foundry deployments.

##### API Key Authentication

In the Azure OpenAI or Microsoft Foundry UI, you can find your API key after you have set up at least one deployment, as well as your endpoint URL.
Both of these values are required when you configure this model provider in Obot.
Use the **Azure** provider for API key-based authentication.

Additionally, you must manually specify the names of all of your deployments, as the API key does not provide the ability to list them.
The format is `name:type`, for example, `gpt-5.2:reasoning-llm`. The supported types are `llm`, `reasoning-llm`, `text-embedding`, and `image-generation`.
If no type is specified, Obot will assume the type is `llm`.
In the Azure portal, find your API key and endpoint URL after setting up at least one deployment — both are required.

If the type specified is `llm` or none at all, and the deployment name starts with the name of a known reasoning model (such as `o3` or `gpt-5`), Obot
will automatically treat it as a reasoning model.
You must also specify deployment names. The format is a comma-separated list of deployment names, optionally as `model:deployment` pairs (e.g. `gpt-4o,gpt-4o-mini` or `gpt-4o:my-gpt4o,gpt-4o-mini:my-mini`).

##### Microsoft Entra Authentication
You can also optionally specify the API version (defaults to `2025-01-01-preview`).

:::important
At this time, Microsoft Entra authentication is only supported for Azure OpenAI deployments and not for the newer Microsoft Foundry deployments.
:::
##### Microsoft Entra ID Authentication

Use the **Azure (Entra ID)** provider for service principal authentication via Microsoft Entra ID. Deployments are discovered automatically from the Azure Management API.

###### 1. Create a service principal

```bash
az ad sp create-for-rbac --name "<sp-name>" \
--role "Cognitive Services OpenAI User" \
--scopes /subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.CognitiveServices/accounts/<account-name>
```

This outputs the `appId` (Client ID), `password` (Client Secret), and `tenant` (Tenant ID) needed below.

###### 2. Find your resource details

```bash
az cognitiveservices account show \
--name <account-name> \
--resource-group <resource-group> \
--query "{endpoint:properties.endpoint, id:id}"
```

Instead of using an API key, you can set up a Microsoft Entra app registration as a service principal to use Azure OpenAI.
###### 3. Configure the provider

Obot requires the Client ID, Client Secret, and Tenant ID of the Entra app, as well as the Endpoint URL, Subscription ID, and Resource Group from Azure OpenAI/Microsoft Foundry.
Obot requires:
- **Azure Endpoint** — your Azure OpenAI endpoint URL (`https://<resource_name>.openai.azure.com`)
- **Client ID** — the Entra app's application (client) ID
- **Client Secret** — the Entra app's client secret
- **Tenant ID** — the Entra app's tenant ID
- **Subscription ID** — the Azure subscription ID containing the Cognitive Services account
- **Resource Group** — the resource group containing the Cognitive Services account
- **Account Name** — the Cognitive Services account name

You do NOT need to manually specify your deployment names, as the Entra app's credentials will be sufficient to list them.
You can also optionally specify the API version (defaults to `2025-01-01-preview`).

After you have created your Entra app registration, you need to go to your Azure OpenAI resource in the Azure portal and add a new role assignment for the app registration, as a service principal.
It needs to have the `Cognitive Services OpenAI User` role.
The service principal requires at minimum the `Cognitive Services OpenAI User` or `Cognitive Services User` role on the account to read deployments. Deployments are discovered automatically — each deployment's base model name becomes the model ID exposed to Obot.

See the [Microsoft docs](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/role-based-access-control?view=foundry-classic#add-role-assignment-to-an-azure-openai-resource) for more details.
See the [Microsoft docs](https://learn.microsoft.com/en-us/azure/foundry/foundry-models/how-to/configure-entra-id) for more details.

#### Ollama

[Ollama](https://ollama.ai/) allows you to run LLMs locally. Two configuration steps are required to use it with Obot:

1. **Expose Ollama to the network** - By default, Ollama only binds to `127.0.0.1:11434`. Since Obot runs in a container, `localhost` addresses resolve to Obot's container, not your host. Set `OLLAMA_HOST=0.0.0.0` before starting Ollama, then use your host's IP address in the endpoint URL.

2. **Use the OpenAI-compatible endpoint** - The endpoint must include the `/v1/` path:
2. **Set the Ollama host**
```
http://<your-host-ip>:11434/v1/
http://<your-host-ip>:11434
```
Using `http://<host>:11434/` without `/v1/` will result in validation errors.
- If you are running Obot in Docker, you should use `http://host.docker.internal:11434`. For linux users, run Obot in Docker with this additional flag `--add-host=host.docker.internal:host-gateway` or use an alternative method of allowing the container to access the host network

See [Ollama's FAQ](https://docs.ollama.com/faq) for platform-specific instructions on setting `OLLAMA_HOST`.
4 changes: 2 additions & 2 deletions docs/docs/configuration/server-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,8 @@ The Obot server is configured via environment variables. The following configura
| `OBOT_SERVER_AUDIT_LOGS_COMPRESS_FILE` | Controls whether or not to compress audit log files | `true` |
| `OBOT_SERVER_AUDIT_LOGS_USE_PATH_STYLE` | Whether to use path style for S3 | - |
| `OBOT_SERVER_MCPBASE_IMAGE` | Deploy MCP servers in the kubernetes cluster or using docker with this base image. | `ghcr.io/obot-platform/mcp-images/phat:v0.20.2` |
| `OBOT_SERVER_MCPREMOTE_SHIM_BASE_IMAGE` | Deploy MCP remote shim servers in the cluster using this base image. | `ghcr.io/nanobot-ai/nanobot:v0.0.68` |
| `OBOT_SERVER_NANOBOT_AGENT_IMAGE` | Deploy the Nanobot agent in the cluster using this image. | `ghcr.io/nanobot-ai/nanobot-agent:v0.0.68` |
| `OBOT_SERVER_MCPREMOTE_SHIM_BASE_IMAGE` | Deploy MCP remote shim servers in the cluster using this base image. | `ghcr.io/nanobot-ai/nanobot:v0.0.69` |
| `OBOT_SERVER_NANOBOT_AGENT_IMAGE` | Deploy the Nanobot agent in the cluster using this image. | `ghcr.io/nanobot-ai/nanobot-agent:v0.0.69` |
| `OBOT_SERVER_MCPHTTPWEBHOOK_BASE_IMAGE` | Deploy MCP HTTP webhook servers in the cluster using this base image. | `ghcr.io/obot-platform/mcp-images/http-webhook-mcp-converter:v0.20.2` |
| `OBOT_SERVER_MCPRUNTIME_BACKEND` | The runtime backend to use for running MCP servers: docker, kubernetes, or local. | `kubernetes` in the helm chart, `docker` otherwise |
| `OBOT_SERVER_MCPCLUSTER_DOMAIN` | The cluster domain to use for MCP services. Only matters if `OBOT_SERVER_MCPBASE_IMAGE` is set. | `cluster.local` |
Expand Down
Loading
Loading