Skip to main content

Capabilities

Besides translating requests, the DIAL OpenAI Adapter adds behavior that DIAL relies on: it reports token usage for every response, counts and truncates prompts, exposes model settings, surfaces reasoning, caches prompts, and passes the native Anthropic API through to Claude. This reference describes each capability and the DIAL Core settings that turn it on. It is for platform engineers who tune models in DIAL; it assumes you know Supported upstreams.

Token usage​

The adapter includes token usage in every chat completion response, which DIAL Core needs for limits and cost.

  • When the upstream returns usage, the adapter passes it on.

  • When it does not, as Azure OpenAI and the OpenAI Platform do for streaming by default, the adapter counts tokens itself. Counting is CPU-heavy and can throttle requests under load.

  • To avoid that, ask the upstream for usage in streaming responses with a model default in DIAL Core:

    "defaults": { "stream_options": { "include_usage": true } }

The adapter counts no tokens for deployments in DATABRICKS_DEPLOYMENTS and MISTRAL_DEPLOYMENTS, and for the legacy Completions, Images, and Responses APIs. For other deployments it uses:

ContentHow it is counted
Texttiktoken, with the encoding from TIKTOKEN_MODEL_MAPPING, the deployment ID itself, or the gpt-4o encoding as the default
ImagesThe Azure image-token rules for deployments in GPT4O_DEPLOYMENTS or GPT4O_MINI_DEPLOYMENTS; zero otherwise
vLLM deploymentsvLLM's own /tokenize endpoint, for the whole request
Documents, audio, videoNot counted: zero tokens

Tokenize and truncate prompt​

  • /tokenize counts the tokens of a request or a string, following the DIAL SDK schema.
  • /truncate_prompt is the dry run of max_prompt_tokens: it reports which messages would be dropped to fit a token budget, without calling the model.
  • max_prompt_tokens in a chat request truncates the conversation before it reaches the model, dropping whole messages from the oldest.

Expose the endpoints to DIAL clients through the model's features:

"features": {
  "tokenizeEndpoint": "<ADAPTER>/openai/deployments/<UPSTREAM_DEPLOYMENT>/tokenize",
  "truncatePromptEndpoint": "<ADAPTER>/openai/deployments/<UPSTREAM_DEPLOYMENT>/truncate_prompt"
}

For Responses API models, only OpenAI Platform upstreams can count tokens. On Azure OpenAI and Amazon Bedrock, /tokenize and /truncate_prompt answer 404, and max_prompt_tokens is ignored with a warning.

Configurable models​

Some models take extra settings in custom_fields.configuration of the chat request. GET /openai/deployments/<id>/configuration returns the JSON Schema of those settings; expose it to clients with features.configurationEndpoint.

ModelsWhat you can configure
DALL·E 3, GPT Image 1Image generation parameters, such as size, style, and quality. The schema is open, so parameters added in later API versions also pass through.
SoraVideo width and height; 480 × 480 by default.
Models on a Responses API upstreamResponses API features, such as reasoning.
ClaudeExtended thinking, reasoning level, beta feature flags, and citations; see the Anthropic adapter.
{
  "messages": [{ "role": "user", "content": "A watercolor of a lighthouse" }],
  "custom_fields": { "configuration": { "size": "1024x1024", "style": "vivid" } }
}

Reasoning​

  • Reasoning content. Reasoning tokens from reasoning models are reported twice: in a DIAL stage titled Reasoning, visible in DIAL Chat, and in the reasoning_content field of the message, which streams like content.

  • Reasoning summaries. For models on a Responses API upstream, set the reasoning parameter in the request:

    "custom_fields": { "configuration": { "reasoning": { "effort": "medium", "summary": "auto" } } }

    Not all models support reasoning; check the provider's documentation first.

  • Web search. Models on an Azure OpenAI Responses API upstream support OpenAI's web search tool. Pass a static function named web_search in tools. Each search becomes a DIAL stage, and URL citations become attachments.

Prompt caching​

ProviderHow to turn it on
Azure OpenAI and Azure AI FoundrySet features.autoCachingSupported: true on the model. Check that the deployment supports prompt caching first.
Alibaba Cloud Model StudioSet extra_data.vendor: "alibaba-cloud" on the upstream. Chat Completions models then honor DIAL cache breakpoints, at the top level or per message; Responses API models cache the conversation prefix for the whole session.
Claude (Anthropic Messages)Automatic and explicit prompt caching through the Anthropic adapter.

See Prompt caching for how DIAL routes cached prompts.

Load balancing and upstream headers​

  • Several upstreams. List several upstreams on one model, for example Azure OpenAI resources in different regions; DIAL Core balances and fails over between them. See Load balancing and upstreams.
  • Header proxying. An upstream's extra_data can name incoming request headers for the adapter to forward to the provider. For example, forwarding DIAL Chat's x-conversation-id lets a vLLM cluster pin all turns of a conversation to one worker. The same field carries AWS credentials for Bedrock upstreams.

Azure API versions​

Azure OpenAI's dated API needs an api-version query parameter, and old versions get retired. API_VERSIONS_MAPPING lets operators manage versions for applications:

  • Map a retired version to a current one: {"2023-01-01-preview": "2025-06-01"}.
  • Set a default for requests without a version: {"": "2025-06-01"}.

API versions do not apply to the v1 Chat Completions API or the Responses API, which are not versioned.

Anthropic Messages passthrough​

The adapter also serves the native Anthropic Messages API at /anthropic, for Claude models on Azure AI Foundry, the Anthropic API, or another provider of the same API. Clients built on the Anthropic SDK, such as Claude Code, then work through DIAL without translation. Declare the interface on the DIAL Core model:

"<DIAL_MODEL_ID>": {
  "type": "chat",
  "overrideName": "<CLAUDE_MODEL>",
  "interfaces": {
    "anthropicMessages": { "base_url": "<ADAPTER>" }
  },
  "upstreams": [
    { "endpoint": "https://api.anthropic.com/v1/messages", "key": "<ANTHROPIC_API_KEY>" }
  ]
}

The proxied endpoints and their errors are described in the Anthropic adapter. To set up Claude Code against DIAL, see Claude Code.

Further reading​