Skip to main content

Configuration

The DIAL OpenAI Adapter is configured with environment variables on its container, while each model is configured in DIAL Core. This reference lists every variable of the adapter: the deployment categories that select an API and a tokenizer, credentials, performance, and logging. It is for platform engineers who run the adapter; for the model side, see Supported upstreams.

Deployment categories​

These variables group deployments that share an API and a tokenization algorithm. Each takes a comma-separated list of upstream deployment IDs, the ID in the adapter URL. A deployment in no category is treated as a text chat or text embeddings model on the OpenAI API.

VariableDefaultDescription
DALLE3_DEPLOYMENTS—Deployments on the DALL·E 3 API, for example dall-e-3,dalle3.
DALLE3_AZURE_API_VERSION2024-02-01The API version for the Azure DALL·E 3 API.
GPT_IMAGE_1_DEPLOYMENTS—Deployments on the GPT Image 1 API, for example gpt-image-1.
GPT_IMAGE_1_AZURE_API_VERSION2024-02-01The API version for the Azure GPT Image 1 API.
MISTRAL_DEPLOYMENTS—Deployments on the Mistral API.
DATABRICKS_DEPLOYMENTS—Databricks chat deployments.
GPT4O_DEPLOYMENTS—GPT-4o deployments, for image token counting.
GPT4O_MINI_DEPLOYMENTS—GPT-4o mini deployments, for image token counting.
VLLM_DEPLOYMENTS—Deployments on a vLLM OpenAI-compatible upstream, including vLLM embeddings and vLLM-Omni text-to-speech. Required for vLLM-Omni when the upstream has no key; otherwise the adapter tries Azure authentication.
QWEN3_ASR_VLLM_DEPLOYMENTS—Qwen3-ASR speech-to-text deployments served by vLLM.
AZURE_AI_VISION_DEPLOYMENTS—Azure AI Vision multimodal embedding deployments.
AUDIO_AZURE_API_VERSION2025-03-01-previewThe API version for the Azure Audio API.

General​

VariableDefaultDescription
DIAL_URL—The URL of DIAL Core. When set, the adapter can upload and download DIAL files.
LOG_LEVELINFOThe log level. Use DEBUG for development only.
TIKTOKEN_MODEL_MAPPING{}A JSON map from a deployment ID to a tiktoken model name, for token counting, for example {"my-gpt-deployment": "gpt-3.5-turbo"}. The gpt-4o tokenizer is the default.
NON_STREAMING_DEPLOYMENTS—Deployments that do not support streaming. The adapter calls them without streaming and returns the answer as a one-chunk stream.
API_VERSIONS_MAPPING{}A JSON map of Azure OpenAI API versions, for example {"2023-03-15-preview": "2023-05-15", "": "2024-02-15-preview"}. The empty key sets the default. See Azure API versions.
ELIMINATE_EMPTY_CHOICESfalseRemoves stream chunks with an empty choices list, for clients that cannot handle them. Azure OpenAI sends such chunks for content filter results and for usage.
SSE_HEARTBEAT_INTERVAL—When set, the adapter sends ping comments on a streaming response after this many idle seconds, so slow upstreams do not trigger read timeouts.
CLAUDE_DEFAULT_MAX_TOKENS1536The default max_tokens for Claude. Prefer a per-model default in DIAL Core instead.

Azure credentials​

VariableDefaultDescription
AZURE_OPEN_AI_SCOPEhttps://cognitiveservices.azure.com/.defaultThe scope of the Azure access token, when an upstream has no key.
ACCESS_TOKEN_EXPIRATION_WINDOW10How many seconds before expiry the Azure access token is renewed.

AWS credentials​

For OpenAI models on Amazon Bedrock, when an upstream sets neither key nor credentials in extra_data.

VariableDefaultDescription
AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY—Static AWS credentials with access to Bedrock.
AWS_SESSION_TOKEN—The session token of temporary credentials.
AWS_ASSUME_ROLE_ARN—A role to assume for Bedrock, for example arn:aws:iam::123456789012:role/RoleName. Ignored when static credentials are set.
AWS_CREDENTIALS_EXPIRATION_WINDOW300How many seconds before expiry the credentials of an assumed role are renewed.
AWS_SESSION_TAGS—A JSON map of AWS STS session tags to pass when assuming a role, for example {"application": "Bedrock.modelId", "project": "UserInfo.project"}. See Session tags.

Performance​

VariableDefaultDescription
WEB_CONCURRENCY1The number of Uvicorn worker processes. More workers use more CPU cores, which helps when the adapter counts tokens.
THREAD_POOL_SIZEmin(32, CPUs + 4)The size of the thread pool for CPU-heavy work, such as token counting and image analysis.
TIMEOUT_KEEP_ALIVE70How long an idle keep-alive connection stays open, in seconds. Keep it 10 seconds above DIAL Core's client.keepAliveTimeout, 60 by default, so Core always closes first.
HTTP_MAX_CONNECTIONS1000The most concurrent upstream connections per worker, shared by all upstreams.
HTTP_MAX_KEEPALIVE_CONNECTIONS100The most idle connections kept for reuse. Ignored with HTTP_TRANSPORT=aiohttp.
HTTP_POOL_TIMEOUT10How long a request waits for a free connection, in seconds. On timeout, the adapter answers 503.
HTTP_TRANSPORTaiohttpThe upstream connection pool: aiohttp or httpx. httpx costs more CPU with hundreds of concurrent streams.

Logging​

Logging comes from the DIAL SDK: the log format, structured JSON output, trace correlation, and OpenTelemetry log export are set by the DIAL_SDK_* variables in the DIAL SDK logging documentation. To log the underlying SDKs, set ANTHROPIC_LOG=debug or OPENAI_LOG=debug.

Private certificate authorities​

To trust a private CA for outbound TLS, set USE_SYSTEM_CA_CERTS to any non-empty value, and mount a directory of PEM *.crt files at /certificates:

docker run --rm \
  -e USE_SYSTEM_CA_CERTS=1 \
  -v "$PWD/certs:/certificates:ro" \
  epam/ai-dial-adapter-openai:0.45.2

At startup, the container combines the system CA bundle with those certificates.

Example​

adapter-dial-openai:
  image: epam/ai-dial-adapter-openai:0.45.2
  environment:
    DIAL_URL: http://core:8080
    LOG_LEVEL: INFO
    GPT4O_DEPLOYMENTS: gpt-4o
    DALLE3_DEPLOYMENTS: dall-e-3
    API_VERSIONS_MAPPING: '{"": "2025-06-01"}'
    WEB_CONCURRENCY: "2"

Further reading​