DIAL RAG environment variables

This reference lists every environment variable of DIAL RAG, with its default and behavior. It is for platform engineers who deploy DIAL RAG. For how the variables combine with the YAML profile and with each request, see Configuration.

Version scope

Defaults below are checked against DIAL RAG 0.43.0 and the DIAL SDK 0.39.0 it locks. Where the image sets a variable, the table gives the image's value. The request settings can also be set in a profile or per request, at the same path without the DIAL_RAG__REQUEST__ prefix.

Service

Variable	Default	Description
DIAL_URL	http://dial-proxy.dial-proxy	The DIAL Core URL. DIAL RAG sends its per-request key only to attachment URLs that start with it.
DIAL_RAG__CONFIG_PATH	/config/azure_description.yaml in the image	The YAML profile. See Profiles. Without the image, no profile is loaded.
ENABLE_DEBUG_COMMANDS	False	Turns on the debug commands /model, /query_model, and /profile. The upstream README says to keep it false in production.
DIAL_RAG__HEADERS_TO_PROXY	[]	A JSON array of header names. DIAL RAG copies these headers of the incoming request into its model calls through DIAL Core.
DIAL_SDK_HEADERS_TO_PROXY	Unset	Comma-separated header names. The DIAL SDK copies these headers of the incoming request into every HTTP call that DIAL RAG makes to DIAL_URL.

Index storage

Variable	Default	Description
DIAL_RAG__INDEX_STORAGE__USE_DIAL_FILE_STORAGE	False	True stores indexes in DIAL file storage. False keeps them only in memory. See Index storage.
DIAL_RAG__INDEX_STORAGE__IN_MEMORY_CACHE_CAPACITY	128MiB	The size of the in-memory index cache of each process, in bytes or as a size such as 1GiB. A larger cache needs more pod memory.

Compute

Variable	Default	Description
DIAL_RAG__CPU_POOLS__INDEXING_CPU_POOL	The CPU count minus 2, at least 1	Threads for parsing, document conversion, and page image extraction. Text embeddings always run in one thread of their own.
BGE_EMBEDDINGS_MODEL_PATH	/embeddings_model/bge-small-en in the image	The epam/bge-small-en model for text embeddings. Outside the image, the default is the model's Hugging Face name.
BGE_EMBEDDINGS_DEVICE	auto	cpu runs the model with OpenVINO, and cuda with PyTorch in half precision. auto uses CUDA when it's available.

Tip

The default CPU pool size comes from Python's CPU count, which is the node's CPU count, not the container's CPU limit. Set DIAL_RAG__CPU_POOLS__INDEXING_CPU_POOL to match the pod's CPU limit.

Request settings

Each variable here is the service-wide default of a request setting. Its per-request path is the variable name without DIAL_RAG__REQUEST__, in lowercase, with . for __.

General

Variable	Default	Description
DIAL_RAG__REQUEST__IGNORE_DOCUMENT_LOADING_ERRORS	False	Answers from the documents that loaded when others fail. Otherwise any failure ends the request with the errors.
DIAL_RAG__REQUEST__USE_PROFILER	False	Adds a CPU profile of the request to its Profiler stage
DIAL_RAG__REQUEST__LOG_DOCUMENT_LINKS	False	Allows document links in log messages above DEBUG. Off, only DEBUG messages may contain them.
DIAL_RAG__REQUEST__DOWNLOAD__TIMEOUT_SECONDS	30	The total timeout of a document download
DIAL_RAG__REQUEST__DOWNLOAD__CONNECT_TIMEOUT_SECONDS	30	The connection timeout of a document download
DIAL_RAG__REQUEST__CHECK_ACCESS__TIMEOUT_SECONDS	30	The total timeout of the access check, which reads the document's metadata
DIAL_RAG__REQUEST__CHECK_ACCESS__CONNECT_TIMEOUT_SECONDS	30	The connection timeout of the access check

Indexing

Variable	Default	Description
DIAL_RAG__REQUEST__INDEXING__PARSER__MAX_DOCUMENT_TEXT_SIZE	5MiB	The most text a document may yield. The limit applies to the extracted text, not the file.
DIAL_RAG__REQUEST__INDEXING__PARSER__UNSTRUCTURED_CHUNK_SIZE	1000	The largest chunk, in characters. A change rebuilds the indexes.
DIAL_RAG__REQUEST__INDEXING__DESCRIPTION_INDEX	Set, with the fields below	The Page image search index. null turns it off.
DIAL_RAG__REQUEST__INDEXING__DESCRIPTION_INDEX__LLM__DEPLOYMENT_NAME	gpt-4.1-mini-2025-04-14	The vision model that describes each page image. It runs once per page, so a cheap, fast model fits.
DIAL_RAG__REQUEST__INDEXING__DESCRIPTION_INDEX__LLM__MAX_PROMPT_TOKENS	0	Not sent when 0. Descriptions use no history.
DIAL_RAG__REQUEST__INDEXING__DESCRIPTION_INDEX__LLM__MAX_RETRIES	1000000000	Retries of a failed model call. The time limit below ends the work instead.
DIAL_RAG__REQUEST__INDEXING__DESCRIPTION_INDEX__LLM__TEMPERATURE	0.0	The model's temperature
DIAL_RAG__REQUEST__INDEXING__DESCRIPTION_INDEX__ESTIMATED_TASK_TOKENS	4000	The tokens one page is expected to use. DIAL RAG reserves this many tokens per page from the caller's daily limit and paces calls to the minute limit.
DIAL_RAG__REQUEST__INDEXING__DESCRIPTION_INDEX__TIME_LIMIT_MULTIPLIER	1.5	The time limit of a document's descriptions is the estimated time times this value.
DIAL_RAG__REQUEST__INDEXING__DESCRIPTION_INDEX__MIN_TIME_LIMIT_SEC	300	The shortest time limit, in seconds
DIAL_RAG__REQUEST__INDEXING__MULTIMODAL_INDEX	null	The Multimodal search index. Setting it, or any of its fields, turns it on.
DIAL_RAG__REQUEST__INDEXING__MULTIMODAL_INDEX__EMBEDDINGS_MODEL	multimodalembedding@001	The multimodal embeddings model, such as azure-ai-vision-embeddings or amazon.titan-embed-image-v1. A change rebuilds the indexes.
DIAL_RAG__REQUEST__INDEXING__MULTIMODAL_INDEX__METRIC	sqeuclidean_dist	sqeuclidean_dist or cosine_sim. Upstream recommends sqeuclidean_dist for multimodalembedding@001 and cosine_sim for azure-ai-vision-embeddings.
DIAL_RAG__REQUEST__INDEXING__MULTIMODAL_INDEX__IMAGE_SIZE	1536	The size of the page images sent to the model
DIAL_RAG__REQUEST__INDEXING__MULTIMODAL_INDEX__ESTIMATED_TASK_TOKENS	500	The tokens one page image is expected to use. Upstream gives 500 for multimodalembedding@001, 1 for azure-ai-vision-embeddings, and 75 for amazon.titan-embed-image-v1.
DIAL_RAG__REQUEST__INDEXING__MULTIMODAL_INDEX__TIME_LIMIT_MULTIPLIER	1.5	As for the description index
DIAL_RAG__REQUEST__INDEXING__MULTIMODAL_INDEX__MIN_TIME_LIMIT_SEC	300	As for the description index

Answer

The chat chain writes the answer from the retrieved chunks.

Variable	Default	Description
DIAL_RAG__REQUEST__QA_CHAIN__CHAT_CHAIN__LLM__DEPLOYMENT_NAME	gpt-4.1-2025-04-14	The chat model. It must accept images while NUM_PAGE_IMAGES_TO_USE is above 0.
DIAL_RAG__REQUEST__QA_CHAIN__CHAT_CHAIN__LLM__MAX_PROMPT_TOKENS	16000	Sent as max_prompt_tokens, so that the history is truncated to fit. The model must support it. 0 doesn't send it.
DIAL_RAG__REQUEST__QA_CHAIN__CHAT_CHAIN__LLM__MAX_RETRIES	2	Retries of a failed model call
DIAL_RAG__REQUEST__QA_CHAIN__CHAT_CHAIN__LLM__TEMPERATURE	0.0	The model's temperature
DIAL_RAG__REQUEST__QA_CHAIN__CHAT_CHAIN__SYSTEM_PROMPT_TEMPLATE_OVERRIDE	Unset	Replaces the built-in system prompt. The built-in one asks the model to cite chunks as <[n]>, which DIAL RAG turns into references.
DIAL_RAG__REQUEST__QA_CHAIN__CHAT_CHAIN__USE_HISTORY	True	Sends the conversation to the model. False sends only the search query.
DIAL_RAG__REQUEST__QA_CHAIN__CHAT_CHAIN__NUM_PAGE_IMAGES_TO_USE	4	The most page images sent with the chunks. 0, with both image indexes off, makes DIAL RAG text-only.
DIAL_RAG__REQUEST__QA_CHAIN__CHAT_CHAIN__PAGE_IMAGE_SIZE	1536	The size of those page images

Search query

The query chain rewrites a follow-up question into a standalone search query.

Variable	Default	Description
DIAL_RAG__REQUEST__QA_CHAIN__QUERY_CHAIN__LLM__DEPLOYMENT_NAME	gpt-4.1-2025-04-14	The query model. It must support tool calling while USE_HISTORY is on.
DIAL_RAG__REQUEST__QA_CHAIN__QUERY_CHAIN__LLM__MAX_PROMPT_TOKENS	8000	As for the chat model
DIAL_RAG__REQUEST__QA_CHAIN__QUERY_CHAIN__LLM__MAX_RETRIES	2	Retries of a failed model call
DIAL_RAG__REQUEST__QA_CHAIN__QUERY_CHAIN__LLM__TEMPERATURE	0.0	The model's temperature
DIAL_RAG__REQUEST__QA_CHAIN__QUERY_CHAIN__USE_HISTORY	True	Rewrites the question when the conversation has more than one user message. False uses the latest message as is.

When the rewrite fails, DIAL RAG logs a warning and searches with the latest message.

Logging

Variable	Default	Description
LOG_LEVEL	INFO	The level of DIAL RAG's logs: CRITICAL, ERROR, WARNING, INFO, or DEBUG
LOG_LEVEL_OVERRIDE	{}	A JSON object of logger names and levels, such as {"aidial_rag": "DEBUG", "urllib3": "ERROR"}
DIAL_SDK_LOG	WARNING	The level of the DIAL SDK's own logger
DIAL_SDK_LOG_FORMAT	text	text or json
DIAL_SDK_TEXT_LOG_FORMAT	%(levelprefix)s | %(asctime)s | %(name)s | %(process)d | %(message)s	The format of text logs
DIAL_SDK_JSON_LOG_FORMAT	level, time, logger, process, and message keys	A JSON object that maps keys to log record fields

Observability

The DIAL SDK sets up OpenTelemetry at startup. Its OTLP exporters use gRPC and read the standard OTEL_EXPORTER_OTLP_* variables.

Variable	Default	Description
OTEL_TRACES_EXPORTER	Unset	otlp exports traces.
OTEL_METRICS_EXPORTER	Unset	Comma-separated. otlp pushes metrics, and prometheus serves them on OTEL_EXPORTER_PROMETHEUS_PORT.
OTEL_LOGS_EXPORTER	Unset	Comma-separated. otlp exports log records, and console writes them to stderr as JSON lines.
OTEL_EXPORTER_PROMETHEUS_PORT	9464	The Prometheus port
OTEL_PYTHON_LOG_CORRELATION	false	true adds trace and span IDs to log lines while traces are on.

Set by the image

Variable	Value	Purpose
DO_NOT_TRACK	true	Turns off usage tracking in the unstructured library
UNSTRUCTURED_XY_CUT_BBOX_SHRINK_FACTOR	1.0	Keeps unstructured from cutting off parts of tables in PDF documents

Further reading

• DIAL RAG API — set request settings per request
• DIAL RAG — how DIAL RAG indexes attached documents and retrieves from them