Evaluation Framework Backend configuration

Evaluation Framework Backend takes its settings from environment variables. This reference lists them for release 0.5.0, grouped by subsystem, with the defaults of the release's application.yml. It is for platform engineers who run the backend; for how it connects to DIAL Admin, see Connect DIAL Admin.

Spring Boot accepts every property under its relaxed-binding name: upper case, with . and - replaced by _, so test-case.multi-turn.max-turns becomes TEST_CASE_MULTI_TURN_MAX_TURNS. Where application.yml reads a different name, this page lists that name. The upstream configuration guide has the property names; where its defaults differ from application.yml, this page follows application.yml.

Minimum settings

Variable	Why
POSTGRES_META_DATASOURCE_URL, _USERNAME, _PASSWORD	The meta database. The defaults are for local development.
POSTGRES_ANALYTICS_DATASOURCE_URL, _USERNAME, _PASSWORD	The analytics database.
DIAL_CORE_URL	DIAL Core.
DIAL_EF_API_KEY	The backend's Core API key. The backend doesn't start without it.
METRIC_PROVIDERS_DIAL_BASE_URL and METRIC_PROVIDERS_SYNC_ENABLED=true	The metrics service, and loading its metric catalog.
PROVIDERS_<ID>_ISSUER, _JWKSETURI, _AUDIENCES, _PRINCIPALCLAIM	Validation of the tokens DIAL Admin forwards.

Server and endpoints

Variable	Default	Description
SERVER_PORT	8080	Port of the REST API, the health checks, and Swagger UI.
MANAGEMENT_SERVER_PORT	9464	Port of Spring Boot Actuator.
MANAGEMENT_ENDPOINTS_WEB_EXPOSURE_INCLUDE	health,info,prometheus,metrics	Actuator endpoints served, such as /actuator/prometheus.
MANAGEMENT_ENDPOINT_HEALTH_SHOW_DETAILS	when_authorized	Detail in /actuator/health: never, when_authorized, or always.
SPRINGDOC_SWAGGER_UI_ENABLED	true	Serve Swagger UI at /swagger-ui.html.
SPRINGDOC_API_DOCS_ENABLED	true	Serve the OpenAPI document at /v3/api-docs.
SERVER_TOMCAT_MAX_HTTP_POST_SIZE	10485760	Largest request body, in bytes.
VIRTUAL_THREADS_ENABLED	true	Run request and run workers on virtual threads. Set false only to profile CPU use.

The health checks at /api/v1/health, /api/v1/health/ready, and /api/v1/health/live need no token. The Actuator readiness group includes the dialFileStorage, database, and analyticsDatabase checks; dialFileStorage stays down until the backend finds its file bucket in DIAL Core with DIAL_EF_API_KEY.

Security

Variable	Default	Description
CONFIG_REST_SECURITY_MODE	oidc	oidc validates a JWT on every API call. none turns authentication off; use it only for local development.
CONFIG_REST_SECURITY_DEFAULT_ALLOWEDROLES	admin	Roles accepted when a provider sets no ALLOWEDROLES.
CONFIG_REST_SECURITY_DISABLE_SWAGGER_AUTHORIZATION	true	Serve Swagger UI and the OpenAPI document without a token.
SECURITY_JWT_USER_CLAIM	sub	Claim stored as the author of test suites and other entities.
SECURITY_JWT_RESOLVE_USER_NAME	false	Store the user's display name from DIAL Core's GET /v1/user/info instead of the claim. DIAL Admin shows the stored value in Created by.

Warning

The backend has no per-user authorization yet. Every user it accepts can read and change all datasets, test suites, and runs.

Identity providers

Each identity provider is a set of PROVIDERS_<ID>_* variables, where <ID> is a name you choose, such as KEYCLOAK. Configure the provider that signs administrators in to DIAL Admin, because Admin forwards that provider's tokens. Lists are comma-separated.

Variable	Required	Description
PROVIDERS_<ID>_ISSUER	Yes	Expected iss claim: the issuer URL as it appears in the tokens, which is the URL the browser uses.
PROVIDERS_<ID>_JWKSETURI	Yes	URL of the provider's signing keys. It must be reachable from the backend.
PROVIDERS_<ID>_AUDIENCES	Yes	Accepted aud claim values, such as DIAL Admin's client ID.
PROVIDERS_<ID>_PRINCIPALCLAIM	Yes	Claim used as the user's name.
PROVIDERS_<ID>_ROLECLAIMS	No	Claims that hold the user's roles. The roles of all listed claims are combined.
PROVIDERS_<ID>_ALLOWEDROLES	No	Roles that may call the API. Defaults to CONFIG_REST_SECURITY_DEFAULT_ALLOWEDROLES. A token with none of them is rejected, and the backend logs Access denied for issuer.
PROVIDERS_<ID>_ALIASES	No	Other accepted iss values, such as Microsoft Entra ID tenant aliases.

API-key authentication

With API-key authentication, callers can send a DIAL Core API key in the Api-Key header instead of a JWT. The backend checks the key with DIAL Core's GET /v1/user/info. eval-cli and Application mode need it. It works only in oidc mode.

Variable	Default	Description
API_KEY_ENABLED	false	Accept the Api-Key header.
API_KEY_CORE_URL	—	DIAL Core URL used to check keys. Required when enabled.
API_KEY_KEY_ROLES_MAPPING	—	JSON object that maps the roles of Core project keys to lists of backend authorities.
API_KEY_JWT_ROLES_MAPPING	—	JSON object that maps the roles in a per-request key's user claims to lists of backend authorities. At least one of the two mappings must be set.
API_KEY_USER_CLAIMS_ROLE_CLAIM	roles	Claim read for roles from a per-request key's user claims, when its issuer matches no identity provider. Otherwise that provider's ROLECLAIMS are used.
API_KEY_CACHE_TTL_SECONDS	60	How long a checked key is cached.
API_KEY_CACHE_MAX_SIZE	10000	Maximum cached keys.
API_KEY_REQUEST_TIMEOUT_MS	3000	Timeout of the call to Core.
API_KEY_STARTUP_PROBE	true	Call Core at startup and fail if it can't be reached.

The two mapping variables were renamed in 0.5.0, without fallback; see Framework Backend 0.5.0.

Application mode

By default, a run calls the target with the token of the user who started it, so a run that outlasts the token fails. In Application mode, DIAL Core starts the run through an application route and gives it a per-request key instead. The backend's files are then shared with the evaluated deployments through Core.

Variable	Default	Description
DIAL_APP_PROXY_ENABLED	false	Run test suites through a DIAL Core application route. Requires API_KEY_ENABLED=true; the backend doesn't start otherwise.
DIAL_APP_PROXY_DEPLOYMENT_NAME	EF	Name of the application in Core's configuration.
DIAL_APP_PROXY_HEARTBEAT_INTERVAL_MS	30000	Heartbeat that keeps the connection from Core open.
DIAL_APP_PROXY_TRIGGER_READ_TIMEOUT_MS	43200000	Timeout of the call that starts a run through Core: 12 hours.

Also raise TEST_SUITE_RUN_SSE_TIMEOUT_MINUTES, such as to 720, so that the status stream outlasts long runs. Register the application in DIAL Core with a route to the backend's run endpoint; this is the form in the upstream configuration guide:

{
  "applications": {
    "EF": {
      "displayName": "Evaluation Framework",
      "routes": {
        "run_execute": {
          "paths": ["/api/internal/runs/[^/]+/execute$"],
          "methods": ["POST"],
          "rewritePath": true,
          "upstreams": [{"endpoint": "http://<EVALUATION_BACKEND_HOST>:8080"}],
          "attachmentPaths": {
            "requestBody": []
          }
        }
      }
    }
  }
}

• The application name must match DIAL_APP_PROXY_DEPLOYMENT_NAME. The 0.5.0 release notes give a fuller version of this entry.
• The application needs access to the backend's file bucket (DIAL_FILE_STORAGE_BUCKET_ALIAS), so that Core can share test case files with the evaluated deployments.
• Keep /api/internal/ off the public ingress. The endpoint is authenticated, but only Core needs it.

See Application routes for the route fields.

Databases

The backend uses two PostgreSQL datasources: meta for datasets, test suites, runs, and metric declarations, and analytics for results and scores. They can be two databases on one server, as in the advanced sample, or two schemas of one database; they can't share a database and schema. The backend applies its Flyway migrations to both at startup, so its database user needs DDL privileges.

Variable	Default	Description
POSTGRES_META_DATASOURCE_URL	jdbc:postgresql://localhost:5432/evaluation_db	JDBC URL of the meta database, without query parameters.
POSTGRES_META_DATASOURCE_USERNAME	postgres	Database user.
POSTGRES_META_DATASOURCE_PASSWORD	postgres	Database password. Override it outside local development.
POSTGRES_META_DATASOURCE_SCHEMA	public	Schema of the meta tables.
POSTGRES_META_DATASOURCE_CONNECTION_PARAMS	—	JDBC query parameters, such as sslmode=require.
POSTGRES_ANALYTICS_DATASOURCE_URL	jdbc:postgresql://localhost:5432/evaluation_analytics_db	JDBC URL of the analytics database.
POSTGRES_ANALYTICS_DATASOURCE_USERNAME	postgres	Database user.
POSTGRES_ANALYTICS_DATASOURCE_PASSWORD	postgres	Database password. Override it outside local development.
POSTGRES_ANALYTICS_DATASOURCE_SCHEMA	public	Schema of the analytics tables.
POSTGRES_ANALYTICS_DATASOURCE_CONNECTION_PARAMS	reWriteBatchedInserts=true	JDBC query parameters.
DATASOURCE_META_AUTH_TYPE	basic	basic uses the user and password. azure uses Microsoft Entra ID tokens for Azure Database for PostgreSQL.
DATASOURCE_ANALYTICS_AUTH_TYPE	basic	The same for the analytics database.
AUTH_AZURE_TYPE	none	Set managed with azure authentication, to use the pod's managed identity. The database user must then be that identity.

DIAL Core

Variable	Default	Description
DIAL_CORE_URL	http://localhost:8085	DIAL Core base URL.
DIAL_EF_API_KEY	—	Required. The backend's Core API key, used for test case files in Core's storage. Calls for deployments, Try Out, runs, and MCP tools use the caller's token.
DIAL_CORE_CONNECT_TIMEOUT_MS	5000	Connection timeout.
DIAL_CORE_READ_TIMEOUT_MS	30000	Read timeout of metadata calls, such as the deployment list.
DIAL_CORE_RETRY_MAX_ATTEMPTS	3	Attempts for failed metadata calls.
DIAL_CORE_RETRY_DELAY_MS	1000	First retry delay.
DIAL_CORE_RETRY_MULTIPLIER	2.0	Backoff multiplier.
DIAL_CORE_TRY_OUT_READ_TIMEOUT_MS	3600000	Read timeout of Try Out calls.
DIAL_MCP_CONNECT_TIMEOUT_MS	5000	Connection timeout of MCP tool calls through Core.
DIAL_MCP_READ_TIMEOUT_MS	120000	Read timeout of MCP tool calls.

Files

Test cases with File fields keep their files in DIAL Core's storage.

Variable	Default	Description
DIAL_FILE_STORAGE_BUCKET_ALIAS	@ef	Alias used in file references, such as @ef/datasets/<DATASET_ID>/<FILE>. The real bucket is found at runtime.
DIAL_FILE_STORAGE_MAX_FILE_SIZE_BYTES	52428800	Largest file, 50 MB. Uploads are also limited by SPRING_SERVLET_MULTIPART_MAX_FILE_SIZE (50MB) and SPRING_SERVLET_MULTIPART_MAX_REQUEST_SIZE (55MB).
DIAL_FILE_STORAGE_MAX_FILES_PER_SUITE	100	Files per test suite.
DIAL_FILE_STORAGE_MAX_FILES_PER_DATASET	100	Files per dataset.
DIAL_FILE_STORAGE_CONNECT_TIMEOUT_MS	5000	Connection timeout.
DIAL_FILE_STORAGE_READ_TIMEOUT_MS	30000	Read timeout.

Run costs

The run cost figures come from the Analytics data-access service, which the backend queries with the caller's token. In 0.5.0 the backend calls its POST /v1/queries/execute-sql endpoint.

Variable	Default	Description
DIAL_ADAS_URL	http://localhost:8087	Base URL of the Analytics data-access service.
DIAL_ADAS_CONNECT_TIMEOUT_MS	5000	Connection timeout.
DIAL_ADAS_READ_TIMEOUT_MS	30000	Read timeout.
QUERY_DSL_EXTENSION_TEST_SUITE_RUN_COST_ENABLED	false	Add each run's total cost to run query results.
QUERY_DSL_EXTENSION_TEST_SUITE_RUN_COST_TIMEOUT_SEC	2	Deadline of that cost lookup.

Metric providers

A metric provider is a service with GET /metrics and POST /evaluate, such as Evaluation Metrics. A sync job reads each enabled provider's catalog; during a run, the backend sends each metric to the provider it came from.

Variable	Default	Description
METRIC_PROVIDERS_SYNC_ENABLED	false	Run the sync job. Set true; otherwise no provider's metrics are loaded.
METRIC_PROVIDERS_SYNC_CRON	-	Spring cron expression with seconds, such as 0 */5 * * * * for every five minutes. With -, the job runs once at startup.
METRIC_PROVIDERS_SYNC_FIXED_DELAY_MS	0	Delay between syncs, used only when the cron is -.
METRIC_PROVIDERS_DIAL_ENABLED	true	Sync the built-in dial provider.
METRIC_PROVIDERS_DIAL_BASE_URL	http://localhost:8086	Base URL of the dial provider: the Evaluation Metrics service, such as http://evaluation-metrics:5000.
METRIC_PROVIDERS_EXTRA_ENABLED	false	Sync the built-in second provider, extra.
METRIC_PROVIDERS_EXTRA_BASE_URL	http://localhost:8087	Base URL of the extra provider. Set it before enabling the provider.
METRIC_PROVIDERS_<ID>_CONNECT_TIMEOUT_MS	5000	Connection timeout of a provider.
METRIC_PROVIDERS_<ID>_READ_TIMEOUT_MS	150000	Read timeout of a provider. A provider defined only through environment variables defaults to 30000.

To add a third provider, set both METRIC_PROVIDERS_<ID>_BASE_URL and METRIC_PROVIDERS_<ID>_ENABLED; a provider without ENABLED stops startup. A disabled provider is skipped by the sync job only: its metrics already loaded still run.

Metric evaluation

Variable	Default	Description
METRIC_EVALUATION_DEFAULT_CONCURRENCY_PER_PROVIDER	5	Evaluations sent to one provider at a time.
METRIC_EVAL_PER_RESULT_TIMEOUT_MS	150000	Time allowed for all metrics of one result. Metrics still running are marked failed.
METRIC_EVALUATION_BATCH_SIZE	100	Scores buffered before they are written.
METRIC_EVALUATION_RETRY_MAX_RETRIES	0	Retries of a failed evaluation call.
METRIC_EVALUATION_RETRY_RETRY_DELAY_MS	1000	First retry delay.
METRIC_EVALUATION_RETRY_RETRY_BACKOFF_MULTIPLIER	2.0	Backoff multiplier, at least 1.0.
METRIC_EVALUATION_RETRY_MAX_RETRY_DELAY_MS	60000	Longest retry delay.

Test suite runs

Variable	Default	Description
TEST_SUITE_RUN_LIMITS_MAX_CONCURRENT_RUNS_GLOBAL	20	Runs executing at once across all test suites.
TEST_SUITE_RUN_LIMITS_MAX_CONCURRENT_RUNS_PER_SUITE	5	Runs executing at once for one test suite.
TEST_SUITE_RUN_RUN_CONFIG_MAX_NUMBER_OF_RUNS	64	Largest Number of runs per test case.
TEST_SUITE_RUN_EXECUTION_DEFAULT_CONCURRENCY_LEVEL	1	Test cases executed in parallel in a run, by default.
TEST_SUITE_RUN_EXECUTION_MAX_CONCURRENCY_LEVEL	50	Upper bound of that parallelism.
TEST_SUITE_RUN_EXEC_DEFAULT_TIMEOUT	600000	Default timeout of a request to the target, in milliseconds.
TEST_SUITE_RUN_EXEC_MAX_TIMEOUT	1200000	Upper bound of that timeout.
TEST_SUITE_RUN_EXECUTION_MAX_RESPONSE_SIZE_BYTES	5242880	Largest stored response, 5 MB; longer responses are cut.
TEST_SUITE_RUN_EXECUTION_RESULT_BATCH_SIZE	100	Results buffered before they are written.
TEST_SUITE_RUN_EXECUTION_HEADER_BLACKLIST	Authorization, Host, Content-Length, Transfer-Encoding, Connection, traceparent, tracestate	Headers removed from test requests before they are sent.
TEST_SUITE_RUN_RETRY_DEFAULT_MAX_RETRIES	0	Default retries of a failed request; at most TEST_SUITE_RUN_RETRY_MAX_MAX_RETRIES (10).
TEST_SUITE_RUN_RETRY_DEFAULT_RETRY_DELAY_MS	1000	Default first retry delay; at most TEST_SUITE_RUN_RETRY_MAX_RETRY_DELAY_MS (60000).
TEST_SUITE_RUN_RETRY_DEFAULT_RETRY_BACKOFF_MULTIPLIER	2.0	Default backoff multiplier; at most TEST_SUITE_RUN_RETRY_MAX_RETRY_BACKOFF_MULTIPLIER (10.0).
TEST_SUITE_RUN_SSE_TIMEOUT_MINUTES	30	Timeout of the run status stream that DIAL Admin listens to.
TEST_SUITE_RUN_SSE_CLEANUP_INTERVAL_MS	300000	Heartbeat interval of status streams.
TEST_SUITE_RUN_RUN_INPUTS_RETENTION_DAYS	1	Days a finished run's inputs are kept.
SSE_EVENT_PROCESSING_MAX_TOTAL_DURATION_MS	3600000	Longest time spent reading one streamed response.
JSONATA_EVALUATION_TIMEOUT_MS	10000	Longest evaluation of one JSONata expression in a request template, response column, or condition.
JSONATA_MAX_RECURSION_DEPTH	1000	Deepest recursion of one JSONata expression.

Datasets and imports

Variable	Default	Description
TEST_CASE_MULTI_TURN_MAX_TURNS	10	Most turns in a multi-turn test case. Longer test cases are saved but marked invalid, and runs skip them.
CSV_IMPORT_MAX_FILE_SIZE	10MB	Largest CSV import.
CSV_IMPORT_MAX_ROWS	100000	Most rows in one import.
CSV_IMPORT_BATCH_SIZE	1000	Rows written per batch.
CSV_IMPORT_ZIP_MAX_ENTRIES	1000	Most entries in a ZIP import.
CSV_IMPORT_ZIP_MAX_TOTAL_UNCOMPRESSED_SIZE	1GB	Largest expanded size of a ZIP import.
CSV_IMPORT_ZIP_MAX_MANIFEST_SIZE	1MB	Largest manifest.json in a ZIP import.
CSV_EXPORT_PAGE_SIZE	500	Rows read per page while exporting CSV.
ANALYTICS_RESULTS_CSV_IMPORT_MAX_FILE_SIZE	10MB	Largest CSV of run results to import, as eval-cli does.
VALIDATION_MAX_TEMPLATE_SIZE_BYTES	65536	Largest request template.
VALIDATION_MAX_BINDINGS_COUNT	64	Most input bindings.
VALIDATION_MAX_WARNINGS_PER_CASE	5	Validation warnings stored per test case.
VALIDATION_REVALIDATION_BATCH_SIZE	500	Test cases revalidated per batch after a schema change.
VALIDATION_REVALIDATION_TIMEOUT_MINUTES	5	Longest revalidation task.

API limits

Variable	Default	Description
PAGINATION_DEFAULT_SIZE	100	Page size when a list request sets none.
PAGINATION_MAX_SIZE	1000	Largest page size.
TEST_CASE_BATCH_MAX_ITEMS	256	Items in one batch update of test cases.
TEST_CASE_BULK_MAX_OPERATIONS	512	Operations in one bulk update.
TEST_CASE_BULK_MAX_ITEM_OPERATIONS	500	Per-row operations in one bulk update.
TEST_CASE_BULK_MAX_IDS_PER_SELECTOR	10000	IDs one bulk operation selects.
TEST_CASE_BULK_MAX_DELETE_IDS	10000	IDs in one bulk delete.
ANALYTICS_RESULTS_BATCH_MAX_ITEMS	10000	Results in one batch write; the body is limited by ANALYTICS_RESULTS_BATCH_MAX_REQUEST_SIZE_BYTES (10485760).
ANALYTICS_EVAL_SUMMARIES_BATCH_MAX_ITEMS	10000	Scores in one batch write; the body is limited by ANALYTICS_EVAL_SUMMARIES_BATCH_MAX_REQUEST_SIZE_BYTES (10485760).
ANALYTICS_COMPARISON_MAX_UNMATCHED_ROWS	5000	Rows of one run that a run comparison may leave unmatched. Above it, the comparison fails with 409.

Inbound MCP server

The backend can serve an MCP endpoint, so that an agent can set up evaluations. In 0.5.0 it has the list_deployments and get_deployment tools. It is authenticated like the REST API.

Variable	Default	Description
SPRING_AI_MCP_SERVER_ENABLED	false	Serve the MCP endpoint. When off, the endpoint answers 404.
SPRING_AI_MCP_SERVER_PROTOCOL	STATELESS	STATELESS answers each request with one JSON response. STREAMABLE keeps sessions in memory, so it needs one replica or sticky sessions.
SPRING_AI_MCP_SERVER_STREAMABLE_HTTP_MCP_ENDPOINT	/mcp	Path of the endpoint.

The other SPRING_AI_MCP_SERVER_* variables are listed in the upstream configuration guide.

Telemetry and logs

Variable	Default	Description
OTEL_SDK_DISABLED	true	OpenTelemetry is off. Set false to export traces, metrics, and logs, with the service name ai-dial-evaluation-framework.
OTEL_EXPORTER_OTLP_ENDPOINT	http://localhost:4318	OTLP collector endpoint.
OTEL_EXPORTER_OTLP_PROTOCOL	http/protobuf	OTLP protocol; grpc also works.
OTEL_TRACES_EXPORTER, OTEL_METRICS_EXPORTER, OTEL_LOGS_EXPORTER	otlp	Exporter of each signal.
GRAFANA_BASE_URL	—	Grafana URL for trace links. When unset, runs and Try Out have no trace links, and DIAL Admin hides Traces in Grafana.
GRAFANA_TEMPO_DATASOURCE_UID	tempo	UID of the Tempo data source in Grafana.
GRAFANA_ORG_ID	1	Grafana organization in the links.
LOGGING_LEVEL_ROOT	INFO	Root log level.
LOGGING_LEVEL_COM_EPAM_AIDIAL	INFO	Log level of the backend's own code.
LOGGING_REQUEST_RESPONSE_ENABLED	false	Log every HTTP request and response with its body.
LOGGER_CONFIGURATION_PATH	/app/log-config/logging.levels.json	File of log levels that the backend re-reads at runtime.
LOGGER_CONFIGURATION_INTERVAL	30	Seconds between reads of that file.

Prometheus metrics are at /actuator/prometheus on port 9464. For the collector and scrape setup of the platform, see Tracing and Metrics.

Container

Variable	Description
JAVA_OPTS	JVM options passed to java.
DEBUG_OPTS	Further JVM options, such as a debug agent.
USE_SYSTEM_CA_CERTS	Any value imports the certificates mounted in /certificates/ at startup.

Further reading

• Metrics configuration — the judge models and limits of the metrics service
• DIAL Admin: Test suites — bundle a target, metrics, and a dataset into a repeatable test
• DIAL Admin: Runs — run statuses and per-test-case metric scores
• DIAL Admin: Datasets — reusable test-case collections with a defined schema
• Configure tracing with OpenTelemetry — follow one request across Core, adapters, and applications