DIAL Analytics Enrichment Runner
DIAL Analytics Enrichment Runner executes pipelines that add computed values to analytics rows or groups of rows. This reference is for platform engineers operating DIAL Analytics. Pipeline creation and the interface are covered in DIAL Admin: Pipelines.
It requires a matching DIAL Analytics version: the two services check a shared contract, and earlier counterparts are incompatible. The upgrade guide describes the contract change.
How enrichment runs
Analytics owns pipeline definitions and all ClickHouse access. The runner synchronizes those definitions, schedules work, and persists cursors, jobs, run history, failures, and spend counters in its PostgreSQL database. Its configuration connects it to Analytics and, for LLM transforms, DIAL Core.
| Execution path | Behavior | Model calls |
|---|---|---|
| Row SQL enrichment | Asks Analytics to materialize computed values inside ClickHouse | No |
| Row LLM enrichment | Scans new rows, calls a model through Core, validates outputs, and writes enrichment rows through Analytics | Yes |
| Group LLM enrichment | Tracks changing groups, assembles their members, and computes group-level outputs through Core | Yes |
Execution is at least once; duplicate writes collapse in the enrichment table. Retries can still repeat model calls and incur cost. The runtime reference covers limits, retries, and spend tracking. Aggregate rollups run inside Analytics and do not use this scheduler.
Dependencies and deployment
| Dependency | Purpose |
|---|---|
| Matching DIAL Analytics | Provides pipeline definitions, governed source reads, SQL materialization, and write-back |
| PostgreSQL | Persists runtime state in a database separate from the Analytics catalog; both databases can share a server |
| DIAL Core and a model deployment | Required for LLM transforms; SQL enrichment does not call a model |
| Identity provider | Required for inbound oidc authentication; API-key authentication also connects to Core |
The image is epam/ai-dial-analytics-enrichment-runner. The Dockerfile runs as UID 1001 and exposes port 5000. A read-only root filesystem requires writable /tmp; persistent state stays in PostgreSQL.
The health configuration separates process liveness from readiness. Readiness requires PostgreSQL and the first successful Analytics pipeline sync. Startup probes need time for the initial sync delay and a polling interval.
Warning
Inbound authentication defaults to
none, so reachable callers can trigger work that spends money. UseSECURITY_MODE=oidc, grant operator access throughFULL_ADMIN, and keep the control plane internal. OutgoingADAS_API_KEYandDIAL_API_KEYvalues do not protect incoming requests.
Connection examples, paired deployment order, and troubleshooting are in Configuration. Spending, retries, and scheduling are described in Environment variables.
Pipeline controls
Analytics' enabled flag is the deliberate run control. Setting it to false stops execution while preserving queued work. POST /v1/pipelines/{name}/pause is removed and answers 404; POST /v1/pipelines/{name}/resume lifts only a circuit-breaker pause and answers 409 for any other pipeline. Backfills keep their own pause and resume endpoints. DIAL Admin 0.22.5 still calls the pipeline pause and resume endpoints, so its pause control fails against this runner. See the upgrade guide.