Skip to main content

Configure metrics

This guide exposes DIAL metrics to Prometheus and wires them into a Prometheus Operator deployment on Kubernetes. It also covers pushing the same metrics to an OpenTelemetry collector instead. It is for DevOps engineers running DIAL on Kubernetes with the DIAL Helm chart. For the conceptual model, see Observability.

DIAL components deliver metrics through two independent mechanisms: Prometheus scrapes a /metrics endpoint on each component, or the component pushes OTLP metrics to a collector. Configure either or both — they are not mutually exclusive, and most of this page covers the Prometheus path.

Prerequisites​

  • DIAL deployed with the dial-core Helm chart
  • Prometheus running in the cluster, ideally via the Prometheus Operator
  • Access to edit the chart's values.yaml

The metrics endpoint​

Each instrumented DIAL component serves Prometheus-format metrics on port 9464 at path /metrics. The port is exposed on the container but the metrics service is disabled until you enable it.

Enable the metrics service​

In the dial-core chart values, turn on metrics and the Prometheus Operator ServiceMonitor:

metrics:
  enabled: true
  serviceMonitor:
    enabled: true        # creates a ServiceMonitor for the Prometheus Operator
    interval: 30s
    path: /metrics
    port: http-metrics
containerPorts:
  metrics: 9464          # change only if 9464 conflicts in your cluster

With metrics.enabled: true, the chart renders a dedicated ClusterIP service on the metrics port. With serviceMonitor.enabled: true, it renders a ServiceMonitor so the Prometheus Operator discovers and scrapes the endpoint automatically.

If you do not run the Prometheus Operator, leave serviceMonitor.enabled: false and add a static scrape job in your Prometheus configuration that targets the metrics service on port 9464.

Enable the exporter on the component​

How you enable the Prometheus endpoint depends on how the component is instrumented:

  • DIAL Core and DIAL Chat expose Prometheus metrics on :9464/metrics without any metrics-exporter environment variable.
  • DIAL Admin Frontend exposes :9464/metrics by default; setting OTEL_METRICS_EXPORTER=otlp switches its metrics to OTLP HTTP. See Admin telemetry settings.
  • Adapters and applications built on the DIAL SDK only serve the endpoint once you set the metrics exporter to include prometheus:
OTEL_METRICS_EXPORTER=otlp,prometheus

This exports metrics to your OTLP collector and serves the Prometheus scrape endpoint at the same time. Use prometheus alone if you do not run a collector.

OTEL_EXPORTER_PROMETHEUS_PORT is optional. It defaults to 9464, matching the chart's containerPorts.metrics and ServiceMonitor port set above — set it only if you changed the port there.

Push metrics through a collector instead​

Components can also push OTEL metrics to a collector instead of, or alongside, being scraped by Prometheus:

OTEL_METRICS_EXPORTER=otlp
OTEL_EXPORTER_OTLP_PROTOCOL=grpc
OTEL_EXPORTER_OTLP_ENDPOINT=http://<YOUR_COLLECTOR_HOST>:4317

Use this when your backend consumes OTLP directly and you don't want to run Prometheus, or set OTEL_METRICS_EXPORTER=otlp,prometheus to run both paths at once.

The combined-exporter example applies to components that support it, such as the Python SDK. Admin Frontend selects a single metrics exporter and uses HTTP/JSON rather than gRPC.

What gets scraped​

warning

DIAL does not define custom Prometheus metric names. The scraped series are the standard metrics produced by the instrumentation layer: JVM and HTTP server metrics from the OpenTelemetry Java agent on DIAL Core, and system metrics (CPU, memory) plus FastAPI request metrics from the DIAL SDK on Python adapters and applications. Do not build dashboards or alerts around DIAL-specific metric names — they do not exist.

Visualize in Grafana​

note

The DIAL Helm chart ships no prebuilt Grafana dashboard for these Prometheus metrics. Use a community dashboard for the underlying instrumentation — for example a JVM or OpenTelemetry Java agent dashboard for DIAL Core — and adapt it to your labels. Ready-made dashboards for the separate usage-analytics plane are covered in Observability with Grafana and Prometheus.

Result​

Prometheus scrapes DIAL components on port 9464, and the series appear in your Prometheus instance and any Grafana datasource that reads from it.

Further reading​