Skip to main content

Deploy DIAL on AWS

Deploy the full DIAL stack to Amazon EKS using the umbrella Helm chart, with Amazon S3 for storage and Amazon Bedrock as the model provider. This guide assumes you operate an EKS cluster and are familiar with Helm and kubectl. For the chart model and infrastructure mapping, read the Helm chart overview first.

Prerequisites​

  • An EKS cluster, version 1.24 or higher, with kubectl and Helm 3.8.0 or higher configured against it

  • The AWS Load Balancer Controller installed in the cluster

  • An S3 bucket for DIAL file storage

  • Amazon Bedrock enabled in your target region, with access granted to at least one model

  • IAM roles for service accounts (IRSA) for DIAL Core's S3 and Bedrock access

  • An Amazon Cognito user pool with an app client and a hosted UI domain, for DIAL Chat sign-in

  • An ACM certificate for your domain (for HTTPS ingress)

  • The DIAL Helm repository added:

    helm repo add dial https://charts.dialx.ai
    helm repo update

Step 1: Create a values file​

Start from the upstream AWS simple example and fill in your environment. The required values are:

core:
  # IRSA role granting access to the S3 bucket
  serviceAccount:
    annotations:
      eks.amazonaws.com/role-arn: <AWS_CORE_ROLE_ARN>
  env:
    aidial.storage.bucket: <AWS_CORE_S3_BUCKET_NAME>
    aidial.identityProviders.cognito.jwksUrl: "https://cognito-idp.<AWS_COGNITO_REGION>.amazonaws.com/<AWS_COGNITO_ID>/.well-known/jwks.json"
    aidial.identityProviders.cognito.rolePath: "cognito:groups"
    aidial.identityProviders.cognito.issuerPattern: '^https:\/\/cognito-idp\.<AWS_COGNITO_REGION>\.amazonaws\.com.+$'
  ingress:
    enabled: true
    ingressClassName: alb
    annotations:
      alb.ingress.kubernetes.io/scheme: internet-facing
      alb.ingress.kubernetes.io/target-type: ip
      alb.ingress.kubernetes.io/healthcheck-path: /health
      alb.ingress.kubernetes.io/certificate-arn: <AWS_CERTIFICATE_ARN>
      alb.ingress.kubernetes.io/ssl-redirect: "443"
    hosts:
      - dial.<DOMAIN>

bedrock:
  # IRSA role granting access to Amazon Bedrock
  serviceAccount:
    annotations:
      eks.amazonaws.com/role-arn: <AWS_BEDROCK_ROLE_ARN>
  env:
    aidial.bedrock.region: <AWS_BEDROCK_REGION>

chat:
  env:
    AUTH_CALLBACK_BASE_URL: "https://chat.<DOMAIN>"
    AUTH_POST_LOGOUT_REDIRECT_URI: "https://chat.<DOMAIN>"
    DIAL_CORE_URL: "http://dial-core.dial.svc.cluster.local"
    CORS_ORIGIN: "https://chat.<DOMAIN>"
  secrets:
    AUTH_SESSION_SECRET: <AUTH_SESSION_SECRET>
    AUTH_COGNITO_CLIENT_ID: <AUTH_COGNITO_CLIENT_ID>
    AUTH_COGNITO_HOST: <AUTH_COGNITO_HOST>
    AUTH_COGNITO_SECRET: <AUTH_COGNITO_SECRET>
  ingress:
    enabled: true
    ingressClassName: alb
    annotations:
      alb.ingress.kubernetes.io/scheme: internet-facing
      alb.ingress.kubernetes.io/target-type: ip
      alb.ingress.kubernetes.io/healthcheck-path: /api/health
      alb.ingress.kubernetes.io/certificate-arn: <AWS_CERTIFICATE_ARN>
      alb.ingress.kubernetes.io/ssl-redirect: "443"
    hosts:
      - chat.<DOMAIN>

DIAL Core and DIAL Chat each get their own Ingress under core.ingress and chat.ingress, so the AWS Load Balancer Controller provisions one ALB per service. The upstream example adds further ALB tuning (target-group stickiness, HTTP/2, a pinned TLS policy) that this snippet omits for brevity; see the AWS example - simple for the full set.

core.env registers Cognito as the identity provider DIAL Core trusts for user tokens; chat.env/chat.secrets configure DIAL Chat's own sign-in flow against the same user pool. In your Cognito app client, set the allowed callback and sign-out URLs to match AUTH_CALLBACK_BASE_URL.

Replace the placeholders:

  • <AWS_CORE_ROLE_ARN> — IRSA role ARN for S3 access
  • <AWS_CORE_S3_BUCKET_NAME> — the S3 bucket name
  • <AWS_COGNITO_REGION> — the AWS region of your Cognito user pool
  • <AWS_COGNITO_ID> — the Cognito user pool ID
  • <AWS_BEDROCK_ROLE_ARN> — IRSA role ARN for Bedrock access
  • <AWS_BEDROCK_REGION> — the region where Bedrock models are available
  • <DOMAIN> — your base domain; DIAL Core is exposed at dial.<DOMAIN> and DIAL Chat at chat.<DOMAIN>, for example example.com
  • <AUTH_COGNITO_CLIENT_ID> — the Cognito app client ID
  • <AUTH_COGNITO_HOST> — the Cognito hosted UI domain, for example https://your-domain.auth.<AWS_COGNITO_REGION>.amazoncognito.com
  • <AUTH_COGNITO_SECRET> — the Cognito app client secret
  • <AUTH_SESSION_SECRET> — a random secret DIAL Chat uses to encrypt session cookies
  • <AWS_CERTIFICATE_ARN> — the ACM certificate ARN for HTTPS

Generate the DIAL API key, encryption key, cache password, and <AUTH_SESSION_SECRET> before installing and supply them as values or Kubernetes secrets.

Step 2: Install the chart​

helm install dial dial/dial \
  -f values.yaml \
  --version <CHART_VERSION> \
  --namespace dial \
  --create-namespace

Pin <CHART_VERSION> to a released chart version.

Step 3: Verify the deployment​

Check that the pods are running:

kubectl get pods -n dial

Once both ALBs have addresses, DIAL Chat is reachable at https://chat.<DOMAIN>/; open it in a browser and sign in through your Cognito user pool. Confirm the Unified API responds:

curl https://dial.<DOMAIN>/openai/deployments \
  -H "Api-Key: <YOUR_DIAL_API_KEY>"

Result​

DIAL Core, DIAL Chat, Valkey, and the Bedrock adapter run on your EKS cluster, with files persisted to S3 and models served from Bedrock.

Note

The bedrock alias of the dial chart deploys the standalone Bedrock adapter, which will be deprecated. DIAL Model Provider replaces it. Deploy it with the DIAL Extension chart and move your models as described in Migrate from the standalone adapters.

Production architecture​

The simple example is a starting point. For a production deployment, size the AWS services and configure redundancy as follows.

AWS serviceConfiguration / tier
Amazon EKSStandard, zone-redundant
Elastic Load Balancing (ALB)Standard
Amazon S3S3 Standard, with Multi-Region Access Points
ElastiCacheStandard, Multi-AZ
Amazon BedrockSized to capacity requirements; usage-based pricing

Dependencies whose technology may follow corporate standards:

DependencyAWS service
SSOAmazon Cognito
FirewallAWS WAF with ALB or CloudFront, plus VPC security groups
Logging and monitoringAmazon CloudWatch and AWS CloudTrail
Container registryAmazon ECR
Secret storageAWS Secrets Manager

Scalability​

DIAL services are stateless, so they run as multiple concurrent copies. As a baseline, run two copies of each service on different nodes for high availability. The services are not resource-intensive and handle thousands of users without further scaling.

To absorb load spikes, configure autoscaling for DIAL Core based on CPU usage (for example, scale out when usage exceeds 80% of the CPU limit for one minute), and enable cluster autoscaling so new nodes are added when existing nodes lack capacity.

Scaling language models is the more common need, because a single Bedrock model instance is often capped by request rate limits. Options, in order of preference:

  1. Request a regional quota increase through the AWS Service Quotas console or AWS Support.
  2. Invoke models in additional regions, optionally with cross-region inference.
  3. Use multiple AWS accounts to access model endpoints.

High availability​

Achieve high availability through Multi-AZ deployments: run DIAL service copies across availability zones and use highly available configurations for the supporting services.

ServiceAvailability SLAResilient to zone-wide outages
Amazon EKS99.95%Yes, if configured explicitly
AWS IAM99.0%–99.99%Yes
Amazon S399.9%Yes, with multi-region replication
ElastiCache (Redis OSS)99.99% with Multi-AZYes, if configured explicitly
Amazon Bedrock99.9%Depends on configuration
AWS Secrets Manager99.99%Yes
Amazon ECR99.9%Yes (regional)

Because Bedrock availability depends on configuration, define multiple model endpoints in different regions. If one region becomes unavailable, DIAL automatically tries the next endpoint in the list.

Disaster recovery​

DIAL has two parts with separate recovery policies: DIAL Core and the optional DIAL Analytics.

DIAL Core consists of stateless services plus a single permanent store on S3 (ElastiCache is a cache and can be safely restarted). For a region-wide failure:

  • S3 — use Cross-Region Replication or Multi-Region Access Points for built-in cross-region redundancy. After a regional failure, review and re-establish the multi-region configuration as needed.
  • All other services — being stateless, redeploy the full set in a secondary region and re-route the DIAL domain name to the new entry point. To meet a tight recovery time objective, keep a passive DIAL Core deployment in the secondary region and activate it on failure.

When the primary region is restored, there is no need to fail back.

DIAL Analytics keeps ingested rows in ClickHouse and its catalog of tables, pipelines, saved queries, and dashboards in PostgreSQL. Back up PostgreSQL to keep those definitions. Replicate the usage-log blobs it reads like any other permanent store: the backfill endpoint can re-ingest a date range from them, up to BACKFILL_MAX_DAYS days per request.

Further reading​