AI DIAL Architecture

Introduction

Modular architecture of AI DIAL allows implementing scalable and customized solutions to fit specific business needs. Its key building blocks include Core (the main and the only mandatory component), Chat (web application with user interface), Adapters (align APIs of LLMs with the Universal API), Applications (any custom logic with a conversation interface packaged as a ready-to-use solution), and a persistent layer that relies on a configured resilient and scalable cloud blob storage, with Redis layer on top to enhance performance. AI DIAL, allows to use various Identity Service Providers (IDPs) to manage user identities.

The Unified API enables universal connectivity between models (including models of different modalities), unified access to different embedding models, and facilitates communication with the AI DIAL Core for both external and internal clients.

Applications can form an ecosystem and interact with each other through the Unified API with access to all DIAL Core features among which is connectivity to models, file storage, access control, per-request API keys and other - see the following illustration. To enhance performance and fault tolerance, AI DIAL Core employs a proprietary load balancer and a retry mechanism. This significantly reduces delays and boosts throughput especially during peak demand. Refer to the document with the overview of the performance tests to learn more.

You can have a minimal installation (it includes only AI DIAL Core) which can be easily installed even on a personal laptop or a desktop computer and is a good starting point for getting familiar with AI DIAL. To engage chat users and access different LLMs, you can have a setup with Core, Chat and LLM adapters – we call it a standard installation. This package is more suitable for use in enterprise-level production environments.

A modular architecture allows adding as many components as needed up to a full platform landscape.

Minimal Installation

AI DIAL has only one required component – AI DIAL Core.

AI DIAL Core is headless. It exposes a Unified Protocol that can be used by custom extensions, internal and external API clients to get full access to all AI DIAL Core features.

Unified Protocol streamlines communication and fosters interoperability by eliminating the need for multiple protocols for each integration. It is fully compatible with OpenAI API and supports streaming, tokens, seeds, tools, and multi-modality.

Standard Installation

A Standard installation of AI DIAL includes, Core (including API), Chat and Adapters to selected LLMs. It can also include custom extension such as Applications, Addons or Assistants.

This configuration enables the utilization of Chat functionality, allowing users to engage with LLMs, as well as the integration of custom extensions with AI DIAL Core through the API.

Full Platform Landscape

The diagram below demonstrates the most complete landscape of AI DIAL ecosystem: