Skip to main content

DIALApp

DIALApp is the entry point for every DIAL SDK application. It extends FastAPI and registers your application logic under the Unified API route structure that DIAL Core expects.

from aidial_sdk import DIALApp

Serve a chat completion​

Implement ChatCompletion, then register it under a deployment name:

import uvicorn
from aidial_sdk import DIALApp
from aidial_sdk.chat_completion import ChatCompletion, Request, Response


class MyApp(ChatCompletion):
    async def chat_completion(
        self, request: Request, response: Response
    ) -> None:
        with response.create_single_choice() as choice:
            choice.append_content("Hello from MyApp!")


app = DIALApp()
app.add_chat_completion("my-app", MyApp())

if __name__ == "__main__":
    uvicorn.run(app, port=5000)

That registration creates the routes DIAL Core calls:

RouteMethodDescription
/openai/deployments/{name}/chat/completionsPOSTChat completion (streaming and non-streaming)
/openai/deployments/{name}/ratePOSTResponse rating
/openai/deployments/{name}/tokenizePOSTToken counting (only if tokenize is overridden)
/openai/deployments/{name}/truncate_promptPOSTPrompt truncation (only if truncate_prompt is overridden)
/openai/deployments/{name}/configurationGETConfiguration endpoint (only if configuration is overridden)

Registration returns the app, so calls can be chained.

The same handlers are also served under /openai/v1/…, where the deployment is resolved from the request headers rather than the path.

Serve embeddings​

app.add_embeddings("my-embeddings", MyEmbeddings())
RouteMethodDescription
/openai/deployments/{name}/embeddingsPOSTEmbeddings generation

See Embeddings.

Call DIAL Core as the original caller​

When your app calls other DIAL deployments — a model, or another app — those calls need the caller's credentials. Turn on propagation and the SDK forwards the incoming api-key for you:

app = DIALApp(
    dial_url="http://core:8080",
    propagate_auth_headers=True,
)

Propagation covers outgoing requests made with aiohttp, httpx, and requests. Only requests whose URL starts with dial_url get the header, so calls to anything else are left alone. dial_url is required when propagation is on.

Add a health check​

app = DIALApp(add_healthcheck=True)

Adds GET /health, which returns {"status": "ok"}. Access logs for that path are suppressed.

Keep long responses alive​

Slow completions can be dropped by proxies before the first token arrives. A heartbeat keeps the connection warm:

app.add_chat_completion("my-app", MyApp(), heartbeat_interval=25)

The interval is in seconds, and applies to SSE streaming responses.

Enable telemetry​

from aidial_sdk.telemetry.types import TelemetryConfig

app = DIALApp(telemetry_config=TelemetryConfig())

Requires the telemetry extras (pip install "aidial-sdk[telemetry]"). See Telemetry.

Extending with FastAPI features​

Because DIALApp extends FastAPI, you can use any FastAPI feature: middleware, dependency injection, custom routes, exception handlers, and lifespan events. Custom routes are served alongside the Unified API routes.

Further reading​