Chat completion

The aidial_sdk.chat_completion module contains the core abstractions for building chat completion applications. This page covers the ChatCompletion abstract base class and the enums shared across the module.

from aidial_sdk.chat_completion import ChatCompletion, Request, Response

How the pieces fit together

When DIAL Core sends a request to your application, the SDK:

1. Parses the incoming HTTP request into a Request object.
2. Creates a Response object.
3. Calls your ChatCompletion.chat_completion(request, response) method.
4. Inside chat_completion, you create one or more Choice objects from the response.
5. Each choice can stream content, add attachments, and create Stage objects for progress reporting.
6. The SDK serializes everything into the Unified API response format (streaming SSE or a single JSON object).

Request ──► ChatCompletion.chat_completion(request, response)
                                              │
                                              ├── response.create_single_choice()
                                              │       │
                                              │       ├── choice.append_content("...")
                                              │       ├── choice.add_attachment(...)
                                              │       └── choice.create_stage("step 1")
                                              │               ├── stage.append_content("...")
                                              │               └── stage.close()
                                              │
                                              └── response.set_usage(prompt_tokens=10, completion_tokens=20)

ChatCompletion

Abstract base class that you subclass to implement application logic.

from aidial_sdk.chat_completion import ChatCompletion

Abstract methods

chat_completion

async def chat_completion(self, request: Request, response: Response) -> None

The main entry point for your application. Receives the parsed request and a response object that you populate with choices, content, and metadata.

This is the only method you must implement.

Optional methods

Override these methods to enable additional endpoints. If not overridden, the corresponding routes are not registered.

rate_response

async def rate_response(self, request: RateRequest) -> None

Handles response rating feedback. Called when a user rates a response in DIAL Chat.

RateRequest fields:

Field	Type	Description
response_id	str	ID of the response being rated.
rate	bool	True for positive, False for negative. Default: False.

tokenize

async def tokenize(self, request: TokenizeRequest) -> TokenizeResponse

Counts tokens for the given inputs. Used by DIAL Core for prompt management.

TokenizeRequest fields:

Field	Type	Description
inputs	list[TokenizeInput]	List of inputs to tokenize. Each input is either a TokenizeInputRequest (a full chat completion request) or a TokenizeInputString (a plain string).

TokenizeResponse fields:

Field	Type	Description
outputs	list[TokenizeOutput]	One result per input. Each is either TokenizeSuccess(token_count=int) or TokenizeError(error=str).

truncate_prompt

async def truncate_prompt(
    self, request: TruncatePromptRequest
) -> TruncatePromptResponse

Determines which messages to discard when the prompt exceeds a token limit.

TruncatePromptRequest fields:

Field	Type	Description
inputs	list[ChatCompletionRequest]	List of chat completion requests to evaluate.

TruncatePromptResponse fields:

Field	Type	Description
outputs	list[TruncatePromptResult]	One result per input. Each is either TruncatePromptSuccess(discarded_messages=list[int]) (indices of messages to remove) or TruncatePromptError(error=str).

configuration

async def configuration(
    self, request: ConfigurationRequest
) -> ConfigurationResponse | dict

Returns application configuration metadata. The response is a free-form dictionary or ConfigurationResponse (a Pydantic model with extra = "allow"). DIAL Chat uses this to discover supported features, such as attachment types or custom properties.

Enums

FinishReason

from aidial_sdk.chat_completion.enums import FinishReason

Value	Description
FinishReason.STOP	Normal completion.
FinishReason.LENGTH	Stopped because max_tokens was reached.
FinishReason.FUNCTION_CALL	Model returned a function call (legacy).
FinishReason.TOOL_CALLS	Model returned one or more tool calls.
FinishReason.CONTENT_FILTER	Content was filtered.

Status

from aidial_sdk.chat_completion.enums import Status

Value	Description
Status.COMPLETED	Stage completed successfully.
Status.FAILED	Stage failed.

Sub-pages

• Request — request model and all sub-models
• Response — response methods for choices, usage, and metadata
• Choice — streaming content, attachments, tool calls, and forms
• Stage — progress reporting within a choice