Response

The Response class is the output side of a chat completion. You receive it as the second argument to ChatCompletion.chat_completion() and use it to create choices, report usage, and set metadata.

from aidial_sdk.chat_completion import Response

Creating choices

create_single_choice

def create_single_choice(self) -> Choice

Creates and returns a single Choice. Use this when your application generates exactly one response (the common case).

Raises RuntimeServerError if called after a choice has already been created. Raises RequestValidationError if the request specifies n > 1.

async def chat_completion(self, request: Request, response: Response) -> None:
    with response.create_single_choice() as choice:
        choice.append_content("Hello!")

create_choice

def create_choice(self) -> Choice

Creates and returns a Choice. Call this multiple times (up to request.n times) to generate multiple response alternatives.

Raises RuntimeServerError if called more times than the requested n.

async def chat_completion(self, request: Request, response: Response) -> None:
    for i in range(response.n):
        with response.create_choice() as choice:
            choice.append_content(f"Alternative {i + 1}")

Usage reporting

Call these methods after all choices have been created.

set_usage

def set_usage(self, prompt_tokens: int = 0, completion_tokens: int = 0)

Reports token usage for the entire response. DIAL Core uses this for billing, rate limiting, and analytics.

Raises RuntimeServerError if called before all choices are created or if called twice.

response.set_usage(prompt_tokens=150, completion_tokens=42)

add_usage_per_model

def add_usage_per_model(
    self,
    model: str,
    prompt_tokens: int = 0,
    completion_tokens: int = 0,
)

Reports token usage broken down by upstream model. Call this once per model when your application delegates to multiple models. Can be called multiple times for different models.

Raises RuntimeServerError if called before all choices are created.

response.add_usage_per_model("gpt-4", prompt_tokens=100, completion_tokens=30)
response.add_usage_per_model("gpt-3.5-turbo", prompt_tokens=50, completion_tokens=12)

set_discarded_messages

def set_discarded_messages(self, discarded_messages: list[int])

Reports which message indices were discarded due to token limits. DIAL Chat uses this to indicate truncation in the UI.

Raises RuntimeServerError if called before all choices are created or if called twice.

response.set_discarded_messages([0, 1])

Metadata

Call these methods before creating any choices. They set values that appear in every streamed chunk.

set_model

def set_model(self, model: str)

Sets the model field in the response. Useful when your application proxies to a model and wants to report which model was actually used.

Raises RuntimeServerError if called after generation has started.

set_response_id

def set_response_id(self, response_id: str)

Overrides the auto-generated response ID. The default is a random UUID.

Raises RuntimeServerError if called after generation has started.

set_created

def set_created(self, created: int)

Overrides the auto-generated created timestamp (Unix epoch seconds). The default is the current time.

Raises RuntimeServerError if called after generation has started.

Flushing

aflush

async def aflush(self)

Waits until all queued chunks have been sent to the client. Use this when you need to ensure the client has received all content up to this point before continuing.

with response.create_single_choice() as choice:
    choice.append_content("Processing...")
    await response.aflush()
    # Client has now received "Processing..."
    choice.append_content(" Done!")

Properties

Property	Type	Description
n	int	Number of choices requested (from request.n, defaults to 1).
stream	bool	Whether the response is streamed.
request	Request	The original request.

Further reading

• Request — request fields, messages, and tools your SDK app receives
• Choice — write content, attachments, and tool calls
• Custom content in Chat — see how response metadata is rendered