Embeddings

The aidial_sdk.embeddings module provides the base class and models for building embedding applications. Register an implementation with app.add_embeddings() to expose it through the Unified API.

from aidial_sdk.embeddings import Embeddings, Request, Response

Embeddings base class

from aidial_sdk.embeddings import Embeddings

Abstract base class with a single method to implement:

embeddings

async def embeddings(self, request: Request) -> Response

Receives an embedding request and returns the computed embeddings. Unlike ChatCompletion, this method returns the response directly (no streaming).

Request

Extends the Azure OpenAI embeddings format with DIAL-specific fields.

Standard fields (Azure OpenAI compatible)

Field	Type	Default	Description
model	str | None	None	Model identifier.
input	str | list[str] | list[int] | list[list[int]]	required	Text or tokenized input to embed.
encoding_format	"float" | "base64"	"float"	Output encoding format.
dimensions	int | None	None	Desired embedding dimensionality (if the model supports it).
user	str | None	None	End-user identifier.

DIAL extension fields

Field	Type	Default	Description
custom_input	list[EmbeddingsMultiModalInput] | None	None	Multimodal input (strings and/or attachments).
custom_fields	EmbeddingsRequestCustomFields | None	None	Custom fields (see below).

EmbeddingsRequestCustomFields

Field	Type	Default	Description
type	str | None	None	Embedding type hint (e.g., "query" or "document").
instruction	str | None	None	Task instruction for the embedding model.

EmbeddingsMultiModalInput

Union[str, Attachment, list[str | Attachment]]

Allows mixing text and file attachments in a single embedding request.

Request properties

Property	Type	Description
deployment_id	str	The deployment name from the URL path.
api_key_secret	str	The API key from the api-key header.
jwt	str | None	The JWT from the Authorization header, if present.
headers	dict[str, str]	All HTTP headers from the original request.

Response

EmbeddingResponse

Field	Type	Default	Description
data	list[Embedding]	required	List of embedding results.
model	str	required	Model used to generate the embeddings.
object	"list"	"list"	Object type (always "list").
usage	Usage	required	Token usage.

Embedding

Field	Type	Default	Description
embedding	str | list[float]	required	Embedding vector (float list) or base64-encoded string.
index	int	required	Index of the input this embedding corresponds to.
object	"embedding"	"embedding"	Object type.

Usage

Field	Type	Description
prompt_tokens	int	Number of tokens in the input.
total_tokens	int	Total tokens processed.

Example

import uvicorn
from aidial_sdk import DIALApp
from aidial_sdk.embeddings import Embeddings, Request, Response
from aidial_sdk.embeddings.response import Embedding, Usage


class SimpleEmbeddings(Embeddings):
    async def embeddings(self, request: Request) -> Response:
        inputs = (
            [request.input]
            if isinstance(request.input, str)
            else request.input
        )
        return Response(
            data=[
                Embedding(
                    embedding=[0.1, 0.2, 0.3],
                    index=i,
                )
                for i in range(len(inputs))
            ],
            model="simple-embeddings",
            usage=Usage(
                prompt_tokens=sum(len(str(inp).split()) for inp in inputs),
                total_tokens=sum(len(str(inp).split()) for inp in inputs),
            ),
        )


app = DIALApp()
app.add_embeddings("simple-embeddings", SimpleEmbeddings())

if __name__ == "__main__":
    uvicorn.run(app, port=5000)

Further reading

• Supported upstreams — find configuration examples for embedding models
• Register an embedding handler — serve the handler through DIALApp
• Configure the model in Core — make the embedding endpoint available to callers
• Unified API — find the embedding request and response schemas