Tutorial: enable and verify prompt caching
In this tutorial, you will enable manual prompt caching on a model deployment, send two chat completion requests that share a long prefix, and confirm from the response that the second request hit the cache. By the end, you will know how to mark a cache breakpoint and how to read cached-token counts from the response usage. This tutorial is for developers who can already call the Unified API; for the concepts behind caching, read How prompt caching works first.
Prerequisites
- A running DIAL deployment and its base URL, referred to below as
<DIAL_URL>. - A DIAL API key, referred to below as
<YOUR_DIAL_API_KEY>. - A
chatmodel deployment in DIAL whose provider supports prompt caching, referred to below as<DEPLOYMENT_NAME>. Confirm caching support with your provider before starting. - Edit access to the DIAL Core
config.json(dynamic settings). curlandjqinstalled.
What you'll do
- Turn on manual caching for the deployment with the
cacheSupportedflag. - Send a first request with a
cache_breakpointafter a long shared prefix and store the cache entry. - Send a second request that shares the same prefix and observe a cache hit in the response usage.
Project structure
By the end of this tutorial, your working directory will look like this:
dial-prompt-caching/
├── request-1.json # first request — creates the cache entry
└── request-2.json # second request — shares the prefix, hits the cache
Step 1: Enable manual caching on the deployment
In the DIAL Core config.json, find your model deployment under models and set features.cacheSupported to true:
{
"models": {
"<DEPLOYMENT_NAME>": {
"type": "chat",
"features": {
"cacheSupported": true
},
"upstreams": [
{ "endpoint": "<UPSTREAM_ENDPOINT>", "key": "<UPSTREAM_KEY>" }
]
}
}
}
cacheSupported is a dynamic setting, so DIAL Core applies it on the next configuration reload without a restart.
Verify: check the DIAL Core logs for a configuration reload entry, or confirm the file is picked up by your reload mechanism. The deployment now accepts cache breakpoints.
Step 2: Create dial-prompt-caching/request-1.json
Create the working directory and the first request:
mkdir dial-prompt-caching && cd dial-prompt-caching
Create dial-prompt-caching/request-1.json. The system message holds a long, unchanging prefix and ends with a cache_breakpoint. The user message is the only part that changes between requests:
{
"messages": [
{
"role": "system",
"content": "You are a meticulous financial analyst. Follow these rules for every answer: 1) cite the figures you rely on; 2) state assumptions explicitly; 3) flag any data older than one quarter; 4) never give investment advice; 5) answer in plain language. (Extend this block with the full instruction set your application uses — the longer and more stable the prefix, the larger the caching benefit.)",
"custom_fields": {
"cache_breakpoint": {}
}
},
{
"role": "user",
"content": "Summarize the key risks in a SaaS company with high net revenue retention but negative free cash flow."
}
]
}
Verify: jq . dial-prompt-caching/request-1.json prints the JSON without errors.
Step 3: Send the first request
Send the request to your deployment. This populates the cache entry for the shared prefix:
curl "<DIAL_URL>/openai/deployments/<DEPLOYMENT_NAME>/chat/completions?api-version=2024-10-21" \
-H "Api-Key: <YOUR_DIAL_API_KEY>" \
-H "Content-Type: application/json" \
-d @request-1.json | jq '.usage'
Verify: the response returns a completion and a usage object. On this first call, the shared prefix is written to the cache, so cached-read tokens are zero or absent. Depending on the provider, you may see cache-write tokens — for example, Anthropic reports cache_creation_input_tokens.
Step 4: Create dial-prompt-caching/request-2.json
Create dial-prompt-caching/request-2.json with the same system message and breakpoint, changing only the user message:
{
"messages": [
{
"role": "system",
"content": "You are a meticulous financial analyst. Follow these rules for every answer: 1) cite the figures you rely on; 2) state assumptions explicitly; 3) flag any data older than one quarter; 4) never give investment advice; 5) answer in plain language. (Extend this block with the full instruction set your application uses — the longer and more stable the prefix, the larger the caching benefit.)",
"custom_fields": {
"cache_breakpoint": {}
}
},
{
"role": "user",
"content": "Now do the same for a hardware company with strong margins but heavy inventory."
}
]
}
Verify: the system message in request-2.json is byte-for-byte identical to the one in request-1.json. Any difference in the prefix produces a different hash and prevents a cache hit.
Step 5: Send the second request and confirm the cache hit
Send the second request:
curl "<DIAL_URL>/openai/deployments/<DEPLOYMENT_NAME>/chat/completions?api-version=2024-10-21" \
-H "Api-Key: <YOUR_DIAL_API_KEY>" \
-H "Content-Type: application/json" \
-d @request-2.json | jq '.usage'
Verify: the usage object reports cached input tokens for the shared prefix. The exact field depends on the provider:
- Anthropic models report
cache_read_input_tokensgreater than zero. - OpenAI models report
prompt_tokens_details.cached_tokensgreater than zero.
A non-zero cached-token count confirms that DIAL Core routed the second request to the same upstream and that the model reused the cached prefix.
What you learned
- How to enable manual caching on a deployment with the
cacheSupportedflag. - How to mark a cacheable prefix with
custom_fields.cache_breakpoint. - How to read cached-token counts from the response
usageto confirm a cache hit.
Further reading
- How prompt caching works — the cache-availability policy and automatic caching
- Models configuration: caching flags — full reference for
cacheSupportedandautoCachingSupported - How DIAL Core is configured — how Core's static and dynamic configuration fit together