Skip to main content

Advanced topics

This section covers optimization and performance techniques for developers who already build with DIAL and want to reduce cost and latency in production. You should be comfortable calling the Unified API and configuring model deployments in DIAL Core.

In this section​

  • How prompt caching works — why prompt caching saves cost and latency, how DIAL Core routes shared prefixes to the same upstream, and the difference between automatic and manual caching.
  • Tutorial: enable and verify prompt caching — a hands-on walkthrough that turns on caching for a model deployment, sends two requests with a shared prefix, and confirms the cache hit in the response usage.