Skip to main content

Analytics

Introduction​

AI DIAL Core collects system logs and chat completion logs:

  • System logs do not include any user data and contain logs of all requests from system components to AI DIAL Core (using the ELK stack (Elasticsearch, Logstash, Kibana) or other log collection system). Refer to Observability to learn about logs, metrics and traces supported in AI DIAL.
  • Chat completion requests logs include information that users send in their requests to LLMs and the information they get in responses.

AI DIAL setup can include a special service called DIAL Analytics Realtime, which uses diverse techniques such as embedding algorithms, clustering algorithms, frameworks, light-weight self-hosted language models, to analyze chat completion logs and extract the needed information, which can be presented in tools such as Grafana for visualization and analytics.

Refer to Analytics Realtime repository to learn more and view the project source code.

Analytics Realtime does not retain any private information, such as user prompts or conversations, beyond the system. Instead, only the computed artifacts are collected and stored in time-series databases like InfluxDB or any scalable database capable of handling voluminous, constantly changing information.

Examples of the computed artifacts:

  • Information about user(user hash, title, and never personal data such as names).
  • What areas have people asked questions about?
  • Are there any recurring patterns?
  • Topics of conversations.
  • Unique users.
  • Sentiments.
  • Cost analysis of the communication.
  • Language of conversations.
  • Any other calculated statistics based on conversations.

Configuration​

Refer to Analytics Realtime Configuration to see configuration guidelines.

Disable Logging of User Information​

AI DIAL Core will log all user requests and deployment responses authorized with an access token (like JWT) or unsecured API keys. To avoid this, create and use a secured API key - in this case the system will not collect logs of user requests and related responses from applications, assistants and models.

Refer to AI DIAL Core configuration to learn how to create secured and other types of API keys.