0.15.0 Release Notes

This version includes SDK usage improvements, monitoring flow with Datadog integration, avoided answer property updates, along with more features, stability and performance improvements, that are part of our 0.15.0 release.

Deepchecks LLM Evaluation 0.15.0 Release

• 🖥️ SDK Updates for Ease of Use
• 🤫 Improved Avoided Answer Property
• 👀 Datadog Monitoring Integration
• 🧮 Robust Tiers and Usage Tracking

What’s New and Improved?

• SDK Updates - important note for SDK use of deepchecks-llm-client>=0.15.0

  • SDK client (previously dc_client ) is now created by instantiating DeepchecksLLMClient.
  • All existing functions, such as log_batch_interactions, now require the app_name, version_name and env_type arguments to enable better control of logged data.

• Avoided Answer Property

  • New and improved algorithmic implementation for the avoided answer property.
  • Achieving an F1 score of above 0.98, on a variety of datasets (with the threshold of 0.5)

• Datadog Monitoring Integration

  • See full documentation here: Datadog Integration

    

  • Deepchecks now suppports seamless integration with Datadog, in order to easily track the metrics over time, view configurable dashboards and receive alerts. This enables Datadog users to have the relevant logs, dashboards and alerts, viewable and configurable all in one place.

• Robust Tiers and Usage Tracking

  • Monthly usage is now aggregated to allow automatic limits and billing, according to the product tier.