0.12.0 Release Notes

This page includes updates from our 0.12.0 Release, which includes new features, stability and performance improvements.

Deepchecks LLM Evaluation 0.12.0 Release

• 🦠New: Pentest Environment for detecting vulnerabilities that your LLM app is prone to
• 🔠 Improved Support for Non-English Use Cases
• 🗒️ Docs Additions: E2E Use case (GVHD), and new demonstrations for many evaluation features
• 🪞"Golden Set" Environment Renamed to "Evaluation" in the UI

🚧

SDK Breaking Changes

The api for determining the version and environment the SDK is set to has changed.

Previously as a method:

Python

dc_client.env_type(EnvType.PROD)
dc_client.version_name('v1')

Now, updating the class member:

Python

dc_client.env_type = EnvType.PROD
dc_client.version_name = 'v1'

What's New and Improved

• Pentest Environment
  • A dedicated environment for testing your system against known attack types. Enable it in the "Workspace Settings" to check it out.
  • Includes Pentesting data for running on your app, which should then be uploaded to Deepchecks to get an evaluation of your app's resilience to different types of attacks.
  • For more info, check out: Pentesting Your LLM-Based App
• Improved Support for Non-English Use Cases
  • Deepchecks now includes built-in support for additional languages.
  • Reach out to us to have it enabled for your organization.
• Docs Additions
  • New GVHD Use Case : Q&A Example use case to run through and get familiar with the system and its values
  • Deep explanations about features and value, such as Hard Sample Mining for Fine-Tuning can be found in the "Usage Scenarios" section.