0.12.0 Release Notes
8 months ago by Shir Chorev
This page includes updates from our 0.12.0 Release, which includes new features, stability and performance improvements.
Deepchecks LLM Evaluation 0.12.0 Release
- 🦠New: Pentest Environment for detecting vulnerabilities that your LLM app is prone to
- 🔠 Improved Support for Non-English Use Cases
- 🗒️ Docs Additions: E2E Use case (GVHD), and new demonstrations for many evaluation features
- 🪞"Golden Set" Environment Renamed to "Evaluation" in the UI
SDK Breaking Changes
The api for determining the version and environment the SDK is set to has changed.
Previously as a method:
dc_client.env_type(EnvType.PROD) dc_client.version_name('v1')Now, updating the class member:
dc_client.env_type = EnvType.PROD dc_client.version_name = 'v1'
What's New and Improved
- Pentest Environment
- A dedicated environment for testing your system against known attack types. Enable it in the "Workspace Settings" to check it out.
- Includes Pentesting data for running on your app, which should then be uploaded to Deepchecks to get an evaluation of your app's resilience to different types of attacks.
- For more info, check out: Pentesting Your LLM-Based App
- Improved Support for Non-English Use Cases
- Deepchecks now includes built-in support for additional languages.
- Reach out to us to have it enabled for your organization.
- Docs Additions
- New GVHD Use Case : Q&A Example use case to run through and get familiar with the system and its values
- Deep explanations about features and value, such as Hard Sample Mining for Fine-Tuning can be found in the "Usage Scenarios" section.