Documentation
DocumentationAPI ReferenceRelease Notes
DocumentationLog In
Release Notes
Back to All

0.8.0 Release Notes

8 months ago by Shir Chorev

This page includes includes updates from our 0.8.0 Release, which includes new features, stability and performance improvements.

Deepchecks LLM Evaluation 0.8.0 Release

  • 🧮 Estimated score & Annotated score
  • 🪞New Versions Screen incl. Per-version Metrics (Score & Properties)
  • 💬 LLM-based properties support
  • 🔄 Auto-annotation pipeline updates & rerun capability
  • 🎨 Sidebar redesign
  • 👀 Calculation in progress status for properties, topics and segments

What's New and Improved

  • Estimated Score & Annotated Score

    • The Overview Screen (and in next version also the Versions Screen) enables seeing the overall score both with and without deepchecks' estimated annotations.
      For example, in the following app, 21% of the interactions that didn't have user annotations (in grey) do have estimated annotations, and the score is updated accordingly.

      Score with and without estimated annotations By default, estimated annotations are included

      Score with and without estimated annotations
      By default, estimated annotations are included

    • Hover on the annotation enables to see the annotations detailed aplit

  • Versions Screen

    • The new versions screen replaces the "data management" screen, and enables metric comparison. This includes: Score & Property values for the production and golden set datasets for each version.

    • Versions can be sorted according to: creation date, last modified date, score or property values. Clicking on a version will open the Overview page of the corresponding application version.

  • LLM Properties

    • LLM-based properties are now supported. They can be customized in the "Custom Properties" Screen. See the LLM Properties docs for more details.
    • Note: LLM-properties calculation are limited to 5K interactions*properties per day and to 20K and per month (e.g. for 1000 interactions with 5 LLM-based properties defined: all will be calculated. If more, than only first 5000 calculations will return property values).

  • Auto-annotation pipeline updates & rerun capability

    • Auto-annotation pipeline can now be triggered to rerun, for example after new configuration is uploaded:

    • Conditions on categorical values are now supported in auto-annotation YAML config.

    • 📘

      New YAML Structure for Auto-annotation Pipeline

      In order to update auto-annotation configuration for new applications, YAML should be downloaded from the config page and edited accordingly. If you have manually saved aside YAML files from previous versions, they won't be supported as-is. To replace them: open the auto annotation config page for the relevant application version and download the configuration in the new format (it will be updated with your customizations if you'd made any).

  • Sidebar Redesign

    • Sidebar structure updated for more convenient browsing

    • Workspace settings, API key & more are now accessible in a sidebar that opens up when selecting the user icon

  • Calculation in progress status

    • Status in system can now be distinguished between "No Value" since the result is N/A or there is not enough data for calculation (e.g. not enough interactions sent for topic calculation), and "Calculation in progress". This is relevant for: Topics & Segments, and status for each is updated accordingly.
    • For properties: when property values are N/A, or when they are still in calculation, the following view will be displayed in the Overview screen: