Documentation
DocumentationAPI ReferenceRelease Notes
DocumentationLog In
Release Notes
Back to All

0.22.0 Release Notes

23 days ago by Shir Chorev

This version adds support for multi-step workflows, by allowing different types of interactions within a single application. Properties and annotations now run on the Interaction Type level. This, alongside additional improvements such as to the Grounded in Context property, UI simplifications, stability improvements and performance enhancements, are part of our 0.22.0 release.

Deepchecks LLM Evaluation 0.22.0 Release

  • 🚀 Enhanced Support for Complex Applications
    • 🧩 New Interaction Types Layer
    • 🔄 SDK Updates
  • ☝️ Improved Grounded in Context Property
  • 🟣 Simplified Versions and Auto-annotation Screen

What’s New and Improved?

  • Enhanced Support for Complex Applications - Interaction Types

    • Applications now natively support multi-phase workflows.
    • Interaction types allow specifying a distinct type for each phase in the application, allowing to adapt the properties and evaluation for that logical phase. Supported predefined types include Q&A, Summarization, Generation, Classification, and Other.
    • For more details about configuring the Properties and annotation on the Interaction Type level, see Properties and Auto-Annotation YAML Configuration.

  • SDK/API Updates

    • The app_type parameter now determines the default interaction type for all interactions within an application. This provides a more intuitive setup and ensures consistent property evaluation.

      # Example usage
dc_client.create_application(APP_NAME,
                             app_type=ApplicationType.QA)

    • The new LogInteraction class introduces support for the optional interaction_type parameter, allowing you to specify the type of interaction directly when logging.
      Note:While LogInteractionType is still supported for backward compatibility, we recommend transitioning to LogInteraction as LogInteractionType will be deprecated in future versions.

      from deepchecks_llm_client.data_types import LogInteraction

single_sample = LogInteraction(
    user_interaction_id="id-1",
    input="my user input1",
    output="my model output1",
    started_at="2024-09-01T23:59:59",
    finished_at=datetime.now().astimezone(),
    annotation="Good",  # Either Good, Bad, Unknown, or None
    interaction_type="Generation"  # Optional. Defaults to the application's default type if not provided.
)

    • Interaction types can now be specified in SDK methods designed for creating or retrieving interactions. Methods for logging interactions, such as log_interaction and log_batch_interactions, now allow assigning interaction types during creation. Similarly, data retrieval methods like get_data and data_iterator support an interaction_types array, enabling filtering and retrieval based on specific interaction types. For more, see Deepchecks' SDK.