0.9.0 Release Notes

This page includes updates from our 0.9.0 Release, which includes new features, stability and performance improvements.

Deepchecks LLM Evaluation 0.9.0 Release

  • πŸƒβ€β™€οΈ Expanded LLM Properties Support: Test Run, See Reasoning
  • πŸ’Ύ Estimated Annotations Saved
  • ☸️ New Applications Management Screen

What's New and Improved

  • Expanded LLM Properties Support

    • LLM Properties can now be tested and results observed on chosen interactions before saving them to run.

    • Score and reasoning of the interactions can be viewed, during test run and also in the full interaction view in the "Data" Screen

  • Estimated Annotations Are Saved

    • Can be viewed and reverted to upon hover on the user's annotation.

  • Applications Management Screen

    • A list of all applications can now be accessed via the sidebar. Enabling a central place to see the applications processing status, application info log, and add or delete applications.

🚧

Breaking Changes (REST API)

Changes in setting verbosity in SDK

SDK init param changed from verbose to silent_mode - Default is False, hence you suppose to expect exceptions to be thrown, if you do not want exception - set it to True
dc_client.init(host=DEEPCHECKS_LLM_HOST, api_token=DEEPCHECKS_LLM_API_KEY, app_name=DEEPCHECKS_APP_NAME, version_name=version, env_type=env_type, silent_mode=False,

REST API changes in the /interaction endpoint.

  • Body params change:
    • data InteractionCreationSchema -> List[InteractionCreationSchema]
    • env_typewas excluded from data and moved to be top level
    • app_namewas excluded from data and moved to be top level
    • version_namewas excluded from data and moved to be top level
  • Before:
    { "user_interaction_id": "string", "full_prompt": "string", "information_retrieval": "string", "input": "string", "output": "string", "annotation": "good", "annotation_reason": "string", "app_name": "string", "version_name": "string", "env_type": "PROD", "raw_json_data": {}, "steps": [ { "name": "string", "annotation": "good", "type": "LLM", "attributes": {}, "started_at": "2024-01-18T07:48:25.156Z", "finished_at": "2024-01-18T07:48:25.156Z", "input": "string", "output": "string", "error": "string" } ], "custom_props": {}, "started_at": "2024-01-18T07:48:25.156Z", "finished_at": "2024-01-18T07:48:25.156Z" }
  • After:
    { "app_name": "string", "version_name": "string", "env_type": "PROD", "interactions": [ { "user_interaction_id": "string", "full_prompt": "string", "information_retrieval": "string", "input": "string", "output": "string", "annotation": "good", "annotation_reason": "string", "raw_json_data": {}, "steps": [ { "name": "string", "annotation": "good", "type": "LLM", "attributes": {}, "started_at": "2024-01-18T07:50:26.707Z", "finished_at": "2024-01-18T07:50:26.707Z", "input": "string", "output": "string", "error": "string" } ], "custom_props": {}, "started_at": "2024-01-18T07:50:26.707Z", "finished_at": "2024-01-18T07:50:26.707Z" } ] }