0.9.0 Release Notes

This page includes updates from our 0.9.0 Release, which includes new features, stability and performance improvements.

Deepchecks LLM Evaluation 0.9.0 Release

• 🏃‍♀️ Expanded LLM Properties Support: Test Run, See Reasoning
• 💾 Estimated Annotations Saved
• ☸️ New Applications Management Screen

What's New and Improved

• Expanded LLM Properties Support

  • LLM Properties can now be tested and results observed on chosen interactions before saving them to run.

  • Score and reasoning of the interactions can be viewed, during test run and also in the full interaction view in the "Data" Screen

    

• Estimated Annotations Are Saved

  • Can be viewed and reverted to upon hover on the user's annotation.

    

• Applications Management Screen

  • A list of all applications can now be accessed via the sidebar. Enabling a central place to see the applications processing status, application info log, and add or delete applications.

    

🚧

Breaking Changes (REST API)

Changes in setting verbosity in SDK

SDK init param changed from verbose to silent_mode - Default is False, hence you suppose to expect exceptions to be thrown, if you do not want exception - set it to True
dc_client.init(host=DEEPCHECKS_LLM_HOST, api_token=DEEPCHECKS_LLM_API_KEY, app_name=DEEPCHECKS_APP_NAME, version_name=version, env_type=env_type, silent_mode=False,

REST API changes in the /interaction endpoint.

• Body params change:
  • data InteractionCreationSchema -> List[InteractionCreationSchema]
  • env_typewas excluded from data and moved to be top level
  • app_namewas excluded from data and moved to be top level
  • version_namewas excluded from data and moved to be top level
• Before:
  { "user_interaction_id": "string", "full_prompt": "string", "information_retrieval": "string", "input": "string", "output": "string", "annotation": "good", "annotation_reason": "string", "app_name": "string", "version_name": "string", "env_type": "PROD", "raw_json_data": {}, "steps": [ { "name": "string", "annotation": "good", "type": "LLM", "attributes": {}, "started_at": "2024-01-18T07:48:25.156Z", "finished_at": "2024-01-18T07:48:25.156Z", "input": "string", "output": "string", "error": "string" } ], "custom_props": {}, "started_at": "2024-01-18T07:48:25.156Z", "finished_at": "2024-01-18T07:48:25.156Z" }
• After:
  { "app_name": "string", "version_name": "string", "env_type": "PROD", "interactions": [ { "user_interaction_id": "string", "full_prompt": "string", "information_retrieval": "string", "input": "string", "output": "string", "annotation": "good", "annotation_reason": "string", "raw_json_data": {}, "steps": [ { "name": "string", "annotation": "good", "type": "LLM", "attributes": {}, "started_at": "2024-01-18T07:50:26.707Z", "finished_at": "2024-01-18T07:50:26.707Z", "input": "string", "output": "string", "error": "string" } ], "custom_props": {}, "started_at": "2024-01-18T07:50:26.707Z", "finished_at": "2024-01-18T07:50:26.707Z" } ] }