0.44.0 Release Notes
Deepchecks LLM Evaluation 0.44.0 Release
Version 0.44 brings GPT-OSS model support, SDK enhancements, and several UX improvements across the platform.
Deepchecks LLM Evaluation 0.44.0 Release:
🧠 GPT-OSS Model Support
📖 Documentation Revamp
🪜 Steps Support in log_spans
📊 Total System Metrics
📥 Prompt Property JSON Download & Upload
💡 Suggested Feedback for Few Shots
🔒 SageMaker VPC Endpoint for Logs
🚨 Deprecated SDK Functions & Endpoints
What's New and Improved?
GPT-OSS Model Support
Deepchecks now supports GPT-OSS models for property evaluation. This allows self-hosted deployments to run LLM-based properties using open-source GPT models, removing the dependency on external LLM providers.
Documentation Revamp
The Deepchecks documentation has undergone a major revamp — restructured, rewritten, and expanded to cover the full platform in a clear and navigable way. Check it out here.
Steps Support in log_spans
The log_spans SDK function now accepts an optional steps parameter on each span. Steps allow you to attach arbitrary custom data to spans - any structured information you want to upload and later use for property calculations.
See more details here.
Total System Metrics
System metrics (input tokens, output tokens, total tokens, cost) now display total values alongside min/avg/max statistics. This gives a clearer picture of aggregate resource consumption per session.
Prompt Property JSON Download & Upload
Prompt property configurations can now be downloaded and uploaded as JSON files. This replaces the previous notebook-based export and makes it easier to version-control, share, and migrate prompt property configurations between environments.
Suggested Feedback for Few Shots
When refining LLM property evaluations with few-shot examples, the system now suggests feedback based on existing data. Suggestions are validated and aligned to help you build high-quality few-shot examples faster.
SageMaker VPC Endpoint for Logs
Self-hosted SageMaker deployments can now route logs through a VPC endpoint, keeping log traffic within your private network and avoiding public internet exposure.
Deprecated SDK Functions & Endpoints
The include_extended_data parameter on GET /applications is deprecated and will be removed in v0.47. Use the new GET /applications/extended endpoint instead, which provides paginated access to application data with full metrics.