Production Monitoring

When your application is not performing it might have a significant impact on your company’s business thus monitoring the performance of your application becomes a critical task.

In the Production Tab, you can see the application behavior over time which includes the estimated score alongside the annotation distribution, the number of samples, and the properties values change over time.

Root-Cause Analysis in Production

Score Breakdown Comparison

Enables comparison of score distributions across different time ranges within the Production environment. This helps identify changes in performance over time and isolate which prompt properties or scoring dimensions contributed to the observed differences.

Interaction-Type Level Insights

Runs automated insights on production data within a selected time window. Similar to the existing insight mechanism for evaluation data, this feature analyzes the data at the interaction-type level and highlights patterns, anomalies, and contributing factors to shifts in performance.

Monitoring with External Tools

In addition to its built-in monitoring capabilities, Deepchecks also offers seamless integration with Datadog and New Relic. Additionally, it provides the ability to create custom integrations through our SDK or versatile webhook options, allowing for tailored solutions to meet specific needs.