Production Monitoring

When your application is not performing it might have a significant impact on your company’s business thus monitoring the performance of your application becomes a critical task.

In the Production Tab, you can see the application behavior over time which includes the estimated score alongside the annotation distribution and the number of samples.

At the bottom of the production environment overview screen, you can toggle between viewing average property scores and score trends over time. This allows you to see which properties have had the greatest impact on your application’s performance.

Root-Cause Analysis in Production

Score Breakdown Comparison

Enables comparison of score distributions across different time ranges within the Production environment. This helps identify changes in performance over time and isolate which prompt properties or scoring dimensions contributed to the observed differences.

Interaction-Type Level Insights

Runs automated insights on production data within a selected time window. Similar to the existing insight mechanism for evaluation data, this feature analyzes the data at the interaction-type level and highlights patterns, anomalies, and contributing factors to shifts in performance.

Monitoring with External Tools

In addition to its built-in monitoring capabilities, Deepchecks also offers seamless integration with Datadog and New Relic. Additionally, it provides the ability to create custom integrations through our SDK or versatile webhook options, allowing for tailored solutions to meet specific needs.