Use Sampling Properly

Sampling is an essential tool for working efficiently with large-scale production data. When your application reaches production and begins generating high volumes of user interactions, it is often unnecessary (and costly) to evaluate every single session. Deepchecks allows you to configure sampling on the application level, helping you balance between cost and insight.

The first thing you should do when uploading production data to Deepchecks is to set a proper sampling sampling ratio. This can be done via the Edit Application window.

For high-volume apps, a 20% sampling rate for evaluated data (set as 0.2 ratio) usually offers statistically significant results while keeping costs reasonable—about 35% of what it would cost to evaluate 100% of sessions. Deepchecks performs randomized selection, so the evaluated sessions represent the full data distribution.

Once the sampling ratio is set, sampled sessions will move through the standard evaluation flow. You’ll be able to apply all Deepchecks tools—built-in and custom property analysis, prompt-based insights, RCA, data exploration, version comparison, and more—to extract value and continuously improve your pipeline. Unevaluated sessions will be stored separately in the Storage screen. This screen offers basic visibility into unprocessed sessions—filterable by app, version, and time range—and allows you to view individual sessions (as raw JSON), search and sort by date or user annotations, and selectively send sessions to full evaluation.

When you're ready to evaluate specific stored sessions, you can do so individually from the session view or in bulk from the storage list. Once you click Evaluate, you'll see a summary of the process and its destination (app, version, environment and data) before you finally decide to evaluate it.

Important: sessions in evaluation process will only become part of the same app, version and environment they were originally sent to. After evaluation, those sessions will move from storage to the regular evaluated datasets and will no longer appear in the Storage view. Each session always resides in either evaluated data or storage—never both.