Properties
Properties are the foundation of evaluation in Deepchecks - individual quality scores calculated on every interaction, used for annotations, comparison, filtering, and root cause analysis.
When your data arrives in Deepchecks, the evaluation pipeline calculates properties on every interaction. Properties are individual quality scores - numeric (e.g., 1-5) or categorical (e.g., a topic label or risk category). They range from simple text characteristics like word count to complex evaluations like hallucination detection, retrieval relevance, or plan efficiency for agents.
Properties are the foundation of everything in Deepchecks. Automatic annotations are built on property scores. Version comparison uses property averages. Root cause analysis drills into property values to find problems. Production monitoring tracks property trends over time.
Types of properties
Built-in Properties
Created and maintained by Deepchecks. They run automatically on every interaction - no setup required. They range from simple text analysis to complex proprietary models, some using LLM calls. Examples include Grounded in Context, Toxicity, Retrieval Relevance, and Sentiment.
→ Built-in Properties - full catalog, organized by category
Prompt Properties
Custom LLM-as-a-judge evaluators you define in natural language. You write guidelines describing what to evaluate, and Deepchecks scores every interaction accordingly. Two types: numerical (1-5 score) and categorical (classification into your defined labels).
→ Prompt Properties - how to create and configure them
User-Value Properties
Values you calculate yourself and send to Deepchecks alongside your interaction data - via the Python SDK or CSV upload. Once uploaded, they behave like any other property: available for annotations, filtering, and analysis.
→ User-Value Properties - how to upload and use them
Session-Level Properties
Evaluate entire multi-turn sessions rather than individual interactions. They detect patterns that only emerge across multiple turns - like whether the user's goal was ultimately achieved.
→ Session-Level Properties - built-in and custom session properties
Default configurations and interaction types
Each interaction type (Q&A, Summarization, Agent, RAG, etc.) comes with its own default set of built-in properties - the ones most relevant for that type. This is a starting point, not a limit. You can add any built-in property to any interaction type, and create prompt or user-value properties for any of them.
Pinned vs. other properties
Deepchecks calculates all configured properties on every interaction - pinned or not. Pinning only affects visibility.
Pinned properties appear prominently across the UI:
- On the Overview screen - as chart widgets showing distribution and trends
- In interaction details - listed first when you open an individual interaction
- In Version Comparison - the first three pinned numerical properties drive the star rating per interaction pair
Non-pinned properties are still fully calculated. They appear under "Other properties" in interaction details and remain available for filtering and annotations.
To pin or unpin a property, go to the Properties screen and click the pin icon next to it.
Tip: If there's a property you are interested to see in the Overview screen, make sure that it's marked as pinned in the Properties screen.
The Properties screen
Accessible from application settings. Shows all active properties for each interaction type, with their name, type, and description. From here you can pin/unpin, add, edit, pause, or remove properties.
Adding: Click Add Properties to select from the built-in bank, or Create Custom Property to build a prompt or user-value property. New properties run on all future interactions; use Recalculate to apply them to existing data.
Editing: Any LLM-based property (built-in or prompt) can be edited after creation. Open the Property Info dialog and click Edit & Test Property. Use the test feature to validate on a sample before saving, then recalculate existing data if needed for consistency.
Removing: Click the remove icon. If the property is referenced in the auto-annotation configuration, you'll need to remove it from that YAML first.
Pausing: Temporarily stops a property from running on new data without removing it. See Activate or Pause Property Calculations.
Updated 23 days ago