Properties
What are properties in Deepchecks LLM Evaluation, what kinds of properties are there and how they are used.
Properties are one-dimensional values that are calculated on each text sample. For example, a property could be simple text characteristics such as the number of words in the text, or more complex properties such identifying if the text contains toxic language, or if a given summary is capturing the key points of the original article.
What kinds of properties are there?
- These are Deepchecks' proprietary implementations, and do not use an LLM. A few examples: Grounded in Context, Avoided Answer, Retrieval Relevance, Toxicity, Sentiment.
- Any numerical or categorical value, supplied by the user, to enrich the interactions, and allow al property flows on them.
LLM Properties π’
- Properties that run using an LLM API (configurable by Deepchecks). Including many built-in and editable templates.
What are properties used for?
Properties measure various aspects of our LLM interactions that we may be interested in. They are used in the following ways:
- Properties are used to calculate the estimated annotations. By defining rules on the calculated properties, you can create a flow that automatically estimates the quality of each LLM interaction. For example, by default summarization interactions with low Conciseness are deemed to be bad interactions.
- Average values of calculated properties are shown in the Dashboard screen. You can then dive in to a specific property and see interactions with extreme values, such as extremely irrelevant answers.
- Properties are shown in the data page, and can be used there to sort and filtered the viewed interactions. This is useful for example if you wish to see only the interactions with Toxicity > 0.5, and perhaps combine that filter with additional ones (e.g., a specific topic).
Updated 16 days ago