LLM Properties

LLM properties are properties evaluated by LLMs, and are used to evaluate the more nuanced qualities of your LLM interactions. These are calculated by asking an LLM to grade your interaction according to given steps using a score of 1-5 (with higher being better).

When you create a task in Deepchecks LLM Evaluation your task is by default initialized with a given set of such properties appropriate for your task type. You can then go on and add new LLM properties by defining the steps the LLM will use when grading your interaction, and the components of the interaction it will be using to grade. For example, the default LLM property "Coherence", used for summarization tasks, is defined as follows:

📘

Recalculate Properties

When a new property is defined, it will be automatically applied to all of the new interactions sent to the system. You can use the "Recalculate" button in order to trigger the LLM properties calculation for existing interactions. If you "recalculate" for interactions that already have values, the "recalculate" will override them. Note: this recalculation is part of the "LLM Tokens Processed" in system usage.