User-Value Properties are any numerical or categorical properties, sent to Deepchecks by the user, that provide additional context or metrics about the interactions.

Once added to the system, User-Value properties can be utilized like any other property. They are available for a wide range of purposes, including annotation, root cause analysis, version comparison, and production monitoring, thus enhancing the depth and versatility of your evaluations.

Types of User-Value Properties

User-Value properties can be broadly categorized into two types: metadata properties and metric properties.

Metadata Properties

Metadata properties are descriptive attributes that provide contextual information about a sample. These properties help in organizing and categorizing samples based on various descriptive factors.

• An example for a metadata property could be the "Region" associated with a sample, such as "North America", "Europe", etc.
• In the system, metadata properties are used as part of the Root Cause Analysis (RCA). Users can receive insights and reports based on custom properties categorization.

Metric Properties

Metric properties are quantitative values that assess certain aspects of the sample. These properties provide measurable data that can be used for evaluation and comparison.

• An example for a metric property could be "Internal Document Quality" representing the vector store retrieval score.
• In the system, metric properties can be integrated into the Automatic Annotation flow and affect the estimated scores provided by Deepchecks.
• You may provide reasons for any given value. For example, you may want to add a reason of "bad grammar" to a low value of the "Internal Document Quality" property for a specific interaction. These reasons can then be viewed in the interaction view in the system.

By leveraging both metadata and metric properties, users can tailor the system to better meet their specific needs, improving the overall management and analysis of samples.

Uploading a Custom User-Value Property

Creating The Property (Optional)

New properties must be defined as either Categorical or Numeric. If a property isn’t manually created before values are sent to the system, the system will automatically assign its type — and this cannot be changed later. To avoid issues, it’s best to manually define the property in advance when possible.

Uploading a Custom Property

Properties can be uploaded either via our SDK or through the UI using CSV Upload. For CSV uploads, the relevant column should use the prefix cp followed by the desired property name. If you're including reasoning for the property, use the prefix reason_cp — this column can be left empty if not needed.