📖

This Page Includes

• Deepchecks' main application types: 💡Q&A , 📚Summarization, 🎨Generation, 🎯Classification
• A few of the key properties for each type
• The data structure to upload for each application type
• 📘 Note 📘: Deepchecks is compatible with all application types. Selecting your specific application type helps you begin with optimized default settings, which you can fully customize to your needs. If you're not sure which configuration would be the best starting point, you can choose "Other", or just consult with us!

Question Answering (RAG) 💡

An application whose end goal is to answer a question provided by the end user. The application can contain multiple intermediate steps such as embeddings-based information retrieval in the classic RAG scheme or SQL query generation to retrieve relevant information.

Q&A Key Properties

Among the various Deepchecks' properties, we found the following to be especially useful for Q&A applications: Grounded in Context, Retrieval Relevance, Relevance, Correctness, and Completeness.

Data to upload for Q&A applications

• Input (str) - should contain the user-provided question.
• Information Retrieval (List[str]) - retrieved information on which the answer should be based. Either documents in the case of RAG or data retrieved from the database.
• History (List[str]) - additional context relevant to the interaction, which wasn't retrieved from a knowledge base. For example: chat history.
• (Optional) Additional Steps - In case your application has additional intermediate steps they should be logged under the steps mechanism. See more details here.
• (Optional) Full Prompt (str) - the full prompt that was sent to the LLM.
• Output (str) - the final response that was displayed to the user.

In cases where the question is asked after a chat between the user and the application, the chat history should be included as part of the input.

See GVHD Q&A Example for a complete RAG application quickstart.

Summarization 📚

An application designed to condense text into either a free-form summary or a specified format. Examples of such applications are meeting-summary bots and bots tasked with summarizing a legal case or a financial report.

Summarization Key Properties

Among the various Deepchecks' properties, we found the following to be especially useful for summarization applications: Coverage, Grounded in Context, Information Density, and Coherence.

Data to upload for Summarization applications

• Input (str) - The text to be summarized
• (Optional) Additional Steps - In case your application has additional intermediate steps they should be logged under the steps mechanism. See more details here.
• (Optional) Full Prompt (str) - the full prompt that was sent to the LLM.
• Output (str) - the generated summary.

Generation 🎨

Applications that utilize data to perform a creative task of some sort. This is the most diverse application type and can include several different kinds of applications. To name a few specific examples:

• Report Generation - receives information about a company or real estate asset, and generates a report in a specific requested format.
• Content Generation - receives details about a subject and generates a web page. For example, generate an "About Us" page on a company's website.
• Docstring Generation - receives a code snippet describing a class or a function and generates a high-quality docstring for the input code.

Generation Key Properties

Among the various Deepchecks' properties, we found the following to be especially useful for generation applications: Grounded in Context, Relevance, and Coherence. In addition, it is likely that different applications could significantly benefit from additional properties so it is highly recommended to check out the full catalog.

Data to upload for Generation Applications

• Input (str) - The user's input to the application, without additional application instruction.
• (Optional) Information Retrieval (List[str]) - retrieved information on which the output should rely.
• (Optional) History (List[str]) - additional context relevant to the interaction, which wasn't retrieved from a knowledge base. For example: chat history.
• (Optional) Additional Steps - In case your application has additional intermediate steps they should be logged under the steps mechanism. See more details here.
• Full Prompt (str) - the full prompt that was sent to the LLM.
• Output (str) - the application's output.

Classification 🎯

Includes both multi-class as well as multi-label classification applications in which the LLM performs the classification.

Classification Key Properties

For classification applications Deepchecks' properties, are mainly useful for segmenting and analyzing different aspects of the user input. Specifically interesting properties for this include Sentiment, Fluency, Formality, and in relevant cases also Content Type. In classification applications, the main tools used for auto annotation are similarity-based annotation and Deepchecks Evaluator.

Data to upload for Classification Applications

• Input (str) - The input to be classified.
• (Optional) Additional Steps - In case your application has additional intermediate steps they should be logged under the steps mechanism. See more details here.
• (Optional) Full Prompt (str) - the full prompt that was sent to the LLM.
• Output (str) - the classification. In the case of multi-label classification should be formatted as Class 1, Class 2, ...