Documentation
DocumentationAPI ReferenceRelease Notes
DocumentationLog In
Documentation

Code Snippets: Full Examples

Ready-to-go end-to-end code snippets for common operations.

Suggest Edits

The code snippets on this page are intended for a straightforward RAG use case using mock data. For a more comprehensive demonstration, please refer to the Q&A Demo: GVHD Use Case or check out Data Upload for additional and advanced data uploading options.

📘

For more details about SDK functions and arguments check out the full Python SDK Reference

Creating an Application and Uploading an Evaluation Set

import pandas as pd
from deepchecks_llm_client.client import DeepchecksLLMClient
from deepchecks_llm_client.data_types import EnvType, LogInteractionType, ApplicationType

DEEPCHECKS_LLM_API_KEY = "YOUR API KEY"
APP_NAME = "DemoApp"

# download data and read as csv
df = pd.read_csv('https://figshare.com/ndownloader/files/49102981')

# Init SDK's client and create an application
dc_client = DeepchecksLLMClient(api_token=DEEPCHECKS_LLM_API_KEY)
dc_client.create_application(APP_NAME, ApplicationType.QA)

# Log a batch of 97 samples used as the evaluation set to the Deepchecks platform
interactions = [
    LogInteractionType(user_interaction_id=idx, input=row["input"], 
                       information_retrieval=row["information_retrieval"], output=row["output"],
                       custom_props={'Question Type': row['cp_Question Type']},
                       annotation="Good",  # If annotations are unknown, remove this argument.
                       ) for idx, row in df.iterrows()]
dc_client.log_batch_interactions(app_name=APP_NAME, version_name="Ground Truth", 
                                 env_type=EnvType.EVAL, interactions=interactions)

Evaluating a New Version

from datetime import datetime

# Retrieve the evaluation set inputs
evaluation_set = dc_client.get_data(
    app_name=APP_NAME, version_name="Ground Truth", env_type=EnvType.EVAL)

interactions = []
# Run v1 version pipeline on the evaluation set
for _, row in evaluation_set.iterrows():
    start_time = datetime.now().astimezone()
    rephrased_input = rephrase(row['input'])
    retrieved_documents = v1_retrieval_func(rephrased_input)
    output = v1_llm_pipeline(rephrased_input, retrieved_documents)
    
    interactions.append(LogInteractionType(
        user_interaction_id=str(row['user_interaction_id']), input=row['input'],
        information_retrieval=retrieved_documents, output=output,
        steps=[Step(name="Rephrase", input=row['input'], output=rephrased_input)],
        started_at=start_time, finished_at=datetime.now().astimezone()))

# Log the interactions into the Deepchecks' platform
dc_client.log_batch_interactions(
    app_name=APP_NAME, version_name="v1_gpt3.5",
    env_type=EnvType.EVAL, interactions=interactions)

Updated 8 days ago

What’s Next