Main SDK Classes

Deepchecks Client

The DeepchecksLLMClient instance serves as a gateway to the Deepchecks platform. All SDK operations will be conducted through it. The silent_mode argument controls the behavior in case something goes wrong. It is recommended to set it to True when using it in a production environment.

from deepchecks_llm_client import DeepchecksLLMClient

dc_client = DeepchecksLLMClient(
    api_token="Fill API Key Here",
    silent_mode=True
)

LogInteraction

An Interaction represents an interaction in the Deepchecks platform and is the primary object for logging data. For additional information about the Deepchecks data structure, see Hierarchy & Data Upload Format.

from deepchecks_llm_client.data_types import LogInteraction

single_sample = LogInteraction(
    user_interaction_id="id-1",
    input="my user input1",
    output="my model output1",
    started_at="2024-09-01T23:59:59",
    finished_at=datetime.now().astimezone(),
    annotation="Good",  # Either Good, Bad, Unkwown(annotator was unsure), or None
    interaction_type="Generation",  # Optional. The Type of interaction. If not provided, the interaction type will default to the application’s default type. 
    session_id="session-1",  # Optional. Groups related interactions; auto-generated if not provided.
)

AnnotationType

In addition to the estimated annotation calculated by Deepchecks, users can (and are encouraged to) supply their own annotations when available. This can be done via the annotation argument which accepts either the AnnotationType enum or string values. Possible annotations are "Good", "Bad" and "Unkown" which should be used only in cases where a human annotator saw the interaction and could not decide on its nature.