Automatically Log Traces to Deepchecks

Deepchecks now supports logging traces and spans directly via OpenTelemetry and OpenInference.

This means that if you’re using an agentic framework that already provides OpenInference instrumentors (such as CrewAI, LangGraph or others), you can automatically log traces and spans to Deepchecks with minimal setup.

This integration makes it easy to capture rich trace data from your agents and pipelines and centralize them in Deepchecks for evaluation, monitoring and debugging.

Installation

First, install the required dependency:

pip install "deepchecks-llm-client[otel]"

Usage Example (CrewAI)

Here’s an example of setting up OpenTelemetry tracing with Deepchecks using CrewAI:

from deepchecks_llm_client.data_types import EnvType
from deepchecks_llm_client.otel import CrewaiIntegration

# Register the Deepchecks exporter
CrewaiIntegration().register_dc_exporter(
    host="https://app.llm.deepchecks.com/",   # Deepchecks endpoint
    api_key="Your Deepchecks API Key",        # API key from your Deepchecks workspace
    app_name="Your App Name",                 # Application name in Deepchecks
    version_name="Your Version Name",         # Version name for this run
    env_type=EnvType.EVAL,                    # Environment: EVAL, PROD, etc.
    log_to_console=True,                      # Optional: also log spans to console
)

How It Works

• OpenTelemetry standardizes trace collection and handling.
• OpenInference provides pre-built instrumentors for popular frameworks, so spans are captured automatically.
• Deepchecks exporter (register_dc_exporter) sends all spans to your Deepchecks workspace.

This allows you to see and evaluate detailed traces and spans, attributes, system metrics and framework events inside the Deepchecks platform, without needing to manually log them.

Add session ID and tags to your traces

Traces can be grouped into sessions and categorized using tags to help with analysis, filtering, and understanding the context of spans.

Session ID

A session ID groups multiple traces into a single session. This is useful in multi-step workflows, such as chat use cases, where you want to see all spans belonging to one logical interaction.

Use the using_session method to set a session ID. Here is an example:

from openinference.instrumentation import using_session

with using_session(session_id="my-session-id"):  
    # Calls within this block will generate spans with the attribute:  
    # "session.id" = "my-session-id"  
    result = crew_instance.kickoff(inputs={  
        "customer_domain": customer_domain,  
        "project_description": project_description,  
    })

Here's how this example will be logged to the Deepchecks platform:

Tags

Tags let you categorize traces or individual spans. For example, you could tag spans by feature, category, or team.

Use the using_tags method to assign tags to all spans within a block. Here is an example:

from openinference.instrumentation import using_tags

tags = ["Football", "Soccer"]
with using_tags(tags):
    result = crew_instance.kickoff(inputs={
        "customer_domain": customer_domain,
        "project_description": project_description,
    })

Here's how this example will be logged to the Deepchecks platform:

You can also add tags to a single span without affecting the rest of the trace. This is useful for highlighting specific operations without categorizing the entire session.