Jump to Content
DeepchecksDocumentation
DocumentationAPI ReferenceRelease Notes
DocumentationLog InDeepchecks
Documentation
Log In
DocumentationAPI ReferenceRelease Notes

Welcome

  • Welcome to Deepchecks LLM Evaluation
  • Getting Started with Deepchecks!

Deepchecks in Action

  • Q&A Demo: GVHD Data
    • Uploading the Data
    • Identify Problems Using Properties, Estimated Annotations and Insights
    • User-Value Properties and Prompt Properties
    • Compare Between Versions
    • Monitor Production Data and Research Degradation
  • Summarization Demo: E-Commerce Data
    • Uploading the Data
    • Configuring the Automatic Annotation
    • Compare Between Versions
    • Production Monitoring
  • Classification Demo: Movie Genre
    • Uploading the Data
    • Evaluation Set Analysis
    • Production Monitoring

User Guide

  • Deepchecks' SDK
    • Setup: Python SDK Installation & API Key Retrieval
    • Main SDK Classes
    • Data Upload
    • Data Download
    • Code Snippets: Full Examples
  • Hierarchy & Data Upload Format
  • Supported Use Cases
  • Agent Evaluation Use-Case
  • Features
    • Automatic Annotations
      • Customizing the Auto Annotation Configuration
    • Version Comparison
    • Root Cause Analysis (RCA)
    • Production Monitoring
    • Experiment Management
    • Additional Features
  • Properties
    • Built-in Properties
      • Retrieval Use-Case Properties
    • Prompt Properties
    • User-Value Properties
  • Deepchecks' UI
  • Langchain Tracing
  • Tracing on the Deepchecks System

USAGE Scenarios

  • Evaluation Dataset Management
    • Sending Evaluation Data via Deepchecks SDK
    • Generating an Initial Evaluation Set (RAG Use Cases Only)
    • Uploading an Existing Evaluation Set via Drag & Drop UI
    • Excluding Undesired Interactions from the Evaluation Set
    • Cloning Interactions from Production into the Evaluation Set
  • Version Comparison
  • AI-Assisted Annotations
  • Hard Sample Mining for Fine-Tuning
  • Pentesting Your LLM-Based App
  • Configuring Nvidia's Guardrails with Deepchecks

Integrations

  • Deepchecks in AWS SageMaker
  • LLMs
    • OpenAI
    • Azure OpenAI
    • Vertex AI
    • Anthropic
    • Nvidia NIM
    • Oracle Cloud (OCI)
    • AWS Bedrock
  • Production Monitoring
    • Datadog Integration
    • New Relic Integration
Powered by 

LLMs

Suggest Edits

This section describes integration to various common LLM models:

OpenAI

Azure OpenAI

Vertex AI

Anthropic

Nvidia NIM

Oracle Cloud (OCI)

AWS Bedrock

Updated about 1 year ago