Deepchecks in AWS SageMaker

Deepchecks runs as a native Partner AI App inside your AWS account. Unlike SaaS, all LLM inference uses your own Bedrock models and API keys - Deepchecks does not manage or touch your models. Your data never leaves your AWS environment.

Key differences from SaaS:

• LLM inference runs through your Bedrock models (you configure and pay for them)
• You manage model capacity and rate limits
• CloudWatch integration is available for observability

Prerequisites

Before you begin:

• An active AWS account (setup guide)
• A Deepchecks subscription via SageMaker Partner AI Apps (free trials available)
• Administrator permissions to set up the Partner AI App and configure user access

Setup

Step 1: Access SageMaker

1. Log in to the AWS Management Console
2. Search for SageMaker in the top search bar
3. Navigate to Amazon SageMaker AI
4. Your organization's administrator should have pre-provisioned a SageMaker Studio domain - launch the Studio environment using the relevant domain and user profile

Follow the Partner AI Apps onboarding guide to configure administrative and user permissions.

Step 2: Open the Deepchecks App

In SageMaker Studio, locate the Deepchecks Partner AI App. A Deployed status means the app is running and ready.

📘

Note the Amazon Resource Name (ARN) and the SDK URL displayed when you open the app - you'll need these for SDK integration.

Step 3: Get Your API Key

Click Open Deepchecks LLM Evaluation to launch the application. Inside the app, generate and copy your Deepchecks API key alongside the ARN and SDK URL.

Step 4: Configure Your LLM Models

Since all LLM inference runs through your Bedrock models, configure the models Deepchecks should use before uploading data at scale.

→ See Working with LLM Features for configuration steps.

Step 5: Start Evaluating

Use the Example notebooks in the Deepchecks app page to get started quickly:

You can also launch JupyterLab within SageMaker and integrate Deepchecks using the Python SDK. Because Deepchecks runs as an AWS Partner AI App, SDK authentication uses AWS SigV4 signing instead of a plain API token - see Using the Python SDK on SageMaker for the full setup.

What's in this section

• Working with LLM Features - configure Bedrock models for Deepchecks features, understand model tiers and rate limits
• Using the Python SDK on SageMaker - SDK installation, SigV4 authentication, environment variables, and initialization examples
• Optimizing LLM Costs - control LLM spend through model selection, sampling, and property configuration
• Using the Owner Panel - manage users and workspace settings in the SageMaker deployment
• AWS CloudWatch Integration - forward Deepchecks metrics to CloudWatch for unified monitoring