Deepchecks Self-Hosted Enterprise runs entirely in your own infrastructure, giving you full control over networking, security, and scaling. The platform is designed to run on Kubernetes to ensure reliable performance and horizontal scalability in production environments. This documentation walks through the infrastructure components and configuration required to deploy Deepchecks successfully.

Deployment Overview

The high-level steps for deploying Deepchecks differ slightly between cloud and self-hosted environments. Use the tabs below to see the deployment journey for your target environment; each step links to the section with full instructions.

Infrastructure Prerequisites

The following diagram illustrates a typical Deepchecks deployment running on AWS:

Prior to deploying Self-Hosted Enterprise, Deepchecks recommends having each of the following infrastructure components ready to go. Select your deployment environment below for provider-specific guidance.

Minimum Resource Requirements

The following table summarizes the default CPU and memory requests for each Deepchecks component. These represent the minimum resources your cluster must be able to schedule.

📘

Node Storage

We recommend that each node has a minimum of 150 GB of available storage.

📘

GPU Worker

The gpu-runner worker must run on a dedicated GPU-powered node and is not included in the CPU/memory totals above. Plan for an additional node with at least 4 vCPU, 16 GiB RAM, and one NVIDIA T4 (or equivalent) GPU. See the Infrastructure Prerequisites section for recommended GPU instance types.

All resource values can be customized in your values.yaml. See the Example Configurations section for details.

Queue Configuration

Overview

Deepchecks uses message queues to coordinate background processing tasks. All queues are created with a configurable prefix that must match the TENANT_NAME environment variable provided during Helm chart deployment. See Environment Variables for further details.

variable "queue_prefix" {
  description = "Prefix for queue names (must match TENANT_NAME)"
  type        = string
  default     = "deepchecks"
}

locals {
  queues = {
    insights-calculator = {
      visibility_timeout = 360
      max_receive_count  = 3
    }
    garak-props-calculator = {
      visibility_timeout = 360
      max_receive_count  = 3
    }
    props-calc-batcher = {
      visibility_timeout = 360
      max_receive_count  = 3
    }
    translation = {
      visibility_timeout = 660
      max_receive_count  = 3
    }
    pre-calc-eng = {
      visibility_timeout = 660
      max_receive_count  = 3
    }
    advanced-llm-prop-calculator = {
      visibility_timeout = 360
      max_receive_count  = 3
    }
    calibrator = {
      visibility_timeout = 360
      max_receive_count  = 3
    }
    notifier = {
      visibility_timeout = 360
      max_receive_count  = 3
    }
    proba-calculator = {
      visibility_timeout = 360
      max_receive_count  = 3
    }
    llm-properties = {
      visibility_timeout = 660
      max_receive_count  = 3
    }
    topics-inference = {
      visibility_timeout = 660
      max_receive_count  = 3
    }
    topics-train = {
      visibility_timeout = 660
      max_receive_count  = 3
    }
    similarity-annotations = {
      visibility_timeout = 660
      max_receive_count  = 3
    }
    properties-calculator = {
      visibility_timeout = 660
      max_receive_count  = 3
    }
    estimate-annotation-calculator = {
      visibility_timeout = 360
      max_receive_count  = 3
    }
    general-calc = {
      visibility_timeout = 660
      max_receive_count  = 3
    }
  }
}

# Dead Letter Queues
resource "aws_sqs_queue" "dlq" {
  for_each = local.queues

  name                       = "${var.queue_prefix}-${each.key}-dlq"
  visibility_timeout_seconds = each.value.visibility_timeout
}

# Main Queues
resource "aws_sqs_queue" "queue" {
  for_each = local.queues

  name                       = "${var.queue_prefix}-${each.key}"
  visibility_timeout_seconds = each.value.visibility_timeout

  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.dlq[each.key].arn
    maxReceiveCount     = each.value.max_receive_count
  })
}

Object Storage

Deepchecks requires access to object storage for storing ML models and application data.

Authentication

End users authenticate to Deepchecks through an external OAuth 2.0 / OIDC identity provider. Select the provider by setting OAUTH.PROVIDER and supply its connection details via the OAUTH.CLIENT_ID, OAUTH.CLIENT_SECRET, OAUTH.SERVER_URL, and OAUTH.TENANT_URL environment variables (see Environment Variables for the full list).

Licensing

Deepchecks deployments are activated using a license key issued by Deepchecks. Two activation modes are supported.

⚠️

Mutually Exclusive

Set either LICENSE_KEY or PRODUCT_KEY — never both. If both are set, LICENSE_KEY takes precedence.

Online activation

Used by all cloud deployments and any on-premises cluster with internet egress.

1. Receive your LICENSE_KEY from Deepchecks support.
2. Add LICENSE_KEY to the Kubernetes Secret referenced by global.secretName.

The application validates the key against the Deepchecks licensing server periodically.

Offline (air-gapped) activation

For clusters with no outbound access to the Deepchecks licensing server, a PRODUCT_KEY is issued against the UUID of the kube-system namespace, binding it to that specific cluster. The Kubernetes cluster must be provisioned before a PRODUCT_KEY can be issued.

1. Retrieve your cluster UUID. Run the following against your target cluster:

   Bash

   kubectl get namespace kube-system -o jsonpath='{.metadata.uid}'

   The UUID is generated when the cluster is created and persists across upgrades and node replacements.

2. Submit the cluster UUID along with your customer details to Deepchecks support.

3. Receive your PRODUCT_KEY from Deepchecks.

4. Provision external dependencies. PostgreSQL, Redis/Valkey, RabbitMQ, and S3-compatible object storage must be running and reachable from the cluster before deploying the Helm chart.

5. Configure the deployment. Add PRODUCT_KEY to the Kubernetes Secret referenced by global.secretName and ensure LICENSE_KEY is not set.

Kubernetes Configuration

Tooling

We require you to install and configure the following Kubernetes tooling:

1. Install helm by following these instructions
2. Install kubectl by following these instructions
3. Configure kubectl to connect to your cluster:

Create a Kubernetes namespace for your Deepchecks deployment (this can also be done as part of the Helm deployment):

kubectl create namespace deepchecks

Configuring Kubernetes Secrets

Sensitive credentials must be available in a Kubernetes Secret during deployment. The secret must be referenced in your values.yaml file at global.secretName. Ensure all required secrets are configured before deploying. See the Environment Variables section for the full list.

📘

External Secrets Operator

While creating a Kubernetes Secret manually will work, we recommend using the External Secrets Operator with your secrets manager of choice (e.g., AWS Secrets Manager, Azure Key Vault, Google Secret Manager) to securely create this secret on your behalf.

kubectl create secret generic deepchecks-secrets \
  --namespace deepchecks \
  --from-literal=DATABASE.URI='postgresql://user:pass@host:5432/dbname' \
  --from-literal=OAUTH.CLIENT_SECRET='your-oauth-secret' \
  --from-literal=WEBAPP.AUTH_JWT_SECRET='your-jwt-secret' \
  --from-literal=LICENSE_KEY='your-license-key'

--from-literal=STORAGE_ACCESS_KEY_ID='your-storage-access-key' \
--from-literal=STORAGE_SECRET_ACCESS_KEY='your-storage-secret-key'

GPU Nodes

In order for Kubernetes to utilize GPU nodes, the NVIDIA GPU Operator must be installed on your cluster. The installation process is the same across all supported Kubernetes distributions (EKS, AKS, GKE, and on-premises).

nodeSelector:
  role: gpu
affinity: null

tolerations:
  - key: CriticalAddonsOnly
    operator: Exists
  - key: nvidia.com/gpu
    operator: Exists

Application Permissions

The Deepchecks application requires access to cloud provider APIs for object storage and message queue operations.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:Get*",
        "s3:List*",
        "s3:Put*",
        "s3:Delete*"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::<BUCKET_NAME>",
        "arn:aws:s3:::<BUCKET_NAME>/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "sqs:SendMessage",
        "sqs:ReceiveMessage",
        "sqs:DeleteMessage",
        "sqs:GetQueueAttributes",
        "sqs:GetQueueUrl"
      ],
      "Resource": "arn:aws:sqs:<REGION>:<ACCOUNT_ID>:<TENANT_NAME>*"
    },
    {
      "Action": [
        "bedrock:InvokeModel"
      ],
      "Effect": "Allow",
      "Resource": "*"
    }
  ]
}

Mirroring Artifacts for Air-Gapped Clusters

📘

On-Premises Only

This section applies only to self-hosted clusters that cannot reach registry.cdn.deepchecks.com. If your cluster has outbound network access, you can skip this section and follow Registry Authentication instead.

Air-gapped deployments must mirror two artifacts to an internal registry before installing:

• The application image (registry.cdn.deepchecks.com/deepchecks/llm), which contains the web server and all background workers.
• The Helm chart (oci://registry.cdn.deepchecks.com/deepchecks/llm-stack), which orchestrates the deployment.

📘

Versions for This Documentation Version

Use application image tag 0.44.0 and Helm chart version 2.2.1 (or later) with this version of the documentation. The application image follows semantic versioning (<MAJOR>.<MINOR>.<PATCH>) and matches the application version in the documentation. The chart and application are versioned independently — see the Helm Chart Reference for the compatibility matrix.

Application Image

1. Pull the image from the Deepchecks registry on a host with outbound network access:

   Bash

   docker login registry.cdn.deepchecks.com
   docker pull registry.cdn.deepchecks.com/deepchecks/llm:0.44.0

2. Re-tag and push the image into your internal registry:

   Bash

   docker tag registry.cdn.deepchecks.com/deepchecks/llm:0.44.0 <INTERNAL_REGISTRY>/deepchecks/llm:0.44.0
   docker push <INTERNAL_REGISTRY>/deepchecks/llm:0.44.0

Helm Chart

1. Pull the chart from the Deepchecks OCI registry on a host with outbound network access. This produces a .tgz archive in the current directory:

   Bash

   helm registry login registry.cdn.deepchecks.com
   helm pull oci://registry.cdn.deepchecks.com/deepchecks/llm-stack --version 2.2.1
   # Produces llm-stack-2.2.1.tgz

2. Push the chart into your internal OCI registry:

   Bash

   helm registry login <INTERNAL_REGISTRY>
   helm push llm-stack-2.2.1.tgz oci://<INTERNAL_REGISTRY>/deepchecks

   Alternatively, transfer the .tgz file directly to your air-gapped environment and install it from disk (helm install deepchecks ./llm-stack-2.2.1.tgz ...) without pushing to a registry.

Configure values.yaml

After mirroring, point the chart at your mirror and disable the model sync job:

1. Override the image repository so the chart pulls the application image from your mirror:

   YAML

   global:
     image:
       repository: <INTERNAL_REGISTRY>/deepchecks

2. Disable the model sync job. It downloads model artefacts from the public internet and is not used in air-gapped deployments:

   YAML

   modelsSync:
     enabled: false

   See Disabling Model Sync (Air-Gapped Deployments) for details on pre-loading the storage bucket.

When you reach the Installation step, install from your mirrored chart instead of the public registry — for example helm install deepchecks oci://<INTERNAL_REGISTRY>/deepchecks/llm-stack --version 2.2.1 --values values.yaml.

Deploying the Deepchecks Helm Chart

Deepchecks is distributed as a Helm chart named llm-stack, which packages the llm application image (web server and all background workers) along with the cluster resources required to run it. The chart is hosted as an OCI artifact at oci://registry.cdn.deepchecks.com/deepchecks/llm-stack; the application image is hosted at registry.cdn.deepchecks.com/deepchecks/llm.

The chart is versioned independently from the application. See the Helm Chart Reference for the compatibility matrix between chart and application versions, upgrade notes, and the values schema.

Registry Authentication

In order to access the Helm chart and container images you will need your credentials from the Deepchecks team.

helm registry login registry.cdn.deepchecks.com

Image Pull Secret

Container images are pulled from a private registry, so the deployment namespace needs an image pull secret referenced from global.imagePullSecrets. Create it with kubectl:

kubectl create secret docker-registry deepchecks-registry \
  --namespace deepchecks \
  --docker-server=registry.cdn.deepchecks.com \
  --docker-username=<USERNAME> \
  --docker-password=<LICENSE_KEY>

Then reference it in values.yaml:

global:
  imagePullSecrets:
    - name: deepchecks-registry

See the Kubernetes documentation for further details.

Installation

helm install deepchecks oci://registry.cdn.deepchecks.com/deepchecks/llm-stack \
  --values values.yaml \
  --namespace deepchecks \
  --create-namespace

⏳

Deployment Timeout

As part of the deployment, a Kubernetes job will run to populate the object storage bucket with required model data. This job requires internet access to pull model artifacts. Typically this takes approximately 2 minutes, but depends on network performance. If your Helm commands are timing out, add --timeout 20m0s.

⚠️

Sync Job

As part of the Helm deployment each Deployment has an initContainer that checks that a 'sync' Job has completed successfully. If the Job is manually deleted, new pods will fail to start. Setting modelsSync.enabled: false removes both the Job and the initContainer check, so air-gapped deployments are unaffected by this constraint.

Disabling Model Sync (Air-Gapped Deployments)

For air-gapped deployments where internet access is unavailable or model syncing is not needed, you can disable the model sync job by setting modelsSync.enabled to false in your values.yaml:

modelsSync:
  enabled: false

⚠️

Pre-load Models

When model sync is disabled, models must be pre-loaded into the storage bucket before deployment. The application will not attempt to download or synchronize model artifacts at startup. Instructions for obtaining the model artifacts will be provided by Deepchecks at deployment time.

DNS Configuration

Configure a DNS record for the FQDN you set in WEBAPP.DEPLOYMENT_URL (for example, deepchecks.example.com) pointing to your ingress controller's external address:

• AWS: a CNAME record pointing to the AWS Load Balancer Controller's provisioned ALB hostname.
• Azure: an A record pointing to the Kong (or other) ingress controller's external IP.
• GCP: an A record pointing to the Google Cloud Application Load Balancer's external IP provisioned by the GKE Ingress Controller.
• On-Premises: an A or CNAME record pointing to your ingress controller's external address.

The application is unreachable until DNS resolves to the ingress endpoint.

KEDA Autoscaling (Optional)

kedaAutoscaling:
  enabled: true
  provider: "aws"
  aws:
    region: "<AWS_REGION>"
    queuePrefix: "<TENANT_NAME>"
    accountId: "<AWS_ACCOUNT_ID>"
    authenticationRef:
      name: "aws-credentials"
      kind: ClusterTriggerAuthentication

kedaTriggerAuthentication:
  enabled: true
  items:
    - name: aws-credentials
      scope: cluster
      podIdentity:
        provider: aws

Environment Variables

Non-sensitive values can be passed directly via the Helm chart's global.env configuration. For sensitive values, create a Kubernetes Secret and reference it via global.secretName.

Example Configurations

The examples below show provider-specific overrides on top of the chart's defaults. To inspect the full default chart values, run:

helm show values oci://registry.cdn.deepchecks.com/deepchecks/llm-stack --version <CHART_VERSION>

For details on the chart's structure and the most commonly customised keys, see the Helm Chart Reference.

serviceAccount:
  annotations:
    eks.amazonaws.com/role-arn: arn:aws:iam::123456789012:role/DeepchecksRole

global:
  image:
    tag: "0.44.0"
  imagePullSecrets:
    - name: deepchecks-registry
  secretName: "deepchecks-secrets"
  env:
    CREATE_ORG_FOR_USER: "[email protected]"
    TENANT_NAME: "example"
    GENERAL.MODELS_BUCKET_NAME: "deepchecks-models"
    LOGGER.LEVEL: "INFO"
    REDIS.HOST: "host.redis.example.com"
    REDIS.PORT: "6379"
    LOGGER.FORMATTER: "TEXT"
    WEBAPP.DEPLOYMENT_URL: "https://deepchecks.example.com"
    OAUTH.PROVIDER: "auth0"
    OAUTH.SERVER_URL: "https://example.auth0.com"
    OAUTH.TENANT_URL: "https://example.auth0.com"
    OAUTH.CLIENT_ID: "your-client-id"

web:
  ingress:
    enabled: true
    className: "alb"
    annotations:
      alb.ingress.kubernetes.io/scheme: internet-facing
    host: deepchecks.example.com
    tls:
      - hosts:
          - deepchecks.example.com
        secretName: deepchecks-tls

workers:
  gpu-runner:
    nodeSelector:
      role: gpu
    tolerations:
      - effect: NoSchedule
        key: nvidia.com/gpu
        operator: Exists