For this evaluation, we will upload three distinct datasets to the Deepchecks platform:

• Evaluation Dataset: This serves as our ground truth data for model assessment.
• Predictions on Evaluation Data: The predictions made by our candidate pipeline, based on GPT-4o, on the evaluation dataset.
• Predictions on Production Data: The predictions made by our candidate pipeline, based on GPT-4o, on the more recent production dataset.

By uploading these three datasets, we create a comprehensive framework for assessing the model's performance on both historical (labeled) data and more recent (unlabeled) production data. Uploading data to the Deepchecks system can be done either via the SDK or via the UI. Both options are explained below.

Use Deepchecks' Python SDK (Option 1)

📘

Open the Demo Notebook via Colab or Download it Locally

Click the badge below to open the Google Colab or click here to download the Notebook, set in your API token (see below), and you're ready to go!

Get your API key from the user menu in the app:

If running locally, we recommend the best practice of using a python virtual environment to install the Deepchecks client SDK.

Or: Use the Deepchecks' UI (Option 2)

📘

Click here to Download the Demo Data

You'll see there the three demo datasets used in this example

1. Click the Upload Data button on the sidebar

   

2. Create a New Application, name it "Movie Genre Classification", Application Type is "Classification", and a New Version named "evaluation set".

   

3. Upload the evaluation_set.csv to the Evaluation environment.

4. Create Custom Properties for Vote Count, Vote Average, and Label.

5. Add a New Version, and name it gpt_4o. Upload the second CSV (gpt_4o_evlaution_set.csv) to the Evaluation environment of the new version.

6. Proceed to upload the gpt_4o_prod CSV to the Production environment of the second version.

👍

Success, the Movie Genre Application is now in the Deepchecks App

• Some properties take a few minutes to calculate, so some of the data - such as properties and estimated annotations will be updated over time.
• You'll see a ✅ Completed Processing Status in the Applications page, when processing is finished.