DocumentationAPI ReferenceRelease Notes
DocumentationLog In
Release Notes
These docs are for v0.8.0. Click to read the latest docs for v0.21.0.
deprecated

0.7.0 API Changes (breaking 0.6.0)

🚧

Summary of Changes

  1. ext_interaction_id was changed into user_interaction_id - This id must be unique within a single version and can be set by the consumer (otherwise deepchecks will generate one for you). This is the only unique id of the interaction.
  2. We've made some name changes - user_input is now input, and response is not output. Additional changes that affect SDK and REST API users are listed below.
  3. We've added an option to set the annotation to None and differentiated it from Unknown.
    1. None - Deepchecks will try to estimate the annotation.
    2. Unknown- the interaction input/output where reviewed, and the annotation for that interaction can't be defined as "Good" or "Bad". Deepchecks will not try to estimate an annotation for this sample.

CSV column name changes:

  1. columnuser_input -> input
  2. columnresponse -> output
  3. columnext_interaction_id -> user_interaction_id

SDK Changes:

  1. Tag.USER_INPUT -> Tag.INPUT
  2. Tag.EXT_INTERACTION_ID -> Tag.USER_INTERACTION_ID
  3. StepType.INFORMATION_RETRIVAL -> StepType.INFORMATION_RETRIEVAL
  4. dc_client.log_interaction() API changed
    1. From:
      def log_interaction(self, user_input: str, model_response: str,..., 
                          ext_interaction_id: str = None, ...)
      
    2. To:
      def log_interaction(self, input: str, output: str,..., 
                          user_interaction_id: str = None, ...)
      
  5. dc_client.annotate() API changed
    1. From:
      def annotate(self, ext_interaction_id: str, annotation: AnnotationType)
      
    2. To:
      def annotate(self, user_interaction_id: str, version_name: str,  
                   annotation: AnnotationType = None, 
                   reason: t.Optional[str] = None)
      
  6. GoldenSetInteraction member name changes:
    1. user_input -> input
    2. response -> output
    3. response_properties -> output_properties
    4. ext_interaction_id -> user_interaction_id
    5. idproperty was dropped

REST API Main Changes:

  1. POST /api/v1/annotations - Create new interaction schema changed
    1. From
      {
        "ext_interaction_id": "string",
        //...
        "user_input": "string",
        "model_response": "string",
        "annotation": "good",
        //...
      }
      
    2. To
      {
        "user_interaction_id": "string",
        //...
        "input": "string",
        "output": "string",
        "annotation": "good",
        "annotation_reason": "string",
        //...
      }
      
  2. GET /api/v1/interactions/{id} - Get single interaction URL changed to: GET /api/v1/application_versions/{app_version_id}/interactions/{user_interaction_id}
  3. PUT /api/v1/annotations- CRUD annotations schema changed
    1. From
      {
        "interaction_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
        "value": "good"
      }
      
    2. To
      {
        "user_interaction_id": "interaction_num_1",
        "application_version_id": 1,
        "value": "good",
        "reason": "user reviewed"
      }