deprecated
0.7.0 API Changes (breaking 0.6.0)
about 1 year ago by Shay Tsadok
Summary of Changes
ext_interaction_id
was changed intouser_interaction_id
- This id must be unique within a single version and can be set by the consumer (otherwise deepchecks will generate one for you). This is the only unique id of the interaction.- We've made some name changes -
user_input
is nowinput
, andresponse
is notoutput
. Additional changes that affect SDK and REST API users are listed below.- We've added an option to set the annotation to
None
and differentiated it fromUnknown
.
None
- Deepchecks will try to estimate the annotation.Unknown
- the interaction input/output where reviewed, and the annotation for that interaction can't be defined as "Good" or "Bad". Deepchecks will not try to estimate an annotation for this sample.
CSV column name changes:
- column
user_input
->input
- column
response
->output
- column
ext_interaction_id
->user_interaction_id
SDK Changes:
Tag.USER_INPUT
->Tag.INPUT
Tag.EXT_INTERACTION_ID
->Tag.USER_INTERACTION_ID
StepType.INFORMATION_RETRIVAL
->StepType.INFORMATION_RETRIEVAL
dc_client.log_interaction()
API changed- From:
def log_interaction(self, user_input: str, model_response: str,..., ext_interaction_id: str = None, ...)
- To:
def log_interaction(self, input: str, output: str,..., user_interaction_id: str = None, ...)
- From:
dc_client.annotate()
API changed- From:
def annotate(self, ext_interaction_id: str, annotation: AnnotationType)
- To:
def annotate(self, user_interaction_id: str, version_name: str, annotation: AnnotationType = None, reason: t.Optional[str] = None)
- From:
GoldenSetInteraction
member name changes:user_input
->input
response
->output
response_properties
->output_properties
ext_interaction_id
->user_interaction_id
id
property was dropped
REST API Main Changes:
POST /api/v1/annotations
- Create new interaction schema changed- From
{ "ext_interaction_id": "string", //... "user_input": "string", "model_response": "string", "annotation": "good", //... }
- To
{ "user_interaction_id": "string", //... "input": "string", "output": "string", "annotation": "good", "annotation_reason": "string", //... }
- From
GET /api/v1/interactions/{id}
- Get single interaction URL changed to:GET /api/v1/application_versions/{app_version_id}/interactions/{user_interaction_id}
PUT /api/v1/annotations
- CRUD annotations schema changed- From
{ "interaction_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "value": "good" }
- To
{ "user_interaction_id": "interaction_num_1", "application_version_id": 1, "value": "good", "reason": "user reviewed" }
- From