deprecated
0.7.0 API Changes (breaking 0.6.0)
almost 2 years ago by Shay Tsadok
Summary of Changes
ext_interaction_idwas changed intouser_interaction_id- This id must be unique within a single version and can be set by the consumer (otherwise deepchecks will generate one for you). This is the only unique id of the interaction.- We've made some name changes -
user_inputis nowinput, andresponseis notoutput. Additional changes that affect SDK and REST API users are listed below.- We've added an option to set the annotation to
Noneand differentiated it fromUnknown.
None- Deepchecks will try to estimate the annotation.Unknown- the interaction input/output where reviewed, and the annotation for that interaction can't be defined as "Good" or "Bad". Deepchecks will not try to estimate an annotation for this sample.
CSV column name changes:
- column
user_input->input - column
response->output - column
ext_interaction_id->user_interaction_id
SDK Changes:
Tag.USER_INPUT->Tag.INPUTTag.EXT_INTERACTION_ID->Tag.USER_INTERACTION_IDStepType.INFORMATION_RETRIVAL->StepType.INFORMATION_RETRIEVALdc_client.log_interaction()API changed- From:
def log_interaction(self, user_input: str, model_response: str,..., ext_interaction_id: str = None, ...) - To:
def log_interaction(self, input: str, output: str,..., user_interaction_id: str = None, ...)
- From:
dc_client.annotate()API changed- From:
def annotate(self, ext_interaction_id: str, annotation: AnnotationType) - To:
def annotate(self, user_interaction_id: str, version_name: str, annotation: AnnotationType = None, reason: t.Optional[str] = None)
- From:
GoldenSetInteractionmember name changes:user_input->inputresponse->outputresponse_properties->output_propertiesext_interaction_id->user_interaction_ididproperty was dropped
REST API Main Changes:
POST /api/v1/annotations- Create new interaction schema changed- From
{ "ext_interaction_id": "string", //... "user_input": "string", "model_response": "string", "annotation": "good", //... } - To
{ "user_interaction_id": "string", //... "input": "string", "output": "string", "annotation": "good", "annotation_reason": "string", //... }
- From
GET /api/v1/interactions/{id}- Get single interaction URL changed to:GET /api/v1/application_versions/{app_version_id}/interactions/{user_interaction_id}PUT /api/v1/annotations- CRUD annotations schema changed- From
{ "interaction_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6", "value": "good" } - To
{ "user_interaction_id": "interaction_num_1", "application_version_id": 1, "value": "good", "reason": "user reviewed" }
- From