Quality evaluation
With the “Quality evaluation” feature, you can evaluate the quality of an agent's responses — both manually and with artificial intelligence.
Important! The “Quality evaluation” feature is available on the “Expert + AI” plan — to enable it, email us at support@usedesk.com. The feature works only in the new Usedesk design.
Important! Access to the “Quality evaluation” section is only available to agents who belong to a group that has the rights to access the “Quality control” section.
To start the evaluation, click the “Quality evaluation” button.
The system will expand the quality evaluation block.
You can manually give a rating for the “Product knowledge” criterion, since AI can't yet evaluate an agent's product knowledge. Click the bar in the corresponding position to give the relevant rating: “Poor”, “Fair”, or “Good”.
To evaluate the quality of the agent's responses with AI, click “Generate evaluation”. An AI evaluation can be requested only once per ticket.
The system will start analyzing the conversation. Depending on the number of messages in the conversation and their length, the analysis can take from a few seconds to a couple of minutes.
If you collapse the quality evaluation window, you can track the evaluation's readiness status using the icon.
When it's ready, the average conversation score as a percentage will appear in the “Quality evaluation” block. The score is calculated only from the AI criteria — the manual “Product knowledge” rating is not taken into account.
If you haven't viewed the detailed quality evaluation yet, a purple dot will be shown next to it.
Clicking the “Quality evaluation” block expands the detailed evaluation across six criteria:
- greeting — the presence and appropriateness of a greeting at the start of the response;
- closing — the correctness of ending the dialog, a polite farewell;
- tone and empathy — friendliness, a respectful tone, and showing empathy;
- response structure — the format of the response, correct writing, and how easy the message is to follow;
- understanding the question — whether the agent interpreted the ticket correctly;
- providing the answer — the completeness, accuracy, and relevance of the answer.
To see a comment for each criterion, click the arrow.
Below the quality evaluation block, key insights and recommendations are shown
How the overall AI score is calculated
- A “Good” (2) rating gives 100% for the criterion,
- A “Fair” (1) rating gives 70% for the criterion,
- A “Poor” (0) rating gives 0% for the criterion.
Each AI automatic evaluation criterion is weighted at 10 or 20 units, and the total weight of all criteria is 100.
The manual “Product knowledge” rating is not included in the “Overall AI score” calculation.
Formula for calculating the ticket's final score as a percentage:
Final ticket score = Criterion_weight_1 × AI_score_1 + Criterion_weight_2 × AI_score_2 + … + Criterion_weight_6 × AI_score_6
Overall AI score = 90% = (10 × 70% + 10 × 100% + 20 × 100% + 20 × 70% + 20 × 100% + 20 × 100%)/100
|
Criterion |
Criterion's weight in the overall score |
AI score |
|
Greeting |
10 |
70% (fair) |
|
Farewell |
10 |
100% (good) |
|
Tone and Empathy |
20 |
100% (good) |
|
Response structure |
20 |
70% (fair) |
|
Understanding the question |
20 |
100% (good) |
|
Providing the answer |
20 |
100% (good) |
|
Total weight of criteria |
100 |
Overall AI score = 90% |