Measuring chatbot outcomes
Important
Power Virtual Agents capabilities and features are now part of Microsoft Copilot Studio following significant investments in generative AI and enhanced integrations across Microsoft Copilot.
Some articles and screenshots may refer to Power Virtual Agents while we update documentation and training content.
Tracking conversation engagement and outcomes is crucial to measure the chatbot performance metrics and spot areas for improvements in the analytics dashboard.
Important
A Microsoft Copilot Studio conversation between an end-user and a chatbot generates one or multiple analytics sessions. All analytics sessions are either Engaged or Unengaged.
Understanding analytics sessions outcomes
Analytics sessions each have an outcome:
- Unengaged sessions always have an outcome of None.
- Engaged sessions have one of the following outcomes:
- Resolved,
- Escalated,
- Abandoned.
Engagement and outcomes
Conversation analytics sessions, their engagement status, and their outcomes are available as part of the analytics dashboard or in the ConversationTranscript
table in Dataverse.
Engagement | Outcome | Definition |
---|---|---|
Unengaged | None (None ) |
All unengaged sessions have an outcome of None. |
Engaged | Escalated (HandOff ) |
A session is considered escalated when the Escalate topic is triggered, or the Transfer Conversation node is reached. |
Engaged | Resolved (Resolved ) |
A session is considered resolved when the End of Conversation topic is triggered and the user confirms that the interaction was a success (impliedSuccess = FALSE ) or doesn't answer and lets the session time out (impliedSuccess = TRUE ). |
Engaged | Abandoned (Abandoned ) |
A session is considered abandoned when an engaged session ends and didn't reach a resolved or escalated state. |
Properly ending conversations to measure outcomes
It’s important to end conversation with the End of Conversation topic, so that the end-user can confirm their issue was resolved or not (and potentially escalate).
When the bot confirms success, a CSAT survey is displayed to the end-user to capture a 0-5 score.
If you’re certain of the outcome of a conversation based on the topic logic, you can directly flag it as a confirmed success or redirect to the Escalate topic.
Tip
In the unified authoring canvas, in the code editor, you can set the conversationOutcome
within an action. For example conversationOutcome: ResolvedConfirmed
for confirmed success or conversationOutcome: ResolvedImplied
for implied success.
Feedback
https://aka.ms/ContentUserFeedback.
Coming soon: Throughout 2024 we will be phasing out GitHub Issues as the feedback mechanism for content and replacing it with a new feedback system. For more information see:Submit and view feedback for