Chatbots analytics
EARLY ACCESSChatbot analytics is the central hub for chatbot performance monitoring and reporting. Track chatbot effectiveness, dialog flow performance, user engagement, and conversation outcomes with real-time updates and comprehensive analytics.
Chatbot analytics consists of two main categories:
- Predefined dashboards - Access standard chatbot performance metrics and KPIs based on fundamental chatbot operations. Monitor conversation flows, user interactions, success rates, and engagement metrics.
- Custom dashboards - Build personalized analytics views to track specific aspects of your chatbot operations. Create custom reports and visualizations that focus on the KPIs that matter most to your business.
Predefined dashboards
Predefined dashboards provide standard chatbot performance metrics and analytics based on industry best practices.
At the top of the page, you can switch between the following tabs:
- Overview
- Sessions
- Messages
- Users
- Dialogs and intents
- Voice of the customer
Overview
The Overview dashboard provides a high-level view of the performance of all your chatbots. It shows fundamental metrics so you can identify whether your chatbots are performing as expected.
By default, the analysis is set to a 30-day period. You can adjust the filters as required.
The Overview report is organized into four main categories:
Summary
The Summary section displays key performance metrics:
| Metric | Description |
|---|---|
| Total sessions | The total number of chat sessions that were initiated by end users. This number includes chats that were timed out or abandoned without any message from the end user. |
| Engaged session rate | An engaged session is one in which an end user initiated the session and then sent a minimum of 1 message to the chatbot. The engaged session rate is the number of engaged sessions as a percentage of the total sessions. Example: If the total sessions is 35 and number of engaged sessions is 28, the engaged session rate is 80%. |
| Average sessions per user | The average number of sessions individual end users initiated. The value is the average number of sessions per user to the nearest decimal point (1.0). |
| Total users | The total number of users who interacted with your chatbots. This number includes all users, including those who abandoned the chat and who continued until the end of the conversation with the chatbot. |
| User retention rate | The rate at which users go on to engage with the chatbot after having initiated the chat. The value is the percentage across all users for the set filters. |
| Session containment rate | The rate at which sessions are closed without the end user having to close the session without attaining a resolution. Sessions which are passed to support agents are considered contained sessions. The value is the percentage of contained sessions over total sessions. |
Session overview
The Session overview shows session trends and outcomes:
| Metric | Description |
|---|---|
| Sessions over time | Visualizes session volume trends across the selected time period. |
| Number of sessions by how they ended | Breakdown of session outcomes showing how conversations concluded. |
Changing filters will affect the view. Hover over any points in the graph for more information.
Users overview
The Users overview displays user engagement patterns:
| Metric | Description |
|---|---|
| Users over time | Shows the volume of users interacting with your chatbots over the selected period. |
| Comparison with engaged users | Compares total users with engaged users to measure engagement effectiveness. |
All chatbots overview
The All chatbots overview provides a comparative view of all your chatbots:
| Metric | Description |
|---|---|
| Chatbot name | Name of the chatbot. |
| Total sessions | Total number of sessions for this chatbot. |
| Engaged sessions | Number of sessions where users sent at least one message. |
| Total users | Total number of users who interacted with this chatbot. |
| Total messages | Combined count of all messages (user and chatbot). |
| User messages | Number of messages sent by users. |
| Chatbot messages | Number of messages sent by the chatbot. |
This overview provides a high-level composition of chatbot-specific metrics. For more information about specific chatbot aspects, see the individual dedicated reports.
Sessions
The Sessions dashboard provides key performance indicators around chatbot sessions. Monitor session engagement, containment rates, and session outcomes to identify areas for optimization.
Sessions refer to the whole conversation from start to end between the chatbot and end user. Sessions are limited to one chat and end when the session is closed, either by initiation or timeout.
Sessions summary
| Metric | Description |
|---|---|
| Total sessions | The number of opened chat sessions initiated by end users regardless if the chat timed out or was abandoned without any dialog having taken place. |
| Engaged sessions | The total number of sessions in which end users initiated a session and then sent a minimum of 1 message to the chatbot. |
| Engaged session rate | The rate at which end users initiated a session and then sent a minimum of 1 message to the chatbot. The value is the percentage of engaged sessions over total sessions. |
| Average sessions per user | The average number of sessions individual end users initiated. |
| Average engaged sessions per user | The average number of sessions in which individual end users initiated a session and then sent a minimum of 1 message to the chatbot. |
| Containment rate | The rate at which sessions are closed without the end user having to close the session without attaining a resolution. Sessions which are passed to support agents are considered contained sessions. The value is the percentage of contained sessions over total sessions. |
Sessions over time
Displays the number of sessions through time, helping you identify when chatbot interactions are most active. You can use this to determine when to plan extra human support if and when required.
| Metric | Description |
|---|---|
| All sessions | Shows total session volume trends across the selected time period. |
| Engaged sessions | Shows engaged session volume trends across the selected time period. |
How sessions ended
Shows how sessions ended, broken down by count and duration.
Count by how sessions ended
Engaged sessions are sessions in which the user sent at least two messages.
| Metric | Description |
|---|---|
| Number of sessions | Total number of sessions grouped by how they ended. |
| Number of engaged sessions | Total number of engaged sessions grouped by how they ended. |
Duration time by how sessions ended
Engaged sessions are sessions in which the user sent at least two messages.
| Metric | Description |
|---|---|
| All session duration | Average duration for all sessions grouped by how they ended. |
| Engaged session duration | Average duration for engaged sessions grouped by how they ended. |
| Interaction time duration | Time users spend interacting with the chatbot grouped by how they ended. |
Sessions
Provides details about the chatbot sessions:
| Column | Description |
|---|---|
| Session ID | Unique identifier for the session. |
| User ID | Identifier for the end user. The format depends on the channel. |
| Session end reason | The reason the session ended. |
| Start date | The date on which the end user sent the first message. |
| Time | The time at which the end user sent the first message. |
| Engaged session | Whether the end user sent a minimum of 1 message during the session. |
| User messages | The total number of messages sent by the end user during the session. |
| Chatbot messages | The total number of messages sent by the chatbot during the session. |
| Duration | How long the session lasted. |
To drill down into a session, select its Session ID. The session transcript opens in a new window where you can view all messages exchanged between the chatbot and the end user.
Call API
Call API element overview with additional details for comparison.
| Column | Description |
|---|---|
| Chatbot info | Name and identifier of the chatbot making the API call. |
| Request type | HTTP method used for the API call (for example, GET, POST). |
| URL endpoint | The URL endpoint that the API call targets. |
| Total requests | Total number of API requests made. |
| Success responses | Number of requests that returned a successful response. |
| Average response time | Average response time for the API call in milliseconds. |
Messages
The Messages dashboard provides key performance indicators around chatbot messages for both chatbot and user. Compare message totals across types to check if users are dropping off too quickly or getting the resolutions they need.
Message summary
| Metric | Description |
|---|---|
| Total messages | The total number of messages exchanged whether users engaged with the chat or not. This number should be as close to the number of engaged messages as possible to indicate if the chatbot is working efficiently. |
| Average number of messages per session | The total average number of messages exchanged across all sessions whether users engaged with the chat or not. |
| Engaged messages | The total number of messages exchanged as part of an engaged session. |
| Average number of messages per engaged session | The total average number of messages exchanged during engaged sessions. |
| Total user messages | The total number of messages received from users (incoming messages). |
| Total chatbot messages | The total number of chatbot messages sent to users (outgoing messages). |
Messages over time
| Metric | Description |
|---|---|
| Total messages over time | Displays the number of exchanged messages through time, helping you identify when chatbot interactions and message levels are most active. |
| Engaged messages over time | Shows engaged message volume trends across the selected time period. |
Number of messages and average messages per session over time
Number of chatbot and user messages and their averages over time.
Messages and averages comparison
| Metric | Description |
|---|---|
| Total messages inbound | Column (left). Total inbound messages over time. |
| Total messages outbound | Column (left). Total outbound messages over time. |
| Average user messages per session | Line (right). Average number of user messages per session over time. |
| Average chatbot messages per session | Line (right). Average number of chatbot messages per session over time. |
Engaged messages and averages comparison
| Metric | Description |
|---|---|
| Engaged user messages | Column (left). Engaged user messages over time. |
| Engaged chatbot messages | Column (left). Engaged chatbot messages over time. |
| Average engaged user messages per session | Line (right). Average engaged user messages per session over time. |
| Average engaged chatbot messages per session | Line (right). Average engaged chatbot messages per session over time. |
Users
The Users dashboard provides key performance indicators around users. Understand how many people are coming back to chatbots for assistance, how many are finding a resolution, and how many are actually engaging with chatbots.
User summary
| Metric | Description |
|---|---|
| Total users | The total number of users tracked who have initiated a conversation with a chatbot. This includes all sessions, engaged or not, as well as new and returning users. |
| New users | The total number of new or unique users initiating chats. This works by tracking the user's UUID and checking if there are any matches in chatbot records. |
| Returning users | The total number of returning users initiating new chats. This works by tracking the user's UUID and checking if there are matches in chatbot records. |
| User retention rate | The percentage of users who managed to get to what would be considered a successful resolution or termination of chat, without abandoning the chat or it timing out. |
| Engaged users | The total number of end users who initiated a session and then sent a minimum of 1 message to the chatbot. Ideally, this number should be close to the value of Total users. |
| User engagement rate | The percentage of end users who engage with the chatbot. Chat engagement occurs when an end user initiates a chat and then sends a minimum of 1 message to the chatbot. |
New and returning users
New and returning users comparison
Returning user is a user that returned to using the chatbot over a selected time frame.
| Metric | Description |
|---|---|
| New users | Total number of new users over time. |
| Returning users | Total number of returning users over time. |
Engaged new and returning users comparison
Engaged users are users who engaged in conversation with chatbot by sending at least two messages.
| Metric | Description |
|---|---|
| Engaged new users | Number of engaged new users over time. |
| Engaged returning users | Number of engaged returning users over time. |
Number of users over time
Number of users with returning and new user comparison over time.
All users over time
Engaged users are users who sent at least two messages.
| Metric | Description |
|---|---|
| Returning users | Number of returning users over time. |
| New users | Number of new users over time. |
Engaged users over time
Engaged users are users who engaged in conversation with chatbot by sending at least two messages.
| Metric | Description |
|---|---|
| Engaged returning users | Number of engaged returning users over time. |
| Engaged new users | Number of engaged new users over time. |
Users
Provides a detailed look into users at a granular level:
| Column | Description |
|---|---|
| User ID | Identifier for the end user. The format depends on the channel. |
| Number of sessions | Total number of sessions the user had with the chatbot. |
| Total messages inbound | The total number of messages sent by the user to the chatbot. |
| New vs. returning user flag | Whether the user is a new or returning user. |
To view further details about individual users, select a user ID to see information about their sessions including:
- Total sessions
- Inbound messages
- First time seen
- Last time seen
- Average session duration
- CSAT score (if the CSAT element is present in chats)
Dialogs and intents
The Dialogs and intents dashboard provides key performance indicators around used dialogs and triggered intents. Understand which parts of the customer journey are performing better than others to learn about your customers and their preferred routes of interaction.
Top dialogs and intents
Most triggered dialogs and most recognized user intents.
| Chart | Description |
|---|---|
| Top 10 triggered dialogs | The 10 most frequently triggered dialogs. |
| Top triggered intents | The most frequently recognized user intents. |
Top start and exit dialogs
Percentage of sessions that started and ended with a dialog.
| Chart | Description |
|---|---|
| Top 10 start dialogs | The 10 most common dialogs where sessions begin. |
| Top 10 exit dialogs | The 10 most common dialogs where sessions end. |
Top user start messages
Top messages users sent to start interacting with the chatbot.
| Chart | Description |
|---|---|
| Top 10 start messages | The 10 most common messages users send to initiate a chat. |
| Top 10 start messages on engaged sessions | The 10 most common messages users send to initiate an engaged session. |
Dialog and intent insights
Dialogs and intents overview with additional details for comparison.
Dialog insight
| Column | Description |
|---|---|
| Dialog name | Name of the dialog. |
| Times triggered | How many times the dialog has been entered by a user. |
| Users | The total number of users who entered this dialog. |
| Sessions | Number of sessions in which the dialog was triggered. |
| Session start (%) | Percentage of sessions that started with this dialog. |
| Session end (%) | Percentage of sessions that ended with this dialog. |
| Expired session | The number of sessions where the user let the session expire in this dialog. |
| Close session | The number of times conversation ended naturally at the designed point for closing sessions. |
| Go to agent action | The number of times the user requested to talk to an agent. |
| Agent's takeover | Agent took over the conversation from the chatbot. |
Intent insight
| Column | Description |
|---|---|
| Intent name | Name of the intent. |
| Times recognized | The number of times the chatbot recognized a user's utterance and forwarded the user to the correct intent. |
| Users | Total number of users routed to the intent. |
| Sessions | Total number of sessions in which the intent was recognized. |
| Session started (%) | Percentage of sessions that started with this intent. |
Unhandled user messages
Messages that did not pass the Get attribute element type validation or did not match with a keyword in the User input element.
User messages not matched to keyword or failed attribute type validation
| Column | Description |
|---|---|
| Session uncertain message | The user message that was not matched or failed validation. |
| Dialog info | Name of the dialog where the message occurred. |
| Dialog element type | The element type that did not recognize the incoming message. |
| Message count | How many times this particular message was received. |
| Session count | How many sessions this happened in. |
| User count | The total number of users who sent the message. |
Voice of the customer
The Voice of the customer dashboard provides key performance indicators around customer satisfaction from the CSAT element.
Customer satisfaction summary
Key performance indicators for the customer satisfaction (CSAT) element.
| Metric | Description |
|---|---|
| Customer satisfaction score | The percentage of happy CSAT session responses in all responses. |
| Customer response rate | The session CSAT survey response rate in %. |
| Average user rating | The average of session CSAT survey score. |
Satisfaction score and response volume
Customer satisfaction score and response volume with comparison.
| Chart | Description |
|---|---|
| Happy and Not Happy comparison | Comparison of happy and not happy CSAT responses over time. |
| Response type comparison | Comparison of response types over time. |
Customer satisfaction
Scores and comments overview with additional details for comparison. The table overview shows up to 100 worst user responses based on the CSAT score.
| Column | Description |
|---|---|
| Customer satisfaction details | Detailed view of individual CSAT responses including scores and comments. |
Custom dashboards
In addition to predefined dashboards, you can create your own dashboards. Custom dashboards enable you to create personalized views of your chatbot performance data. You can select, arrange, and visualize the metrics that matter most to your business.
Use custom dashboards to monitor key performance indicators, track trends, and make data-driven decisions tailored to your specific needs.
The Custom dashboards page contains two tabs:
- Custom dashboards
- Visualizations
Custom dashboards
Dashboards are visual representations of data. Create custom dashboards to obtain specific, relevant information about the performance of your chatbots.
With custom dashboards, you can:
- Track changes to your KPIs
- Compare current data across time periods
- Add, slice, and filter data
- Compare data in multiple visualizations
Custom dashboards can include one or more of the following:
- Visualizations - Multi-layer analytics that enable you to add and slice data based on multiple metrics
- Key Performance Indicators (KPIs) - Single-metric analytics displayed in numeric format (whole number, decimal number, or percentage)
- Attribute filter - Additional filters that you can add to standard filtering options to filter reports by specific chatbot components
Custom dashboards can contain more than one item of the same type (for example, multiple visualizations). All elements on the custom dashboard must work together to fulfill a common purpose.
To access custom dashboards, go to Insights and Analytics → Analytics → Chatbots → Custom Dashboards.
You can:
- View custom dashboards - By default, the latest custom dashboard that you created is displayed. To view other custom dashboards, choose from the Custom Dashboards menu
- Create a custom dashboard - Either use custom metrics to customize an existing predefined dashboard or create a new custom dashboard from scratch
- Edit a custom dashboard - Select a custom dashboard and then select Edit
- Export a custom dashboard - Select a custom dashboard, then in the menu on the right, select Export to PDF. Supported file type is
.pdf - Duplicate a custom dashboard - Select an existing custom dashboard and then select Save as new
- Share a custom dashboard - Share with other people in your organization. Select a custom dashboard and then select Share
Create dashboard from a predefined dashboard
- On a predefined dashboard, select the menu button and then select Explore from here.
- The report opens in the visualization editor.
- Add other reporting elements to the editor.
- Select Save.
Create dashboard from scratch
To create a new dashboard:
- Go to Insights and Analytics → Analytics → Chatbots → Custom Dashboards, and select Add dashboard.
- Enter the dashboard name. Choose one of the following dashboard availability options:
- Public - Accessible by everyone who has access to Chatbots analytics
- Shared - Shared with specific users or groups
- Private - Accessible only to the creator of the dashboard
- Select Create.
- Drag one or more of the following items to the dashboard editor. You can add more than one item of the same type:
- Visualization - Create a new visualization to display results by multiple data types
- Filter - Add the Filter element and then search for or choose an attribute from the list (for example, channel name or CSAT score). For each attribute, choose the values to filter
- Column container - Create a vertical group of widgets placed in a single column inside your section
- Visualization switcher - Combine multiple visualizations into a single dashboard. At a time, only one visualization is displayed
- Saved Visualizations - Search for a visualization or choose an existing one from the list. You can add visualizations that you created or those that are shared with you
- Rich text - Add rich text such as images, links, code blocks, lists, formatting, and text to your dashboard using markdown syntax
- Add titles and descriptions for each section.
- (Optional) To configure or remove a visualization, select it to get the menu options, and then select Remove from dashboard.
- (Optional) To remove an attribute filter, select the item and drag it to the left.
- Select Save.
Visualizations
Visualizations are interactive, multi-layer analytics that enable you to add and slice data based on multiple metrics. Visualizations are the most extensive report types and can be used as standalone reports or as part of your custom dashboards.
To access visualizations, go to Insights and Analytics → Analytics → Chatbots → Visualizations in the web interface.
You can:
- View a list of visualizations
- Create a new visualization
- View details of a visualization by selecting the information icon
- Search for visualizations by partial or complete name
- Filter visualizations by visualization type or last updated
- Sort visualizations using the table headers
- Export visualizations
- Set the number of visualizations displayed on a single page
Visualizations editor
Use the Visualizations editor to:
- Create a visualization
- Edit a visualization
- Export a visualization
- Duplicate an existing visualization by selecting Save as new
To access the editor, go to Insights and Analytics → Analytics → Chatbots → Custom Dashboards → Visualizations tab and select Create visualization.
The editor contains:
| Component | Description |
|---|---|
| Data catalog | Contains metrics and attributes. Expand menu items to view available data items. |
| Metrics | Quantitative data that you can add to the visualization. |
| Attributes | Qualitative data that you can add to the visualization. |
| Filter options | Select the relevant data type (metrics and attributes) to view a list of data items. |
| Search | Search for data items by partial or full name. |
| Visualization type | Change the chart type as needed. The system updates the visualization to match the selected type. |
| Active zones | Drag items from the data catalog to these areas of the canvas. |
| Ad hoc metrics | Use basic arithmetic functions to create metrics. |
| Filter | Filter the visualizations by specific criteria. Drag the Date or any attribute from the data catalog. |
| Recommendations | Modify the visualization based on usage patterns. |
| Configuration | Format the visualization (for example, change colors or position the legend). |
| Undo, Redo, and Clear | Undo or redo your changes. Select Clear to remove all data from the visualization. |
| Open | View a visualization. Filter to view only the visualizations that you created. |
| Sort | Sort data alphabetically, numerically, or chronologically. |
| Save | Save your visualization. |
| Export | Export your visualization. |
Create visualization
To create a visualization:
- Insights and Analytics → Analytics → Chatbots → Custom Dashboards → Visualizations tab and select Create visualization.
- Choose a visualization type. Each chart type supports specific data types.
- Drag one or more data items (metric, attribute) from the data catalog to the active zones. You can view the data immediately after you add items to the fields.
- (Optional) Customize your visualization:
- To get more detailed data, drag the Date or an Attribute to the Rows/Columns field (only for tables) or to the View by, Trend by, or Attribute fields (for other chart types)
- To split the data into stacked groups, drag an Attribute to the Stack by or Segment by field
- To format the chart, use the Configuration options
- To make changes, use the Undo, Redo, and Clear options. To delete a data item, select the item in the editor and drag it to the left
- Use additional customization options: refine data items or add filters
- Name the visualization.
- Select Save.
Refine data items
You can refine metrics and attributes by selecting the arrow next to a data item in the Metrics section.
You can:
- Display the filter as a percentage
- Filter the attribute - Select Add attribute filter and then either choose a filter from the list or search for a filter. You can add the date and a maximum of 20 data items (metrics and attributes)
- Format the data values
Add filters to the visualization
To filter the visualization:
- Drag Date and other attributes to the filter at the top.
- (Optional) Select the item added to the filter and choose the required options.
- (Optional) Select the filter icon and choose the filters to apply.
Export visualization
On the Visualizations tab, select the three dots next to a visualization and choose one of the following file formats:
.csv.xlsx
The visualization is downloaded as a static file.