Analyze autonomous agent performance

The Analytics page in Copilot Studio provides an aggregated insight into the overall health of your agent with event triggers across analytics sessions. The page also displays an Overview area that provides high-level, key performance indicator (KPI) metrics for your agent, a Savings area that analyzes time and cost savings attributable to your agent or your agent's tools, and a Summary area that provides key analytic insights into your agent's performance.

For more information about:

• The Summary and Overview areas, see Analytics overview
• The Savings area, see Analyze time and cost savings for agents

There are four core areas to focus on when reviewing and improving autonomous agent performance:

• Run outcomes: Knowing the end result of a session started by a trigger helps you begin to identify where your agent is succeeding and where it needs improvement.
• Trigger use: Seeing which triggers are used and how often helps you understand what your agent is doing and why.
• Tool use: Learning how often tools are used and how often they succeed can help you understand if those tools are useful and successful.
• Knowledge source use: Learning how often individual knowledge sources are used and how often the agent returned errors.

To change the time range, select the range dropdown at the top of the Analytics page. You can view analytics for sessions that occurred in the last 90 days.

Run outcomes

The Run outcomes section shows a chart that tracks the type of outcome for each session.

To download run outcome data (data visualized in the graph), select the menu icon and select Download CSV.

You can also see a breakdown of session durations. Select See details on the Run outcomes chart. A side panel opens with the average duration for successful runs and failed sessions and how the length of sessions changed over time.

Long session durations can cause delays in workflows your agent is a part of. You can improve your agent's speed by providing more specific instructions at the trigger or agent level.

Trigger use

The Trigger use section shows a chart and metrics that track how often each trigger initiated a session. The chart shows the change in trigger use over time.

Select See details to view a breakdown of all triggers used during the selected time period, the most common events that triggered a run, and which events resulted in the most failed runs.

You can use this information to target specific runs for improvements, and understand which kind of events your agents responds to the most.

Tool use

The Tool use section shows a chart and metrics that track how often your tools are started over time, and how often your agent used those tools successfully. It also shows trend indicators for how often your agent uses each tool and the percentage of called tools used successfully.

The chart displays the top five tools used over the date period defined at the top of the Analytics page.

In the legend below the chart, hover over any of the tools to highlight that tool in the chart.

To open a side panel with a list of all tools used in the specified time period, along with trend indicators, select See details on the chart. On the Tool use panel, you can display calculations of the percentage of questions used for each tool. If your agent has child agents, you can choose to display calculations for both the main agent and child agents (All), the Main agent only, or the Child agent only.

Knowledge source use

The Knowledge source use section provides insight into how your agent uses it knowledge sources and how effectively those sources support your agent. The panel shows the top five knowledge sources used over the date range defined, including each source's type, total questions, response quality, and reactions. Clicking any of the listed sources opens a page of all the user questions that referenced that source. Select the See all button at the top right corner of the panel to open a page of all knowledge used by your agent.

Select See details to open a side panel with knowledge source use trend and error rates. You can use these charts to identify which knowledge sources work well to help users, and which to target for improvements.

On the side panel, select All to display metrics for both the main agent and child agents, Main agent for metrics about the main agent only, or Child agent for metrics about child agents only.

• Source use trend is a chart of how often your knowledge sources were used by your agent over the selected time period. In the legend below the chart, hover over any of the knowledge sources to highlight that knowledge source in the chart, select the knowledge sources to enable/disable it from the chart view.
• Errors shows the percentage of sessions that used each knowledge source type (for example, SharePoint) that resulted in an error.