AI Impact Dashboard for GitHub Copilot (Public Beta)

We're excited to invite you to our beta program for our Copilot Dashboard in Engineering Intelligence. Using this dashboard, you’ll be able to measure adoption and impact of AI coding tools, starting with GitHub Copilot, across your engineering teams.

What's new

Impact by team AI adoption rate

View each team’s Copilot adoption rate, and correlate team adoption rate to delivery and reliability metrics. Adoption rates are provided daily by comparing active users against Copilot seats. Choose between different performance metrics by using the dropdown on the top right area of this chart.

Impact between Copilot users vs Non-Copilot users

Compare delivery and reliability side-by-side between users who leveraged AI tools within the last 7 days, and those who did not. Understand whether recent AI usage affects engineering performance. Choose between different performance metrics by using the dropdown on the top right area of this chart.

Cortex will automatically create an AI Usage user label for your team members, and keep that updated based on whether each member was active in Copilot within the last 7 days or not.

AI adoption insights

Track how many developers are actively usually Copilot and see how adoption has trended over time.

Dive deeper in Metrics Explorer

See two new metrics, AI Adoption Rate and Active AI Users, within a new AI Tools category in Metrics Explorer. Group any Eng Intel Metric by the Cortex-generated AI Usage user label to compare performance between AI Users (Last 7 days) and Non-AI Users (Last 7 days).

Getting started

Add data from Copilot

Note that Copilot usage history cannot be backfilled. Your dashboard will populate over time as usage data is collected.

Data integration for Copilot metrics is the first step. If you have already configured the GitHub integration, you must update your permissions to enable the ability to pull in Copilot data.

See the instructions below for your integration method:

• Cortex GitHub app

  • Re-authorize your Cortex GitHub app to accept two new permissions.

• Custom GitHub App

  • Add the following permission to your custom GitHub App: Read-only access to Administration.

• Personal Access Token

  • Add the following permission to your PAT: Read-only access to Administration.

View the Copilot Dashboard

Once your integration is configured with the necessary permissions, navigate to Eng Intelligence > Dashboards > Copilot Dashboard.

Note that Copilot usage history cannot be backfilled. Your dashboard will populate over time as usage data is collected.

See the full documentation: Copilot Dashboard.

Questions or feedback?

We'd love your input as we refine this dashboard. Please reach out to your account team or contact our product team ([email protected]) directly with any questions, suggestions, or issues you encounter during the beta period.