Skip to main content

Eng Intelligence

Eng Intelligence provides users with key metrics and high-level data to gain insights into services, bottlenecks in the pull request lifecycle, incident response, and more. These metrics can indicate areas that need deeper investigation, allowing you to quickly remediate and improve productivity across your teams.

Users with the View Eng Intelligence permission can access Eng Intelligence. Users with the Configure Eng Intelligence permission can configure Eng Intelligence settings.

Accessing Eng Intelligence​

To view, click Eng Intelligence in the main nav:
Click Eng Intelligence in the left navigation menu

tip

If you do not see Eng Intelligence, please contact your Cortex Customer Success Manager.

Using Eng Intelligence​

Eng Intelligence aggregates data from your connected entities to calculate critical metrics based on your organization's priorities. The data is presented by team, group, or individual, and can be filtered by time range. Cortex provides a set of default metrics, but you can also create custom metrics to track here.

These values are recalculated every hour. For count metrics, like "PRs opened," you’ll see a "0" if there are no data available. For average metrics, like "Avg PR open to close time," you’ll see "N/A" if no data are available to calculate averages.

Apply time range and team filters​

To filter by time range:
In the upper right corner of Eng Intelligence, click the Time range dropdown then select a time range for your metrics display: Click the time range filter

To filter by team, group, or owner:

  1. Click Filter by in the upper right corner.
    Click "Filter by"
  2. Select a team, group, and/or owner.
  3. Click Save filters.

Group by team hierarchy​

By default, each Team entity in Cortex is displayed in its own dedicated row. To group by the team hierarchies you've created, click View as hierarchy.

Group by entity type​

Click the Group by dropdown to select and view the metrics for each entity type: team, group, user, service, owner, domain. Click "Group by" then select an entity type

View more details for an entity​

To better understand the data behind a trend you see in Eng Intelligence, click an entity to open a side panel with more information. In the panel, see the available metrics a historical performance graph for each.

eng intelligence 2

You can adjust the time range for the graphs to be anywhere between 7 days and 6 months. This will update the graph view and maps to the table, so all metrics will reflect the new timeframe.

Show Scorecard view​

The Show Scorecard view overlays Eng Intelligence with Scorecard performance by level when grouped by team or service. This view is not available when grouping by group, user, or owner.

eng intelligence 3

When you select a Scorecard from the Show Scorecard dropdown, the icon representing the level achieved by each entity will appear next to its name.

Metrics​

Users with the Configure custom metrics permission can create custom metrics for Eng Intelligence, or you can use the built-in metrics listed below.

Eng intelligence reports on metrics for deploys, GitHub, GitLab, Jira, and PagerDuty.

Deploy metrics​

Avg deploys/week
  • Calculates the average number of deploys per week over the selected time range.
  • Pulls deploy data added via the Cortex deploys API.
Deploy change failure rate
  • Displays the number of rollbacks divided by number of deploys in a given time range.
  • Pulls deploy data added via the Cortex deploys API.

Git metrics​

Avg PR open to close time
  • Calculates the average time to close pull requests for each PR opened and merged during the selected time range.
  • Pulls data from GitHub and GitLab.

This metric provides insight into how long it takes to merge something, such as build time, reviews, conversations, fixing linter issues, etc.

If your Average PR open to close time is high, it’s worth investigating to identify the part of the development cycle that contribute the most to this time.

Average PR open to close time is related to other metrics, such as time to review and bottlenecks in average PRs reviewed each week. The key here is to examine the time and quantity of a particular activity.

Note that if some teams are using draft pull requests, their numbers may be higher.

Avg time to first review
  • Determines average time from first open to first review of a pull request for any PR that has been opened during the selected time range.
  • Pulls data from GitHub and GitLab.

For a subset of pull requests, this metric can provide insight into potential inefficiencies. For high figures, investigate whether this is due to the software process or roadblocks faced by team members.

Note that if some teams are using draft pull requests, their numbers may be higher.

Avg time to approval
  • Displays average time from when a pull request was first opened to when it was first approved for any PR opened during the selected time range.
  • Pulls data from GitHub and GitLab.

Average time to approval can capture review-related bottlenecks in the PR cycle. When this figure is high, there may be opportunities to improve processes and PR sizes.

Note that if some teams are using draft pull requests, their numbers may be higher.

PRs opened
  • Displays a count of pull requests opened during the selected time range.
  • Pulls data from GitHub and GitLab.

Pull requests opened is particularly useful as a throughput metric. When reviewing this data, consider the expected minimum activity for a developer.

On an individual level, evaluate how much time a team member spends building features versus supporting others. You can also assess how much time a team is spending shipping code versus other teams.

Note that while this metric provides useful insight, weekly PRs merged may be a more meaningful figure.

Weekly PRs merged
  • Calculates how many pull requests were opened each week, averaged across the weeks in the selected time range, to determine how many PRs were opened and merged each week.
  • Pulls data from GitHub and GitLab.

This throughput metric provides insight into how many things make it to the default branch and are closed out.

In theory, this figure should match the trend for Average PR open to close time, since you don’t want too many pull requests kept open.

Average PRs reviewed/week
  • Calculates the number of pull requests that were reviewed each week, averaged across the selected time frame.
  • Pulls data from GitHub and GitLab.

This metric helps users understand bottlenecks in the review stage due to load balancing work, education gaps, onboarding, career progression, and domain mastery.

Note that this figure has been deduplicated on a per user basis, so if a user reviews a pull request multiple times, it will only display once within Eng Intelligence.

You may be spending too much time in the review stage if this figure is high, but you have a low number of commits and a low number of merged pull requests. If this is the case, other parts of the PR lifecycle may be at risk.

Average commits per PR
  • Displays the number of commits required from PR open to close, and averaged across all PRs, for any PR opened and merged during the selected time range.
  • Pulls data from GitHub and GitLab.

This metric provides insight into activity trends by team members, as greater activity indicates more engagement.

Average commits per PR can be helpful during the onboarding process, so you can gauge how long it takes for a developer to reach the team’s baseline for activity.

Note that if some teams are using draft pull requests, their numbers may be higher.

Avg LOC changed per PR
  • Displays the average number of lines added plus lines deleted for pull requests that were opened/merged during the selected timeframe.
  • Pulls data from GitHub and GitLab.

This metric can provide information about pull request size. Ideally, developers should open consumable PRs that are easy to review, and thus are easy to push into production.

This figure can impact other metrics related to the PR cycle.

Jira metrics​

For information on using Jira metrics in Eng Intelligence, see Jira metrics.

PagerDuty metrics​

Mean time to resolve incidents
  • Calculates average amount of time from incident open to resolution in the selected time range.
  • Pulls data from PagerDuty.
Incidents opened
  • Displays sum of incidents opened for that time range; based on the most recently assigned user/team for each incident.
  • Pulls data from PagerDuty.
Incidents opened/week
  • Displays sum of of incidents opened, divided by the number of weeks in the selected time range; based on the most recently assigned user/team for each incident.
  • Pulls data from PagerDuty.

Eng Intelligence settings​

Change Eng Intelligence appearance​

From the Eng Intelligence tab of Appearance settings, users with the Configure Eng Intelligence permission can also choose which columns to display and adjust the order of columns.

Set filtering for metric calculation​

In the Filters tab on Settings > Eng Intelligence, users with the Configure Eng Intelligence permission can set filters for some pre-defined metrics:
On the Filters page, set deploy environments and exclude pull request authors

  • Under Deploys, select the deploy environments you want to include in the calculation of deploy frequency and deploy failure rate.
    • If none are selected, all deploys will be included.
  • Under Pull requests, select the authors you want to exclude from the calculation of PR-related metrics.
    • If none are selected, PRs from all authors will be included.
    • By default, Cortex filters out pull requests opened by bots in GitHub but does not do this automatically for GitLab.