LogoLogo
Login to CortexBook a DemoCortex Academycortex.io
  • Cortex Docs
  • Cortex Quick Start
  • Ingesting data into Cortex
    • Managing Entities
      • Adding entities
        • Add services
        • Add domains
        • Add teams
        • Add custom entity types
        • Defining dependencies
      • Entity details page
      • Defining ownership
      • Defining relationship types
      • Grouping entities
      • Adding external documentation
      • Adding Deploy data
      • Adding custom data
      • Viewing discovered entities
      • Archiving entities
      • Relationship graph
      • Using On-call Assistant for incidents
      • Managing Terraform infra in Cortex
    • Managing Catalogs
    • Integrations
      • Internally hosted integrations
      • ArgoCD
      • AWS
      • Azure DevOps
      • Azure Resources
      • BambooHR
      • Bitbucket
      • BugSnag
      • Buildkite
      • Checkmarx
      • CircleCI
      • ClickUp
      • Codecov
      • Coralogix
      • Datadog
      • Dynatrace
      • Entra ID (Azure AD)
      • FireHydrant
      • GitHub
      • GitLab
      • Google
      • Grafana
      • incident.io
      • Instana
      • Jenkins
      • Jira
      • Kubernetes
      • LaunchDarkly
      • Lightstep
      • Mend
      • Microsoft Teams
      • New Relic
      • Okta
      • Opsgenie
      • PagerDuty
      • Prometheus
      • Rollbar
      • Rootly
      • Sentry
      • ServiceNow
      • Slack
      • Snyk
      • SonarQube
      • Splunk Observability Cloud (SignalFx)
      • Splunk On-Call (VictorOps)
      • Sumo Logic
      • Veracode
      • Wiz
      • Workday
      • xMatters
  • Scorecards
    • Initiatives and Action items
      • Creating issues based on Initiatives
    • Scorecard rule exemptions
    • Scorecard rule filters
    • Scorecard examples
    • Scorecards as code
  • Reports
    • Executive report
    • All Scorecards report
    • Bird's eye report
    • Progress report
    • Report card
  • Eng Intelligence
    • Custom Metrics
    • Jira Metrics
    • Metrics Explorer (Beta)
  • Cortex Query Language (CQL)
    • Using CQL reports
    • Using JQ in Cortex
  • Workflows
    • Running a Workflow
    • Blocks
    • Registering a Scaffolder template
      • Scaffolder advanced usage
    • Using a Workflow to sync in ArgoCD
    • Kicking off a Jenkins pipeline in a Workflow
  • Plugins
    • Creating a plugin
      • Creating a plugin proxy
    • Migrating Backstage plugins to Cortex
  • Developer homepage
  • Workspace Settings
    • Using GitOps for Cortex
      • GitOps logs
    • Managing users
      • Roles and permissions
        • Custom roles
        • Team ownership entity editing
      • Configuring SSO
        • Microsoft Entra ID
        • Google
        • Other OIDC providers
        • Okta
          • Okta SCIM
      • Configuring identity mappings
      • Onboarding management
    • API keys, secrets, and tokens
      • Secrets
      • Personal tokens
    • Audit logs
    • Entity settings
      • Data verification
      • Auto archiving entities
    • IP allowlist
    • Notifications
      • Notification logs
    • Customizing your workspace
    • Using search in Cortex
  • Cortex API
    • REST API operations
      • API Keys
      • Audit Logs
      • Catalog Entities
      • Custom Data
        • Custom Data (Advanced)
      • Custom Events
      • Custom Metrics
      • Dependencies
      • Deploys
      • Discovery Audit
      • Docs
      • Eng Intel: User Labels
      • Entity Types
      • GitOps Logs
      • Groups
      • Initiatives
      • Integrations APIs
        • Azure Active Directory (Entra ID) API
        • Azure Resources API
        • AWS API
        • Azure DevOps API
        • CircleCI API
        • Coralogix API
        • Datadog API
        • GitHub API
        • GitLab API
        • incident.io API
        • LaunchDarkly API
        • New Relic API
        • PagerDuty API
        • Prometheus API
        • SonarQube API
      • IP Allowlist
      • Notification Logs
      • On call
      • Packages
      • Plugins
      • Queries
      • SCIM
      • Scorecards
      • Secrets
      • Team Hierarchies
      • Teams
      • Workflows
Powered by GitBook
On this page
  • Prerequisites
  • Accessing Eng Intelligence
  • Using Eng Intelligence
  • Apply time range and team filters
  • Group by team hierarchy
  • Group by entity type
  • Group by user label
  • View more details for an entity
  • Show Scorecard view
  • Metrics
  • Deploy metrics
  • Git metrics
  • Jira metrics
  • PagerDuty metrics
  • Eng Intelligence settings
  • Change Eng Intelligence appearance
  • Set filtering for metric calculation
  • Create and manage user labels for grouping

Was this helpful?

Export as PDF

Eng Intelligence

Last updated 2 days ago

Was this helpful?

Eng Intelligence provides you with key metrics and high-level data to gain insights into services, bottlenecks in the pull request lifecycle, incident response, and more. These metrics can give context to cross-team activities and indicate areas that need deeper investigation, allowing you to quickly remediate and improve productivity across your teams.

See to learn about the new experience of visualization for Eng Intelligence.

Eng intelligence reports on metrics for , git (, , , and ), issue tracking (), and incident management (). See the available metrics under the header below.

Prerequisites

Before using Eng Intelligence, make sure you have configured your Git providers and Jira with the proper permissions. See each integration's documentation page for required permissions and configuration instructions:

    • Bitbucket data in Eng Intelligence is in private beta. Please contact your Cortex Customer Success Manager for access.

    • When using Bitbucket in Eng Intelligence, it is highly recommended to use the .

Cortex users with the View Eng Intelligence can access Eng Intelligence. Users with the Configure Eng Intelligence permission can configure Eng Intelligence settings.

If you do not have Eng Intelligence in your Cortex instance, please contact your Cortex Customer Success Manager.

Accessing Eng Intelligence

To view, click Eng Intelligence in the main nav:

Using Eng Intelligence

These values are recalculated every hour. For count metrics (e.g., PRs opened) , 0 is displayed if no data is available. For average metrics (e.g., average PR open to close time), N/A is displayed if no data is available to calculate averages.

Apply time range and team filters

By default, Eng Intelligence displays data from the last 7 days.

To filter by team, group, or owner:

  1. Click into Group, Owner, or Team, and select filter options.

  2. Click Apply.

Group by team hierarchy

Group by entity type

By default, Eng Intell displays Team data. In the upper left corner, click the Team dropdown to select a different entity type:

Group by user label

Click the Group by dropdown and select a label you want to group by. The grouping will be added as a row to the metrics table, along with separate rows for each member of the grouping.

View more details for an entity

To better understand the data behind a trend you see in Eng Intelligence, click an entity to open a side panel with more information:

  • Under the Related activity tab, see available metrics and recent activity.

  • Under the Trends tab, see a historical performance graph for each metric.

In the upper right corner of the panel, you can adjust the time range for the graphs to be anywhere between the last 7 days and 6 months. This will update the graph view and maps to the table, so all metrics will reflect the new timeframe.

Show Scorecard view

In the upper right corner, click Display. In this drop-down, you can choose whether to display entities in their associated hierarchies and you can select a Scorecard.

When you select a Scorecard, Scorecard performance is overlayed in Eng Intelligence when grouped by team or service. This view is not available when grouping by group, user, or owner. The icon representing the Scorecard level achieved by each entity will appear next to the entity name:

Metrics

Deploy metrics

Avg deploys/week

Calculates the average number of deploys per week over the selected time range.

Deploy change failure rate

Displays the number of rollbacks divided by number of deploys in a given time range.

Git metrics

Avg PR open to close time
  • Calculates the average time to close pull requests for each PR opened and merged during the selected time range.

  • Pulls data from Azure DevOps, Bitbucket, GitHub, and GitLab.

This metric provides insight into how long it takes to merge something, such as build time, reviews, conversations, fixing linter issues, etc.

If your Average PR open to close time is high, it’s worth investigating to identify the part of the development cycle that contribute the most to this time.

Average PR open to close time is related to other metrics, such as time to review and bottlenecks in average PRs reviewed each week. The key here is to examine the time and quantity of a particular activity.

Note that if some teams are using draft pull requests, their numbers may be higher.

Avg time to first review
  • Determines average time from first open to first review of a pull request for any PR that has been opened during the selected time range.

  • Pulls data from Azure DevOps, Bitbucket, GitHub, and GitLab.

For a subset of pull requests, this metric can provide insight into potential inefficiencies. For high figures, investigate whether this is due to the software process or roadblocks faced by team members.

Note that if some teams are using draft pull requests, their numbers may be higher.

Avg time to approval
  • Displays average time from when a pull request was first opened to when it was first approved for any PR opened during the selected time range.

  • Pulls data from Azure DevOps, Bitbucket, GitHub, and GitLab.

Average time to approval can capture review-related bottlenecks in the PR cycle. When this figure is high, there may be opportunities to improve processes and PR sizes.

Note that if some teams are using draft pull requests, their numbers may be higher.

PRs opened
  • Displays a count of pull requests opened during the selected time range.

  • Pulls data from Azure DevOps, Bitbucket, GitHub, and GitLab.

Pull requests opened is particularly useful as a throughput metric. When reviewing this data, consider the expected minimum activity for a developer.

On an individual level, evaluate how much time a team member spends building features versus supporting others. You can also assess how much time a team is spending shipping code versus other teams.

Note that while this metric provides useful insight, weekly PRs merged may be a more meaningful figure.

Weekly PRs merged
  • Calculates how many pull requests were opened each week, averaged across the weeks in the selected time range, to determine how many PRs were opened and merged each week.

  • Pulls data from Azure DevOps, Bitbucket, GitHub, and GitLab.

This throughput metric provides insight into how many things make it to the default branch and are closed out.

In theory, this figure should match the trend for Average PR open to close time, since you don’t want too many pull requests kept open.

Avg PRs reviewed/week
  • Calculates the number of pull requests that were reviewed each week, averaged across the selected time frame.

  • Pulls data from Azure DevOps, Bitbucket, GitHub, and GitLab.

This metric helps users understand bottlenecks in the review stage due to load balancing work, education gaps, onboarding, career progression, and domain mastery.

Note that this figure has been deduplicated on a per user basis, so if a user reviews a pull request multiple times, it will only display once within Eng Intelligence.

You may be spending too much time in the review stage if this figure is high, but you have a low number of commits and a low number of merged pull requests. If this is the case, other parts of the PR lifecycle may be at risk.

Avg commits per PR
  • Displays the number of commits required from PR open to close, and averaged across all PRs, for any PR opened and merged during the selected time range.

  • Pulls data from Azure DevOps, GitHub, and GitLab.

This metric provides insight into activity trends by team members, as greater activity indicates more engagement.

Average commits per PR can be helpful during the onboarding process, so you can gauge how long it takes for a developer to reach the team’s baseline for activity.

Note that if some teams are using draft pull requests, their numbers may be higher.

Avg LOC changed per PR
  • Displays the average number of lines added plus lines deleted for pull requests that were opened/merged during the selected timeframe.

  • Pulls data from GitHub and GitLab. This metric is not supported for Azure DevOps.

This metric can provide information about pull request size. Ideally, developers should open consumable PRs that are easy to review, and thus are easy to push into production.

This figure can impact other metrics related to the PR cycle.

Jira metrics

PagerDuty metrics

Mean time to resolve incidents
  • Calculates average amount of time from incident open to resolution in the selected time range.

  • Pulls data from PagerDuty.

Incidents opened
  • Displays sum of incidents opened for that time range; based on the most recently assigned user/team for each incident.

  • Pulls data from PagerDuty.

Incidents opened/week
  • Displays sum of of incidents opened, divided by the number of weeks in the selected time range; based on the most recently assigned user/team for each incident.

  • Pulls data from PagerDuty.

Eng Intelligence settings

Change Eng Intelligence appearance

Set filtering for metric calculation

  • Under Deploys, select the deploy environments you want to include in the calculation of deploy frequency and deploy failure rate. - If none are selected, all deploys will be included.

  • Under Pull requests, select the authors you want to exclude from the calculation of PR-related metrics.

    • If none are selected, PRs from all authors will be included.

    • By default, Cortex filters out pull requests opened by bots in GitHub but does not do this automatically for GitLab.

Create and manage user labels for grouping

User labels in Eng Intelligence allow you to group users into cohorts to analyze metrics based on different factors. This can be useful for benchmarking one engineer’s metrics against the average within a cohort, comparing metrics between engineers who use different tools to complete their work, and understanding metrics by different variables: location (e.g., in office or remote), engineer level (staff vs. lead engineer), tech stack (frontend vs. backend), and more.

Users who have the Configure user labels permission can create and apply labels.

The instructions below describe how to use this feature in the Cortex UI. See the Cortex API documentation for instructions on creating and managing user labels programmatically.

Create a user label

    1. Click your avatar in the lower left then click Settings.

    2. Under Eng Intelligence, click User labeling.

  1. In the upper right corner, click Create label.

  2. Fill out the “Create label” form:

    • Name: Enter a descriptive name, e.g., Location.

    • Description: Optionally enter a description, such as "This label helps us understand metrics by location."

    • Values: Enter possible values for the label, e.g., New York, California, Remote.

  3. Click Create label.

After saving, the label will appear under the Label management tab in the Eng Intelligence settings page.

View applied user labels

Assign a user label to a user

  1. In the list, locate the user you want to add a label to. Under the "Labels" column for that user, click the pencil icon.

  2. In the side panel, click into the dropdown to select a pre-existing label. To create a new label, type in a name then click +Create in the dropdown.

  3. At the bottom of the side panel, click Set labels.

Assign user labels in bulk

  1. In the bulk edit modal, enter the labels you want to add to the users, then click Set labels.

Review trends in Eng Intelligence and use that knowledge to inform your . While viewing an Eng Intelligence report, in the upper right corner of the page click Create Scorecard. You will be redirected to a configurable Scorecard template that measures performance, activity, and flow metrics that impact productivity.

Eng Intelligence aggregates data from your connected entities to calculate critical metrics based on your organization's priorities. The data is presented by team, group, or individual, and can be filtered by time range. Cortex provides a set of , but you can also create to track here.

To filter by time range: In the upper right corner of Eng Intelligence, click Last 7 days, then select a new time range for your metrics display:

Click Filter in the upper right corner.

By default, each Team entity in Cortex is displayed in its own dedicated row. To group by the you've created, click View as hierarchy.

After you have , you can group by labels in Eng Intelligence.

Users with the Configure custom metrics permission can create for Eng Intelligence, or you can use the built-in metrics listed below.

Eng intelligence reports on metrics for , git (Azure DevOps, Bitbucket, GitHub, and GitLab), Jira, and PagerDuty.

Pulls deploy data added via the .

Pulls deploy data added via the .

For information on using Jira metrics in Eng Intelligence, see .

From the , users with the Configure Eng Intelligence permission can also choose which columns to display and adjust the order of columns.

Under Settings > Eng Intelligence, in , users with the Configure Eng Intelligence permission can set filters for some pre-defined metrics:

In Cortex, navigate to the .

In the under the User labeling tab, you can view a list of users and their applied labels. Note that these labels are only displayed in Eng Intelligence, and not in other pages within Cortex.

In Cortex, navigate to the .

In Cortex, navigate to the .

Check the boxes next to the users you want to edit. As you check names, a banner will appear at the bottom of the page showing how many users are selected. In that banner, click Edit labels.

After applying labels to users, you can while viewing Eng Intelligence metrics.

Scorecards
team hierarchies
custom metrics
deploys
Cortex deploys API
Cortex deploys API
Jira metrics
Eng Intelligence tab of Appearance settings
the Filters tab
User labeling settings
Eng Intelligence settings page
User labeling settings
User labeling settings
custom metrics
default metrics
set up user labels
group by user label
Metrics Explorer
deploys
Azure DevOps
Bitbucket
GitHub
GitLab
Jira
PagerDuty
Deploys
permission
Metrics
PagerDuty
The "Related activity" tab shows recent activity and metrics.
A Scorecard level icon appears next to an entity name.
GitHub
GitLab
Jira
Azure DevOps
Bitbucket
workspace token configuration