Azure DevOps

Azure DevOps is a Microsoft-owned version control system used for managing the software development lifecycle.

Integrating Cortex with Azure DevOps allows you to:

• Automatically discover and track ownership of Azure DevOps entities

• Follow a GitOps workflow with Azure DevOps

• View information about your Azure DevOps repositories on an entity's details page, including: The repo associated with the entity, recent commits and releases in the event timeline, the most-used language in the files for that entity, the top code contributors, and their number of contributions

  • If you pull in Azure DevOps pipeline data, you can also see pipeline runs and builds in the CI/CD section of an entity's details page.

  • If you enable the option to pull in Azure DevOps work items, you will also see a list of open work items on entity pages.

• View information about pull requests and work items in the engineering homepage

• Create a work item from an Initiative issue in Cortex

• Use Azure DevOps metrics in Eng Intelligence to understand key metrics and gain insight into services, incident response, and more

• Create Scorecards that track progress and drive alignment on projects involving your repositories, Azure DevOps work items, and Azure DevOps pipeline data

How to configure Azure DevOps with Cortex

Prerequisites

Before you get started:

• Add a Azure DevOps personal access token with at least the following scopes enabled:

  • Analytics: read

  • Build: read

  • Code: read

    • If using the Scaffolder with Azure DevOps, you must also enable:

      • Code: write

      • Code: manage

  • Graph & Identity: read

  • Work Items: read and write

Configure the integration in Cortex

1. In Cortex, navigate to the Azure DevOps settings page:

   1. In Cortex, click your avatar in the lower left corner, then click Settings.

   2. Under "Integrations", click Azure DevOps.

2. Click Add configuration.

3. Configure the Azure DevOps integration form:

   • Organization: Enter the slug for your Azure DevOps organization.

   • Username: Enter the username for your personal access token.

   • Personal access token: Enter your Azure DevOps personal access token.

   • Host: Optionally, if you are using a self-managed setup, enter your hostname.

4. Click Save.

Cortex supports mapping multiple identities for a single user if you have multiple configurations of Azure DevOps. See the Identity mapping documentation for more information.

Enable or disable Azure DevOps work items

On the Azure DevOps settings page in Cortex, you can choose whether Azure DevOps work items should be pulled in from Azure DevOps. Cortex recommends disabling this option if your organization does not use work items or if you are worried about running into rate limit issues.

How to connect Cortex entities to Azure DevOps

Import entities from Azure DevOps

See the Create services documentation for instructions on importing entities.

Editing the entity descriptor

In an entity's YAML, you can define a repository, work items, ownership, and pipelines.

Define a repository

To define an Azure DevOps repository for a given entity, add the x-cortex-git block to the entity's descriptor.

x-cortex-git:
  azure:
    project: cortex
    repository: docs
    basepath: myService
    alias: accountAlias

Field	Description	Required
project	The name of the project as listed under the "Projects" tab when you are logged into Azure DevOps (on the https://dev.azure.com// screen)	✓
repository	The repo name you see when you navigate to the "Repos" section of Azure DevOps	✓
basepath	If the entity is in a monorepo (e.g. in a subdirectory), use this field to define the subdir
alias	Alias for the configuration in Cortex (only needed if you have opted into multi-account support)

Only one repository can be defined for in a given entity's YAML in the x-cortex-git block.

Define work items

Before adding work items to your entity YAML, make sure you have enabled the option to pull in Azure DevOps work items in your integration settings.

To define Azure DevOps work items for a given entity, add the x-cortex-azure-devops block to the entity's descriptor. If there is no work item registrations, but the entity matches a repository, we will pull in all work items from the repository's project with a tag that matches the Cortex entity name, Cortex entity tag, or the repository name.

Example WIQL

The example YAML below is based on the following example WIQL:

SELECT
    [System.Id],
    [System.AssignedTo],
    [System.State],
    [System.Title],
    [System.Tags]
FROM workitems
WHERE
    '[System.TeamProject] = Design Agile'
    AND '[System.WorkItemType] = User Story'
    AND '[System.State] = Active'
ORDER BY [System.ChangedDate] DESC
ASOF '02-11-2020'

Learn more about WIQL in Microsoft's WIQL syntax reference.

x-cortex-azure-devops:
    workItems:
        projects:
            - name: projectName1
              wiqls:
                - "[System.TeamProject] = 'Design Agile'"
                - "[System.WorkItemType] = 'User Story'"
                - "[System.State] = 'Active'"
                - ORDER BY [System.ChangedDate] DESC ASOF '02-11-2020'
            - name: projectName2
              alias: alias1
            - name: projectName3
              alias: alias2

Field	Description	Required
projects	List of the projects	✓
name	The project name as listed under the "Projects" tab when you are logged into Azure DevOps (on the https://dev.azure.com// screen)	✓
wiqls	List of WIQL conditions to filter work items fetched
alias	Alias for the configuration in Cortex (only needed if you have opted into multi-account support)

Define ownership

Ownership of each entity through Azure DevOps is defined through an owner of type group.

x-cortex-owners:
  - type: group
    name: My Azure DevOps Team
    provider: AZURE_DEVOPS
    description: This is a description for this owner # optional

name is a case-sensitive field that corresponds to the upstream identifier of your owner from Azure DevOps.

Learn more about ownership in Defining ownership.

Define pipelines

You can add Azure DevOps pipelines under the x-cortex-azure-devops block:

  x-cortex-azure-devops:
    pipelines:
      projects:
      - name: projectName1
        pipelines:
        - id: 1
      - name: projectName2
        pipelines:
        - id: 2

Field	Description	Required
piepelines	List of the pipelines	✓
projects	List of the projects	✓
name	The project name as listed under the "Projects" tab when you are logged into Azure DevOps (on the https://dev.azure.com// screen)	✓
pipelines:id	The Azure DevOps system.definitionID for the pipeline.	✓

Learn more about Azure DevOps pipelines in Microsoft's documentation.

Identity mappings for Azure DevOps

Cortex maps users' email addresses to discovered Azure DevOps accounts.

You can confirm users' Azure DevOps accounts are connected from Azure DevOps identity mappings in settings.

Create a work item from an Initiative issue

Initiatives allow you to set deadlines for specific rules or a set of rules in a given Scorecard and send notifications to users about upcoming due dates.

From the Issues tab of an Initiative, you can automatically create a Azure DevOps work item from a failing rule:

1. Click Create issue.

2. In the modal that appears, fill out the form:

   • Integration: If you have multiple task tracking tools, select Azure DevOps from the Integration dropdown.

   • Name: Enter a name for the configuration.

   • Project: Select from the dropdown.

     • Options available in the dropdown are pulled in from the specific Azure DevOps instances configured in Settings.

3. Select the Work item type and the Sub-item Type from the respective dropdowns. Then, select how the sub-items's fields should be populated on issue creation and status change.

4. Choose to include or exclude groups of entities, or define a more advanced filter.

The issue configuration will apply to all entities that meet the filter criteria. Once an entity is passing the rule, Cortex will automatically close the associated ticket.

Using the Azure DevOps integration

View Azure DevOps data on entity pages in Cortex

The Azure DevOps integration will populate the Repo detail block on an entity's details page.

In the Recent activity preview, you'll find the recent commits and releases. These will also appear in the event timeline.

These data will appear for entities imported from a Git source or those that have a Git repo defined in their YAMLs.

Events

On an entity's Events page, you can find all of the commits and releases associated with that entity. Each is hyperlinked to the commit or release page in Azure DevOps and includes a timestamp.

CI/CD

From the CI/CD > Azure DevOps page in the entity's sidebar, see a history of pipeline runs.

Repository

You can access more detailed information pulled from Azure DevOps under Repository in the sidebar. At the top of the repository page, you'll find the repo associated with that entity and the most-used language in files for that entity. In the Top contributors block, you'll find the three users who have contributed the most code and the number of their contributions.

In the Commits section, you'll find the 10 most recent commits and metadata about each. Below Commits is the Recent releases section, which includes the 5 most recent releases.

Issue tracking

In the Issue tracking section, you can find a list of open Azure DevOps work items. Each work item will show the title, summary, assignees, priority, and date created.

Packages

Packages are automatically scraped from your Git repos or they can be submitted via the packages API. The package file must be in the root of your repository — or, if you're using basepath, in the root of the subdirectory — to be scraped by Cortex. You can query an entity's packages in CQL explorer using packages().

To view packages, click Packages in the entity's sidebar.

The following package types are automatically scraped from repositories:

• JavaScript / Node.js: package.json, package-lock.json, yarn.lock, pnpm-lock.yaml

• Python: requirements.txt, pipfile.lock

• .NET (C#): packages.lock.json

• Java: pom.xml

• Go: go.sum

All other files of these types can be added via the packages API.

Engineering homepage

The Azure DevOps integration enables Cortex to pull information about pull requests and work items into the homepage. You can find your open pull requests, any pull requests assigned to you for review, and any work items assigned to you.

Pull requests and work items from Azure DevOps are refreshed every 5 minutes.

Eng Intelligence

The Eng Intelligence tool also uses pull request data from Azure DevOps to generate metrics:

• Average PR open to close time

• Avg time to first review

• Avg time to approval

• PRs opened

• Weekly PRs merged

• Avg PRs reviewed/week

• Avg commits per PR

Read more about how Eng Intelligence tracks metrics for teams and users in the Eng Intelligence documentation.

Scorecards and CQL

With the Azure DevOps integration, you can create Scorecard rules and write CQL queries based on Azure DevOps work items.

See more examples in the CQL Explorer in Cortex.

Approvals required to merge

Total number of approval required to merge a pull request into a repository. Defaults to 0 if no approvals are defined.

Definition: git.numOfRequiredApprovals()

Examples

For a security or development maturity Scorecard, you can write a rule to make sure at least one approval is required for a pull request:

git.numOfRequiredApprovals() >= 1

By having a rigorous PR process in place for a repo, you can make sure changes aren't made that create vulnerabilities. This kind of rule could also be used in a best practices or project standards Scorecard.

You can also use a similar expression in the Query Builder to find entities lacking approval:

git.numOfRequiredApprovals() < 1

Branches

List all live branches with some basic metadata:

• Head

• Is protected

• Name

Definition: git.branches(): List<GitBranch>

Example

For a development best practices Scorecard, you can make sure that branches associated with an entity match a standard naming convention:

git.branches().all((branch) => branch.name.matches("(main|master|feat-.*|bug-.*|task-.*

Branch protection details

Find details for a specified branch or default branch if none is specified.

Definition: git.branchProtection(branchName: Text?): GitBranchProtection

Examples

For a security Scorecard, you can write a rule to make sure the default branch is protected:

git.branchProtection() != null

Or to make sure the main branch is protected:

git.branchProtection("main") != null

Because vulnerabilities in the default branch are critical, this rule should be in one of the first couple levels. A higher-level rule might make sure that branch protection checks are set:

git.branchProtection("main").requiredStatusChecks.length > 0

You can also use the Query Builder to find entities with unprotected default branches:

git.branchProtection() = null

Commits

Get the latest commits (to a maximum of 100) for a defined lookback period (defaulting to 7 days).

These results can be filtered based on branch name, using the default branch if no other branch is provided.

Definition: git.commits()

Examples

You can use the git.commits() expression in a security Scorecard to make sure entities have fewer than three commits to a "security-fixes" branch in the last week:

git.commits(branch="security-fixes").length < 3

Entities passing this rule will include those that haven't needed three or more security fixes. This can indicate that there aren't vulnerabilities in a given entity's code, but could also suggest that fixes aren't being implemented.

Using this rule in conjunction with one focused on vulnerabilities could provide the extra context needed to gain a better understanding of what's happening.

Default branch

Default branch for the entity's repository or main when null.

Definition: git.defaultBranch()

Examples

If default branches should always be named "main," you can write a rule in a best practices Scorecard to make sure entities are compliant:

git.defaultBranch().matches("main")

File contents

Load the contents of a file from the entity's associated repository.

The contents can be validated by using string comparison operations or parsed by the built-in jq function. The jq function will automatically coerce file contents of JSON or YAML formats.

Definition: git.fileContents(<filename: Text>)

Examples

For a Scorecard focused on development maturity, you could use the git.fileContents() rule to enforce that a CI pipeline exists, and that there is a testing step defined in the pipeline:

git.fileContents(“circleci/config.yml”).matches(“.*npm test.*”) - Enforce that a CI pipeline exists, and there is a testing step defined in the pipeline

A best practices Scorecard, meanwhile, could use this expression for a number of rules:

• To make sure node engine version in specified in the package.json file:

  jq(git.fileContents("package.json"), ".engines.node") != null

• To make sure TypeScript projects have a tsconfig.json file checked in:

  jq(git.fileContents("package.json"), ".devDependencies | with_entries(select(.key == \"typescript\")) | length") == 0 or git.fileExists("tsconfig.json")

• To make sure projects using yarn do not allow NPM:

  jq(git.fileContents("package.json"), ".engines.yarn") == null or jq(git.fileContents("package.json"), ".engine.npm") = "please-use-yarn"

• And to ensure the yarn version being used is not deprecated:

  jq(git.fileContents("package.json"), ".engines.yarn") == null or !(semver("1.2.0") ~= semverRange(jq(git.fileContents("package.json"), ".engines.yarn")))

File exists

Check if file exists from within the entity's associated repository.

Definition: git.fileExists(<filename: Text>)

Examples

For a development best practices Scorecard, this expression can be used for a rule that makes sure developers are checking in lockfiles to ensure repeatable builds:

git.fileExists(“package-lock.json”)

In the Query builder, you can use this expression with a wildcard to find entities with unit tests enabled:

git.fileExists(*Test.java”)

Or to find entities with an outdated Terraform version:

git.fileExists("terraform/versions.tf") and !git.fileContents("terraform/versions.tf").matchesIn("required_version =! \"[~>= ]{0,3}0\\.(12|13)")

Has Cortex YAML (GitOps)

When enabling GitOps to manage entity descriptors, Cortex checks for a checked in file ./cortex.yaml at the root directory. This rule can help track migrations from UI editing to GitOps for entity descriptor management.

Definition: git.hasCortexYaml()

Examples

If you're using a Scorecard to track a migration from Cortex UI to GitOps, you can use this rule to make sure entities are set up for GitOps management of entity descriptors:

git.hasCortexYaml() == true

Git repository set

Check if entity has a registered Git repository.

Definition: git (==/!=) null

Examples

A Scorecard focused on best practices or standards will likely include a rule in its first level making sure a Git repository is set up:

git != null

If an entity is failing this rule, it can indicate broader issues with the integration or explain why an entity isn't functioning as expected.

Last commit details

Provides last commit details.

Definition: git.lastCommit()

Examples

Depending on best practices at your organization, you may want to confirm the last commit for a given entity is no older than 3 days:

datetime(git.lastCommit().date).fromNow() > duration("P-3D")

Confirming whether a service was updated recently can help team members catch outdated code sooner. Plus, if there is a security issue, you can quickly determine which services have or have not been updated to patch the vulnerability.

For a best practices Scorecard, you can also use this expression to make sure the entity's last commit message follows conventional commit guidelines:

git.lastCommit().message.matches("^(feat|fix|docs|style|refactor|test|chore)(\\(.+\\))?:*")

List pipelines

List pipelines with metadata, including name, id, url, and project name.

Definition: azureDevops.pipelines()

Examples

You could write a Scorecard rule to ensure that the entity has at least 1 pipeline that scans vulnerabilities:

azureDevops.pipelines().any(pipeline => pipeline.name.matchesIn("Vuln scan"))

You could write a Scorecard rule to ensure that the entity has an Azure DevOps pipeline set:

azureDevops.pipelines() != null

Pipeline build success rate

Percentage of build pipelines that complete successfully.

This is calculated against builds on the default branch for commits in the last 30 days: # successful builds / (# successful + # failed).

Definition: git.percentBuildSuccess()

Examples

This expression can be used in a development maturity Scorecard to write a rule making sure at least 95% of build runs are successful:

git.percentBuildSuccess() >= 0.95

Pipeline metrics

List pipelines with metrics. Metrics contains the successRate & averageDuration of a pipeline.

Definition: azureDevops.pipelineMetrics()

Examples

You could write a Scorecard rule to ensure pipelines have a successRate higher than 95%:

azureDevops.pipelineMetrics().all((pipeline) => pipeline.metrics.successRate > 0.95)

Pipeline runs

Get pipelines runs meeting the given filter criteria, which includes results, states. Results include "succeeded", "failed", "canceled", "unknown", states include "completed", "inProgress", "canceling", "unknown".

Definition: azureDevops.pipelineRuns()

Examples

List pipeline runs that are opened and reviewed within 1 day:

azureDevops.pipelineRuns().filter(run => run.run.completedDate != null).map((run) => run.run.createdDate.until(run.run.completedDate)).averageDuration() < duration("P1D")

Recency of last commit

Calculates the duration of time between Scorecard evaluation and the date of the last commit from the entity's Git repository.

Definition: git.lastCommit().freshness

Examples

One of the first rules you might write for a Scorecard focused on development maturity or security is one validating that the last commit was within the last month:

git.lastCommit().freshness < duration("P1M")

As counterintuitive as it may seem, services that are committed too infrequently are actually at more risk. People who are familiar with the service may leave a team, institutional knowledge accumulates, and from a technical standpoint, the service may be running outdated versions of your platform tooling.

Reviews

List reviews left during a defined lookback period.

• Organization

• Repository

• Review date

• Reviewer

Definition: git.reviews()

Examples

A development maturity Scorecard might use the git.reviews() expression to make sure that there is a rigorous review process in place before changes are implemented:

git.reviews(lookback=duration("P7D")).length > 25

This rule makes sure that there are more than 25 reviews left in the last week.

Search repository files

Find all instances of code search query from within a repository.

Can filter by path, file name (extension required in file name), or extension. Filters can use * for glob matching. Supplying a query is required.

Definition: git.codeSearch((query = <query: Text>) (, path = <path: Text>) (, fileName = <fileName: Text>) (, fileExtension = <fileExtension: Text>)): List<GitSearchResult>

Examples

You can use the git.codeSearch() expression to query for entities that have certain components, like icons:

git.codeSearch(query = "icon", fileExtension = "css").length > 0

Top repository language

Find top used language for a repository, if available.

Definition: git.topLanguage()

Examples

Let's say the primary language developers should be using is Kotlin. You can write a rule to make sure that the top language associated with entities is Kotlin:

git.topLanguage() == "kotlin"

You can also use this expression to query for entities that don't have Kotlin as the top language to identify those that need to be updated:

git.topLanguage() != "kotlin"

Work items

Number of unresolved work items associated with the entity, where unresolved is defined as the WIQL [System.State] NOT IN ('Closed', 'Done', 'Completed', 'Inactive', 'Removed').

Definition: azureDevops.workItems()

Examples

For a Scorecard measuring entity maturity, you can use this expression to make sure entities have fewer than 10 Azure DevOps work items:

azureDevops.workItems().length <= 10

Work items from WIQL query

Number of work items associated with the entity based on arbitrary WIQL query.

Definition: azureDevops.workItems(query: Text | Null)

Examples

For a more specific rule in an entity maturity Scorecard, you can use this expression with a WIQL query to make sure entities have no more than 3 tickets with "Doing" status and highest priority.

jira.workItems("System.State = \"Doing\" AND Microsoft.VSTS.Common.Priority = 1").length <= 3

Background sync

Cortex conducts a background sync of Azure DevOps identities every day at 10 a.m. UTC. Pull requests and work items are refreshed every 5 minutes.

Still need help?​

The following options are available to get assistance from the Cortex Customer Engineering team:

• Email: [email protected], or open a support ticket in the in app Resource Center

• Chat: Available in the Resource Center

• Slack: Users with a connected Slack channel will have a workflow added to their account. From here, you can either @CortexTechnicalSupport or add a :ticket: reaction to a question in Slack, and the team will respond directly.

Don’t have a Slack channel? Talk with your Customer Success Manager.