Codecov

Scorecards

Codecov is a code coverage reporting platform that that monitors how much of your code has been tested and validated. Codecov analytics can be used to drive visibility into your microservice architecture and understand coverage trends over time.

Integrating Cortex with Codecov allows you to:

• View code coverage details for entities directly in Cortex
• Create Scorecards that track progress and drive alignment on projects involving your Codecov code coverage metrics

How to configure Codecov with Cortex

Prerequisites

Before getting started:

• Create a Codecov access token.

Configure the integration in Cortex

1. In Cortex, navigate to the Codecov settings page:
   1. In Cortex, click your avatar in the lower left corner, then click Settings.
   2. Under "Integrations," click Codecov.
2. Configure the Codecov integration form:
   • API token: Enter your Codecov access token.
   • Host: If you're using a custom Codecov instance, enter your host URL.
     • Make sure to enter the URL without the API path (e.g., https://codecov.getcortexapp.com).
3. Click Save.

caution

If you do not see the settings page you're looking for, you may not have permission to access that page. Please contact your admin for assistance.

If you’ve set everything up correctly, you’ll see the option to Remove Integration in settings.

You can also use the Test configuration button to confirm that the configuration was successful. If your configuration is valid, you’ll see a banner that says “Configuration is valid. If you see issues, please see documentation or reach out to Cortex support.”

How to connect Cortex entities to Codecov

Auto discovery of Codecov projects

Cortex will use the GitHub, GitLab, Bitbucket, or Azure DevOps repository as the "best guess" for the corresponding Codecov project, since Codecov projects are connected to repositories. For example, if the GitHub repo associated with your Codecov instance is my-org/repo, then the entities in Cortex should also be associated with my-org/repo.

You can find the repository for a given entity in its YAML, defined in a block like the one below:

x-cortex-git:
  github:
    repository: cortexapps/sample-repo

If the Codecov project you want to associate isn't the same as the repository, you can override this in the Cortex entity descriptor.

Warning

While Cortex will use the entity tag for discovery with many integrations, the repository is used for Codecov projects.

Editing the entity descriptor

If you need to override automatic discovery, you can define the following block in your Cortex entity descriptor.

x-cortex-static-analysis:
  codecov:
    owner: org-name
    repo: my-project
    provider: AZURE_DEVOPS | BITBUCKET | BITBUCKET_SERVER | GITHUB | GITHUB_ENTERPRISE | GITLAB | GITLAB_ENTERPRISE
    flag: flag

Field	Description	Required
owner	Name of the Git organization	✓
repo	Git repository (without the organization)	✓
provider	One of the Git providers in the sample YAML	✓
flag	Pulls from isolated and categorized coverage reports

The value for repo should be the full repository because Codecov maps projects by repo.

Flags

Codecov's flags are used to categorize coverage reports for various features and tests in a given project. Flags allow you to set different statistics for different areas of your code base. For example, if you have a monorepo with multiple unique projects, you can use Codecov flags to evaluate each project with different test coverage metrics.

To pull flags into Cortex, define the flag line in the entity descriptor block.

caution

If you choose to configure with flags, discovery will be disabled; you would need to define the owner, repo, and provider lines.

Expected results

Entity pages

With the Codecov integration, you can find code coverage details on an entity's details page as long as that entity is associated with a repo linked to your Codecov instance.

From the Integrations tab in the sidebar, you can open the Codecov page to find the code coverage for that entity.

Scorecards and CQL

With the Codecov integration, you can create Scorecard rules and write CQL queries based on Codecov code coverage metrics.

See more examples in the CQL Explorer in Cortex.

Code coverage

Code coverage for an entity's Git repository (out of 100)

Definition: codecov.codeCoverage()

Details

Example

For a Scorecard focused on development maturity, you can set a rule to make sure code coverage for a given entity is at least 95%:

codecov.codeCoverage() >= 95

Set a threshold that is both challenging and realistic so there's an incentive for developers to improve.

tip

Setting up a rule based on code coverage can serve as a secondary check to confirm an entity is synced with Codecov and reporting frequently.

Still need help?

The following are all the ways to get assistance from our customer engineering team. Please use the option that is best for your users:

• Email: help@cortex.io, or open a support ticket in the in app Resource Center
• Chat: Available in the Resource Center
• Slack: Users with a connected Slack channel will have a workflow added to their account. From here, you can either @CortexTechnicalSupport or add a :ticket: reaction to a question in Slack, and the team will respond directly.

Don’t have a Slack channel? Talk with your customer success manager.