GitOps considerations
Before getting started, note the following:- Cortex will only check for files in the repository’s default branch, unless otherwise specified. Cortex defaults to
mainif there is no default branch defined. - Cortex does not delete Scorecards if a corresponding Scorecard YAML is deleted. You can enable automatic archival of entities through GitOps by toggling on “Enable auto archiving of services” in the Entities settings page. Read more in the auto-archival docs.
- Domain, team, and Scorecard definitions must be in the
.cortex/domains,.cortex/teams, and.cortex/scorecardsfolders, respectively, or in any subdirectory beneath them. - The hierarchy of entities in Cortex is based on that hierarchy being defined in the entity’s YAML file; Cortex does not set hierarchies or entity relationships based on a YAML file’s location in your repository.
- Learn more about defining a hierarchy in the YAML in the Team docs, Domain docs, and the Entity relationship docs.
- You can define any number of entities within the same repository.
- GitOps is not designed for high-volume or bulk updates. We recommend keeping batch sizes under 1,000 updates per hour to avoid hitting your Git provider’s rate limits. For large-scale entity updates, we recommend using the Cortex API.
- The recommended placement for entity descriptor files is in the root of the repository, or in the appropriate
.cortex/catalogfolder.- For Bitbucket, Bitbucket Server, GitHub, or GitLab, the descriptor can be located anywhere in the repository as long as the file is named
cortex.yamlorcortex.yml. You can also use a single repository and place descriptor files in the appropriate.cortexsubdirectory. - For Azure DevOps, the
cortex.yamlfile must be stored at the root of the repository of the default branch. It is possible to work around this for unique cases. You can also use a single repository and place descriptor files in the appropriate.cortexsubdirectory. - When using a single repository structure, as described in this example, the
.cortexsubdirectory only respectscatalog,scorecards,domains,workflows, andteamssubdirectories. Do not place an entity’s YAML file in the.cortexdirectory unless it is in one of the supported subdirectories.
- For Bitbucket, Bitbucket Server, GitHub, or GitLab, the descriptor can be located anywhere in the repository as long as the file is named
Step 1: Disable UI editing
When following a GitOps approach, you make changes to entities via their entity descriptor file and sync the changes using a Cortex git integration or programmatically using the Cortex API. You must disable UI editing to ensure consistency. If the UI editor is enabled, then changes made via git will not be processed in Cortex. Confirm that the Cortex UI editor is disabled for each entity type you want to use a GitOps approach for:- Navigate to the GitOps page in Settings.
- Disable the toggles for UI editing next to services, domains, teams, and other entity types.
Step 2: Configure a Git integration
Before you can move to a GitOps approach, Cortex must be integrated with GitHub, GitLab, Azure DevOps, or Bitbucket. Expand the tabs below for instructions on each provider.Azure DevOps
Azure DevOps
Azure DevOps
- Follow the instructions to integrate Cortex with Azure DevOps.
- Add a webhook for Azure DevOps:
- In Cortex, navigate to Settings > Azure DevOps and validate your Azure DevOps integration.
- Click Create a new webhook and copy the unique webhook URL.
- Follow the instructions from Azure on adding a webhook.
- Set the event type to
Code pushedand use the URL from the previous step.
- Set the event type to
Bitbucket
Bitbucket
The Bitbucket integration is pre-configured with the ability to use the GitOps approach in Cortex. If you are using Bitbucket Server, you will also need to add a webhook.
- In Bitbucket, enable development mode.
- You can find this setting in your Bitbucket instance at
https://bitbucket.org//workspace/settings/addon-management/.
- You can find this setting in your Bitbucket instance at
- Follow the instructions to integrate Cortex with Bitbucket.
- If you are not using Bitbucket Server, then your integration is complete and you are ready to use GitOps.
- If you are using Bitbucket Server, add a webhook for Bitbucket:
- Navigate to Settings > Bitbucket and validate your configuration.
- Enter a secret token for your Bitbucket Server webhook.
- Click Create a new webhook and copy the unique webhook URL.
- Follow the Bitbucket Server instructions for adding a project-level or repository-level webhook.
- Set the event type to
repository push. Use the secret and the URL from the previous steps.
- Set the event type to
GitHub
GitHub
The Cortex app for GitHub is pre-configured with the ability to use the GitOps approach in Cortex. If you configure the integration using a personal access token, you will also need to add a webhook.
- Follow the instructions to integrate Cortex with GitHub.
- If you integrated using the Cortex app, then your integration is complete and you are ready to use GitOps.
- If you configured your GitHub integration using a personal access token, create a webhook:
- Enter a secret passphrase for your GitHub webhook in the Secret field.
- After saving the passphrase, a unique webhook URL is displayed.
- The URL will end with
{alias}- make sure to replace this with the alias you used when configuring your GitHub integration.
- The URL will end with
- Follow the instructions from GitHub on creating a webhook. Cortex recommends adding an organization webhook, but you can also define the webhook for a repository in the org.
- Set the content type to
application/json. Use the secret passphrase and the URL from the previous steps.
- Set the content type to
GitLab
GitLab
- Follow the instructions to integrate Cortex with GitLab.
- Create a webhook:
- In the GitLab settings in Cortex, click Create a new token.
- The token and the webhook URL will be displayed.
- In GitLab, create a system hook(recommended) or a project hook.
- Enable
Push eventsfor the webhook. Use the token and the webhook URL from the previous steps.
- Enable
- In the GitLab settings in Cortex, click Create a new token.
cortex.yaml files and processes them. For subsequent webhooks, Cortex only processes files with a change in the webhook event. A maximum of 3,000 changed files will be reported per commit.
Additional configuration
Cortex’s out-of-the-box GitOps configuration suits most common use cases, but you may have a scenario that requires additional configuration, such as a monorepo in Bitbucket, a repository with different projects in multiple branches, a need to restrict which repositories to import from, and more. See Additional configuration options for GitOps below for more information.
Multi-account configuration
It is possible to configure multiple account integrations with Cortex for each of the git integrations. If you’re creating or editing a cortex.yaml in the non-default configuration, you must reference the alias you used for that integration when you configured it.
For example, if you added a second GitHub configuration called non-default-example, you would define the following block in the entity descriptor:
Step 3: Get started with managing entities
After completing your GitOps configuration, you can start managing entities via GitOps. Before following these steps:- We recommend reviewing the Entities documentation to understand the basics about working with entities in Cortex.
Create an entity
Create an entity
Create an entityTo create a new entity from a new repository:Learn more about creating entity YAMLs in Defining entities via YAML file.Verify that your entity was createdTo verify that the new entity was created:
- Create a repository in your git account.
- Create a new
cortex.yamlorcortex.ymlfile in the repo you just created, including the entity’s details. - Commit and push your changes.
The GitHub app has a built-in linter, so if an entity descriptor file is invalid, the GitHub app will comment on the pull request with outstanding issues.
- View the GitOps logs page in Cortex, which displays all changes made in your Cortex workspace. If your entity creation was successful, you should expect to see it at the top of the list.
- You must have the
View GitOps logspermission to view this page.
- You must have the
- Search the Catalogs > All entities page for the new entity’s name or tag. If your creation was successful, it will appear in the search results.
If you do not see your changes, refresh the browser window where you are logged in to Cortex.
Edit an entity
Edit an entity
Edit an entityAny changes you commit to an entity will appear under Recent activity on the entity page overview.
- Navigate to the YAML file for an entity.
- Add data to the entity
- Commit and push your changes.
- View the GitOps logs page in Cortex, which displays all changes made in your Cortex workspace. If your entity edit was successful, you should expect to see it at the top of the list.
- You must have the
View GitOps logspermission to view this page.
- You must have the
- Search the Catalogs > All entities page for the entity’s name or tag. If your edit was successful, it will appear in the search results.
0 entities if this is the case. When you open the commit, the panel will show No changes processed, and the cortex.yaml file will appear under Omitted files.In the Cortex UI on an entity details page, entities created or updated via GitOps will display the file path of the entity’s YAML file and a preview of the last GitOps log. You must have the
View GitOps logs permission.Additional configuration options for GitOps
Cortex’s standard GitOps configuration suits most common use cases:- Single or many projects per repo
- Only one branch needs to be processed for
cortex.yaml - The
cortex.yamlfile is in the default ormainbranch
- Monorepos in Bitbucket: Multiple projects in a single repo, split into subfolders.
- Branches: Non-main or non-default branches, or different projects in multiple branches.
Restrict which repositories to import from
By default, Cortex will check all repositories for services, domains, teams, and other entity types. It is possible to restrict which repositories entities are imported from:- Navigate to the Settings > GitOps.
- Under Options by entity type, find the dropdown labeled Entity GitOps repository allowlist for new entity types.
- Select the repositories you want to import from.
cortex.yaml file.
Using GitOps for a monorepo in Bitbucket
Using GitOps with a monorepo - one Git repository with multiple entities - is supported out-of-the-box for Azure DevOps, GitHub, and GitLab when you use thebasepath field to specify the subdirectory in the entity descriptor (see an example of this in the docs for Azure DevOps, GitHub, and GitLab).
When using a monorepo with Bitbucket, you must use the cortex-properties.yaml file:
- The file is automatically processed, just like the entity descriptor.
- It should live in the
defaultbranch for the repo, regardless of which branches it states Cortex should use to find entity descriptor files. - When using the
cortex-properties.yaml file, thebasePathmay not function as expected if you have service code elsewhere in your repository.
Using GitOps in a non-default branch
You can configure Cortex to automatically processcortex.yaml files in non-standard branches, multiple branches, or both.
Consider the following example scenario:
- You have a project where
mainis protected and is the default branch. - You want to include
cortex.yamlin thedevelopbranch. - You also have a separate project version in a
stagingbranch with its owncortex.yamlfile.
- Add a
cortex-properties.yamlfile in the default branch of your repo - Define the
branchesfield with a list of branches to process.- The default branch must be explicitly defined if you are using an advanced configuration and you want Cortex to search for a
cortex.yamlfile in the default branch.
- The default branch must be explicitly defined if you are using an advanced configuration and you want Cortex to search for a
cortex-properties.yaml file does not contain a branches field, Cortex will continue to process the default branch.
Cortex will process the most recently modified cortex.yaml file across all branches listed in the cortex-properties.yaml file.
When using the
branches field in your cortex-properties.yaml file, make sure to include the default branch if you want Cortex to continue looking for a cortex.yaml file in the default branch.Using multiple source directories
When using Azure DevOps for GitOps, you can configure Cortex to look forcortex.yaml files in multiple subdirectories.
Consider the following example scenario:
- You have a monorepo structure where all projects live in a single repository.
- Each project lives in a subdirectory in the main repository (
project1/,project2/, etc.). - Each project has its own
cortex.yamlfile.
- Add a
src-dirsfield in acortex-properties.yamlfile at the root of the repository, containing a list of directories to process.
cortex.yaml file found in the root of the repository.
Troubleshooting and FAQ
See frequently asked questions below.Conflicts between UI editing, GitOps, and the Cortex API
Conflicts between UI editing, GitOps, and the Cortex API
If GitOps has previously been enabled, but UI editing is temporarily turned on, any changes made in Cortex to applicable entities will not be reflected in Git. When the file is next changed through your Git provider, it will override changes made in the UI.The last received change in a
cortex.yaml file will override previous changes, whether it originated from the create/update entity API or a push from your Git provider. Changes are not appended and the last submitted entire file takes precedence, so fields omitted in cortex.yaml will be removed.Will my cortex.yaml file be picked up immediately?
Will my cortex.yaml file be picked up immediately?
If you already have a
cortex.yaml file when you set up GitOps, Cortex will automatically process it. However, the file will not be processed until UI editing is disabled.The entity I created appears in GitOps logs, but displays 0 entities 0 scorecards in the Entities column.
The entity I created appears in GitOps logs, but displays 0 entities 0 scorecards in the Entities column.
First, use the YAML linter to validate your
cortex.yaml file. Then, confirm GitOps settings are configured correctly:- Make sure UI editing is disabled for the entity type that you’re trying to create.
- Check repositories in the GitOps repository allowlist. If there are repositories selected for the entity type you’re working with, confirm that you’re working from an allowed repo.
My entities were syncing from a directory under .cortex, and now they aren't.
My entities were syncing from a directory under .cortex, and now they aren't.
Cortex matches directory names exactly. It doesn’t scan similarly named directories such as
.cortex/catalog-pilot/, .cortex/catalog_v2/, or .cortex/teams-old/.If files in one of these directories were being discovered before, move them into catalog, domains, teams, scorecards, or workflows, or into a subdirectory beneath one of them. Cortex doesn’t remove entities that were already created from those directories. They stop receiving updates, and you can archive or delete them when you’re ready.