> ## Documentation Index
> Fetch the complete documentation index at: https://docs.cortex.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Keep your catalog up to date

> Automate how entities enter, change, and leave your Cortex catalog so it stays accurate as new services ship

Most operational questions start with "what do we have, who owns it, and what does it depend on?" If you can't answer that reliably, everything after it runs on guesswork.

A catalog is only useful if people trust it. New services ship, teams reorganize, and cloud resources get torn down, and every change that doesn't reach Cortex makes the catalog a little less accurate. Scorecards, notifications, data verification, and Eng Intelligence all depend on that data being right.

This recipe covers the methods Cortex customers use to keep their catalogs current with as little manual work as possible, organized around the lifecycle of an entity: how it gets into the catalog, how it stays accurate, and how it leaves.

## Quick checklist

* [ ] Each entity type has one management method (UI, API, or GitOps), and your GitOps settings match it.
* [ ] New entities are created automatically, from discovery, a template, or the workflow that creates the service.
* [ ] Descriptor changes are validated before they're applied.
* [ ] Teams sync from an identity provider, Git provider, or HR system, and teams (not individuals) own entities.
* [ ] Auto-archival is on, so entities that no longer exist leave the catalog.
* [ ] A Scorecard flags incomplete entities, and a recurring data verification period asks owners to confirm their data.

## Before you start

**1. Decide how each entity type is managed**

You can create and update entities in three ways. The UI and API update Cortex directly, whereas GitOps changes go through your Git provider first. Choose one method per entity type.

* **Cortex UI** - People edit entities directly in Cortex. Best for smaller catalogs or entity types that change rarely.
* **Cortex API** - Your own scripts and pipelines write descriptors to Cortex through the [Catalog entities API](https://docs.cortex.io/api/rest/catalog-entities). The [Cortex CLI](https://docs.cortex.io/configure/cortex-command-line-interface.md) and [Terraform provider](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/terraform/terraform-provider.md) use the API too. Best for large-scale or programmatic updates driven by another system.
* **GitOps** - Descriptors live as `cortex.yaml` files in Git, and Cortex processes them when changes are committed. Best when you want review, history, and rollback for catalog changes. See [GitOps](https://docs.cortex.io/configure/gitops).

If you use GitOps for a type, disable UI editing for it in [GitOps settings](https://docs.cortex.io/configure/settings/gitops-settings.md). While UI editing is enabled, Cortex doesn't process committed changes to that type's descriptors.

> [!WARNING]
>
> Disabling UI editing doesn't block the API. An API update to a GitOps-managed entity succeeds, but the next commit to that entity's `cortex.yaml` overwrites it. If an entity type is managed with GitOps, make changes in Git, not through the API.

**2. Connect your Git provider**

Many of the patterns below rely on a [GitHub](https://docs.cortex.io/ingesting-data-into-cortex/integrations/github.md), [GitLab](https://docs.cortex.io/ingesting-data-into-cortex/integrations/gitlab.md), [Bitbucket](https://docs.cortex.io/ingesting-data-into-cortex/integrations/bitbucket.md), or [Azure DevOps](https://docs.cortex.io/ingesting-data-into-cortex/integrations/azuredevops.md) integration. If you use GitOps, see [Using GitOps for Cortex](https://docs.cortex.io/configure/gitops/using-gitops-for-cortex.md) for setup steps.

**3. Agree on your descriptor conventions**

Decide on a naming convention for entity tags (`x-cortex-tag`), and which fields every entity should have, such as owners, groups, a parent domain, and links. Remember that an entity's tag can't be changed later: changing `x-cortex-tag` creates a new entity.

## Step 1: Choose how new entities get into the catalog

Most teams combine more than one approach. Use discovery to catch what already exists, and automation to register new services the moment they're created.

| Approach | Best for | Learn more |
| - | - | - |
| Discovery and import | Finding repositories, cloud resources, and APM services that aren't in Cortex yet | [Discovered entities](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/discovery-audit.md) |
| Auto import | Cloud resources from AWS, Azure, and Google Cloud, teams from Workday, and domains from ServiceNow | [Entity settings](https://docs.cortex.io/configure/settings/entity-settings) |
| GitOps descriptors | Services and other entities defined in a `cortex.yaml` file, either in each repository or in a central `.cortex/catalog` repository | [Using GitOps for Cortex](https://docs.cortex.io/configure/gitops/using-gitops-for-cortex.md) |
| API, CLI, or Terraform | Scripted or large-scale updates from your own systems | [Catalog entities API](https://docs.cortex.io/api/rest/catalog-entities), [Cortex CLI](https://docs.cortex.io/configure/cortex-command-line-interface.md), [Terraform provider](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/terraform/terraform-provider.md) |
| Scaffolder and Workflows | Creating a new service and its catalog entry in one step | [Scaffolder](https://docs.cortex.io/streamline/workflows/scaffolder) |

> [!TIP]
>
> GitOps isn't designed for bulk updates. Keep batches under 1,000 updates per hour to avoid your Git provider's rate limits, and use the [Cortex API](https://docs.cortex.io/api/rest/catalog-entities) for large-scale changes.

## Step 2: Generate descriptors from metadata you already have

If you manage entities with the API or GitOps, don't write each descriptor by hand. Render descriptors from a template (Jinja2 or similar) filled in from data you already have, like a service registry, a deployment manifest, or repository metadata. The template enforces your conventions, so every entity gets the same tag format, parent domain, groups, owners, and links.

With GitOps, teams usually commit the rendered descriptors to a central `.cortex/catalog` repository. With the API, your pipeline sends the rendered descriptor to the [Create or update entity](https://docs.cortex.io/api/rest/catalog-entities/create-or-update-entity.md) endpoint.

For example, a template might render a descriptor like this:

```yaml theme={null}
openapi: 3.0.1
info:
  title: Payments API
  description: Handles payment processing and refunds.
  x-cortex-tag: payments-api
  x-cortex-type: service
  x-cortex-parents:
    - tag: payments-domain
  x-cortex-groups:
    - payments
    - tier-1
  x-cortex-owners:
    - type: group
      name: payments-team
      provider: CORTEX
  x-cortex-git:
    github:
      repository: my-org/payments-api
  x-cortex-link:
    - name: Runbook
      type: DOCUMENTATION
      url: https://wiki.example.com/payments-api/runbook
```

For the full list of fields, see [Defining entities via YAML file](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/yaml).

**Start from what's already in Cortex.** If you're moving existing entities to a templated approach, export their current descriptors first and use them as the starting point. With the [Cortex CLI](https://docs.cortex.io/configure/cortex-command-line-interface.md), run `cortex catalog descriptor -y -t <tag>` to print an entity's descriptor as YAML. You can also use the [Retrieve entity descriptor](https://docs.cortex.io/api/rest/catalog-entities/retrieve-entity-descriptor.md) and [List entity descriptors](https://docs.cortex.io/api/rest/catalog-entities/list-entity-descriptors.md) API endpoints.

**Use rendered YAML only.** Cortex can't process a descriptor that still contains template syntax like `{{ variable }}`. Render the descriptor in your pipeline, then commit or send the result.

## Step 3: Validate descriptors before they merge

How you validate depends on how you manage the entity type.

* **UI** - Cortex validates changes when you save, so no setup is needed.
* **API** - Before your pipeline writes a change, send the descriptor to the [Create or update entity](https://docs.cortex.io/api/rest/catalog-entities/create-or-update-entity.md) endpoint with `dryRun=true`. Cortex validates it and returns any errors or warnings without changing the entity. Only write the change if the dry run passes.
* **GitOps** - Open a pull request for every descriptor change instead of pushing straight to the default branch.
  * **GitHub** - The Cortex GitHub app includes a linter that checks descriptors on pull requests and comments with any issues. In [GitOps settings](https://docs.cortex.io/configure/settings/gitops-settings.md#linter-checks), choose which repositories it runs on and whether warnings fail the check.
  * **Other Git providers** - Add a CI step that calls the Create or update entity endpoint with `dryRun=true`.
  * After a change merges, confirm Cortex processed it on the [GitOps logs](https://app.getcortexapp.com/settings/gitops-logs) page. An entry that shows `0 entities` usually means UI editing is still enabled for that entity type, or the repository isn't on the GitOps repository allowlist.
    To check a single descriptor, paste it into the [YAML linter](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/yaml/linter.md) under **Tools > YAML linter**.

## Step 4: Register new services where they're created

Build registration into the process that creates the service, and the catalog stays complete without anyone having to remember.

* **Trigger on your deployment manifests.** This pairs well with the template approach in Step 2. For example, a CI job (like a GitHub Action) that runs when a new production Helm chart, Terraform module, or other deployment manifest lands can render the descriptor from the manifest and repository metadata. With GitOps, the job opens a pull request against your catalog repository. With the API, it sends the descriptor to Cortex directly.
* **Use the Scaffolder.** When the [Scaffolder](https://docs.cortex.io/streamline/workflows/scaffolder) creates a new service from a template, it can register the service in Cortex at the same time. Turn on **Create cortex.yaml in git when creating a new service** to commit the descriptor to the new repository; otherwise Cortex adds the entity through the API.
* **Wrap it in a Workflow.** A [Workflow](https://docs.cortex.io/streamline/workflows) can run the Scaffolder, collect the service name and tag from a form, set custom data, and open tickets for follow-up work. See [Standardize new service creation](https://docs.cortex.io/guides/production-readiness/standardize-new-service-creation.md) for a complete example. Even if you build your own pipeline, it's a useful model for the steps to include.
* **Use Terraform.** If new services are provisioned through Terraform, use the `cortex_catalog_entity` resource in the [Cortex Terraform provider](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/terraform/terraform-provider.md) to create the entity alongside the infrastructure.

## Step 5: Keep ownership current

Ownership goes stale faster than any other catalog data, because people change teams and teams reorganize. For a deeper guide, see [Ownership best practices](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/ownership/ownership-best-practices.md).

* **Assign teams, not people.** Team owners survive turnover. Individual owners don't update when someone leaves.
* **Sync teams from the system that already knows your org,** such as an identity provider, Git provider, or HR system, instead of maintaining membership by hand.
* **Fill gaps with recommendations.** The [Ownership tool](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/ownership/assigning-owners-to-entities.md#assigning-ownership-via-cortex-recommendations) recommends owners for unowned entities. If you manage ownership through GitOps, accepting a recommendation opens a pull request that adds the team to `x-cortex-owners`, so your repository stays the source of truth. See [Accepting recommendations for GitOps-managed entities](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/ownership/assigning-owners-to-entities.md#accepting-recommendations-for-gitops-managed-entities).
* **Replace the owners block when you update owners through the API.** Write the full `x-cortex-owners` block with the current owners instead of appending to it, so former owners don't pile up. The [Create or patch entity](https://docs.cortex.io/api/rest/catalog-entities/create-or-patch-entity.md) endpoint replaces arrays by default; don't set `appendArrays=true` for owner changes. If the entity is managed with GitOps, update `x-cortex-owners` in its `cortex.yaml` instead. An API change would be overwritten on the next commit.

## Step 6: Retire entities that no longer exist

A catalog full of deleted repositories and torn-down resources is as misleading as one with gaps.

* **Turn on auto-archival.** [Auto-archival](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/archiving-entities/auto-archive.md) archives entities automatically when they're no longer detected in an integration, when their YAML file is deleted, or both. Set the source for each entity type in **Settings > Entities**.
* **Set an auto-deletion policy for short-lived resources.** If you ingest resources that come and go often, an [auto-deletion policy](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/archiving-entities/auto-delete.md) permanently deletes archived entities after a set time, so they don't pile up.
* **Review entities that aren't detected anymore.** The [discovered entities list](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/discovery-audit.md) flags catalog entities that Cortex no longer finds in your integrations, such as **AWS resource not detected**. Archive or delete them from there.

> [!NOTE]
>
> Changing an entity's `x-cortex-tag` creates a new entity instead of renaming the existing one. To change a tag, archive or delete the old entity, then create the new one.

## Step 7: Catch drift before it spreads

Automation covers most changes, but not all of them. Build in checks that surface what slipped through.

* **Review discovered entities on a schedule.** Check the [discovered entities list](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/discovery-audit.md) regularly for new repositories and resources that haven't been imported. Ignore the ones you don't need to track, like test repositories, so the list stays short.
* **Score catalog completeness.** Create a Scorecard that checks the fields your conventions require, so incomplete entities show up as tasks on their owners' homepages. Start from the **Ownership Verification** template in [Create a Scorecard](https://docs.cortex.io/standardize/scorecards/create.md#option-1-use-a-template), or add a rule like `ownership.teams().length > 0` to an existing Scorecard.
* **Run recurring data verification.** A [data verification period](https://docs.cortex.io/configure/settings/entity-settings/verification.md) asks owners to confirm that their entities' details are still accurate. Schedule one after large imports or reorgs, and on a regular cadence after that. See [Data verifications for audit readiness](https://docs.cortex.io/guides/compliance/data-verification.md) for an example.

## Related resources

* [Using GitOps for Cortex](https://docs.cortex.io/configure/gitops/using-gitops-for-cortex.md)
* [Catalog entities API](https://docs.cortex.io/api/rest/catalog-entities)
* [Defining entities via YAML file](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/yaml)
* [Discovered entities](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/discovery-audit.md)
* [Ownership best practices](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/ownership/ownership-best-practices.md)
* [Archiving entities automatically](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/archiving-entities/auto-archive.md)
* [Standardize new service creation](https://docs.cortex.io/guides/production-readiness/standardize-new-service-creation.md)


## Related topics

- [Catalogs](https://docs.cortex.io/guides/product-areas/catalogs.md)
- [Archiving entities automatically](https://docs.cortex.io/ingesting-data-into-cortex/entities-overview/entities/archiving-entities/auto-archive.md)
- [Data verification](https://docs.cortex.io/guides/product-areas/data-verification.md)
