LogoLogo
Login to CortexBook a DemoCortex Academycortex.io
  • Cortex Docs
  • Cortex Quick Start
  • Ingesting data into Cortex
    • Managing Entities
      • Adding entities
        • Add services
        • Add domains
        • Add teams
        • Add custom entity types
        • Defining dependencies
      • Entity details page
      • Defining ownership
      • Defining relationship types
      • Grouping entities
      • Adding external documentation
      • Adding Deploy data
      • Adding custom data
      • Viewing discovered entities
      • Archiving entities
      • Relationship graph
      • Using On-call Assistant for incidents
      • Managing Terraform infra in Cortex
    • Managing Catalogs
    • Integrations
      • Internally hosted integrations
      • ArgoCD
      • AWS
      • Azure DevOps
      • Azure Resources
      • BambooHR
      • Bitbucket
      • BugSnag
      • Buildkite
      • Checkmarx
      • CircleCI
      • ClickUp
      • Codecov
      • Coralogix
      • Custom webhook integrations
      • Datadog
      • Dynatrace
      • Entra ID (Azure AD)
      • FireHydrant
      • GitHub
      • GitLab
      • Google
      • Grafana
      • incident.io
      • Instana
      • Jenkins
      • Jira
      • Kubernetes
      • LaunchDarkly
      • Lightstep
      • Mend
      • Microsoft Teams
      • New Relic
      • Okta
      • Opsgenie
      • PagerDuty
      • Prometheus
      • Rollbar
      • Rootly
      • Sentry
      • ServiceNow
      • Slack
      • Snyk
      • SonarQube
      • Splunk Observability Cloud (SignalFx)
      • Splunk On-Call (VictorOps)
      • Sumo Logic
      • Veracode
      • Wiz
      • Workday
      • xMatters
  • Scorecards
    • Initiatives and Action items
      • Creating issues based on Initiatives
    • Scorecard rule exemptions
    • Scorecard rule filters
    • Scorecard examples
    • Scorecards as code
  • Reports
    • Executive report
    • All Scorecards report
    • Bird's eye report
    • Progress report
    • Report card
  • Eng Intelligence
    • Custom Metrics
    • Jira Metrics
    • Metrics Explorer (Beta)
  • Cortex Query Language (CQL)
    • Using CQL reports
    • Using JQ in Cortex
  • Workflows
    • Creating a Workflow
      • Workflows as code
    • Blocks
    • Running a Workflow
    • Registering a Scaffolder template
      • Scaffolder advanced usage
    • Using a Workflow to sync in ArgoCD
    • Kicking off a Jenkins pipeline in a Workflow
    • Calling internal service endpoints in a Workflow
  • Plugins
    • Creating a plugin
      • Creating a plugin proxy
    • Migrating Backstage plugins to Cortex
  • Engineering homepage
  • Workspace Settings
    • Using GitOps for Cortex
      • GitOps logs
    • Managing users
      • Roles and permissions
        • Custom roles
        • Team ownership entity editing
      • Configuring SSO
        • Microsoft Entra ID
        • Google
        • Other OIDC providers
        • Okta
          • Okta SCIM
      • Configuring identity mappings
      • Onboarding management
    • API keys, secrets, and tokens
      • Secrets
      • Personal tokens
    • Audit logs
    • Entity settings
      • Data verification
      • Auto archiving entities
    • IP allowlist
    • Notifications
      • Notification logs
    • Customizing your workspace
    • Using search in Cortex
  • Cortex API
    • REST API operations
      • API Keys
      • Audit Logs
      • Catalog Entities
      • Custom Data
        • Custom Data (Advanced)
      • Custom Events
      • Custom Metrics
      • Dependencies
      • Deploys
      • Discovery Audit
      • Docs
      • Eng Intel: User Labels
      • Entity Relationship Types (Beta)
      • Entity Relationships (Beta)
      • Entity Types
      • GitOps Logs
      • Groups
      • Initiatives
      • Integrations APIs
        • Azure Active Directory (Entra ID) API
        • Azure Resources API
        • AWS API
        • Azure DevOps API
        • CircleCI API
        • Coralogix API
        • Datadog API
        • GitHub API
        • GitLab API
        • incident.io API
        • LaunchDarkly API
        • New Relic API
        • PagerDuty API
        • Prometheus API
        • SonarQube API
      • IP Allowlist
      • Notification Logs
      • On call
      • Packages
      • Plugins
      • Queries
      • SCIM
      • Scorecards
      • Secrets
      • Team Hierarchies
      • Teams
      • Workflows
Powered by GitBook
On this page
  • Defining custom data
  • Data source hierarchy
  • Use cases
  • Cataloging
  • Scorecards

Was this helpful?

Export as PDF
  1. Ingesting data into Cortex
  2. Managing Entities

Adding custom data

Last updated 13 hours ago

Was this helpful?

Custom data extends the out-of-the-box metadata that Cortex provides via integrations, enhancing your use of catalogs and augmenting details available for your entities. This data can also be used to write CQL queries and .

Custom data can be defined manually in the , added to entities programmatically via a , or sent through a webhook. To learn about sending data via webhook, see .

Defining custom data

There are a few ways you can add custom data to an entity:

  • In the entity descriptor

    • This is optimal for low-volume, human-maintained data because it requires updating the entity's YAML when the data changes.

  • Via a POST REST API endpoint

    • Using a is a better option for data that comes from an automated process, like a CI/CD pipeline.

  • A .

    • This can be done without auth or an explicit entity tag in the URL if you do not have access to the entity tag or the ability to add authentication headers.

Editing the entity descriptor

Learn more about in the Managing entities documentation.

The simplest way to describe information in the YAML is to define an object.

x-cortex-custom-metadata:
  key-example: value-example
  arbitrary-custom-data: hippocampus
  regions-of-brain:
    value: 3
    description: cerebrum, cerebellum, brainstem
  cognition: true
Field
Description
Required

key

Key or title for the custom data. Anything defined before the : will serve as the key.

✓

value

Value for the custom data. Anything defined after the : is the value.

✓

Custom data can be of any type, including scalars (strings, numbers, booleans), objects, and lists.

When you add custom data to an entity's YAML, you'll see the key and value pairs on an entity's Custom data page.

Custom data added via entity descriptor will display with a YAML tag.

Adding descriptions

You can add a description to data by explicitly defining it along with the key and value pairs. While key and value pairs can be defined with any terms when a description is not added, value: must explicitly be defined when a description is included.

x-cortex-custom-metadata:
  regions-of-brain:
    value: 3
    description: cerebrum, cerebellum, brainstem
  brain-hemispheres:
    value: 2
    description: The brain has a left and a right hemisphere.
Field
Description
Required

key

Key or title for the custom data. Anything defined before the : will serve as the key.

✓

value

Value for the key; should be defined explicitly with value:

✓

description

Description of the custom data

✓

Note: While the description field is always optional, it is technically "required" to add a description to custom data.

Adding custom data via the Cortex API

You can pipe custom data directly into Cortex by POSTing to /api/v1/catalog/{tag}/custom-data where {tag}is the x-cortex-tag for a given entity.

The request body requires the following in JSON format:

Field
Type
Description
Required

key

string

Key or title for the custom data

✓

value

object

Value for the custom data

✓

description

string

Description of the custom data

If a key is defined in the entity descriptor, the API cannot override the key. You'll see YAML as the source value in the response body if you encounter this situation.

To explicitly overwrite values set in the YAML file, use the force=true flag as a query parameter.

If you find yourself using the force flag, it may be better to remove the field from the YAML file or update the value in the cortex.yaml file instead to maintain the source of truth.

Custom data added via API will display with a API tag.

Bulk upload entity keys

You can use the bulk upload endpoint to upload multiple keys for one or more entities.

PUT data in the following format to /api/v1/catalog/custom-data:

{
  "values": {
    "<entity-tag>": [
      {
        "key":"example-key",
        "value":"example value",
        "description":"A description of these data."
      }
    ],
    "<another-entity-tag>": [
      {
        "key":"example",
        "value":{
          "my-data":"my-value",
          "complex": "data",
          "v": 0
        }
      }
    ]
  }
}

You can include multiple key and value objects for each tag. The object conforms to the same shape as the single upload API.

Data source hierarchy

Cortex applies a data source hierarchy to determine how to handle a case where the same key is defined from multiple sources.

You can override keys defined in the YAML by adding a force=true parameter when using an API, but when the entity descriptor is eventually re-processed, the data provided via the API will be overwritten.

Use cases

Cataloging

Catalogs contain a lot of useful metadata about entities, including ownership and data from integrations. However, you may have your own custom fields that you want to include that would make exploring the catalog easier.

These can include things like:

  • Which AWS zones is this deployed in?

  • What databases does the entity consume?

  • When was the last successful CI run?

Custom data for cataloging makes the most sense when the data is more flexible.

Scorecards

See the for authentication details.

If a key has already been set through the , an API call will NOT overwrite the existing value. Instead, it will simply return the existing value.

Learn more in .

The is the source of truth. If a key is defined there, the API or webhooks cannot override the key.

The and approaches are at the same level and can override each other.

If the answers to these questions fit a pre-enumerated list, consider using . Groups are displayed on entity detail pages and can easily be applied in catalog filters.

Custom data can then be queried against by using the , explored with , or viewed on an .

Cortex makes it easy to write based on custom data that exists in your catalogs. When adding a rule, you can select Custom data from Choose an integration in the form editor and follow the flow. Cortex will automatically supply variables based on the custom data you've defined.

Using the to push data into Cortex and extend your Scorecards. You can send entire JSON payloads and use jq to process them in a Scorecard, or use it as an input to a custom OPA Policy as a Scorecard rule.

Custom webhook integrations
groups
Scorecard rules
custom data API
entity descriptor
entity descriptor
API
webhook
Scorecard rules
Custom webhook integrations
webhook
entity descriptor
REST API
REST API
CQL reports
entity's details page
API docs
entity YAML descriptors
Query builder