Adding custom data
Last updated
Was this helpful?
Last updated
Was this helpful?
Custom data extends the out-of-the-box metadata that Cortex provides via integrations, enhancing your use of catalogs and augmenting details available for your entities. This data can also be used to write CQL queries and .
Custom data can be defined manually in the , added to entities programmatically via a , or sent through a webhook. To learn about sending data via webhook, see .
There are a few ways you can add custom data to an entity:
In the entity descriptor
This is optimal for low-volume, human-maintained data because it requires updating the entity's YAML when the data changes.
Via a POST
REST API endpoint
Using a is a better option for data that comes from an automated process, like a CI/CD pipeline.
A .
This can be done without auth or an explicit entity tag in the URL if you do not have access to the entity tag or the ability to add authentication headers.
The simplest way to describe information in the YAML is to define an object.
key
Key or title for the custom data. Anything defined before the :
will serve as the key
.
✓
value
Value for the custom data. Anything defined after the :
is the value
.
✓
Custom data can be of any type, including scalars (strings, numbers, booleans), objects, and lists.
When you add custom data to an entity's YAML, you'll see the key
and value
pairs on an entity's Custom data page.
Custom data added via entity descriptor will display with a YAML
tag.
You can add a description to data by explicitly defining it along with the key
and value
pairs. While key
and value
pairs can be defined with any terms when a description is not added, value:
must explicitly be defined when a description is included.
key
Key or title for the custom data. Anything defined before the :
will serve as the key
.
✓
value
Value for the key; should be defined explicitly with value:
✓
description
Description of the custom data
✓
Note: While the description
field is always optional, it is technically "required" to add a description to custom data.
Cortex applies a data source hierarchy to determine how to handle a case where the same key
is defined from multiple sources.
You can override keys defined in the YAML by adding a force=true
parameter when using an API, but when the entity descriptor is eventually re-processed, the data provided via the API will be overwritten.
Catalogs contain a lot of useful metadata about entities, including ownership and data from integrations. However, you may have your own custom fields that you want to include that would make exploring the catalog easier.
These can include things like:
Which AWS zones is this deployed in?
What databases does the entity consume?
When was the last successful CI run?
Custom data for cataloging makes the most sense when the data is more flexible.
See the for authentication details.
If a key has already been set through the , an API call will NOT overwrite the existing value. Instead, it will simply return the existing value.
Learn more in .
The is the source of truth. If a key is defined there, the API or webhooks cannot override the key.
The and approaches are at the same level and can override each other.
If the answers to these questions fit a pre-enumerated list, consider using . Groups are displayed on entity detail pages and can easily be applied in catalog filters.
Custom data can then be queried against by using the , explored with , or viewed on an .
Cortex makes it easy to write based on custom data that exists in your catalogs. When adding a rule, you can select Custom data from Choose an integration in the form editor and follow the flow. Cortex will automatically supply variables based on the custom data you've defined.
Using the to push data into Cortex and extend your Scorecards. You can send entire JSON payloads and use jq
to process them in a Scorecard, or use it as an input to a custom OPA Policy
as a Scorecard rule.