Datadog
Summary
Datadog is an observability service for cloud-scale applications, providing monitoring of servers, databases, tools, and services. You can use Datadog to drive insights into values such as:
- Service discovery
- Monitors
- SLOs
- Dependencies
Setup
In order to connect Cortex to your Datadog instance, you’ll need to add a Datadog Application Key and create a Datadog API key, and add it under Settings → Datadog.
If you do not see the Settings page you're looking for in the sidebar, you likely don't have the proper permissions and need to contact your admin.
Registration
Discovery
By default, Cortex will use the entity tag (e.g. my-service
) as the "best guess" for the Datadog tag. For example, if your entity tag is my-service
, then the corresponding tag in Datadog should also be my-service
.
If your Datadog tags don’t cleanly match the Cortex entity tag, you can override this in the Cortex Catalog Descriptor.
Catalog Descriptor
If you need to override the automatic discovery, you can define the following block in your Cortex Catalog Descriptor.
x-cortex-apm:
datadog:
serviceTags: # List of tags & values
- tag: service
value: brain
- tag: backend
value: brain
These tags are used to "discover" your monitors and SLOs. Cortex will look up monitors and SLOs by querying for tag:value OR tag:value ...
If you want to hardcode/override discovery altogether, use the following sections:
Monitors
Adding monitors let you see information about their current status directly in the Catalog, under the Datadog
section. You can find your monitors at https://app.datadoghq.com/monitors/manage.
The ID of a monitor is found in the URL when you click on a monitor in your Datadog dashboard, for example https://app.datadoghq.com/monitors/**<MONITOR_ID>**
.
info:
x-cortex-apm:
datadog: # List of monitor IDs.
monitors:
- 12345
- 67890
SLOs
Datadog SLOs can be listed in the Catalog under the Datadog
section. You can find the SLOs at https://app.datadoghq.com/slo.
The ID for the SLO can be found in the URL when you click on an SLO in the datadog dashboard, for example https://app.datadoghq.com/slo?slo_id=**<SLO_ID>**&timeframe=7d&tab=status_and_history
.
info:
x-cortex-slos:
datadog: # List of SLO ids
- id: 0b73859a3e2504bf09ad23a161702654
- id: 228499184a9efe34d4e4e9df838c7fa1
Cortex will do an OR
query on Datadog using service:brain OR backend:brain
to find the SLOs & Monitors.
Dependencies
Cortex automatically syncs dependencies from Datadog's tracing tool into Cortex. By default, we map
entries found in Datadog's tracing map using the Service Identifier (x-cortex-tag
).
For example, if you have two services "service-one" and "service-two" that have a dependency edge in Datadog tracing, we expect both services in Cortex to exist with the same service identifiers.
You can override this by setting service tags where tag
= service
, and value
= the name of the service as found in Datadog's dependency map.