# Prometheus

{% hint style="info" %}
Cortex connects to many third-party vendors whose system interfaces frequently change. As a result, integration behavior or configuration steps may shift without notice. If you encounter unexpected issues, check with your system administrator or refer to the vendor's documentation for the most current information. Additionally, integration sync times vary and are subject to scheduling overrides and timing variance.
{% endhint %}

## Overview

[Prometheus](https://prometheus.io/) is an open-source monitoring and analytics platform that allows customers to analyze, visualize, automate, and alert on metrics data.

Integrating Cortex with Prometheus allows you to:

* [View SLO information](#view-prometheus-data-in-entity-pages) from Prometheus on entity pages in Cortex
* Create [Scorecards](#scorecards-and-cql) that track progress and drive alignment on projects involving Prometheus SLOs

## How to configure Prometheus with Cortex

There are two options for integrating Prometheus: the default configuration method and Cortex Axon Relay, a relay broker allows you to securely connect your on-premises Prometheus data.

### Prerequisite

Before getting started, set up [basic authentication](https://prometheus.io/docs/guides/basic-auth/) credentials in Prometheus.

{% tabs %}
{% tab title="Default configuration" %}
**Configure the integration in Cortex**

1. In Cortex, navigate to the [Prometheus settings page](https://app.getcortexapp.com/admin/integrations/prometheus):
   * Click **Integrations** from the main nav. Search for and select **Prometheus**.
2. Click **Add integration**.
3. Configure the Prometheus integration form:
   * **Account alias**: Enter your account alias.
   * **Username** and **Password**: Enter your Prometheus basic auth credentials.
   * **Host**: Enter your self-managed Prometheus hostname.
   * **Tenant ID**: Optionally, enter your tenant ID.
     * If you have multiple tenants, you can enter an ID here to monitor a specific tenant.
4. Click **Save**.
   {% endtab %}

{% tab title="Relay broker" %}
**Configure Prometheus with Cortex Axon Relay**

See [Internally hosted integrations](https://docs.cortex.io/ingesting-data-into-cortex/integrations/axon-relay) for instructions. Make sure to follow the Prometheus-specific instructions for the docker-compose.yml file.
{% endtab %}
{% endtabs %}

**Configure the integration for multiple Prometheus accounts**[**​**](https://docs.cortex.io/docs/reference/integrations/prometheus#configure-the-integration-for-multiple-propsintegration-accounts)

The Prometheus integration has multi-account support. You can add a configuration for each additional by repeating the process above.

Each configuration requires an alias, which Cortex uses to correlate the designated with registrations for various entities. Registrations can also use a default configuration without a listed alias. You can edit aliases and default configurations from the Prometheus page in your Cortex settings. Select the edit icon next to a given configuration and toggle **Set as default** on. If you only have one configuration, it will automatically be set as the default.

## Connecting Cortex entities to Prometheus

### Linking SLOs in Cortex

You can create and manage SLOs by listing relevant SLIs through queries.

```yaml
x-cortex-slos:
  prometheus:
    - errorQuery: sum(rate(http_server_requests_seconds_count{code=~"(5..|429)"}[5m]))
      totalQuery: sum(rate(http_server_requests_seconds_count[5m]))
      slo: 99.95
      alias: my-prometheus-instance # alias is optional and only relevant if you have opted into multi account support
      name: my-slo-name
```

| Field      | Description                                                                                                                                                                                                         |
| ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| errorQuery | Query that indicates error events for your metric.                                                                                                                                                                  |
| totalQuery | Query that indicates all events to be considered for your metric.                                                                                                                                                   |
| slo        | Target number for SLO.                                                                                                                                                                                              |
| alias      | Ties the SLO registration to a Prometheus instance listed under Settings → Prometheus. The alias parameter is optional, but if not provided the SLO will use the default configuration under Settings → Prometheus. |
| name       | The SLO's name in Prometheus. The name parameter is optional.                                                                                                                                                       |

### How Cortex calculates Prometheus SLOs

When Cortex gets an SLO from Prometheus, the following query is calculated for it:

```
(1 - ({errorQuery}) / ({totalQuery}))
```

This value is calculated and resolved on an hour window, and calculated back for 7 days. Cortex averages the value for each 1-hour window, then averages each of those hourly averages across the lookback period, before displaying it in your Cortex workspace.

The value is updated when the entity page is loaded and when Scorecards are evaluated.

## Using the Prometheus integration

### View Prometheus data in entity pages

When an SLO is defined in an entity's descriptor, you'll see detailed data about SLOs in the **Monitoring** page in the sidebar of an [entity details page](https://docs.cortex.io/ingesting-data-into-cortex/entities/details). See the SLO query, target, the current value for each SLO, and the period of time the SLO is being calculated for. For example, if the time listed is "7 days ago," then the SLO is looking at the time range starting 7 days ago to now\..

### Scorecards and CQL

With the Prometheus integration, you can create Scorecard rules and write CQL queries based on Prometheus SLOs.

See more examples in the [CQL Explorer](https://app.getcortexapp.com/admin/cql-explorer) in Cortex.

<details>

<summary>SLOs</summary>

SLOs associated with the entity via ID or tags. You can use this data to check whether an entity has SLOs associated with it, and if those SLOs are passing.

**Definition:** `slos: List<SLO>`

**Examples**

In a Scorecard, you can use this expression to make sure an entity is passing its SLOs:

```
slos().all((slo) => slo.passing) == true
```

Use this expression to make sure latency Service Level Indicator (SLI) value is above 99.99%:

```
slos().filter((slo) => slo.name.matchesIn("latency") and slo.sliValue >= 0.9999).length > 0
```

</details>

### View integration logs <a href="#still-need-help" id="still-need-help"></a>

## Still need help?[​](https://docs.cortex.io/docs/reference/integrations/aws#still-need-help) <a href="#still-need-help" id="still-need-help"></a>

The following options are available to get assistance from the Cortex Customer Engineering team:

* **Email**: <help@cortex.io>, or open a support ticket in the in app Resource Center
* **Slack**: Users with a connected Slack channel will have a workflow added to their account. From here, you can either @CortexTechnicalSupport or add a `:ticket:` reaction to a question in Slack, and the team will respond directly.

Don’t have a Slack channel? Talk with your Customer Success Manager.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.cortex.io/ingesting-data-into-cortex/integrations/prometheus.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
