Cortex Self-Managed
For organizations with strict security and compliance requirements, Cortex can be deployed on-premises into your own Kubernetes (k8s) cluster.
The Cortex installation is provided as a Helm chart.
Getting started with a Cortex self-managed account
To deploy Cortex on-premises, you will download and extract the Cortex Helm chart, edit its configuration, then install it from the local copy.
Prerequisites
Before getting started, you will need:
- A running k8s cluster, configured with permission to talk to the public internet (our Docker images are hosted on GitHub)
- Permission to add image pull secrets to the cluster
- kubectl installed and connected to your k8s cluster
- Helm Package Manager installed
- A GitHub personal access token (PAT) and a JWT license key, both provided to you by the Cortex team
Step 1: Configure your Kubernetes cluster
- Run the following command to configure your cluster to talk to Cortex:
kubectl create secret docker-registry cortex-docker-registry-secret \
--docker-server=ghcr.io \
--docker-username=cortex-image-bot \
--docker-password=<enter the GitHub PAT provided by Cortex> \
--docker-email=<yourname@example.com>
- Make sure to replace
<enter the GitHub PAT provided by Cortex>
with the value of your PAT, and replace<yourname@example.com>
with an email address.
- Run the following command to create a secret for your Cortex license:
kubectl create secret generic cortex-secret \
--from-literal ENTITLEMENTS_JWT=your_JWT
- Make sure to replace
your_JWT
with the value of your JWT license key.
Step 2: Set up a persistent database
Cortex requires a Postgres Database (version 15), and does not support other datastores. We recommend using your standard database provider, for example a hosted RDS or Cloud SQL instance.
Prerequisites
You will need the following resources:
- 2 instances of backend container, with 8GB memory per instance (2 cores recommended)
- 1 instance of frontend container
- < 500MB memory is generally sufficient, but you can adjust based on your available resources.
- Postgres DB v15 with 15GB storage and 4GB memory, with a maximum number of connections ≥100.
- The maximum number of connections should be ≥2x the backend's connection pool size, which is 50 by default (25 per instance).
Create a Postgres Database
Note: You must create (or utilize an existing) an instance of Postgres and the database within.
- Create a Postgres Database (version 15) with UTF8 encoding that will be accessible from your instance. Create the following and note their values, as you will need them in the next steps:
- The database instance and database with:
instance URI
,database name
, andinstance port
- The username and password to access the database
- The database instance and database with:
- Set up database user permissions on the default
public.schema
:- Add
ALL
on the public schema to create/modify tables - Add
INSERT
,SELECT
,UPDATE
,DELETE
on all tables in the public schema
- Add
Step 3: Fetch the helm chart
- In command line, run the following commands to download and unzip the latest version of the Cortex chart into a
cortex
directory:
helm repo add cortex https://helm-charts.cortex.io
helm pull cortex/cortex
tar -xvzf cortex-*.tgz
Step 4: Configure and install the helm chart
Note the following:
- The default values for resource allocation in the chart are provided as guidance, but you may need to adjust these depending on your resource consumption.
- Auth is disabled by default. Any users who want to access and test Cortex will have access to the dashboard through a “demo” user.
- The chart does not ship with built in TLS or certificates. If deployed into a cluster behind your existing load balancer with TLS, change the
protocol
value in the chart.
Step 4.1: Create a k8s secret for your database and license
-
Run the following command to create a k8s secret that will hold your database and license credentials:
kubectl create secret generic cortex-secret \
--from-literal DB_HOST="hostname (without jdbc:postgresql:// or the port)" \
--from-literal DB_PORT=5432 \
--from-literal DB_USERNAME=postgres \
--from-literal DB_NAME=postgres \
--from-literal DB_PASSWORD=mysecretpassword \
--from-literal ENTITLEMENTS_JWT='your_JWT_value'- Make sure to substitute the placeholders in this command with the actual values of your port, username, password, database, and JWT.
- See the Kubernetes documentation for more information on managing secrets using
kubectl
.
-
Continue to the next steps to configure the chart.
Step 4.2: Configure the helm chart
- In command line,
cd
to thecortex
directory, where you downloaded the helm chart in the previous steps. - Using a text editor such as
vi
, open thevalues.yaml
file. - You may need to make modifications, such as the following:
app.service.type
: Configure this field based on your infrastructure. For example, if you're using EKS, AKS, or GKE, configure the type asLoadBalancer
. If you're configuring this locally on your system, configureNodePort
.app.ingress
: Configure these fields if you are using a load balancer.- Under the
Hosted DATABASE
section:- Locate the line
# secret: cortex-secret
. Uncomment this line by deleting the#
.- Make sure to remove the additional space before
secret
to maintain the correct YAML formatting.
- Make sure to remove the additional space before
app.hostnames
: Configure the frontend and backend hostnames that the servers will be expecting to be reached on.- In a load balancer setup, the frontend hostname should be pointed at the external IP of the load balancer.
app.protocol
: If your cloud provider or load balancer automatically uses TLS or SSL, changeapp.protocol
tohttps
.- Note that this field does not set up TLS; it is used for URL generation.
- Locate the line
- Under the
Cortex API (Backend) Configuration
section:backend.replicaCount
: This field determines how many containers to create. Depending on your resource constraints, you may need to change this to1
.backend.containerConfiguration.resources
: Depending on your resource constraints, you may need to adjustrequests.memory
andlimits.memory
. By default, each container will consume a minimum of 8GB and a limit of 16GB.backend.containerConfiguration.livenessProbe.periodSeconds
andbackend.containerConfiguration.livenessProbe.timeoutSeconds
: You may need to extend these settings if your containers aren't starting after you install the helm chart.
- For any containers that you do not require for your use case, you can disabled them by setting their
enabled
field tofalse
.
The README contains tips on configuring Cortex when running locally in a Minikube instance.
Step 4.3: Install the helm chart
- To install the helm chart, run
helm install cortex .
at the root of the chart. - Run
kubectl get all
to see the status of the containers.- If the backend container makes multiple attempts to start and continues to fail, you may need to extend the
backend
timeout settings invalues.yaml
. - The containers may also fail to start if there are issues with outbound HTTP access or your GitHub PAT.
- If the backend container makes multiple attempts to start and continues to fail, you may need to extend the
Additional configuration for a cloud cluster installation
If you're installing Cortex into a cloud cluster, for example EKS or GKE, you may need to install required plugins to your cluster. For example, kubectl apply
the nginx plugin for AWS
Additional configuration options
Modifying the configuration after installing
After making additional modifications to values.yaml
, apply the changes by running helm upgrade
.
Self-Managed SSL Certificates
In some cases, you may want to enable your Cortex deployment to access managed services within your infrastructure (such as a self-hosted GitLab instance) that have their own SSL Certificates. Follow these steps to add a certificate to the trusted keystores:
- Create a kubernetes secret in the same namespace as your Cortex helm installation called
tls.cert
, containing the contents of the certificate file:
kubectl create secret generic tls-secret
--from-literal tls.crt="Replace with contents of cert file"
- Open your
values.yaml
file. EnableselfSignedCerts
and ensure the secret name is the same as the one created in the last step. In the example below, the secret is calledtls-secret
:
app:
...
backend:
enabled: true
...
selfSignedCerts:
enable: true
secret: tls-secret
worker:
enabled: true
...
selfSignedCerts:
enable: true
secret: tls-secret
- Repeat the previous step for each service in your
values.yaml
where you want to use the provided certificate. - Run
helm upgrade
.
Enabling Redis
Cortex provides optional Redis support by provisioning Redis in Kubernetes as part of the Helm chart. Enable Redis to gain support for outbound rate limiting of integration API calls.
Redis is enabled via a third-party Helm chart; a full reference to all of the possible settings can be found in this repository on GitHub.
To enable Redis, set redis.enabled
to true
in your values.yaml
:
redis:
enabled: true
It will only be accessible from within the Kubernetes cluster, and authentication will not be enabled by default.
Set authentication for Redis
If you enable auth for Redis:
- Create a secret:
kubectl create secret generic cortex-helm-redis \
--from-literal redis-password=$(openssl rand -base64 12 | tr '/@:' _)
- Configure the Redis subchart to use that secret:
redis:
auth:
enabled: true
existingSecret: cortex-helm-redis
existingSecretPasswordKey: redis-password
Enabling metrics
Prometheus
Prometheus metrics can be published from the backend and worker pods
by setting app.shared.prometheusMetrics.enabled
to true
in your
values.yaml
:
app:
shared:
prometheusMetrics:
enabled: true
With this enabled, Prometheus metrics will be published on port 8181
at the /actuator/prometheus
endpoint.
Debugging
Cortex provides a CLI to debug the on-prem installation.
All commands create a data
folder that has the data needed for the Cortex team to debug the issue.
Helm / K8s dump
To dump all the information about the installation, run the following command:
brain-freeze k8s dump [options]
Note: The CLI only dumps non-sensitive information. For the secrets
, the values are masked.
K8s logs
To fetch the logs for all Cortex related deployments, run the following command:
brain-freeze k8s logs [options]
Cortex-specific info
After a successful installation, the Cortex team might ask for settings or configurations on your setup. To get this information, run the following command:
/brain-freeze backend info --token="[CORTEX PAT]" --url="[BACKEND HOST]"
You can create a Cortex PAT in your Cortex workspace under Settings > Personal access tokens.
Product analytics
Cortex collects basic, anonymized data from self-managed customers. If you would like to opt out, please reach out to your Cortex Customer Success Manager.