Kubernetes
Summary
The Kubernetes integration provides deep visibility into the infrastructure of how your services are actually deployed. You can use the Kubernetes integration to:
- Easily one-click import services from multiple k8s clusters into Cortex
- Build Scorecard rules to enforce your org's k8s best practices
- View information from k8s in the catalog, letting you easily access information such as active replicas, currently deployed versions, resource requirements, and more.
Setup
In order to connect Cortex to your Kubernetes instance, you’ll need to install the Cortex K8s agent in your Kubernetes cluster. You can follow the instructions to do so in Settings → Kubernetes.
If you do not see the Settings page you're looking for in the sidebar, you likely don't have the proper permissions and need to contact your admin.
Registration
Discovery
Entity tag discovery
By default, Cortex will use the entity tag (e.g. my-service
) as the "best guess" for the Kubernetes resource. For example, if your entity tag is my-service
, then the corresponding resource in Kubernetes should also be my-service
.
If your Kubernetes resources don’t cleanly match the Cortex entity tag, you can override this in the Cortex Catalog Descriptor.
Metadata label discovery
Instead of using entity tag discovery, you can choose to have Cortex discover Kubernetes resources using their metadata labels. Specify a list of metadata labels in the Settings → Kubernetes page, then Cortex will discover all Kubernetes resources whose metadata labels contain a key in that list where the value equals the Cortex entity tag. For example, let's say you have two Cortex entities foo
and bar
, and a Kubernetes json blob
{
"name": "lorem ipsum",
"metadata": {
"labels": {
"app": "foo",
"another": "bar"
}
}
}
Initially, foo
and bar
will have no Kubernetes resource mappings. If the list of metadata labels is set to ["app"]
, then entity foo
will be associated with the Kubernetes resource. If the list is set to ["app", "another"]
, then both foo
and bar
will be associated with the resource.
Annotation
Alternatively, you can link your Kubernetes deployment to a Cortex entity by adding an annotation to your deployment metadata. Use cortex.io/tag
as the key and use the value of x-cortex-tag
in the Cortex entity's cortex.yaml
as the value.
For example, if the cortex.yaml
file contains
openapi: 3.0.0
info:
title: My Service
x-cortex-tag: my-service
x-cortex-type: service
description: This is my cool service.
Then the deployment.yaml
file should be configured as
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-name
namespace: my-namespace
annotations:
cortex.io/tag: my-service
Annotation customization
You can customize how Cortex maps your Kubernetes annotations in your deployment metadata to Cortex entity tags by setting a custom JQ expression in the Settings → Kubernetes page. For example, if your deployment.yaml
has the metadata configuration
metadata:
name: my-name
namespace: my-namespace
annotations:
cortex.io/tag: my.service
and should be mapped to a Cortex entity whose cortex.yaml
file has x-cortex-tag: my-service
, you could set the JQ expression .metadata.annotations."cortex.io/tag" | gsub("\\."; "-")
(which converts all periods in the deployment annotation tag to dashes) in the settings to achieve that mapping.
Catalog descriptor
If you need to override the automatic discovery, you can define the following block in your Cortex Catalog Descriptor.
Cortex accepts a list of K8s resources, which can be on different clusters or of different types.
Each Resource type has the same field definitions.
Field | Description | Required |
---|---|---|
identifier | namespace/name as found in Kubernetes | true |
cluster | The name of the cluster, which is set when deploying the agent | false |
Deployments
x-cortex-k8s:
deployment:
- identifier: namespace/name
cluster: dev # optional
- identifier: experiment/scratch
cluster: dev
- identifier: default/cortex
cluster: prod
ArgoCD Rollout
x-cortex-k8s:
argorollout:
- identifier: namespace/name
cluster: dev
StatefulSet
x-cortex-k8s:
statefulset:
- identifier: namespace/name
cluster: dev
CronJob
x-cortex-k8s:
cronjob:
- identifier: namespace/name
cluster: dev
Cortex k8s agent
The k8s integration is powered by a lightweight, custom agent we've developed. The agent is deployed into your cluster, and periodically sends information back to Cortex.
Basics
The k8s agent collects information from your cluster (such as the
current list of deployments
). Using a Cortex API key, it sends this information
back to Cortex, where it's exposed through the dashboard.
The k8s agent is lightweight and adds negligible impact to your cluster. It's essentially a simple cron job that collects and sends information back to your Cortex account.
The agent collects information every five minutes, by default.
Deployment
To deploy the agent, you can use our helm chart which allows for an easy installation. The helm chart is provided to you upon request.
Security
Security of your cluster is extremely important to us (and to you!). The agent's push-model ensures that you do not need to expose your cluster to the public internet.
Additionally, the helm chart comes with a predefined ClusterRole
that provides the correct RBACs, which are:
- Permissions:
["get", "watch", "list"]
- Resources:
["deployments", "services", "pods", "replicationcontrollers", "statefulsets", "rollouts", "cronjobs"]
- API groups:
["apps", "argoproj.io", "batch"]
Communication out of the cluster to Cortex happens over HTTPS. There is no inbound traffic to the agent.