Kubernetes

CatalogDiscoveryScorecards

Summary

The Kubernetes integration provides deep visibility into the infrastructure of how your services are actually deployed. You can use the Kubernetes integration to:

• Easily one-click import services from multiple k8s clusters into Cortex
• Build Scorecard rules to enforce your org's k8s best practices
• View information from k8s in the catalog, letting you easily access information such as active replicas, currently deployed versions, resource requirements, and more.

Setup

In order to connect Cortex to your Kubernetes instance, you’ll need to install the Cortex K8s agent in your Kubernetes cluster. You can follow the instructions to do so in Settings → Kubernetes.

caution

If you do not see the Settings page you're looking for in the sidebar, you likely don't have the proper permissions and need to contact your admin.

Registration

Discovery

Entity tag discovery

By default, Cortex will use the entity tag (e.g. my-service) as the "best guess" for the Kubernetes resource. For example, if your entity tag is my-service, then the corresponding resource in Kubernetes should also be my-service.

If your Kubernetes resources don’t cleanly match the Cortex entity tag, you can override this in the Cortex Catalog Descriptor.

Metadata label discovery

Instead of using entity tag discovery, you can choose to have Cortex discover Kubernetes resources using their metadata labels. Specify a list of metadata labels in the Settings → Kubernetes page, then Cortex will discover all Kubernetes resources whose metadata labels contain a key in that list where the value equals the Cortex entity tag. For example, let's say you have two Cortex entities foo and bar, and a Kubernetes json blob

{
  "name": "lorem ipsum",
  "metadata": {
    "labels": {
      "app": "foo",
      "another": "bar"
    }
  }
}

Initially, foo and bar will have no Kubernetes resource mappings. If the list of metadata labels is set to ["app"], then entity foo will be associated with the Kubernetes resource. If the list is set to ["app", "another"], then both foo and bar will be associated with the resource.

Annotation

Alternatively, you can link your Kubernetes deployment to a Cortex entity by adding an annotation to your deployment metadata. Use cortex.io/tag as the key and use the value of x-cortex-tag in the Cortex entity's cortex.yaml as the value.

For example, if the cortex.yaml file contains

openapi: 3.0.0
info:
  title: My Service
  x-cortex-tag: my-service
  x-cortex-type: service
  description: This is my cool service.

Then the deployment.yaml file should be configured as

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-name
  namespace: my-namespace
  annotations:
    cortex.io/tag: my-service

Annotation customization

You can customize how Cortex maps your Kubernetes annotations in your deployment metadata to Cortex entity tags by setting a custom JQ expression in the Settings → Kubernetes page. For example, if your deployment.yaml has the metadata configuration

metadata:
  name: my-name
  namespace: my-namespace
  annotations:
    cortex.io/tag: my.service

and should be mapped to a Cortex entity whose cortex.yaml file has x-cortex-tag: my-service, you could set the JQ expression .metadata.annotations."cortex.io/tag" | gsub("\\."; "-") (which converts all periods in the deployment annotation tag to dashes) in the settings to achieve that mapping.

Catalog descriptor

If you need to override the automatic discovery, you can define the following block in your Cortex Catalog Descriptor.

Cortex accepts a list of K8s resources, which can be on different clusters or of different types.

Each Resource type has the same field definitions.

Field	Description	Required
identifier	namespace/name as found in Kubernetes	true
cluster	The name of the cluster, which is set when deploying the agent	false

Deployments

x-cortex-k8s:
  deployment:
    - identifier: namespace/name
      cluster: dev # optional
    - identifier: experiment/scratch
      cluster: dev
    - identifier: default/cortex
      cluster: prod

ArgoCD Rollout

x-cortex-k8s:
  argorollout:
    - identifier: namespace/name
      cluster: dev

StatefulSet

x-cortex-k8s:
  statefulset:
    - identifier: namespace/name
      cluster: dev

CronJob

x-cortex-k8s:
  cronjob:
    - identifier: namespace/name
      cluster: dev

Cortex k8s agent

The k8s integration is powered by a lightweight, custom agent we've developed. The agent is deployed into your cluster, and periodically sends information back to Cortex.

Basics

The k8s agent collects information from your cluster (such as the current list of deployments). Using a Cortex API key, it sends this information back to Cortex, where it's exposed through the dashboard.

The k8s agent is lightweight and adds negligible impact to your cluster. It's essentially a simple cron job that collects and sends information back to your Cortex account.

The agent collects information every five minutes, by default.

Deployment

To deploy the agent, you can use our helm chart which allows for an easy installation. The helm chart is provided to you upon request.

Security

Security of your cluster is extremely important to us (and to you!). The agent's push-model ensures that you do not need to expose your cluster to the public internet.

Additionally, the helm chart comes with a predefined ClusterRole that provides the correct RBACs, which are:

• Permissions: ["get", "watch", "list"]
• Resources: ["deployments", "services", "pods", "replicationcontrollers", "statefulsets", "rollouts", "cronjobs"]
• API groups: ["apps", "argoproj.io", "batch"]

Communication out of the cluster to Cortex happens over HTTPS. There is no inbound traffic to the agent.