Connect Kubernetes

CostReveal divides what each node costs, as the bill states it, among the workloads that ran on it, by the CPU, memory and GPUs they requested or used of that node's capacity, so a core on a costly node costs more than one on a cheap node, and a GPU node's GPUs are paid for by the workloads that requested them. Capacity no workload took is idle, and shown on its own, never spread across workloads.

What it needs

• The cluster's nodes on a connected bill. A cluster's cost is found by a tag every node carries: kubernetes.io/cluster/<name> with the value owned for Amazon EKS, goog-k8s-cluster-name for Google GKE, or any tag you name. On AWS, activate the tag as a cost allocation tag so the bill carries it.
• The OpenTelemetry Collector in the cluster, from its official Helm chart, read only. CostReveal installs nothing of its own, and nothing runs inside your workloads.
• An API key with the scope Send data only, kept in a Kubernetes secret. It can read nothing.

Set it up

1. In Kubernetes, choose Add cluster: name it, and say where it runs, which fills in its billing tag.
2. Create the key in Settings, API keys.
3. Follow Set up the Collector: create the secret, and install the chart twice with the values shown, at the version CostReveal is certified against. One Collector reads each node's allocatable CPU and memory, each container's requests, each pod's labels, and each node's provider ID and GPUs and each container's GPUs from the Node and Pod objects, from the Kubernetes API; one on every node reads what each container uses from its kubelet. Both send to CostReveal alone, naming the cluster. Each object is cut before it leaves the cluster: a node to its name, labels, provider ID, allocatable GPUs and, on GKE, its instance ID annotation; a pod to its UID, node and each container's name and GPUs, so nothing else of its spec, such as its environment, is sent.

A Collector you already run can send CostReveal the same metrics with one more exporter: k8s.container.cpu_request, k8s.container.memory_request, k8s.node.allocatable_cpu and k8s.node.allocatable_memory from the k8s_cluster receiver, with allocatable_types_to_report: [cpu, memory]; container.cpu.usage and container.memory.working_set from the kubeletstats receiver; labels from the k8sattributes processor under its own names, k8s.<source>.labels.<key>, where the source is pod, deployment, statefulset, daemonset, job, cronjob, namespace or node, with read access to those objects; and, so each node is priced by its own line on the bill and GPUs are counted, Node and Pod objects from the k8s_objects receiver, sent as logs. Cut each object to what CostReveal reads before it leaves, as the values shown do. The Helm chart's preset that extracts every pod label names each attribute by the bare label instead, which CostReveal cannot tell from other attributes, so it reads no owners from it.

How a node is matched to the bill

Each node's provider ID names the machine the cloud bills it as. A cluster line is a node's own when it bills that machine:

Cloud	Provider ID	The node's line on the bill
Amazon EKS	aws:///<zone>/<instance ID>	The instance's line, by its ID
Google GKE	gce://<project>/<zone>/<name>	The instance's line, by its name, or by the instance ID GKE states on the node
Azure AKS	azure:///subscriptions/…/virtualMachineScaleSets/<set>/virtualMachines/<n>	The scale set's line, which its nodes divide together
Oracle OKE	oci://<instance OCID>	The instance's line, by its OCID

The cluster's other lines, such as disks, load balancers and the control plane, are no node's, and are divided across the whole cluster. A node's line on a day it sent no metrics is unmeasured. A cluster whose nodes no cloud bills, or whose Collector sends no Node objects, is divided across the whole cluster, and the Kubernetes page says so.

How a day is divided

• Each day on its own. A day is summed once it is over, about an hour after midnight UTC, and is then closed: metrics for it that arrive later are refused, and say so.
• A workload takes the greater of what its containers requested and what they used of CPU and memory, each over the day, and the GPUs its containers requested for as long as they ran, NVIDIA's and AMD's, on each node it ran on. A node's line is divided among the workloads on that node, by that node's capacity; the cluster's shared lines by the whole cluster's.
• Each resource's part of the cost is divided by what of it was taken. On a node with GPUs, the GPUs' part is 80 percent unless you set the cluster's GPU part otherwise: a single GPU AWS instance costs about five times a general purpose one of the same CPU and memory. The rest, and all of a node with no GPUs, is CPU's and memory's: half each, unless you set the cluster's CPU part otherwise.
• Idle is what no workload took, on each node, GPUs no workload requested included. Unmeasured is what was billed on days with no metrics; it is not divided.
• Owners come from labels, read with the project's tag keys, so a pod labeled team: payments is the payments team's, as a tagged resource would be. A pod's own label comes first, then its workload's, a Deployment's, StatefulSet's, DaemonSet's, Job's or CronJob's, then its namespace's, then its node's, so a team that labels its Deployments, its namespaces or a node pool needs to label nothing else. Owners are decided on each node a workload runs on, so a DaemonSet's pods on nodes of different pools can have different owners. Explore shows what workloads explain as allocated. A workload with no such label stays unknown on that dimension. Custom dimensions are read from the same labels by their own tag keys.
• Explore groups by cluster, namespace and workload, with idle capacity and unmeasured days as rows of their own, and filters and crosses each with the provider, account, region and service, as any other dimension.
• Tags come first. A node line tagged for a team stays that team's; the cluster divides only what tags, named owners and assign rules leave unknown, before any split rule.

Changing a cluster, or removing it, explains every month again.

Who can set it up

Owners and admins add, change and remove clusters. Everyone who can see a project's costs sees its clusters and what they cost. Every change is recorded in Audit logs.