11 min read

Learning Objectives

By the end of this module, you will be able to:

  • Separate what the managed control plane gives you from what the node pools in your VDC cost and carry
  • Design node pools around a node template that is fixed at creation and nodes that are replaced, not patched
  • Plan autoscaling with a floor of one warm node per autoscaled pool
  • Design around the four boundaries of the service: LoadBalancer services, log paths, network filtering and compliance attribution
  • Place the control plane for a sovereign design and choose the cluster layout for FinCorp

Unit 6.1: Kubernetes Platform Design

Introduction

Managed Kubernetes splits a cluster into two halves with different owners. The control plane is run by IONOS CLOUD, out of your sight. The node pools are servers in your own VDC, and most design decisions land there: their shape, their network, their maintenance and their cost.

The design work is to know where each half ends. A LoadBalancer service, a log stream, a security group or a compliance claim that works on a VM can behave differently on a cluster. This unit sets out the platform's behaviour and the boundaries, and fixes FinCorp's cluster layout. Units 6.2 and 6.3 build it.

1. Two Halves: Control Plane and Node Pools

1.1 What You Pay For and What Is Covered

The control plane is free of charge. You pay for the infrastructure that the cluster uses: the nodes are Compute Engine servers with Block Storage volumes, provisioned into your VDC. An extra cluster therefore adds no control-plane charge; its cost is its nodes. Dedicated Core node pools are eligible for Cloud Savings Plans (Unit 2.4).

The service levels follow the same split. The per-service uptime SLA of 99.95 percent covers the control plane, with the Kubernetes API as its scope. The node pools fall under the Compute Engine terms, because they are Compute Engine servers. Write both lines into the availability design: an API outage stops changes to the cluster, while running workloads depend on the nodes.

1.2 The Hidden Regional Control Plane

The control plane components, such as kube-apiserver, kube-scheduler and kube-controller-manager, are not visible to you and cannot be modified. You reach kube-apiserver only through its REST API. The control plane runs on VMs in a geo-redundant cluster in the chosen region, and traffic between the nodes and the control plane is secured by mutual TLS.

Two of its properties are fixed inputs to your design. First, each cluster runs two kube-apiserver replicas with 8 GiB of memory each, and a replica that reaches the limit is restarted. Workloads keep running, but operators and controllers that depend on the API can be disrupted, so very large numbers of objects and heavy list traffic are a design risk. Second, the Container Network Interface (CNI) is Calico, and you cannot choose or install another one. Calico also provides Kubernetes network policies, which matter in Section 3.3.

2. Node-Pool Design

2.1 The Node Template Is a Creation-Time Decision

A node pool is a group of nodes that share one configuration. Its node template sets the server type, Dedicated Core or vCPU, and the cores, RAM, availability zone, storage type (Essential or Performance) and storage size. Regardless of the server type, 1 provisioned core equals 2 Managed Kubernetes CPUs.

Cores, RAM, availability zone and storage type cannot be changed after the pool is created; the server type of an existing pool can be switched between Dedicated Core and vCPU. A new node shape is therefore a new pool: create it, move the workloads with labels, taints and node selectors, and delete the old pool. The same rule makes zone design explicit. Because the availability zone is a template field, a cluster that must survive the loss of one zone needs one pool per zone.

The service limits set the outer frame:

Limit Value
Pods per node 110 (the Kubernetes default)
Nodes per node pool 100; 20 recommended
Node pools per cluster 500; 50 recommended
Nodes per cluster 5,000
Volumes per node 20, often fewer, because each node has 24 PCI slots shared among all peripherals

As shown above, the recommended values are far below the hard limits. Section 2.2 explains why the recommended pool size matters.

2.2 Immutable Nodes and the Maintenance Window

Nodes are immutable. IONOS CLOUD does not install software updates on running nodes; it replaces them. During maintenance of a pool, nodes are replaced one by one, starting with the oldest: a new node is created, the old one is drained and then deleted. The pool is locked until the work is done.

The maintenance window starts at the time you set and remains open for four hours. Nodes that are not rebuilt within the window are replaced at the next scheduled maintenance, which is why pools of no more than 20 nodes are recommended. Version updates are yours to start; node pools may run an older Kubernetes version than the control plane, but the minor versions must not differ by more than 1. A version that is too old to support is updated by force, and affected users typically receive a support notification two weeks before.

Three design rules follow. Every workload must tolerate a node drain, so set Pod Disruption Budgets that allow progress. Public-pool nodes receive new public IP addresses when they are replaced; pre-specify a list of public IP addresses, so that new nodes take theirs from a known set that a partner allowlist can hold. And a rebuild needs headroom: while a failed node is rebuilt, the pool has an extra billable active node, and the rebuild needs server quota.

2.3 The Autoscaling Floor

The cluster autoscaler adds nodes when pods cannot be scheduled for lack of resources, and removes nodes that are underutilized for an extended period when their pods fit elsewhere. You set a minimum and a maximum per pool, between 1 and 100. The autoscaler never goes outside them, never exceeds the contract quota, and cannot reduce a pool to 0 nodes. Pools with and without autoscaling can share a cluster.

There is no scale-to-zero. The floor of every autoscaled pool is one warm node that you pay for around the clock, so a cluster with many small autoscaled pools has a high idle cost. Keep the number of pools small and put bursty batch work in one pool whose minimum is 1. Scale-down also has blockers: pods with local storage such as emptyDir block it unless they carry the annotation cluster-autoscaler.kubernetes.io/safe-to-evict: "true", and nodes running only DaemonSet or system pods may not be removed.

3. The Four Boundaries

3.1 A LoadBalancer Service Is a Static IP on One Node

A Service of type LoadBalancer does not provision an external load balancer. IONOS CLOUD assigns a pre-reserved static public IP address, or reserves a new one, and attaches it as a secondary IP address to one worker node. That node acts as the ingress node, and kube-proxy forwards traffic to the correct pod. If you do not specify a loadBalancerIP, the address is released when the service is deleted.

The consequences are measurable. Any single node provides up to 2 Gbit/s on its public interface. When kube-proxy forwards to another node, the NAT operation replaces the client IP with an internal node IP; externalTrafficPolicy: Local preserves it, but then the IP must sit on a node that runs the ingress pod. Scaling beyond one node means several LB IP addresses spread across ingress nodes and published in DNS. Private node pools do not support the LoadBalancer type at all.

The pattern is therefore one ingress controller exposed as the only LoadBalancer service, on IP addresses reserved in IP Management, with a separately provisioned Application Load Balancer (ALB) in front where you need managed Layer 7 routing (Unit 3.3). Unit 6.2 builds it.

3.2 Two Log Paths, Both Yours to Wire

Cluster events reach you through two separate paths. Audit logging is a cluster setting: when activated, audit logs are stored in an existing IONOS Cloud Object Storage bucket that you name. Application and node logs reach the Logging Service only if you ship them: a Fluent Bit agent that you deploy in the cluster sends them to a logging pipeline with the Kubernetes log source. Neither path is on by default, so the observability design must name the bucket, the pipeline and who reads each.

3.3 Network Filtering Ends at the Node Pool

Nodes of Managed Kubernetes node pools cannot be added to Network Security Groups (NSGs), so the tier security groups of Unit 3.2 do not reach the cluster. Filtering moves to three other places:

Layer Control
Kubernetes API Allowed IPs on the cluster restrict API access to an IP address range; empty means access from anywhere
Traffic between pods Kubernetes network policies, enforced by Calico
Traffic from outside The ALB in front, and the targets it may reach (Unit 3.3)
Who manages the cluster IAM group grants on the cluster, and the Create Kubernetes Clusters privilege (Unit 2.2)

As shown above, a cluster's filtering lives in Kubernetes objects and in the edge services, not in the VDC firewall model. Start every namespace with a default-deny network policy and allow flows explicitly.

3.4 Compliance Covers the Service, Not Your Workloads

Managed Kubernetes is within the scope of the ISO 27001 certificate on the basis of IT-Grundschutz, BSI-IGZ-0730-2025, which covers the provision and operation of the service by IONOS SE in Germany (Unit 1.4). For BSI C5, confirm the service in the attestation report before you cite it. Either way, the credential describes how IONOS CLOUD runs the platform. Your images, namespaces, RBAC rules, network policies and secrets are your controls, and your evidence pack must cover them separately.

4. Control-Plane Placement and Sovereignty

You create clusters in Frankfurt am Main (de/fra) or in the United States: Newark (us/ewr), Las Vegas (us/las) and Lenexa (us/mci). The control plane runs in the region you choose, and the location cannot be changed later. A cluster with private node pools is bound to one region, and its node pools must be in VDCs in the same location as the cluster.

For a sovereign design the location filter of Unit 1.4 applies to the control plane as well as to the nodes: the Kubernetes API and its secrets live where the control plane lives, and its audit trail lives in the Object Storage bucket you name for audit logging. FinCorp therefore creates clusters only in de/fra.

FinCorp's layout follows from Sections 1 to 3. A public cluster, fc-k8s-portal, runs the portal and partner API services in fc-prod-fra-core behind the ALB fc-alb-edge (Unit 6.2). A private cluster, fc-k8s-ai, runs the retrieval and inference workers in fc-prod-fra-ai, with no public node addresses (Unit 6.3). Both use Dedicated Core pools, one per zone, of at most 20 nodes, so that maintenance finishes in its window and the committed cores can sit in a Savings Plan.

Decision Summary

Decision Choose Because
Number of clusters One per trust zone (public edge, private AI) The control plane is free; clusters cost only their nodes
Node pool shape One pool per shape and per zone Cores, RAM, zone and storage type are fixed at creation
Pool size At most 20 nodes Maintenance replaces nodes within a four-hour window
Autoscaling Few autoscaled pools, minimum 1 No scale-to-zero; every autoscaled pool keeps one warm node
Exposure One ingress controller as the only LoadBalancer service, ALB in front A LoadBalancer service is a static IP on one node; private pools do not support it
Filtering Allowed IPs, network policies, IAM grants Node pool nodes cannot be NSG members
Logs Audit logs to a bucket, Fluent Bit to the Logging Service Both paths must be configured by you
Location de/fra for FinCorp The control plane runs in the chosen region, fixed at creation

Summary

Managed Kubernetes gives you a free, hidden, geo-redundant control plane in the region you choose, and puts everything else in your VDC as node pools of immutable Compute Engine servers. Design the pools around a template that cannot change, a four-hour maintenance window and an autoscaling floor of one node. Then place the controls that a VM design takes for granted: an ALB instead of a LoadBalancer service, your own log wiring, network policies instead of NSGs, and evidence for your workloads beside the provider's certificate.

Key Points:

  • The control plane is free; node pools are billable Compute Engine servers under the Compute Engine terms.
  • Cores, RAM, zone and storage type of a pool are fixed at creation; nodes are replaced, never patched.
  • Autoscaled pools range from 1 to 100 nodes and never scale to zero.
  • A LoadBalancer service attaches a static IP to one node, with up to 2 Gbit/s per node; private pools do not support it.
  • Node pool nodes cannot join NSGs; filter with Allowed IPs, Calico network policies and the edge services.

Important Terminology:

  • Ingress node: the worker node that holds a LoadBalancer service's static IP as a secondary address and forwards its traffic.
  • Node template: the server type and hardware settings of a node pool, set when the pool is created.