Cluster Node Configuration Recommendation

1. Master Configuration Recommendation

The specification of the Master is related to the scale of the cluster. The larger the scale of the cluster, the higher the specification of the Master required. The recommended Master node configuration for different cluster scales is as follows:

The default size of the UK8S Master node system disk is 40G (minimum), which is used to store ETCD information and related configuration files.

If the cluster scale is raised and there is a demand to upgrade the Master node configuration, please change the configuration one by one on the cloud server node management page. Each one needs to be upgraded to the same configuration.

Before upgrading to the next Master node, make sure the other two Master nodes are in the Ready state and the status of all the Kubernetes-related core components on the Master node is in the active state. For troubleshooting methods of the Master node core components, please refer to: Common Fault Handling of Node

2. How to Choose the Size of Node Configuration

UK8S cluster requires the Node configuration to be no less than 2C4G and the default system disk is 40G (minimum), used for storing related configuration files and so on.

About Node Resource Reservation Strategy

Before determining the configuration of UK8S Node nodes, you need to know that the UK8S Node reserves 1GB of memory and 0.2 CPU cores by default to ensure the stable operation of the system. These reserved resources are used for the system and Kubernetes-related service processes.

When the available memory is less than 5%, eviction will start based on the resource priority of the pods. The actual memory available for pods is approximately {Memory of Node} - 1GB - 5% (for example, with 4GB of memory, the available memory is about 2.8GB). Meanwhile, the number of pods that can be created on a single node is related to the number of CPU cores of the Node. The number of Pods = CPU cores × 8 (for example, 2 cores support up to 16 pods, and 4 cores support up to 32 pods).

Therefore, we recommend that the Node configuration is ≥ 2C4G, which is the basic configuration to ensure the normal operation of the cluster.

For storage resources, the system disk of the UK8S Node node. When creating a node, you can choose to mount a data disk (which can also be mounted on the host side after the node is created). If the node is mounted with a data disk, it will be used for Docker to store local images; otherwise, local images will be stored in the system disk. Please ensure that the disk has sufficient space to avoid automatic cleaning of images or pods triggered by insufficient space.

Recommendations for Production Environment Node Configuration Options

The configuration options for production environment Nodes should be comprehensively considered based on the total number of CPU cores used daily by the entire cluster, fault tolerance, and business types. The details are as follows:

Assuming the total number of CPU cores in the cluster is set to 240 (determined based on historical operation data) and can tolerate 10% errors, 10 UHosts with 24 cores each can be selected, and the peak operation load should not exceed 240 × 90% = 216 cores. If the tolerance is less than 5%, 15 UHosts with 16 cores each can be selected, so that even if one node fails, the remaining nodes can still support the normal operation of existing services (with workloads automatically migrated).

From the perspective of providing fault tolerance, the lower the node configuration, the more nodes there will be, and the corresponding availability will increase. However, there are two other drawbacks: first, excessive resources need to be reserved for K8S, causing waste; second, it is not conducive to container scheduling, and may even lead to some containers failing to be registered. An extreme example: 3 nodes with 8 cores each can create 6 Pods that require 4 cores reserved, while 12 nodes with 2 cores each cannot respond to a Pod request that requires 4 cores reserved.

Considering both factors of effective resource utilization and fault tolerance, without considering business hybrid deployment and the overall scale of the business, we recommend that the CPU of production environment Node nodes should be between 4 cores and 32 cores.

As for the CPU: Memory ratio, it is recommended to apply for appropriate machine types according to your own application types. For example, CPU-intensive services can apply for machine types with a ratio of 1:2, Java-based applications can apply for machine types with a ratio of 1:4 or 1:8. If different services are hybrid-deployed, it is best to label the Nodes and use nodeAffinity (node affinity) to reasonably schedule Pods.