Schedule Hazelcast pods

Kubernetes supports several policies for assigning pods to nodes. Operator uses the same policies to allow or disallow specified assignments for Hazelcast members and Management Center.

For more information about scheduling policies, see the Kubernetes documentation.

Node selector

A node selector is a hard requirement that defines the nodes on which pods should run. If no node matches the requirements then the pods are not scheduled to run. To define the nodes on which Hazelcast pods should run, use the nodeSelector field.

For example, to run Hazelcast pods on a node in the us-west1 region, apply the following configuration. This uses the built-in topology.kubernetes.io/region label to specify that the pod should be run on nodes in the us-west1 region.

Hazelcast
Management Center

apiVersion: hazelcast.com/v1alpha1
kind: Hazelcast
metadata:
  name: hazelcast
spec:
  scheduling:
    nodeSelector:
      topology.kubernetes.io/region: us-west1

apiVersion: hazelcast.com/v1alpha1
kind: ManagementCenter
metadata:
  name: managementcenter
spec:
  scheduling:
    nodeSelector:
      topology.kubernetes.io/region: us-west1

For more information about node selectors, see the Kubernetes documentation.

Node affinity

Node affinity allows you to define both hard and soft requirements for the nodes on which pods should run. To define a node affinity, use the nodeAffinity field.

For example, you can set a hard requirement for Hazelcast pods to run only on nodes with AMD64 architecture and Linux operating system, and a soft requirement to prefer nodes in us-west1 or us-west2 regions.

Hazelcast
Management Center

apiVersion: hazelcast.com/v1alpha1
kind: Hazelcast
metadata:
  name: hazelcast
spec:
  scheduling:
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                  - amd64
              - key: kubernetes.io/os
                operator: In
                values:
                  - linux
        preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 15
            preference:
              matchExpressions:
                - key: topology.kubernetes.io/region
                  operator: In
                  values:
                    - us-west1
                    - us-west2

apiVersion: hazelcast.com/v1alpha1
kind: ManagementCenter
metadata:
  name: managementcenter
spec:
  scheduling:
    affinity:
      nodeAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/arch
                operator: In
                values:
                  - amd64
              - key: kubernetes.io/os
                operator: In
                values:
                  - linux
        preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 15
            preference:
              matchExpressions:
                - key: topology.kubernetes.io/region
                  operator: In
                  values:
                    - us-west1
                    - us-west2

For more information about node affinity, see the Kubernetes documentation.

Pod affinity and anti-affinity

You can specify pod affinity and pod anti-affinity to define which nodes Hazelcast instances should run on with respect to other pods. To define a pod affinity, use the podAffinity field.

For example, you may want to run Hazelcast pods on the same nodes as your application to improve performance. You may also prefer that Hazelcast pods run on separate nodes, but you don’t want to block the scheduler if the number of Hazelcast pods is greater than the number of nodes.

Hazelcast
Management Center

apiVersion: hazelcast.com/v1alpha1
kind: Hazelcast
metadata:
  name: hazelcast
spec:
  scheduling:
    affinity:
      podAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
                - key: app
                  operator: In
                  values:
                    - app1
            topologyKey: kubernetes.io/hostname
      podAntiAffinity:
        preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 10
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                  - key: app.kubernetes.io/instance
                    operator: In
                    values:
                      - hazelcast
              topologyKey: kubernetes.io/hostname

apiVersion: hazelcast.com/v1alpha1
kind: ManagementCenter
metadata:
  name: managementcenter
spec:
  scheduling:
    affinity:
      podAffinity:
        requiredDuringSchedulingIgnoredDuringExecution:
          - labelSelector:
              matchExpressions:
                - key: app
                  operator: In
                  values:
                    - app1
            topologyKey: kubernetes.io/hostname
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 10
              podAffinityTerm:
                labelSelector:
                  matchExpressions:
                    - key: app.kubernetes.io/instance
                      operator: In
                      values:
                        - hazelcast
                topologyKey: kubernetes.io/hostname

For more information about pod affinity, see the Kubernetes documentation.

Tolerations

Tolerations and corresponding taints are mechanisms to repel pods from certain nodes. Nodes can be tainted with key, value and an effect.

This example taints the node node1, preventing any Hazelcast pods from being scheduled on node1:

kubectl taint nodes node1 forbidden:NoSchedule

To allow the pods to run on the node node1, you need to add tolerations to the Hazelcast pods:

Hazelcast
Management Center

apiVersion: hazelcast.com/v1alpha1
kind: Hazelcast
metadata:
  name: hazelcast
spec:
  scheduling:
    tolerations:
      - key: "forbidden"
        operator: "Exists"
        effect: "NoSchedule"

apiVersion: hazelcast.com/v1alpha1
kind: ManagementCenter
metadata:
  name: managementcenter
spec:
  scheduling:
    tolerations:
      - key: "forbidden"
        operator: "Exists"
        effect: "NoSchedule"

For more information about tolerations and taints, see the Kubernetes documentation.

Topology spread constraints

Topology spread constraints control how pods are spread across the cluster.

The following configuration will ensure that Hazelcast pods will be spread across all the nodes evenly with at most one pod difference between the nodes.

Hazelcast

apiVersion: hazelcast.com/v1alpha1
kind: Hazelcast
metadata:
  name: hazelcast
spec:
  scheduling:
    topologySpreadConstraints:
      - maxSkew: 1
        topologyKey: kubernetes.io/hostname
        whenUnsatisfiable: DoNotSchedule
        labelSelector:
          matchLabels:
            app.kubernetes.io/name: hazelcast
            app.kubernetes.io/instance: hazelcast
            app.kubernetes.io/managed-by: hazelcast-platform-operator

Note that Management Center has only one instance, so it is not sensible to use topology spread constraints for the ManagementCenter Custom Resource.

For more information about topology spread constraints, see the Kubernetes documentation.