Moving from Node Groups to NodePools on Amazon EKS

Why Should You Transition from Node Groups to NodePools?

In the rapidly changing world of Kubernetes, moving from Amazon EKS Managed Node Groups to NodePools represents a significant shift in infrastructure management. Since its launch in November 2019, EKS Managed Node Groups have simplified the provisioning and management of EC2 instances. However, as the need for dynamic and cost-effective solutions increases, NodePools powered by Karpenter are becoming the preferred choice for developers.

Karpenter introduces features like dynamic node provisioning, which allows developers to scale their clusters more effectively. This flexibility is crucial for modern applications that must respond quickly to fluctuating workloads. Transitioning to NodePools enhances cost management and optimizes resource utilization.

What Are NodePools in EKS?

How Do NodePools Work?

NodePools provide a new approach to defining and controlling compute requirements for Kubernetes workloads. Unlike static Node Groups, NodePools offer granular control over node configurations, enabling the use of spot instances and Graviton architecture.

Here’s a quick overview of the benefits of using NodePools:

• Dynamic Provisioning: Automatically adjusts resources based on workload needs.
• Cost Efficiency: Utilizes spot instances for significant savings.
• Flexible Architecture: Supports various instance types, including ARM-based Graviton instances.
• Improved Management: Simplifies node management through Karpenter's intelligent scheduling.

How Can You Create a NodePool?

Step-by-Step Configuration Guide

To create a NodePool that leverages ARM-based Graviton instances and spot capacity types, define a YAML configuration file. Here’s a simplified example:

apiVersion: karpenter.sh/v1
kind: NodePool
metadata:
  name: arm-mixed-capacity
spec:
  disruption:
    consolidationPolicy: WhenEmptyOrUnderutilized
    consolidateAfter: 10m
    budgets:
      - nodes: "2"
        schedule: "0 9 * * mon-fri"
        duration: 8h
      - nodes: "10%"
  template:
    spec:
      terminationGracePeriod: 24h
      requirements:
        - key: kubernetes.io/arch
          operator: In
          values: ["arm64"]
        - key: karpenter.sh/capacity-type
          operator: In
          values: ["spot", "on-demand"]
        - key: node.kubernetes.io/instance-type
          operator: In
          values:
            - "m7g.medium"
            - "m7g.large"
            - "m7g.xlarge"
            - "m7g.2xlarge"
      nodeClassRef:
        group: eks.amazonaws.com
        kind: NodeClass
        name: default
      taints:
        - key: arch
          value: arm64
          effect: NoSchedule
  limits:
    cpu: "1000"
    memory: 1000Gi
  weight: 10

How to Apply the Configuration

After creating the arm-nodepool.yaml file, deploy it using the following command:

$ kubectl apply -f arm-nodepool.yaml

This command creates the NodePool defined in your configuration file.

How to Deploy Your Application?

Using Node Affinity and Taints

To ensure your application runs on the new NodePool, define a Deployment that specifies the appropriate node affinity. For instance, if you are deploying an Nginx application on ARM architecture, your deployment YAML might look like this:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: arm-app
  labels:
    app: arm-app
spec:
  replicas: 3
  selector:
    matchLabels:
      app: arm-app
  template:
    metadata:
      labels:
        app: arm-app
    spec:
      nodeSelector:
        kubernetes.io/arch: arm64
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              preference:
                matchExpressions:
                  - key: karpenter.sh/capacity-type
                    operator: In
                    values: ["spot"]
      containers:
        - name: nginx
          image: nginx:latest
          ports:
            - containerPort: 80
          resources:
            requests:
              cpu: 500m
              memory: 512Mi
            limits:
              cpu: 1000m
              memory: 1Gi

How to Apply the Deployment

Deploy your application using:

$ kubectl apply -f arm-deployment.yaml

How to Monitor Node Behavior?

Once your deployment is active, monitor node behavior and resource utilization with these commands:

• List Nodes:

$ kubectl get nodes -L kubernetes.io/arch,karpenter.sh/capacity-type

• View NodeClaims:

$ kubectl get nodeclaims

• Describe Pods:

$ kubectl describe pod -l app=arm-app | grep -A 20 Events

These commands provide insights into the architecture and capacity type of your nodes, ensuring everything functions as expected.

Conclusion

Transitioning from Node Groups to NodePools on Amazon EKS enables more dynamic and cost-efficient management of Kubernetes workloads. With Karpenter, developers can utilize features like spot instances, Graviton architecture, and automated provisioning to enhance resource utilization. By following the steps outlined in this blog post, you can easily set up a NodePool tailored to your application’s needs, helping you stay competitive in the cloud landscape.