rekuberate-io/sleepcycles: Reclaim your unused Kubernetes resources

A sophisticated custom Kubernetes controller that shuts down and wakes up your Kubernetes workloads on user-defined schedules.

Introduction

In today’s world of cloud-native applications and dynamic workloads, efficient resource management is key. Kubernetes has revolutionized container orchestration, but managing the power consumption of resources within a cluster is still a challenge yet to be tackled. This is where rekuberate-io/sleepcycles comes into play, offering a powerful solution to optimize power consumption and cost spending by defining sleep and wake-up cycles for Kubernetes resources.

Playing with the words Kubernetes and recuperate, the word rekuberate was born. As defined in Merriam-Webster dictionary, recuperate means:

1.- to get back : REGAIN

2.- to bring back into use or currency : REVIVE

Nowadays, is predominantly used as term in automotive industry where recuperation means converting the kinetic energy during braking into electrical energy. This leads to a significant increase in range by recharging your battery by regaining energy otherwise lost.

By scheduling your workloads (Deployments, Stateful Sets, Cron Jobs & Horizontal Pod Autoscalers) you can depressurize your clusters, schedule resource-hungry tasks on times that are not impacting your business, decrease costs, reduce power consumption and lower your carbon footprint.

Cutting down the active hours of a workload — assuming you will shut it down 8 hours every night and keep it shut during the weekends — will lead to a reduction of a whopping 368 hours per month in a total of 730 billable hours.

How it works

You’ll need of course a Kubernetes cluster. You can use KIND or K3D to get a local cluster for testing, or run against a remote cluster. If you don’t have one in place you can follow plenty of guides that you can find in my articles list:

Install Kubernetes 1.29 using Vagrant in under 10 minutes

Core element of rekuberate-io/sleepcycles is a new Custom Resource (CR) that is called SleepCycle:

apiVersion: core.rekuberate.io/v1alpha1
kind: SleepCycle
metadata:
  name: sleepcycle-app-1
  namespace: app-1
spec:
  shutdown: "1/2 * * * *"
  shutdownTimeZone: "Europe/Athens"
  wakeup: "*/2 * * * *"
  wakeupTimeZone: "Europe/Dublin"
  enabled: true

Every SleepCycle has the following mandatory properties:

• shutdown: cron expression for your shutdown schedule
• enabled: whether this sleepcycle policy is enabled

and the following non-mandatory properties:

• shutdownTimeZone: the timezone for your shutdown schedule, defaults to UTC
• wakeup: cron expression for your wake-up schedule
• wakeupTimeZone: the timezone for your wake-up schedule, defaults to UTC
• successfulJobsHistoryLimit: how many completed CronJob Runner Pods to retain for debugging reasons, defaults to 1
• failedJobsHistoryLimit: how many failed CronJob Runner Pods to retain for debugging reasons, defaults to 1
• runnerImage: the image to use when spawn CronJob Runner pods, defaults to akyriako78/rekuberate-io-sleepcycles-runners

You should not add seconds or timezone information to you cron expressions. The former is not needed and the latter is taken care of shutdownTimeZone and wakeupTimeZone properties.

Ok, but how it *really* works?

1. The sleepcycle-controller watches periodically, every 1 minute, all the SleepCycle custom resources for changes (in all namespaces).
2. The controller, for every SleepCycle resource within the namespace app-1, collects all the resources that have been marked with the label rekuberate.io/sleepcycle: sleepcycle-app1.
3. It provisions, for every workload — in this case deployment deployment-app1 a CronJob for the shutdown schedule and optionally a second CronJob if a wake-up schedule is provided.
4. It provisions a ServiceAccount, a Role and a RoleBinding per namespace, in order to make possible for runner-pods to update resources' specs.
5. The Runner pods will be created automatically by the cron jobs and are responsible for scaling the resources up or down.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: app-1
  namespace: app-1
  labels:
    app: app-1
    rekuberate.io/sleepcycle: sleepcycle-app-1
spec:
  replicas: 9
  selector:
    matchLabels:
      app: app-1
  template:
    metadata:
      name: app-1
      labels:
        app: app-1
    spec:
      containers:
        - name: app-1
          image: traefik/whoami
          imagePullPolicy: IfNotPresent

The diagram above described the scenario of scheduling a Deployment. The same steps count for a StatefulSet and a HorizontalPodAutoscaler. There are two exceptions though:

a HorizontalPodAutoscaler will scale down to 1 replica and not to 0

a CronJob has no replicas to scale up or down, it is going to be resumed or suspended respectively.

See rekuberate-io/sleepcycles in action via K9S

Any workload in namespace kube-system marked with the labelrekuberate.io/sleepcycle will be ignored by the controller by design.

Integrating with ArgoCD

You can combine rekuberate.io/sleepcycles with applications provisioned with ArgoCD, as long as you disable self-healing when an automatic sync policy is enabled, otherwise ArgoCD's sync mechanism will work against the shutdown and wakeup cronjobs of rekuberate.io/sleepcycles In that case ArgoCD will always automatically revert to the desired state described in the git manifests overpowering the effect of the sleepcycle schedule.

Leave SELF HEAL option, under SYNC POLICY, unchecked.

An ArgoCD application with Manual or Automatic sync policy without self-healing is the only recommended configuration in order to achieve a synergistic relationship between ArgoCD and rekuberate.io/sleepcycles.

There is a GitHub repository in place, with all the necessary artifacts to deploy via ArgoCD an nginx application and a preconfigured sleepcycle. You can find the git repo here and experiment yourself.

See rekuberate-io/sleepcycles in action in ArgoCD dashboard

Future work

Yet another project is on rails, rekuberate-io/carbon, a custom Kubernetes controller that will measure and attribute the power consumed by every individual Pod and combine them with the corresponding carbon intensity coefficient of your DC at the period of measurement in order to calculate the actual CO2 emission footprint of your workloads. Transforming your workloads and consequently Kubernetes to a carbon-aware platform and shifting your resources, temporally or spatially, by aiming to minimize your emissions footprint will be of paramount importance in the years to come.

If you find rekuberate-io/sleepcycles interesting, don’t forget to ⭐️ the repository in GitHub and give a shout out to this article.

GitHub - rekuberate-io/sleepcycles: Define sleep & wake up cycles for your Kubernetes resources…

Stay tuned…