April 1, 2021 - infrastructure kubernetes

Schedule Kubernetes CronJob to restart pods automatically

Xavier Geerinck

@XavierGeerinck

For a project I needed a way to automatically schedule pods to be deleted every 3 hours. This was due to the nature of the project where we wanted a cleanup in memory due to zombie processes appearing. It's not a clean fix, but it was necessary for production to run stable while we were patching things.

Setting this up is quite easy actually! Just create the script below and save it to a file of your preferences (e.g. deploy-cronjob.yaml), finally, applying it through kubectl apply -f deploy-cronjob.yaml.

What this YAML does:

  1. It creates the necessary Service Accounts and Roles and Role Bindings to have the permissions to let an automated job run kubectl delete
  2. It spins up a bitnami/kubectl container
  3. It runs the /bin/sh -c <your_command> that will get executed

⚠️ We utilize /bin/sh -c since we utilize command substitution which is only available in the Shell.

deploy-cronjob.yaml

# Apply With:
# kubectl apply -f deploy-cronjob.yaml
# Validate with:
# kubectl get cronjob
---
kind: ServiceAccount
apiVersion: v1
metadata:
name: deleting-pods
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: deleting-pods
namespace: default
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "patch", "list", "watch", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: deleting-pods
namespace: default
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: deleting-pods
subjects:
- kind: ServiceAccount
name: deleting-pods
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: deleting-pod-myname
namespace: default
spec:
concurrencyPolicy: Forbid
schedule: "0 */3 * * *" # At minute 0 past every 6 hours
jobTemplate:
spec:
backoffLimit: 2
activeDeadlineSeconds: 600
template:
spec:
serviceAccountName: deleting-pods
restartPolicy: Never
containers:
- name: kubectl
image: bitnami/kubectl
command: [ "/bin/sh", "-c" ]
args:
- 'kubectl delete pod $(kubectl get pod -l app=<your_label> -o jsonpath="{.items[0].metadata.name}")'

Did you enjoy reading? Or do you want to stay up-to-date of new Articles?

Consider sponsoring me or providing feedback so I can continue creating high-quality articles!

Xavier Geerinck © 2020

Twitter - LinkedIn