Using Kubernetes Jobs in Harness

Introduction

Kubernetes provides developers the ability to create a resource called Jobs. A Kubernetes Job creates “one or more Pods and ensures that a specified number of them successfully terminate.” (kubernetes.docs.io) When the number of successful completion of a task is reached, the job is done. A user can terminate the Job and it will clean up the associated pods.

Use Cases

  1. Create a new pod if the first pod fails or deleted due to a node hardware failure or node reboot
  2. Create a job that cleans up the configuration of an environment, to create a fresh environment for deployment
  3. Having a Job that spins down the replica count of service, to save on cost
  4. Run Tests against the deployed instance in Harness (i.e. Selenium Tests, Ruby Cucumber Tests, Any Scripted Tests)
  5. Simulate Traffic for Harness Continuous Verification

Sample Job YAML Flow

Example 1:

The Job below is a countdown, job. It will countdown from 9 to 1 and then print out done

apiVersion: batch/v1

kind: Job

metadata:

name: countdown

spec:

template:

metadata:

name: countdown

spec:

containers:

- name: counter

image: centos: 7

command:

- "bin/bash"

- "-c"

- "for i in 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 ; do echo $i ; done"

restartPolicy: Never

To see what the job is a user can run a command to view the status of it

kubectl describe jobs/countdown

To get the pod associated with the job

kubectl get pods --show-all

The one with the name countdown is the one we need to track the logs for!

kubectl logs <countdown_job_pod>

A User should see an output like so:

kubectl logs countdown-lc80g

15

14

13

12

11

10

9

8

7

6

5

4

3

2

1

Once the job is complete, a user can delete the job and the corresponding pod will go with it,

kubectl delete job countdown

Example 2:

This Job computes Pi to 2000 places and prints it out.

apiVersion: batch/v1

kind: Job

metadata:

name: pi

spec:

template:

spec:

containers:

- name: pi

image: perl

command: [ "perl" , "-Mbignum=bpi" , "-wle" , "print bpi(2000)" ]

restartPolicy: Never

backoffLimit: 4

To describe the job, run

kubectl describe jobs/pi

Retrieve the Pods that are associated with the job, all should have a prefix of “pi”

kubectl get pods --show-all

To see the logs of the job:

kubectl log <pi_pods>

In Harness, how do we use Jobs?

A Sample Job could be it’s on Service, or it can be part of a larger Service. In this example, I’ve created a Job as its own service.

To reference the Centos Library, create an Artifact Source for the service,

The manifest files for this service:

Configuration Variable for the Service

For quick copy:

countdown.yaml

apiVersion: batch/v1

kind: Job

metadata:

name: {{.Values.name}}

spec:

template:

metadata:

name: {{.Values.name}}

spec:

containers:

- name: counter

image: {{.Values.image}}

command:

- "bin/bash"

- "-c"

- "for i in 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 ; do echo $i ; done"

restartPolicy: Never

values.yaml

name: ${serviceVariable.JobName}

image: ${artifact.metadata.image}

Create a Rolling Workflow and delete the Rollout Deployment step.

Click on Add Command and Select “Apply”

Your Apply Step should look something like this, be sure to skip dry run because of the resource hasn’t been created yet!

!

Create a workflow variable called jobName and set the default value to your choosing. In the field, I have noticed some customers trigger Kubernetes Jobs in their Jenkins Pipelines and pass the name of the job through the trigger.

A shortcut to check if the jobs are complete can be written in a one-liner like so:

kubectl wait -- for =condition=complete --timeout=30s jobs/${workflow.variables.JobName} -n ${infra.kubernetes.namespace}

Your Final Workflow Should look like this:

And the Deployment Execution should look like this:

You can verify the output yourself, or in Harness, we do show that the Status of a job is completed:

Conclusion

With this basic setup, you should be able to configure a Kubernetes Job within Harness and have Harness manage and deploy it. You can get a status of the job when it is done and port it into an Email body, Jira Ticket, or a variable for later use.

Most users I’ve worked with have been using the Jobs to simulate tests after the Service has been deployed via Canary or Rolling Workflow. This technique isn’t limited to these workflow types, it can be leveraged in Blue Green as well.

Our Docs team has a documented processes as well: https://docs.harness.io/article/qpv2jfdjgm-run-kubernetes-jobs

1 Like