Supercharge Continuous Delivery in Jenkins with Workflow

on

This article is written for and published on SitePoint.

Recently, I wrote a series of articles about Jenkins. In these articles, we took a quick look at how we can assure the quality of our PHP projects. In this article, we’ll take a dive into the Jenkins plugin “Workflow”, an open source plugin created and maintained by CloudBees.

As enterprises scale up their use of Jenkins and bring more and more workflows into their systems, automating them can become a challenge. The Workflow plugin helps to build and automate complex workflows. By scripting, you can mix multiple actions like a parameterized trigger, copied artefacts, promoted builds and conditional build steps, for example. Through the Workflow plugin, it becomes relatively easy to write such a script.

Installing and creating a workflow

Before you start, you might want to make sure you have basic knowledge of Jenkins by reading my previous articles. If you haven’t done so, I suggest to keep those articles open as a reference.

Let’s get started by installing the Workflow plugin to get a good view on this particular plugin. Within the plugin page, search for Workflow: Aggregator and install it. All the dependencies will be installed automatically for you. When done, make sure you restart Jenkins.

Now it’s time to create a new job by clicking new item in the left menu. Fill in the form and make sure you pick workflow as your type. When done, submit the form and head straight to the configuration page for this job.

Workflow

The configuration looks very basic from the outside, but it’s very powerful. The most important part is the script field, in which you’ll script your workflow. Scripting is done in the groovy language. Let’s start with the old fashioned hello world example, so we get the very basics of this mighty plugin. Add the following content to your script field and click save.

echo 'Hello SitePoint!'

If you start a build and you take a look at the console output, you’ll see the following content.

Started by user Peter Nijssen
Running: Print Message
Hello SitePoint!
Running: End of Workflow
Finished: SUCCESS

Of course, it’s nice to see we can echo a message to the console, but we want to script our whole build process. Let’s assume we want to build our project Jumph, from the previous series of articles.

If you’re new to the groovy language like I am, you might wonder how to start. CloudBees helpfully added a snippet generator to the plugin, which will help you start quite fast. Below the script field, you’ll see a snippet generator.

Mark the checkbox in front of snippet generator and select build from job. Fill in Jumph and click generate groovy. You will get the following output, which you can insert in your script text area.

build 'Jumph'

Workflow

If you now run a build, you’ll notice Jumph is actually being kicked off. Nice! However, we could of course have started the build process of the project ourselves. This however is a simple example of a so-called step. A step can also be just the archiving the artefacts of your project or sending an email when done. Let’s take a deep dive into workflow, to get a better understanding of it’s power.

Let’s say we want to build the project for deployment. We want to create a tar.gz file which later can be used for deployment. With Workflow, we can script this process.

def src = 'https://github.com/peternijssen/Jumph.git'

stage 'Build'
node {
    git url: src
    sh 'export SYMFONY_ENV=prod && composer install --no-dev --optimize-autoloader'
    sh 'bower install'
}

In the above script, we first define our URL for the git repository. Within the current node, we pull in this git repository and run composer install while we set the Symfony environment to production. Next we run Bower to install our front-end dependencies. If we were to run this script, our project would be prepared to be deployed.

Perhaps you have a tool like Gulp or Grunt running along to prepare your front-end. You can easily expand this script to run multiple scripts to prepare your application for deployment.

The next step we want to do is pack the application in a tar.gz file and move it to our public root, so it can be downloaded.

def timestamp = new Date().format("yyyyMMddHHmmss")
def fileName = 'jumph-' + timestamp + '.tar.gz'

stage 'Package', concurrency: 1
node {
    sh 'tar -zcvf ' + fileName + ' app/ bin/ src/ vendor/ web/'
    sh 'cp ' + fileName + ' /srv/builds/'
   mail body: 'A new build is awaiting: ' + fileName, subject: 'New build available', to: 'example@sitepoint.com'
}

We’re also sending an email and making sure our file name includes a timestamp, so we’ll always be aware of when the file was actually created.

If we think we’re going to use this part more than once, we can wrap it in a ‘function’.

node {
    packageApp(fileName)
   mail body: 'A new build is awaiting: ' + fileName, subject: 'New build available', to: 'example@sitepoint.com'
}

/**
 * Package the app in tar.gz file
 *
 * @param fileName name of the file
 */
def packageApp(fileName) {
    sh 'tar -zcvf ' + fileName + ' app/ bin/ src/ vendor/ web/'
    sh 'cp ' + fileName + ' /srv/builds/'
}

In the end, your complete script will look like this.

def src = 'https://github.com/peternijssen/Jumph.git'
def timestamp = new Date().format("yyyyMMddHHmmss")
def fileName = 'jumph-' + timestamp + '.tar.gz'

stage 'Build'
node {
    git url: src
    sh 'export SYMFONY_ENV=prod && composer install --no-dev --optimize-autoloader'
    sh 'bower install'
}

stage 'Package', concurrency: 1
node {
    packageApp(fileName)
    mail body: 'A new build is awaiting: ' + fileName, subject: 'New build available', to: 'example@sitepoint.com'
}

/* FUNCTIONS */

/**
 * Package the app in tar.gz file
 *
 * @param fileName name of the file
 */
def packageApp(fileName) {
    sh 'tar -zcvf ' + fileName + ' app/ bin/ src/ vendor/ web/'
    sh 'cp ' + fileName + ' /srv/builds/'
}

We still have a pretty simple script compared with what we can actually do with Workflow. For example, we could also start by running some simple tests to make sure our build is in the correct shape before we actually package the app. Instead of packaging, we could also deploy the app straight to our staging environment. After checking our staging environment, we could also let Workflow ask you for permission to deploy it to the production environment.

If you’re interested in these kind of flows, you can check out the Java examples which you can find here and here.

Perhaps you also noticed we divided the script into two stages. A stage named build and a stage named package. By setting the concurrency to one on the package stage, we make sure that only the most recent build, that passed all the previous stages, will be performed. Next to that, a stage will also divide the script into sections.

As your whole build process grows, you can imagine it can take a lot of time for it to be completed. If your server then experiences problems or gets shut down, your whole process could get disrupted. However, Workflow is suspendable. This means Jenkins will remember at which part of the script it stopped, and will continue executing as soon as it’s back up. Through Workflow, you can make sure your process keeps on running where it left just before the shutdown.

Conclusion

As seen in the article, Workflow makes it very easy and convenient to combine jobs, making continuous delivery with Jenkins even easier. You can just write a simple script to create a continuous delivery pipeline from a simple job.

If you’re interested in what Workflow can do, check out this tutorial to get a better idea. If you’re an author of a Jenkins plugin, you might be interested in getting it to work in combination with Workflow through DSL.

If you need an even more advanced package then the Workflow plugin, you might want to take a look at CloudBees Jenkins enterprise solution on their website.

Leave a Reply

Your email address will not be published. Required fields are marked *