This article is written for and published on SitePoint.
Picture this: You start using Jenkins, probably start with one machine. As both the number of projects and the size of the projects themselves start to grow, you start to use slaves. Slowly, your cluster of Jenkins servers grows, and you slowly start to loose your grip. What if your master server crashes? What if the slave does not respond anymore? If you recognize this feeling or you want to avoid this, you definitely should read on.
In this article we’ll take a small peak at Jenkins Operation Center. JOC, created by CloudBees, offers you a lot of additional functionality to take your Jenkins cluster to the next level. The key features for JOC are:
- Consolidated navigation experience across all the client masters
- Shared build slave resources that can be used by any client masters.
- Control of authentication and authorization schemes used by client masters. This enables features such as: single sign-on and consolidated permission schemes.
- Management of update centers used by client masters.
- Consolidated management of Jenkins Enterprise licenses.
- Management and enforcement of certain key security settings on client masters.
Within this article we’ll just use a small amount of the available features. We’ll be creating a client master and a shared slave. For this we will be using three separate servers. The first one will run JOC, the second one will run Jenkins Enterprise and the third will be used as a shared slave. We will be explaining everything in the next couple of chapters.
Preparation
I’ll be using three virtual machines on my local computer. If you want to follow along, you might want to make sure your own computer is powerful enough. If you happen to have three spare servers somewhere, you can also try it directly on those servers.
The three servers will be managed through Vagrant. I created a multi-machine Vagrant file which will boot all three servers. If you’re not familiar with Vagrant yet, you might want to read this article.
Let’s start off with our Vagrant file. You can find the content here
Note: All three boxes are running CentOS 7. You can easily replace them with Ubuntu boxes if you prefer, but do note that all commands below are meant for CentOS.
The three servers which will be created when you run vagrant up. They all got a different IP:
- 192.168.56.105: Jenkins Operation Center (joc)
- 192.168.56.106: Jenkins Enterprise – Client Master (master)
- 192.168.56.107: Shared Slave (slave)
Installing Jenkins Operation Center
We start by installing JOC on our first server. JOC is our main control hub to manage all Jenkins instances. We’ll be using it to create our client master and shared slave. After logging in to the first Vagrant machine by running vagrant ssh joc in your terminal, you can perform the following commands to install JOC.
sudo rpm --import http://nectar-downloads.cloudbees.com/jenkins-operations-center/latest/rpm/jenkins-ci.org.key
sudo wget -O /etc/yum.repos.d/jenkins-oc.repo http://nectar-downloads.cloudbees.com/jenkins-operations-center/latest/rpm/jenkins-oc.repo
sudo yum update
sudo yum install java jenkins-oc
sudo chkconfig jenkins-oc on
sudo service jenkins-oc start
You will receive an ok message to indicate JOC has been successfully installed and started.
Before we can actually start using JOC, we’ll have to open the web service port to allow access as well as allow our other servers to access this server. We can do this by changing the firewall by running the following commands.
sudo firewall-cmd --zone=public --permanent --add-port=8888/tcp --permanent
sudo firewall-cmd --zone=public --permanent --add-rich-rule='rule family="ipv4" source address="192.168.56.105/32" accept'
sudo firewall-cmd --zone=public --permanent --add-rich-rule='rule family="ipv4" source address="192.168.56.106/31" accept'
sudo firewall-cmd --reload
By now, you should be able to reach JOC with the following URL: http://192.168.56.105:8888/.
After entering the above URL, a registration form will show up. You can register for an evaluation license if you haven’t bought the product yet from CloudBees. Just fill in your details and press Next.
When you’ve registered yourself, I recommend you update all the plugins. JOC requires some specific versions of plugins so we want to make sure we have these. Generally, it’s not recommended to just update plugins when it’s not needed, as indicated in this article, 7 ways to improve Jenkins.
Note: Problems installing JOC, or using a different server setup? Have a look at the official manual for troubleshooting.
Preparing a Client Master
On the client master server, we’ll install Jenkins Enterprise. Jenkins Enterprise is a commercial product like JOC by CloudBees. It’ll enhance your Jenkins experience by installing several plugins.
You can enter the client master server by running vagrant ssh master on the command line. Now you can run the following commands to install Jenkins.
sudo rpm --import https://jenkins-ci.org/redhat/jenkins-ci.org.key
sudo wget -O /etc/yum.repos.d/jenkins.repo http://pkg.jenkins-ci.org/redhat-stable/jenkins.repo
sudo yum update
sudo yum install java jenkins
sudo chkconfig jenkins on
sudo service jenkins start
Just as we did when installing JOC, we have to configure our firewall again.
sudo firewall-cmd --zone=public --permanent --add-port=8080/tcp --permanent
sudo firewall-cmd --zone=public --permanent --add-rich-rule='rule family="ipv4" source address="192.168.56.105/32" accept'
sudo firewall-cmd --zone=public --permanent --add-rich-rule='rule family="ipv4" source address="192.168.56.106/31" accept'
sudo firewall-cmd --reload
When you’ve performed these steps on the server, you can enter the Jenkins instance by entering http://192.168.56.106:8080/ in your favorite web browser. Our next step is to convert this Jenkins instance into an Enterprise edition. You can do so by going to the plugin page and installing the plugin named Install Jenkins Enterprise by CloudBees. After that, continue your way to Manage Jenkins -> Install Jenkins Enterprise by CloudBees. Since we got ourselves an up and running JOC, you can continue by pressing Install Operations Center client only.
The process can take a couple of minutes. When it’s finished, reload Jenkins and fill in the registration form again to get an evaluation license.
When completed, make sure you go to Manage Jenkins -> Configure System and set the number of executors to 0. By setting it to 0, we make sure that later on this client master uses any of the available shared slaves instead of building on the server itself.
Adding a Client Master to JOC
Install Jenkins Enterprise is one step. To create a client master out of it, we have to configure it in JOC. After entering JOC again, we click on new item in the menu and choosing client master as our option on the JOC instance.
After creating the client master, you can define several settings like which license type to use and whether there’s a specific user who manages this master. The latter option can come in quiet handy when different people manage certain Jenkins instances. You can find more information about the available settings in the official manual. For now, set the license to License (no dedicated executors).
We can now connect this client master with the actual server. By filling in http://192.168.56.106:8080/ in Client Master URL and clicking Push Notification, we can connect our master instance with JOC.
After confirming this connection, you’ll find a connected client master in the overview in JOC.
Adding a Shared Slave
The third server is our shared slave. A shared slave is, as the name implies, a slave which can be shared between multiple client masters. Whenever a client master needs a slave, it will retrieve a slave from the pool in JOC. Whichever slave is connected to the client master and is available, will be attached to the client master.
A shared slave doesn’t necessary needs its own server. It can run on the same servers as our other instances or even multiple slaves can be ran on one server. However, to spread load it’s better to use a separate instance like we’re doing now.
We can add a shared slave to JOC by going to new item in the menu and choosing shared slave as your option on the JOC instance.
Change the following settings:
- Number of executors: 1
- Remote FS root: /var/jenkins
- Launch method: Launch slave agents via Java Web Start
It’s better to actually set up a SSH connection between your servers, but for this article, we’ll use Java Web Start for now.
After clicking save, your shared slave is correctly configured. Now we need to connect our slave server to JOC. You will need to run two commands on the shared slave. The first command gets the jar file we need to run and the second command will connect your shared slave to JOC. The URL will be different on your system, so make sure to copy it from the shared slave page.
wget http://localhost.joc.com:8888/jnlpJars/slave.jar
java -jar slave.jar -jnlpUrl http://192.168.56.105:8888/jnlpSharedSlaves/SitePoint%20Shared%20Slave/slave-agent.jnlp -secret 81ca1aefc6b45b4dd67216d036549b38e00eb878d8bb246f1b215f60d3b959fc
Creating a Project
Until now, we’ve only been preparing our servers. We can now actually start to run a project and see the use of a shared slave. Log into our client master and create a new freestyle job by clicking on new item and then choosing freestyle project.
We add a new shell build step with the following content.
sleep 10
date
When you start building this project, you’ll notice two things. In the client master, you’ll notice that the project is being build on the shared slave. In JOC you will notice that the slave is being marked as ‘in use’.
Of course this project is not realistic, but you can imagine your own project is now being built. The next step you probably want to do is add more shared slaves, so your client master has more slaves to choose from. Also, you want to create more client masters and assign your colleagues to a client master to manage. This way you can keep control using JOC while having multiple Jenkins servers dealing with your projects.
Conclusion
Unfortunately, we’re heading to the end of the article. As you can imagine, we’re not quite production ready. To get to that stage, you’ll probably want to secure all your Jenkins instances with a single sign on and configure multiple shared slaves or perhaps a shared cloud. Also you could dive into generating analytics or configure the update centers.
As you can see, Jenkins Operations Center is a great tool to manage your Jenkins instances and this article just showed you a small part of horizontal scaling your Jenkins cluster.