Continuous Integration and deployment (CI/CD Pipeline) with Jenkins and Node.js

Write code, test and deploy

This is one of the common development cycles of any developer.

Most of the time testing and deployment steps do not change frequently and in order to keep the developer focus on writing code, we do the automation of testing and deployment.

This automation is called “continuous integration and deployment” OR CI/CD Pipeline.

In this tutorial, we will learn how to set up continuous integration and deployment (CI/CD) infrastructure for the Node.js project.

Prerequistics

I am going to use Jenkins as the automation tool and Github as source code management system.

Jenkins is free for use but we require public server in order to integrate the Github repository.

For the server, I will be using $5 droplet on Digitalocean. In case you wish to buy it, you can use this link to get the $10 off for the first month.

How it’s going to work

Here is the complete flow of the system.

  • You made some changes in your project.
  • You push those changes in Github on master or any branch.
  • Github will notify Jenkins about the new push. ( We will configure it )
  • Jenkins will then run the commands you ask it to run.

Those commands will contain the following.

  • Test script.
  • Deployment script.

Deployment script will be added to Project only and Jenkins will use that to communicate to Server and perform the push.

Basically, what you do manually such as running test command, login in to Server, performing Git pull, and then restarting the build server, we will do all of them automatically by writing the script.

Continuous Integration and deployment with Jenkins and Node.js

Creating a project

You need a Node.js project hosted on Github to perform the testing and deployment. I am sure most of you already have projects at Github written in Node.

For those who do not have one, let’s create one together.

Go to github.com to create new repository. Give it proper name and initiate it with README file and choose Node as .gitignore file.

Devops nodejs

After creating a new repository, clone the project on your local system using the following command.

git clone https://github.com/<username>/repo-name.git

For.eg

git clone https://github.com/codeforgeek/demoapp

Let’s create a simple Express project to deliver the “Hello World” message. Switch to the clone folder and run npm init -y command to generate the package.JSON file.

package.json
{
  "name": "sampleCode",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo "Error: no test specified" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC"
}

Install express using the following command.

npm i --save express

Create new file named “app.js” and put the following code in it.

app.js
var express = require('express');
var app = express();

app.get('/',function(req,res) {
  res.send("Hello World");
});

app.listen(4000);

Run the code using pm2 start app.js command and visit http://localhost:4000 from your browser to view the app.

PM2 is process manager and we recommend you to use it. Here is the article on same.

So we have a basic app ready to go. Let’s write one simple test case which perform the home page accesibility checking.

First, install supertest, mocha and should packages.

npm i --save-dev supertest should mocha

We need mocha globally installed too. Run this command to do so.

sudo npm i -g mocha

Create new folder named “test” and new file inside it named “test.js”.

mkdir test && cd test

Paste the following code in it.

/test/test.js
var supertest = require("supertest");
var should = require("should");

// This agent refers to PORT where the program is running.

var server = supertest.agent("http://localhost:4000");

// UNIT test begin

describe("SAMPLE unit test",function(){

  // #1 should return home page
  it("should return home page",function(done){
    // calling home page
    server
    .get("/")
    .expect("Content-type",/text/)
    .expect(200) // THis is HTTP response
    .end(function(err,res){
      // HTTP status should be 200
      res.status.should.equal(200);
      done();
    });
  });

});

Run your node app then run the following command from another tab.

mocha

You should be able to see the console like this.

Mocha Devops

Ok, so we have our app running with the test. Let’s push it to Github.

First add all files.

git add .

Add new commit.

git commit -m "first push"

Push it.

git push

Adding Github webhook to push events to Jenkins

When any changes i.e commits are pushed to the Github repository, we need a mechanism to notify that event to our Jenkins Server which we going to configure in next section.

Github allows us to add Webhook services to achieve same. Open GitHub repository, go to settings and click “Webhooks and Services” from side panel. Click on “Add Services” and search Jenkins, click on “Jenkins Github Plugin”.

Note: Don’t choose Git one.

Adding Github webhook

After that, add the webhook URL which is in the following format.

http://yourdomain.com-OR-IP/github-webhook

webhook screen

Click on Add Service and you are good.

Creating droplet and Installing Jenkins

I am using DigitalOcean as Web Hosting. You can use any Server as you like, in case you want to host your app on DigitalOcean and looking forward to buying it, you can use this link to get $10 discount on our side.

Click here to go to DigitalOcean droplet creation screen.

Continous integration for Node

Choose $5 plan and create the droplet.

After creating a droplet wait for some time, DigitalOcean will send you an email containing the credentials.

Login to your droplet using the following command.

ssh username@dropletip

Provide the password given in the email and you are good to go. Once you are log in, update the kernel using the following command.

sudo apt-get update

We need the Java runtime system to execute Jenkins. Run the following commands to install it.

sudo apt-get install default-jre
sudo apt-get install default-jdk

Run following commands one by one to install Jenkins.

sudo wget -q -O - http://pkg.jenkins-ci.org/debian/jenkins-ci.org.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins-ci.org/debian binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt-get update
sudo apt-get install jenkins

After installation, Jenkins will automatically start itself and you can visit the domain-name:8080 to access it.

Continous integration for Node

We have installed Jenkins, let’s add the Github plugin in Jenkins in order to integrate the project.

Adding Github plugin in Jenkins

Click on “Manage Jenkins” and then click on “Manage Plugins”.

Plugin Jenkins

Now search for “GitHub” from search bar placed at right side of the screen and choose the Github plugin. There are many more with similar names but choose the one shown below.

Github plugin

Click on “Install without restart” and in a few seconds plugin will be installed.

plugin installed

Configuring project with Jenkins

We have our project ready and hosted at Github and Jenkins installed, all we need now is to integrate both of them so that when we push any changes to Github, Jenkins knows about it and does the testing and deployment.

Click on “New Item” and fill in the proper project name. Choose “Freestyle project” among the options.

Continious integration using Node.js

In configuration screen, check Github project and add the Github project link. Under “Source Code Management” select Git and add the Repository URL with .git extension.

You can choose which branches you want to cover up in the deployment process or leave it blank for any branch.

Under “Build Trigger” section select “Build when a change is pushed to GitHub” option. This is very important.

Under “Build” section, Click on “Add build step” button and then click “Execute shell” option.

In that for instance type “npm install” and then click on Save. Here is full-page screenshot.

Jenkins configuration

Let’s check if we have Github and Jenkins talking to each other.

Make any changes in the project and do the commit and push. As soon as you push new changes Jenkins will show you new build in the project dashboard.

Building process Jenkins

You can replace the command with mocha in order to do testing first.

Configuring deployment process

We have Github communicating to Jenkins, it’s time to perform the deployment to our Server. One thing, Jenkins will not directly communicate to your web server. All it can do is to run instruction which you want it to run.

So let’s first list down the steps we manually perform to do the deployment to any Server.

  • Login to Server using SSH.
  • Switching to project directory.
  • Pulling code from Github.
  • Restarting process manager ( say pm2 ).

We will write down all of these commands in one file and tell Jenkins to execute them.

But, what about the SSH username and password? Of course, Jenkins won’t enter it as we do. We need to make sure Jenkins can perform SSH login without using the password to our Server.

To achieve that, we will generate the SSH key manually from Jenkins Server and add it to our Development Server.

Let’s do it.

Login to Jenkins Server using Terminal and switch to root user using this command.

sudo su

Jenkins automatically creates a new user after installation. Switch to it using this command.

su jenkins

Let’s generate an RSA key, run the following command.

ssh-keygen -t rsa

Press Enter for the location and do not type any password when it asks to, just hit enter.

Once the process is completed, print the public key information using this command.

cat ~/.ssh/id_rsa.pub

It should start with ssh-rsa and ends with jenkins@droplet-ip. Copy the key.

Now login to your development server and switch to ~/.ssh directory using following command.

cd ~/.ssh

If it’s not present, create one using this command.

mkdir ~/.ssh

Now open the file named authorized_keys if its present, else create one.

nano authorized_keys

Paste the key in the file, if there is already some information is present, just append the key in the new line and do not change anything else.

Once done save the file.

In order to validate whether keys are properly configured or not, switch to Jenkins Server and try to login to the development server using SSH.

ssh userName@development-ip

If it’s not asking for any password and log in successfully, you are good to go.

Now we have both the Server connecting each other. Let’s write the deployment script and finish this job.

Create a new file in your project directory with no extension. I name it deploy.

Paste the following code in it.

-#!/bin/sh     
ssh userName@development-server-ip <<EOF       
 cd /your-project-path 
 git pull      
 npm install --production      
 pm2 restart all
 exit      
EOF

Save the file. Each line of the code in this file is what you actually do manually, I don’t think so I need to explain this.

Make the file executable using the following command. This is important for the Shell script to execute.

chmod +x deploy

Go to Jenkins project page, from left navigation click on “Configure” and scroll down to “Build” section.

Add “./deploy” in the build commands. Here is screenshot for same.

Adding deployment script

Click on Save.

Now in the project directory, add the deploy file, commit and push it to Github.

Check the second build in Jenkins.

Build process

Refresh your project page and you shall see the changes updated in the project.

It Works. We have automated the process to do the testing and deployment every time we make some changes in the project.

Tips for integrating Github private repository or organization with Jenkins

Setting up Jenkins for the private repository is similar to what we did above. I personally configured the continuous integration and deployment environment for my organization using the same steps I mentioned above except some little extra efforts due to Server configuration ( This is my guess ).

When Jenkins was trying to pull the code, it was throwing an error due to the HTTPS method used by Github. I stumbled upon various methods and nothing worked for me. I changed the HTTPS origin to SSH origin and updated the repository origin too in the Development server.

I also configured my account to use the SSH key in order to perform the Git operation without authentication and that you may need in order to let Jenkins do the code update.

Here is what I did.

Go to this official doc of Github to generate and add the SSH key in Github. You need to generate an SSH key in your development Server and add the Public key in Github.

Link to official doc.

Once that is done, change the origin of the Github repository from HTTPS to SSH.

Then log in to the development Server and switch to the project directory and run this command to change the origin to SSH.

 git remote set-url origin <copy it from github>

I have solved the issue using this technique, if there is still a better way and I am sure there is, let me know in the comment or email me.

Further Study

Conclusion

Continuous integration and deployment using Jenkins are super easy and this approach would save a lot of time for developers. They need to just focus on writing better code and Jenkins will do the rest boring stuff.

Pankaj Kumar
Pankaj Kumar
Articles: 208