Write code, test and deploy
This is one of the common development cycles of any developer.
Most of the time testing and deployment steps do not change frequently and in order to keep the developer focus on writing code, we do the automation of testing and deployment.
This automation is called “continuous integration and deployment” OR CI/CD Pipeline.
In this tutorial, we will learn how to set up continuous integration and deployment (CI/CD) infrastructure for the Node.js project.
Prerequistics
I am going to use Jenkins as the automation tool and Github as source code management system.
Jenkins is free for use but we require public server in order to integrate the Github repository.
For the server, I will be using $5 droplet on Digitalocean. In case you wish to buy it, you can use this link to get the $10 off for the first month.
How it’s going to work
Here is the complete flow of the system.
- You made some changes in your project.
- You push those changes in Github on master or any branch.
- Github will notify Jenkins about the new push. ( We will configure it )
- Jenkins will then run the commands you ask it to run.
Those commands will contain the following.
- Test script.
- Deployment script.
Deployment script will be added to Project only and Jenkins will use that to communicate to Server and perform the push.
Basically, what you do manually such as running test command, login in to Server, performing Git pull, and then restarting the build server, we will do all of them automatically by writing the script.
Continuous Integration and deployment with Jenkins and Node.js
Creating a project
You need a Node.js project hosted on Github to perform the testing and deployment. I am sure most of you already have projects at Github written in Node.
For those who do not have one, let’s create one together.
Go to github.com to create new repository. Give it proper name and initiate it with README file and choose Node as .gitignore file.
After creating a new repository, clone the project on your local system using the following command.
For.eg
Let’s create a simple Express project to deliver the “Hello World” message. Switch to the clone folder and run npm init -y command to generate the package.JSON file.
"name": "sampleCode",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"test": "echo "Error: no test specified" && exit 1"
},
"keywords": [],
"author": "",
"license": "ISC"
}
Install express using the following command.
Create new file named “app.js” and put the following code in it.
var app = express();
app.get('/',function(req,res) {
res.send("Hello World");
});
app.listen(4000);
Run the code using pm2 start app.js command and visit http://localhost:4000 from your browser to view the app.
PM2 is process manager and we recommend you to use it. Here is the article on same.
So we have a basic app ready to go. Let’s write one simple test case which perform the home page accesibility checking.
First, install supertest, mocha and should packages.
We need mocha globally installed too. Run this command to do so.
Create new folder named “test” and new file inside it named “test.js”.
Paste the following code in it.
var should = require("should");
// This agent refers to PORT where the program is running.
var server = supertest.agent("http://localhost:4000");
// UNIT test begin
describe("SAMPLE unit test",function(){
// #1 should return home page
it("should return home page",function(done){
// calling home page
server
.get("/")
.expect("Content-type",/text/)
.expect(200) // THis is HTTP response
.end(function(err,res){
// HTTP status should be 200
res.status.should.equal(200);
done();
});
});
});
Run your node app then run the following command from another tab.
You should be able to see the console like this.
Ok, so we have our app running with the test. Let’s push it to Github.
First add all files.
Add new commit.
Push it.
Adding Github webhook to push events to Jenkins
When any changes i.e commits are pushed to the Github repository, we need a mechanism to notify that event to our Jenkins Server which we going to configure in next section.
Github allows us to add Webhook services to achieve same. Open GitHub repository, go to settings and click “Webhooks and Services” from side panel. Click on “Add Services” and search Jenkins, click on “Jenkins Github Plugin”.
Note: Don’t choose Git one.
After that, add the webhook URL which is in the following format.
http://yourdomain.com-OR-IP/github-webhook
Click on Add Service and you are good.
Creating droplet and Installing Jenkins
I am using DigitalOcean as Web Hosting. You can use any Server as you like, in case you want to host your app on DigitalOcean and looking forward to buying it, you can use this link to get $10 discount on our side.
Click here to go to DigitalOcean droplet creation screen.
Choose $5 plan and create the droplet.
After creating a droplet wait for some time, DigitalOcean will send you an email containing the credentials.
Login to your droplet using the following command.
Provide the password given in the email and you are good to go. Once you are log in, update the kernel using the following command.
We need the Java runtime system to execute Jenkins. Run the following commands to install it.
Run following commands one by one to install Jenkins.
After installation, Jenkins will automatically start itself and you can visit the domain-name:8080 to access it.
We have installed Jenkins, let’s add the Github plugin in Jenkins in order to integrate the project.
Adding Github plugin in Jenkins
Click on “Manage Jenkins” and then click on “Manage Plugins”.
Now search for “GitHub” from search bar placed at right side of the screen and choose the Github plugin. There are many more with similar names but choose the one shown below.
Click on “Install without restart” and in a few seconds plugin will be installed.
Configuring project with Jenkins
We have our project ready and hosted at Github and Jenkins installed, all we need now is to integrate both of them so that when we push any changes to Github, Jenkins knows about it and does the testing and deployment.
Click on “New Item” and fill in the proper project name. Choose “Freestyle project” among the options.
In configuration screen, check Github project and add the Github project link. Under “Source Code Management” select Git and add the Repository URL with .git extension.
You can choose which branches you want to cover up in the deployment process or leave it blank for any branch.
Under “Build Trigger” section select “Build when a change is pushed to GitHub” option. This is very important.
Under “Build” section, Click on “Add build step” button and then click “Execute shell” option.
In that for instance type “npm install” and then click on Save. Here is full-page screenshot.
Let’s check if we have Github and Jenkins talking to each other.
Make any changes in the project and do the commit and push. As soon as you push new changes Jenkins will show you new build in the project dashboard.
You can replace the command with mocha in order to do testing first.
Configuring deployment process
We have Github communicating to Jenkins, it’s time to perform the deployment to our Server. One thing, Jenkins will not directly communicate to your web server. All it can do is to run instruction which you want it to run.
So let’s first list down the steps we manually perform to do the deployment to any Server.
- Login to Server using SSH.
- Switching to project directory.
- Pulling code from Github.
- Restarting process manager ( say pm2 ).
We will write down all of these commands in one file and tell Jenkins to execute them.
But, what about the SSH username and password? Of course, Jenkins won’t enter it as we do. We need to make sure Jenkins can perform SSH login without using the password to our Server.
To achieve that, we will generate the SSH key manually from Jenkins Server and add it to our Development Server.
Let’s do it.
Login to Jenkins Server using Terminal and switch to root user using this command.
Jenkins automatically creates a new user after installation. Switch to it using this command.
Let’s generate an RSA key, run the following command.
Press Enter for the location and do not type any password when it asks to, just hit enter.
Once the process is completed, print the public key information using this command.
It should start with ssh-rsa and ends with jenkins@droplet-ip. Copy the key.
Now login to your development server and switch to ~/.ssh directory using following command.
If it’s not present, create one using this command.
Now open the file named authorized_keys if its present, else create one.
Paste the key in the file, if there is already some information is present, just append the key in the new line and do not change anything else.
Once done save the file.
In order to validate whether keys are properly configured or not, switch to Jenkins Server and try to login to the development server using SSH.
If it’s not asking for any password and log in successfully, you are good to go.
Now we have both the Server connecting each other. Let’s write the deployment script and finish this job.
Create a new file in your project directory with no extension. I name it deploy.
Paste the following code in it.
ssh userName@development-server-ip <<EOF
cd /your-project-path
git pull
npm install --production
pm2 restart all
exit
EOF
Save the file. Each line of the code in this file is what you actually do manually, I don’t think so I need to explain this.
Make the file executable using the following command. This is important for the Shell script to execute.
Go to Jenkins project page, from left navigation click on “Configure” and scroll down to “Build” section.
Add “./deploy” in the build commands. Here is screenshot for same.
Click on Save.
Now in the project directory, add the deploy file, commit and push it to Github.
Check the second build in Jenkins.
Refresh your project page and you shall see the changes updated in the project.
It Works. We have automated the process to do the testing and deployment every time we make some changes in the project.
Tips for integrating Github private repository or organization with Jenkins
Setting up Jenkins for the private repository is similar to what we did above. I personally configured the continuous integration and deployment environment for my organization using the same steps I mentioned above except some little extra efforts due to Server configuration ( This is my guess ).
When Jenkins was trying to pull the code, it was throwing an error due to the HTTPS method used by Github. I stumbled upon various methods and nothing worked for me. I changed the HTTPS origin to SSH origin and updated the repository origin too in the Development server.
I also configured my account to use the SSH key in order to perform the Git operation without authentication and that you may need in order to let Jenkins do the code update.
Here is what I did.
Go to this official doc of Github to generate and add the SSH key in Github. You need to generate an SSH key in your development Server and add the Public key in Github.
Once that is done, change the origin of the Github repository from HTTPS to SSH.
Then log in to the development Server and switch to the project directory and run this command to change the origin to SSH.
I have solved the issue using this technique, if there is still a better way and I am sure there is, let me know in the comment or email me.
Further Study
- Pro MERN Stack: Full Stack Web App Development with Mongo, Express, React, and Node
- Node.js MySQL Tutorial
- Top 5 Node.js Frameworks to Increase Coding Productivity
- Learning Continuous Integration with Jenkins
Conclusion
Continuous integration and deployment using Jenkins are super easy and this approach would save a lot of time for developers. They need to just focus on writing better code and Jenkins will do the rest boring stuff.
