Computer Vision in nutshell is defined as “making computers see and understand things the way we humans do”. This seems easy to understand but it is really complex.
If you really want to grasp the importance and use case of Computer Vision then do watch the video. See how Amazon is using this technology to build next-generation shopping malls.
Pretty amazing, isn’t it?
So let’s just learn and understand about Computer Vision and how to get started with it using NodeJS.
Learning Computer Vision
It’s just not possible to explain the complete field of Computer Vision in one article or probably even in series of articles. Let’s learn the basics enough to go ahead with development.
You can, of course, take courses to learn more about it from the experts at sites like EDX.
There is an open-source organization working towards computer vision called OpenCV. This library official supports C, C++, Python, and Java. There is no official module for Node.js but some awesome people work together to create OpenCV module called “node-opencv”. We will use it in the upcoming section.
In this article we will cover the following:
- OpenCV Installation.
- OpenCV Node.js module.
- Building live face detector.
Let’s begin.
OpenCV Installation
You need to install the OpenCV library on your computer. If you are using Ubuntu then these commands will install the OpenCV. I have tested it on Ubuntu 16.04.
First, we need to install the dependencies required by OpenCV.
sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
Now we can install OpenCV using the following command.
However, you can also install it using the latest code shipped by OpenCV team. Refer the official installation document here.
You can also install it in Windows by following this article.
And for Mac, you can either build OpenCV from source or install it using brew. I’ll mention the brew version here.
brew install opencv
Node-OpenCV Module
OpenCV officially does not provide a Node.js driver. However, there is alternative available on NPM made by Peter Braden and team. This module is still in development and does not cover all of the OpenCV API’s.
As of now, the latest build on Github is not merged with NPM module of node-opencv hence if you directly install from NPM you may face building issues.
Install the module directly from Github using the following command.
Let’s start coding!
Building Live WebCam Face Detector
Before we start detecting the faces in the live webcam feed, let’s detect faces from images first.
So how come we detect faces from images? OpenCV provides us various classifiers which you can use to detect faces, eyes, cars, etc. These classifiers, however, are a simple one and not trained one using machine learning, so we may expect approximately 80 percent (rough estimate) of accuracy in detecting faces.
Here is the image of my friends from one of the traditional parties at the office.
Let’s write a code to detect faces. Here is the piece of code which reads the file from the disk and generates the output image.
cv.readImage("./friends.jpg", function(err, im){
if (err) throw err;
if (im.width() < 1 || im.height() < 1) throw new Error('Image has no size');
im.detectObject(cv.FACE_CASCADE,{}, function(err, faces){
if (err) throw err;
for (var i = 0; i < faces.length; i++){
var face = faces[i];
im.ellipse(face.x + face.width / 2, face.y + face.height / 2, face.width / 2, face.height / 2);
}
im.save('./friends-faces.jpg');
console.log('Image saved as friends-faces.png');
});
});
Here is the image after face detection.
I know it’s not accurate, but that’s a good start.
Let’s do the same using Webcam.
First, we need to connect to the webcam and show the live feed. Here is the code which does the same.
try {
var camera = new cv.VideoCapture(0);
var window = new cv.NamedWindow('Video', 0)
setInterval(function() {
camera.read(function(err, im) {
if (err) throw err;
console.log(im.size())
if (im.size()[0] > 0 && im.size()[1] > 0){
if (err) throw err;
window.show(im);
}
window.blockingWaitKey(0, 50);
});
}, 20);
} catch (e){
console.log("Couldn't start camera:", e)
}
Run the code using the following command.
and you shall see the desktop window popping up with a feed from the webcam. Similar to as shown below, except with your face.
I swear I look better in person, this webcam sucks 😀
Let’s go through the code.
var window = new cv.NamedWindow('Video', 0)
Here we are trying to access the webcam video feed and creating a new window to show that video.
Now, to show the live feed instead of the still image we need to refresh the window with the latest image received from the webcam. We do it by using the interval function and refreshing the window in 20ms.
camera.read(function(err, im) {
if (err) throw err;
console.log(im.size())
if (im.size()[0] > 0 && im.size()[1] > 0){
if (err) throw err;
window.show(im);
}
window.blockingWaitKey(0, 50);
});
}, 20);
The initial image comes as 0x0 pixel hence we do not need to show that. That if condition takes care of that.
Alright, so now we know how to detect faces from the image and how to show live feed from the webcam, let’s merge both and we will have our live face detector ready to go!
Here is the code to detect faces from the webcam.
try {
var camera = new cv.VideoCapture(0);
var window = new cv.NamedWindow('Video', 0)
// face detection properties
var rectColor = [0, 255, 0];
var rectThickness = 2;
setInterval(function() {
camera.read(function(err, im) {
if (err) throw err;
console.log(im.size())
if (im.size()[0] > 0 && im.size()[1] > 0){
im.detectObject(cv.FACE_CASCADE, {}, function(err, faces) {
if (err) throw err;
for (var i = 0; i < faces.length; i++) {
face = faces[i];
im.rectangle([face.x, face.y], [face.width, face.height], rectColor, rectThickness);
}
window.show(im);
});
}
window.blockingWaitKey(0, 50);
});
}, 20);
} catch (e){
console.log("Couldn't start camera:", e)
}
Let’s run the code.
Here is the output.
It’s not accurate but it provides at least 80% of accuracy (rough estimate though).
Conclusion
OpenCV is an amazing open-source project. We can use it for free to develop some useful real-world application like motion tracker, face detector and hook it with machine learning to train the existing classifiers and provide deep learning. I hope they officially support and work on the Node.js client module to avail more APIs of OpenCV in the Node.js client module.
