Asynchronous programming in Node.js

JavaScript is asynchronous in nature and so is Node. Asynchronous programming is a design pattern which ensures the non-blocking code execution.

Non blocking code do not prevent the execution of piece of code. In general if we execute in Synchronous manner i.e one after another we unnecessarily stop the execution of those code which is not depended on the one you are executing.

Asynchronous does exactly opposite, asynchronous code executes without having any dependency and no order. This improves the system efficiency and throughput.

Asynchronous programming is great for faster execution of programs but it comes with price. That’s right, its difficult to program and most of the time we end up having callback hell scenario.

This tutorial is about explaining each of the Asynchronous scenarios which you may face while coding. We will also learn about how to efficiently avoid callback hell situation.

Getting started

Consider following code.

In above code, file has been read first and then console.log() executes.

Now consider following code.

Here output will be as follows :

Here we have got console.log() content first and then file content. This is because code is Asynchronous and event loop executes that later.

This also covers one of the interview question where interviewer asks to write program to print same code.

Related learning : How Node.js event loop works ?

What is callback hell

Familiar with something like this.

Note : Above code is just an example not a working code.

This happens due to the Asynchronous nature of the JavaScript. We want to execute tasks which are dependent on each other hence we wrap them into the callbacks of each function and hence caught into callback hell situation.

How to avoid callback hell

To avoid callback hell, follow one or combination of the following :

• Modularise your code.
• Use generators.
• Use promises
• Use event-driven programming.
• Use Async.js

Modularise your code

Consider following code.

The callback function is a closure and can only be accessed inside the function. However you can create separate function by providing some name and pass that function as callback.

Like this.

The only drawback is you need to create lot of function as code grows.

Use generators

In simple words Generators provides you the ability to convert asynchronous code to synchronous one. Lets learn using example,consider following.

This is asynchronous code so console.log() will execute prior to readFile().

In order to avoid putting our console.log() inside the callback closure we can use generators to convert the asynchronous nature of readFile() into synchronous one.

Consider following code.

Try to execute this code, you will see the content of file first and then console.log().

I will cover generators in more detail in upcoming tutorials.

Use promises

Promise represents the result of asynchronous function. Promises can be used to avoid chaining of callbacks. In Node.js, you can use Q or Bluebird modules to avail the feature of promise.

If we want to convert file read code callback into promises here is how we can do that.

Make sure you have bluebird ( npm install bluebird ) installed.

Checkout bluebird API reference for more information.

Use event-driven programming

Node.js provides EventEmitter module that can help you to program using events. You can also use it to structure your code and avoid callback hell. However this may not help you in large structure code but it is an option.

Related learning : About Event Emitter in details

Consider below code.

Here we have created one generic function which perform the file read. You can call this function by emitting events. You can also extend it to emit the events when file reading are done.

Use Async.js

Async.js is utility module provides various functions (around 70) to handle the Asynchronous JavaScript. This is most preferred way and recommended way by us.

Let’s look at some common situation which you may across while writing code and proposed solution for same using Async.js.

Scenario : Run multiple tasks that does not depend on each other and when they all finish do something else

In this scenario, we want to execute multiple asynchronous function which are not dependent on each other i.e We don’t need to block them.

Solution : Use async.parallel() function

Here is code to explain same.

Scenario : Run multiple tasks one after another and once they are finish execute something else

No need to explain the scenario.

Solution : Use async.series() function

Here is code to explain same.

Scenario : Run multiple tasks one after another and exchange data between them and once they are finish execute something else

This is the scenario very similar to above one except that we need to pass some data to the next function. Async.series() will pass each functions data to final callback function not to the next one.

Solution : Use async.waterfall() function

Here is code to explain same.

Run multiple parallel task for by iterating over array ( any collection ) and once all of them are finish execute something else

Consider a scenario where you need to send an email to 1000 users. You have email pushed in an array and you want to execute email() function independent of each other.

Solution : Use async.each()

Here is code to explain same.

Run multiple parallel task by iterating over array ( any collection ) in a set and once all of them are finish execute something else

Same scenario as mentioned above, instead we need limit by which we divide our data. Say you have 1 million email and you want to process 1000 emails at a time.

Solution : use async.eachLimit()

Here is code to explain same.

Run multiple serial task for by iterating over array ( any collection ) and once all of them are finish execute something else

Same scenario as above, instead of going parallel we need to go for one task at a time.

Solution: You can either use async.eachLimit() with 1 as concurrency limit OR go for async.eachSeries()

Run multiple parallel task by iterating over array ( any collection ) and inside each parallel task run some tasks in series and once all of them are finish execute something else

Consider same email scenario, after sending an email you also need to update the database. That means you need to execute two task for each emails and that too in series manner by passing data from email() function to database one.

Use async.each() with async.waterfall()

Here is code to explain same.

Basically you can use any combination of async.js functions to fulfill your requirement. Here is some combination example.

Performing Queue operation

Considering mass mailer in real world scenario, you cannot invoke like millions of callback function at one particular time and it is because of resource limitation and buffer.

In this kind of situation where operation should invoke in batch say 10000 emails at once and keep executing till everything is done you can use async.queue().

Here is very simple queuing example to send an Email. You can refer this tutorial for mailer code.

Wrapping up

We have covered some important and useful async.js function, however there are many more and you can surely use them according to your programming scenario.

If you come across any situation which we have not covered and you stuck on it, let us know in comments and we will surely look at it.

Conclusion

Async.js is no doubt very useful package for Node.js developer. It will save a lot of time and make your code looks good as well. Star it on Github to show your support.