Buffers in Node.js: A 3-Minute Guide To Get You Started

A computer can only understand binary data, that is, data in the form of 0’s and 1’s, this data on the Internet transfer information from one point to another. The sequential movement of these data is called a stream. Stream data in broken parts called chunks, the computer starts processing the data as soon as it receives a chunk, not waiting for the whole data.

Sometimes, the processing speed is less than the rate of receiving chunks or faster than the rate of receiving chunks, in both cases, it is necessary to hold the chunks because processing requires a minimum amount of it which is done using the buffers.

The above definition of the stream is just a way to give you an understanding of the buffer concept, but we highly recommend that you read our guide on streams: The Ultimate Guide to Understanding Streams and The Ultimate Guide to Working with Streams.

Buffer in Node.js

The buffer in Node.js is created using the Buffer class. The Buffer class is available in the global scope in Node.js, so you don’t need to import it using the require module.

Buffers in Nodejs are less popular as Node developers do not interact with system memory on a day-to-day basis.

Developers who work with Go, C or C++, or any other system programming languages, are very familiar with the concept of the buffer.

Node.js has a fixed size of buffer memory that cannot be resized. We can think of buffers in Nodejs as an array of integers, each defining the memory size in bytes of the data.

Why is it useful to use the buffer in Nodejs?

Buffer in Node.js is required when a stream processor receives the data at a faster rate than it can digest it.

Buffers in Node.js are pure JavaScript. While they work very well with Unicode-encoded strings, they don’t get along well with binary data. However, this is fine for browsers, as most data are strings.

Node.js can use both streams and strings to read and write to the file system. However, there is a problem with the strings as both of these tasks demand dealing with binary data streams.

Using string slows down the process because you have to use an API compatible with strings and not binary data.
There is also a tendency for code to break in unexpected ways.

So, why use binary strings when you can use buffers in Nodejs!

Creating a Buffer in Nodejs

Buffers in Nodejs can be created using Buffer.from(), Buffer.alloc() and Buffer.allocUnsafe() methods.

Buffer.from() Method

It takes data as an argument and creates a buffer with that data, unlike the alloc() and allocUnsafe() method which creates a blank buffer with a custom size.

Syntax:

Buffer.from(initialization)

You may even use the following syntax to create buffers using from() method in Node.js:

Buffer.from(array)
Buffer.from(arrayBuffer[, byteOffset[, length]])
Buffer.from(buffer)
Buffer.from(string[, encoding])

Buffer.alloc() and Buffer.allocUnsafe() Method

These methods take the buffer’s size as an argument to create it.

Syntax:

Buffer.alloc(size)
//or
Buffer.allocUnsafe(size)

We may also allocate a size when initializing a buffer. Let’s create a 2KB buffer:

const buf = Buffer.alloc(2048)
//or
const buf = Buffer.allocUnsafe(2048)

Note:

The alloc() and allocUnsafe() both define the size of the buffer in bytes, but there are differences between the two.

The alloc() method is initialized with zeroes and the allocUnsafe() method remains uninitialized. This means that, although allocUnsafe() might be comparatively faster than alloc(), the allocated piece of data may contain old and potentially sensitive data.

If there are older data present in memory, can be accessed or leaked when the Buffer is read. This is precisely why it is named as allocUnsafe, and this is what makes it unsafe! It is recommended you take extra care when using this method.

Accessing Contents of a Buffer

Buffers in Nodejs are more like an array of bytes. Hence, as we access data from a simple array, we can do it similarly here to access the contents of a buffer using indexing.

const buf = Buffer.from('Hey!')
console.log(buf[0]) //72
console.log(buf[1]) //101
console.log(buf[2]) //121

In the above code, the numbers are UTF-8 bytes identifying the characters in that buffer (H → 72, e → 101, y → 121). You see this because UTF-8 is the default Unicode encoding. It is important to note that some characters may take up more of a byte of memory.

To print the entire contents of the buffer, use the toString() method:

console.log(buf.toString())

By default, toString() also uses UTF-8 encoding.

When you initialize a buffer with a pre-specified number for size, you will not get an empty buffer, rather you would find some random data inside it.

Changing Contents of a Buffer

You can write an entire string of data to buffers in Nodejs by making use of the write() method.

const buf = Buffer.alloc(4)
buf.write('Dan!')

Just as we change data in an array by first accessing it, we can do the same here to change the contents of buffers in Nodejs:

const buf = Buffer.from('Dan!')
buf[2] = 79 // y in UTF-8
console.log(buf.toString()) //Day!

Conclusion

Buffers are used to store binary data. In Node.js we can create our own custom buffer with a specific size using alloc() and allocUnsafe() methods, we can write data to it, and read data from it.

To perform operations on the buffer using the different methods, we have to use the Buffer class, which is global, so you can use it directly without any import statement. After reading this tutorial we recommend you try it which helps you wrap up the concept learned in its tutorial. We hope this tutorial helped you understand the use of buffers in Node.js.