And if you already understand what Streams are, you’re all set, mi amigo!
Getting back to the point, to understand what are buffers in Nodejs, you first need to understand what streams are.
Correlating Streams with Buffers in Nodejs
Assuming you understand what I am talking about at this point, buffers in Nodejs refer to temporary data that a stream holds onto before it is consumed.
The size of a buffer in a stream is defined by a property on the stream instance known as highWatermark. This property is a number that defines the size of a buffer in bytes.
Now, when some data is pushed into a stream, it resides in a stream buffer until the stream gets consumed. When the buffer memory is full, and we try to forcefully push data into the stream, the data is rejected and the output returned is false for the push action.
Buffers in Nodejs – A Standalone Definition
The Buffer class is available in the global scope in Nodejs and hence, you do not ever have to import it using the require module.
Buffers in Nodejs are of less popularity or acquaintance so to say, to Node developers since they do not interact with system memory on a day-to-day basis.
Developers who work with Go, C or C++, or any other system programming language, are much familiar with the Buffers concept.
Buffers in Nodejs are a fixed size of memory that cannot be resized. As mentioned above, their size is defined by the highWatermark property on the stream instance.
Why Using Buffers in Nodejs is Useful?
Buffers in Node.js should not be confused with the concept of buffering. This is only the case when a stream processor receives data at a faster pace than it can digest!
Originally, Nodejs used strings to deal with TCP streams and reading & writing to the file system. However, there’s a problem with strings. Both these tasks demand dealing with binary data streams.
Although you could use strings, however, that would only bring trouble your way. First, this approach slows down the process since you’ll have to use an API compatible with strings and NOT binary data.
Second, there’s also a tendency for the code to break in unexpected ways.
So, why use binary strings, when you can use buffers in Nodejs!
Creating a Buffer in Nodejs
Buffers in Nodejs can be created using Buffer.from(), Buffer.alloc() and Buffer.allocUnsafe() methods.
You may even use the following syntax to create buffers in Nodejs:
- Buffer.from(arrayBuffer[, byteOffset[, length]])
- Buffer.from(string[, encoding])
We may also allocate a size when initializing a buffer. Let’s create a 2KB buffer:
const buf = Buffer.allocUnsafe(2048)
There are certain things you should know when working with buffers in Nodejs. Although both alloc and allocUnsafe define a size to a Buffer in bytes, there are differences between the two.
The alloc() method is initialized with zeroes and the allocUnsafe() method remains uninitialized. Meaning which, although allocUnsafe() might be comparatively faster than alloc(), the allocated piece of data may contain old and potentially sensitive data.
If there are older data present in memory, can be accessed or leaked when the Buffer is read. This is exactly why it is named as allocUnsafe, and this is what makes it unsafe!
It is recommended you take extra care when using this method.
Accessing Contents of a Buffer
Earlier in this article, I mentioned Buffers in Nodejs are more like an array of bytes. Hence, as we access data from a simple array, we can do it similarly here to access the contents of a buffer using indexing.
The numbers you see are UTF-8 bytes identifying the characters in that buffer (H → 72, e → 101, y → 121). You see this because UTF-8 is the default Unicode encoding. It is important to note that some characters may take up more of a byte of memory.
To print the entire contents of the buffer, use the toString() method:
By default, buf.toString() also uses UTF-8 encoding.
When you initialize a buffer with a pre-specified number for size, you will not get an empty buffer, rather you would find some random data inside it.
Changing Contents of a Buffer
You can write an entire string of data to buffers in Nodejs by making use of the write() method.
Just as we change data in an array by first accessing it, we can do the same here to change the contents of buffers in Nodejs:
buf = 79 // y in UTF-8
To understand what are buffers in Nodejs, you first need to understand what streams are. Read this post about buffers in Nodejs that explains what it is, how it works, and how you can use buffers in Nodejs projects.
- Nodejs.dev Official Docs
- Nodejs.org Official Docs
- Nodejs.org Official Blog
- UTF Chart