There are multiple ways to create a Buffer object in Node.js. The two most important ones for now are the following:
const buf = Buffer.alloc(3)
console.log(buf)
// -> <Buffer 00 00 00>
Buffer.from(*) creates a Buffer object from a variety of sources, including arrays and ArrayBuffer objects, other Buffer objects, and, most importantly for us, strings.
When creating a buffer from a string, you can specify two arguments: Buffer.
from(string, encoding) where encoding is optional and defaults to 'utf8'
Once created, Buffer objects contain a variety of properties and methods, some of which we’ll encounter throughout this book. For now, it’s worth highlighting two of them:
buf.toString(encoding) is a method that returns the string representation of
the buffer in the specified encoding (which defaults to ‘utf8’ if not set); see this, for example:
const buf = Buffer.from('Hello world!', 'utf8')
console.log(buf.toString('utf8'))
// -> 'Hello world!'
buf.toString('utf8') would have been identical to
buf.toString().buf.length is a property that contains the length of the buffer, in bytes; see this, for example:
const buf = Buffer.from('Hello world!', 'utf8')
console.log(buf.length)
// -> 12
Hex encoding (HEX ↔️ STRING)
const buf = Buffer.from('48656C6C6F20776F726C6421', 'hex')
console.log(buf.toString('utf8'))
// -> 'Hello world!'
const buf = Buffer.from('Hi, Buffer!', 'utf8')
console.log(buf.toString('hex'))
// -> '48692c2042756666657221
Many programming languages, JavaScript included, allow you to write numbers in your code directly using their hexadecimal notation by adding the 0x prefix. For example, the following
console.log(0xB4 === 180) // -> true
hex encoding is highly inefficient in terms of storage requirements, as it doubles the size of our data, it is often used during development as it has three interesting properties
“Base64 standard encoding,” as defined by RFC 4648 Section 4, uses the following 64 symbols ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz01234
56789+/
“Base64 URL encoding,” as defined by RFC 4648 Section 5, uses the following 64 symbols: ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz01234
56789-_
unlike hex, these are case-sensitive
In fact, many web applications prefer to use the second format because the characters + and / are not URL-safe, so they have to be encoded when used in a URL with the usual percentage encoding, becoming %2B and %2F, respectively. Instead, - and _ are URL-safe and do not require encoding when used in URLs.
sometimes up to 2 padding characters, =, are added to make the length of base64-encoded strings an exact multiple of 4
The good news is that Node.js supports base64 encoding natively in the Buffer APIs, with ‘base64’ and ‘base64url’ available as values for the encoding arguments in the methods we saw previously
const buf1 = Buffer.from('SGk=', 'base64')
console.log(buf1.toString())
// -> 'Hi'
const buf2 = Buffer.from('8424bff8', 'hex')
console.log(buf2.toString('base64'))
// -> 'hCS/+A=='
console.log(buf2.toString('base64url'))
// -> 'hCS_-A'
Generating cryptographically secure random byte sequences
When building applications that leverage cryptography, it’s very common to have to generate random byte sequences (encryption keys, salt etc)
Node.js already includes a function to generate random data in the crypto
module: randomBytes(size, callback)
The importance of randomness
True Random Number Generator (TRNG) devices exist and are generally based on the observation of quantum effects; however, these are uncommon.
Instead, for most practical applications, we rely on Cryptographically Secure PseudoRandom Number Generators (CSPRNGs), which use various sources of entropy (or “noise”) to generate unpredictable numbers.
Math.random() (which is available in Node.js too) are not cryptographically safe, and should not be used for generating random numbers or byte sequences for use in cryptographic operations.
Using crypto.randomBytes
const crypto = require('crypto')
const {promisify} = require('util')
const randomBytes = promisify(crypto.randomBytes)
;(async function() {
const buf = await randomBytes(32)
console.log(buf.toString('hex'))
})()