Asynchronous Code with Promises and Generators

While dealing with Node, we basically have no other choice than write code filled with callbacks. But why do we have callbacks in the first place?

The Node engine is single-threaded and it means we share its only thread with all active users. With this single thread limitation comes another term—Event Loop. The event loop is the state when an application waiting for events that may occurred. What kind of events? It may be a finished database query or an incoming HTTP request.

When an event occurs, Node will add a corresponding event handler—a callback—to the execution queue. Since we stuck with a single thread, the event handlers from the queue will be executed one at the time. If any of callbacks contain an interaction with a database or any other I/O operation, Node will leave the execution of this operation to the system and continue to process other handlers from the queue. When the system finishes the interaction with the I/O, it will notice the Node and a new event handler will be added to the execution queue. This is asynchronous execution with the idea of non-blocking I/O.

The standard Node API contains some methods for performing synchronous interaction with the file system. But using them we are blocking the application from doing something useful at the same time. For example, while synchronously copying files for some user, we block the application from the processing of incoming HTTP requests from other users.

The Apache web server in this situation will create a new thread for each user. And every HTTP-request will be processed without waiting while the file copying for the first user is completed. But at what cost? Every new thread will consume memory and increase the CPU load. Every new thread will have its own connection to the database, so it does not only affect the web server but the whole system.

With Node-based applications, we may get the better performance with spending fewer resources.

It may sound reasonable, but on a practice, we may end writing code like this:

someAsyncMethod(function(resA) {
  someAsyncMethod(resA, function(resB) {
    someAsyncMethod(resB, function(resC) {
      someAsyncMethod(resC, function(resD) {
        someAsyncMethod(resD, function(resE) {
          someAsyncMethod(resE, function() {
            // ...
          });
        });
      });
    });
  });
});

There are different tactics to write asynchronous code and prevent similar situations. Let’s see what modern JavaScript may offer.

Promises

The release of the ECMAScript 2015 specification brings us a native (but not new) approach to handling asynchronous code—Promises. With Promises, we can structure our code to the chain of methods when each method will be executed after the previous one is completed.

The example below shows how to wrap an asynchronous function into a Promise object:

function readFile(file, encoding) {
  return new Promise(function(resolve, reject) {
    fs.readFile(file, encoding, function(err, data) {
      if (err) return reject(err);

      resolve(data);
    });
  });
}

In a callback of an asynchronous function, we can pass a result of the execution further by calling the resolve function which is given as an argument. This process called as fulfilling a promise.

In the case of an error—like a file doesn’t exist or can’t be opened from the example above—a promise can be rejected.

When a function returns a Promise object, it contains the then method that accepts two callbacks as arguments:

readFile('input.txt', 'utf-8').then(function(data) {
  console.log(data);
}, function(err) {
  console.error(String(err));
});

The first callback (onFulfilled) contains a fulfilled value or values. The second one (onRejected) contains a rejected value, usually an error.

It’s possible to omit the second callback and catch thrown errors like in the example bellow:

function readFile(file, encoding) {
  return new Promise(function(resolve) {
    fs.readFile(file, encoding, function(err, data) {
      // instead of a rejection, we can throw an error
      if (err) throw err;

      resolve(data);
    })
  });
}

readFile('input.txt', 'utf-8').then(function(data) {
  console.log(data);
}).catch(function(err) {
  console.error(String(err));
});

And when all asynchronous functions wrapped into promises, we can get rid of nested callbacks and have a code structure like this:

someAsyncMethod().then(function(resultA) {
  return someAsyncMethod(resultA)
}).then(function(resultB) {
  return someAsyncMethod(resultB)
}).then(function(resultC) {
  return someAsyncMethod(resultC)
}).then(function(resultD) {
  console.log(resultD);
}).catch(function(err) {
  console.error(String(err));
});

Generators

Another great feature of ECMAScript 2015 is a generator. A generator function allows us to pause the function at the time of execution and resume it later.

A generator function can be declared using this expression:

function* getValues() {
  // generator body
}

Notice, the asterisk sign after the function keyword. It is what makes a function generator.

Let’s have a look at a generator in action:

function* getValues() {
  yield 'first';
  yield 'second';
}

const values = getValues();

console.log(values.next());
// { value: 'first', done: false }
console.log(values.next());
// { value: 'second', done: false }
console.log(values.next());
// { value: undefined, done: true }

The call of a generator doesn’t execute a whole function body. Instead, it returns an iterator object. This object contains the next method. With this method, we execute the function body until the first yield statement. After it, the function pauses.

If we need to resume the execution, we call again the next method. Every time we call the method, it returns the current state of a function with a yield value and the done property. This property stores a boolean value, by which we can determine if the whole generator function has been executed—all yield values has been received.

Combining Promises and Generators

We can combine promises and generators and instead of yielding simple values, we will yield promises. Let’s see the full example, in which we read a text file, process the text, and save the result into another file. At the end of the function, we return the result of the text processing.

The example below uses arrow functions. Make sure you understand the concept.

'use strict';

const fs = require('fs');

function readFile(file, encoding) {
  return new Promise((resolve) => {
    fs.readFile(file, encoding, (err, data) => {
      if (err) throw err;

      resolve(data);
    })
  });
}

function writeFile(file, data, encoding) {
  return new Promise((resolve) => {
    fs.writeFile(file, data, encoding, (err) => {
      if (err) throw err;

      resolve();
    })
  });
}


function* processFile(inputFilePath, outputFilePath, encoding) {
  try {
    const text = yield readFile(inputFilePath, encoding);
    const processedText = text.toUpperCase();

    yield writeFile(outputFilePath, processedText, encoding);

    return processedText;
  } catch (e) {
    console.error(String(e));
  }
}

function execute(generator, yeildValue) {
  const next = generator.next(yeildValue);

  if (!next.done) {
    next.value.then(result => execute(generator, result))
              .catch(err => generator.throw(err));
  } else {
    console.log(next.value);
  }
}

execute(processFile('./input1.txt', './output.txt', 'utf-8'));

Our script consists of the two promises for handling reading and writing files. Next, we have the generator function in which we write the logic of the script. To provide the step-by-step execution of statements, every promise object should be yielded.

Another important function called execute takes a generator function and a yielded value as arguments:

function execute(generator, yeildValue) {
  const next = generator.next(yeildValue);

  if (!next.done) {
    next.value.then(result => execute(generator, result))
              .catch(err => generator.throw(err));
  } else {
    console.log(next.value);
  }
}

This function recursively executes the generator by calling the next method. Until the generator is fully executed, the function receives promise objects at their completion.

If some promise throws an error, we catch it and throw it into the generator by calling the generator.throw method. We catch the same error twice. Since we want to make this function universal with the ability to work with any generator that yields promises, it’s a good way to handle the concrete error situation in the generator function itself.

At the end of execution of the generator when its property done equal to true, we will get the final result of the generator function.

Conclusion

Handling asynchronous execution with a proper code structure is hard. Combining JavaScript promises and generators is a good way to achieve a similar to synchronous step-by-step execution without losing advantages of asynchronous execution.