Programming Async (1): Wrapping Functions with Callbacks as Promises

TL;DR This is a series of unsorted notes to record my learnings when writing async programs.

Many functions in JavaScript take callbacks to "pass" the outputs to the consumers. In fact, people could easily encounter callback hell if they are not careful enough. There are several ways to work around it, but still in some cases we may want to simply treat such functions as async function and have the consumers await them instead of passing in callbacks.

For example, the readFile function in the fs module takes a callback to handle the file reading results:

fs.readFile(filename, 'utf-8', function(err, data) {
    ...
})

While it's perfectly correct to use this function in its originally intended way, I found it a bit less intuitive when using it in an async program. For example:

function readFileAndFilterContent(filename) {
    fs.readFile(filename, 'utf-8', (err, content) => {
        if (err) throw err;
        let processed_content = process_func(content);
        // okay now the content is loaded and processed, now what?
    });
}

Since the readFile function is async, the callback won't get executed until the file content is loaded. We can't return the processed file content outside of the callback passed to readFile in the function body of readFileAndFilterContent, unless we have yet another callback like this:

function readFileAndFilterContent(filename, callback) {
    fs.readFile(filename, 'utf-8', (err, content) => {
        if (err) throw err;
        let processed_content = process_func(content);
        // okay now the content is loaded and processed, call the external callback to pass back the results
        callback(processed_content);
    });
}

Oh well, this is exactly how the callback hell gets created :/ With this version, we'd have to write code like

readFileAndFilterContent(filename, (processed_content) => {
    // do some stuff with processed_content
});

This doesn't look so bad to be honest, BUT there is huge catch here. If the results produced from further computation based on processed_content is needed outside the local scope where readFileAndFilterContent is called, we likely need to call yet another callback inside the anonymous callback passed into readFileAndFilterContent... Thing will eventually get out of control :(

One way to get things to work without nesting callbacks further is to use Promise:

function readFileAndFilterContent(filename) {
    return new Promise((resolve, reject) => {
        fs.readFile(filename, 'utf-8', (err, content) => {
            if (err) return reject(err);
            // okay now the content is loaded, process it as needed and return the results
            let processed_content = process_func(content);
            return resolve(processed_content);
        });
    })
}

This way, we can write code like

readFileAndFilterContent(filename).then((processed_content) => {
    // do some stuff with processed_content and return the results normally
    // the return value will be wrapped into a `Promise` automatically.
});

This version doesn't seem very different from the callback-based version, but it has a major advantage: the return value of the then part is yet another Promise that can be easily passed around by binding it to a variable or passed out of the local scope with a return, and we can perform further computation on top of it using another then where needed - all without involving more callbacks! Yay! We seem to have found a way to break the callback hell whenever we want!

Well, there is still a catch with the Promise trick tho, as mentioned in this post. To put it in a simple way: chaining operations with then could still create callback hell if not used carefully/correctly.