Async/Await and JavaScript Arrays

Async/Await and JavaScript Arrays

The Comtravo backend is built from a number of micro-services, most of which run on Node.js. Like any large scale business application we need to do a lot of communication with internal as well as external API endpoints and services. In this post I will outline how using asynchrounous calls to perform that communication is often easier and produces cleaner syntax.

In general, asynchronous code is a formalism that allows non-blocking function calls. When a non-blocking function is called, the calling code receives a future (immediately) and can continue executing other commands instead of having to wait for the called task to finish, for instance waiting for network I/O to be completed. The future represents a promise that the called function will eventually complete (or fail). However, the actual return value of the called function is available only at some later point in time not immediately.

In JavaScript futures are implemented as a Promise. The native Promise API is rather limited. For instance, the only way to handle multiple Promises at once is to use Promise.all. We make heavy use of asynchronous calls and Promises at Comtravo and these limitations have become problematic. For instance, using a Promise returned from an asynchronous call as an array filter condition is not possible because the Promise object itself evaluates to true.

Previous iterations of asynchronous code in JavaScript utilised callbacks (AsyncJS). In this post I’ll explore some initial ideas on how to use async calls and promises on array operations.

Asynchronous Operations and JavaScript Arrays

Consider a use case where we hold a list of IDs on the client side and need to filter the IDs based on the status of the objects on some remote service. Querying the remote service for the status requires network I/O and can’t therefore complete immediately. This is where the asynchrounous calls come in handy.

I started using Node.js v8.10.0 with Async/Await, but I could’t use Async/Await with Array methods such as filter(), every(), find(), findIndex(), forEach(), map(), reduce(), reduceRight() and some(). This post outlines how those functions could be modified to support asynchrounous calls. You can find a more thorough implementation of the ideas presented here in the Async-Ray library on GitHub or the NPM package.

Let’s start with a simple example. Filtering a list of IDs in JavaScript is trivially easy when the filter function is syncrohonous

const UIDs = [1, 2, 3, 4, 5, 6]; // a long list of IDs to perform some operation on
UIDs.filter((e) => e > 2).map( e => expensiveOperation(e) );

Asynchronous Filtering

If we replace the filter function (e) => e > 2 with an async call the code will not work as expected (see below). As mentioned earlier the return value of the Async call is a Promise that evaluates to true, so nothing will ever be filtered as the filtering condition will always be true.

UIDs.filter(await asyncOp(e)).map(e => expensiveOperation(e) );

In order to produce the intended behaviour we need to write some rather convoluted code and introduce an additional list to hold the filtered values.

const filteredList = [];
for (let e of UIDs) {
    // Call the async operation
    if (await asyncOp(e)) {
        filteredList.push(e);
    }
}
filteredList.map(e => expensiveOperation(e));

With an imaginary asyncFilter that supports async/await we can make the above code much cleaner

const mappedValues = (await UIDs.asyncFilter( async (e) => await asyncOp(e) )
    ).map(e => expensiveOperation(e))

Of course that imaginary Array.asyncFilter does not in fact exist, but I thought of doing something like the example below. The filter function (aFilter) takes an array of values to be filtered and a callback and returns a promise. Please note that all the code snippets below are in TypeScript.

// Signature of the callback
type CallBackFilter<T> = (
  value: T,
  index?: number,
  collection?: T[]
) => Promise<boolean>;

/**
 * Async Filter function
 *
 * @export
 * @template T
 * @param {T[]} elements
 * @param {CallBackFilter<T>} cb
 * @returns {Promise<T[]>}
 */
async function aFilter<T>(
  elements: T[],
  cb: CallBackFilter<T>
): Promise<T[]> {
  const filteredResults: T[] = [];
  for (const [index, element] of elements.entries()) {
    if (await cb(element, index, elements)) {
      filteredResults.push(element);
    }
  }

  return filteredResults;
}

This can be used as follows with the same list of UIDs already used earlier

const output = await aFilter<number>(UIDs, async (i) => {
  return Promise.resolve(i > 2);
});

Below are example implementations for the other Array methods (Array.every, Array.find etc.). However, one issue still remains with these examples: chaining calls is quite cumbersome as the await calls still need to be wrapped in brackets. Luckily, there’s a solution for that as well. You can find the details in my Async-Ray library on GitHub.

Footnotes

Array.every

// Signature of the callback
type CallBackEvery<T> = (
  value: T,
  index?: number,
  collection?: T[]
) => Promise<boolean>;

/**
 * Async Every function
 *
 * @export
 * @template T
 * @param {T[]} elements
 * @param {CallBackEvery<T>} cb
 * @returns {Promise<boolean>}
 */
async function aEvery<T>( elements: T[], cb: CallBackEvery<T> ): Promise<boolean> {
  for (const [index, element] of elements.entries()) {
    if (!(await cb(element, index, elements))) {
      return false;
    }
  }

  return true;
}
// You can use as follows
const array = [1, 2, 3, 4];

const output = await aEvery<number>(array, async (i) => {
 return Promise.resolve(i > 2);
});

Array.find

// Signature of the callback
type CallBackFind<T> = (
  value: T,
  index?: number,
  collection?: T[]
) => Promise<boolean>;

/**
 * Async Find function
 *
 * @export
 * @template T
 * @param {T[]} elements
 * @param {CallBackFind<T>} cb
 * @returns {Promise<T | undefined>}
 */
async function aFind<T>( elements: T[], cb: CallBackFind<T> ): Promise<T | undefined> {
  for (const [index, element] of elements.entries()) {
    if (await cb(element, index, elements)) {
      return element;
    }
  }

  return undefined;
}

// You can use as follows
const array = [1, 2, 3, 4];

const output = await aFind<number>(array, async (i) => {
 return Promise.resolve(i === 2);
});

Array.findIndex

// Signature of the callback
type CallBackFindIndex<T> = (
  value: T,
  index?: number,
  collection?: T[]
) => Promise<boolean>;

/**
 * Async FindIndex function
 *
 * @export
 * @template T
 * @param {T[]} elements
 * @param {CallBackFind<T>} cb
 * @returns {Promise<number>}
 */
async function aFindIndex<T>( elements: T[], cb: CallBackFindIndex<T> ): Promise<number> {
  for (const [index, element] of elements.entries()) {
    if (await cb(element, index, elements)) {
      return index;
    }
  }

  return -1;
}
// You can use as follows
const array = [1, 2, 3, 4];

const output = await aFindIndex<number>(array, async (i) => {
 return Promise.resolve(i === 2);
});

Array.forEach

// Signature of the callback
type CallBackForEach<T> = (
  value: T,
  index?: number,
  collection?: T[]
) => Promise<void>;

/**
 * Async ForEach function
 *
 * @export
 * @template T
 * @param {T[]} elements
 * @param {CallBackForEach<T>} cb
 * @returns {Promise<void>}
 */
async function aForEach<T>( elements: T[], cb: CallBackForEach<T> ): Promise<void> {
  for (const [index, element] of elements.entries()) {
    await cb(element, index, elements);
  }
}

// You can use as follows
const array = [1, 2, 3, 4];

const output: number[] = [];

await aForEach<number>(array, async (i) => {
  output.push(i);
});

Array.map

// Signature of the callback
type CallBackMap<T, R> = (
  value: T,
  index?: number,
  collection?: T[]
) => Promise<R>;

/**
 * Async Map function
 *
 * @export
 * @template T
 * @template R
 * @param {T[]} elements
 * @param {CallBackMap<T, R>} cb
 * @returns {Promise<R[]>}
 */
async function aMap<T, R>(
  elements: T[],
  cb: CallBackMap<T, R>
): Promise<R[]> {
  const mappedResults: R[] = [];

  for (const [index, element] of elements.entries()) {
    const mappedResult = await cb(element, index, elements);
    mappedResults.push(mappedResult);
  }

  return mappedResults;

}


// You can use as follows
const array = [1, 2, 3, 4];

const output = await aMap<number, string>(array, async (i) => {
  return Promise.resolve(i.toString(10));
});

Array.reduce

// Signature of the callback
type CallBackReduce<T, R> = (
  accumulator: R,
  value: T,
  index?: number,
  collection?: T[]
) => Promise<R>;

/**
 * Async Reduce function
 *
 * @export
 * @template T
 * @template R
 * @param {T[]} elements
 * @param {CallBackReduce<T, R>} cb
 * @param {R} [initialValue]
 * @returns {Promise<R>}
 */
async function aReduce<T, R>( elements: T[], cb: CallBackReduce<T, R>, initialValue?: R ): Promise<T | R> {
  if (!elements.length && initialValue === undefined) {
    throw new Error('Reduce of empty array with no initial value');
  }

  let reducedValue: T | R;
  let index = 0;

  if (initialValue === undefined) {
    reducedValue = elements[0] as T;
    index++;
  } else {
    reducedValue = initialValue;
  }

  for (; index < elements.length; index++) {
    reducedValue = await cb(reducedValue, elements[index], index, elements);
  }

  return reducedValue;
}

// You can use as follows
const array = [1, 2, 3, 4];

const output = await aReduce<number, string>(array, async (acc, i,) => {
  return Promise.resolve(`${acc}${i.toString(10)}`);
}, '');

Array.reduceRight

/** returns any type value */
type CallBackReduceRight<T, R> = (
  accumulator: T | R,
  value: T,
  index?: number,
  collection?: T[]
) => Promise<T | R>;

/**
 * Async ReduceRight function
 *
 * @export
 * @template T
 * @template R
 * @param {T[]} elements
 * @param {CallBackReduceRight<T, R>} cb
 * @param {R} [initialValue]
 * @returns {Promise<T | R>}
 */
async function aReduceRight<T, R>( elements: T[], cb: CallBackReduceRight<T, R>, initialValue?: R ): Promise<T | R> {
  if (!elements.length && initialValue === undefined) {
    throw new Error('Reduce of empty array with no initial value');
  }

  let reducedValue: T | R;
  let index = elements.length - 1;

  if (initialValue === undefined) {
    reducedValue = elements[index] as T;
    index--;
  } else {
    reducedValue = initialValue;
  }

  for (; index >= 0; index--) {
    reducedValue = await cb(reducedValue, elements[index], index, elements);
  }

  return reducedValue;
}

// You can use as follows
const array = [1, 2, 3, 4];

const output = await aReduceRight<number, number>(array, async (acc, i, index, collection) => {
    return acc + (await Promise.resolve(i));
  }, 1);

Array.some

/** returns boolean */
type CallBackSome<T> = (
  value: T,
  index?: number,
  collection?: T[]
) => Promise<boolean>;

/**
 * Async Some function
 *
 * @export
 * @template T
 * @param {T[]} elements
 * @param {CallBackSome<T>} cb
 * @returns {Promise<boolean>}
 */
export async function some<T>( elements: T[], cb: CallBackSome<T> ): Promise<boolean> {
  for (const [index, element] of elements.entries()) {
    if (Prawait cb(element, index, elements)) {
      return true;
    }
  }

  return false;
}

// You can use as follows
const array = [1, 2, 3, 4];

const output = await aSome<number>(array, async (i) => {
 return Promise.resolve(i > 2);
});

The above methods are the ones I find feasible. As I mentioned earlier, all these methods are compiled into an NPM Package Async-Ray, please check it out on GitHub and feel free to contribute. The project is MIT licensed.


Written by

Ruwan Geeganage

Senior Software Engineer

Ruwan is an experienced Software Engineer with 9+ years of working experience. He is an enthusiastic team player dedicated to streamlining processes and efficiently resolving project issues. He excels at NodeJs, JavaScript, MongoDB, including coordinating ground-up planning, programming, and implementation of core modules.


Updated