Serial and parallel operations in JavaScript

Serial and parallel operations in JavaScript

1. Introduction

This article describes the solutions for asynchronous functions, serial execution, and parallel execution in es5 and es6 in js . Examples have been used in combination with serial and parallel.

2. es5 method

Before es6 came out, the nodejs community already had a promise solution to deal with callback hell. If there are multiple asynchronous functions, how should the execution sequence be arranged? How can all asynchronous functions be executed faster and then proceed to the next step? Here comes the problem of serial execution and parallel execution of js.

3. Asynchronous function serial execution

var items = [ 1, 2, 3, 4, 5, 6 ];
var results = [];

function async(arg, callback) {
  console.log('Parameter is ' + arg +', return result after 1 second');
  setTimeout(function () { callback(arg * 2); }, 1000);
}

function final(value) {
  console.log('Completed: ', value);
}

function series(item) {
  if(item) {
    async(item, function(result) {
      results.push(result);
      return series(items.shift()); // recursively execute all data });
  } else {
    return final(results[results.length - 1]);
  }
}

series(items.shift());

4. Parallel execution of asynchronous functions

The above functions are executed one by one, and the next one is executed after the previous one is finished, which is similar to async and await in es6 (collectively referred to as es6 after es5). Is there something like promise.all that can execute all in parallel?

We can write:

var items = [ 1, 2, 3, 4, 5, 6 ];
var results = [];

function async(arg, callback) {
  console.log('Parameter is ' + arg +', return result after 1 second');
  setTimeout(function () { callback(arg * 2); }, 1000);
}

function final(value) {
  console.log('Completed: ', value);
}

items.forEach(function(item) {// loop complete async(item, function(result){
    results.push(result);
    if(results.length === items.length) {// Determine whether the number of completed functions is equal to the number of functions to be executed final(results[results.length - 1]);
    }
  })
});

5. Combination of serial and parallel execution of asynchronous functions

If many asynchronous data (hundreds of them) are executed in parallel, and each asynchronous data contains a lot of (https) request data, it is bound to cause insufficient TCP connections or accumulation of countless call stacks leading to memory overflow. Therefore, it is not easy to execute too much data in parallel, so a combination of parallel and serial methods emerged.

The code can be written as follows:

var items = [ 1, 2, 3, 4, 5, 6 ];
var results = [];
var running = 0;
var limit = 2;

function async(arg, callback) {
  console.log('Parameter is ' + arg +', return result after 1 second');
  setTimeout(function () { callback(arg * 2); }, 1000);
}

function final(value) {
  console.log('Completed: ', value);
}

function launcher() {
  while(running < limit && items.length > 0) {
    var item = items.shift();
    async(item, function(result) {
      results.push(result);
      running--;
      if (items.length > 0) {
        launcher();
      } else if(running == 0) {
        final(results);
      }
    });
    running++;
  }
}

launcher();

6. es6 method

es6 naturally comes with serial and parallel execution methods. For example, serial can use async and await (explained in the previous article), and parallel can use promise.all and so on. Then for combining serial and parallel operations and limiting the number of concurrent promise all calls, the community also has some solutions, such as

tiny-async-pool, es6-promise-pool, p-limit


Simple encapsulation of a promise all concurrent number limit solution function

function PromiseLimit(funcArray, limit = 5) { // Execute 5 data concurrently let i = 0;
  const result = [];
  const executing = [];
  const queue = function() {
    if (i === funcArray.length) return Promise.all(executing);
    const p = funcArray[i++]();
    result.push(p);
    const e = p.then(() => executing.splice(executing.indexOf(e), 1));
    executing.push(e);
    if (executing.length >= limit) {
      return Promise.race(executing).then(
        () => queue(),
        e => Promise.reject(e)
      );
    }
    return Promise.resolve().then(() => queue());
  };
  return queue().then(() => Promise.all(result));
}

use:

// Test code const result = [];
for (let index = 0; index < 10; index++) {
  result.push(function() {
    return new Promise((resolve, reject) => {
      console.log("start" + index, new Date().toLocaleString());
      setTimeout(() => {
        resolve(index);
        console.log("End" + index, new Date().toLocaleString());
      }, parseInt(Math.random() * 10000));
    });
  });
}

PromiseLimit(result).then(data => {
  console.log(data);
});

Modify the test code and add random failure logic

// Modify the test code to fail or succeed randomly const result = [];
for (let index = 0; index < 10; index++) {
  result.push(function() {
    return new Promise((resolve, reject) => {
      console.log("start" + index, new Date().toLocaleString());
      setTimeout(() => {
        if (Math.random() > 0.5) {
          resolve(index);
        } else {
          reject(index);
        }
        console.log("End" + index, new Date().toLocaleString());
      }, parseInt(Math.random() * 1000));
    });
  });
}
PromiseLimit(result).then(
  data => {
    console.log("success", data);
  },
  data => {
    console.log("failed", data);
  }
);

7. async and await combined with promise all

async function PromiseAll(promises,batchSize=10) {
 const result = [];
 while(promises.length > 0) {
   const data = await Promise.all(promises.splice(0,batchSize));
   result.push(...data);
 }
return result;
}

There are two problems with this writing:

  • 1. promises are created before calling Promise.all , promise are actually executed
  • 2. Your implementation must wait for the previous batchSize個promise resolve before running the next batchSize , that is, promise all must succeed.

The improvements are as follows:

async function asyncPool(array,poolLimit,iteratorFn) {
  const ret = [];
  const executing = [];
  for (const item of array) {
    const p = Promise.resolve().then(() => iteratorFn(item, array));
    ret.push(p);

    if (poolLimit <= array.length) {
      const e = p.then(() => executing.splice(executing.indexOf(e), 1));
      executing.push(e);
      if (executing.length >= poolLimit) {
        await Promise.race(executing);
      }
    }
  }
  return Promise.all(ret);
}

use:

const timeout = i => new Promise(resolve => setTimeout(() => resolve(i), i));
return asyncPool( [1000, 5000, 3000, 2000], 2,timeout).then(results => {
    ...
});

This is the end of this article about serial and parallel operations in JavaScript asynchronous operations. For more relevant content about serial and parallel operations in JavaScript asynchronous operations, please search for previous articles on 123WORDPRESS.COM or continue to browse the following related articles. I hope everyone will support 123WORDPRESS.COM in the future!

You may also be interested in:
  • Summary of several common processing methods for JavaScript asynchronous operations
  • Solve the problem of using $q.all to sort the order of Angularjs asynchronous operation background requests
  • Detailed explanation of asynchronous operation examples of ES6 javascript
  • async/await and promise (asynchronous operation problem in nodejs)
  • Thinkjs page jump synchronous and asynchronous operation
  • JavaScript-Solve the asynchronous operation of mongoose data query
  • A simple Node.js asynchronous operation manager sharing

<<:  HTML css js implements Tab page sample code

>>:  How to change fixed positioning of child elements to absolute positioning by CSS3 transform

Recommend

MySQL Basic Tutorial Part 1 MySQL5.7.18 Installation and Connection Tutorial

Starting from this article, a new series of artic...

Reasons and solutions for the failure of React event throttling effect

Table of contents The problem here is: Solution 1...

Javascript tree menu (11 items)

1. dhtmlxTree dHTMLxTree is a feature-rich Tree M...

Summary of the differences between global objects in nodejs and browsers

In Node.js, a .js file is a complete scope (modul...

Implementation of CSS dynamic height transition animation effect

This question originated from a message on Nugget...

CSS sets the box container (div) height to always be 100%

Preface Sometimes you need to keep the height of ...

Differences between ES6 inheritance and ES5 inheritance in js

Table of contents Inheritance ES5 prototype inher...

Notes on upgrading to mysql-connector-java8.0.27

Recently, an online security scan found a vulnera...

Installation tutorial of mysql5.7.21 decompression version under win10

Install the unzipped version of Mysql under win10...

Cross-browser local storage Ⅰ

Original text: http://www.planabc.net/2008/08/05/...

Tips for List Building for Website Maintenance Pages

And, many times, maintenance requires your website...

Handwritten Vue2.0 data hijacking example

Table of contents 1: Build webpack 2. Data hijack...

How to migrate the data directory in mysql8.0.20

The default storage directory of mysql is /var/li...