JS array deduplication details

1 Test Cases

// Test case const a = {};
const b = { c: 1 };
const array = [
  1, 1, "1", "1",
  {}, {}, { c: 1 }, { c: 1},
  a, a, b, b, 
  [], [], [1], [1],
  undefined, undefined,
  null, null,
  NaN, NaN,
];

2 JS array deduplication 4 types

2.1 Element Comparison

This type removes duplicates by comparing array elements.

2.1.1 Double-layer for loop comparison (commonly used in es5)

Use a double for loop to compare array elements one by one, and use splice method to remove duplicate elements

// Double for loop function uniq1(arr) {
    for (let i = 0; i < arr.length; i++) {
        for (let j = i + 1; j < arr.length; j++) {
            if (arr[i] === arr[j]) {
                arr.splice(j, 1)
                j--
            }
        }
    }
    return arr
}

// Deduplication result// [1,'1',{},{},{c:1},{c:1},{},{c:1},[],[],[1],[1],undefined,null,NaN,NaN]

By comparing the results before and after deduplication, the duplicate NaN are not removed because NaN === NaN is false

2.1.2 Sorting adjacent comparison

Use the sort() method to sort the array elements, then compare adjacent elements and use splice method to remove duplicate elements.

function uni2(arr) {
    arr.sort();
    for (let i = 0; i < arr.length - 1; i++) {
        arr[i] === arr[i + 1] && arr.splice(i + 1, 1) && i--;
    }
    return arr;
}

You can also create a new array and put non-repeating elements into the new array

function uniq3(arr) {
    arr = arr.sort()
    const newArr = [arr[0]]
    for (let i = 1; i < arr.length; i++) {
        if (arr[i] !== arr[i - 1]) {
            newArr.push(arr[i])
        }
    }
    return newArr
}

// Deduplication result// [[],[],1,'1',[1],[1],NaN,NaN,{},{},{c:1},{c:1},{},{c:1},null,undefined]

Duplicate NaN are not removed because NaN === NaN為false
sort The default sorting order NaN === NaN為false
sort is to convert elements to strings. Objects converted to strings are all [object Object] , so sort method cannot sort objects in an array, and it may not be able to remove duplicate objects unless the duplicate objects are adjacent.

2.2 Find element position type

This type removes duplicates by finding the first occurrence of an element.

2.2.1 indexOf

Use indexOf to find out whether the position where the current element first appears is the current position. If so, put it into the new array.

function uniq4(arr) {
    let res = []
    for (let i = 0; i < arr.length; i++) {
        if (arr.indexOf(arr[i]) === i) {
            res.push(arr[i])
        }
    }
    return res
}

// Deduplication result// [1,'1',{},{},{c:1},{c:1},{},{c:1},[],[],[1],[1],undefined,null]

Similarly, because NaN === NaN is false , searching for NaN with indexOf always results in -1, so there will be no NaN in the new array.

2.2.2 findIndex

Use findIndex to find out whether the position where the current element first appears is the current position. If so, put it into the new array.

function uniq5(arr) {
    let res = []
    for (let i = 0; i < arr.length; i++) {
        if (arr.findIndex(item => item === arr[i]) === i) {
            res.push(arr[i])
        }
    }
    return res
}
// Deduplication result// [1,'1',{},{},{c:1},{c:1},{},{c:1},[],[],[1],[1],undefined,null]

Similarly, because NaN === NaN is false , the result of searching for NaN with findIndex is always -1, so there will be no NaN in the new array.

2.3 Element Existence Type

This type removes duplicates by determining whether the current element exists in the new array.

2.3.1 includes

The includes method is used to determine whether an array contains a specified value

function uniq6(arr) {
    let res = []
    for (let i = 0; i < arr.length; i++) {
        if (!res.includes(arr[i])) {
            res.push(arr[i])
        }
    }
    return res
}
// Deduplication result// [1,'1',{},{},{c:1},{c:1},{},{c:1},[],[],[1],[1],undefined,null,NaN]

includes uses a zero-value equality algorithm to determine whether a given element is found, so it can determine whether NaN exists in the new array.

2.3.2 some

The some method is used to test whether at least one element in the array passes the test of the provided function.

function uniq7(arr) {
    let res = []
    for (let i = 0; i < arr.length; i++) {
        if (!res.some(item => item === arr[i])) {
            res.push(arr[i])
        }
    }
    return res
}
// Deduplication result// [1,'1',{},{},{c:1},{c:1},{},{c:1},[],[],[1],[1],undefined,null,NaN,NaN]

Again, === is still used here to compare elements, because NaN === NaN is false , so there will be multiple NaN in the new array.

2.4 Relying on data structure characteristics

This type uses the non-repeatable characteristics of the data structures Map and Set provided by ES6 to deduplicate

2.4.1 Map

The Map structure provided by ES6 can use various types of values ​​(including objects) as keys, and the keys are unique.

function uniq8(arr) {
    const map = new Map()
    for (let i = 0; i < arr.length; i++) {
        !map.has(arr[i]) && map.set(arr[i], true)
    }
    return [...map.keys()]
}
// Deduplication result// [1,'1',{},{},{c:1},{c:1},{},{c:1},[],[],[1],[1],undefined,null,NaN]

map.has method also works for NaN

2.4.2 Set (ES6 most commonly used)

The values ​​of the members of the Set structure are unique and there are no duplicate values.

function uniq9(arr) {
    return [...new Set(arr)]
}

// Deduplication result// [1,'1',{},{},{c:1},{c:1},{},{c:1},[],[],[1],[1],undefined,null,NaN]

3 Supplement

The methods mentioned above can be modified using different Api . For example, where splice method is used to remove array elements, we can use filter method to filter the array to get a new array.

For example, in the includes method, instead of traversing the array with for loop, the reduce method is used instead.

In short, there are many methods, but they all remain the same.

Some deduplication methods are not valid for NaN because NaN === NaN is false . If necessary, you can use Object.is(NaN, NaN) to be true to modify it.

In practical applications, the most common method is to use Set, or you can use the third-party library lodash to process

This is the end of this article about the details of JS array deduplication. For more relevant JS array deduplication content, please search 123WORDPRESS.COM’s previous articles or continue to browse the following related articles. I hope everyone will support 123WORDPRESS.COM in the future!