0

I seem to only find ways to remove duplicates from an array using other JS libraries but I'm looking to do it in pure JS or typescript since I'm working on an Angular project.

My problem is that I may get an array that has duplicate entries, such as this one:

data [0: {Id: 1, Definition: "House"},
      1: {Id: 1, Definition: "House"}]

And I want to filter it out so that I only get

data [0: {Id: 1, Definition: "House"}]

I've tried it using this method but I still get duplicate entries

let uniqueArray = data.filter(function(item, pos) {
    return data.indexOf(item) == pos;
})
3
  • This data.indexOf(item) == pos will always be true in your case, since find will test Object identity. You need something like: jsfiddle.net/khrismuc/hnLowjy2
    – user5734311
    Sep 26, 2018 at 9:17
  • how do you want to filter out the duplicate entries? Is it only based on the id key?
    – Ashish
    Sep 26, 2018 at 9:23
  • @Ashish either compare the ID or the String in "Definition"... both will work
    – bobdolan
    Sep 26, 2018 at 9:26

3 Answers 3

6

You can achieve what you want in this way:

You can check if the value is already there in your final array using 'some'

data = [{Id: 1, Definition: "House"}, {Id: 1, Definition: "House"}]

const finalOut = []
data.forEach((value) => {
    if (!finalOut.some(x=> (x.Id === value.Id || x.Definition === value.Definition))) 
   {
        finalOut.push(value)
    }
})

You can also achieve this by 'reduce' in clean and elegant way:

const finalOut2 = data.reduce((acc, cur) => acc.some(x=> (x.Id === cur.Id || x.Definition === cur.Definition)) ? acc : acc.concat(cur), [])

As suggested by @Ezequiel using some inside forEach or reduce making the time complexity of order of n square. For smaller sets of data using reduce and some is an elegant approach. But if you are dealing with arrays of very large length, you must avoid order of n square time complexity Here is one such approach with filter:

//Here storing every value of data is inside lookupObj after filtering it. 
//And checking if value is filtered based on if key of the value inside lookupObj

const lookupObj = {} 
const finalOut3 = data.filter(
    x => {
        const is_unique = !(lookupObj[`Id_${x.Id}`] || lookupObj[`Id_${x.Definition}`])
        lookupObj[`Id_${x.Id}`] = true
        lookupObj[`Id_${x.Definition}`] = true
        return is_unique
    }
)
3
  • Thank you, this finally worked!
    – bobdolan
    Sep 26, 2018 at 9:34
  • Do be aware that this method runs in O(n²) or quadratic time, because hidden in here is a nested loop. This isn't a problem with small arrays, but gets slow if data.length becomes large. The other methods outlined in this answer run in O(n) or linear time and don't suffer the same fate. Sep 26, 2018 at 9:51
  • I have updated the answer with new function to reduce O(n²) complexity.
    – Ashish
    Jun 18, 2019 at 9:41
2

You can use Id key of the object to get the unique object using array#reduce in an object accumulator and get all the object of this object using Object.values().

let data = [{Id: 1, Definition: "House"},{Id: 1, Definition: "House"}, {Id: 2, Definition: "House2"}, {Id: 2, Definition: "House2"}],
    result = Object.values(data.reduce((r, o) =>  {
      r[o.Id] = r[o.Id] || {...o};
      return r;
    },{}));
console.log(result);

2
  • sadly I get a syntax error [ts] Property "values" is not available for type "ObjectConstructor"
    – bobdolan
    Sep 26, 2018 at 9:30
  • You can use Object.prototype.values = function(obj) { var res = []; for (var i in obj) { if (obj.hasOwnProperty(i)) { res.push(obj[i]); } } return res; }; Sep 26, 2018 at 9:33
0

If the ID is what determines identity, then you could use a Map or a plain Object to dedupe. Same IDs will take the same spot on the map, so you end up with unique objects (if the above assumptions about IDs hold).

let data = [{Id: 1, Definition: "House"}, {Id: 1, Definition: "House"}];
let idToObj = {};
data.forEach((o) => idToObj[o.Id] = o);
let uniqueArray = Object.values(idToObj);

EDIT: In case that you want the same but you may have objects with the same ID but differing in other fields, a Map comes in handy because it can take whole objects as keys:

let data = [{Id: 1, Definition: "House1"}, {Id: 1, Definition: "House2"}];
let map = new Map();
data.forEach((o) => map.set(o, true));
let uniqueArray = [...map.keys()];
1

Not the answer you're looking for? Browse other questions tagged or ask your own question.