19

I have a list of sets :

L = [set([1, 4]), set([1, 4]), set([1, 2]), set([1, 2]), set([2, 4]), set([2, 4]), set([5, 6]), set([5, 6]), set([3, 6]), set([3, 6]), set([3, 5]), set([3, 5])]

(actually in my case a conversion of a list of reciprocal tuples)

and I want to remove duplicates to get :

L = [set([1, 4]), set([1, 2]), set([2, 4]), set([5, 6]), set([3, 6]), set([3, 5])]

But if I try :

>>> list(set(L))
TypeError: unhashable type: 'set'

Or

>>> list(np.unique(L))
TypeError: cannot compare sets using cmp()

How do I get a list of sets with distinct sets?

2
  • One way could be to convert the list of set to list of list and then remove the duplicates and then converting the list elements back to set.
    – ZdaR
    Commented Aug 30, 2015 at 13:18
  • @ZdaR I'm going to say this in bold: Your hint is wrong. Following the part of your instructions which say "convert the list of set to list of list and then remove the dupicates", we get that list(map(list, [{3, 11}, {11, 3}])) outputs [[3, 11], [11, 3]].
    – wlad
    Commented Mar 25, 2017 at 13:15

4 Answers 4

26

The best way is to convert your sets to frozensets (which are hashable) and then use set to get only the unique sets, like this

>>> list(set(frozenset(item) for item in L))
[frozenset({2, 4}),
 frozenset({3, 6}),
 frozenset({1, 2}),
 frozenset({5, 6}),
 frozenset({1, 4}),
 frozenset({3, 5})]

If you want them as sets, then you can convert them back to sets like this

>>> [set(item) for item in set(frozenset(item) for item in L)]
[{2, 4}, {3, 6}, {1, 2}, {5, 6}, {1, 4}, {3, 5}]

If you want the order also to be maintained, while removing the duplicates, then you can use collections.OrderedDict, like this

>>> from collections import OrderedDict
>>> [set(i) for i in OrderedDict.fromkeys(frozenset(item) for item in L)]
[{1, 4}, {1, 2}, {2, 4}, {5, 6}, {3, 6}, {3, 5}]
2
  • @PM2Ring It exists in 2.7 so that statement is not strictly true.
    – Dan D.
    Commented Aug 30, 2015 at 14:05
  • @DanD: Oops! I originally just looked at the Python 3 docs, where it says "New in version 3.1". But I just now checked in the Python 2 docs & it says "New in version 2.7."; I should've checked there to see if it had been back-ported. Sorry about that. I'll remove my comment(s).
    – PM 2Ring
    Commented Aug 30, 2015 at 14:34
4

An alternative using a loop:

result = list()
for item in L:
    if item not in result:
        result.append(item)
8
  • 1
    It is better to use [] than list() to create a list Commented Aug 30, 2015 at 13:33
  • 1
    @ReblochonMasque: it's because [] is literal syntax (allowing the empty list to be instantiated at compile time) whereas list() is a function call (the function name must be looked up at run time and then called to return the empty list). The latter is slightly more expensive.
    – Alex Riley
    Commented Aug 30, 2015 at 16:51
  • 2
    My preference for list() is merely aesthetic. Up until today I didn't even know it was slightly more expensive. If that's the only issue, I don't think it's that big of a deal. Commented Aug 30, 2015 at 17:34
  • 1
    @ReblochonMasque I investigated a little and they behave differently, but it's an insignificant difference, as you say. Commented Aug 31, 2015 at 9:21
  • 1
    @thefourtheye I think that's a bit of a stretch, unless I'm missing something. You'd have to make the mistake of assigning to list instead of assigning list() to something. Commented Aug 31, 2015 at 9:25
1

Here is another alternative

yourNewSet = map(set,list(set(map(tuple,yourSet))))
1
  • This answer is wrong. Two equal sets can be mapped to two different tuples. I have seen this happen. For example: ss = [{3, 11}, {11, 3}]; list(map(tuple, ss)) outputs [(3, 11), (11, 3)]
    – wlad
    Commented Mar 25, 2017 at 13:00
0

There is another alternative.

import itertools
list_sets = [set(['a', 'e', 'f']), set(['c', 'b', 'f']), set(['a', 'e', 'f']), set(['a', 'd']), set(['a', 'e', 'f'])]

lists = [list(s) for s in list_sets] # convert a list of sets to a list of lists
lists.sort()
lists_remove_duplicates = [lists for lists,_ in itertools.groupby(lists)]
print(lists_remove_duplicates)

# output
[['a', 'd'], ['a', 'e', 'f'], ['c', 'b', 'f']]

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.