5

Say I have a list of names in python, such as the following:

names = ['Alice','Bob','Carl','Dave','Bob','Earl','Carl','Frank','Carl']

Now, I want to get rid of the fact that there are duplicate names in this list, but I don't want to remove them. Instead, for each name that appears more than once in this list, I want to append a suffix to that name, where the suffix is the n-th time the name has appeared, while preserving the order of the list. Since there are 3 Carls in the list, I want to be able to refer to them as Carl_1, Carl_2, and Carl_3 respectively. So in this case the desired output is as follows:

names = ['Alice','Bob_1','Carl_1','Dave','Bob_2','Earl','Carl_2','Frank','Carl_3']

I can do this by looping through the list and modifying each name if it needs to be modified, for example with something like the following code.

def mark_duplicates(name_list):
    output = []
    duplicates = {}
    for name in name_list:
        if name_list.count(name) = 1:
            output.append(name)
        else:
            if name in duplicates:
                duplicates['name'] += 1
            else:
                duplicates['name'] = 1
            output.append(name + "_" + str(duplicates['name']))
    return output

However this is a lot of work and a lot of lines of code for something that I suspect shouldn't be very hard to do. Is there a simpler way to accomplish what I want to do? For example, using something such as list comprehension or a package like itertools or something?

4
  • I don't think there is a reasonable one-liner, if that is what you are looking for. Anyway, what do you expect to get if input is ['Alice', 'Alice', 'Alice_1', 'Alice_2']?
    – zvone
    Commented Oct 25, 2016 at 20:19
  • Theoretically that type of input will never happen. If it does, however, then I guess I'd be screwed because our resulting list would be ['Alice_1','Alice_2','Alice_1','Alice_2'].
    – K. Mao
    Commented Oct 25, 2016 at 20:21
  • The reason I asked is because you maybe wanted result to be ['Alice_1','Alice_2','Alice_3','Alice_4'] (I don't know what it's for). That would require a different algorihm altogether.
    – zvone
    Commented Oct 25, 2016 at 20:24
  • I don't really have a good answer for how the problem you raised should be handled actually. The underscore deliminator was chosen because I didn't expect it to appear in any of the "names". Really I could have chosen a strange separator that would never appear like 'Alice[__|]1', but I assumed that underscore wouldn't appear in a normal name. If I need to change it I can always modify the separator into some other string.
    – K. Mao
    Commented Oct 25, 2016 at 20:30

5 Answers 5

8

collections.Counter can help cut down on the bookkeeping a bit:

In [106]: out = []

In [107]: fullcount = Counter(names)

In [108]: nc = Counter()

In [109]: for n in names:
     ...:     nc[n] += 1
     ...:     out.append(n if fullcount[n] == 1 else '{}_{}'.format(n, nc[n]))
     ...:

In [110]: out
Out[110]:
['Alice', 'Bob_1', 'Carl_1', 'Dave', 'Bob_2', 'Earl', 'Carl_2', 'Frank', 'Carl_3']
0

If you don't care about the initial order, you could think of this this way:

  • Count the number of times each name appears
  • Generate a list where, if the name only appears once, we don't add anything, but if it appears more than once, it adds _1, _2... to the second and subsequent appearances.

This means, you could use a collections.Counter to get the job done:

import collections

names = ['Alice', 'Bob', 'Carl', 'Dave', 'Bob', 'Earl', 'Carl', 'Frank', 'Carl']

counter = collections.Counter(names)
print("Counter: %s" % counter)

result = []
for name, counts in counter.iteritems():
    result.append(name)
    for i in range(1, counts):
        result.append("%s_%d" % (name, i))
print(result)

Which outputs:

Counter: Counter({'Carl': 3, 'Bob': 2, 'Earl': 1, 'Frank': 1, 'Alice': 1, 'Dave': 1})
['Earl', 'Frank', 'Alice', 'Dave', 'Carl', 'Carl_1', 'Carl_2', 'Bob', 'Bob_1']

If you wanted to add the _1, _2 suffix to all the names that have more than one occurrence in the list, but leave the names that only occur once untouched, you could do:

import collections

names = ['Alice', 'Bob', 'Carl', 'Dave', 'Bob', 'Earl', 'Carl', 'Frank', 'Carl']

counter = collections.Counter(names)
print("Counter: %s" % counter)

result = []
for name, counts in counter.iteritems():
    if counts == 1:
        result.append(name)
    else:
        for i in range(counts):
            result.append("%s_%d" % (name, i + 1))
print(result)

Which outputs:

Counter: Counter({'Carl': 3, 'Bob': 2, 'Earl': 1, 'Frank': 1, 'Alice': 1, 'Dave': 1})
['Earl', 'Frank', 'Alice', 'Dave', 'Carl_1', 'Carl_2', 'Carl_3', 'Bob_1', 'Bob_2']
0

If ['Alice', 'Bob', 'Carl', 'Dave', 'Bob_2', 'Earl', 'Carl_2', 'Frank', 'Carl_3'] is acceptable output (the first person not having a _1 added on) then I would suggest the following:

counts = {}
def append(name):
    try:
        counts[name] += 1
        return True
    except:
        counts[name] = 1
        return False

def get_duplicates():
    return ['_'.join([name, str(counts[name])]) if append(name) else name for name in names]

The benefit to this approach is I only go through names a single time, which is why I cannot know ahead of time whether more will appear.


To meet the spec, I can further modify append:

def append(name):
    if names.count(name) != 1:
        try:
            counts[name] += 1
        except:
            counts[name] = 1
        return True
    else:
        return False

which will give the expected result:

['Alice', 'Bob_1', 'Carl_1', 'Dave', 'Bob_2', 'Earl', 'Carl_2', 'Frank', 'Carl_3']
0

Another solution which uses enumerate:

>>> names = ['Alice','Bob','Carl','Dave','Bob','Earl','Carl','Frank','Carl']
>>> processed = []
>>> for n in names:
...     if n not in processed:
...         indices = [i for i,name in enumerate(names) if name == n]
...         if len(indices) > 1:
...             suffix = 1
...             for i in indices:
...                 names[i] = "{}_{}".format(names[i], suffix)
...                 suffix += 1
...     if n.split('_')[0] not in processed:
...         processed.append(n)
...
>>>
>>> names
['Alice', 'Bob_1', 'Carl_1', 'Dave', 'Bob_2', 'Earl', 'Carl_2', 'Frank', 'Carl_3']
0

The following code should do what you're looking for and uses comprehensions:

def get_duplicates(names):
    counts = { k: 0 for k in names }
    output = []
    for name in names:
        if count[name] == 0:
            output.append(name)
            counts[name] += 1
        else:
            output.append("{}_{}".format(name, counts[name]))
            counts[name] += 1
    return output

Update: I fixed the code in my answer to properly return what OP was looking for. Not the best way, but it doesn't require use of another library and uses 1 dict comprehension and 1 loop.

6
  • counts[name] needs to be wrapped in str() Commented Oct 25, 2016 at 20:22
  • I believe that should be fixed now that I switched to format. Commented Oct 25, 2016 at 20:23
  • It puts a count on items that appear only once though.
    – Francisco
    Commented Oct 25, 2016 at 20:24
  • 1
    return ['_'.join([name, str(counts[name])]) if counts[name] > 1 else name for name in names] fixed. List comprehension if/else format is [this if condition else that for iterator] Commented Oct 25, 2016 at 20:24
  • 1
    This one puts the same number after each of the names (e.g., the three Carls in the example come back as ['Carl_2', 'Carl_2', 'Carl_2'] rather than ['Carl_1', 'Carl_2', 'Carl_3']).
    – Randy
    Commented Oct 25, 2016 at 20:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.