1

I recently started coding in Python 2.7. I'm a molecular biologist. I'm writing a script that involves creating lists like this one:

mylist = [[0, 4, 6, 1], 102]

These lists are incremented by adding an item to mylist[0] and summing a value to mylist[1].

To do this, I use the code:

def addres(oldpep, res):
    return [oldpep[0] + res[0], oldpep[1] + res[1]]

Which works well. Since mylist[0] can become a bit long, and I have millions of these lists to take care of, I thought that using append or extend might make my code faster, so I tried:

def addres(pep, res):
    pep[0].extend(res[0])
    pep[1] += res[1]
    return pep

Which in my mind should give the same result. It does give the same result when I try it on an arbitrary list. But when I feed it to the million of lists, it gives me a very different result. So... what's the difference between the two? All the rest of the script is exactly the same. Thank you! Roberto

8
  • 1
    What reason did you have to believe that using append or extend might make your code faster? What part of things were you trying to improve?
    – abarnert
    Commented Nov 6, 2013 at 21:35
  • I noticed in a couple of other scripts that if a is a very long list, then the code a = a + [b] is much slower than a.append(b) ... Am I wrong? Thanks!
    – Roberto
    Commented Nov 6, 2013 at 21:42
  • @Roberto You're right. The reason, though, is exactly because of the copying behavior that's breaking your code. The difference is that a.append(b) modifies the list a to have an extra element, where a += [b] creates a new list containing the contents of a followed by the element b, and then assigns it to the name a. If nobody else is using the list that used to be called a, it's then thrown away, for the same net effect. The problem is that in your code, apparently, someone else is still using the old list by that name.
    – Danica
    Commented Nov 6, 2013 at 21:46
  • a = a + [b] is creating a new list containing one element (b), then combining the two lists together into another new list; a.append(b) is just adding a new item to the existing list a -- so the second operation is faster because it's doing significantly less work. Commented Nov 6, 2013 at 21:49
  • @Roberto: But even with lists of tens of millions of values, all those appends add up to milliseconds, and in most programs that's unlikely to be anywhere near a bottleneck. It's always important to figure out what's making your code slow and speed up that part, not to just pick something arbitrary and speed it up.
    – abarnert
    Commented Nov 6, 2013 at 21:51

2 Answers 2

3

The difference is that the second version of addres modifies the list that you passed in as pep, where the first version returns a new one.

>>> mylist = [[0, 4, 6, 1], 102]
>>> list2 = [[3, 1, 2], 205]
>>> addres(mylist, list2)
[[0, 4, 6, 1, 3, 1, 2], 307]
>>> mylist
[[0, 4, 6, 1, 3, 1, 2], 307]

If you need to not modify the original lists, I don't think you're going to really going to get a faster Python implementation of addres than the first one you wrote. You might be able to deal with the modification, though, or come up with a somewhat different approach to speed up your code if that's the problem you're facing.

3
  • Hmm this is right, and I see your point. Thanks! I thought pep would only change within the function, my mistake. But... the problem persists if I use newpep = list(pep) then modify and return newpep. So I'm still doing something silly.
    – Roberto
    Commented Nov 6, 2013 at 21:38
  • @Roberto list(pep[0]).extent(res[0]) is not really going to be any faster then pep[0] + res[0]
    – cmd
    Commented Nov 6, 2013 at 21:42
  • 1
    The problem is that list(pep) only does a shallow copy, so that although the lists themselves are different, their first element points to the same object in memory. That is, if you do newpep = list(pep), then newpep[0] = "something else" won't change pep[0]. But what you're doing is newpep[0].extend(res[0]), which changes the list pointed to by both pep[0] and newpep[0]. (Also, as @cmd points out, there's not really any speed reason to do it this way.)
    – Danica
    Commented Nov 6, 2013 at 21:42
0

List are objects in python which are passed by reference.

a=list()

This doesn't mean that a is the list but a is pointing towards a list just created.

In first example, you are using list element and creating a new list, an another object while in the second one you are modifying the list content itself.

2
  • 1
    All values in Python are passed by reference - it's just that it doesn't matter for the immutable types. Commented Nov 6, 2013 at 22:06
  • I think this terminology is highly misleading and confusing. Python doesn't pass by reference in the same sense that C-family languages mean—it doesn't pass references to variables, it passes references to values. Which Java explicitly calls "pass by value". So, what is a good name for it? There is no name that answers the question or gives any information without a corresponding explanation.
    – abarnert
    Commented Nov 6, 2013 at 22:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.