6

This is the python script that I'm trying to run:

n = 50000000000 ##50 billion 
b = [0]*n
for x in range(0,n):
    b[x] = random.randint(1,899999)

... But the output I'm getting is:

E:\python\> python sort.py
Traceback (most recent call last):
  File "E:\python\sort.py", line 8, in <module>
    b = [0]*n
MemoryError

So, what do I do now?

15
  • 7
    You're out of memory. The error message says so.
    – Torxed
    May 18, 2017 at 19:49
  • You are trying to allocate a 40GB array
    – litelite
    May 18, 2017 at 19:51
  • An int takes at least 24 bytes; you have 5,000,000,000 ints. That's 111GB right there.
    – chepner
    May 18, 2017 at 19:51
  • Maybe it would be more useful if you explained what you were planning to do with so many random numbers. Almost certainly there's a way to do it without blowing out your memory.
    – alexis
    May 18, 2017 at 19:53
  • 1
    @litelite No, an int is a Python object, representing an integer with arbitrary precision, not a machine word.
    – chepner
    May 18, 2017 at 19:53

3 Answers 3

10

The size of the list you are generating (which is 50 billion not 5).

An int object instance takes 24 bytes (sys.getsizeof(int(899999)), the upper limit of your random numbers), so that list would take 50,000,000,000 * 24 bytes, which is about 1.09 TB.

In other words to create such a list you would need at least 1118 GB of RAM in your computer.

I don't know what your use case is, but you should consider a different approach to what you are trying to solve (maybe define a generator, or just don't store your numbers in memory and instead directly use the numbers in the for loop).

6
  • That's actually a lower limit generally. The smallest int takes that much, but int objects take variable amounts of memory, e.g. sys.getsizeof(1000000000000000000) == 32, whereas sys.getsizeof(1000) == 28, May 18, 2017 at 20:05
  • @juanpa.arrivillaga In the author's question, the maximum attainable number is 899999, and int objects from 0 up to that number are all 24 bytes in size. (I updated the answer to show how this upper limit is calculated)
    – Tenchi2xh
    May 18, 2017 at 20:06
  • I'm getting 28 on my system... 64bit python 3.5, Mac OSX May 18, 2017 at 20:07
  • Python 3 apparently has 4 more bytes of overhead compared to Python 2
    – Tenchi2xh
    May 18, 2017 at 20:09
  • 1
    Also, just to pile on, we've only been talking about the memory needed for the int objects themselves. The list is another 64 bytes plus 50,000,000,000 pointers (4-8 bytes, depending on your architecture) to reference each int object.
    – chepner
    May 18, 2017 at 20:42
2

Since other people already answered your question here's a quick tip when dealing with big numbers: you can use "_" to separate the digits of your numbers as you wish:

n = 50_000_000_000

is the same as

n = 50000000000

but the former is much easier on the eyes

0

One other possibility is to increase you computers vitual memory. It helped me in my code. I had a max 3000MB virtual memory, when I increased it to 5000MB the memory error was gone.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.