14

I'm using PyCharm, and typically run parts of my scripts in its Python console for debugging purposes. However, when I have to run something on a "big" variable (that consumes a lot of memory), the console becomes very slow.

Say df is a huge pandas data frame, as soon as I type df. in the console, it won't react anymore for 10-15 seconds. I can't tell whether this is pandas specific, since the only "big" variables I use come from pandas.

I'm running the community edition 3.4.1, pandas 0.14, Python 2.7.3, on Mac OS X 10.9.4 (with 8 GB of ram).

Size of df:

In[94]: df.values.nbytes + df.index.nbytes + df.columns.nbytes
Out[94]: 2229198184
2
  • Have you tried running it in another IDE, like Komodo or PyDev? Do they behave in a similar manner? Sep 18, 2014 at 20:10
  • Also happens with PyCharm 2017, Python 3.6.2 and pandas 0.23
    – fidlr
    Jul 4, 2018 at 9:15

4 Answers 4

0

I've been having the same problem, and have yet to find a proper solution, but this is a workaround that I've been using, from PyCharm hanging for a long time in iPython console with big data.

Instead of typing df.columns type d.columns and then go back and add the f. This at least prevents the hanging, although it renders the auto-completion a bit useless.

0

Have you tried changing the "Variable Loading Policy" to "On Demand"?

0

I have also faced this problem since long. I always deactivate "Show Variables" option and run code. This increases speed everytime I do this.

Then Enable it again when you want to see variables.

0

Did you print df in the console? So the console will apply huge memory to show words to you. Here suggest that only print several columns or cells, refer to pandas.DataFrame.columns

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.