2

I am using scipy.interpolate.griddata in a loop and it is causing Python to accumulate memory with each loop. This is using Python 2.7.5 and SciPy 0.12.0 (installed through Macports) on Mac OS X 10.6.8.

This code below, adapted from the scipy.interpolate.griddata reference guide, illustrates my point.

import numpy as np
from scipy.interpolate import griddata

def func(x, y):
    return x*(1-x)*np.cos(4*np.pi*x) * np.sin(4*np.pi*y**2)**2

grid_x, grid_y = np.mgrid[0:1:100j, 0:1:200j]
points = np.random.rand(1000, 2)
values = func(points[:,0], points[:,1])

for t in xrange(10000):
    griddata(points, values, (grid_x, grid_y), method='nearest')
    griddata(points, values, (grid_x, grid_y), method='linear')
    griddata(points, values, (grid_x, grid_y), method='cubic')

As I increase the loop, Python will consume more memory.

loop      memory
1        48.4 MB
10       52.7 MB
100      94.6 MB
1000    500.9 MB

I am not sure if this is the intended behaviour of griddata, nor am I certain that this does not happen in past versions of SciPy. According to this question, there is a memory leak associated with Cython (I am using ver. 0.19.1), but it should have been resolved in the final release of SciPy 0.12.0.

I appreciate any suggestions on how I can fix or workaround this issue, or any advice if I'm doing something wrong. Let me know if you need more information.

1 Answer 1

2

If still important, the problem is that scipy 0.12.0 and 0.12.1 has probably been released without cython recompilation of the relevant files.

So the following worked for me:

First updating cython to the most recent version 0.19.2, then recompiling scipy from sources (but the ones from github, not from .tar.gz):

git clone https://github.com/scipy/scipy.git
cd scipy
git checkout maintenance/0.12.x
python setup.py build
cd ..
pip install -U ./scipy

Then the example from the post works without leaking.

UPDATE: Actually I've realised that by mistake did /0.13.x branch instead -- the /0.12.x still shows the memory leak...

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.