Questions tagged [scipy]
SciPy is an open source library of algorithms and mathematical tools for the Python programming language.
17,725
questions
456
votes
12answers
831k views
How do I read CSV data into a record array in NumPy?
I wonder if there is a direct way to import the contents of a CSV file into a record array, much in the way that R's read.table(), read.delim(), and read.csv() family imports data to R's data frame?
...
435
votes
10answers
423k views
Read .mat files in Python
Is it possible to read binary MATLAB .mat files in Python?
I've seen that SciPy has alleged support for reading .mat files, but I'm unsuccessful with it. I installed SciPy version 0.7.0, and I can't ...
376
votes
14answers
360k views
Sorting arrays in NumPy by column
How can I sort an array in NumPy by the nth column?
For example,
a = array([[9, 2, 3],
[4, 5, 6],
[7, 0, 5]])
I'd like to sort rows by the second column, such that I get back:
...
263
votes
7answers
315k views
dropping infinite values from dataframes in pandas?
what is the quickest/simplest way to drop nan and inf/-inf values from a pandas DataFrame without resetting mode.use_inf_as_null? I'd like to be able to use the subset and how arguments of dropna, ...
262
votes
8answers
51k views
Relationship between SciPy and NumPy
SciPy appears to provide most (but not all [1]) of NumPy's functions in its own namespace. In other words, if there's a function named numpy.foo, there's almost certainly a scipy.foo. Most of the time,...
232
votes
25answers
917k views
Import Error: No module named numpy
I have a very similar question to this question, but still one step behind. I have only one version of Python 3 installed on my Windows 7 (sorry) 64-bit system.
I installed numpy following this link -...
231
votes
9answers
85k views
Does Conda replace the need for virtualenv?
I recently discovered Conda after I was having trouble installing SciPy, specifically on a Heroku app that I am developing.
With Conda you create environments, very similar to what virtualenv does. ...
225
votes
9answers
297k views
How to smooth a curve in the right way?
Lets assume we have a dataset which might be given approximately by
import numpy as np
x = np.linspace(0,2*np.pi,100)
y = np.sin(x) + np.random.random(100) * 0.2
Therefore we have a variation of 20% ...
222
votes
28answers
414k views
Moving average or running mean
Is there a SciPy function or NumPy function or module for Python that calculates the running mean of a 1D array given a specific window?
207
votes
15answers
358k views
Installing SciPy with pip
It is possible to install NumPy with pip using pip install numpy.
Is there a similar possibility with SciPy? (Doing pip install scipy does not work.)
Update
The package SciPy is now available to ...
204
votes
16answers
392k views
Calculating Pearson correlation and significance in Python
I am looking for a function that takes as input two lists, and returns the Pearson correlation, and the significance of the correlation.
197
votes
3answers
132k views
What are the differences between Pandas and NumPy+SciPy in Python? [closed]
They both seem exceedingly similar and I'm curious as to which package would be more beneficial for financial data analysis.
184
votes
7answers
289k views
How to do exponential and logarithmic curve fitting in Python? I found only polynomial fitting
I have a set of data and I want to compare which line describes it best (polynomials of different orders, exponential or logarithmic).
I use Python and Numpy and for polynomial fitting there is a ...
183
votes
13answers
297k views
Is there a library function for Root mean square error (RMSE) in python?
I know I could implement a root mean squared error function like this:
def rmse(predictions, targets):
return np.sqrt(((predictions - targets) ** 2).mean())
What I'm looking for if this rmse ...
178
votes
6answers
287k views
How to add a new row to an empty numpy array
Using standard Python arrays, I can do the following:
arr = []
arr.append([1,2,3])
arr.append([4,5,6])
# arr is now [[1,2,3],[4,5,6]]
However, I cannot do the same thing in numpy. For example:
arr =...
177
votes
7answers
123k views
Does Python SciPy need BLAS?
numpy.distutils.system_info.BlasNotFoundError:
Blas (http://www.netlib.org/blas/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site....
172
votes
2answers
190k views
numpy.where() detailed, step-by-step explanation / examples [closed]
I have trouble properly understanding numpy.where() despite reading the doc, this post and this other post.
Can someone provide step-by-step commented examples with 1D and 2D arrays?
171
votes
8answers
189k views
Specifying and saving a figure with exact size in pixels
Say I have an image of size 3841 x 7195 pixels. I would like to save the contents of the figure to disk, resulting in an image of the exact size I specify in pixels.
No axis, no titles. Just the ...
161
votes
2answers
265k views
A tool to convert MATLAB code to Python [closed]
I have a bunch of MATLAB code from my MS thesis which I now want to convert to Python (using numpy/scipy and matplotlib) and distribute as open-source. I know the similarity between MATLAB and Python ...
158
votes
8answers
232k views
Installing SciPy and NumPy using pip
I'm trying to create required libraries in a package I'm distributing. It requires both the SciPy and NumPy libraries.
While developing, I installed both using
apt-get install scipy
which installed ...
155
votes
7answers
310k views
How to normalize a NumPy array to within a certain range?
After doing some processing on an audio or image array, it needs to be normalized within a range before it can be written back to a file. This can be done like so:
# Normalize audio channels to ...
150
votes
10answers
141k views
Fitting empirical distribution to theoretical ones with Scipy (Python)?
INTRODUCTION: I have a list of more than 30,000 integer values ranging from 0 to 47, inclusive, e.g.[0,0,0,0,..,1,1,1,1,...,2,2,2,2,...,47,47,47,...] sampled from some continuous distribution. The ...
150
votes
9answers
138k views
Peak-finding algorithm for Python/SciPy
I can write something myself by finding zero-crossings of the first derivative or something, but it seems like a common-enough function to be included in standard libraries. Anyone know of one?
My ...
145
votes
15answers
225k views
scipy.misc module has no attribute imread?
I am trying to read an image with scipy. However it does not accept the scipy.misc.imread part. What could be the cause of this?
>>> import scipy
>>> scipy.misc
<module 'scipy....
145
votes
2answers
172k views
Should I use scipy.pi, numpy.pi, or math.pi?
In a project using SciPy and NumPy, should I use scipy.pi, numpy.pi, or math.pi?
138
votes
14answers
195k views
Multiple linear regression in Python
I can't seem to find any python libraries that do multiple regression. The only things I find only do simple regression. I need to regress my dependent variable (y) against several independent ...
136
votes
3answers
75k views
Confusion between numpy, scipy, matplotlib and pylab
Numpy, scipy, matplotlib, and pylab are common terms among they who use python for scientific computation.
I just learn a bit about pylab, and I got confused.
Whenever I want to import numpy, I can ...
135
votes
6answers
103k views
Numpy: Divide each row by a vector element
Suppose I have a numpy array:
data = np.array([[1,1,1],[2,2,2],[3,3,3]])
and I have a corresponding "vector:"
vector = np.array([1,2,3])
How do I operate on data along each row to either subtract ...
134
votes
4answers
218k views
Compute a confidence interval from sample data
I have sample data which I would like to compute a confidence interval for, assuming a normal distribution.
I have found and installed the numpy and scipy packages and have gotten numpy to return a ...
131
votes
5answers
79k views
How to check BLAS/LAPACK linkage in NumPy and SciPy?
I am builing my numpy/scipy environment based on blas and lapack more or less based on this walk through.
When I am done, how can I check, that my numpy/scipy functions really do use the previously ...
128
votes
6answers
204k views
How to create a density plot in matplotlib?
In R I can create the desired output by doing:
data = c(rep(1.5, 7), rep(2.5, 2), rep(3.5, 8),
rep(4.5, 3), rep(5.5, 1), rep(6.5, 8))
plot(density(data, bw=0.5))
In python (with matplotlib)...
120
votes
21answers
228k views
Can't install Scipy through pip
When installing scipy through pip with :
pip install scipy
Pip fails to build scipy and throws the following error:
Cleaning up...
Command /Users/administrator/dev/KaggleAux/env/bin/python2.7 -c "...
119
votes
13answers
249k views
ImportError in importing from sklearn: cannot import name check_build
I am getting the following error while trying to import from sklearn:
>>> from sklearn import svm
Traceback (most recent call last):
File "<pyshell#17>", line 1, in <module>
...
118
votes
6answers
183k views
binning data in python with scipy/numpy
is there a more efficient way to take an average of an array in prespecified bins? for example, i have an array of numbers and an array corresponding to bin start and end positions in that array, and ...
116
votes
9answers
207k views
Numpy Resize/Rescale Image
I would like to take an image and change the scale of the image, while it is a numpy array.
For example I have this image of a coca-cola bottle:
bottle-1
Which translates to a numpy array of shape (...
113
votes
11answers
97k views
Principal component analysis in Python
I'd like to use principal component analysis (PCA) for dimensionality reduction. Does numpy or scipy already have it, or do I have to roll my own using numpy.linalg.eigh?
I don't just want to use ...
112
votes
6answers
166k views
Histogram Matplotlib
So I have a little problem. I have a data set in scipy that is already in the histogram format, so I have the center of the bins and the number of events per bin. How can I now plot is as a histogram. ...
111
votes
1answer
29k views
How can I perform two-dimensional interpolation using scipy?
This Q&A is intended as a canonical(-ish) concerning two-dimensional (and multi-dimensional) interpolation using scipy. There are often questions concerning the basic syntax of various ...
110
votes
9answers
146k views
How to calculate cumulative normal distribution?
I am looking for a function in Numpy or Scipy (or any rigorous Python library) that will give me the cumulative normal distribution function in Python.
100
votes
7answers
323k views
Plotting a fast Fourier transform in Python
I have access to NumPy and SciPy and want to create a simple FFT of a data set. I have two lists, one that is y values and the other is timestamps for those y values.
What is the simplest way to feed ...
98
votes
9answers
204k views
How to calculate probability in a normal distribution given mean & standard deviation?
How to calculate probability in normal distribution given mean, std in Python? I can always explicitly code my own function according to the definition like the OP in this question did: Calculating ...
94
votes
12answers
79k views
scipy: savefig without frames, axes, only content
In numpy/scipy I have an image stored in an array. I can display it, I want to save it using savefig without any borders, axes, labels, titles,... Just pure image, nothing else.
I want to avoid ...
92
votes
9answers
105k views
ImportError: cannot import name NUMPY_MKL
I am trying to run the following simple code
import scipy
scipy.test()
But I am getting the following error
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
...
91
votes
13answers
316k views
ImportError: No module named scipy
I am using Python 2.7 and trying to get PyBrain to work.
But I get this error even though scipy is installed -
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
...
88
votes
9answers
116k views
Quantile-Quantile Plot using SciPy
How would you create a qq-plot using Python?
Assuming that you have a large set of measurements and are using some plotting function that takes XY-values as input. The function should plot the ...
88
votes
4answers
114k views
How to transform numpy.matrix or array to scipy sparse matrix
For SciPy sparse matrix, one can use todense() or toarray() to transform to NumPy matrix or array. What are the functions to do the inverse?
I searched, but got no idea what keywords should be the ...
87
votes
2answers
74k views
Two-sample Kolmogorov-Smirnov Test in Python Scipy
I can't figure out how to do a Two-sample KS test in Scipy.
After reading the documentation scipy kstest
I can see how to test where a distribution is identical to standard normal distribution
from ...
86
votes
5answers
150k views
How can I plot a histogram such that the heights of the bars sum to 1 in matplotlib?
I'd like to plot a normalized histogram from a vector using matplotlib. I tried the following:
plt.hist(myarray, normed=True)
as well as:
plt.hist(myarray, normed=1)
but neither option produces a ...
85
votes
3answers
140k views
How to implement band-pass Butterworth filter with Scipy.signal.butter
UPDATE:
I found a Scipy Recipe based in this question! So, for anyone interested, go straight to: Contents » Signal processing » Butterworth Bandpass
I'm having a hard time to achieve what seemed ...
85
votes
4answers
119k views