17

I am using scipy's curve_fit to fit a function to some data, and receive the following error;

Cannot cast array data from dtype('O') to dtype('float64') according to the rule 'safe'

which points me to this line in my code;

popt_r, pcov = curve_fit(
                    self.rightFunc, np.array(wavelength)[beg:end][edgeIndex+30:], 
                    np.dstack(transmitted[:,:,c][edgeIndex+30:])[0][0],
                    p0=[self.m_right, self.a_right])

rightFunc is defined as follows;

def rightFunc(self, x, m, const):

    return np.exp(-(m*x + const))

As I understand it, the 'O' type refers to a python object, but I can't see what is causing this error.

Complete Error:

Image detailing the error received

Any ideas for what I should investigate to get to the bottom of this?

3
  • 1
    What is the type of wavelength? You have it wrapped in a call to np.array(), so I assume it is not a numpy array already. Presumably it is some sort of container (python list? Pandas DataFrame or Series? Something else?). What does the data in wavelength look like? Also ask the same questions about transmitted. Commented Sep 12, 2016 at 16:05
  • wavelength is a python list of floats transmitted is a 3d numpy array (constructed from numpy.zeros and then filled in later)
    – jm22b
    Commented Sep 12, 2016 at 16:10
  • 1
    Have you tried casting you array type to float using .astype(float)? this solved my problem. more info here. Commented Jun 17, 2018 at 18:19

5 Answers 5

16

Just in case it could help someone else, I used numpy.array(wavelength,dtype='float64') to force the conversion of objects in the list to numpy's float64. Works well for me.

5

Typically these scipy functions require parameters like:

curvefit( function, initial_values, (aux_values,), ...)

where the tuple of aux_values is passed through to your function along with the current value of the main variable.

Is the dstack expression this aux_values? Or a concatenation of several. It may need to be wrapped in a tuple.

(np.dstack(transmitted[:,:,c][edgeIndex+30:])[0][0],)

We may need to know exactly where this error arises, not just which line of your code does it. We need to know what value is being converted. Where is there an array with dtype object?

5
  • the scipy documentation asks for the following; curve_fit(function, xdata, ydata, p0) where p0 are some initial guesses for the parameters passed to my function. the dstack expression is my y-value array. I will add more detailed information about the error to my question
    – jm22b
    Commented Sep 12, 2016 at 16:25
  • It's leastsq that operates as I guessed, and curvefit packs your first 3 arguments into the args tuple for that. It's not getting deep in to leastsq, so it must be checking the dtypes of the inputs. So what's the dtype of your arguments?
    – hpaulj
    Commented Sep 12, 2016 at 16:40
  • the data types for x, y, and the two items in p0 are float32
    – jm22b
    Commented Sep 12, 2016 at 17:10
  • 2
    An O dtype often results when constructing an array from a list of sublists that differ in size. If np.array(...) can't make a clean n-d array of numbers, it resorts to making an array of objects.
    – hpaulj
    Commented Sep 12, 2016 at 17:15
  • I have resolved the issue. I was passing an array with one element to the p0 list, rather than the element itself. Thank you for your help
    – jm22b
    Commented Sep 12, 2016 at 17:51
4

Just to clarify, I had the same problem, did not see the right answers in the comments, before solving it on my own. So I just repeat them here:

I have resolved the issue. I was passing an array with one element to the p0 list, rather than the element itself. Thank you for your help – Jacobadtr Sep 12 at 17:51

An O dtype often results when constructing an array from a list of sublists that differ in size. If np.array(...) can't make a clean n-d array of numbers, it resorts to making an array of objects. – hpaulj Sep 12 at 17:15

That is, make sure that the tuple of parameters you pass to curve_fit can be properly casted to an numpy array

0

I had the same error when attempting to run numpy.interp on data with string values. The error did not occur on data with numeric values.

So the error could be due to (some of) the data in your wavelength array being non numeric.

-1

From here, apparently numpy struggles with index type. The proposed solution is:

One thing you can do is use np.intp as dtype whenever things have to do with indexing or are logically related to indexing/array sizes. This is the natural dtype for it and it will normally also be the fastest one.

Does this help?

4
  • I need the arrays to have floating point data types. My error seems to think I am casting a python object to float64.
    – jm22b
    Commented Sep 12, 2016 at 14:59
  • what is the dtype of beg, end, and edgeIndex? Commented Sep 12, 2016 at 15:03
  • int, int and numpy.int64 That might very well be the problem - I shall test and come back
    – jm22b
    Commented Sep 12, 2016 at 15:12
  • Making sure they are all of type int did not resolve my issue
    – jm22b
    Commented Sep 12, 2016 at 16:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.