I'm importing some data from a csv file. The file has nan values flagged with text 'NA'. I import the data with:
X = genfromtxt(data, delimiter=',', dtype=float, skip_header=1)
I the use this code to replace nan with a previosly calculated column mean.
inds = np.where(np.isnan(X))
X[inds]=np.take(col_mean,inds[1])
I then run a couple of checks and get empty arrays:
np.where(np.isnan(X))
np.where(np.isinf(X))
Finally I run a scikit classifier:
RF = ensemble.RandomForestClassifier(n_estimators=100,n_jobs=-1,verbose=2)
RF.fit(X, y)
and get the following error:
File "C:\Users\m&g\Anaconda\lib\site-packages\sklearn\ensemble\forest.py", line 257, in fit
check_ccontiguous=True)
File "C:\Users\m&g\Anaconda\lib\site-packages\sklearn\utils\validation.py", line 233, in check_arrays
_assert_all_finite(array)
File "C:\Users\m&g\Anaconda\lib\site-packages\sklearn\utils\validation.py", line 27, in _assert_all_finite
raise ValueError("Array contains NaN or infinity.")
ValueError: Array contains NaN or infinity.
Any ideas why it is telling me that there are NaN or infinity? I read this post and tried to run:
RF.fit(X.astype(float), y.astype(float))
but I get the same error.
np.max(np.abs(X))return? – larsmans Jan 23 at 22:24