18

I'm trying out multidimensional scaling with sklearn, pandas and numpy. The data file Im using has 10 numerical columns and no missing values. I am trying to take this ten dimensional data and visualize it in 2 dimensions with sklearn.manifold's multidimensional scaling as follows:

import numpy as np
import pandas as pd
from sklearn import manifold
from sklearn.metrics import euclidean_distances

seed = np.random.RandomState(seed=3)
data = pd.read_csv('data/big-file.csv')

#  start small dont take all the data, 
#  its about 200k records
subset = data[:10000]
similarities = euclidean_distances(subset)

mds = manifold.MDS(n_components=2, max_iter=3000, eps=1e-9, 
      random_state=seed, dissimilarity="precomputed", n_jobs=1)

pos = mds.fit(similarities).embedding_

But I get this value error:

Traceback (most recent call last):
  File "demo/mds-demo.py", line 18, in <module>
    pos = mds.fit(similarities).embedding_
  File "/Users/dwilliams/Desktop/Anaconda/lib/python2.7/site-packages/sklearn/manifold/mds.py", line 360, in fit
    self.fit_transform(X, init=init)
  File "/Users/dwilliams/Desktop/Anaconda/lib/python2.7/site-packages/sklearn/manifold/mds.py", line 395, in fit_transform
eps=self.eps, random_state=self.random_state)
  File "/Users/dwilliams/Desktop/Anaconda/lib/python2.7/site-packages/sklearn/manifold/mds.py", line 242, in smacof
eps=eps, random_state=random_state)
  File "/Users/dwilliams/Desktop/Anaconda/lib/python2.7/site-packages/sklearn/manifold/mds.py", line 73, in _smacof_single
raise ValueError("similarities must be symmetric")
ValueError: similarities must be symmetric

I thought euclidean_distances returned a symmetric matrix. What am I doing wrong and how do I fix it?

6
  • 2
    first check that np.allclose(similarities, similarites.T) is True. when I try this with random input it works. can u try with random input? Jun 7, 2013 at 21:48
  • Try using scipy.spatial.distance_matrix? Or if you're just using euclidean distance anyways, you can let sklearn compute them using dissimilarity="euclidean". Aug 11, 2013 at 21:40
  • I ran into a similar issue and I had to patch L71 in sklearn/manifold/mds.py by multiplying the tolerance by 20 (np.abs(similarities - similarities.T).max()was ~1e-12 for me instead of < 1e-13, so the check was too stringent and it failed.
    – jorgeca
    Oct 25, 2013 at 15:50
  • Would be great if down raters could provide a comment. It's not constructive in any way to down vote and bounce without input. Wasted time. Nov 14, 2013 at 19:40
  • You need to answer Phillip Cloud's question, as well as the other questions posted in the comments. You also need to provide test data to replicate the problem.
    – cxrodgers
    Nov 17, 2013 at 20:57

2 Answers 2

13

I ran across the same problem; it turned out that my data was an array of np.float32 and the reduced float precision caused the distance matrix to be asymmetric. I fixed the issue by converting my data to np.float64 before running MDS on it.

Here's an example that uses random data to illustrate the issue:

import numpy as np
from sklearn.manifold import MDS
from sklearn.metrics import euclidean_distances
from sklearn.datasets import make_classification

data, labels = make_classification()
mds = MDS(n_components=2)

similarities = euclidean_distances(data.astype(np.float64))
print np.abs(similarities - similarities.T).max()
# Prints 1.7763568394e-15
mds.fit(data.astype(np.float64))
# Succeeds

similarities = euclidean_distances(data.astype(np.float32))
print np.abs(similarities - similarities.T).max()
# Prints 9.53674e-07
mds.fit(data.astype(np.float32))
# Fails with "ValueError: similarities must be symmetric"
1
  • Thx,It really works. but after changed to float64, I got another warning: /Library/Python/2.7/site-packages/sklearn/manifold/mds.py:396: UserWarning: The MDS API has changed. fit now constructs an dissimilarity matrix from data. To use a custom dissimilarity matrix, set dissimilarity='precomputed'. Nov 12, 2015 at 4:16
6

Had the same problem a while ago. Another solution, which I believe much more efficient, is to compute the distance only for the upper triangular matrix, and later copy to the lower part.

It can be done with scipy as follows:

from scipy.spatial.distance import squareform,pdist                                                              
similarities = squareform(pdist(data,'speuclidean'))

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.