31

I applied clustering on a set of text documents (about 100). I converted them to Tfidf vectors using TfIdfVectorizer and supplied the vectors as input to scikitlearn.cluster.KMeans(n_clusters=2, init='k-means++', max_iter=100, n_init=10). Now when I

model.fit()
print model.score()

on my vectors, I get a very small value if all the text documents are very similar, and I get a very large negative value if the documents are very different.

It serves my basic purpose of finding which set of documents are similar, but can someone help me understand what exactly does this model.score() value signify for a fit? How can I use this value to justify my findings?

0

3 Answers 3

16

The word chosen by the documentation is a bit confusing. It says "Opposite of the value of X on the K-means objective." It means negative of the K-means objective.

K-Means Objective

The objective in the K-means is to reduce the sum of squares of the distances of points from their respective cluster centroids. It has other names like J-Squared error function, J-score or within-cluster sum of squares. This value tells how internally coherent the clusters are. (The less the better)

The objective function can be directly obtained from the following method.

model.inertia_

14

In the documentation it says:

Returns:    
score : float
Opposite of the value of X on the K-means objective.

To understand what that means you need to have a look at the k-means algorithm. What k-means essentially does is find cluster centers that minimize the sum of distances between data samples and their associated cluster centers.

It is a two-step process, where (a) each data sample is associated to its closest cluster center, (b) cluster centers are adjusted to lie at the center of all samples associated to them. These steps are repeated until a criterion (max iterations / min change between last two iterations) is met.

As you can see there remains a distance between the data samples and their associated cluster centers, and the objective of our minimization is that distance (sum of all distances).

You naturally get large distances if you have a big variety in data samples, if the number of data samples is significantly higher than the number of clusters, which in your case is only two. On the contrary, if all data samples were the same, you would always get a zero distance regardless of number of clusters.

From the documentation I would expect that all values are negative, though. If you observe both negative and positive values, maybe there is more to the score than that.

I wonder how you got the idea of clustering into two clusters though.

2
  • thanks. that helps... I've been experimenting with the number of clusters. Like you mentioned, if the documents are similar, the distance would always be zero (or very close to it). it's just that i need to know what exactly (in terms of cluster evaluation) does the score() function return. In classification, for example, the score() returns the accuracy. Sep 3, 2015 at 9:15
  • To know what the score exactly returns, you probably have to look at the software itself. But the objective of K-means is defined here: en.wikipedia.org/wiki/K-means_clustering#Description As you can see it is very simple. For me it seems like you obtain a measure that is somewhat helpful for you but only an approximation of what you want to measure with some side effects.
    – ypnos
    Sep 3, 2015 at 11:06
7

ypnos is right, you can find some detail here: https://github.com/scikit-learn/scikit-learn/blob/51a765a/sklearn/cluster/k_means_.py#L893

inertia : float
    Sum of distances of samples to their closest cluster center.
"""

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.