19
from sklearn.feature_extraction.text import TfidfVectorizer
import numpy as np
from sklearn import linear_model
arr=['dogs cats lions','apple pineapple orange','water fire earth air', 'sodium potassium calcium']
vectorizer = TfidfVectorizer()
X = vectorizer.fit_transform(arr)
feature_names = vectorizer.get_feature_names()
Y = ['animals', 'fruits', 'elements','chemicals']
T=["eating apple roasted in fire and enjoying fresh air"]
test = vectorizer.transform(T)
clf = linear_model.SGDClassifier(loss='log')
clf.fit(X,Y)
x=clf.predict(test)
#prints: elements

In the above code, clf.predict() prints only 1 best prediction for a sample from list X. I am interested in top 3 predictions for a particular sample in the list X, i know the function predict_proba/predict_log_proba returns a list of all probabilities for each feature in list Y, but it has to sorted and then associated with the features in list Y before getting the top 3 results. Is there any direct and efficient way?

1
  • @user1269942: Very helpful addition there! However, I did not totally understand the function of the "truths" variable. Can you elaborate please?
    – Statmonger
    Feb 24, 2019 at 12:34

6 Answers 6

21

There is no built-in function, but what is wrong with

probs = clf.predict_proba(test)
best_n = np.argsort(probs, axis=1)[-n:]

?

As suggested by one of the comment, should change [-n:] to [:,-n:]

probs = clf.predict_proba(test)
best_n = np.argsort(probs, axis=1)[:,-n:]
7
  • 1
    No names associated with the corresponding probability! Sep 11, 2015 at 18:44
  • 3
    clf.classes_ provides these. Sep 13, 2015 at 15:31
  • 9
    Shouldn't the slicing be best_n = np.argsort(probs, axis=1)[:, -n:] ?
    – MCMZL
    Aug 3, 2018 at 8:09
  • 1
    Hey @AndreasMueller, exactly how would you map the clf.classes_ to the probs? I am new to Python and have spent the last 90 minutes trying to figure out how to map the indices of the classes to the actual classes...thanks! Mar 30, 2019 at 8:16
  • one more question: the slicing with best_n provides 3rd best, 2nd best, then 1st best. how could you reverse the order so it's 1st, 2nd, then 3rd? thanks again! Mar 30, 2019 at 8:30
11

I know this has been answered...but I can add a bit more...

#both preds and truths are same shape m by n (m is number of predictions and n is number of classes)
def top_n_accuracy(preds, truths, n):
    best_n = np.argsort(preds, axis=1)[:,-n:]
    ts = np.argmax(truths, axis=1)
    successes = 0
    for i in range(ts.shape[0]):
      if ts[i] in best_n[i,:]:
        successes += 1
    return float(successes)/ts.shape[0]

It's quick and dirty but I find it useful. One can add their own error checking, etc..

5

Hopefully, Andreas will help with this. predict_probs is not available when loss='hinge'. To get top n class when loss='hinge' do:

calibrated_clf = CalibratedClassifierCV(clfSDG, cv=3, method='sigmoid')
model = calibrated_clf.fit(train.data, train.label)

probs = model.predict_proba(test_data)
sorted( zip( calibrated_clf.classes_, probs[0] ), key=lambda x:x[1] )[-n:]

Not sure if clfSDG.predict and calibrated_clf.predict will always predict the same class.

1
  • Given that log-loss for SGDClassifier is OvR, you could also just rank by decision_function and it wouldn't really be worse in any way. Using CalibratedClassifierCV is probably better, but orthogonal to the question. I would use LogisticRegression(multiclass='multinomial'). Aug 9, 2017 at 20:07
2

argsort gives results in ascending order, if you want to save yourself with unusual loops or confusion you can use a simple trick.

probs = clf.predict_proba(test)
best_n = np.argsort(-probs, axis=1)[:, :n]

Negating the probabilities will turn smallest to largest and hence you can take top-n results in descending order.

0

As @FredFoo described in How do I get indices of N maximum values in a NumPy array? a faster method would be to use argpartition.

Newer NumPy versions (1.8 and up) have a function called argpartition for this. To get the indices of the four largest elements, do

>>> a = np.array([9, 4, 4, 3, 3, 9, 0, 4, 6, 0])
>>> a array([9, 4, 4, 3, 3, 9, 0, 4, 6, 0])
>>> ind = np.argpartition(a, -4)[-4:]
>>> ind array([1, 5, 8, 0])
>>> a[ind] array([4, 9, 6, 9])

Unlike argsort, this function runs in linear time in the worst case, but the returned indices are not sorted, as can be seen from the result of evaluating a[ind]. If you need that too, sort them afterwards:

>>> ind[np.argsort(a[ind])] array([1, 8, 5, 0]) 

To get the top-k elements in sorted order in this way takes O(n + k log k) time.

0

I wrote a function that outputs a dataframe with the top n predictions and their probabilities, and ties it back to class names. Hope this is helpful!

def return_top_n_pred_prob_df(n, model, X_test, column_name):
  predictions = model.predict_proba(X_test)
  preds_idx = np.argsort(-predictions) 
  classes = pd.DataFrame(model.classes_, columns=['class_name'])
  classes.reset_index(inplace=True)
  top_n_preds = pd.DataFrame()
  for i in range(n):
        top_n_preds[column_name + '_prediction_{}_num'.format(i)] =     [preds_idx[doc][i] for doc in range(len(X_test))]
    top_n_preds[column_name + '_prediction_{}_probability'.format(i)] = [predictions[doc][preds_idx[doc][i]] for doc in range(len(X_test))]
    top_n_preds = top_n_preds.merge(classes, how='left', left_on= column_name + '_prediction_{}_num'.format(i), right_on='index')
    top_n_preds = top_n_preds.rename(columns={'class_name': column_name + '_prediction_{}'.format(i)})
    try: top_n_preds.drop(columns=['index', column_name + '_prediction_{}_num'.format(i)], inplace=True) 
    except: pass
  return top_n_preds

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.