8

I am trying out to create a Random Forest regression model on one of my datasets. I need to find the order of importance of each variable along with their names as well. I have tried few things but can't achieve what I want. Below is the sample code I tried on Boston Housing dataset:

from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_boston
from sklearn.ensemble import RandomForestRegressor
import pandas as pd
import numpy as np
boston = load_boston()
rf=RandomForestRegressor(max_depth=50)
idx=range(len(boston.target))
np.random.shuffle(idx)
rf.fit(boston.data[:500], boston.target[:500])
instance=boston.data[[0,5, 10]]
print rf.predict(instance[0])
print rf.predict(instance[1])
print rf.predict(instance[2])
important_features=[]
for x,i in enumerate(rf.feature_importances_):
      important_features.append(str(x))
print 'Most important features:',', '.join(important_features)

Most important features: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12

If I print this:

impor = rf.feature_importances_
impor

I get below output:

array([  3.45665230e-02,   4.58687594e-04,   5.45376404e-03,
     3.33388828e-04,   2.90936201e-02,   4.15908448e-01,
     1.04131089e-02,   7.26451301e-02,   3.51628079e-03,
     1.20860975e-02,   1.40417760e-02,   8.97546838e-03,
     3.92507707e-01])

I need to get the names associated with these values and then pick the top n out of these features.

2
  • I don't think there would be any way that I know of. But I would suggest calculate entropy for each feature and highest entropy features would be the important ones. Commented Feb 9, 2017 at 4:59
  • @AnwarShaikh: You do get the amount of importance of each feature in some value using rf.feature_importances_ giving some values. We need to print the names along with its order.
    – CodeHunter
    Commented Feb 9, 2017 at 5:04

5 Answers 5

14

First, you are using wrong name for the variable. You are using important_features. Use feature_importances_ instead. Second, it will return an array of shape [n_features,] which contains the values of the feature_importance. You need to sort them in order of those values to get the most important features. See the RandomForestRegressor documentation

Edit: Added code

important_features_dict = {}
for idx, val in enumerate(rf.feature_importances_):
    important_features_dict[idx] = val

important_features_list = sorted(important_features_dict,
                                 key=important_features_dict.get,
                                 reverse=True)

print(f'5 most important features: {important_features_list[:5]}')

This will print the index of important features in decreasing order. (First is most important, and so on)

6
  • Thanks for pointing that out. I modified the code. Added my edits here. But my problem is still the same.
    – CodeHunter
    Commented Feb 9, 2017 at 5:25
  • @ashishkumar I have added code to achieve what you wanted Commented Feb 9, 2017 at 5:26
  • @ashishkumar Sorry, typo in dict name in for loop. Corrected. Try it now Commented Feb 9, 2017 at 5:36
  • Thanks Vivek! It helped!
    – CodeHunter
    Commented Feb 9, 2017 at 5:44
  • 1
    @Ike Thanks, I have updated the answer. Commented Nov 18, 2021 at 4:57
3
importances = rf.feature_importances_

sorted_indices = np.argsort(importances)[::-1]

sorted_indices
2
  • 4
    Welcome to SO! While this code may solve the question, including an explanation of how and why this solves the problem would really help to improve the quality of your post. Remember that you are answering the question for readers in the future, not just the person asking now. Please edit your answer to add an explanation.
    – Nick
    Commented Aug 29, 2020 at 2:38
  • 2
    Please edit this answer to add the information provided by your other answer to this question and then delete the other answer. And still, I doubt that this helps or even works at all. To convince me otherwise please explain this code.
    – Yunnosch
    Commented Aug 29, 2020 at 7:37
2

You can print the order like this:


importances = brf.feature_importances_

sorted_indices = np.argsort(importances)[::-1]

print(*X_train.columns[sorted_indices], sep = "\n")
3
  • I doubt that this helps or even works at all. To convince me otherwise please explain this code.
    – Yunnosch
    Commented Aug 29, 2020 at 7:35
  • 1
    Also, please do not create multipe answers to the same question without explaining how they are fundamentally different approaches. One answer with different angels is preferred.
    – Yunnosch
    Commented Aug 29, 2020 at 7:37
  • Hi Abuubakry. Can I ask you a questiön concerning your posts? Like, which one is the correct one? Or do both work? What is the difference? Your posts and silence together are currently giving the impression that you do not understand the concepts of this community, cannot appropriatly use the available tools and are unable to communicate in the language chosen for this community. Please answer and let me help you.
    – Yunnosch
    Commented Aug 30, 2020 at 10:03
0

By the following code, you should be able to see the features in descending order with their names as well:

Create an empty list

featureImpList= []

Run the for loop:

for feat, importance in zip(train_df.columns, clf_ggr.feature_importances_):  
    temp = [feat, importance*100]
    featureImp.append(temp)

fT_df = pd.DataFrame(featureImp, columns = ['Feature', 'Importance'])
print (fT_df.sort_values('Importance', ascending = False))
-1
# list of column names from original data
cols = data.columns
# feature importances from random forest fit rf
rank = rf.feature_importances_
# form dictionary of feature ranks and features
features_dict = dict(zip(np.argsort(rank),cols))
# the dictionary key are the importance rank; the values are the feature name
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.