88

Not quite sure why I can't figure this out. I'm looking to slice a Pandas dataframe by using index numbers. I have a list/core index with the index numbers that i do NOT need, shown below

 pandas.core.index.Int64Index

 Int64Index([2340, 4840, 3163, 1597, 491 , 5010, 911 , 3085, 5486, 5475, 1417, 2663, 4204, 156 , 5058, 1990, 3200, 1218, 3280, 793 , 824 , 3625, 1726, 1971, 2845, 4668, 2973, 3039, 376 , 4394, 3749, 1610, 3892, 2527, 324 , 5245, 696 , 1239, 4601, 3219, 5138, 4832, 4762, 1256, 4437, 2475, 3732, 4063, 1193], dtype=int64)

How can I create a new dataframe excluding these index numbers. I tried

df.iloc[combined_index]

and obviously this just shows the rows with those index number (the opposite of what I want). any help will be greatly appreciated

1
  • 1
    Do you want to remove the rows with those indices, or the rows with those positions? iloc is used when you care more about the locations (you want to refer to the 5th row, say, regardless of index).
    – DSM
    Jan 31, 2015 at 21:54

5 Answers 5

114

Not sure if that's what you are looking for, posting this as an answer, because it's too long for a comment:

In [31]: d = {'a':[1,2,3,4,5,6], 'b':[1,2,3,4,5,6]}

In [32]: df = pd.DataFrame(d)

In [33]: bad_df = df.index.isin([3,5])

In [34]: df[~bad_df]
Out[34]: 
   a  b
0  1  1
1  2  2
2  3  3
4  5  5
2
  • 2
    Does this work for numpy? I suspect not because ~[2,1] is [-3,-2], for example. So if X is a np array, X[ix] and X[~ix] are not 'complementary' as with pandas. Jan 26, 2017 at 22:08
  • This works only if the DataFrame index is sequential, otherwise it won't work
    – kennyFF92
    Oct 7, 2020 at 10:05
60

Just use .drop and pass it the index list to exclude.

import pandas as pd

df = pd.DataFrame({"a": [10, 11, 12, 13, 14, 15]})


df.drop([1, 2, 3], axis=0)

Which outputs this.

    a
0  10
4  14
5  15
2
  • This can run into issues for dataframes with non-integer indices, or dataframes with integer indices that skip certain numbers. Aug 25, 2020 at 23:20
  • True, you must pass the index that you want to drop. If you need to use a range then first reset the index.
    – Chris Farr
    Aug 26, 2020 at 2:28
19

Probably an easier way is just to use a boolean index, and slice normally doing something like this:

df[~df.index.isin(list_to_exclude)]
0
9

You could use pd.Int64Index(np.arange(len(df))).difference(index) to form a new ordinal index. For example, if we want to remove the rows associated with ordinal index [1,3,5], then

import numpy as np
import pandas as pd

index = pd.Int64Index([1,3,5], dtype=np.int64)
df = pd.DataFrame(np.arange(6*2).reshape((6,2)), index=list('ABCDEF'))
#     0   1
# A   0   1
# B   2   3
# C   4   5
# D   6   7
# E   8   9
# F  10  11

new_index = pd.Int64Index(np.arange(len(df))).difference(index)
print(df.iloc[new_index])

yields

   0  1
A  0  1
C  4  5
E  8  9
3

Assuming there exists a DataFrame df:

In [4]: df = pd.DataFrame({'a': range(4), 'b': ['a', 'b', 'c', 'd']})

In [5]: df
Out[5]: 
   a  b
0  0  a
1  1  b
2  2  c
3  3  d

and you want to remove index [1, 3], you can use query:

In [5]: df.query('index != [1,3]')
Out[5]: 
   a  b
0  0  a
2  2  c

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.