23

I have this large dataframe I've imported into pandas and I want to chop it down via a filter. Here is my basic sample code:

import pandas as pd
import numpy as np
from pandas import Series, DataFrame

df = DataFrame({'A':[12345,0,3005,0,0,16455,16454,10694,3005],'B':[0,0,0,1,2,4,3,5,6]})

df2= df[df["A"].map(lambda x: x > 0) & (df["B"] > 0)]

Basically this displays bottom 4 results which is semi-correct. But I need to display everything BUT these results. So essentially, I'm looking for a way to use this filter but in a "not" version if that's possible. So if column A is greater than 0 AND column B is greater than 0 then we want to disqualify these values from the dataframe. Thanks

5
  • 5
    Read the documentation on boolean masking here : pandas.pydata.org/pandas-docs/stable/…. You can use ~ as "not" df2= df[~df["A"].map(lambda x: x > 0) & (df["B"] > 0)]
    – Thtu
    May 27, 2016 at 23:38
  • this is a step in the right direction but it leaves me only with 2 records instead of 5
    – staten12
    May 27, 2016 at 23:42
  • 2
    Sorry, the snippet I posted needs to be applied to the entire mask, not just the first one. df[~(df["A"].map(lambda x: x > 0) & (df["B"] > 0))]
    – Thtu
    May 27, 2016 at 23:43
  • Why can't you then do the reverse? if column A is less than 0 or column b is less then 0 May 28, 2016 at 0:24
  • @Thomas Tu that works thank you!!
    – staten12
    May 28, 2016 at 1:06

4 Answers 4

43

No need for map function call on Series "A".

Apply De Morgan's Law:

"not (A and B)" is the same as "(not A) or (not B)"

df2 = df[~(df.A > 0) | ~(df.B > 0)]
1

There is no need for the map implementation. You can just reverse the arguments like ...

df.ix[(df.A<=0)|(df.B<=0),:]

Or use boolean indexing without ix:

df[(df.A<=0)|(df.B<=0)]
0
1

you could use: wrap everything in a bracket and use a ~ (tilde) outside. in place of not.

df[~((df['A'] >0) & (df['B']>0))]

answer:

    A       B
0   12345   0
1   0       0
2   3005    0
3   0       1
4   0       2
0

Try

df2 = df[df["A"].map(lambda x: x <= 0) | (df["B"] <= 0)]

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.