19

I have a dataframe that looks like shown below

                               mean
comp_name  date                      
Appdynamics 2012-05-01 00:18:15.910000
            2012-05-01             NaT
            2012-05-01             NaT
            2012-05-02 00:20:12.145200
            2012-05-02             NaT
            2012-05-02             NaT

Here the comp_name and date form multiindex. I want to get rid of the NaT values and obtain only those rows where the mean(timedelta64) is not NaT.

                               mean
comp_name  date                      
Appdynamics 2012-05-01 00:18:15.910000
            2012-05-02 00:20:12.145200

Any ideas on this?

2
  • 3
    does dropna() not work for this?
    – EdChum
    Jul 3, 2014 at 12:08
  • 1
    Unfortunately not.
    – sparrow
    Jul 27, 2021 at 16:15

2 Answers 2

25

pandas.notnull() takes a series and returns a Boolean series which is True where the input series is not null (None, np.NaN, np.NaT). Then you can slice a dataframe by the Boolean series:

df[pandas.notnull(df['mean'])]
3
  • How can you check for 2 columns at once, eg, df['mean', 'score']? Jun 20, 2015 at 4:08
  • 1
    Depending on what you want: df['mean','score'].isnull().any(axis=1) or df['mean','score'].isnull().all(axis=1)
    – exp1orer
    Jun 22, 2015 at 20:39
  • In recent versions, it's better to do the slicing in this way: df.loc[df.mean.notnull()]
    – Elias
    Jan 28, 2021 at 7:58
2

In Pandas 1.4.1 dropna gets rid of NaT values. Source: documentation and I am using it. So now it is as simple as

df = df.dropna()

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.