32

How to subset data in R without losing NA rows?

The post above subsets using logical indexing. Is there a way to do it in dplyr?

Also, when does dplyr automatically delete NAs? In my experience, it removes NA when I filter out a specific string, eg:

b = a %>% filter(col != "str")

I would think this would not exclude NA values but it does. But when I use other format of filtering, it does not automatically exclude NA, eg:

b = a %>% filter(!grepl("str", col))

I would like to understand this feature of filter. I would appreciate any help. Thank you!

1

3 Answers 3

42

The documentation for dplyr::filter says... "Unlike base subsetting, rows where the condition evaluates to NA are dropped."

NA != "str" evaluates to NA so is dropped by filter.

!grepl("str", NA) returns TRUE, so is kept.

If you want filter to keep NA, you could do filter(is.na(col)|col!="str")

1
  • 2
    This solution won't do since if col != "str" returns FALSE but is.na(col) returns TRUE, it will be kept. So the filter fails. I'm actually using both filter examples in one filter. So it goes b = a %>% filter(col != "str1", !grepl("str2", col)). This is what I do but it also filters out NA...that's the problem Sep 25, 2017 at 7:58
18

If you want to keep NAs created by the filter condition you can simply turn the condition NAs into TRUEs using replace_na from tidyr.

a <- data.frame(col = c("hello", NA, "str"))
a %>% filter((col != "str") %>% replace_na(TRUE))
1
  • This is the only solution that actually worked for me! Very elegant too! Thank you!! Feb 11, 2022 at 16:30
3

I just ran into this issue. It is very easy to miss, and I must say I find this behavior somewhat unintuitive. Based on qwr's answer, this is becoming a staple in all my projects from now on:

filter_na <- function(tbl, expr){
   tbl %>% filter({{expr}} %>% replace_na(T))
}
1
  • The behavior matches SQL WHERE in that it drops UNKNOWN in SQL's 3VL.
    – qwr
    Aug 23, 2023 at 2:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.