34

I have a data.frame:

SelectVar
     a   b  c   d   e   f   g   h   i j k l ll m n o p  q   r
1 Dxa8 Dxa8 0 Dxa8 Dxa8 0 Dxa8 Dxa8 0 0 0 0  0 0 0 0 0 Dxc8 0
2 Dxb8 Dxc8 0 Dxe8 Dxi8 0 tneg tpos 0 0 0 0  0 0 0 0 0 Dxi8 0

I would like to remove the columns with zero values in both rows from the data frame, so it yields a data frame as below:

SelectVar
     a   b    d    e    g   h     q   
1 Dxa8 Dxa8 Dxa8 Dxa8 Dxa8 Dxa8  Dxc8 
2 Dxb8 Dxc8 Dxe8 Dxi8 tneg tpos  Dxi8 

Have tried:

SelectVar!=0

which yields a True/False dataframe, and:

SelectVar[, colSums(abs(SelectVar)) ! == 0]

which yields an error.

How could I remove the columns with zero values in each row?

10 Answers 10

61

You almost have it. Put those two together:

 SelectVar[, colSums(SelectVar != 0) > 0]

This works because the factor columns are evaluated as numerics that are >= 1.

12
  • 4
    No, it does not work with NA values. If NA value are present, replace the test with !is.na(colSums(SelectVar != 0)) & colSums(SelectVar != 0) > 0 (or equivalent). Feb 3, 2014 at 15:12
  • 5
    With NA just try: SelectVar[, colSums(SelectVar != 0, na.rm = TRUE) > 0]
    – mpalanco
    Jul 14, 2015 at 9:50
  • @MatthewLundberg - Will this SelectVar[, colSums(SelectVar == 0) == 0] remove all columns with sum as zero? Aug 7, 2017 at 21:13
  • 1
    @M.Mashaye That will return identical results in all cases. abs(x) == 0 is equivalent to x == 0. Jan 23, 2018 at 20:24
  • 1
    @TheGoat That's the name of the data.frame in the question. Jun 19, 2018 at 14:14
11

A dplyr friendly solution:

SelectVar %>% select_if(colSums(.) != 0)

2
  • but what if there are negative values and negatives and positivea add up to 0? you would be removing those columns with this method, right? Oct 31, 2020 at 2:57
  • Good catch @charlesdarwin! Most likely, yes. Though I've recently found another dplyr/tidyverse friendly solution purrr::discard(~all(is.na(.)))
    – seapen
    Dec 14, 2020 at 21:30
7

One option since dplyr 1.0.0 could be:

df %>%
 select(where(~ any(. != 0)))

     a    b    d    e    g    h    q
1 Dxa8 Dxa8 Dxa8 Dxa8 Dxa8 Dxa8 Dxc8
2 Dxb8 Dxc8 Dxe8 Dxi8 tneg tpos Dxi8
5

Try also

SelectVar[, !apply(SelectVar == 0, 2, all)]

This was taken from here:

Delete all columns with 0 from matrix

2
  • This is likely to be slower though, particularly on larger datasets. Also, I think the OP is looking for any not all. Feb 3, 2014 at 15:40
  • Maybe, i just copy paste the post. It seems to work ok.
    – Fernando
    Feb 3, 2014 at 15:45
5

The faster option, by about 40% according to mean execution times, is

df[,-(which(colSums(df)==0))]

We can benchmark the two options with a simple example data frame consisting of 3,000 columns and two observations.

# Create simple 2 X 3000 data frame with many 1s and 0s
# 500 columns have all 0s
df = matrix(c(rep(c(0,1,1),1000),rep(c(1,0,0),1000)),nrow=2)
df = as.data.frame(df)

# Benchmark the two options in milliseconds, 100 times
library(microbenchmark)
microbenchmark(
  df[,colSums(df != 0) > 0],
  df[,-(which(colSums(df)==0))]
  )

Unit: milliseconds
                             expr     min       lq     mean   median       uq      max neval
       df[, colSums(df != 0) > 0] 23.3844 24.77905 30.24852 26.37730 29.17175 140.6486   100
 df[, -(which(colSums(df) == 0))] 17.3664 19.12815 21.58901 20.59055 22.29905  41.9485   100

1

simple answer that remove columns with any zeros:

columns_to_keep = (colSums(SelectVar != 0) == nrow(SelectVar))
NewTable = SelectVar[, columns_to_keep]
3
  • Do you mean SelectVar[, columns_to_keep]? And to be technically correct, you might use colSums(SelectVar != 0), using != instead of > in case there are negative numbers. Jan 4, 2022 at 17:08
  • I corrected the answer, thank you! Jan 5, 2022 at 18:16
  • I think it's done now, sorry I'm a new user. Jan 19, 2022 at 19:58
0

To remove any and all columns that contain only zeros, simply pass your data frame into the following function:

remove_zero_cols <- function(df) {
  rem_vec <- NULL
  for(i in 1:ncol(df)){
    this_sum <- summary(df[,i])
    zero_test <- length(which(this_sum == 0))
    if(zero_test == 6) {
      rem_vec[i] <- names(df)[i]
    }
  }
  features_to_remove <- rem_vec[!is.na(rem_vec)]
  rem_ind <- which(names(df) %in% features_to_remove)
  df <- df[,-rem_ind]
  return(df)
}

Example:

iris$Sepal.Width <- 0
new_df <- remove_zero_cols(iris)
print(new_df)
2
  • What is 6 in if(zero_test == 6) Jun 22, 2018 at 17:12
  • The summary function on line 4 gives a 6 number summary of a column. If all 6 values are 0 then the column contains all zeros, and is thus removed.
    – Cybernetic
    Jun 22, 2018 at 18:31
0

you can try something like

   [row, column] = SelectVar.shape
    for j in range(column):
       if np.all(SelectVar.iloc[:, j] == 0):
           SelectVar = SelectVar.drop(SelectVar.columns[j], axis=1, inplace=True)
0

Late answer, but one other base R approach which can work here would be to assert that either the minimum or maximum value in each column is not zero:

colMax <- sapply(SelectVar, max, na.rm=TRUE)
colMin <- sapply(SelectVar, min, na.rm=TRUE)
SelectVar[, colMin != 0 | colMax != 0]
0

All above answers are valid -
but for some of us, perhaps the most intuitive solution to read following the goal remove columns with zero values is:

df %>% select(where(~ sum(.) != 0))

You can read it as:
For every column (~), remove it (!=) if its sum (sum(.)) is zero

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.