38

I have a data.frame:

SelectVar
     a   b  c   d   e   f   g   h   i j k l ll m n o p  q   r
1 Dxa8 Dxa8 0 Dxa8 Dxa8 0 Dxa8 Dxa8 0 0 0 0  0 0 0 0 0 Dxc8 0
2 Dxb8 Dxc8 0 Dxe8 Dxi8 0 tneg tpos 0 0 0 0  0 0 0 0 0 Dxi8 0

I would like to remove the columns with zero values in both rows from the data frame, so it yields a data frame as below:

SelectVar
     a   b    d    e    g   h     q   
1 Dxa8 Dxa8 Dxa8 Dxa8 Dxa8 Dxa8  Dxc8 
2 Dxb8 Dxc8 Dxe8 Dxi8 tneg tpos  Dxi8 

Have tried:

SelectVar!=0

which yields a True/False dataframe, and:

SelectVar[, colSums(abs(SelectVar)) ! == 0]

which yields an error.

How could I remove the columns with zero values in each row?

11 Answers 11

68

You almost have it. Put those two together:

 SelectVar[, colSums(SelectVar != 0) > 0]

This works because the factor columns are evaluated as numerics that are >= 1.

12
  • 4
    No, it does not work with NA values. If NA value are present, replace the test with !is.na(colSums(SelectVar != 0)) & colSums(SelectVar != 0) > 0 (or equivalent). Commented Feb 3, 2014 at 15:12
  • 6
    With NA just try: SelectVar[, colSums(SelectVar != 0, na.rm = TRUE) > 0]
    – mpalanco
    Commented Jul 14, 2015 at 9:50
  • @MatthewLundberg - Will this SelectVar[, colSums(SelectVar == 0) == 0] remove all columns with sum as zero? Commented Aug 7, 2017 at 21:13
  • 1
    @M.Mashaye That will return identical results in all cases. abs(x) == 0 is equivalent to x == 0. Commented Jan 23, 2018 at 20:24
  • 1
    @TheGoat That's the name of the data.frame in the question. Commented Jun 19, 2018 at 14:14
12

A dplyr friendly solution:

SelectVar %>% select_if(colSums(.) != 0)

2
  • but what if there are negative values and negatives and positivea add up to 0? you would be removing those columns with this method, right? Commented Oct 31, 2020 at 2:57
  • Good catch @charlesdarwin! Most likely, yes. Though I've recently found another dplyr/tidyverse friendly solution purrr::discard(~all(is.na(.)))
    – seapen
    Commented Dec 14, 2020 at 21:30
10

One option since dplyr 1.0.0 could be:

df %>%
 select(where(~ any(. != 0)))

     a    b    d    e    g    h    q
1 Dxa8 Dxa8 Dxa8 Dxa8 Dxa8 Dxa8 Dxc8
2 Dxb8 Dxc8 Dxe8 Dxi8 tneg tpos Dxi8
6

The faster option, by about 40% according to mean execution times, is

df[,-(which(colSums(df)==0))]

We can benchmark the two options with a simple example data frame consisting of 3,000 columns and two observations.

# Create simple 2 X 3000 data frame with many 1s and 0s
# 500 columns have all 0s
df = matrix(c(rep(c(0,1,1),1000),rep(c(1,0,0),1000)),nrow=2)
df = as.data.frame(df)

# Benchmark the two options in milliseconds, 100 times
library(microbenchmark)
microbenchmark(
  df[,colSums(df != 0) > 0],
  df[,-(which(colSums(df)==0))]
  )

Unit: milliseconds
                             expr     min       lq     mean   median       uq      max neval
       df[, colSums(df != 0) > 0] 23.3844 24.77905 30.24852 26.37730 29.17175 140.6486   100
 df[, -(which(colSums(df) == 0))] 17.3664 19.12815 21.58901 20.59055 22.29905  41.9485   100

5

Try also

SelectVar[, !apply(SelectVar == 0, 2, all)]

This was taken from here:

Delete all columns with 0 from matrix

2
  • This is likely to be slower though, particularly on larger datasets. Also, I think the OP is looking for any not all. Commented Feb 3, 2014 at 15:40
  • Maybe, i just copy paste the post. It seems to work ok.
    – Fernando
    Commented Feb 3, 2014 at 15:45
1

All above answers are valid -
but for some of us, perhaps the most intuitive solution to read following the goal remove columns with zero values is:

df %>% select(where(~ sum(.) != 0))

You can read it as:
For every column (~), remove it (!=) if its sum (sum(.)) is zero

1

simple answer that remove columns with any zeros:

columns_to_keep = (colSums(SelectVar != 0) == nrow(SelectVar))
NewTable = SelectVar[, columns_to_keep]
3
  • Do you mean SelectVar[, columns_to_keep]? And to be technically correct, you might use colSums(SelectVar != 0), using != instead of > in case there are negative numbers. Commented Jan 4, 2022 at 17:08
  • I corrected the answer, thank you! Commented Jan 5, 2022 at 18:16
  • I think it's done now, sorry I'm a new user. Commented Jan 19, 2022 at 19:58
0

To remove any and all columns that contain only zeros, simply pass your data frame into the following function:

remove_zero_cols <- function(df) {
  rem_vec <- NULL
  for(i in 1:ncol(df)){
    this_sum <- summary(df[,i])
    zero_test <- length(which(this_sum == 0))
    if(zero_test == 6) {
      rem_vec[i] <- names(df)[i]
    }
  }
  features_to_remove <- rem_vec[!is.na(rem_vec)]
  rem_ind <- which(names(df) %in% features_to_remove)
  df <- df[,-rem_ind]
  return(df)
}

Example:

iris$Sepal.Width <- 0
new_df <- remove_zero_cols(iris)
print(new_df)
2
  • What is 6 in if(zero_test == 6) Commented Jun 22, 2018 at 17:12
  • The summary function on line 4 gives a 6 number summary of a column. If all 6 values are 0 then the column contains all zeros, and is thus removed.
    – Cybernetic
    Commented Jun 22, 2018 at 18:31
0

you can try something like

   [row, column] = SelectVar.shape
    for j in range(column):
       if np.all(SelectVar.iloc[:, j] == 0):
           SelectVar = SelectVar.drop(SelectVar.columns[j], axis=1, inplace=True)
0

Late answer, but one other base R approach which can work here would be to assert that either the minimum or maximum value in each column is not zero:

colMax <- sapply(SelectVar, max, na.rm=TRUE)
colMin <- sapply(SelectVar, min, na.rm=TRUE)
SelectVar[, colMin != 0 | colMax != 0]
0
SelectVar <- data.frame(
  a = c("Dxa8", "Dxb8"),
  b = c("Dxa8", "Dxc8"),
  c = c(0, 0),
  d = c("Dxa8", "Dxe8"),
  e = c("Dxa8", "Dxi8"),
  f = c(0, 0),
  g = c("Dxa8","tneg"),
  h = c("Dxa8", "tpos"),
  i = c(0, 0),
  j = c(0, 0),
  k = c(0, 0),
  l = c(0, 0),
  ll = c(0, 0),
  m = c(0, 0),
  n = c(0, 0),
  o = c(0, 0),
  p = c(0, 0),
  q = c("Dxc8", "Dxi8"),
  r = c(0, 0)
)


print(SelectVar)
colSums(SelectVar == 0) == nrow(SelectVar)
    a     b     c     d     e     f     g     h     i     j     k     l    ll     m     n     o     p     q 
FALSE FALSE  TRUE FALSE FALSE  TRUE FALSE FALSE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE FALSE 
    r 
 TRUE 
SelectVar[, colSums(SelectVar == 0) != nrow(SelectVar), drop=TRUE]

output is

a    b    d    e    g    h    q
1 Dxa8 Dxa8 Dxa8 Dxa8 Dxa8 Dxa8 Dxc8
2 Dxb8 Dxc8 Dxe8 Dxi8 tneg tpos Dxi8

or

# Identify columns with only zeros
zero_columns <- sapply(SelectVar, function(col) all(col == 0))

# Keep columns without only zeros
result <- SelectVar[, !zero_columns]
print(result)

or using library purr

library(purrr)
# Keep columns without only zeros
SelectVar %>%
  keep(~ any(. != 0))
print(result)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.