3

I have a data frame similar to the example below (which is a small section of my data frame). I would like to filter out rows based on the ‘pos’ column whereby I remove rows where the value is within 50 of one or more other rows, unless all of those nearby rows have values of less than 0.05 in either the ‘m_q_prop’ and ‘f_q_prop’ columns or ‘m_p_prop’ and ‘f_p_prop’ columns. In other words, remaining rows have a ‘pos’ value at least 50 away from other rows, or any rows within 50 of ‘pos’ have ‘m_q_prop’ and ‘f_p_prop’ or ‘m_p_prop’ and ‘f_p_prop’ values of less than 0.05. However, rows must be grouped by the ‘contig’ column, so that ‘pos’ values are only compared to other rows in the same ‘contig’ group.

I have attempted to use many various combinations of functions, such as those below, but all have produced errors. My assumption was that I would need to split the data frame by the ‘contig’ groups then filter the rows before rejoining the data frame. For the middle step, I tried to either directly filter the rows, or to create a new column of categories (e.g. suitable and unsuitable) that I could use to filter afterwards. The columns are also all characters so need to be converted to numeric, aside the ‘contig’ column (although the example data frame produces factors, and I am not sure how to convert those to characters to match my actual data).

Example data frame:

df <- data.frame(contig=c("Contig101702", "Contig101702", "Contig103637", 
"Contig105622", "Contig105622", "Contig105622", "Contig105901", 
"Contig105901", "Contig117031", "Contig118816", "Contig120914", 
"Contig120914"), 
pos=c("12048", "13281", "1398", "1078", "1090", "1125", "7178", "7180", 
"3236", "3700", "56298", "56326"),
m_p_prop=c("0.9789", "0.9792", "0.9845", "0.9787", "0.9839", "0.9826", 
"0.9468", "0.9468", "0.9713", "0.9794", "0.0195", "0.0048"),
f_p_prop=c("0.5047", "0.5000", "0.5089", "0.5000", "0.5000", "0.5050", 
"0.4867", "0.4867", "0.4810", "0.5086", "0.0491", "0.0012"),
m_q_prop=c("0.0211", "0.0208", "0.0155", "0.0213", "0.0161", "0.01744", 
"0.0532", "0.0532", "0.0287", "0.0206", "0.9805", "0.0052"),
f_q_prop=c("0.495", "0.5000", "0.4911", "0.5000", "0.5000", "0.4950", 
"0.5133", "0.5133", "0.5190", "0.4914", "0.9509", "0.9988"))

df

Expected output following filtering:

df_output

Examples of my attempts to filter (at this stage I was just attempting to filter by the 'pos' values, before I added the 'prop' requirements):

df_output <- df %>%
  split(df$contig) %>%
  by(function(x) {filter(!between(as.numeric(df$pos), 
                                 as.numeric(df$pos)-50, 
                                 as.numeric(df$pos)+50))}) %>%
  do.call(rbind, .)

df_output <- df %>%
  split(df$contig) %>%
  sapply( function(x) {filter(!as.numeric(df$pos)>=as.numeric(df$pos)-50 |
      !as.numeric(df$pos)<=as.numeric(df$pos)+50)}) %>%
  do.call(rbind, .)

df_output <- df %>% as.numeric(df$pos) %>%
  split(df$contig) %>%
  sapply( function(x)
   {mutate(suitable=ifelse(as.numeric(df$pos)>=as.numeric(df$pos)-50 |
                                            as.numeric(df$pos 
                                            <=as.numeric(df$pos)+50),
                              "bad", "good")}) %>%
  do.call(rbind,.)
3
  • could you give the desired output for your example ?
    – denis
    Jan 29, 2018 at 16:54
  • It is there already: it is the 'df_output' link to an image which is under the 'Expected output following filtering' heading
    – Katie
    Jan 29, 2018 at 17:03
  • First, why you don't have numeric variables from the beginning? Second, be careful when you transform factor variables to numeric. Try df$pos; as.numeric(df$pos) using the dataset you posted and see the difference.
    – AntoniosK
    Jan 29, 2018 at 17:26

1 Answer 1

0

A solution using . First, I converted all your numbers to numeric.

library(tidyverse)

df <- df %>% mutate_at(vars(-contig), funs(as.numeric(as.character(.))))

After that, I created a list column called Diff with all the combinations of numbers in each contig. I then created a column called Flag, which is TRUE if all number pairs are larger than 50 or there is only one number. Finally, I merged the data frame back to the original data frame. i then filtered the data frame as filter(Flag | (m_p_prop < 0.05 & f_p_prop < 0.05) | m_q_prop < 0.05 & f_q_prop < 0.05). df2 is the final output.

If you don't want to use map_lgl, replace it with sapply would also work.

df2 <- df %>%
  group_by(contig) %>%
  summarise(Diff = ifelse(n() > 1, list(combn(pos, 2)), list(NA))) %>%
  mutate(Flag = map_lgl(Diff, function(x){
    if (is.null(dim(x))){
      return(TRUE)
    } else {
      return(all(abs(x[1, ] - x[2, ]) > 50))
    }
  })) %>%
  right_join(df, by = "contig") %>%
  filter(Flag | (m_p_prop < 0.05 & f_p_prop < 0.05) | (m_q_prop < 0.05 & f_q_prop < 0.05)) %>%
  select(-Diff, -Flag)

df2
# A tibble: 7 x 6
#   contig         pos m_p_prop f_p_prop m_q_prop f_q_prop
#   <fct>        <dbl>    <dbl>    <dbl>    <dbl>    <dbl>
# 1 Contig101702 12048  0.979    0.505    0.0211     0.495
# 2 Contig101702 13281  0.979    0.500    0.0208     0.500
# 3 Contig103637  1398  0.984    0.509    0.0155     0.491
# 4 Contig117031  3236  0.971    0.481    0.0287     0.519
# 5 Contig118816  3700  0.979    0.509    0.0206     0.491
# 6 Contig120914 56298  0.0195   0.0491   0.980      0.951
# 7 Contig120914 56326  0.00480  0.00120  0.00520    0.999
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.