I have a data frame that has 5 variables and 800 rows:
head(df)
V1 variable value element OtolithNum
1 24.9835 V7 130230.0 Mg 25
2 24.9835 V8 145844.0 Mg 25
3 24.9835 V9 126126.0 Mg 25
4 24.9835 V10 103152.0 Mg 25
5 24.9835 V11 129571.9 Mg 25
6 24.9835 V12 114214.0 Mg 25
I need to perform the following:
- identify all values (from the "value" variable) that are > 2 Standard Deviations from the median, grouped by the element variable.
- remove the outliers from the dataframe (or create a new dataframe with the outliers excluded.
I have been using dplyr package and have used the following code to group by the "element" variable, and provide the mean values:
df1=df %>%
group_by(element) %>%
summarise_each(funs(mean), value)
Can you please help me manipulate or add to the code above in order to remove outliers (defined above, as >2 sd from the median) grouped by the "element" variable, before I extract the means.
I have tried the following code from another posting (thats why the data names don't match with my personal data above), without luck:
#standardize each column (we use it in the outdet function)
scale(dat)
#create function that looks for values > +/- 2 sd from mean
outdet <- function(x) abs(scale(x)) >= 2
#index with the function to remove those values
dat[!apply(sapply(dat, outdet), 1, any), ]