4

Here is my problem, just hard for me...

I want to generate multiple datasets, then apply a function to these datasets and output corresponding output in single or multiple dataset (whatever possible)...

My example, although I need to generate a large number of variables and datasets

seed <- round(runif(10)*1000000)

datagen <- function(x){
set.seed(x)
var <- rep(1:3, c(rep(3, 3)))
yvar <- rnorm(length(var), 50, 10)
matrix <- matrix(sample(1:10, c(10*length(var)), replace = TRUE), ncol = 10)
mydata <- data.frame(var, yvar, matrix)
}

gdt <- lapply (seed,  datagen) 

# resulting list (I believe is correct term) has 10 dataframes: 
# gdt[1] .......to gdt[10]

# my function, this will perform anova in every component data frames and 
#output probability coefficients...  
anovp <- function(x){
          ind <- 3:ncol(x) 
          out <- lm(gdt[x]$yvar ~ gdt[x][, ind[ind]])
          pval <- out$coefficients[,4][2]
          pval <- do.call(rbind,pval) 
         }

plist <- lapply (gdt,  anovp) 

Error in gdt[x] : invalid subscript type 'list'

This is not working, I tried different options. But could not figure out...finally decided to bother experts, sorry for that...

My questions are:

(1) Is this possible to handle such situation in this way or there are other alternatives to handle such multiple datasets created?

(2) If this is right way, how can I do it?

Thank you for attention and I will appreciate your help...

1 Answer 1

7

You have the basic idea right, in that you should create a list of data frames and then use lapply to apply the function to each element of the list. Unfortunately, there are several oddities in your code.

There is no point in randomly generating a seed, then setting it. You only need to use set.seed in order to make random numbers reproducible. Cut the lines

seed <- round(runif(10)*1000000)

and maybe

set.seed(x)

rep(1:3, c(rep(3, 3))) is the same as rep(1:3, each = 3).


Don't call your variables var or matrix, since they will mask the names of those functions. since it's confusing.


3:ncol(x) is dangerous. If x has less than 3 columns it doesn't do what you think it does.


... and now, the problem you actually wanted solving.

The problem is in the line out <- lm(gdt[x]$yvar ~ gdt[x][, ind[ind]]).

lapply passes data frames into anovp, not indicies, so x is a data frame in gdt[x]. Which throws an error.


One more thing. While you are rewriting that line, note that lm takes a data argument, so you don't need to do things like gdt$some_column; you can just reference some_column directly.


EDIT: Further advice.

You appear to always use the formula yvar ~ X1 + X2 + X3 + X4 + X5 + X6 + X7 + X8 + X9 + X10. Since its the same each time, create it before your call to lapply.

independent_vars <- paste(colnames(gdt[[1]])[-1:-2], collapse = " + ")
model_formula <- formula(paste("yvar", independent_vars, sep = " ~ "))

I probably wouldn't bother with the anovp function. Just do

models <- lapply(gdt, function(data) lm(model_formula, data))

Then include a further call to lapply to play with the coefficients if necessary. The next line replicates your anovp code, but won't work because model$coefficients is a vector (so the dimensions aren't right). Adjust to retrieve the bit you actualy want.

coeffs <- lapply(models, function(model) do.call(rbind, model$coefficients[,4][2]))
4
  • Good advice except for the rationale for not using names of functions. Names of objects are kept in a separate list. Creating an object named "var" will not mask the var function, but it's still not a good idea because the human brain is not as carefully organized.
    – IRTFM
    Sep 4, 2011 at 14:35
  • @DWin: True, R is reasonable at figuring out whether to use a variable or a function, but the there are some ambiguous cases where masking does occur. For example, type var at the command prompt, and R prints the function definition. Now define var <- 1:5 and repeat. This time the varaible is printed. Sep 4, 2011 at 14:53
  • @RichieCotton Nice answer, but I agree with @DWin about masking - this is not an issue. See stackoverflow.com/q/6135868/602276 for an explanation. The code x <- 1:5; var(x); var <- "a"; var; var(x) will do everything as you expect, despite the fact that there is a new var called var.
    – Andrie
    Sep 4, 2011 at 17:33
  • thank you so much...I tried the following - still unsuccessful..still I am not grasping what you mean... "anovp <- function(x){ ind <- 3:ncol(x) out <- lm(x[,3] ~ x[, ind]) pval <- out$coefficients[,4][2] pval <- do.call(rbind,pval) }
    – jon
    Sep 4, 2011 at 22:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.