I am working with data from a meta-analysis of studies, where not every study had data for every genetic variant. I am trying to solve the following problem:
- I have a data frame ("studies") with information about 10 studies, which includes the number of cases and controls in each study and the total study sample size (N = cases + controls).
- I also have a data frame ("rawresults") with approximately 10 million rows, each of which was generated by summing some unknown subset of the sample sizes (N) from the table of studies.
- I need to figure out which studies were added up to create each total
Here are examples of what the data looks like:
> head(studies,2)
study cases controls N
1 A 3747 8024 11771
2 B 5367 5780 11147
> head(rawresults,2)
ID N
1 rs58241367 65280
2 rs85436064 107624
Here is an example of what I want to make from it (the contributing_studies column in optional, I can get rid of it if doing so allows for a better solution):
> head(final,2)
ID contributing_studies cases controls N
1 rs10685984 B,C,D,F,G 26221 19987 46208
2 rs12123751 A,C,D,G,J 25631 23509 49140
So far my best idea for how to approach this problem is to brute-force it. Each of the ten studies has two possible states (contributing vs. not contributing) for a given total, so that's 2^10 = 1024 possible sums. Some sums may not be unique (there may be more than one combination of study Ns that could create that sum) and I would plan to exclude those as ambiguous. I have included code below with an example of that solution as an answer.
What I want to ask: Does R have a better way to do this? Perhaps some library or function that exists for handling this kind of problem? Or is there something else I could do to make it faster and more efficient?
Here is code to simulate data for the scenario:
set.seed(1)
# Make "studies"
studies <- data.frame(toupper(letters[1:10]),round(rnorm(10,5000,2000)),round(rnorm(10,5000,2000)),stringsAsFactors=F)
colnames(studies) <- c('study','cases','controls')
studies$N <- studies$cases + studies$controls
# Make "rawresults"
rawresults <- data.frame(character(length=50),numeric(length=50),stringsAsFactors=F)
colnames(rawresults) <- c('ID','N')
for(i in seq(1,50)) {
numstudies <- sample(seq(5,10),1)
rawresults[i,'N'] <- sum(sample(studies$N,numstudies))
rawresults[i,'ID'] <- paste0('rs',sample(seq(1,99999999),1))
}
Edit: Faster code to simulate data for the scenario, so it will be possible to simulate millions of rawresults rows. Inspired by Allan Cameron's solution below, which also uses combn.
set.seed(1)
# Make "studies"
studies <- data.frame(toupper(letters[1:10]),round(rnorm(10,5000,2000)),round(rnorm(10,5000,2000)),stringsAsFactors=F)
colnames(studies) <- c('study','cases','controls')
studies$N <- studies$cases + studies$controls
# Make "rawresults"
num_results <- 50 # Number of results to simulate
possible_ns <- unlist(sapply(1:10,combn,x=studies$N,sum))
rawresults <- data.frame(paste0('rs',sample(1:99999999,num_results)),sample(possible_ns,num_results,rep=T),stringsAsFactors=F)
colnames(rawresults) <- c('ID','N')
x1,x2,x3
contribution fromstudy1, study2,study3
that makes up the total.. what i meant is, isx1,x2,x3
the same for every row. maybe I am complicating things.