11

I want to put some R code plus the associated data file (RData) on Github.

So far, everything works okay. But when people clone the repository, I want them to be able to run the code immediately. At the moment, this isn't possible because they will have to change their work directory (setwd) to directory that the RData file was cloned (i.e. downloaded) to.

Therefore, I thought it might be easier, if I changed the R code such that it linked to the RData file on github. But I cannot get this to work using the following snippet. I think perhaps there is some issue text / binary issue.

x <- RCurl::getURL("https://github.com/thefactmachine/hex-binning-gis-data/raw/master/popDensity.RData")
y <- load(x)

Any help would be appreciated.

Thanks

3 Answers 3

11

This works for me:

githubURL <- "https://github.com/thefactmachine/hex-binning-gis-data/raw/master/popDensity.RData"
load(url(githubURL))
head(df)
#          X        Y        Z
# 1 16602794 -4183983 94.92019
# 2 16602814 -4183983 91.15794
# 3 16602834 -4183983 87.44995
# 4 16602854 -4183983 83.79617
# 5 16602874 -4183983 80.19643
# 6 16602894 -4183983 76.65052

EDIT Response to OP comment.

From the documentation:

Note that the https:// URL scheme is not supported except on Windows.

So you could try this:

download.file(githubURL,"myfile")
load("myfile")

which works for me as well, but this will clutter your working directory. If that doesn't work, try setting method="curl" in the call to download.file(...).

3
  • Thanks for posting. Getting the following error message when I try the code above: Error in load(url(githubURL)) : cannot open the connection I am using R Studio. Commented Jul 20, 2014 at 1:52
  • I think the problem is to do with using https. See my edits for alternatives that (might) work - it's difficult to know since all of these work on my system. What happens if you just change the url to http://...? That works for me too.
    – jlhoward
    Commented Jul 20, 2014 at 16:42
  • 2
    The load from download.file causes bad restore file magic number (file may be corrupted) -- no data loaded. What worked to me: load(url(githubURL)). Commented Mar 12, 2019 at 14:35
2

I've had trouble with this before as well, and the solution I've found to be the most reliable is to use a tiny modification of source_url from the fantastic [devtools][1] package. This works for me (on a Mac).

load_url <- function (url, ..., sha1 = NULL) {
  # based very closely on code for devtools::source_url
  stopifnot(is.character(url), length(url) == 1)
  temp_file <- tempfile()
  on.exit(unlink(temp_file))
  request <- httr::GET(url)
  httr::stop_for_status(request)
  writeBin(httr::content(request, type = "raw"), temp_file)
  file_sha1 <- digest::digest(file = temp_file, algo = "sha1")
  if (is.null(sha1)) {
    message("SHA-1 hash of file is ", file_sha1)
  }
  else {
    if (nchar(sha1) < 6) {
      stop("Supplied SHA-1 hash is too short (must be at least 6 characters)")
    }
    file_sha1 <- substr(file_sha1, 1, nchar(sha1))
    if (!identical(file_sha1, sha1)) {
      stop("SHA-1 hash of downloaded file (", file_sha1, 
           ")\n  does not match expected value (", sha1, 
           ")", call. = FALSE)
    }
  }
  load(temp_file, envir = .GlobalEnv)
}

I use a very similar modification to get text files from github using read.table, etc. Note that you need to use the "raw" version of the github URL (which you included in your question).

[1] https://github.com/hadley/devtoolspackage

1

load takes a filename.

x <- RCurl::getURL("https://github.com/thefactmachine/hex-binning-gis-data/raw/master/popDensity.RData")
writeLines(x, tmp <- tempfile())
y <- load(tmp)
1
  • 1
    Thanks for posting. I tried the solution above but get the following message. I tried a few different files. Error: bad restore file magic number (file may be corrupted) -- no data loaded In addition: Warning message: file ‘filef00b79947a46’ has magic number '' Use of save versions prior to 2 is deprecated Commented Jul 20, 2014 at 1:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.