3

I'm attempting to download zip files directly from the Kaggle space in my R code itself. Unfortunately, it's not working out right. Here's what's happening:

For the San Francisco Crime Data set at https://www.kaggle.com/c/sf-crime/data

Take the first data set: test.csv.zip: https://www.kaggle.com/c/sf-crime/download/test.csv.zip

I'm using R code:

download.file(url='https://www.kaggle.com/c/sf-crime/download/test.csv.zip', destfile = 'test.zip',method = 'curl')

In place of the original 18.75MB file, R only downloads a 183byte file.

Session output:

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0100   183  100   183    0     0    665      0 --:--:-- --:--:-- --:--:--   667

What am I doing wrong?

Thanks in advance, Rahul

3
  • 2
    Have you logged into Kaggle?
    – PereG
    Feb 9, 2016 at 23:04
  • You're probably just receiving this: <html><head><title>Object moved</title></head><body> <h2>Object moved to <a href="/account/login?ReturnUrl=%2fc%2fsf-crime%2fdownload%2ftest.csv.zip">here</a>.</h2> </body></html>
    – Josh W.
    Feb 9, 2016 at 23:07
  • Yep, I have logged into Kaggle.
    – Rahul
    Feb 10, 2016 at 14:47

1 Answer 1

5
library(RCurl)

#Set your browsing links 
loginurl = "https://www.kaggle.com/account/login"
dataurl  = "https://www.kaggle.com/c/titanic/download/train.csv"

#Set user account data and agent
pars=list(
  UserName="suiwenfeng@live.cn",
  Password="-----"
)
agent="Mozilla/5.0" #or whatever 

#Set RCurl pars
curl = getCurlHandle()
curlSetOpt(cookiejar="cookies.txt",  useragent = agent, followlocation = TRUE, curl=curl)
#Also if you do not need to read the cookies. 
#curlSetOpt(  cookiejar="", useragent = agent, followlocation = TRUE, curl=curl)

#Post login form
welcome=postForm(loginurl, .params = pars, curl=curl)

bdown=function(url, file, curl){
  f = CFILE(file, mode="wb")
  curlPerform(url = url, writedata = f@ref, noprogress=FALSE, curl = curl)
  close(f)
}

ret = bdown(dataurl, "c:\\test.csv",curl)

rm(curl)
gc()

FYI : use RCurl like a web client.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.