3

I have a wget_string and commands made as follows:

wget_string <- paste("wget --user=" , u_name , " --password=", p_word,     " ", my_urls,' -qO ', file_name,   sep="")
 system(wget_string)
 readLines(file_name)

Which works, but then I have to use readLines() to read the file into R. I would like to run the command so that file is available directly in R, without being saved to the hard disk and then loaded from it.

I'm hoping to save resources by loading the file straight into R from the web. Can't use readlines from the beginning because of the secure server. What are the options for this?

2
  • Note there's a security issue with putting passwords on command lines like this - they can be read by anyone else on the system. Better to put them in .wgetrc (read the wget help for more).
    – Spacedman
    Jan 9, 2013 at 17:23
  • Thanks, appreciated. I was worried about that.
    – Yoda
    Jan 9, 2013 at 20:34

2 Answers 2

14

An intern=TRUE parameter the system function has. Capture the output from the command it does. Use the force, and the right options to wget to print to stdout:

> wget_string="wget -qO- http://www.google.com"
> s = system(wget_string,intern=TRUE)

If your returned data is a CSV file, textConnection you can use, feed it to read.csv you can.

0
5

If you use the -O - option of wget you set the output to the standar output (writing directly on the screen). In this way you can read directly from the output of the wget command.

E.g.

 wget -O - http: //www.address.com

Will download the web page and print it directly to the standard output. So you can directly read the output of the system(wget_string).

From the wget man page:

-O file --output-document=file The documents will not be written to the appropriate files, but all will be concatenated together and written to file. If - is used as file, documents will be printed to standard output, disabling link conversion. (Use ./- to print to a file literally named -.)

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.