3

I use RSelenium to run a scraping loop which sometimes (infrequently) meets an error and then stops.

The problem for me is that when this happens and I don't check in on the RSelenium session for a while (for like half an hour or so..?), the RSelenium session closes automatically, which removes logs from the session that I want to check.

How can I stop this from happening -- or more precisely, how can I prevent the RSelenium session (and the Firefox browser opened from RSelenium) from closing when left idle for an extended time period?

The following is how I start the scraping -- I open the Firefox browser like this, then go to the URL that I want and then start scraping.

library(RSelenium)
# Running with the browser open ------------------------------------------------
rD <- RSelenium::rsDriver(port = 4454L, browser = "firefox")

remDr <- rD$client
remDr$open()

P.S. Just to clarify, it's okay that the scraping stops once in a while -- that's how I can check for loopholes that I am missing. What I need is a way for me to stop the RSelenium session from closing when left idle. Thank you in advance for any help you can give!

1 Answer 1

0

Found a similar issue, https://github.com/ropensci/RSelenium/issues/241

chrome_prefs = 
  list(  
   # chrome prefs
    "profile.default_content_settings.popups" = 0L,
    "download.prompt_for_download" = FALSE
    )

chrome_args = 
  c(
  # chrome command arguments
    '--headless',
    '--window-size=1200,1800',
    '-sessionTimeout 57868143'
    )

eCaps_notimeout = 
  list(chromeOptions = 
         list(
           prefs = chrome_prefs,
           args = chrome_args 
  ))


  remDr <- remoteDriver(
    browserName = "chrome",
    extraCapabilities = eCaps_withhead
  )

Further reference Is there a way too prevent selenium automatically terminating idle sessions?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.