0

Im using HtmlUnit to grab content of web page (having javascript)

   Stopwatch timer = new Stopwatch().start();
   final WebClient webClient = new WebClient(BrowserVersion.CHROME);
   webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
   webClient.getOptions().setJavaScriptEnabled(true);
   webClient.getOptions().setActiveXNative(false);
   webClient.getOptions().setAppletEnabled(false);
   webClient.getOptions().setCssEnabled(false);
   webClient.getOptions().setDoNotTrackEnabled(true);
   webClient.getOptions().setGeolocationEnabled(false);
   webClient.getOptions().setPopupBlockerEnabled(true);
   webClient.getOptions().setPrintContentOnFailingStatusCode(false);
   webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
   webClient.getOptions().setThrowExceptionOnScriptError(false);
   webClient.getOptions().setUseInsecureSSL(true);
   webClient.setCssErrorHandler(new SilentCssErrorHandler());
   webClient.getCookieManager().setCookiesEnabled(false);
   webClient.getOptions().setRedirectEnabled(false);
   webClient.getOptions().setTimeout(900);

   System.out.println("1.0 : " + timer.elapsed(TimeUnit.MILLISECONDS));
   final HtmlPage page = webClient.getPage(url);
   System.out.println(page.getWebResponse().getLoadTime());
   System.out.println("1.1 : " + timer.elapsed(TimeUnit.MILLISECONDS));

then the result i got is like this

   1.0 : 2
   707
   1.1 : 8003

Why webclient consuming more time (more than webresponse load time)? how can i reduce that time?

2
  • i refered here as well.
    – RDD
    Aug 25, 2014 at 7:57
  • i found that time increasing because of time consuming for javascript execution. Is there any way to reduce it?
    – RDD
    Aug 26, 2014 at 4:20

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.