210

I do a lot of HTML parsing in my line of work. Up until now, I was using the HtmlUnit headless browser for parsing and browser automation.

Now, I want to separate both tasks.

I want to use a light HTML parser because it takes much time in HTMLUnit to first load a page, then get the source, and then parse it.

I want to know which HTML parser can parse HTML efficiently. I need

  1. Speed
  2. Ease to locate any HtmlElement by its "id" or "name" or "tag type".

It would be ok for me if it doesn't clean the dirty HTML code. I don't need to clean any HTML source. I just need the easiest way to move across HtmlElements and harvest data from them.

1
  • The slowness may be if the parser downloads DTD's and/or XSD's off the internet. Commented Oct 1, 2015 at 15:30

3 Answers 3

432

jsoup

Self plug: I have just released a new Java HTML parser: jsoup. I mention it here because I think it will do what you are after.

Its party trick is a CSS selector syntax to find elements, e.g.:

String html = "<html><head><title>First parse</title></head>"
  + "<body><p>Parsed HTML into a doc.</p></body></html>";
Document doc = Jsoup.parse(html);
Elements links = doc.select("a");
Element head = doc.select("head").first();

See the Selector javadoc for more info.

This is a new project, so any ideas for improvement are very welcome!

1
  • jsoup works very well, highly recommended! Commented Feb 21 at 23:06
33

The best I've seen so far is HtmlCleaner:

HtmlCleaner is open-source HTML parser written in Java. HTML found on Web is usually dirty, ill-formed and unsuitable for further processing. For any serious consumption of such documents, it is necessary to first clean up the mess and bring the order to tags, attributes and ordinary text. For the given HTML document, HtmlCleaner reorders individual elements and produces well-formed XML. By default, it follows similar rules that the most of web browsers use in order to create Document Object Model. However, user may provide custom tag and rule set for tag filtering and balancing.

With HtmlCleaner you can locate any element using XPath.

For other html parsers see this SO question.

3
  • 2
    You have to handle the proxy yourself, then using HtmlCleaner to process your stream instead. => It is not convenient. T__T
    – kidnan1991
    Commented Oct 4, 2016 at 2:40
  • Wouldn't HTMLTidy be a better option here? html-tidy.org Commented Oct 25, 2016 at 12:03
  • 2
    Similar rules that the most of web browsers use - This is not very convincing Commented Jul 5, 2017 at 3:30
10

I suggest Validator.nu's parser, based on the HTML5 parsing algorithm. It is the parser used in Mozilla from 2010-05-03

1
  • 3
    I tried this and found it super hard to use. Very little documentation / support. Commented Jan 3, 2014 at 18:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.