0

I have a source to a web page and I need to extract the body. So anything between </head><body> and </body></html>.

I've tried the following with no success:

var match = Regex.Match(output, @"(?<=\</head\>\<body\>)(.*?)(?=\</body\>\</html\>)");

It finds a string but cuts it off long before </body></html>. I escaped characters based on the RegEx cheat sheet.

What am i missing?

3
  • Generally it's ill-advised to use regular expressions to parse irregular language like HTML. Use an XML/HTML DOM parser like the HTML Agility Pack: htmlagilitypack.codeplex.com
    – Chad Levy
    Sep 21, 2011 at 21:04
  • 1
    I agree about regex not being the BEST choice, but there is an answer to this simple case. ::MY OPINION:: If you think a question is bad, then vote it down. If there is no way to answer a question the way that it is asked, then suggest an alternative to the idea so that the asker can get the same result. If there is an answer, THEN PROVIDE IT... don't simply poo poo an answerable question... that's my opinion of how this should work, anyways... Sep 21, 2011 at 21:18
  • It finds a string but cuts it off The way I see it, if that regex finds a string, the contents of capture buffer 1, are surrounded as the assertions dictate in the real text. Have you examined capture buffer 1 ? Even so, you should use 'i' option (single line) or use ([\S\s]*?)
    – user557597
    Sep 21, 2011 at 21:51

4 Answers 4

6

I'd recommend using the HtmlAgilityPack instead - parsing HTML with regular expressions is very, very fragile.

The latest version even supports Linq so you can get your content like this:

HtmlWeb web = new HtmlWeb();
HtmlDocument doc = web.Load("http://stackoverflow.com");
string html = doc.DocumentNode.Descendants("body").Single().InnerHtml;
1
  • Wow, had no idea this existed. Awesome, thanks. Exactly what I was looking for. Sep 21, 2011 at 21:18
2

Regex is not meant for such html handling, as many here would say. Without having your sample web page / html, I can only say that try removing the non-greedy ? quantifier in (.*?) and try. After all, a html page will have only one head and body.

1

Though regexes are definitely not the best tool for this task, there are a few suggestions and points I would like to make:

  1. un-escape the angle brackets - with the @ before your string, they are going through to the regex and they do not need to be escaped for a .NET regex
  2. with your regex, you need to make sure that the head/body tag combinations do not have any white-space between them.
  3. with your regex, the body tag cannot have any attributes.

I would suggest something more like:

(?<=</head>\s*<body(\s[^>]*)?>)(.*?)(?=</body>\s*</html>)

this seems to work for me on the source of this page!

1
  • regexes would need to support all the different possibilities of different things that could be inserted into various places in the DOM -- whitespace is allowed in html, and start-tags may or may not have attributes - something that this expression supports - but there may be more things that are not supported that I'm not thinking of! Sep 21, 2011 at 21:14
0

As the others have said, the correct way to handle this is with an HTML-specific tool. I just want to point out some problems with that cheat-sheet.

First, it's wrong about angle brackets: you do not need to escape them. In fact, it's wrong twice: it also says \< and \> match word boundaries, which is both incorrect for .NET, and incompatible with the advice about escaping angle brackets.

That cheat-sheet is just a random collection of regex syntax elements; most of them will work in most flavors, but many are guaranteed not to work in your particular flavor, whatever it happens to be. I recommend you disregard it and rely instead on .NET-specific documents or Regular-Expressions.info. The books Mastering Regular Expressions and Regular Expressions Cookbook are both excellent, too.

As for your regex, I don't see how it could behave the way you say it does. If it were going to fail, I would expect it to fail completely. Does your HTML document contain a CDATA section or SGML comment with </body></html> inside it? Or is it really two or more HTML documents run together?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.