1

I know this is probably simple and has probably been asked before, but I'm having trouble coming up with a solution.

I am parsing some RSS feeds which include HTML as CDATA blocks. One example is here: http://g.msn.com/1ewenus50/news2

The feed changes a lot, but there are almost always some extended characters in it. For example if I make a simple console app and use WebClient.DownloadString and look at the result, I see things like

"learned of the alleged attempted Flight 253 bomber’s extremist links while he was mid-flight on Christmas Day. NBC’s Savannah Guthrie reports. (Today Show)"

However those weird characters should be apostrophes, quote marks, em dashes, etc.

What is the trick for getting these to decode correctly?

If it wasn't clear, I'm using C# / .NET for this. In the end this content will be rendered in Silverlight, but I'm seeing the issue in the full .NET 3.5 runtime as well.

4
  • "has probably been asked before" did you have a look in SO to see whether it had? Jan 7, 2010 at 18:28
  • I did search around a bit but did not find the same issue. That doesn't mean it's not there, though. Jan 7, 2010 at 18:35
  • Did the content look ok in the browser? Jan 7, 2010 at 18:36
  • The bytes of the CDATA data in the RSS feed you mention are UTF-8 encoded, which is the same encoding as the XML itself. I would have expected .NET's XML parser to decode the CDATA bytes automatically, since CDATA is just a way of escaping XML markup characters, but it sounds like it is not fully decoding the CDATA correctly, or you are not parsing the XML correctly to allow it to handle CDATA correctly. Can you show your actual code? Jan 7, 2010 at 18:40

2 Answers 2

0

Download it in binary form and parse it as XML. That should get it right - the XML document should be self-describing in terms of the encoding, but I wouldn't put it past some webservers to advertise it (in headers) as having a different encoding, which would confuse DownloadString.

In general, when XML is involved it's worth doing as much as possible within an XML API rather than with the raw data.

1
  • There you go. Thanks. This works: byte[] bar = w.DownloadData(new Uri("g.msn.com/1ewenus50/news2")); string baz = new UTF8Encoding().GetString(bar); var x = XDocument.Parse(baz); Jan 7, 2010 at 18:39
0

You are probably using the wrong text encoding... I'm not sure which one you are using or which is the right one, but this might put you on the path.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.