0

I'm getting some foreign characters (Chinese in this case) in the response body received by the browser being replaced by the replacement character, � (Unicode: U+fffd). What was one Chinese character become 2 or 3 replacement characters. It seems like the Unicode for these characters somehow becomes corrupt by the time the response hits the browser.

It looks similar to what was asked about here and here, but neither had a real solution. I'm hoping that with further information I put, someone might have an idea.

The JSON blob containing the foreign characters is about 500KB. The full page response is about 700KB. The corruption or replacement is spread out over the JSON blob. It can happen at the beginning of, in the middle of and at the end of the foreign word, but only affecting about 2 to 5 characters.

The response has "Content-Type:text/html; charset=utf-8". On the page load from the initial GET request, the handful of characters that are replaced are part of what will be passed as a JavaScript object to an AMD module via an ASP.NET ScriptManager inside an HTML script tag on the main page document. What is really interesting is that this same module can be reloaded with an ASP.NET postback. The postback response does not have any of the foreign characters replaced.

Also, I can get to this dynamically created page with different URLs. If I get there with just the base URL, I get some characters that are replaced. If I go to /default.aspx, I get different characters and amount (still just a handful) that are replaced. I can also go to /default.aspx?=XXXX to get there and I then see even different characters replaced. So either the replacement is dependent on the content length? or maybe just the URL and method? I get the same result for the same URL regardless of which browser I use.

Any ideas will be greatly appreciated.

3
  • take a look at this answer: stackoverflow.com/questions/13701648/…
    – Aristos
    May 26, 2016 at 17:23
  • Thanks for the idea. I will try removing the content-length header. I have already tried turning on and off both static and dynamic compression in IIS. I think the default is gzip. Nothing changed when I changed the compression settings. I don't believe there is any other compression occurring. But we will see what removing the content length does.
    – MDavid
    May 26, 2016 at 17:26
  • So, It looks like the Content-Length header wasn't being set explicitly anywhere. IIS was already setting it. And without any compression happening that I can see, It looks like that isn't the culprit.
    – MDavid
    May 26, 2016 at 20:31

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.