2

I am trying to get the source from a URI. It's reported as UTF-8. I have also tried ISO-8859-1, ISO-8859-1 Windows-1250 and ISO-8859-2.

Here is my code of the latest attempt (trying ISO-8859-2):

public static String getPage(String page,String charset) throws IOException{
        URL url=new URL(page);

        return org.apache.commons.io.IOUtils.toString(url.openConnection().getInputStream(),charset);
    }

    public static void main(String args[])throws Exception{
        String page=getPage("http://buscon.rae.es/drae/srv/search?val=aba","ISO-8859-2");
        System.out.println(page);
    }

But the result is :

apÄ?ge 'quita, aparta', y este del gr. á¼?Ï?αγε)

instead of:

(Del lat. apăge 'quita, aparta', y este del gr. ἄπαγε).

Likewise UTF-8 (which works with other code, and in browsers) and other encoding names, also fail in a similar manner.

8
  • The entity that URI returns in your example uses UTF-8 (good for them, there's no call to be using ISO-8859-2 in this day and age, especially on the web). We can see this from the text/html; charset=UTF-8 header. It's also in Spanish (with some Latin and some Greek in the etymology). Is this the real URI? It can't duplicate the problem to look at a UTF-8 entity.
    – Jon Hanna
    Aug 7, 2012 at 16:14
  • Bye the bye. Doing a quick .NET downloader, I get (Del lat. apăge 'quita, aparta', y este del gr. ἄπαγε). if treating it as UTF-8. It's definitely UTF-8.
    – Jon Hanna
    Aug 7, 2012 at 16:17
  • If I UTF-8 in Java it doesn't return the same
    – Lengoman
    Aug 7, 2012 at 17:21
  • Sadly, I know the web and character-encoding part of this question, not the Java part. The URI you give returns perfectly good UTF-8, it's the Java bit that's wrong. (Unless you've a different URI in use, where does Romanian come into this?).
    – Jon Hanna
    Aug 7, 2012 at 17:24
  • 1
    Here's a question. How were you looking at the output? Could there be an encoding complication there? It's strange that you got Ïαγε in your ISO-8859-2 attempt, because some of those characters aren't in ISO-8859-2! Could you try your UTF-8 again, and save output to a UTF-8 text file rather than writing to the console? Maybe the code was perfect with UTF-8, but the console is letting you down.
    – Jon Hanna
    Aug 7, 2012 at 19:42

1 Answer 1

3

U+0103 (ă) is encoded as the byte sequence C4 83; this data is UTF-8.

The bug is likely due to the other transcoding operation you are performing via the PrintStream attached to System.out. This will encode the data to the system encoding, which may be a lossy conversion and may cause corruption if the device being written to doesn't use a matching encoding.

You can read some analysis of this with respect to the Windows console here.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.