215

Charset issues are confusing and complicated by themselves, but on top of that you have to remember exact names of your charsets. Is it "utf8"? Or "utf-8"? Or maybe "UTF-8"? When searching internet for code samples you will see all of the above. Why not just make them named constants and use Charset.UTF8?

4
  • 19
    +1: This was also bugging me all the time. The same story goes on for MessageDigest#getInstance() by the way.
    – BalusC
    Commented Nov 5, 2009 at 22:34
  • 2
    For the real answer, you'd need to ask someone at Sun. Good luck with that :-)
    – Stephen C
    Commented Nov 6, 2009 at 5:31
  • 1
    Stephen C: I believe it has been discussed on a public mailing list. -Someone at Sun. Commented Nov 6, 2009 at 15:17
  • 1
    see this question
    – yegor256
    Commented Sep 27, 2012 at 9:16

6 Answers 6

162

The simple answer to the question asked is that the available charset strings vary from platform to platform.

However, there are six that are required to be present, so constants could have been made for those long ago. I don't know why they weren't.

JDK 1.4 did a great thing by introducing the Charset type. At this point, they wouldn't have wanted to provide String constants anymore, since the goal is to get everyone using Charset instances. So why not provide the six standard Charset constants, then? I asked Martin Buchholz since he happens to be sitting right next to me, and he said there wasn't a really particularly great reason, except that at the time, things were still half-baked -- too few JDK APIs had been retrofitted to accept Charset, and of the ones that were, the Charset overloads usually performed slightly worse.

It's sad that it's only in JDK 1.6 that they finally finished outfitting everything with Charset overloads. And that this backwards performance situation still exists (the reason why is incredibly weird and I can't explain it, but is related to security!).

Long story short -- just define your own constants, or use Guava's Charsets class which Tony the Pony linked to (though that library is not really actually released yet).

Update: a StandardCharsets class is in JDK 7.

6
  • Just curious, any idea when there will be a release (alpha / beta / whatever) of Guava? The project homepage is a bit curt on this.
    – Jonik
    Commented Nov 6, 2009 at 2:02
  • No turkey for me til it's out! Commented Nov 7, 2009 at 4:43
  • the reason why is incredibly weird and I can't explain it, but is related to security - you can create a modifiable String via custom charsets, yet they could have been made works even faster than string (which actually looks up the charset). It's an omission/neglect how String(byte bytes[], int offset, int length, Charset charset) is implemented. In fact, the performance hit is not trivial at all when creating a small string from a large byte[].
    – bestsss
    Commented Jan 16, 2013 at 7:24
  • 7
    No fair! You have access to such great resources. =( I saw another answer where you once said, "Yeah, so I asked Josh [Bloch] about that..."
    – kevinarpe
    Commented Jul 20, 2013 at 13:33
  • PrintStream doesn't support Charset
    – rofrol
    Commented Nov 24, 2014 at 12:35
105

Two years later, and Java 7's StandardCharsets now defines constants for the 6 standard charsets.

If you are stuck on Java 5/6, you can use Guava's Charsets constants, as suggested by Kevin Bourrillion and Jon Skeet.

29

I'd argue that we can do much better than that... why aren't the guaranteed-to-be-available charsets accessible directly? Charset.UTF8 should be a reference to the Charset, not the name as a string. That way we wouldn't have to handle UnsupportedEncodingException all over the place.

Mind you, I also think that .NET chose a better strategy by defaulting to UTF-8 everywhere. It then screwed up by naming the "operating system default" encoding property simply Encoding.Default - which isn't the default within .NET itself :(

Back to ranting about Java's charset support - why isn't there a constructor for FileWriter/FileReader which takes a Charset? Basically those are almost useless classes due to that restriction - you almost always need an InputStreamReader around a FileInputStreamor the equivalent for output :(

Nurse, nurse - where's my medicine?

EDIT: It occurs to me that this hasn't really answered the question. The real answer is presumably either "nobody involved thought of it" or "somebody involved thought it was a bad idea." I would strongly suggest that in-house utility classes providing the names or charsets avoid duplication around the codebase... Or you could just use the one that we used at Google when this answer was first written. (Note that as of Java 7, you'd just use StandardCharsets instead.)

8
  • 2
    +1. But as a method rather than a field so as to allow lazy loading (okay, you are probably going to want UTF-8, but there are a few other charsets about and you might want similar facilities for them). Unfortunately this doesn't seem to be very popular with those making the decisions. Commented Nov 5, 2009 at 22:22
  • I'd be happy enough with a method, although I hope that eagerly loading those very few charsets wouldn't be a significant cost.
    – Jon Skeet
    Commented Nov 5, 2009 at 22:24
  • 1
    We're on a crusade to stop eager class loading. / Just did a search of a JDK for "UTF-8". Found 270 match(es) in 165 file(s). Although a lot of that is in old Apache junk (I believe contributed by my team). Commented Nov 5, 2009 at 22:33
  • 1
    @tackline: I suppose eager classloading is one of those things that mounts up over time. A few classes here, a few classes there - each one individually sounding innocuous enough - could make a big difference.
    – Jon Skeet
    Commented Nov 5, 2009 at 22:46
  • @TomHawtin-tackline accessing a compile-time constant via a name (a static final field) does not cause class loading. There never was a reason to use a literal "UTF-8" in thousand places instead of one canonical symbol. So it’s the opposite, using a method for that would actually cause class loading when the method is invoked.
    – Holger
    Commented Jan 23, 2019 at 13:53
28

In Java 1.7

import java.nio.charset.StandardCharsets

ex: StandardCharsets.UTF_8 StandardCharsets.US_ASCII

5

The current state of the encoding API leaves something to be desired. Some parts of the Java 6 API don't accept Charset in place of a string (in logging, dom.ls, PrintStream; there may be others). It doesn't help that encodings are supposed to have different canonical names for different parts of the standard library.

I can understand how things got to where they are; not sure I have any brilliant ideas about how to fix them.


As an aside...

You can look up the names for Sun's Java 6 implementation here.

For UTF-8, the canonical values are "UTF-8" for java.nio and "UTF8" for java.lang and java.io. The only encodings the spec requires a JRE to support are: US-ASCII; ISO-8859-1; UTF-8; UTF-16BE; UTF-16LE; UTF-16.

1
  • 2
    I don't begrudge the PrintStream one, as the class does clearly say "The PrintWriter class should be used in situations that require writing characters rather than bytes." (Which is, like, all situations...) Commented Nov 5, 2009 at 23:46
2

I have long ago defined a utility class with UTF_8, ISO_8859_1 and US_ASCII Charset constants.

Also, some long time ago ( 2+ years ) I did a simple performance test between new String( byte[], Charset ) and new String( byte[], String charset_name ) and discovered that the latter implementation is CONSIDERABLY faster. If you take a look under the hood at the source code you will see that they indeed follow quite a different path.

For that reason I included a utility in the same class

public static String stringFromByteArray (
    final byte[] array,
    final Charset charset
)
{
    try
    {
        return new String( array, charset.name( ) )
    }
    catch ( UnsupportedEncodingException ex )
    {
        // cannot happen
    }
}

Why the String( byte[], Charset ) constructor does not do the same, beats me.

1
  • 1
    The Charset need not be registered, so the exception can happen. IIRC, there were some changes in JDK7 to make it faster for known-good Charset implementations (eliminate the extra copy). Commented Nov 6, 2009 at 0:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.