Charset issues are confusing and complicated by themselves, but on top of that you have to remember exact names of your charsets. Is it "utf8"
? Or "utf-8"
? Or maybe "UTF-8"
? When searching internet for code samples you will see all of the above. Why not just make them named constants and use Charset.UTF8
?
6 Answers
The simple answer to the question asked is that the available charset strings vary from platform to platform.
However, there are six that are required to be present, so constants could have been made for those long ago. I don't know why they weren't.
JDK 1.4 did a great thing by introducing the Charset type. At this point, they wouldn't have wanted to provide String constants anymore, since the goal is to get everyone using Charset instances. So why not provide the six standard Charset constants, then? I asked Martin Buchholz since he happens to be sitting right next to me, and he said there wasn't a really particularly great reason, except that at the time, things were still half-baked -- too few JDK APIs had been retrofitted to accept Charset, and of the ones that were, the Charset overloads usually performed slightly worse.
It's sad that it's only in JDK 1.6 that they finally finished outfitting everything with Charset overloads. And that this backwards performance situation still exists (the reason why is incredibly weird and I can't explain it, but is related to security!).
Long story short -- just define your own constants, or use Guava's Charsets class which Tony the Pony linked to (though that library is not really actually released yet).
Update: a StandardCharsets
class is in JDK 7.
-
Just curious, any idea when there will be a release (alpha / beta / whatever) of Guava? The project homepage is a bit curt on this.– JonikCommented Nov 6, 2009 at 2:02
-
-
the reason why is incredibly weird and I can't explain it, but is related to security - you can create a modifiable String via custom charsets, yet they could have been made works even faster than string (which actually looks up the charset). It's an omission/neglect how
String(byte bytes[], int offset, int length, Charset charset)
is implemented. In fact, the performance hit is not trivial at all when creating a small string from a large byte[].– bestsssCommented Jan 16, 2013 at 7:24 -
7No fair! You have access to such great resources. =( I saw another answer where you once said, "Yeah, so I asked Josh [Bloch] about that..." Commented Jul 20, 2013 at 13:33
-
Two years later, and Java 7's StandardCharsets now defines constants for the 6 standard charsets.
If you are stuck on Java 5/6, you can use Guava's Charsets constants, as suggested by Kevin Bourrillion and Jon Skeet.
I'd argue that we can do much better than that... why aren't the guaranteed-to-be-available charsets accessible directly? Charset.UTF8
should be a reference to the Charset
, not the name as a string. That way we wouldn't have to handle UnsupportedEncodingException
all over the place.
Mind you, I also think that .NET chose a better strategy by defaulting to UTF-8 everywhere. It then screwed up by naming the "operating system default" encoding property simply Encoding.Default
- which isn't the default within .NET itself :(
Back to ranting about Java's charset support - why isn't there a constructor for FileWriter
/FileReader
which takes a Charset
? Basically those are almost useless classes due to that restriction - you almost always need an InputStreamReader
around a FileInputStream
or the equivalent for output :(
Nurse, nurse - where's my medicine?
EDIT: It occurs to me that this hasn't really answered the question. The real answer is presumably either "nobody involved thought of it" or "somebody involved thought it was a bad idea." I would strongly suggest that in-house utility classes providing the names or charsets avoid duplication around the codebase... Or you could just use the one that we used at Google when this answer was first written. (Note that as of Java 7, you'd just use StandardCharsets
instead.)
-
2+1. But as a method rather than a field so as to allow lazy loading (okay, you are probably going to want UTF-8, but there are a few other charsets about and you might want similar facilities for them). Unfortunately this doesn't seem to be very popular with those making the decisions. Commented Nov 5, 2009 at 22:22
-
I'd be happy enough with a method, although I hope that eagerly loading those very few charsets wouldn't be a significant cost. Commented Nov 5, 2009 at 22:24
-
1We're on a crusade to stop eager class loading. / Just did a search of a JDK for "UTF-8". Found 270 match(es) in 165 file(s). Although a lot of that is in old Apache junk (I believe contributed by my team). Commented Nov 5, 2009 at 22:33
-
1@tackline: I suppose eager classloading is one of those things that mounts up over time. A few classes here, a few classes there - each one individually sounding innocuous enough - could make a big difference. Commented Nov 5, 2009 at 22:46
-
@TomHawtin-tackline accessing a compile-time constant via a name (a
static final
field) does not cause class loading. There never was a reason to use a literal"UTF-8"
in thousand places instead of one canonical symbol. So it’s the opposite, using a method for that would actually cause class loading when the method is invoked.– HolgerCommented Jan 23, 2019 at 13:53
In Java 1.7
import java.nio.charset.StandardCharsets
ex:
StandardCharsets.UTF_8
StandardCharsets.US_ASCII
The current state of the encoding API leaves something to be desired. Some parts of the Java 6 API don't accept Charset
in place of a string (in logging
, dom.ls
, PrintStream
; there may be others). It doesn't help that encodings are supposed to have different canonical names for different parts of the standard library.
I can understand how things got to where they are; not sure I have any brilliant ideas about how to fix them.
As an aside...
You can look up the names for Sun's Java 6 implementation here.
For UTF-8, the canonical values are "UTF-8"
for java.nio
and "UTF8"
for java.lang
and java.io
. The only encodings the spec requires a JRE to support are: US-ASCII; ISO-8859-1; UTF-8; UTF-16BE; UTF-16LE; UTF-16.
-
2I don't begrudge the PrintStream one, as the class does clearly say "The PrintWriter class should be used in situations that require writing characters rather than bytes." (Which is, like, all situations...) Commented Nov 5, 2009 at 23:46
I have long ago defined a utility class with UTF_8, ISO_8859_1 and US_ASCII Charset constants.
Also, some long time ago ( 2+ years ) I did a simple performance test between new String( byte[], Charset )
and new String( byte[], String charset_name )
and discovered that the latter implementation is CONSIDERABLY faster. If you take a look under the hood at the source code you will see that they indeed follow quite a different path.
For that reason I included a utility in the same class
public static String stringFromByteArray (
final byte[] array,
final Charset charset
)
{
try
{
return new String( array, charset.name( ) )
}
catch ( UnsupportedEncodingException ex )
{
// cannot happen
}
}
Why the String( byte[], Charset ) constructor does not do the same, beats me.
-
1The
Charset
need not be registered, so the exception can happen. IIRC, there were some changes in JDK7 to make it faster for known-goodCharset
implementations (eliminate the extra copy). Commented Nov 6, 2009 at 0:49
MessageDigest#getInstance()
by the way.