117

So I know about String#codePointAt(int), but it's indexed by the char offset, not by the codepoint offset.

I'm thinking about trying something like:

But my concerns are

  • I'm not sure whether codepoints which are naturally in the high-surrogates range will be stored as two char values or one
  • this seems like an awful expensive way to iterate through characters
  • someone must have come up with something better.

4 Answers 4

158

Yes, Java uses a UTF-16-esque encoding for internal representations of Strings, and, yes, it encodes characters outside the Basic Multilingual Plane (BMP) using the surrogacy scheme.

If you know you'll be dealing with characters outside the BMP, then here is the canonical way to iterate over the characters of a Java String:

final int length = s.length();
for (int offset = 0; offset < length; ) {
   final int codepoint = s.codePointAt(offset);

   // do something with the codepoint

   offset += Character.charCount(codepoint);
}
14
  • 26
    But you shouldn't. For instance if your program outputs XML and if someone gives it some obscure mathematical operator, suddenly your XML may be invalid. Commented Jul 15, 2012 at 1:18
  • 4
    @PaulGroke Yes there is. The function offsetByCodePoints (it redirects to Character.offsetByCodePoints) is like 50 lines long with loops and stuff, meanwhile charCount is just a one liner with a numeric if, so I guess there is a lot of performance loss.
    – Sipka
    Commented Aug 30, 2015 at 20:20
  • 3
    @Mechanicalsnail I don't understand your comment. Why would outputting XML cause this answer to misbehave?
    – Gili
    Commented Sep 22, 2015 at 18:41
  • 5
    @Gili the answer is fine. He was referring to @Jonathan Feinberg's comment in which he advocates for using charAt() which is a bad idea Commented Feb 17, 2018 at 18:28
  • 3
    Small modification to make it more continue-friendly: final int length = s.length(); for (int codepoint, offset = 0; offset < length; offset += Character.charCount(codepoint)) { codepoint = s.codePointAt(offset); // do something with the codepoint }
    – imgx64
    Commented Apr 9, 2019 at 6:03
87

Java 8 added CharSequence#codePoints which returns an IntStream containing the code points. You can use the stream directly to iterate over them:

string.codePoints().forEach(c -> ...);

or with a for loop by collecting the stream into an array:

for(int c : string.codePoints().toArray()){
    ...
}

These ways are probably more expensive than Jonathan Feinbergs's solution, but they are faster to read/write and the performance difference will usually be insignificant.

2
  • 4
    for (int c : (Iterable<Integer>) () -> string.codePoints().iterator()) also works.
    – user4910279
    Commented Jul 12, 2017 at 23:13
  • 2
    Slightly shorter version of @saka1029:s code: for (int c : (Iterable<Integer>) string.codePoints()::iterator) ...
    – Lii
    Commented Mar 24, 2018 at 9:40
10

Thought I'd add a workaround method that works with foreach loops (ref), plus you can convert it to java 8's new String#codePoints method easily when you move to java 8:

You can use it with foreach like this:

 for(int codePoint : codePoints(myString)) {
   ....
 }

Here's the method:

public static Iterable<Integer> codePoints(final String string) {
  return new Iterable<Integer>() {
    public Iterator<Integer> iterator() {
      return new Iterator<Integer>() {
        int nextIndex = 0;
        public boolean hasNext() {
          return nextIndex < string.length();
        }
        public Integer next() {
          int result = string.codePointAt(nextIndex);
          nextIndex += Character.charCount(result);
          return result;
        }
        public void remove() {
          throw new UnsupportedOperationException();
        }
      };
    }
  };
}

Or alternately if you just want to convert a string to an array of int codepoints (if your code could use a codepoint int array more easily) (might use more RAM than the above approach):

 public static List<Integer> stringToCodePoints(String in) {
    if( in == null)
      throw new NullPointerException("got null");
    List<Integer> out = new ArrayList<Integer>();
    final int length = in.length();
    for (int offset = 0; offset < length; ) {
      final int codepoint = in.codePointAt(offset);
      out.add(codepoint);
      offset += Character.charCount(codepoint);
    }
    return out;
  }

Thankfully uses "codePointAt" which safely handles the surrogate pair-ness of UTF-16 (java's internal string representation).

6

Iterating over code points is filed as a feature request at Sun.

See Bug Report

There is also an example on how to iterate over String CodePoints there.

1

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.