How can I match characters (with the intention of removing them) from outside the unicode Basic Multilingual Plane in java?
2 Answers
To remove all non-BMP characters, the following should work:
String sanitizedString = inputString.replaceAll("[^\u0000-\uFFFF]", "");
-
Have you actually tested this? Because your character range includes the surrogate range used to construct non-BMP codepoints.– AnonOct 27, 2010 at 17:32
-
3@Anon: As you pointed out in your own answer, regexps are evaluated at the level of codepoints, not codeunits, so it doesn't see surrogates.– axtavtOct 27, 2010 at 17:35
-
-
@axtavt - actually, I assumed that regex was evaluated at the character level, and that the non-BMP codepoint was simply translated into surrogates.– AnonOct 27, 2010 at 17:42
-
1@Anon - More info on supplementary characters in java: java.sun.com/developer/technicalArticles/Intl/Supplementary Oct 27, 2010 at 17:44
Are you looking for specific characters or all characters outside the BMP?
If the former, you can use a StringBuilder
to construct a string containing code points from the higher planes, and regex will work as expected:
String test = new StringBuilder().append("test").appendCodePoint(0x10300).append("test").toString();
Pattern regex = Pattern.compile(new StringBuilder().appendCodePoint(0x10300).toString());
Matcher matcher = regex.matcher(test);
matcher.find();
System.out.println(matcher.start());
If you're looking to remove all non-BMP characters from a string, then I'd use StringBuilder
directly rather than regex:
StringBuilder sb = new StringBuilder(test.length());
for (int ii = 0 ; ii < test.length() ; )
{
int codePoint = test.codePointAt(ii);
if (codePoint > 0xFFFF)
{
ii += Character.charCount(codePoint);
}
else
{
sb.appendCodePoint(codePoint);
ii++;
}
}