how do i get the number of visible charcters in a UTF8String regardless of how many bytes it takes to hold the string?
If you under Windows, try CharNextW:
uses
Winapi.Windows,
System.SysUtils;
function GetCharsCount(const AStr: UTF8String): Integer;
var
P: PWideChar;
begin
Result := 0;
P := PWideChar(UnicodeString(AStr));
while P <> '' do begin
Inc(Result);
P := CharNextW(P);
end;
end;
-
It might be worth pointing out that this approach can also be used for ordinary
string
s, not onlyUTF8String
s (indeed, this code begins by converting the latter to the former). – Andreas Rejbrand Nov 27 '20 at 21:28 -
@DavidHeffernan actually, it does: "This function works with default "user" expectations of characters when dealing with diacritics. For example: A string that contains U+0061 U+030a "LATIN SMALL LETTER A" + COMBINING RING ABOVE" — which looks like "å", will advance two code points, not one. A string that contains U+0061 U+0301 U+0302 U+0303 U+0304 — which looks like "a´^~¯", will advance five code points, not one, and so on." However,
P <> ''
does need to be changed toP^ <> #0
, at least – Remy Lebeau Nov 28 '20 at 0:35 -
I think this is fairly close. Take
∫àéôx̅ȧa̎b𝕴
, for example. There are arguably 9 visible characters.Length(S)
returns17
, whileGetCharsCount
returns10
. And how often do you use U+200B: ZERO WIDTH SPACE anyway? (Or do you have some other counterexample in mind, @DavidHeffernan?) Also, since the OP is talking aboutUTF8String
and not defining "visible characters" very precisely, maybe the OP would even considerLength(string(U))
to be good enough for his/her needs? – Andreas Rejbrand Nov 28 '20 at 0:38 -
Interestingly,
GetCharsCount
fails to treat a leading surrogate pair as a single character:b𝕴
(without the space) is said to be 2 chars, but𝕴b
is said to be 3 chars. Isn't that a bit strange? – Andreas Rejbrand Nov 28 '20 at 1:15 -
Sorry, comment deleted. My mistake. Answer would benefit from an explanation of all this though. Or perhaps just made a dupe of remy's linked answer. – David Heffernan Nov 28 '20 at 7:34
Length
of aUTF8String
is the number of encoded UTF codeunits, not the number of Unicode codepoints, or the number of visual graphemes the codepoints produce. – Remy Lebeau Nov 27 '20 at 18:51U
is aUTF8String
, thenLength(string(U))
is an often acceptable approximation to the actual number of "visible characters". In many applications, the number is exactly right. But as Remy points out, there are quite a few exceptions. (Even for an ordinarystring
,Length(s)
is only an approximation to the number of "visible characters".) – Andreas Rejbrand Nov 27 '20 at 18:59