15

Given this area of the Unicode table, for instance:

  ...
π‘Ž    U+1D44E Dec:119886       MATHEMATICAL ITALIC SMALL A 𝑎
𝑏    U+1D44F Dec:119887       MATHEMATICAL ITALIC SMALL B 𝑏
𝑐    U+1D450 Dec:119888       MATHEMATICAL ITALIC SMALL C 𝑐
𝑑    U+1D451 Dec:119889       MATHEMATICAL ITALIC SMALL D 𝑑
𝑒    U+1D452 Dec:119890       MATHEMATICAL ITALIC SMALL E 𝑒
𝑓    U+1D453 Dec:119891       MATHEMATICAL ITALIC SMALL F 𝑓
𝑔    U+1D454 Dec:119892       MATHEMATICAL ITALIC SMALL G 𝑔
𝑖    U+1D456 Dec:119894       MATHEMATICAL ITALIC SMALL I 𝑖 # what?!
𝑗    U+1D457 Dec:119895       MATHEMATICAL ITALIC SMALL J 𝑗
π‘˜    U+1D458 Dec:119896       MATHEMATICAL ITALIC SMALL K 𝑘
𝑙    U+1D459 Dec:119897       MATHEMATICAL ITALIC SMALL L 𝑙
π‘š    U+1D45A Dec:119898       MATHEMATICAL ITALIC SMALL M 𝑚
𝑛    U+1D45B Dec:119899       MATHEMATICAL ITALIC SMALL N 𝑛
π‘œ    U+1D45C Dec:119900       MATHEMATICAL ITALIC SMALL O 𝑜
  ...

I would naturally expect u+1d455 to be MATHEMATICAL ITALIC SMALL H. But it seems not defined on any table I look around.

Why are there holes in Unicode table? (also U+1d49d, u+1d53a, etc.)
Is there any way I can fill them?


[EDIT]: These links do state:

The "holes" in the alphabetic ranges are filled by previously defined characters in the Letter like Symbols block shown below.

and

The Unicode Consortium adds new codepoints to the standard all the time. Visit their website to find out about pending codepoints and whether this one is in the pipe. The following table shows typical representations of how the codepoint would look, if it existed. This may help you when debugging, but is not of real use otherwise.

But I just... don't understand what they mean :\

13
  • codepoints.net/U+1D455
    – SLaks
    Nov 9, 2017 at 15:49
  • The first paragraph of your first link: "The "holes" in the alphabetic ranges are filled by previously defined characters in the Letter like Symbols block shown below."
    – JJJ
    Nov 9, 2017 at 15:50
  • 6
    Unicode wasn't built in one day, as the saying goes. For example, h was defined in an earlier block as U+210E (h being one of the very common symbols as it denotes the Planck constant.) Later the full alphabet was added but they didn't want to duplicate characters so the spot where the earlier symbol would have gone was left blank.
    – JJJ
    Nov 9, 2017 at 15:57
  • 2
    Let's say your software checks if a character is the "mathematical italic small h": if(chr == U+210E) ... Later Unicode is updated with new characters and there are now two codepoints for h. Suddenly the software has a bug and needs an update to check if(chr == U+210E || chr == U+1D455) ... Not to mention if the software tries to check if two characters are the same just in general: if( chr1 == chr2 || (chr1 == U+210E && chr2 == U+1D455) || ...
    – JJJ
    Nov 9, 2017 at 16:12
  • 1
    Well, in practice it is. In any modern language (constant) string comparison is very inexpensive because you can just check if the strings' memory addresses match, regardless of how long the strings are. If you have to check every character one by one, you'll go from O(1) complexity to O(n) and it's gets worse and worse the more duplicates you have. You'd be forced to choose between accuracy and performance. It would be a horrible mess.
    – JJJ
    Nov 9, 2017 at 16:23

2 Answers 2

17

From the comments (cheers guys), I have learnt that these holes are due to some characters being already assigned in Unicode when the whole alphabet had been added.

For instance: before U+1D4* MATHEMATICAL ITALIC SMALL * identifiers were defined, β„Ž was already known in the table as

β„Ž    U+210E Dec:008462        PLANCK CONSTANT ℎ # here it is

So in order to keep consistency in numbering but NOT to duplicate β„Ž id, a hole has been inserted at U+1D455 position.


Similarly, ℬ is known as U+212C SCRIPT CAPITAL B rather than U+1D49D - - - reserved in the MATHEMATICAL SCRIPT CAPITAL letters family.

Similarly, β„‚ from MATHEMATICAL DOUBLE-STRUCK CAPITAL letters family is not U+1D53A because it was already known as U+2102 DOUBLE-STRUCK CAPITAL C.

This was a difficult choice, having to deal with retro-compatibility, consistency and reliability altogether :)

1
  • 1
    Did you find any references that can be used to map the missing characters to the other existing characters? Oct 3, 2022 at 2:23
1

First of all, sorry for necroposting, but I believe that if I ended up here through a Google search where it was the first or second result, many other people might, too, and they will be as confused as I was.

I don't have a final answer, but I wanted to point out that iago-lito's answer is not completely rightβ€”it seems to be a legitimate mistake, whether from the Unicode Consortium, the operating systems I've used to check, or the typeface designers. Well, at least in the case of that specific h: there is the β„Ž that's used for the Plack constant, but there is no glyph that would fit what we would consider the mathematical italic small hβ€”that is, a regular width italic serif lowercase h, actually.

My suspicions are that, at the time, most people used serif typefaces everywhere, as Times New Roman is both the default typeface for LaTeX and for many scientific writing guides, such as APAβ€”not to mention browsers, which usually have Times New Roman as the default serif and default typeface. So it could be that the Planck constant h was always rendered as serif, but now, since we use sans-serif typefaces, it's displayed as sans-serif, and there seems to be no way to get a proper, regular weight serif small letter h. Bear in mind that the Planck constant address doesn't have a specific glyph; font files just "redirect" the address to the glyph of whatever letter h they use, so that's why I think that's a possibility, even if it doesn't make so much sense when you think about it.

It's also important to note that many characters have various identical versions throughout Unicode, and, in fact, there is the entire sans-serif alphabet under between 0x1D5A0 - mathematical sans-serif capital a, and 0x1D5D3 - mathematical sans-serif small z, so it's puzzling why they decided not to add this one letterβ€”though people have speculated that it's because of how 'famous' the other one was, and you do want backwards-compatibility. But that doesn't answer it for me, as that actually wouldn't break compatibility. It would just mean that they used the wrong one, and now there is a right one.

Of course, I'm not entirely sure it is a problem in the Unicode Consortium's standard. It could be a mistake in the typeface; maybe the typefaces should have used a serif h as Planck's constant. But this seems to be wide-spread regardless of font file, and, at the very least, there isn't clarity on what typeface designers should have done.

I have, now, submitted a request for information to the Unicode Consortium as to whether they plan to add the letter. Hopefully, they will add it, as the byte number does exist. At least they were this smart.

Meanwhile you can use the mathematical bold italic small h, 𝒉, which is represented in 8-bit as 0x1D489, or in html as 𝒉. That's all for now, at least.

6
  • 1
    Sure there's a way to "get a proper, regular weight serif small letter h" - use a serif font! The Unicode description of the various characters don't specify sans or serif, so there's no requirement to make it one way or the other. You'd expect it to follow the style of the rest of the font. Oct 3, 2022 at 2:22
  • It's not possible to use a Serif font on most mobile text editors and on many other situations. Also, there is a requirement, which is the fact that among the "mathematical alphanumeric symbols" section of Unicode there is always specification for font, including sans-serif, except one. And every other letter on the set that's missing specification is serif. Oct 5, 2022 at 18:52
  • Also, please don't take this the wrong way (seriously πŸ€—) but I don't believe we should ever cast doubt on whether someone needs what they are asking. I'm not saying you did it, I think you had a fair shot at a recommendation (about using a serif typeface). I just wanted to mention it because people get frisky about it, many times people are being very patronizing. Personally, I only think it's ever acceptable to ask "why do you need that" if you are legitimately trying to find a way out through a different method. Oct 5, 2022 at 19:01
  • I hear you. I have an ex-co-worker who refuses to use Stack Overflow, because he found himself constantly arguing with people who were questioning his motives for asking the question. As for your own answer, I don't know enough about the history of these characters to judge it. If there's a legitimate reason that Planck's constant should be a different glyph than mathematical small h, then indeed the Unicode Consortium made a mistake but left themselves a way of fixing it if they so choose. I somehow doubt that will ever happen though. Oct 6, 2022 at 21:45
  • 1
    Looking at The Unicode Standard, Version 15.0, the table on page 2 shows not only is 'h' missing but so are 11 other letters. The following pages are similar. It's rather frustrating not being able to increment a letter by 1 in order to get the next one, especially when they have left each of the slots available for that purpose. Leaving them empty seems worse than pointless; it's like they went out of their way to be annoying. Jun 20 at 3:26

Your Answer

By clicking β€œPost Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.