I'm writing some code (intended for use from both C and C++) that does alpha compositing using integer math (fixed point), and I'm coming across the problem that 0xff
isn't quite 1.0
. This is particularly problematic in the alpha channel. For example (0xff * 0xff) >> 8
equals 0xfe
. So if you have a white pixel with alpha 0xff
and RGB 0xff
, and you try to premultiply your alpha channel into the color channels, you end up with 0xfe
in the RGB channels, so you no longer really have white.
Here are some example functions that do alpha premultiplication and then alpha-compositing with premultiplied alpha, both of which have some variant of this problem. These are for little endian RGBA/BGRA, let's ignore byte order issues for now:
uint32_t premultiply_alpha(uint32_t color)
{
uint32_t alpha = ((color>>24) & 0xff);
uint32_t rb = color & 0xff00ffU;
uint32_t g = color & 0x00ff00U;
rb = (rb * alpha) >> 8;
g = (g * alpha) >> 8;
return (color & 0xff000000) | (rb & 0xff00ff) | (g & 0x00ff00);
}
uint32_t rgb_blend(uint32_t background, uint32_t color)
{
uint32_t alpha = color>>24;
uint32_t rb = background & 0xff00ffU;
uint32_t g = background & 0x00ff00U;
rb = (color & 0xff00ff) + ((0xff - alpha) * rb >> 8);
g = (color & 0x00ff00) + ((0xff - alpha) * g >> 8);
uint32_t dest_alpha = background>>24;
dest_alpha = alpha + dest_alpha - ((alpha*dest_alpha) >> 8);
dest_alpha = dest_alpha < 255 ? dest_alpha : 255;
return (dest_alpha << 24) | (rb & 0xff00ff) | (g & 0x00ff00);
}
The first function has the exact issue I described in prose above, and the second one has a similar problem in the expression (0xff - alpha)
, which can correctly express zero, but not 1.0.
How do people usually handle this without making a round trip through floating point? I'm sure people have already studied and solved this issue. Thanks.
Edit: as pointed out in the helpful comment by John, the issue is that >> 8
is equivalent to division by 0x100
, not 0xff
. Therefore, replacing the shift by / 0xff
solves the problem. However, since the divisor is constant, the compiler will avoid a div
instruction using the trick from Chapter 10 of Hacker's Delight (I confirmed that GCC does this even without optimization turned on). It seems empirically that it's not safe to process r and b in the same register, as I was doing here, in the presence of that optimization. I'm still interested in hearing if there are "standard" solutions to this / what other people tend to do.
>> 8
which is equivalent to/ 0x100
. What happens if you try/ 0xFF
instead of>> 8
?unsigned
-- In C++, the data types in <cstdint> would be used to guarantee the bitness of the type.