1

I'm writing some code (intended for use from both C and C++) that does alpha compositing using integer math (fixed point), and I'm coming across the problem that 0xff isn't quite 1.0. This is particularly problematic in the alpha channel. For example (0xff * 0xff) >> 8 equals 0xfe. So if you have a white pixel with alpha 0xff and RGB 0xff, and you try to premultiply your alpha channel into the color channels, you end up with 0xfe in the RGB channels, so you no longer really have white.

Here are some example functions that do alpha premultiplication and then alpha-compositing with premultiplied alpha, both of which have some variant of this problem. These are for little endian RGBA/BGRA, let's ignore byte order issues for now:

uint32_t premultiply_alpha(uint32_t color)
{
        uint32_t alpha = ((color>>24) & 0xff);
        uint32_t rb = color & 0xff00ffU;
        uint32_t g  = color & 0x00ff00U;

        rb = (rb * alpha) >> 8;
        g  = (g  * alpha) >> 8;

        return (color & 0xff000000) | (rb & 0xff00ff) | (g & 0x00ff00);
}
uint32_t rgb_blend(uint32_t background, uint32_t color)
{
        uint32_t alpha = color>>24;
        uint32_t rb = background & 0xff00ffU;
        uint32_t  g = background & 0x00ff00U;
        rb = (color & 0xff00ff) + ((0xff - alpha) * rb >> 8);
        g  = (color & 0x00ff00) + ((0xff - alpha) * g  >> 8);

        uint32_t dest_alpha = background>>24;
        dest_alpha = alpha + dest_alpha - ((alpha*dest_alpha) >> 8);
        dest_alpha = dest_alpha < 255 ? dest_alpha : 255;

        return  (dest_alpha << 24) | (rb & 0xff00ff) | (g & 0x00ff00);
}

The first function has the exact issue I described in prose above, and the second one has a similar problem in the expression (0xff - alpha), which can correctly express zero, but not 1.0.

How do people usually handle this without making a round trip through floating point? I'm sure people have already studied and solved this issue. Thanks.


Edit: as pointed out in the helpful comment by John, the issue is that >> 8 is equivalent to division by 0x100, not 0xff. Therefore, replacing the shift by / 0xff solves the problem. However, since the divisor is constant, the compiler will avoid a div instruction using the trick from Chapter 10 of Hacker's Delight (I confirmed that GCC does this even without optimization turned on). It seems empirically that it's not safe to process r and b in the same register, as I was doing here, in the presence of that optimization. I'm still interested in hearing if there are "standard" solutions to this / what other people tend to do.

6
  • 5
    I guess the problem is >> 8 which is equivalent to / 0x100. What happens if you try / 0xFF instead of >> 8?
    – john
    Commented Jun 29 at 18:52
  • People handle it differently in the distinct languages. Pick only one in the tags.
    – 3CxEZiVlQ
    Commented Jun 29 at 19:15
  • @john thanks, you're totally right. I'm still interested in hearing whether this is the standard way people approach this or not (if there is a standard way), but this is clearly one valid way. Commented Jun 29 at 20:19
  • 3CxEZiVlQ, the example code I posted is completely reasonable in both C and C++, and I'd be interested in solutions leveraging the features of either one. Commented Jun 29 at 20:34
  • the example code I posted is completely reasonable in both C and C++ -- unsigned -- In C++, the data types in <cstdint> would be used to guarantee the bitness of the type. Commented Jun 29 at 21:01

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.