Graphic Twist

GPU Image Compositing without Programmable Shaders

Rectangles with gradient fill are drawn through curve stroke mask. Bitmaps are drawn through curve fill mask.

Image compositing is a feature that is becoming more and more important in modern computer graphics and graphical user interfaces. The basic idea is to perform a compositing operation with three key surfaces (which could be abstracted to “a bunch of pixels”): source, mask and target. Source gives the colors that we want to draw. Mask determines which pixels should be visible and which not or further, how much is each pixel visible. Target specifies where to draw the source pixels that pass through the mask.

Source is filtered through mask and remaining pixels displayed on destination surface.

Programmable shaders

Currently programmers are widely using GPU programmable shaders to perform image compositing. The source and mask surfaces are passed to a compositing shader program as two textures and the destination is set as the current drawing buffer. The shading program is written in a way to suit the desired compositing effect.

Alternative solution

However, the mask filtering effect can be also achieved without programmable shaders and without intermediate drawing of the desired graphics into textures. This makes it available to older graphic cards that don’t support GPU programming, such as the one I own. In this post I’ll describe how to perform compositing operations in real-time using OpenGL. I will not go into details of every used function, but will only provide a brief description of what they do. If you need a more detailed description, see the OpenGL specification.

The two key functionalities that are needed (and are also supported on older systems) are stencil buffer and blending operations. Stencil buffer will act as an all-or-nothing filter which will let through only the desired pixels which have at least a little opacity. After that, the blending operations will give the desired level of opacity/transparency to the rest of source pixels.

Stencil buffer

Stencil buffer is just another GPU buffer aside to the color and depth buffers. To write into the stencil buffer, we have to set the appropriate stencil function and stencil operation. Stencil function specifies which pixels will be passed on in the GPU pipeline and which will be discarded during the stencil test. The stencil function checks the current value in the stencil buffer at the coordinates of the incoming pixel and fails or passes according to the desired logical operation with the current stencil value. We can set the stencil function to check if the target stencil buffer value is greater, lower or equal (or any combination of them) to a reference value. Further we can specify a stencil test mask to use just a few bits of the target stencil and reference values when comparing them. The stencil operations specifies what is done with the target stencil value when the stencil test passes or fails. The value can be kept, replaced by the reference value of the stencil function, inverted, increased, decreased etc.

Blending

Blending operations take two colors (source and destination) and blend their red, green, blue and alpha components together according to the specified blending function to produce a new destination color. They usually take advantage of alpha layer, but the blending function can also be specified so as to use other means of providing the “weight” factors for the source and destination color components. Source color is the one of incoming pixel and destination is the value in the color buffer at the coordinates of the incoming pixel. To use blending functionality, we have to tell OpenGL were to take weights of each source and destination color component from and what to do with the weighted color components. We do this by setting so called blending function and blending equation. Blending function can take weight factors from the source alpha component, destination alpha component, component-wise from source and destination colors, component-wise from a specific constant color or any of 1 minus previous mentioned. Blending equation can add weighted components, subtract them etc.

Preparing the scene

Since we will use the stencil test and alpha blending, we must clear the stencil buffer and set the alpha of the color buffer to zero. A color mask is set so that we allow writing only to the alpha components of the color buffer. We should enable stencil test and blending only after clearing buffers, otherwise they could affect the clearing operation.

Drawing the mask

To gain the best performance we want to discard fully transparent pixels already before blending, because it is time-consuming. That’s why we use stencil buffer. We will set stencil buffer value for every mask pixel drawn to 1, and later discard all the pixels that have the stencil value not equal to 1. The stencil configuration is following: stencil function always passes and uses reference value of 1 ; stencil operation replaces the current stencil value with the reference value.

When we draw the mask we actually want only to set the alpha values of the destination surface. That’s why we should keep the previous color mask. In order not to override the destination alpha if the same pixel is drawn twice, we must use blending that accumulates alpha values. The needed blending configuration is following: take source alpha value and add destination alpha, weighted by source alpha.

Drawing the source

Now we got the compositing mask stored in the destination alpha color components, plus we can check the stencil buffer to know which pixels are fully transparent. Stencil test is configured this way: stencil function checks if the target stencil value is equal to the reference value, which is set to 1 ; stencil operation is not important. For blending, we use a typical transparency configuration, but instead of source alpha component, destination (mask) alpha is used. Color mask should be returned to default value.

Conclusion

If we use this kind of stencil and blending setup, everything that is drawn after mask-drawing configuration, will act as a compositing mask surface and only the pixels’ alpha component will be used while the destination colors will be untouched. Every pixels that later come after source-drawing configuration will be filtered through the mask and their color components blended with the destination according to the mask’s alpha values.

This way of color compositing is useful because it doesn’t need textures for specifying the compositing surfaces. The mask and source are specified on the fly through the drawn geometry which can, if one so wishes, still be a textured quad, but can also be any other type of geometry. If we want to additionally use source alpha values and combine them with the alpha values of the mask, a different blending setup that combines the two could easily be used before drawing the source pixels.

Links:

• Open Class Collection