Chapter 5. Blending, Keying, and Transitions

This chapter explains how to combine video frame information and computer-generated graphics on the Indigo2 workstation.

Use the VL and the Indigo2 Video board for Indigo2 IMPACT to perform three types of blending:

The choice “Blend/Wipe Node” in the Pro menu of the panel vcp, a graphical user interface for VL and the Indigo2 Video board for Indigo2 IMPACT, provides convenient access to blending, keying, and transition controls.

This chapter explains

The chapter concludes with descriptions of example application programs that are included in the software.

The Blend Node

Blending takes place in the VL's internal blend node, which is created with the vlGetNode() function. The code fragment in Example 5-1 sets up source, drain, and blend nodes.

Example 5-1. Setting Up Source, Drain, and Blend Nodes

/* variable definitions */
{
 VLServer vlSvr;
 VLPath path;
 VLNode drn_scr, drn_vid, src_scr, src_vid, blend_node;
}

/* Open a video device */ 
vlSvr = vlOpenVideo(“”);

/* Set up drain nodes on the screen and video */
drn_scr = vlGetNode(vlSvr, VL_DRN, VL_SCREEN, VL_ANY);
drn_vid = vlGetNode(vlSvr, VL_DRN, VL_VIDEO, VL_ANY);

/* Set up source nodes on the screen and video */
src_scr = vlGetNode(vlSvr, VL_SRC, VL_SCREEN, VL_ANY);
src_vid = vlGetNode(vlSvr, VL_SRC, VL_VIDEO, VL_ANY);

/* Set up internal blending node */
blend_node = vlGetNode(vlSvr, VL_INTERNAL, VL_BLENDER,
                       VL_ANY);

The blend node is supplied by up to four independent inputs:

  • pixel from a foreground source (A)

  • the alpha value for source A

  • pixel from a background source (B)

  • the alpha value for source B

Figure 5-1 diagrams the blend node.

Figure 5-1. Blend Node


The keyer can serve as the alpha value for source A.

The blend node has four multiplier stages, indicated by ⊗ in Figure 5-1, and one adder stage, indicated by ⊕. The values in the four multiplier stages are based on the blending functions selected and on the input normalization controls.

The rest of this section explains

  • using VL to set up the blend node

  • using VL blending controls to set blend function values

  • using vcp to set blend function values

Using VL to Set Up the Blend Node

Figure 5-2 diagrams setting up the blend node.

Figure 5-2. Setting Up the Blend Node


The blend node mixes the foreground and background video signals by applying a blend function to the foreground and background pixels.

The blend node then sends the data to the drain node. For example, blending analog video with part of a graphics screen and sending it to video out can be diagrammed as shown in Figure 5-3.

Figure 5-3. Blending Analog Video With Part of a Graphics Screen


Blending analog video with static frame data and sending it to video out can be diagrammed as shown in Figure 5-4.

Figure 5-4. Blending Analog Video With Static Frame Data


Using VL Blending Controls to Set Blend Function Values

The VL uses blending controls to set blending options.

All blending controls—that is, all the controls discussed in this chapter—apply only to blend nodes, except for VL_EV1_ALPHA_NOT_PIXEL, which applies to drain nodes. The order of blending and zooming depends on the node type: for a source, zooming takes place before blending; for a drain, blending takes place before zooming.

Table 5-1 summarizes the VL controls for blending.

Table 5-1. VL Blend Controls

Control

Values

Selects

VL_BLEND_A
type intVal

VLNode type, derived from vlGetNode();
must be one of the two source nodes

Input source for foreground image

VL_BLEND_B
type intVal

VLNode type, derived from vlGetNode();
must be one of the two source nodes

Input source for background image

VL_BLEND_A_ALPHA
type intVal

VLNode type, derived from vlGetNode();
must be one of the two source nodes

Input source for foreground alpha

VL_BLEND_B_ALPHA
type intVal

VLNode type, derived from vlGetNode();
must be one of the two source nodes

Input source for background alpha

VL_BLEND_A_FCN
type intVal

VL_BLDFCN_ZERO
VL_BLDFCN_ONE
VL_BLDFCN_B_ALPHA
background_alpha/255
VL_BLDFCN_MINUS_B_ALPHA:
1 - (background_alpha/255)

Blend function that controls mixing of foreground signals

VL_BLEND_B_FCN
type intVal

VL_BLDFCN_ZERO
VL_BLDFCN_ONE
VL_BLDFCN_A_ALPHA
foreground_alpha/255
VL_BLDFCN_MINUS_A_ALPHA
1 - (foreground_alpha/255)

Blend function that controls mixing of background signals

VL_BLEND_A_NORMALIZE
type boolVal

(0,1)
0 = off, 1 = on

Off is not supported by Indigo2 Video for Indigo2 IMPACT

VL_BLEND_B_NORMALIZE
type boolVal

(0,1)
0 = off, 1 = on

Follows Porter-Duff model (background pixels premultiplied by their corresponding alphas before blending)

VL_BLEND_OUT_NORMALIZE
type boolVal

1 = on

Not supported by IMPACT Indigo2 Video


Using vcp to Set Blend Function Values

The blender has two inputs, path A and path B. Each has an associated alpha that is separately controllable. IMPACT Indigo2 Video-specific VL controls set these inputs; they are discussed in “Using VL to Set Up the Blend Node,” later in this chapter. Settings in the panel control blending of these paths.

To use the panel to set the blend functions that control mixing of frames from paths A and B, select “Blend/Wipe Node” in the Pro menu. The following choices appear:

  • All

  • Keying Controls

  • Path Controls

  • Blending Controls

  • Orientation Controls

  • Rates Controls

  • Visual Quality Controls

Select “Path Controls”; the Blend/Wipe Node-Path Controls window appears, as shown in Figure 5-5.

Figure 5-5. vcp Path Controls Window


Notice that Blend Function A serves as foreground and Blend Function B serves as background.

The choices for foreground and background pixel and alpha values differ, however, as shown in Table 5-2.

Table 5-2. Choices for Foreground and Background Pixel and Alpha Values

Fg (A) Pixel

Bg (B) Pixel

Fg (A) Alpha

Bg (B) Alpha

Analog Video

Analog Video

Analog Video

Analog Video

IndyCam[tm]

IndyCam

IndyCam

IndyCam

Frame Buffer

Frame Buffer

Frame Buffer

Frame Buffer

Graphics A

Graphics A

Graphics A

Graphics A

Graphics B

Graphics B

Graphics B

Graphics B

 

Blender Alpha

Alpha = Keyer

Alpha = 0.0

 

Blender Pixel

Alpha = 1.0

Alpha = 1.0

Table 5-3 shows the choices for the two blend functions A and B, which correspond exactly.

Table 5-3. Choices for Blend Functions A and B

Blend Function A

Blend Function B

f(A) = 0.0

f(B) = 0.0

f(A) = 1.0

f(B) = 1.0

f(A) = f(A)

f(B) = f(B)

f(A) = 1 - f(A)

f(B) = 1 - f(B)

The value 0.0 sets the display to black (cut the foreground or background value); the value 1.0 sets the display to white (pass the foreground or background value):

  • If both foreground and background are set to 0.0, the result is black (both foreground and background are cut).

  • If foreground is set to 0.0 and background is set to 1.0, foreground is cut (ignored) and background is passed (displayed).

  • If foreground is set to 1.0 and background is set to 1-f(A), background obscures and overlaps foreground, resulting in compositing.

For foreground to background wipes, background alpha is set to a constant value of 1.0 so that the background shows through the foreground.

The checkboxes at the top of the Path Controls window set normalization on for each blending source. Normalized background pixels for a frame are premultiplied by their corresponding alphas before they are blended (Porter-Duff model). Figure 5-6 gives some examples of compositing, assuming normalized inputs.

Figure 5-6. Binary Compositing


Normally, chroma is multiplied (scaled) by the selected alpha. For example, the value on source A can be multiplied by its own alpha value or that from source B. In a normal blend, f(A), the incoming alpha of source A is applied to the value for A. In the inverse of this blend, f(A)=1-f(A), the region that was considered opaque (turned off), that is, outside the volume defined for keying, is applied to source A.

In another way of blending, the alpha from source B can be applied to the component represented by source A. In the inverse of this blend, f(A)=1-f(B), the region that was turned off for source B is applied to source A.

Keying

For each kind of keying—luma keying, chroma keying, and transitions—further VL controls enable you to specify the properties of the blend.

The values for the Indigo2 Video board for Indigo2 IMPACT “master” keyer control, VL_EV1_KEYER_MODE, determine the type of keying performed:

  • luma keying: VL_EV1_KEYERMODE_LUMA

  • chroma keying: VL_EV1_KEYERMODE_CHROMA

  • transitions, that is, fades, tiles, or wipes: VL_EV1_KEYERMODE_SPATIAL

For example, the following fragment specifies that chroma keying is to be performed:

VLControlType val;
val.intVal = VL_EV1_WIPETYPE_FADE;
vlSetControl(vlSvr, vlPath, blend_node, VL_EV1_WIPE_TYPE,
             &val); 

Keying controls fall into three groups:

  • luma keying

  • chroma keying

  • fades, tiles, and wipes

Each type of keying is explained separately in this section. The section concludes with a discussion of the Indigo2 Video for Indigo2 IMPACT keyer.

Figure 5-7 shows relationships between the Indigo2 Video board keying controls.

Figure 5-7. Indigo2 Video for Indigo2 IMPACT Keying Controls


Luma Keying

Luma keying is typically used to overlay a fixed image on video, such as the name and title of an individual being interviewed, a cable channel's logo, or a symbol that denotes an ongoing news story during a newscast. Figure 5-8 diagrams an application.

Figure 5-8. Luma Keying Application: Titling


The Indigo2 Video for Indigo2 IMPACT luma keying controls are summarized in Table 5-4; each is of type intVal.

Table 5-4. Indigo2 Video for Indigo2 IMPACT Luma Keying Controls

Control

Range

Sets

VL_EV1_KEYER_VALUE_LUMA

(0,255)

Central luma value. This control sets the luma value at which the background shows through the foreground.

VL_EV1_KEYER_RANGE_LUMA

(0,255)

One-sided range of the center value. This control determines the range of luma values where the background shows through the foreground.

VL_EV1_KEYER_FG_OPACITY

(0,255)

Opacity of the foreground, thus limiting the value of foreground alpha at any point.

VL_EV1_KEYER_DETAIL

(-8,7)

Sharpness of transition between foreground and background allowing blurring of edges. The value -8 yields the most gradual transition, +7 the sharpest.

Figure 5-9 diagrams the relationships between these controls.

Figure 5-9. Relationships Between Indigo2 Video for Indigo2 IMPACT Luma Keying Controls


Chroma Keying

Chroma keying overlays one image on another based on the color value. Figure 5-10 diagrams a common application.

Figure 5-10. Chroma Keying Application: TV Weather Map


Table 5-5 summarizes the controls for Indigo2 Video for Indigo2 IMPACT chroma keying and gives their ranges. These controls are all of type intVal.

Table 5-5. Indigo2 Video for Indigo2 IMPACT Chroma Keying Controls

Control

Range

Sets

VL_EV1_KEYER_VALUE_CHROMA_U

(-226,226)

Central U value at which the background shows through the foreground.

VL_EV1_KEYER_RANGE_CHROMA_U

(0,452)

One-sided range of U where the background shows through the foreground.

VL_EV1_KEYER_VALUE_CHROMA_V

(-179,179)

Central V value at which the background shows through the foreground.

VL_EV1_KEYER_RANGE_CHROMA_V

(0,358)

One-sided range of V where the background shows through the foreground.

VL_EV1_KEYER_DETAIL

(-8,7)

Sharpness of transition between foreground and background



Note: VL_EV1_KEYER_FG_OPACITY has no effect on Indigo2 Video for Indigo2 IMPACT in chroma key mode.

Figure 5-11 diagrams the relationships between these controls.

Figure 5-11. Relationships Between Indigo2 Video for Indigo2 IMPACT Chroma Keying Controls


Fades, Tiles, and Wipes

The values used with the control VL_EV1_WIPE_TYPE determine the type of blending performed:

  • from all-foreground to all-background: VL_EV1_WIPETYPE_FADE

  • from all-foreground to all-background by randomly tiling screen with rectangles of a specified size: VL_EV1_WIPETYPE_TILE

  • wipe to cross the screen as a vertical, diagonal, or horizontal “front,” with a specified angle: VL_EV1_WIPETYPE_SINGLE

  • wipe in two orthogonal directions simultaneously (two single wipes at the same time): VL_EV1_WIPETYPE_DOUBLE

  • wipe in two orthogonal directions, with the perpendicular position locked to the normal, or in-line position: VL_EV1_WIPETYPE_CORNER

For example, the following fragment specifies that a fade is to be performed:

VLControlType val;
val.intVal = VL_EV1_WIPETYPE_FADE;
vlSetControl(vlSvr, vlPath, blend_node, VL_EV1_WIPE_TYPE,
             &val); 

Fades, tiles, and wipes go from all-foreground (VL_EV1_WIPE_POSN=0) to all-background (VL_EV1_WIPE_POSN=1000), unless VL_EV1_WIPE_INVERT control is set, in which case they go from all-background (VL_EV1_WIPE_POSN = 0) to all-foreground (VL_EV1_WIPE_POSN = 1000).

Table 5-6 summarizes controls common to all wipe types.

Table 5-6. Controls for Fades, Tiles, and Wipes

Control

Values

Sets

VL_EV1_WIPE_POSN
type fractVal

Numerator (0,1000)
Denominator (1000)

Amount of progress of wipe, from none (numerator = 0) to full (numerator = 1000).

VL_EV1_WIPE_REPT
type intVal

(0,15)

Number of repetitions of pattern in direction of wipe, usually louvers on single, corner, or double wipe, and length of one side of rectangles for a tile wipe.
Note that this control does not apply to fades.

VL_EV1_WIPE_INVERT
type intVal

(0,1)
0 = off, 1 = on

Reversal of foreground and background regions of a wipe. When set to 0, wipes proceed from foreground (position = minimum) to background (position = maximum). When set to 1, wipes proceed from background (position = minimum) to foreground (position = maximum).

This value is buffered (does not go into effect) until another blending control is set.

Table 5-7 summarizes the controls specific to wipes or that work differently for wipes. Some of these controls work in conjunction with each other.

Table 5-7. Indigo2 Video for Indigo2 IMPACT Controls Specific to Wipes

Control

Values

Sets

VL_EV1_WIPE_DIRECTION
type intVal

VL_EV1_WIPEANGLE_E
VL_EV1_WIPEANGLE_NE
VL_EV1_WIPEANGLE_N
VL_EV1_WIPEANGLE_N W
VL_EV1_WIPEANGLE_W
VL_EV1_WIPEANGLE_SW
VL_EV1_WIPEANGLE_S
VL_EV1_WIPEANGLE_SE

Wipe vector direction, that is, the direction in which the wipe appears to be proceeding as its position increases.

Note: VL_EV1_WIPEANGLE_N and VL_EV1_WIPEANGLE_S do not work for the wipe types VL_EV1_WIPETYPE_DOUBLE and VL_EV1_WIPETYPE_CORNER

 

 

VL_EV1_WIPE_FUZZ
type intVal

(-8,7)

Sharpness of wipe transition band. As for VL_EV1_KEYER_DETAIL, -8 is most gradual, +7 is sharpest.

VL_EV1_WIPE_SYMMETRY
type intVal

(0,1)
0 = off, 1 = on

Wipe symmetry (on or off) so that wipe proceeds in both directions at once from the center line. Effect depends on type of wipe: no effect for fades or tiling; enables VL_EV1_WIPE_CENT for single, double, and corner wipes; enables VL_EV1_WIPE_CENT_PERP control for double and corner wipes.

VL_EV1_WIPE_POSN_PERP
type fractVal

numerator (0,1000)
denominator (1000)

Amount of progress of wipe, from none (numerator = 0) to full (numerator = 1000), along a direction perpendicular to normal wipe position VL_EV1_WIPE_POSN.

VL_EV1_WIPE_CENT
type fractVal

numerator (0,1000)
denominator (1000)

Offset that is center of a symmetrical wipe along wipe position. 0 means center is where VL_EV1_WIPE_POSN is 0, and 1000 means center is where VL_EV1_WIPE_POSN is 1000. For this control to work for single, double, and corner wipes, VL_EV1_WIPE_SYMMETRY must be on.

VL_EV1_WIPE_CENT_PERP
type fractVal

numerator (0,1000)
denominator (1000)

Offset that is center of a symmetrical wipe along a perpendicular wipe position. 0 means center is where VL_WIPE_POSN_PERP is 0, and 1000 means center is where VL_WIPE_POSN_PERP is 1000. VL_WIPE_SYMMETRY must be on for this control to work for double and corner wipes.

VL_EV1_WIPE_REPT_PERP
type intVal

(0,15)

Number of repetitions perpendicular to wipe direction for single, double, and corner wipes, and length of other side of rectangles for tile wipe.

In the vcp panel, the Blend/Wipe Node-All window is linked to the controls VL_EV1_WIPE_POSN_PERP, VL_EV1_WIPE_CENT, VL_EV1_WIPE_CENT_PERP, and VL_EV1_WIPE_REPT_PERP.

The Keyer

The role of the keyer is to take a pixel stream and produce an alpha stream. It generates a key for each pixel in each source node:

  • If luma keying is set, the keyer assesses the brightness of each pixel.

  • If chroma keying is set, the keyer assesses the color of each pixel.

  • If spatial, or transition, keying (fade, tile, wipe) is set, the keyer assesses the (x,y) coordinates for each pixel.

The keyer determines the alpha value (opacity) of a pixel and sets a value for it ranging from 0 (completely transparent) to 1 (completely opaque). This alpha value can be used downstream for further layering operations. The program simpleblend.c illustrates this procedure; it is included in the software and described at the end of this chapter.

In the video panel vcp, select the “Blend/Wipe Node” choice in the Pro menu. Select “Keying Controls”; The Blend/Wipe Node-Keying window appears, as shown in Figure 5-12.

Figure 5-12. Chroma Keying Controls


In this window, Keyer Mode can be ALPHA MODE, LUMA KEY, CHROMA KEY, or TRANSITION. The keyer mode setting determines the values you can set with the sliders in this window.

The sliders in this Blend/Wipe Node-Keying window have the following purposes:

  • Keyer Luma Value sets brightness level

  • Keyer Chroma U Value and Keyer Chroma V Value set chrominance

  • Keyer Luma Range and Keyer Chroma Range set the dimensions of the foreground area

  • Foreground Opacity limits the value of foreground alpha at any point

  • Keyer Detail sets the sharpness of the edge between the foreground and background: -8 is the softest transition and +7 is the sharpest transition

Figure 5-13 shows the relationships between value, range, and detail (transition) for a single channel (for example, A).

Figure 5-13. Value, Range, and Transition (Keyer Detail) for a Channel


Use the sliders to set the values you want, based on what you see on the screen. Once you are satisfied with the settings, use the values shown at the sliders to insert into your code.

VL Blending Examples

This section explains two example programs from /usr/people/4Dgifts/examples/dmedia/video/vl:

  • simpleblend.c

  • simplewipe.c

Because the programs are lengthy, they are not duplicated here. Look at the source code in a separate window, or print them out to look at while you read their descriptions.


Caution: To simplify the code, these examples do not check returns. You should, however, always check returns.


Blending Video and Graphics

simpleblend.c, which blends video with graphics and outputs it to both a graphics window and video out. The program

  • constrains the window's aspect ratio

  • checks that the device the user requested is in the device list

  • sets up a path between the source (screen) and the drain (video)

  • adds video source and a screen drain nodes to create the blend

  • sets the keyer mode, keyer source, and blend controls

  • displays the drain window and sets the video to appear in it

  • specifies appropriate event handling

  • starts data transfer

  • specifies that video is updated if the user changes the size of the window

Creating a Simple Wipe Effect

Like simpleblend.c, simplewipe.c blends video with graphics and outputs it to a graphics window and video out. When the user presses the w key, it executes a wipe.

Specifically, in addition to doing everything that simpleblend.c does, simplewipe.c

  • sets up blend parameters (VL_WIPE_TYPE, VL_WIPE_ANGLE, VL_WIPE_CENT, VL_WIPE_REPT)

  • calls a loop that sets the keyer mode to spatial and sets the position in the loop; doswitchloop() and dowipe() execute the loop

  • checks for the w key and calls dowipe(), which in turn calls doswitchloop()