The Video Library for the OCTANE Digital Video option contains a texture node for capturing video streams into the texture memory on OCTANE graphics. Once captured, a video field can be used as a texture just as if it were an image loaded into texture memory through function calls to the OpenGL graphics library. [3]
The texture map interface hardware has two inputs, link A and link B.
The texture-map interface hardware also has a scaler chip for extracting a rectangular subregion of the input video field and shrinking it either horizontally or vertically before it is transferred to TRAM.
Finally, the texture-map interface hardware also contains a real-time mipmap generator, which can create and transfer the levels of detail necessary to display the video field in mipmap mode on the graphics subsystem (see the OpenGL Programming Guide for a discussion of mipmapping).
Double-buffered video texture applications require 4 MB of TRAM, which is available with OCTANE graphics.
Figure 9-1 diagrams the hardware configuration that supports the texture node.
This chapter covers the following topics:
Initially, you use VL calls to create a path between a source node and the texture drain node. Typically, the source node is a video input node. The texture drain node supports both single-link and dual-link transfers.
For single-link transfers, use the VL_MGV_TEXTURE_INPUT_LINK control to specify the input link that supplies the data sent to TRAM. In addition, you can use the VL_MGV_TEXTURE_AUTOSWAP control to make the input link selection toggle automatically after each capture into TRAM. This feature is useful for applications that require two live video textures simultaneously.
For dual-link transfers, an RGBA stream is sent to TRAM. In this mode, the RGB data is extracted from link A and the alpha data is extracted from link B.
Once the path is set up, use the vlBeginTransfer() routine. At this point, data is being transferred from the video source to the texture drain. However, to capture this data into the TRAM on the OCTANE graphics board, you must explicitly tell OpenGL to do it; otherwise the data is simply lost.
The OpenGL function glCopyTexSubImage2DEXT() captures into TRAM the data that the VL is transferring. To specify to this function that a video transfer is occurring, the GLX read drawable must be set to a video input stream. The function glXCreateGLXVideoSource() creates a handle for a video input stream. This handle can then be passed as the read-drawable parameter to the function glXMakeCurrentReadSGI(). Once this handle is passed, each call to glCopyTexSubImage2DEXT() begins a capture into TRAM of the data that is being sent to the VL texture drain node. It is important to realize that this function begins a capture into TRAM and then returns to the caller. Thus, the caller can continue executing while the video field is being captured.
To use the simplest example, you create a loop that calls glCopyTexSubImage2DEXT(), and then draw with the captured video field. To display completely drawn fields, use double-buffered drawing. Even if the drawing routine is very fast, however, this implementation can capture only every other video field that is being sent to the texture node. Although the framebuffer used for drawing is double buffered, the texture buffer itself is not. Any attempt to draw with the single-buffered texture causes the OpenGL draw routine to wait for the video field to finish loading into TRAM. When this load finishes, the drawing begins. However, the video field being sent to the texture node while the drawing is occurring is not captured.
The solution to this problem is to double-buffer the TRAM loads. Then, while a video field is being captured into one of the texture buffers, the other one can be used for drawing. This overlapping of loading one video field into TRAM at the same time that another one is being used for drawing enables you to do real-time video texturing. You can use the OpenGL function glHint() to enable double-buffer texture loads on an OCTANE graphics system that has enough TRAM to support it.
 | Note: Double-buffered video texture applications require 4 MB of TRAM, which you can obtain by installing the Texture Upgrade to 4 MB of Texture Memory for OCTANE graphics. |
If you want your application to use the texture node's ability to do real-time mipmap generation, several issues should be considered. Before a texture is created, the OpenGL minifying filter must be set to a filter that enables mipmapping. You set it by calling the OpenGL function glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, filter) and passing one of the following values for filter:
GL_NEAREST_MIPMAP_NEAREST
GL_LINEAR_MIPMAP_NEAREST
GL_NEAREST_MIPMAP_LINEAR
GL_LINEAR_MIPMAP_LINEAR
When a texture is created using the OpenGL function glTexImage2D(), ten Level-Of-Details (LODs) must be defined (rather than just one for non-mipmap applications). These LODs must be sized so that LOD 0 is 512 horizontally by 256 vertically, and each subsequent LOD (except LOD 9) is half the size, both horizontally and vertically, of the preceding one. LOD 9 must be sized as 1 × 1.
Once the VL texture node control VL_MGV_TEXTURE_MIPMAP_MODE is set to VL_MGV_TEXTURE_MIPMAP_ON, each subsequent call to the OpenGL function glCopyTexSubImage2DEXT() generates all ten LODs for the current video field and loads them into TRAM.
To scale the entire video input field in mipmap mode, the zoom and aspect controls must be set correctly. Normally, you set the VL_ZOOM control to 1. For the VL_MGV_HASPECT control, the numerator would be set to 512 and the denominator set to the width of the active video field in pixels. For the VL_MGV_VASPECT control, the numerator would be set to 256 and the denominator set to the larger of 256 and the height of the active video field in lines.
For real-time video texturing on an OCTANE workstation with the OCTANE Digital Video option, these two subsystems must be frame-locked. If they are not, you can still perform video texturing, but real-time performance cannot be guaranteed.
To frame-lock the graphics and video hardware, use setmon. Before running a video texture application, enter one of these commands:
NTSC/525-line systems: /usr/gfx/setmon -Fe 1280x1024_59
PAL/625-line systems: /usr/gfx/setmon -Fe 1280x1024_49
The texture map interface hardware on the OCTANE Digital Video board contains a field buffer that introduces a one-field delay between the input source and the texture memory on the graphics board. This one-field delay is hidden by the fact that whenever you call the glCopyTexSubImage2DEXT() function to capture a video field, the actual capture does not occur until the beginning of the next video field. This delay occurs regardless of whether the call to glCopyTexSubImage2DEXT() occurs during the video vertical blanking interval or during the active video interval.
For example, the /usr/share/src/dmedia/video/impact/vidtotex.c program is a simple application of video texturing. It captures each video input field into TRAM, and draws a flat polygon in an X window with it. In this example, the following actions occur:
The call to glCopyTexSubImage2DEXT() occurs when video field n is being sent into the texture-map interface input select hardware (see Figure 9-1).
When the next video field, n + 1, is being sent to the texture map interface hardware, field number n is being transferred out of the field buffer and captured into TRAM on the graphics board, as diagrammed in Figure 9-2.
During the video field time for field n + 2, field n is being transferred out of TRAM and into the graphics framebuffer, as diagrammed in Figure 9-3.
During the video field time for field n + 3, field n is displayed on the screen, as diagrammed in Figure 9-4.
Thus, for this application, the call to glCopyTexSubImage2DEXT() results in the capture of the video field that is entering the texture map interface input select hardware at the time of the call. This field is displayed on the screen three video fields later.
This section describes the controls available on the VL texture drain node. Changing these controls is unrestricted; for example, they can be changed before or after vlBeginTransfer() has been called. However, they must match those that are set up through calls to OpenGL; otherwise, the call to glCopyTexSubImage2DEXT() returns an error.
VL_CAP_TYPE specifies the type of field to capture even fields, odd fields, all fields, and non-interleaved frames. Specifying even or odd fields causes each call to OpenGL to initiate a capture to block until the correct field type is present.
If you specify the non-interleaved frame capture type, use the field dominance control (see VL_MGV_DOMINANCE_FIELD, below) to determine the field of the frame to capture first.
VL_OFFSET specifies the upper left corner of a subregion of the active video region used to generate the texture. The x value is the horizontal offset from the left-hand edge in pixels, and the y value is the vertical offset from the top edge in lines. Along with VL_SIZE, this control can be used to extract any subregion of the input video field.
VL_OFFSET operates on the unzoomed (undecimated) image; it does not change if the zoom factor is changed.
The minimum horizontal offset is 0. The minimum vertical offset depends on the video standard:
NTSC/525-line: -13
PAL/625-line: -18 if VL_CAP_TYPE is VL_CAPTURE_ODD_FIELDS; otherwise -19
The maximum horizontal and vertical offsets are restricted: unzoomed size plus offset must not extend past the end of the active video region. The horizontal offset must be an even number.
VL_PACKING specifies the current packing format. For single-link transfers, this control can be set to either VL_PACKING_RGB_8 or VL_PACKING_RGBA_8. For dual-link transfers, this control is always set to VL_PACKING_RGBA_8.
VL_RATE is a read-only control that specifies the number of textures per second to capture into TRAM. Since each capture is initiated by a call to OpenGL, you should make at least the same number of calls per second as is specified by this control.
The VLTransferComplete event is generated at the end of a capture into TRAM. The VLSequenceLost event is generated each time a capture into TRAM is missed. However, these events are reported only if they are enabled via the vlSelectEvents() routine.
VL_SIZE specifies the amount of the image to send to texture RAM; that is, how much clipping takes place. The x value is interpreted as the subregion width in pixels, and the y value is interpreted as the subregion height in lines.
This control operates on the decimated image. For example, when the image is decimated to half size using VL_ZOOM, the limits of the VL_SIZE control change by a factor of 2.
Along with the VL_OFFSET control, VL_SIZE can be used to extract any subregion of the input video field. Controlling size is useful for cropping bad data at the edges of the video region.
The minimum width is 4 pixels; the minimum height is 3 lines. The maximum width is equal to the active line length multiplied by the product of the zoom and the horizontal aspect values. The maximum height is equal to the total field height minus 4, multiplied by the product of the zoom and the vertical aspect values. The width must be an even number.
maxwidth = active_line_length x (zoom x horiz_aspect) maxheight = (total_field_height - 4) x (zoom x vert_aspect) |
VL_ZOOM specifies the amount of scaling applied to the video subregion used as a texture source. Because only decimation is allowed on the texture node, the zoom value is always less than or equal to 1. For different zoom factors along the horizontal and vertical axes, use VL_MGV_HASPECT and VL_MGV_VASPECT controls. The zoom factor along each axis is equal to the product of the VL_ZOOM value and the aspect value for that axis.
VL_MGV_DOMINANCE_FIELD specifies the field dominance when VL_CAP_TYPE is set to non-interleaved frames. In that case, VL_MGV_DOMINANCE_FIELD specifies the field of the frame to be captured into TRAM first.
VL_MGV_HASPECT specifies the scaling along the horizontal axis that is applied to the video subregion used as a texture source. The overall scale factor horizontally is the product of this value and the VL_ZOOM value. This control enables you to scale the horizontal axis independently of the vertical axis, which is useful when mipmapping is enabled.
To scale the entire video input field horizontally when mipmapping is on, set the zoom to 1, and then set the numerator for VL_MGV_HASPECT to 512 and the denominator to the width of the video field in pixels.
VL_MGV_VASPECT specifies the scaling along the vertical axis that is applied to the video subregion used as a texture source. The overall scale factor vertically is the product of this value and the VL_ZOOM value. This control enables you to scale the vertical axis independently of the horizontal axis, which is useful when mipmapping is enabled.
To scale the entire video input field vertically when mipmapping is on, set the zoom to 1, and then set the numerator for VL_MGV_VASPECT to 256 and the denominator to the larger of 256 and the height of the video field in lines.
VL_MGV_TEXTURE_ROUND_MODE specifies the type of rounding used to convert from 10-bit to 8-bit input. You can either round
normally: VL_MGV_TEXTURE_ROUND_8BIT
with a pseudonumber generator: VL_MGV_TEXTURE_ROUND_RNG
with a pseudonumber generator that resets each field: VL_MGV_TEXTURE_ROUND_RNGFRM
The VL_MGV_TEXTURE_MIPMAP_MODE control turns mipmap mode off or on. When mipmap mode is on, the texture node generates mipmaps sized as follows:
512 × 256
256 × 128
128 × 64
64 × 32
32 × 16
16 × 8
8 × 4
4 × 2
2 × 1
1 × 1
VL_MGV_TEXTURE_INPUT_LINK specifies the input link, A or B, that sends pixel data to TRAM. This control is available only in single-link mode.
VL_MGV_TEXTURE_AUTOSWAP turns autoswap mode off or on. When autoswap mode is on, the input link toggles automatically after each capture into TRAM.
If VL_CAP_TYPE is set to non-interleaved frame, toggling occurs after frame captures. VL_MGV_TEXTURE_AUTOSWAP is useful for applications that require two live video textures simultaneously, and is available only in single-link mode.
This section explains how to use OpenGL functions for video texture mapping. For complete information on these functions, see the OpenGL Programming Guide.
glGenTexturesEXT(GLsizei count, GLuint *texnames)
This function generates texture names, which are unsigned integers, and puts count texture names in texnames. The generated texture names have no dimensionality, but assume that of the texture target to which they are first bound (see glBindTexureEXT(), below).
For applications that use more than one video texture at a time (for example, double-buffering the TRAM loads), use this function to get the names of textures so that they can later be passed to glBindTexturesEXT() in order to identify them.
glBindTextureEXT(GLenum target, GLuint texname)
This function makes it possible to use named textures besides the usual OpenGL texture targets. While a texture is bound, GL operations on the target to which it is bound affect the bound texture. For video texturing applications, target should always be GL_TEXTURE_2D.
glHint(GLenum target, GLenum mode)
You can control TRAM double buffering with hints (like other aspects of OpenGL behavior). To enable double buffering, call this function with target set to GL_TEXTURE_MULTI_BUFFER_HINT_SGIX and mode set to GL_FASTEST. To disable double buffering, use the same target, but set mode to GL_NICEST.
-
Use this function to enable texturing. Setting capability to GL_TEXTURE_2D enables two-dimensional texturing.
glXCreateGLXVideoSourceSGIX(Display *dpy, int screen, VLServer svr, VLPath path, int nodeClass, VLNode node)
This function creates a GLX handle for a video input stream that can be used as the read parameter on a call to glXMakeCurrentReadSGI(). Thereafter, the GL transfers video data into texture memory when glCopyTexSubImage2DEXT() is called.
If any control that affects the transfer of video data is changed on the video transfer path for which a particular GLX video source was created, destroy the GLX video source and create a new one. Otherwise, the data read from the source will be undefined. The configuration of a GLX video source is static, and is fixed when the GLX video source is created.
glXMakeCurrentReadSGI(Display *dpy, GLXDrawable draw, GLXDrawable read, GLXContext gc)
This function attaches a GLX context to separate read and write drawables. The handle returned by glXCreateGLXVideoSourceSGIX() can be passed as the read parameter. Thereafter, the GL transfers video data into texture memory when glCopyTexSubImage2DEXT() is called.
glTexParameteri(GLenum target, GLenum pname, GLint param)
This function assigns the value in param to the texture parameter specified by pname for the target given by target. To enable or disable mipmapping, set target to GL_TEXTURE_2D and pname to GL_TEXTURE_MIN_FILTER. To turn mipmapping off, set param to either GL_NEAREST or GL_LINEAR. To turn mipmapping on, set param to one of the following: GL_NEAREST_MIPMAP_NEAREST, GL_NEAREST_MIPMAP_LINEAR, GL_LINEAR_MIPMAP_NEAREST, or GL_LINEAR_MIPMAP_LINEAR. See the OpenGL Programming Guide for a description of these parameters.
glTexImage2D(GLenum target, GLint level, GLint components, GLsizei width, GLsizei height, GLint border, GLenum format, GLenum type, const GLvoid *pixels)
This function defines a texture image. The arguments describe the parameters of the texture image, such as height, width, level-of-detail number, and the internal resolution and format used to store the image.
The height and width must be powers of two. When mipmapping is not used, only level-of-detail number 0 need be defined; otherwise all ten LODs must be defined. For video textures, pixels can be set to NULL, because the texture data itself is loaded through calls to glCopyTexSubImage2DEXT().
glCopyTexSubImage2DEXT(GLenum target, GLint level, GLint xoffset, GLint yoffset, GLint x, GLint y, GLsizei width, GLsizei height)
This function replaces a rectangular portion of a two-dimensional texture image with pixels from the current GL_READ_BUFFER. The screen-aligned pixel rectangle with lower left corner at (x,y) having width width and height height replaces the portion of the texture array with x indices xoffset through xoffset + width - 1, inclusive, and y indices yoffset through yoffset + height - 1, inclusive, at the mipmap level specified by level.
However, for video texture loads, to load all 10 mipmap levels, enable mipmapping using glTexParameteri() and call this function with level set to 0. For video textures, this setting initiates a load into texture memory from the video port.
If any OpenGL texture parameters do not match the corresponding parameters set up through the Video Library, no load occurs and the GL error flag (see glGetError()), is set accordingly.
-
This function returns the value of the error flag. Each detectable error is assigned a numeric code and a symbolic constant. When an error occurs, the error flag is set to the appropriate error code value. No other errors are recorded until glGetError() is called, the error code is returned, and the flag is reset to GL_NO_ERROR.
For video texture loads, a mismatch between the OpenGL parameters and the Video Library parameters generates an error that can be detected with this function.
The example program /usr/share/src/dmedia/video/impact/vidtotex.c continuously captures a stream of live video from a video source and repeatedly loads it into an OpenGL texture map. Those texture maps are then used to texture a polygon in an X window on the graphics display.
The video source can be selected via a command-line option, or via the default input device control on vcp. The output of vlinfo shows valid video source numbers and their mappings.
[3] For a more detailed explanation of the OpenGL routines mentioned in this chapter, see the OpenGL Programming Guide.