Chapter 4. Using the Compression Library With OCTANE Compression

This chapter gives specific information for using the CL with OCTANE Compression. Besides the interfaces presented in Chapter 2, the CL includes JPEG-specific and board-specific CL parameters.

In this chapter:


Note: For information on tuning the JPEG algorithm, trading quality for compression ratio, and vice versa, see Chapter 7, “Using Compression Library Algorithms.”


Adding OCTANE Compression Support to an Application

To add OCTANE Compression support to your application, follow these steps:

  1. Include the dmedia/cl_impactcomp.h header in order to get definitions for OCTANE Compression:

    #include <dmedia/cl_impactcomp.h> 
    

  2. Set OCTANE Compression-specific compression parameters:

    • Set image formats as described in “Using OCTANE Compression Image Formats” on page 88.

    • Enable CL_ENABLE_IMAGEINFO as described in “Getting Compressed Image Information” on page 90.

  3. Query the CL to determine the appropriate scheme argument for clOpenCompressor() when opening a compressor or clOpenDecompressor() when opening a decompressor, as described in “Opening a Compression Session” in Chapter 2.

    Only two OCTANE Compression codecs are available concurrently. An error is returned if no OCTANE Compression codec is available.

  4. Compress or decompress frames.

Determining the JPEG Codec

OCTANE Compression has two independent JPEG codecs. When an application opens a compressor or decompressor, one of these codecs is allocated to the application.

The control CL_IMPACT_VIDEO_INPUT_CONTROL is used by the application to determine which codec was allocated; when CL_EXTERNAL_DEVICE is used, it specifies the CL_CODEC node to be used by the VL.

Values for this control are CL_IMPACT_VIDEO_CHANNEL0 and CL_IMPACT_VIDEO_CHANNEL1; they depend upon which codec was allocated. There is no default value for this parameter.

An application can query this parameter at any time, but can set it only before a call to a data-processing routine, such as clCompress() or clDecompress(). Since the codec channels are identical, it is not usually necessary to select a specific channel.

Although you can try to set the value of this parameter to the other possible value, success is not guaranteed; another application might have the other codec allocated. Example 4-1 shows use of this control.

Example 4-1. Capture Using CL_IMPACT_VIDEO_INPUT_CONTROL


channel = clGetParam(clHandle,CL_IMPACT_VIDEO_INPUT_CONTROL); 
...
vlGetNode(videoServer,VL_DRN,VL_CODEC,channel);

Controlling Compression and Decompression Operation

An application can control compression or decompression with the CL parameter CL_IMPACT_CODEC_CONTROL. The default value of this parameter is CL_IMPACT_START.

If this value of CL_IMPACT_CODEC_CONTROL is CL_IMPACT_START, the operation begins immediately when clCompress() or clDecompress() is called. If the value is CL_IMPACT_STOP, the CL configures and initializes the hardware necessary for the operation, but does not begin the operation until the value is set to CL_IMPACT_START. This feature allows more precise control over the time that the operation begins.

After a codec has begun operation, setting the parameter CL_IMPACT_CODEC_CONTROL to the value CL_IMPACT_STOP halts the compression or decompression operation. If clCompress() or clDecompress() was called with CL_CONTINUOUS_BLOCK, the function returns. If clCompress() or clDecompress() was called with CL_CONTINUOUS_NONBLOCK, the associated thread terminates.

Setting the value to CL_IMPACT_START on a codec that is already processing data has no effect, nor does setting the value to CL_IMPACT_STOP on a codec that is not processing data.

Using OCTANE Compression Image Formats

The Compression Library works with data that is contained in frames. A frame is defined as a sample in time so that:

width * height * components * bitsPerComponent/8 = n bytes

For video compression or decompression, images must be supplied as fields. Because the JPEG compression algorithm processes images in blocks of 16 × 8 pixels, OCTANE Compression requires that input images have a height that is a multiple of 8 pixels and a width that is a multiple of 16 pixels. The CL associates two sets of image dimensions with an instance of a video compressor or decompressor:

  • CL_IMAGE_WIDTH and CL_IMAGE_HEIGHT

  • CL_INTERNAL_IMAGE_WIDTH and CL_INTERNAL_IMAGE_HEIGHT

For compression operations, CL_IMAGE_WIDTH and CL_IMAGE_HEIGHT equal the original, uncompressed image size, and CL_INTERNAL_IMAGE_WIDTH and CL_INTERNAL_IMAGE_HEIGHT equal the final compressed image size.

For decompression operations, CL_IMAGE_WIDTH and CL_IMAGE_HEIGHT equal the final, uncompressed image size, and CL_INTERNAL_IMAGE_WIDTH and CL_INTERNAL_IMAGE_HEIGHT equal the original, compressed image size.

Table 4-1 summarizes the image format parameters.

Table 4-1. OCTANE Compression Image Format Parameters

Image Attribute

Description

Parameter

Values

Pixel format

The option supports 32-bit RGB or YCrCb 4:2:2 for memory-to-memory transfers and YCrCb 4:2:2 only for video-to-memory transfers.

CL_ORIGINAL_FORMAT

CL_RGBX

CL_YUV

Interlacing

The option operates on interlaced NTSC or PAL video data for video-to-memory compression and memory-to-video decompression. Even and odd fields are compressed as separate images.

DM_IMAGE_INTERLACING

NTSC or CCIR(525):
DM_IMAGE_INTERLACED_EVEN

PAL or CCIR(625):
DM_IMAGE_INTERLACED_ODD

Orientation

The option compresses/decompresses images that have top-to-bottom or bottom-to-top orientation.

CL_ORIENTATION

CL_TOP_DOWN

CL_BOTTOM_UP

DM_TOP_TO_BOTTOM
(for Silicon Graphics movies)

Dimensions in pixels

Compression operations: original, uncompressed image height.

Decompression operations: final, uncompressed image height

CL_IMAGE_HEIGHT

Range: 16–4088, in multiples of 8 (NTSC must use either 240 or 248)
Memory-to-memory decompression can be any size less than or equal to CL_INTERNAL_IMAGE_HEIGHT
Default: 248

 

Compression operations: original, uncompressed image height.

Decompression operations: final, uncompressed image width

CL_IMAGE_WIDTH

Range: 16-4080 in multiples of 16 Memory-to-memory decompression can be any size less than or equal to CL_INTERNAL_IMAGE_WIDTH
Default: 640

Dimensions in pixels

Compression operations: final, compressed image height).

Decompression operations: original, compressed image height.

CL_INTERNAL_IMAGE_HEIGHT

Range: 16–4088, in multiples of 8

Default:

 

Compression operations: final, compressed image width.

Decompression operations: original, compressed image width.

CL_INTERNAL_IMAGE_WIDTH

Range: 16-4080, in multiples of 16

Default: 320


Getting Compressed Image Information

The CL provides a function used exclusively by hardware-assisted JPEG operations that lets you get information such as the size, timestamp, and a relative image index value for images (fields or frames) as they are compressed or decompressed through OCTANE Compression. For compressing from external video, the timestamp returned represents the time at which the first line of the uncompressed field arrived at the OCTANE Compression board.


Note: If an application attempts to decompress data that is not valid JPEG data, the decompressor can hang.

To get compressed image information, follow these steps:

  1. Call clSetParam() to set the CL_ENABLE_IMAGEINFO parameter to TRUE before compressing or decompressing any frames.

  2. Call clGetNextImageInfo() to get a structure containing information about the compressed image:

    int clGetNextImageInfo(CL_Handle handle, CLimageInfo *info,
        int sizeofimageinfo)
    

    handle 

    specifies an open handle that is actively compressing or decompressing

    info 

    is a pointer where a CLimageInfo structure is to be placed

    sizeofimageinfo 

    specifies the size of the CLimageInfo structure in bytes

    The CLimageInfo structure is defined in dmedia/cl.h and has the following fields:

    typedef struct {
        unsigned size; /* size of compressed image in bytes */
        long long ustime;            /* time in nanoseconds */
        unsigned imagecount;       /*  media stream counter */
        unsigned status;   /* additional status information */
    } CLimageInfo;
    

    The ustime field returns a meaningful value only when compressing from or decompressing to an external device. The status field is reserved for future use.


Note: To get valid JPEG data, an application using the compressor must enable clGetNextImageInfo() by setting CL_ENABLE_IMAGEINFO, and then read a CLimageInfo structure corresponding to each compressed image, before calling clQueryValid to read the compressed image data.

For the decompressor, you do not need to read CLimageInfo structures. When clGetNextImageInfo() is called, the CL queries the hardware for information pertaining to the field that was most recently displayed on the video hardware. clGetNextImageInfo() blocks only when it is waiting for the first valid decompressed field to exit the decompressor.

Specifying Memory-to-Memory Compression and Decompression

You can use OCTANE Compression to compress images from a memory archive to a buffer. For example, you can use OCTANE Compression to compress images from a movie file to a buffer, and then insert the JPEG-compressed images into a movie file to create a compressed movie. Taking this idea a step further, you can then use OCTANE Compression to scale down the images as it decompresses them, in order to display thumbnail images similar to the ones in Movie Player.

Memory-to-Memory Compression

To compress frames into memory using OCTANE Compression:

  1. Open an OCTANE Compression compressor.

  2. Set the CL image parameters to characterize the input image data.

  3. Compress images into memory.

When compressing images from memory into a buffer, OCTANE Compression supports image widths of 16–4080 (in multiples of 16 pixels) and image heights of 16–4088 (in multiples of 8 pixels). Images may be scaled down to one half horizontally and/or one half vertically. Images may also have black padding regions added to the image prior to the scaling operation.

The CL parameters CL_IMAGE_WIDTH and CL_IMAGE_HEIGHT specify the original uncompressed image size, and the parameters CL_INTERNAL_IMAGE_WIDTH and CL_INTERNAL_IMAGE_HEIGHT specify the final compressed image size.

The uncompressed data format must be 32-bit RGB (CL_RGBX) or YUV 4:2:2 (CL_YUV), and the uncompressed image size cannot be larger than 4080 × 4088 pixels.

NTSC video frames have a height of 243 lines, but OCTANE Compression supports only input image heights that are multiples of 8. For NTSC, you must specify an image height of either 240 (causing the image to be cropped 3 lines from the bottom) or 248 (causing the image to be padded with 5 extra lines of black).

Example 4-2 demonstrates memory-to-memory compression of NTSC video.

Example 4-2. Memory-to-Memory Compression


#include <dmedia/cl.h>
...
    int pbuf[][2] = {
        CL_IMAGE_WIDTH,  0,
        CL_IMAGE_HEIGHT, 0,
        CL_COMPRESSED_BUFFER_SIZE, 0
    };
     ...
    scheme = clQuerySchemeFromName (CL_ALG_VIDEO, "impact");
    if (scheme < 0) {
	        fprintf(stderr, "compression scheme ;'impact' is"
                        " not configured\n");
        return;
    }
    clOpenCompressor (scheme, &handle);

    /* set parameters */
    pbuf[0][1] = 640;
    pbuf[1][1] = 240;
    clSetParams(handle, (int *)pbuf, 3);

    /* allocate the required size buffer */
    clGetParams(handle, (int *)pbuf, 6);
    compressedBuffer = malloc(pbuf[2][1]);

    for(i = 0; i < numberOfFrames; i++)
    {
        /* Get a frame from somewhere */
        ...
        clCompress(handle, 1, frameBuffer,
                   &compressedBufferSize, compressedBuffer);
        /* Write the compressed data to somewhere else. */
        ...
    }
    clCloseCompressor(handle);

After compressing the images, you can use mvInsertCompressedImage() to insert the compressed images into a movie file, as described in Chapter 5 of the Digital Media Programming Guide (007-1799-060 or later).

Memory-to-Memory Decompression

To decompress JPEG images from memory using OCTANE Compression, follow these steps:

  1. Open an OCTANE Compression decompressor.

  2. Set the CL image parameters to characterize the output image data.

  3. Decompress images into a buffer.

You can shrink the images as they are decompressed, which is useful for displaying thumbnail images. When decompressing images from memory into a buffer, OCTANE Compression supports image widths of 16 to 768 and image heights of 16 to 336.

Scaling can be arbitrary, that is, you can scale the image dimensions down by any amount, and the output image dimensions do not have to be multiples of 8. To shrink images as they are decompressed, make the uncompressed image dimensions (CL_IMAGE_WIDTH and CL_IMAGE_HEIGHT) less than the corresponding compressed image dimensions (CL_INTERNAL_IMAGE_WIDTH and CL_INTERNAL_IMAGE_HEIGHT).

For information on padding and scaling images on capture, see “Padding and Scaling” in Chapter 3.

Interleaving

OCTANE Compression supports interleaving fields as they are being decompressed to memory, and deinterleaving as the fields are compressed from memory. This functionality is useful, for example, for taking field-captured media such as that captured from a video source and converting it to a frame medium, such as for display on the graphics monitor.

The interleaving and deinterleaving capability is available only in the memory-to-memory modes of operation. In other modes, use VL controls to select interleaving; see “Using VL_CAP_TYPE and VL_RATE” in Chapter 3.

The CL parameters that control interleaving are CL_IMPACT_FRAME_INTERLEAVE and CL_IMPACT_INTERLEAVE_MODE:

  • The CL_IMPACT_FRAME_INTERLEAVE parameter's two possible values, TRUE and FALSE (the default), turn interleaving on and off.

  • The way that the fields are actually interleaved into memory is controlled by the CL_IMPACT_INTERLEAVE_MODE parameter.

When CL_IMPACT_FRAME_INTERLEAVE is TRUE, the CL_IMPACT_INTERLEAVE_MODE parameter specifies which of the two fields occupies the top line of the uncompressed region of memory:

  • CL_IMPACT_INTERLEAVE_EVEN specifies that the first field decompressed or compressed occupies the first (top) line of the uncompressed memory buffer. This value is appropriate for PAL and CCIR(625) captured media.

  • CL_IMPACT_INTERLEAVE_ODD (the default) specifies that the first field decompressed or compressed occupies the second line of the uncompressed memory buffer. This value is appropriate for NTSC and CCIR(525) captured media.


Note: The width and height of each field to be interleaved must be the same. During compression, the width has the same value as CL_IMAGE_WIDTH; during decompression, the width has the same value as CL_INTERNAL_IMAGE_WIDTH.


Compressing and Decompressing Video Through External Connections

You can use OCTANE Compression as a real-time JPEG codec between your application and the analog video ports on OCTANE Compression or the OCTANE Digital Video option.

Video-to-Memory Compression

To capture video from an external video device using OCTANE Compression, follow these steps:

  1. Connect the video device to the appropriate port. For example, use either analog port 1 or digital port 1. Video port connections are managed from the videopanel control panel.

  2. Open a compressor as described in Example 4-1.

  3. Query the CL to retrieve the appropriate VL_CODEC drain node identifier.

  4. Open a connection to the video server by calling vlOpenVideo("").

  5. Create the video transfer paths.

    • Get the source (VL_SRC) node for the video signal connection by calling vlGetNode().

    • Specify the drain node using the drain node identifier from step 3.

    • Create the path from source to drain by calling vlCreatePath().

    • Set up the path to share (VL_SHARE) data by calling vlSetupPaths().

  6. Set the CL parameters for image dimensions, quality factor, and compressed image information (CL_ENABLE_IMAGEINFO).

  7. Start the video transfer.

  8. Use the CL buffered interface to compress frames by calling clCompress() with CL_CONTINUOUS_NONBLOCK as the framecount parameter and CL_EXTERNAL_DEVICE as the frameBuffer parameter.

  9. Call clGetNextImageInfo() to get a structure containing information about the compressed image.


Note: Instead of using CL_CONTINUOUS_NONBLOCK, you can call clCompress() from a separate thread with the value CL_CONTINUOUS_NONBLOCK. In this case, clCompress() does not return until the transfer is complete.

See capture.c in /usr/share/src/dmedia/dmrecord/dmrecord.cosmo/ for an example of capturing external video through OCTANE Compression.

Video fields entering OCTANE Compression from the direct video connection are captured into an array of field buffers. The field buffers support field widths from 640 to 768 and field heights from 16 to 336. Field dimensions depend on the video timing, as shown in Table 4-2.

Table 4-2. OCTANE Compression Video Field Dimensions

Video Format

WIdth (Pixels)

Height (Pixels)

NTSC

640

243

PAL

768

288

CCIR(525)

720

243

CCIR(625)

720

288

When the compressed image's height is less than the height of the incoming video fields, the video fields are clipped from the bottom before they are sent to the compressor. When the compressed image's height is greater than the height of the incoming video fields, additional lines of black data are appended to the valid video data before the data is sent to the compressor.


Note: NTSC fields have a height (243 pixels) that is not a multiple of 8. For NTSC capture, you can choose to have your application either throw away 3 lines from the bottom of each field (giving a 240 pixel height) or append 5 extra blank lines to the bottom of each field (giving a 248 pixel height) before compression.

You can scale the captured image to half-size before compressing it. This allows for an additional increase in data compression by factor of 4.

Specify vertical decimation by setting the compressed image height (CL_INTERNAL_IMAGE_HEIGHT) to half the size of the uncompressed image height (CL_IMAGE_HEIGHT). Compressed image heights can range from 16 to 168, and uncompressed image heights can range from 32 to 336.

Specify horizontal decimation by setting the compressed image width (CL_INTERNAL_IMAGE_WIDTH) to half the size of the uncompressed image width (CL_IMAGE_WIDTH) as indicated in Table 4-3.


Note: When CCIR(525) or CCIR(625) images are decimated to one half, they are 360 pixels wide, which is not a multiple of 16 pixels. It is for this reason that one-half horizontal decimation is not available for these image sizes.


Table 4-3. OCTANE Compression Field Widths for Compression With Decimation

Video Format

CL_IMAGE_WIDTH (Pixels)

CL_INTERNAL_IMAGE_WIDTH (Pixels)

NTSC

640

320

PAL

768

384

CCIR(525)

720

360

CCIR(625)

720

360

During video compression from an external device, CLimageInfo.imagecount is initialized to 1 when the first field is received by the compressor after calling clCompress(). The count advances when a new field arrives. If the compression data buffer fills up, then a field will be dropped, but the imagecount continues to increase. An application can thus detect a dropped field by noticing a jump in the imagecount field of more than one. The ustime indicates the time the uncompressed field entered the compressor.

To select the fields to capture, the application can modify the video parameters associated with the VL_CODEC node. It is here that the application specifies the capture type (any fields, paired fields, odd or even fields only) and the rate at which they should be captured (30/60 fields per second, 10/30 frames per second). See “Setting Parameters for Data Transfer to or From Memory or Codec Nodes” in Chapter 3.

Memory-to-Video Decompression

The connections for decompressing from memory to an external video are set up similarly to those for capturing video, except that a decompressor is opened. See clInit.c in /usr/share/src/examples/dmedia/dmplay/dmplay.cosmo/ for example code that initializes the CL for JPEG decompression (optionally through OCTANE Compression) from memory to external video.

Video and audio playback of the decompressed frames require media synchronization. See dmplay.c and streamDecompress.c in /usr/share/src/examples/dmedia/dmplay for more information.

Uncompressed fields leaving the JPEG decompressor can optionally be scaled up by a factor of 2 in the horizontal and/or vertical dimensions. NTSC, PAL or CCIR(525)/CCIR(625) fields are then scanned out of the array of field buffers. Horizontal scaling is performed by pixel replication; vertical scaling is performed by line doubling.

If the uncompressed fields leaving the decompressor have fewer lines than the field height required by the NTSC/PAL or CCIR(525)/CCIR(625) connection (after optional pistoling), additional lines of black data are added at the bottom of the uncompressed images. If the uncompressed fields leaving the decompressor have more lines than the NTSC/PAL/CCIR(525)/CCIR(625) field height (after optional pistoling), lines are clipped from the bottom of the uncompressed images.

If the uncompressed image is too narrow (less than 640 pixels wide for square-pixel NTSC), the OCTANE Compression board adds extra (black) pixels to make the image the correct width. For example, if the image is 400 pixels wide, the OCTANE Compression board adds 240 black pixels.

Specify horizontal scaling by setting the uncompressed image width (CL_IMAGE_WIDTH) that is twice the compressed image width (CL_INTERNAL_IMAGE_WIDTH) as indicated in Table 4-4.

Table 4-4. OCTANE Compression Field Widths for Decompression

Video Format

CL_IMAGE_WIDTH (Pixels)

CL_INTERNAL_IMAGE_WIDTH (Pixels)

NTSC

640

320

PAL

768

384

CCIR(525)

720

Not available

CCIR(625)

720

Not Available

Specify vertical scaling by setting the uncompressed image height (CL_IMAGE_HEIGHT) to twice the size of the compressed image height (CL_INTERNAL_IMAGE_HEIGHT). Compressed image heights can range from 16 to 168, and uncompressed image heights can range from 32 to 336.

During video decompression to an external device, CLimageInfo.imagecount reflects the count of fields sent by the application to the decompressor. The ustime indicates the time that field left the decompressor. In certain situations, fields are repeated on output, in which case the imagecount remains the same, but the ustime increases.

Use the CL parameter CL_IMPACT_CODEC_CONTROL to control compression or decompression. When a codec is opened, this parameter is initialized with the value CL_IMPACT_START. If this value is CL_IMPACT_START when clCompress() or clDecompress() is called, the operation begins immediately. If the value is CL_IMPACT_STOP, the operation configures and initializes the hardware necessary for the operation, but does not begin the operation until the value is set to CL_IMPACT_START.

After a codec has begun operation, setting CL_IMPACT_CODEC_CONTROL to the value CL_IMPACT_STOP halts the compression or decompression operation. If clCompress() or clDecompress() was called with CL_CONTINUOUS_BLOCK, the function returns. If clCompress() or clDecompress() was called with CL_CONTINUOUS_NONBLOCK, the associated thread terminates.

Setting Interlacing for NTSC and PAL

For video-to-memory compression and memory-to-video decompression, use the control DM_IMAGE_INTERLACING to set interlacing for NTSC or PAL video data:

  • for NTSC or CCIR(525), use DM_IMAGE_INTERLACED_EVEN

  • for PAL or CCIR(625), use DM_IMAGE_INTERLACED_ODD

Even and odd fields are compressed as separate images.