Chapter 9. Introduction to FFIO

This chapter provides an overview of the capabilities of the flexible file input/output (FFIO) system, sometimes called the FFIO system or layered input/output (I/O). The FFIO system is used to perform many I/O-related tasks. For details about each individual I/O layer, see Chapter 13, “FFIO Layer Reference ”.

Layered I/O

The FFIO system is based on the concept that for all I/O a list of processing steps must be performed to transfer the user data between the user's memory and the desired I/O device. Computer manufacturers have always provided I/O options to users because I/O is often the slowest part of a computational process. In addition, it is extremely difficult to provide one I/O access method that works optimally in all situations.

The following figure depicts the typical flow of data from the user's variables to and from the I/O device.

Figure 9-1. Typical data flow

Typical data flow

It is useful to think of each of these boxes as a stopover for the data, and each transition between stopovers as a processing step.

Each transition has benefits and costs. Different applications might use the total I/O system in different ways. For example, if I/O requests are large, the library buffer is unnecessary because the buffer is used primarily to avoid making system calls for every small request. You can achieve better I/O throughput with large I/O requests by not using library buffering.

If library buffering is not used, I/O requests should be on 4096-byte block boundaries; otherwise, I/O performance will be degraded. On the other hand, if all I/O requests are very small, the library buffer is essential to avoid making a costly system call for each I/O request.

It is useful to be able to modify the I/O process to prevent intermediate steps (such as buffering of data) for existing programs without requiring that the source code be changed. The assign(1) command lets you modify the total user I/O path by establishing an I/O environment.

The FFIO system lets you specify each stopover in Figure 9-1. You can specify a comma-separated list of one or more processing steps by using the assign -F command:

assign -F spec1,spec2,spec3...

Each spec in the list is a processing step that requests one I/O layer, or logical grouping of layers. The layer specifies the operations that are performed on the data as it is passed between the user and the I/O device. A layer refers to the specific type of processing being done. In some cases, the name corresponds directly to the name of one layer. In other cases, however, specifying one layer invokes the routines used to pass the data through multiple layers. See the INTRO_FFIO (3f) man page for details about using the -F option to the assign command.

Processing steps are ordered as if the -F side (the left side) is the user and the system/device is the right side, as in the following example:

assign -F user,bufa,system

With this specification, a WRITE operation first performs the user operation on the data, then performs the bufa operation, and then sends the data to the system. In a READ operation, the process is performed from right to left. The data moves from the system to the user. The layers closest to the user are higher-level layers; those closer to the system are lower-level layers.

The FFIO system has an internal model of the world of data, which it maps to any given actual logical file type. Four of these concepts are basic to understanding the inner workings of the layers.

Concept 

Definition

Data 

Data is a stream of bits.

Record marks 

End-of-record marks (EOR) are boundaries between logical records.

File marks 

End-of-file marks (EOF) are special types of record marks that exist in some file formats.

End-of-data (EOD) 

An end-of-data (EOD) is a point immediately beyond the last data bit, EOR, or EOF in the file.

All files are streams of 0 or more bits that may contain record or file marks.

Individual layers have varying rules about which of these things can appear and in which order they can appear in a file.

Fortran users can use the assign(1) command to specify these FFIO options. For C users, the FFIO layers are available only to programs that call the FFIO routines directly ( ffopen(3c), ffread(3c), and ffwrite(3c)).

You can use FFIO with the following Fortran I/O forms:

  • Buffer I/O

  • Unformatted sequential

  • Unformatted direct access

  • Formatted sequential

  • Namelist

  • List-directed

Using Layered I/O

The specification list on the assign -F command comprises all of the processing steps that the I/O system performs. If assign -F is specified, any default processing is overridden. The FFIO system provides detailed control over I/O processing requests. However, to effectively use any FFIO option, you must understand the I/O processing details.

As a very simple example, suppose you were making large I/O requests and did not require buffering or blocking on your data. You could specify the following:

assign -F system

The system layer is a generic system interface that chooses an appropriate layer for your file. If the file is on disk, it chooses the syscall layer, which maps each user I/O request directly to the corresponding system call. A Fortran READ statement is mapped to one or more read(2) system calls and a Fortran WRITE statement to one or more write(2) system calls. This results in almost the same processing as would be done if the assign -s u command was used.

If you want your file to be COS blocked, you can specify the following:

assign -F cos,system

If you want your file to be F77 blocked, you can specify the following:

assign -F f77,system

These two specs request that each WRITE request first be blocked (blocking adds control words to the data in the file to delimit records). The cos layer then sends the blocked data to the system layer. The system layer passes the data to the device.

The process is reversed for READ requests. The system layer retrieves blocked data from the file. The blocked data is passed to the next higher layer, the cos layer, where it is deblocked. The deblocked data is then presented to the user.

I/O Layers

Several different layers are available for the spec argument. Each layer invokes one or more layers, which then handles the data it is given in an appropriate manner. For example, the syscall layer essentially passes each request to an appropriate system call.

The following tables list the classes you can specify for the spec argument to the assign -F option:

Available I/O Layers

Layer

Function

bufa

Asynchronous buffering layer

cache

Memory cached I/O

cachea

Asynchronous memory cached I/O

cos or blocked

COS blocking

fd

File descriptor open

f77

Record blocking common to most UNIX Fortran implementations

global

Distributed cache layer

null

Syntactic convenience for users (does nothing)

site

Site-specific layer

syscall

System call I/O

system

Generic system interface

text

Newline separated record formats

tmf

IRIX tape management facility

user

User-written layer

Layered I/O Options

You can modify the behavior of each I/O layer. The following spec format shows how you can specify a class and one or more opt and num fields:

class.opt1.opt2:num1:num2:num3

For class, you can specify one of the layers listed in the previous tables. Each of the layers has a different set of options and numeric parameter fields that can be specified. This is necessary because each layer performs different duties. The following rules apply to the spec argument:

  • The class and opt fields are case-insensitive. For example, the following two specs are identical:

    Ibm.VBs:100:200

    IBM.vbS:100:200

  • The opt and num fields are usually optional, but sufficient separators must be specified as placeholders to eliminate ambiguity. For example, the following spec s are identical:

    cos..::40, cos.::40
    cos::40

    In this example, opt1, opt2 , num1, and num2 can assume default values. Similarly, the sds layer also allows optional opt and num fields and it sets opt1, opt2, num1, num2, and num3 to default values as required.

  • To specify more than one spec, use commas between specs. Within each spec, you can specify more than one opt and num. Use periods between opt fields, and use colons between num fields.