This chapter provides an overview of the capabilities of the flexible file input/output (FFIO) system, sometimes called the FFIO system or layered input/output (I/O). The FFIO system is used to perform many I/O-related tasks. For details about each individual I/O layer, see Chapter 14, “FFIO Layer Reference ”.
The FFIO system is based on the concept that for all I/O a list of processing steps must be performed to transfer the user data between the user's memory and the desired I/O device. Computer manufacturers have always provided I/O options to users because I/O is often the slowest part of a computational process. In addition, it is extremely difficult to provide one I/O access method that works optimally in all situations.
The following figure depicts the typical flow of data from the user's variables to and from the I/O device.
It is useful to think of each of these boxes as a stopover for the data, and each transition between stopovers as a processing step.
Each transition has benefits and costs. Different applications might use the total I/O system in different ways. For example, if I/O requests are large, the library buffer is unnecessary because the buffer is used primarily to avoid making system calls for every small request. You can achieve better I/O throughput with large I/O requests by not using library buffering.
If library buffering is not used, I/O requests should be on sector boundaries; otherwise, I/O performance will be degraded. On the other hand, if all I/O requests are very small, the library buffer is essential to avoid making a costly system call for each I/O request.
It is useful to be able to modify the I/O process to prevent intermediate steps (such as buffering of data) for existing programs without requiring that the source code be changed. The assign(1) command lets you modify the total user I/O path by establishing an I/O environment.
The FFIO system lets you specify each stopover in Figure 10-1. You can specify a comma-separated list of one or more processing steps by using the assign -F command:
assign -F spec1,spec2,spec3... |
Each spec in the list is a processing step that requests one I/O layer, or logical grouping of layers. The layer specifies the operations that are performed on the data as it is passed between the user and the I/O device. A layer refers to the specific type of processing being done. In some cases, the name corresponds directly to the name of one layer. In other cases, however, specifying one layer invokes the routines used to pass the data through multiple layers. See the INTRO_FFIO(3f) man page for details about using the -F option to the assign command.
Processing steps are ordered as if the -F side (the left side) is the user and the system/device is the right side, as in the following example:
assign -F user,blankx,system |
With this specification, a WRITE operation first performs the user operation on the data, then performs the blankx operation, and then sends the data to the system. In a READ operation, the process is performed from right to left. The data moves from the system to the user. The layers closest to the user are higher-level layers; those closer to the system are lower-level layers.
The FFIO system has an internal model of the world of data, which it maps to any given actual logical file type. Four of these concepts are basic to understanding the inner workings of the layers.
| Concept | Definition | |
| Data | Data is a stream of bits. | |
| Record marks | End-of-record marks (EOR) are boundaries between logical records. | |
| File marks | End-of-file marks (EOF) are special types of record marks that exist in some file formats. | |
| End-of-data (EOD) | An end-of-data (EOD) is a point immediately beyond the last data bit, EOR, or EOF in the file. |
All files are streams of 0 or more bits that may contain record or file marks.
Individual layers have varying rules about which of these things can appear and in which order they can appear in a file.
Fortran programmers and C programmers can use the capabilities described in this document. Fortran users can use the assign(1) command to specify these FFIO options. For C users, the FFIO layers are available only to programs that call the FFIO routines directly (ffopen(3c), ffread(3c), and ffwrite(3c)).
You can use FFIO with the following Fortran I/O forms:
Buffer I/O
Unformatted sequential
Unformatted direct access
Word addressable
Mass Storage (MS) and Direct Random (DR) packages
Formatted sequential
Namelist
List-directed
Asynchronous queued I/O (AQIO)
The MS package and the DR package includes the OPENMS, WRITMS, READMS, FINDMS, CHECKMS, WAITMS, ASYNCMS, SYNCMS, STINDX, CLOSMS, OPENDR, WRITDR, READDR, and CLOSDR library routines.
The specification list on the assign -F command comprises all of the processing steps that the I/O system performs. If assign -F is specified, any default processing is overridden. For example, unformatted sequential I/O is assigned a default structure of cos on UNICOS systems and UNICOS/mk systems. The -F cos option provides the same structure. The FFIO system provides detailed control over I/O processing requests. However, to effectively use the cos option (or any FFIO option), you must understand the I/O processing details.
As a very simple example, suppose you were making large I/O requests and did not require buffering or blocking on your data. You could specify the following:
assign -F system |
The system layer is a generic system interface that chooses an appropriate layer for your file. If the file is on disk, it chooses the syscall layer, which maps each user I/O request directly to the corresponding system call. A Fortran READ statement is mapped to one or more read(2) system calls and a Fortran WRITE statement to one or more write(2) system calls. This results in almost the same processing as would be done if the assign -s u command was used.
If you want your file to be COS blocked (the default blocking for Fortran unformatted I/O on UNICOS and UNICOS/mk systems), you can specify the following:
assign -F cos,system |
If you want your file to be F77 blocked (the default blocking for Fortran unformatted I/O on IRIX systems), you can specify the following:
assign -F f77,system |
These two specs request that each WRITE request first be blocked (blocking adds control words to the data in the file to delimit records). The cos layer then sends the blocked data to the system layer. The system layer passes the data to the device.
The process is reversed for READ requests. The system layer retrieves blocked data from the file. The blocked data is passed to the next higher layer, the cos layer, where it is deblocked. The deblocked data is then presented to the user.
A COS blocked blank-compressed file can also be read. The following are the processing steps necessary to do this:
Issue system calls to read data from the device.
Deblock the data and deliver blank-compressed characters.
Decompress the characters and deliver them to the user.
In this case, the spec with system is on the right end and would be as follows:
-F blankx,cos,system |
You do not need to specify the systemspec because it is always implied on the right end. To read the COS blocked blank-compressed file, use the following specification:
assign -F blankx,cos |
Because the systemspec is assumed, it is never required.
Several different layers are available for the spec argument. Each layer invokes one or more layers, which then handles the data it is given in an appropriate manner. For example, the syscall layer essentially passes each request to an appropriate system call. The tape layer uses an array of more sophisticated system calls to handle magnetic tape I/O. The blankx layer passes all data requests to the next lower layer, but it transforms the data before it is passed. The mr layer tries to hold an entire file in a buffer that can change size as the size of the file changes; it also limits actual I/O to lower layers so that I/O occurs only at open, close, and overflow.
The following tables list the classes you can specify for the spec argument to the assign -F option:
I/O Layers available on all hardware platforms
| Layer | Function |
| bufa | Asynchronous buffering layer |
| cache | Memory cached I/O |
| cachea | Asynchronous memory cached I/O |
| cos or blocked | COS blocking |
| fd | File descriptor open |
| f77 | Record blocking common to most UNIX Fortran implementations |
| global | Distributed cache layer |
| null | Syntactic convenience for users (does nothing) |
| site | Site-specific layer |
| syscall | System call I/O |
| system | Generic system interface |
| text | Newline separated record formats |
| user | User-written layer |
Deferred implementation for IRIX systems
| Layer | Function |
| event | Monitors I/O layers |
| ibm | IBM file formats |
| mr | Memory-resident file handlers |
| tape or bmx | UNICOS online tape handling |
| vms | VAX/VMS file formats |
|
You can modify the behavior of each I/O layer. The following spec format shows how you can specify a class and one or more opt and num fields:
class.opt1.opt2:num1:num2:num3 |
For class, you can specify one of the layers listed in the previous tables. Each of the layers has a different set of options and numeric parameter fields that can be specified. This is necessary because each layer performs different duties. The following rules apply to the spec argument:
The class and opt fields are case-insensitive. For example, the following two specs are identical:
Ibm.VBs:100:200
IBM.vbS:100:200
The opt and num fields are usually optional, but sufficient separators must be specified as placeholders to eliminate ambiguity. For example, the following spec s are identical:
cos..::40, cos.::40 cos::40
In this example, opt1, opt2, num1, and num2 can assume default values. Similarly, the sds layer also allows optional opt and num fields and it sets opt1, opt2, num1, num2, and num3 to default values as required.
To specify more than one spec, use commas between specs. Within each spec, you can specify more than one opt and num. Use periods between opt fields, and use colons between num fields.
The following options all have the same effect. They all specify the sds layer on UNICOS systems and set the initial SDS allocation to 100 512-word sectors:
-F sds:100 -F sds.:100 -F sds..:100 |
The following option contains one spec for an sds layer that has an opt field of scr (which requests scratch file behavior):
-F sds.scr |
The following option requests two class es with no opt s:
-F cos,sds |
The following option contains two specs and requests two layers: cos and sds. The cos layer has no options; the sds layer has options scr and ovfl, which specify that the file is a scratch file that is allowed to overflow, and that the maximum SDS allocation is 1000 sectors:
-F cos,sds.scr.ovfl::1000 |
When possible, the default settings of the layers are set so that optional fields are seldom needed.
The UNICOS operating system supports a number of library parameters that can be tuned. Sites can use these parameters to change both the performance of the libraries and some of their limits. Through a similar technique, users can also change these parameters when linking an application.
When SEGLDR is invoked, one of its first actions is to read the /lib/segdirs file, which defines the parameters of SEGLDR; this file contains an LINCLUDE directive for the file /usr/lib/segdirs/def_lib, which by default is empty. An administrator can place directives in this file to modify the SEGLDR behavior.
The following HARDREF directives select optional capabilities of the FFIO package to include in the standard libraries compiled into user programs by default.
HARDREF Directives
| HARDREF = | FFIO option | |
| _f_ffvect | F-type records, fixed length | |
| _v_ffvect | V-type records, variable length | |
| _x_ffvect | X-type records | |
| _cos_ffvect | COS-type records, COS blocking | |
| _tape_ffvect | Magnetic tape handlers | |
| _cdc_ffvect | CDC 60-bit record handlers | |
| _sds_ffvect | SDS-resident file handlers | |
| _mr_ffvect | Memory-resident file handlers | |
| _trc_ffvect | Trace layer | |
| _txt_ffvect | Text-type records, newline separated records | |
| _fd_ffvect | Specified file descriptor | |
| _blx_ffvect | Blank compression handlers | |
| _cch_ffvect | Cache layer |
Each of these directives refers to a list of function pointers. Each function-pointer list represents the set of routines necessary to process one or more options on the assign(1) and/or asgcmd(1) commands. Some of these layers are tied to specific hardware, such as tape or SDS. Others are foreign conversion options such as ETA System V-format data. Not all of these layers are loaded into user programs by default. As delivered, the UNICOS operating system can read and write data in many different ways, however, only a subset of these capabilities is loaded into user programs by default, so that user executables are smaller.
If UNICOS source code is available, it is better to change the switches in fdcconfig.h, rather than to use these HARDREF directives, primarily because assign and asgcmd still issue warnings to users who use layers disabled in fdcconfig.h. Also, changing fdcconfig.h is the only way to disable layers that are shipped enabled by default.