After a kernel-level driver has been designed and coded, it must be compiled, linked, and installed. The topics in this chapter describe the major steps of this process, as follows:
“Defining Device Numbers” covers the choice of major and minor device numbers.
“Defining Device Special Files” describes options for creating the file or files controlled by the driver.
“Compiling and Linking” covers the compiler and linker options used for driver modules.
“Configuring a Nonloadable Driver” describes the configuration files used to set up a driver loaded at boot time.
“Configuring a Loadable Driver” describes the additional configuration needed for a loadable driver.
The topics “Major Device Number” in Chapter 2 and “Minor Device Number” in Chapter 2 cover the purpose and use of the device numbers. Major and minor numbers were once very important in the device driver design because they were the primary input that distinguished a device to a device driver upper-half entry point. In current IRIX, this is only the case for legacy drivers in older machines. Contemporary drivers take their input from a vertex of the hwgraph (see “Hardware Graph” in Chapter 2).
The historical use of device numbers can be summarized as follows:
Both numbers are encoded in the inode of a device special file in /dev.
The major number selects the device driver.
The minor number specifies the logical unit, and can encode device features.
Both numbers are passed as a parameter to driver entry points.
Part of creating and installing a device driver is the selection of device numbers and the definition of device special files.
If your driver does not use the hwgraph, you must select a major number to stand for your driver. The numbers that already exist are listed in sys/major.h. However, the major number should not be coded into the driver. Typically the driver code does not need to know its major number, and if it does, the driver should discover its major number dynamically. A method of doing this is discussed under “Variables Section”.
A driver is associated with its major number in the master.d configuration file. When the driver discovers this number dynamically, the system administrator is free to change major numbers in /var/sysgen/master.d files to correct conflicts between one product and another.
When a driver is called to service a device special file defined only in /dev, it receives a device minor number comprising 18 bits of information. You design the content of these numbers to give your driver the information it needs about each device special file. Typically you encode a unique device unit number so the driver can index device information from an array. (When the hwgraph is used, a pointer to the device information is stored in the hwgraph vertex instead.)
Examine the /dev/MAKEDEV script to see some techniques for assembling minor numbers dynamically based on the hardware inventory and other commands.
As described under “Device Special Files” in Chapter 2, the association between a device and a driver is established when a process opens a device special file in the /hw or /dev directory. Without at least one device special file, a device can never be opened.
The system administrator can create device special files using mknod or install (see “Making Conventional Device Files” in Chapter 2). This can be done manually, or through an installation script, or as an exit operation of the software manager program. The device special files can be created at any time—whether or not the device driver is installed, and whether or not the hardware exists. The special device files continue to exist until the administrator removes them.
A more sophisticated approach is to have the device special files created, or recreated, dynamically each time the system boots. This was the purpose for which /dev/MAKEDEV (see “The Script MAKEDEV” in Chapter 2) was introduced—it removes and redefines device special files based on information in the hardware inventory. In current IRIX, all entries in the /hw filesystem are created dynamically by device drivers as devices are attached.
The kernel creates the upper levels of the hardware graph to represent addressable units of hardware in the basic system—modules, buses, and slots. While probing buses, it finds devices, and calls upon device drivers to attach them (see “Entry Point attach()” in Chapter 7 and “Entry Point edtinit()” in Chapter 7). At these times, the driver has the responsibility of extending the hwgraph with vertexes that provide access to the device (see “Extending the hwgraph” in Chapter 8).
Because hwgraph entries are always created dynamically, and can be created and destroyed while the system is running, the initial set of pathnames in /hw are not stable and should not be written into user scripts and source code. Your driver can create additional vertexes in the hwgraph (see “Extending the hwgraph” in Chapter 8), both when attaching a device and later, when ioconfig runs (see “Using ioconfig for Global Controller Numbers” in Chapter 2).
You compile a kernel device driver to an ELF binary using shared libraries. The compile options differ between 32-bit and 64-bit modules.
If you are building a device driver that you wish to use on multiple platforms, you should build a different driver for each CPU board type (for example, IP22) that you want to run it on. You can use the hinv command to determine the host architecture (see hinv (1M)) and then specify the board type in the Makefile as described in the next section.
The file /var/sysgen/Makefile.kernio is a sample Makefile for compiling kernel modules. You can include it from a Makefile of your own to set the definitions of compiler variables and compiler options appropriately for different CPUs and module types.
The Makefile.kernio file tests the following environment variables, which you set:
CPUBOARD | Set to the type of CPU used in the target system, for example IP19, IP22, IP27 (see the sys/cpu.h header file). |
COMPILATION_MODEL | Set to 64 for a 64-bit kernel module, or to 32 for a 32-bit kernel module. |
The purpose of the rules in Makefile.kernio is to set numerous compiler variables appropriately for the CPU type and execution model. It also sets compiler options into a Make variable CFLAGS. Owing to the number of compiler variables and the importance of getting them right for each CPU type, Silicon Graphics strongly recommends that you invoke Makefile.kernio from your own makefile.
 | Note: Makefile.kernio is designed for nonloadable drivers. In particular it sets the compiler option -G8, which is valid for nonloadable drivers. For loadable drivers, use the file /var/sysgen/Makefile.kernloadio as a sample Makefile. This sets the -G0 flag and other options appropriate for loadable drivers. |
The compiler variables listed in Table 9-1 are tested in system header files. They are usually defined on the compiler command line. The rules in Makefile.kernio set definitions of these variables appropriately for different CPU types.
Table 9-1. Compiler Variables Tested by System Header Files
Variable | Meaning |
|---|---|
_KERNEL | Compile for a kernel module, not a user program. |
MP | Compile for a multiprocessor. |
_MP_NETLOCKS | Compile network driver (only) for multiprocessor TCP/IP. |
STATIC=static | Use of pseudo-declarator STATIC is converted to real static. |
_PAGESZ=16384 | Compile for a kernel using 16K memory pages. |
_PAGESZ=4096 | Compile for a kernel using 4K memory pages. |
_MIPS3_ADDRSPACE | Kernel for a MIPS3 machine. |
R10000 | Target machine is the R10000. |
TFP | Target machine is the R8000. |
R4000 | Target machine is the R4000. |
IPnn | Target machine uses the IPnn CPU module, one of IP19, IP20, IP21, IP22, IP25, IP26, IP27, IP28, IP30, and IP35 are currently supported. |
EVEREST | Compile for a Challenge or Onyx system. |
BADVA_WAR, JUMP_WAR, PROBE_WAR | Compile workaround code for bugs in certain R4x00 revisions. |
_IP26_SYNC_WAR, _NO_UNCCHED_MEM_WAR | Compile workaround code for IP26 bugs. |
R10000_SPECULATION_WAR | Compile workaround code for bug in certain R10000 revisions. |
USE_PCI_PIO | Compile workaround for IP32 PIO bug (see sys/PCI/pciio.h). |
Some of the cc and ld options needed to compile and link a kernel-level driver are shown in Table 9-2. The complete and most current set is defined in Makefile.kernio.
Table 9-2. Compiler Options Kernel Modules
Option | Purpose |
|---|---|
-non_shared | Do not compile for shared libraries (no dynamic linking). |
-elf | Compile and link an ELF binary. |
-64 | Set for any kernel using the 64-bit execution model. 32-bit kernel does not set any specific flag. |
-mips4 , -mips2 | Select the MIPS4 instruction set only for the R10000 CPU. Use MIPS2 for others. |
-G 8 | In a nonloadable driver, use the global table for objects up to 8 bytes. |
-G 0 | In a loadable driver, do not use the global table. Refer to the gp_overflow(5) reference page for a discussion of the global table. |
-r | Linker to retain symbols—for all drivers (required by loadable drivers, and needed for lboot). |
-d | Force definition of common storage even though -r used. |
-Wc,-pic0 | Do not allocate stack space used by shared objects. |
-jalr | In loadable drivers only, use jalr (jump-and-link register) instead of jal, whose 26-bit operand may not be enough for subroutine calls from a loaded module to the kernel. |
-TARG:t5_no_spec_stores | Crucial setting for Indigo2 R10000 only; without it, kernel memory corruption can occur. |
-TENV:kernel -TENV:misalignment=1 | Execution environment options for 64-bit compiler. |
-OPT:space -OPT:quad_align_branch_targets=F ALSE -OPT:quad_align_with_memops=FALS E -OPT:unroll_times=0 | Specific optimization constraints for 64-bit compiler. |
When the driver is not loadable, it is linked as part of the IRIX kernel. The following steps are needed to make the driver usable:
Some of these steps are elaborated in the following topics.
The process of naming a kernel-level driver begins in a file in /var/sysgen/system, such as /var/sysgen/system/irix.sm. Names are used as follows:
A USE, INCLUDE, or VECTOR statement in /var/sysgen/system specifies a name, for example
USE hypothetical
This statement directs lboot to read a file of the same name in /var/sysgen/master.d. In this example, the file would be /var/sysgen/master.d/hypothetical.
The file in /var/sysgen/master.d specifies the prefix for driver entry points, for example hypo_.
The same name with the suffix .o, is searched for in /var/sysgen/boot—in this example, /var/sysgen/boot/hypothetical.o. This object file is linked with the kernel.
The public symbols in the object file are searched for names that start with the prefix, for example hypo_attach(). These are noted in the kernel switch table so the driver can be called as needed.
The /var/sysgen/boot directory, where the kernel object modules reside, is actually a symbolic link to one of the directories /usr/cpu/sysgen/IPnnboot, where nn is the number of one of the CPU modules supported by the current release of IRIX (see “CPU Modules” in Chapter 1). When you place the object file of a driver in /var/sysgen/boot, you actually place it in the CPU directory for the system in use.
You describe your driver in a file with the name of the driver in /var/sysgen/master.d. The format of these files is described in two places: the master(4) reference page, and in /var/sysgen/master.d/README. In addition, you can examine the many examples in the distributed system.
The first noncomment line of the master file contains a series of fields, delimited by white space, to describe the driver. These fields are listed in Table 9-3.
Table 9-3. Fields of Descriptive Line in Master File
Field Number | Usage | Details |
|---|---|---|
1 | Flags | See Table 9-4 . |
2 | Prefix | The string of 1-14 characters that identify the public symbols of driver entry points. |
3 | Major number | The major device number found in device special files managed by this driver. When the driver uses the hwgraph, this field contains only a hyphen (-). |
4 | Number of sub-devices | Size of the driver's static arrays of device information, or given as a hyphen “-” when the driver stores device information in the hwgraph. |
5 | Dependencies | A list of other modules that must be in the kernel for this driver to work—usually omitted except for SCSI drivers. |
The important flag values for nonloadable drivers are listed in Table 9-4.
Table 9-4. Flag Values for Nonloadable Drivers
Letter | Meaning |
|---|---|
b or c | Block (b) or character (c) device. One or the other is essential for any device driver. |
f or m | STREAMS driver (f) or STREAMS module (m). Omit for device driver. |
s | Software driver, either a pseudo-device or a SCSI driver. |
The s (software-only) flag tells lboot not to attempt to probe for hardware. This is the case with software-only (pseudo-device) drivers, and with SCSI drivers. If lboot tries to probe for a SCSI device, it fails, and assumes that the device is not present, and does not include your SCSI device driver.
Additional flags (d, r, D, N, R) for loadable drivers are discussed later in the section “Configuring a Loadable Driver”.
The descriptive line ends with a comma-separated list of other loadable kernel modules on which this driver depends. The lboot command makes sure that it will not load this module if it fails to load a dependency.
In most cases, an OEM driver does not have any dependencies. However, a SCSI driver (see Chapter 16, “SCSI Device Drivers”) should list the name scsi, since it depends on the inner SCSI driver. A STREAMS driver might list the name of a STREAMS support module such as clone (see “Support for CLONE Drivers” in Chapter 22).
It is possible for you to design a driver in the form of multiple, loadable modules. In that case, you would name your support modules in this field.
Noncomment lines that follow the descriptive line and precede a line beginning “$” are used by library modules—not by device drivers or STREAMS drivers. Each such line specifies an entry point that this module provides, and which is used by the kernel or some other loadable module. These lines instruct lboot in how to create a harmless “stub” function in the event that this driver is not included in the kernel—for example, because it is specified by an EXCLUDE line in the system file. The format is discussed in the master(4) reference page.
Since a device or STREAMS driver provides only standard entry points that are accessed via the switch tables rather than by linking, drivers do not need to define any stubs.
Following the descriptive line (and the stubs section, if any), you can place a line that begins with “$” and, following this, you can write definitions of global variables using C syntax. This text is compiled into a module linked with the kernel. You refer to these variables as extern in the driver source code.
The advantage of defining global variables in the master file is that the initializing expressions for these variables can include values taken from the descriptive line. The following special symbols can be used:
##E | The integer coded as the major number in the descriptive line. The first integer, if a list of major numbers is given. |
##C | The number of controllers (bus adapters) of this type. |
##D | The number of sub-devices as coded in the fourth field of the descriptive line. |
You can use these symbols to compile run-time values for the major device number and the number of supported sub-devices, as specified in the descriptive line of the file, without coding them as constants in the driver. In the source code you can write
extern major_t myMajNum; extern int myDevLimit; |
In the master file you can implement the variables using the code in Example 9-1.
Example 9-1. Defining Variables in Master Descriptive File
$$$ major_t myMajNum = ##E; int myDevLimit = ##C; |
(In a loadable driver this technique requires one additional step; see “Master File for Loadable Drivers”.)
Once you have placed the binary in /var/sysgen/boot and the master file in /var/sysgen/master.d, you can configure and generate a new kernel. This is done using the autoconfig command, which in turn calls lboot to actually create a new bootable file.
The lboot program only loads modules that are specified in a file in /var/sysgen/system. One command is required to specify the new driver; the command is one of the following:
VECTOR | To specify hardware details, to request a hardware probe at boot time, to load the driver and invoke pfxedtinit(). |
INCLUDE | To load the driver and invoke pfxinit(). |
USE | To load the driver and invoke pfxinit() only if the master file exists in master.d. |
The form of these commands is detailed in the system(4) reference page. In addition, you should examine the distributed files in /var/sysgen/system, especially irix.sm, which contains many comments on the meaning and use of different entries. Specific uses of the VECTOR statement are discussed in the following topics: The form of VECTOR lines for VME devices is discussed under “Configuring VME Devices” in Chapter 12.
You could place the VECTOR, USE, or INCLUDE line for your driver in irix.sm. However, since lboot treats all files in /var/sysgen/system as a single file, you can create a small file unique to your driver. The name of this file is not significant, but a good name is the driver name with the suffix .sm.
The autoconfig script invokes lboot to read the system files, read the master files, and link all the kernel executables. Provided there are no errors, the output is a new file /unix.install. At boot time this file is moved to the name /unix and used as the kernel.
During the testing period you may want to keep the original kernel file available as /unix.original. A simple way to retain this file is to create a link to it using the ln command.
You compile and configure a loadable driver very much as you would a nonloadable driver (so you should read “Configuring a Nonloadable Driver” before reading this section). The differences are as follows:
For more background on loadable modules, see the mload(4) and ml(1) reference pages.
| Note: You may not call sthread_create() in a loadable driver, because the stack must be in direct mapped (K0) space. The sthreads facility has been superseded by pthreads(5) . |
To be loadable, a driver must specify a pfxdevflag entry point containing the D_MP or D_MT flag (see “Driver Flag Constant” in Chapter 7).
Any loadable module must define a public name pfxmversion, declared as follows:
#include <sys/mload.h> char *pfxmversion = M_VERSION; |
Note the exact spelling of the variable; it is easy to overlook the letter “m” after the prefix. If the variable does not exist or is incorrectly spelled, an attempt to load the driver will fail.
Use the -G 0 option when compiling and linking a loadable driver, since the global option table is not available to a loadable driver. You must also use the -jalr option in a loadable driver (see “Compiler Options”).
In a loadable driver, link using the -r and -d options to retain the symbol table yet generate a bss segment.
The file in /var/sysgen/master.d for a loadable driver has different flags.
In the flags field of the descriptive line of the master file (see “Descriptive Line”), you specify that the driver is loadable, and when it should be loaded. The possible flags are listed in Table 9-5.
Table 9-5. Flag Values for Loadable Drivers
Letter | Meaning |
|---|---|
b or c | Block (b) or character (c) device. One or the other is essential for any device driver. |
f or m | STREAMS driver (f) or STREAMS module (m). Omit for device driver. |
s | Software driver, either a pseudo-device or a SCSI driver. |
d | Specifies that this is a loadable driver. |
R | Auto-register the module (discussed in text). |
D | Load, then unload, at boot time, in order to let the driver initialize the hardware inventory. |
N | Prevent this module from being automatically unloaded even when it has a pfxunload() entry point. |
When the d flag is given for an included module, lboot parses the master file for the driver. Global variables defined in the variables section of the master file (see “Variables Section”) are defined and included in the kernel. However, object code of the driver is not included in the kernel, and the names of its entry points are not entered into the kernel switch table.
Such a driver has to be manually loaded with the ml or lboot command before it can be used; and it cannot be used from the miniroot.
A loadable driver can be loaded by the lboot command at boot time, and by the ml command while the system is running. The following steps occur when a driver is loaded:
The object file header is read.
Memory is allocated for the driver's text, data, and bss segments.
The driver's text and data are read.
The text and data are relocated. References to kernel names and to global variables named in the master file are resolved.
Entry points are noted in the appropriate kernel switch table.
The pfxinit() entry point is called if one is defined.
If the driver is named in a VECTOR statement and has a pfxedtinit() entry point, that entry point is called for each VECTOR statement that names the driver.
Space allocated for the module's text, data, and bss is located in node 0 of an Origin2000 system. Static buffers in loadable modules are not necessarily physically contiguous in memory.
A variety of errors can occur when a module is loaded. See the mload(4) reference page for a list of possible causes.
Normally a loadable driver is not loaded at boot time. It must be loaded sometime after boot using the ml command. When the D flag is included in the descriptive line in the descriptive file, lboot loads the driver at boot time, and immediately after calling pfxstart(), unloads the driver. This permits the driver to test the hardware and set up the hwgraph and hardware inventory.
A loadable module is “registered” by loading it, then placing a stub entry in the pfxopen() and pfxattach() column of its row of the switch table, and unloading it again. The stub entry points are invoked when the driver is needed, and the code of the entry points initiates a load of the driver.
Registration of this kind can be done automatically during bootstrap, or later using the ml command. Once it has been registered, a driver is loaded automatically the first time the kernel needs to attach a device supported by this driver, or the first time a process attempts to open a device special file managed by this driver. You can also load a registered driver in advance of any use with the ml command—loading implies registration.
| Note: Try not to confuse this “registration” with a driver's registration with the kernel to handle a particular type of device. |
Registration is done automatically for each master descriptive file that contains the d (loadable) and R (register) flags. Autoregistration is done at bootstrap phase 2. It is initiated by the script /etc/rc2/S23autoconfig. Registration can be initiated manually at any time after bootstrap by using the ml or lboot command with the reg option (see the ml(1M) and lboot(1M) reference pages).
When a registered driver is reloaded, the sequence of events listed under “Loading” occurs again. There is one exception: the pfxreg() entry point is not called when a registered driver is reloaded from a stub. (The complete sequence occurs when an unregistered driver is explicitly loaded by the ml command.)
A module can be unloaded only when it provides an pfxunload() entry point (see “Entry Point unload()” in Chapter 7). The N flag can be specified in the master file to prevent automatic unloading in any case.
A loaded module is automatically unloaded following a delay after the last close of a device it supports. The delay is configurable using systune, as the module_unld_delay variable (see the systune(1) reference page). You can use ml to specify an unloading delay for a particular module.
The lboot or ml command can be used to unload a module before the delay expires, or to manually override the N flag.
The unload sequence is as follows:
The kernel verifies that all opens of the driver's devices have been closed. The driver cannot be unloaded if it has open devices or active mmaps.
The pfxunreg() entry point is called, if one exists. This gives the driver a chance to unregister as a provider of service for a particular device type. If pfxunreg() returns nonzero, the process stops.
The pfxunload() entry point is called. If it returns nonzero, the process stops.
The module is removed from memory. If it had been registered (R flag), stubs are again placed in the pfxopen() and pfxattach() column of its row of the switch table.
Experience has shown that most of the problems with loadable drivers arise from unloading and reloading. The precautions to take are described under “Entry Point unload()” in Chapter 7.