The SGI computer systems supported by IRIX 6.5 can attach multiple SCSI adapters, as follows:

In all systems, DMA mapping hardware allows a SCSI adapter to treat discontiguous memory locations as if they were a contiguous buffer, providing scatter and gather support.

The IRIX kernel contains two levels of SCSI support. An inner SCSI driver, the host adapter driver, manages all communication with a SCSI hardware adapter. The kernel-level SCSI device driver for a particular device prepares SCSI commands and calls on the host adapter driver to execute them. This design centralizes the management of SCSI adapters. Centralization is necessary because the use of the SCSI bus is shared by many devices, while recovery and error-handling are handled at the adapter level. In addition, use of the host adapter driver makes it simpler to write a SCSI device driver.

When planning a SCSI device driver, it is informative to spend some time exploring the rather complex network of hwgraph vertexes that is set up by the existing SCSI drivers. Your tools for this are the find and grep commands. For example:

find /hw -print | grep scsi | grep -v disk

The result is voluminous even on a relatively small system and reveals that there are many vertexes created for each logical unit (LUN) on each controller. Here is a sample with repetitions edited out:

houston 30% find /hw -print | grep scsi | grep -v disk
/hw/module/1/slot/io1/baseio/pci/0/scsi_ctlr
/hw/module/1/slot/io1/baseio/pci/0/scsi_ctlr/0
/hw/module/1/slot/io1/baseio/pci/0/scsi_ctlr/0/bus
/hw/module/1/slot/io1/baseio/pci/0/scsi_ctlr/0/target
/hw/module/1/slot/io1/baseio/pci/0/scsi_ctlr/0/target/1
/hw/module/1/slot/io1/baseio/pci/0/scsi_ctlr/0/target/1/lun
/hw/module/1/slot/io1/baseio/pci/0/scsi_ctlr/0/target/1/lun/0
/hw/module/1/slot/io1/baseio/pci/0/scsi_ctlr/0/target/1/lun/0/scsi
/hw/module/1/slot/io1/baseio/pci/0/scsi_ctlr/0/target/1/lun/1
...
/hw/module/1/slot/io1/baseio/pci/0/scsi_ctlr/0/target/2
...
/hw/module/1/slot/io1/baseio/pci/1/scsi_ctlr
...
/hw/scsi_ctlr
/hw/scsi_ctlr/0
/hw/scsi_ctlr/1
/hw/scsi
/hw/scsi/sc0d1l0
/hw/scsi/sc0d2l0

/hw/module/1/slot/io1/baseio/pci/0/scsi_ctlr/0/target/1/lun/0/scsi

ls -l /hw/module/1/slot/io1/baseio/pci/0/scsi_ctlr/0/target/1/lun/0
total 0
drwxr-xr-x    2 root     sys            0 Mar 12 14:30 disk
crw-------    1 root     sys        0,116 Mar 12 14:30 scsi

% ls -l /hw/scsi
total 0
crw-------    1 root     sys        0,116 Mar 12 14:41 sc0d1l0
crw-------    1 root     sys        0,133 Mar 12 14:41 sc0d2l0
crw-------    1 root     sys        0,150 Mar 12 14:41 sc0d3l0
crw-------    1 root     sys        0,167 Mar 12 14:41 sc0d4l0
crw-------    1 root     sys        0,184 Mar 12 14:41 sc0d5l0

% ls -S /hw/scsi/sc0d1l0
/hw/scsi/sc0d1l0 ->
/hw/module/1/slot/io1/baseio/pci/0/scsi_ctlr/0/target/1/lun/0/scsi

ls -l /dev/scsi
lrwxr-xr-x    1 root     sys            8 Jan 10 15:33 /dev/scsi->/hw/scsi
% ls -l /dev/scsi/sc0d1l0
crw-------    1 root     sys        0,116 Mar 12 15:21 /dev/scsi/sc0d1l0

% ls -l /hw/scsi_ctlr
total 0
lrw-------    1 root     sys           47 Mar 12 14:42 0 ->
/hw/module/1/slot/io1/baseio/pci/0/scsi_ctlr/0
lrw-------    1 root     sys           47 Mar 12 14:42 1 ->
/hw/module/1/slot/io1/baseio/pci/1/scsi_ctlr/0

% find /hw -print | grep scsi | grep disk | \
sed `s/hw.*_ctlr/.../' | more
/.../0/target/1/lun/0/disk
/.../0/target/1/lun/0/disk/volume
/.../0/target/1/lun/0/disk/volume/char
/.../0/target/1/lun/0/disk/volume_header
/.../0/target/1/lun/0/disk/volume_header/block
/.../0/target/1/lun/0/disk/volume_header/char
/.../0/target/1/lun/0/disk/partition
/.../0/target/1/lun/0/disk/partition/0
/.../0/target/1/lun/0/disk/partition/0/block
/.../0/target/1/lun/0/disk/partition/0/char
/.../0/target/1/lun/0/disk/partition/1
/.../0/target/1/lun/0/disk/partition/1/block
/.../0/target/1/lun/0/disk/partition/1/char
/.../0/target/2/lun/0/disk
...

Some bus protocol features such as connect and disconnect are controlled by configuration files that are used by the host adapter drivers. For example, the wd93 driver has a number of configurable options coded in the descriptive file /var/sysgen/master.d (for the format of descriptive files, see “Describing the Driver in /var/sysgen/master.d” in Chapter 9).

The QLogic driver ql takes its options by the more modern route of the DEVICE_ADMIN statement (see “Storing Device and Driver Attributes” in Chapter 2 and “Retrieving Administrator Attributes” in Chapter 8). You can peruse /var/sysgen/system/irix.sm to see DEVICE_ADMIN statements addressed to “ql_” and associated comments.

---

Note: The connect/disconnect strategy is enabled on any SCSI bus by default (the option is controlled by a constant defined in the host adapter driver descriptive file in /var/sysgen/master.d). When disconnect is enabled on a bus, and a device on that bus refuses to disconnect, it can cause timeouts on other devices.

---

IRIX uses host adapter drivers because the SCSI bus is shared among multiple devices of different types, each type controlled by a different driver. A disk, a tape, a CDROM, and a scanner could all be cabled from the same SCSI adapter. Each device has a different driver, but each driver needs to use the adapter, a single chip-set, to communicate with its device.

If IRIX allowed multiple drivers to operate the host adapter, there would be confusion and errors from the conflicting uses. IRIX puts the management of each host adapter under the control of a host adapter driver, whose job is to issue commands on its bus and report the results. The host adapter driver is tailored to the hardware of the particular host adapter and to the architecture of the host system.

The interface to the host adapter driver is the same no matter what type of hardware the adapter uses. This insulates the individual device drivers from details of the adapter hardware.

The driver for each type of device is responsible for preparing the SCSI command bytes for its device, for passing the command requests to the correct host adapter driver, and for interpreting sense and status data as it comes back.

The host adapter driver is the driver that is called by the kernel to attach the SCSI controller at boot time while the kernel is exploring the hardware and building the hwgraph (see “Entry Point attach()” in Chapter 7).

The host adapter driver places information in the hwgraph vertex that represents the controller, and extends the hwgraph with subordinate vertexes that represent targets and LUNs (see “SCSI Devices in the hwgraph”).

A SCSI driver is called to manage one or more SCSI target devices. Each target is physically connected to a SCSI adapter. The hwgraph echoes this connectivity: the SCSI target vertex is connected to the SCSI adapter vertex. Thus when the SCSI driver knows its target device vertex, it can access the corresponding host adapter vertex, and through this vertex, can invoke the host adapter driver.

Each host adapter driver provides the same functional interface and supports the four functions listed in Table 16-1.

---

Note: The scsi_reset() function that formerly existed has been removed.

---

A SCSI device driver can be asked to manage devices on different adapters. Different adapters may use the same or different hardware, and be managed by the same or different host adapter drivers. How does the driver locate the correct host adapter function for a given device?

The answer is that each host adapter driver places a set of vectors to its functions in the hwgraph vertex for the controller. Using macros defined in sys/scsi.h, the driver invokes the function it needs indirectly, by way of pointers stored in the controller vertex.

Vertex Information Structures

The host adapter driver constructs the arrangement of hwgraph vertexes and data structures illustrated in Figure 16-1.

Figure 16-1. SCSI Vertexes and Data Structures

The main features of this arrangement are as follows:

• The vertex for the controller anchors a scsi_ctlr_info_t, which contains the vectors to the host adapter functions.

• The vertex for any target anchors a scsi_targ_info_t, which contains the target number and a pointer to the scsi_ctlr_info_t for that target's controller.

• The vertex for any LUN anchors a scsi_lun_info_t, which contains the unit number and a pointer to the scsi_targ_info_t for that LUN's target.

Before a SCSI driver tries to access a device, it must call the host adapter scsi_info() function, passing the vertex handle for the LUN. This function issues an Inquiry command to the adapter, target, and logical unit. If the Inquiry is not successful—or if the adapter, target, or LUN are not valid—the return value is NULL. Otherwise, the return value is a pointer to a scsi_target_info structure.

The SCSI driver can learn the following things from a call to scsi_info():

You can also call scsi_info() at other times; some of the returned information can be useful in error recovery. However, be aware that scsi_info() for some host adapters is slow, and can use serialized access to hardware. (See also reference page scsi_info(d3x) .)

Depending on its particular design, the host adapter driver may need to allocate memory for data structures, DMA maps, or buffers for sense and inquiry data, before it is ready to execute commands to a particular target. The call to scsi_alloc() gives the host adapter driver the opportunity to prepare in these ways. (See also reference page scsi_alloc(d3x) .)

Because the host adapter driver may allocate virtual memory, it may sleep. Some host adapter drivers allocate all the resources they need on the first call to scsi_alloc() and do little or nothing on subsequent calls.

A call to scsi_alloc() specifies these parameters:

The option parameter may include the SCSIALLOC_EXCLUSIVE flag to request exclusive use of the target. If another driver has allocated a path to the same device, scsi_alloc() returns EBUSY. For example, a tape device driver might require exclusive access, while a disk device driver would not.

The option parameter may include SCSIALLOC_NOSYNC to specify that this device should not, or cannot, use synchronous transfer mode. That setting can be overridden for single commands by a flag to scsi_command() (see Table 16-4).

The option parameter can also include a small integer value indicating the maximum queue depth (the number of SCSI commands the driver would like to start before any have completed). The call to scsi_info() returns the default queue depth that will be used if you do not pass a nonzero value here (typically the default is 1).

The callback function address can be specified as NULL. The specified callback function is called only when sense data is gotten from the allocated device (regardless of which driver initiated the command that resulted in sense data). Only one driver that allocates a path to a device can specify a callback function. If the path is not held exclusively, any other drivers must specify a null address for their callback functions.

A SCSI driver typically calls scsi_free() from the pfxclose() entry point. That is the time when the driver knows that no processes have the device open, so the host adapter should be allowed to release any resources it is holding just for this device.

In addition, scsi_free() releases the device for use by other drivers, if the driver had allocated it for exclusive use.

A SCSI device driver sends SCSI commands to its device by storing information in a scsi_request structure and passing the structure to the scsi_command() function for the adapter. The host adapter driver schedules the command on the SCSI bus that it manages, and returns to the caller. When the command completes, a notify function is invoked. (See also reference page scsi_command(d3x) .)

---

Tip: When debugging a driver using a debugging kernel (see “Preparing the System for Debugging” in Chapter 10), you can display the contents of a scsi_request structure using symmon or idbg (see “Commands to Display I/O Status” in Chapter 10).

---

The purpose of the scsi_abort() function is to abort all pending or executing commands on a device. The prototype of the function is:

SCI_ABORT(SLI_CTLR_INFO(scsi_lun_info_get(lun_vhdl))
               (struct scsi_request *req);

The only fields of the scsi_request that are input to this function are those that identify the device: sr_dev_vhdl (always!), sr_ctlr, sr_target, and sr_lun. The ABORT command is issued on the bus as soon as possible but there could be a delay if the bus is busy. Status is returned in sr_status. The function returns a nonzero value when the ABORT command is issued successfully, and a zero when the ABORT command fails (which probably indicates a serious bus problem).

---

Note: Not all devices and not all host adapters support this operation. Error recovery of queued commands is up to the driver.

---

A SCSI driver can be either a block or a character driver, or it can support both interfaces. When preparing the descriptive file for /var/sysgen/master.d, you must use the s flag, specifying a software-only driver, and list scsi as a dependency in the description line. See “Describing the Driver in /var/sysgen/master.d” in Chapter 9.

Registration is a step that tells the kernel how to associate a device with a driver. The driver must register with the kernel or it will not be able to access a device.

At boot time, the host adapter driver discovers the complement of devices by probing the bus. A SCSI device is identified by its device type, a number defined as shown in Table 16-9

In addition to the device type, SCSI devices supply vendor ID and product ID strings. When the kernel finds a device, it needs to associate it with a driver. For SCSI devices, the kernel looks through a list of drivers that have registered as supporting SCSI devices of the particular type. If a driver of that type has registered, and the kernel finds an entry for a driver of that type with vendor ID and product ID strings that match the ones found at device discovery, it calls pfxattach() (see “About Attaching a Device”).

Registration tables for driver types are defined in master.d/scsi.h. The entries in scsi_drivers[] list the device types supported (by default, only the type 1 table is defined, but you may define other types):

scsi_type_reg_s scsi_drivers[] = {
{ 0, NULL                    },      /* Type 0 driver reg table */
{ 0, scsi_drivers_type1      },      /* Type 1 driver reg table */
{ 0, NULL                    },      /* Type 2 driver reg table */
{ 0, NULL                    },      /* Type 3 driver reg table */
{ 0, NULL                    },      /* Type 4 driver reg table */
{ 0, NULL                    },      /* Type 5 driver reg table */
{ 0, NULL                    },      /* Type 6 driver reg table */
{ 0, NULL                    },      /* Type 7 driver reg table */
{ 0, NULL                    },      /* Type 8 driver reg table */
{ 1, NULL                    },      /* Terminator - don't remove */
};

A driver is associated with a device type in the master.d/scsi file with a four part entry for the specific table type. The four fields are strings that contain the SCSI vendor ID, the SCSI product ID, the driver prefix, and a hwgraph pathname component.

For example, consider the following entry for a type 1 (tape) driver:

scsi_driver_reg_s scsi_drivers_type1[] = {
    // Type 1 - sequential access devices, tapes //
    { “Fujitsu”, “Diana-1”, “fuj”, “fujitsu-tape” },
    { NULL, NULL, NULL, NULL }
};

The vendor ID is Fujitsu, the product ID is Diana-1, the prefix is fuj, and the hwgraph pathname component is fujitsu-tape. So, if a SCSI Inquiry on the device at device discovery time returns a type 1, and the vendor and product ID returned are Fujitsu and Diana-1, the kernel will address the driver entry points with the prefix fuj, passing the hwgraph pathname suffix fujitsu-tape.

Your driver registers by calling the scsi_driver_register() function (see master.d/scsi):

int scsi_driver_register(int unit_type, char *driver_prefix)

This call specifies the SCSI device type int (see Table 16-9) and the driver's prefix character string that you define. The prefix string is configured in the driver's descriptive file (see “Describing the Driver in /var/sysgen/master.d” in Chapter 9). The kernel uses this string to find the addresses of driver entry points. Note that you may call this function multiple times if your driver supports more than one SCSI device type.

You should call scsi_driver_register() from the pfxreg() entry point. Be aware that, if there is an available device of the specified type, pfxattach() can be called immediately, before the scsi_driver_register() function returns.

The order in which drivers are called to attach a device is not defined.

At device discovery during the boot sequence, the kernel identifies SCSI devices by device type and by the vendor ID and product ID strings. It then searches the device type table for matching strings (see “About Registration”). When it finds a match, it uses the associated prefix string in the table entry to call pfxattach() and passes the hwgraph vertex handle, which represents the “connection point” of the device—typically the LUN vertex handle (for example .../scsi_ctlr/0/target/0/lun/0). The driver adds more vertexes connected to this one to represent the logical devices. The handle of the connection point is needed in several kernel functions, so you should save it as part of the device information.

When the pfxopen() entry point is called, the SCSI driver uses the appropriate scsi_info() function to verify the device and get hardware dependent Inquiry data from it. If the device is not operational, the driver can return ENODEV. If the device is operational, the driver calls scsi_alloc() to open a communications path to it.

The pfxopen entry point is passed the edge vertex, which you can use with device_info_get to access the device info pointer (see “Hardware Graph Management” in Chapter 8).

Refer to “Host Adapter Facilities” for information on how your driver can interact with the device.

In general, it is simplest to put all access to a device within a pfxstrategy() entry point, even in a character device driver. When the pfxread(), pfxwrite(), or pfxioctl() entry point needs to read or write data, it can prepare a uio_t to describe the data, and call uiophysio() to direct the operation through the single pfxstrategy() entry point (see “Calling Entry Point strategy() From Entry Point read() or write()” in Chapter 7).

The notify routine passed in the sr_notify field plays the same role as the pfxintr() entry point in other device drivers. It is called asynchronously, when the SCSI command completes. It may not call a kernel function that can sleep. However, it does not have to be named pfxintr(), and a SCSI driver does not have to provide a pfxintr() entry point.

Your pfxdetach entry point is where you remove hwgraph vertexes added with pfxattach. Note that if you create aliases with an ioctl (see “Creating Device Aliases”), you should remove them in your pfxdetach routine as well. As a result of this practice, the hwgraph will represent the actual available devices.

When a loadable SCSI driver is called at its pfxunload() entry point, indicating that the kernel would like to unload it, the driver must take pains not to leave any dangling pointers (as discussed under “Entry Point unload()” in Chapter 7). A driver should not unload when it has any registered interrupt or error handlers.

A driver does not have to unregister itself as a SCSI driver before unloading. Nor does it have to detach any devices it has attached. However, if any devices are open or memory mapped, the driver should not unload.

If the driver has been autoregistered (see “Registration” in Chapter 9), stub functions are placed in the switch tables for the attach and open functions. When the kernel discovers a new device and wants this driver to attach it, or when a process attempts to open a device for which this driver created the vertex, the kernel reloads the driver.

A device alias is a convenient shorthand path which refers to the same device as the full hwgraph entry. If you want your driver to create aliases for hwgraph entries, create a file in /var/sysgen/ioconfig, for example, /var/sysgen/ioconfig/xyz. This file allows you to choose a stable controller number for your device alias, and to specify an ioctl number used by ioconfig and your driver to create the alias. (See /var/sysgen/ioconfig/README and ioconfig(1M) for details of the file syntax.)

Create a pfxioctl entry point that is responsible for creating device aliases, for example, a path under /hw/disk and /hw/rdisk corresponding to the actual hwgraph entry. The pfxioctl entry point might be called with XYZ_ALIAS, for example, which is a numerical value specified in the last entry in /var/sysgen/ioconfig/xyz.

Use device_controller_num_get(dev) (see sys/invent.h), where dev is the hwgraph vertex on which your driver added inventory information, to get the controller number that has been assigned by ioconfig. The controller number supplied will be the starting one claimed in /var/sysgen/ioconfig, so your aliases will remain associated with the actual hwgraph entries.

For example, if you have specified that controller numbers should start at 10, you can be assured that you will always use 10 as your first controller number and, by picking a relatively high number, your driver should have no effect on ioconfig's default controller number assignments for other controllers of the class.

Refer to “Convenience Vertexes” for information on how the SCSI host adapter driver creates the hwgraph aliases.

The host adapter drivers send error messages to the system log using the cmn_err() function (see “Producing Diagnostic Displays” in Chapter 10).

These messages almost always contain the adapter number (sometimes called the bus number or controller number). They sometimes contain the number of the target device, and sometimes add the number of the logical unit that was addressed.

Messages from the wd93 driver specify the adapter number as Bus=n. The target device is shown as ID=n and the logical unit as LUN=n.

Messages from the wd95 and jag drivers contain one, two, or three or more decimal numbers. In all cases, the first number is the adapter number, the second is the target ID, and the third (when present) is the logical unit number.

When error messages list a sense code, refer to “SCSI Sense Codes (Table scsi_key_msgtab) ” and to “Additional Sense Codes (Table scsi_addit_msgtab)”.

When the error message reports an error from the adapter itself, refer to “Adapter Error Codes (Table scsi_adaperrs_tab)”.

The scsi module contains a set of error message tables that you can use to generate error messages based on SCSI sense codes and other data. The contents of these tables is documented here for reference, and to assist in decoding messages from SCSI drivers.

Each table is an array of pointers to strings; for example, the definition of the scsi_key_msgtab table begins as follows:

char *scsi_key_msgtab[SC_NUMSENSE] = {
   "No sense",          /* 0x0 */
   "Recovered Error",   /* 0x1 */
...};

Each of the tables is declared as extern in sys/scsi.h.