There is no data type associated with the edge as such. The data type of a graph vertex is the vertex_hdl_t, an opaque, 32-bit number. When you create a vertex, a vertex_hdl_t is returned. When you store data in a vertex, or get data from one, you pass a vertex_hdl_t as the argument.

The device number type, dev_t, is an important type in classical driver design (see “Device Number Types”). In IRIX 6.4, the dev_t and the vertex_hdl_t are identical. That is, when a driver is called to open or operate a device that is represented as a vertex in the hardware graph, the value passed to identify the device is simply the handle to the hwgraph vertex for that device.

When a driver is called to open a device that is only represented as a special file in /dev (as in IRIX 6.3 and earlier—there are no such devices supported by IRIX 6.4, but such support is provided for third-party drivers in IRIX 6.5), the identifying value is an o_dev_t, containing major and minor numbers and identical to the traditional dev_t.

Most hwgraph functions have graph error codes as their explicit result type. The graph_error_t is an enumeration declared in sys/graph.h (included by sys/hwgraph.h) having these values:

One uio_t describes data transfer to or from a single address space, either the address space of a user process or the kernel address space. The address space is indicated by a flag value, either UIO_USERSPACE or UIO_SYSSPACE, in the uio_segflg field.

The total number of bytes remaining to be transferred is given in field uio_resid. Initially this is the total requested transfer size.

Although the transfer is to a single address space, it can be directed to multiple segments of data within the address space. Each segment of data is described by a structure of type iovec_t. An iovec_t contains the virtual address and length of one segment of memory.

The number of segments is given in field uio_iovcnt. The field uio_iov points to the first iovec_t in an array of iovec_t structures, each describing one segment. of data. The total size in uio_resid is the sum of the segment sizes.

For a simple data transfer, uio_iovcnt contains 1, and uio_iov points to a single iovec_t describing a buffer of 1 or more bytes. For a complicated transfer, the uio_t might describe a number of scattered segments of data. Such transfers can arise in a network driver where multiple layers of message header data are added to a message at different levels of the software.

In the pfxread() and pfxwrite() entry points, you can test uio_segflag to see if the data is destined for user space or kernel space, and you can save the initial value of uio_resid as the requested length of the transfer.

In a character driver, you fetch or store data using functions that both use and modify the uio_t. These functions are listed under “Transferring Data Through a uio_t Object”. When data is not immediately available, you should test for the FNDELAY or FNONBLOCK flags in uio_fmode, and return when either is set rather than sleeping.

The fields of the buf_t are declared in sys/buf.h, which is included by sys/ddi.h. This header file also declares the names of many kernel functions that operate on buf_t objects. (Many of those functions are not supported as part of the DDI/DKI. You should only use kernel functions that have reference pages.)

Because buf_t is used by so many software components, it has many fields that are not relevant to device driver needs, as well as some fields that have multiple uses. The relevant fields are summarized in Table 8-1.

No other fields of the buf_t are designed for use by a driver. In Table 8-1, “read-only” access means that the driver should never change this field in a buf_t that is owned by the kernel. When the driver is working with a buf_t that the driver has allocated (see “Allocating buf_t Objects and Buffers”) the driver can do what it likes.

The logical block number is the number of the 512-byte block in the device. The “device” is encoded by the minor device number that you can extract from b_edev. It might be a complete device surface, or it might be a partition within a larger device (for example, the IRIX disk device drivers support different minor device numbers for different disk partitions).

The pfxstrategy() routine may have to translate the logical block number based on the driver's information about device partitioning and device geometry (sector size, sectors per track, tracks per cylinder).

The data buffer represented by a buf_t can be in one of two places, depending on bits in b_flags.

When the macro BP_ISMAPPED(buf_t-address) returns true, the buffer is in kernel virtual memory and its virtual address is in b_un.b_addr.

When BP_ISMAPPED(buf_t-address) returns false, the buffer is described by a chain of pfdat structures (declared in sys/pfdat.h, but containing no fields of any use to a device driver). In this case, b_un.b_addr contains only an offset into the first page frame of the chain. See “Managing Buffer Virtual Addresses” for a method of mapping an unmapped buffer.

Historically, a driver used the major device number to learn which device driver has been called. This was important only when the driver supported multiple interfaces, for example both character and block access to the same hardware.

Also historically, the driver used the minor device number to distinguish one hardware unit from another. A typical scheme was to have an array of device-information structures indexed by the minor number. In addition, mode of use options were encoded in the minor number, as described under “Minor Device Number” in Chapter 2.

You can still use major and minor numbers the same way, but only when the device is represented by a device special file that is created with the mknod command, so that it contains meaningful major and minor numbers. The kernel functions related to dev_t use are summarized in Table 8-2.

The most important of the functions in Table 8-2 are

The makedevice() function, which combines a major_t and a minor_t to form a traditional dev_t, is useful only when creating a “clone” driver (see “Support for CLONE Drivers” in Chapter 22).

When the device is represented as a hwgraph vertex, the driver does not receive useful major and minor numbers. Instead, the driver uses the device-unique information that the driver itself has stored in the hwgraph vertex.

An historical driver makes only historical use of the dev_t, using the functions listed in the preceding topic. Such a driver makes no use of the hwgraph, and can only manage devices that are opened as device special files in /dev.

A contemporary driver creates hwgraph vertexes to represent its devices (see “Extending the hwgraph”); makes no use of the major and minor device numbers; and uses the dev_t as a handle to a hwgraph vertex. Such a driver can only manage devices that are opened as device special files in /hw, or devices that are opened through symbolic links in /dev that refer to /hw.

It might possibly be necessary to merge the two approaches. This can be done as follows. In each upper-half entry point, apply getemajor() to the dev_t. When the result is nonzero, the dev_t is conventional and geteminor() will return a useful minor number. Use it to locate the device-specific information.

When getemajor() returns 0, the dev_t is a vertex handle. Use device_info_get() to retrieve the address of device-specific information.

Table 8-6 summarizes the functions you use to allocate and free the pollhead structure that is used within the pfxpoll() entry point (see “Entry Point poll()” in Chapter 7). Typically you would call phalloc() while initializing each minor device, and call phfree() in the pfxunload() entry point.

There are symmetrical pairs of functions to allocate and free all types of lock and synchronization objects. These functions are summarized together with the other locking functions under “Waiting and Mutual Exclusion”.

The argument to the pfxstrategy() entry point is a buf_t structure that describes a buffer (see “Entry Point strategy()” in Chapter 7 and “Structure buf_t”).

Ordinarily, both the buf_t and the buffer are allocated and initialized by the kernel or the filesystem that calls pfxstrategy(). However, some drivers need to create a buf_t and associated buffer for special uses. The functions summarized in Table 8-7 are used for this.

To allocate a buf_t and its associated buffer in kernel virtual memory, use either geteblk() or ngeteblk(). Free this pair of objects using brelse(), or by calling biodone().

You can allocate a buf_t to describe an existing buffer—one in user space, statically allocated in the driver, or allocated with kmem_alloc()—using getrbuf(). Free such a buf_t using freerbuf().

The bcopy() and bzero() functions are used to copy and clear data areas within the kernel address space, for example driver buffers or work areas. These are optimized routines that take advantage of available hardware features.

The bcopy() function is not appropriate for copying data between a buffer and a device; that is, for copying between virtual memory and the physical memory addresses that represent a range of device registers (or indeed any uncached memory). The reason is that bcopy() uses doubleword moves and any other special hardware features available, and devices many not be able to accept data in these units. The hwcpin() and hwcpout() functions copy data in 16-bit units; use them to transfer bulk data between device space and memory. (Use simple assignment to move single words or bytes.)

The copyin() and copyout() functions take a kernel virtual address, a process virtual address, and a length. They copy the specified number of bytes between the kernel space and the user space. They select the best algorithm for copying, and take advantage of memory alignment and other hardware features.

If there is no current context, or if the address in user space is invalid, or if the address plus length is not contained in the user space, the functions return -1. This indicates an error in the request passed to the driver entry point, and the driver normally returns an EFAULT error.

The functions fubyte(), subyte(), fuword(), and suword() are used to move single items to or from user space. When only a single byte or word is needed, these functions have less overhead than the corresponding copyin() or copyout() call. For example you could use fuword() to pick up a parameter using an address passed to the pfxioctl() entry point. When transferring more than a few bytes, a block move is more efficient.

The functions summarized in Table 8-13 are used to explicitly create and manage alenlists. For details see reference page alenlist_ops(d3x).

Typically you create an alenlist implicitly, as a side-effect of loading it (see next topic). However you can use alenlist_create() to create an alenlist. Then you can be sure that there will never be an unplanned delay for memory allocation while using the list.

Whenever the driver is finished with an alenlist, release it using alenlist_destroy().

The functions summarized in Table 8-14 are used to populate an alenlist with one or more address/length pairs to describe memory.

Each of the functions buf_to_alenlist(), kvaddr_to_alenlist(), and uvaddr_to_alenlist() take an alenlist address as their first argument. If this address is NULL, they create a new list and use it. If the input list is too small, any of the functions in Table 8-14 can allocate a new list with more entries. Either of these allocations may sleep. In order to avoid an unplanned delay, you can create an alenlist in advance, fill it to a planned size with null items, and clear it.

The functions buf_to_alenlist(), kvaddr_to_alenlist(), and uvaddr_to_alenlist() add entries to an alenlist to describe the physical address of a buffer. Before using uvaddr_to_alenlist() you must be sure that the pages of the user buffer are locked into memory (see “Converting Virtual Addresses to Physical”).

The kernel support for the PCI bus includes functions that translate an entire alenlist from physical memory addresses to corresponding addresses in the address space of the target bus. For PCI functions see “Mapping an Address/Length List” in Chapter 21.

You use a cursor to read out the address/length pairs from an alenlist. The cursor management functions are summarized in Table 8-15 and detailed in reference page alenlist_ops(d3x).

Each alenlist includes a built-in cursor. If you know that only one process or thread is using the alenlist, you can use this built-in cursor. When more than one process or thread might use the alenlist, each must create an explicit cursor. A cursor is associated with one alenlist and must always be used with that alenlist.

The functions that retrieve data based on a cursor are summarized in Table 8-16.

The alenlist_get() function is the key function for extracting data from an alenlist. Each call returns one address and its associated length. However, these address/length pairs are not required to match exactly to the items in the list. You can extract address/length pairs in smaller units. For example, suppose the list contains address/length pairs that describe 4 KB pages. You can read out sequential address/length pairs with maximum lengths of 512 bytes, or any other smaller length. The cursor remembers the position in the list to the byte level.

You pass to alenlist_get() a maximum length to return. When that is 0 or large, the function returns exactly the address/length pairs in the list. When the maximum length is smaller than the current address/length pair, the function returns the address and length of the next sequential segment not exceeding the maximum. In addition, when the maximum length is an integral power of two, the function restricts the returned length so that the returned segment does not cross an address boundary of the maximum length.

These features allow you to read out units of 512 bytes (for example), never crossing a 512-byte boundary, from a list that contains address/length pairs in other lengths. The alenlist_cursor_offset() function returns the byte-level offset between the first address in the list and the next address the cursor will return. 

In some systems, the buffers used for DMA must be aligned on a boundary the size of a cache line in the current CPU. Although not all system architectures require cache alignment, it does no harm to use cache-aligned buffers in all cases. The size of a cache line varies among CPU models, but if you obtain a DMA buffer using the KMEM_CACHEALIGN flag of kmem_alloc(), the buffer is properly aligned. The buffer returned by geteblk() (see “Allocating buf_t Objects and Buffers”) is cache-aligned.

Why is cache alignment necessary? Suppose you have a variable, X, adjacent to a buffer you are going to use for DMA write. If you invalidate the buffer prior to the DMA write, but then reference the variable X, the resulting cache miss brings part of the buffer back into the cache. When the DMA write completes, the cache is stale with respect to memory. If, however, you invalidate the cache after the DMA write completes, you destroy the value of the variable X.

The maximum size for a single DMA transfer can be set by the system tuning variable maxdmasz, using the systune command (see the systune(1) reference page). A single I/O operation larger than this produces the error ENOMEM.

The unit of measure for maxdmasz is the page, which varies with the kernel. Under IRIX 6.2, a 32-bit kernel uses 4 KB pages while a 64-bit kernel uses 16 KB pages. In both systems, maxdmasz is shipped with the value 1024 decimal, equivalent to 4 MB in a 32-bit kernel and 16 MB in a 64-bit kernel.

There are no legitimate reasons for a device driver to convert a kernel virtual memory address to a physical address in IRIX 6.5. This translation is fraught with complexity and strongly dependent on the hardware of the system. For these and other reasons, the kernel provides a wide variety of address-translation functions that perform the kinds of translations that a driver requires.

In the simpler hardware architectures of past systems, there was a straightforward mapping between the addresses used by software and the addresses used by a bus master for DMA. This is no longer the case. Some of the complexities are sketched under the topic “PIO Addresses and DMA Addresses” in Chapter 1. In the Origin2000 architecture, the address used by a bus master can undergo two or three different translations on its way from the device to memory. There is no way in which a device driver can get the information to prepare the translated address for the device to use.

Instead, the driver uses translations based on opaque software objects such as PIO maps, DMA maps, and alenlists. Translations are bus-specific, and the functions for them are presented in the chapters on those buses.

You can load an alenlist with physical address/length pairs based on a kernel virtual address using buftoalenlist() (see “Loading Alenlists”). Some older drivers might still contain use of the  kvtophys() function, which takes a kernel virtual address and returns the corresponding system bus physical address. This function is still supported (see the kvtophys(D3) reference page). However, you should be aware that the physical address returned is useless for programming an I/O device.

Block device drivers operate upon data buffers described by buf_t objects (see “Structure buf_t”). Kernel functions to manipulate buffer page mappings are summarized in Table 8-17.

When a pfxstrategy() routine receives a buf_t that is not mapped into memory (see “Buffer Location and b_flags”), it must make sure that the pages of the buffer space are in memory, and it must obtain valid kernel virtual addresses to describe the pages. The simplest way is to apply the bp_mapin() function to the buf_t. This function allocates a contiguous range of page table entries in the kernel address space to describe the buffer, creating a mapping of the buffer pages to a contiguous range of kernel virtual addresses. It sets the virtual address of the first data byte in b_un.b_addr, and sets the flags so that BP_ISMAPPED() returns true—thus converting an unmapped buffer to a mapped case.

---

Note: The reference page for the userdma() function is out of date as shipped in IRIX 6.4. The correct prototype for this function, as coded in sys/buf.h, is

---

int userdma(void *usr_v_addr, size_t num_bytes, int rw, void *MBZ);

The fourth argument must be a zero. The return value is not the same as stated. The function returns 0 for success and a standard error code for failure.

Some kernel functions used for ensuring cache coherency are summarized in Table 8-18.

The functions for cache invalidation are essential when doing DMA on a uniprocessor. They cost very little to use in a multiprocessor, so it does no harm to call them in every system. You call them as follows:

In the IP28 CPU you must invalidate the cache both before and after a DMA input; see “Uncached Memory Access in the IP26 and IP28” in Chapter 1.

The flushbus() function is needed because in some systems the hardware collects output data and writes it to the bus in blocks. When you write a small amount of data to a device through PIO, delay, then write again, the writes could be batched and sent to the device in quick succession. Use flushbus() after PIO output when it is followed by PIO input from the same device. Use it also between any two PIO outputs when the device is supposed to see a delay between outputs. 

The basic functions for constructing edges and vertexes are summarized in Table 8-21. The most commonly-used are hwgraph_char_device_add() and hwgraph_block_device_add(), functions that create leaf vertexes that users can open.

Suppose the kernel is probing a PCI bus and finds a veeble device plugged into slot 2. The kernel knows that a driver with the prefix veeble_ has registered to handle this type of device. The kernel calls veeble_attach(), passing the handle of the vertex that represents the point of attachment, which might be /hw/module/1/io/pci/slot/2.

Suppose that a veeble device permits only character-mode access and there are no optional modes of use. In this simple case, the driver needs to add only one vertex, a device special file connected by one edge having a label such as “veeble.” The result will be that the device can be opened under the pathname /hw/module/1/io/pci/slot/2/veeble. Parts of the code in veeble_attach() would resemble Example 8-2.

int veeble_attach(vertex_hdl_t vh)
{
   VeebleDevInfoStruct_t * vdis;
   vertex_hdl_t vv;
   graph_error_t ret;
   /* allocate memory for per-device structure */
   vdis = kmem_zalloc(sizeof(*vdis),KM_SLEEP);  
   if (!vdis) return ENOMEM;
   /* create device vertex below connect-point */ 
   ret = hwgraph_char_device_add(vh, "veeble", "veeble_", &vv);
   if (ret != GRAPH_SUCCESS)
      { kmem_free(vdis); return ret; }  
   /* here initialize contents of vdis->information struct */
   /* here initialize the device itself */
   /* set info struct in the device vertex */
   device_info_set(vv,vdis); 
   return 0;
} 

In Example 8-2, the important variables are:

The steps performed are:

An additional step not shown is the storing of hardware inventory information that can be reported by hinv using device_inventory_add().

A point to note is that in a multiprocessor system, a user process could try to open the new “veeble” vertex as soon as (or even before) hwgraph_char_device_add() returns. This would result in an entry to the veeble_open() entry point of the driver, concurrent with the continued execution of the veeble_attach() entry point. However, note the two statements in Example 8-1:

   devInfo_t *pdi = device_info_get(dev);
   if (!pdi) return ENODEV;

At any time before veeble_attach() executes its call to device_info_set(), a call to veeble_open() for this vertex returns ENODEV. Needless to say, all the hwgraph functions are multiprocessor-aware and use locking as necessary to avoid race conditions.

In a more complicated case, a vooble device permits access as a block device or as a character device. The device should be accessible under names vooble/char and vooble/block. In this case the driver proceeds as follows:

The subordinate block and character vertexes are device special files that can be opened by user code. Handles to these vertexes will be passed in to other driver entry points. There are a variety of ways to store device information in the three vertexes:

As you plan this arrangement of data structures, bear in mind that the pfxopen() entry point receives a flag telling it whether the open is for block or character access (see “Entry Point open()” in Chapter 7); and that other entry points are called only for block, or only for character, devices.

Possibly the device has multiple modes of use, as for example a tape device has byte-swapped and non-swapped access, fixed-block and variable-block access, and so on. Traditionally these modes of access were encoded in the device minor number as well as in the device name (see “Creating Conventional Device Names” in Chapter 2). Current practice is to create a separate vertex for each mode of use (see “Multiple Device Names” in Chapter 2).

When using the hwgraph, you represent each mode of access as a separate name in the /hw filesystem. Suppose that a PCI device of type flipper supports two modes of use, “flipped” and “flopped.” It is the job of the flipper_attach() entry point to set up hwgraph vertexes so that one device can be opened under different pathnames such as /hw/module/1/io/pci/slot/2/flipper/flipped and /hw/module/1/io/pci/slot/2/flipper/flopped. The problem is very similar to creating separate block and character vertexes for one device, with the additional problem that the device information stored in each vertex should reflect the desired mode of use, flipped or flopped. The code might resemble in part that shown in Example 8-3.

typedef struct flipperDope_s {
   vertex_hdl_t floppedMode; /* vertex for flopped */
   ...many other fields for management of one flipper dev...
} flipperDope_t;
int flipper_attach(vertex_hdl_t connv)
{
   flipperDope_t *pfd = NULL;
   vertex_hdl_t masterv = GRAPH_VERTEX_NONE;
   vertex_hdl_t flippedv = GRAPH_VERTEX_NONE;
   vertex_hdl_t floppedv = GRAPH_VERTEX_NONE;
   graph_error_t ret = 0;
   if (!pfd = kmem_zalloc(sizeof(*pfd),KM_SLEEP))  
   { ret = ENOMEM; goto done; }
   ret = hwgraph_vertex_create(&masterv);  
   if (ret != GRAPH_SUCCESS) goto done;
   ret = hwgraph_edge_add(connv,masterv,"flipper");  
   if (ret != GRAPH_SUCCESS) goto done;
   ret = hwgraph_char_device_add(masterv, "flipped", "flipper_", &flippedv);  
   if (ret != GRAPH_SUCCESS) goto done;
   ret = hwgraph_char_device_add(masterv, "flopped", "flipper_", &floppedv);
   if (ret != GRAPH_SUCCESS) goto done;
   pfd->floppedMode = floppedv; /* note which vertex is "flopped" */
...here initialize other fields of pfd->flipperDope...
   device_info_set(flippedv,pfd);
   device_info_set(floppedv,pfd);
done: /* If any error, undo all partial work */
   if (ret)
   {
      if (floppedv != GRAPH_VERTEX_NONE) hwgraph_vertex_destroy(floppedv);
      if (flippedv != GRAPH_VERTEX_NONE) hwgraph_vertex_destroy(flippedv);
      if (masterv != GRAPH_VERTEX_NONE)
      {
         hwgraph_edge_remove(rootv,"flipper",NULL);  
         hwgraph_vertex_destroy(masterv); 
      }
      if (pfd) kmem_free(pfd);  
   }
   return ret;
}

After successful completion of flipper_attach() there are two character special devices with paths /hw/.../flipper/flipped and /hw/.../flipper/flopped. A pointer to a single device information structure (a flipperDope_t object) is stored in both vertexes. However, the vertex handle of the flopped vertex is saved in the floppedMode field of the structure. Whenever the device driver is entered, it can retrieve the device information with a statement such as the following:

flipperDope_t *pfd = device_info_get(dev);

Whenever the driver needs to distinguish between “flipped” and “flopped” modes of access, it can do so with code such as the following:

if (dev == pfd->floppedMode)
{ ...this is flopped-mode...}
else
{ ...this is flipped-mode...}

The driver is allowed to create vertexes and attach them anywhere in the hwgraph. The connection point of a device is often at the end of a long path that is hard for a human to read or type. The driver can use hwgraph_vertex_create() and hwgraph_edge_add() to create a shorter, more readable path to any of the leaf vertexes it creates. For example, the hypothetical veeble_ driver of Example 8-2 might like to make the devices it attaches available via paths like /hw/veebles/1 and /hw/veebles/2.

At the time a driver is called to attach a device, the driver has no way to tell how many of these devices exist in the system. Also, recall that the pfxattach() entry point can be called concurrently on multiple CPUs to attach devices in different slots on different buses. The attach code has no basis on which to assign ordinal numbers to devices; that is, no way to know that a particular device is device 1, and another is device 2. These questions cannot be answered until the entire hardware complement has been found and attached.

The purpose of the ioconfig command is to call drivers one more time, before user processes start but after the hwgraph is complete, so they can create convenience vertexes. This use of ioconfig is described under “Device Management File” in Chapter 2. You direct ioconfig to assign controller numbers to your devices. After it does so, it opens each device (resulting in the first entry to pfxopen() for that device vertex), and optionally issues an ioctl against the open device passing a command number you specify. Upon either the first open of a device or in pfxioctl(), you can create convenience vertexes that include the assigned controller number of the device to make the names unique.

The assigned controller numbers are stable from one boot time to the next, so you can also create symbolic links in /dev naming them.

The use of device_info_set() is discussed under two other topics: “Allocating Storage for Device Information” in Chapter 7 and “Extending the Graph With a Single Vertex”. Every device needs such an information structure—if for no other reason than to contain a lock used to ensure that each upper-half entry point has exclusive use of the device.

When the driver creates multiple vertexes for a particular device, the driver can store the same address in every vertex (as shown in Example 8-2 and Example 8-3). Yet another design option is to have each vertex contain the address of a small structure containing optional information unique to that view of the device, and a pointer to a single common structure for the device.

The device_inventory_add() function stores the fields of one inventory_t record in a vertex. The driver can store multiple inventory_t records in a single vertex, but it is customary to store only one. There is no facility to delete an inventory record from a vertex.

The device_inventory_get_next() function is used to read out each of the inventory_t structures in turn. Normally the driver does not have any reason to inspect these. However, the function does not return a copy of the structure; it returns the address of the actual structure in the vertex. The fields of the structure can be modified by the driver.

One field of the inventory_t is particularly important: the controller number is conventionally used to provide ordinal numbering of similar devices. The device_controller_number_get() function returns the controller number from the first (and usually the only) inventory_t structure in a vertex. It fails if there is no inventory data in the vertex.

When the driver can assign an ordinal numbering to multiple devices, it should record that numbering by setting unique controller numbers in each master vertex for the similar devices. This can be done most easily by calling device_controller_number_set(). Typically this would be done in an ioctl call from the application that has determined a stable, global numbering of devices (see “Device Management File” in Chapter 2).

A file attribute is an arbitrary block of information associated with a file inode. Attributes were introduced with the XFS filesystem (see the attr(1) and attr_get(2) reference pages), but the /hw filesystem also supports them. You can store file attributes in hwgraph vertexes, and they can be retrieved by user processes.

The functions that a driver uses to manage attributes are summarized in Table 8-22 (all are detailed in the reference page hwgraph.lblinfo(d3x)).

An attribute consists of a name (a character string), a pointer-sized integer, and a length. When the length is zero, the attribute is “unexported,” that is, not visible to the attr command nor to the attr_get() function. All attributes are initially unexported. An unexported attribute can be retrieved by a driver, but not by a user process.

The value of an attribute is just a pointer; it can be an integer, a vertex handle, or an address of any kind of information. You can use attributes to hold any kind of information you want to associate with a vertex. (For one example, you could use an attribute to contain mode-bits that determine how a device should be treated.)

Attribute storage is not sophisticated. Attribute names are stored sequentially in a string table that is part of the vertex, and looked up in a sequential search. The attribute scheme is meant for convenient storage of a few attributes per vertex, each having a short name.

When you export an attribute, you assert that the value of the attribute is a valid address in kernel virtual memory, and the export length is its correct length. The attr_get() function relies on these points. A user process can retrieve a copy of an attribute by calling attr_get(). The attribute value is copied from the kernel address space to the user address space. This is a convenient route by which you can export driver internal data to user processes, without the complexity of memory mapping or ioctl calls.

The system administrator can use the DEVICE_ADMIN statement to attach a labelled attribute to any device special file in the hwgraph, and can use DRIVER_ADMIN to store a labelled attribute for the driver (see “Storing Device and Driver Attributes” in Chapter 2).

These statements are processed at boot time. At this time, the driver might not be loaded, and the device special file might not have been created in the hwgraph. However, the attributes are saved. When a driver creates a hwgraph vertex that is the target of a DEVICE_ADMIN statement, the labelled attributes are attached to the vertex automatically.

Your driver can request an administrator attribute for a specific device using hwgraph_info_get_LBL() directly, as described above under “Attaching Attributes”. Or you can call device_admin_info_get() (see the reference page hwgraph.admin(d3x) ). The returned value is the address of a read-only copy of the value string.

Your driver can request an attribute that was addressed to the driver with DRIVER_ADMIN using device_driver_admin_info_get(). The returned value is the address of a read-only copy of the value string from the DRIVER_ADMIN statement.

As their name suggests, mutex locks are designed for mutual exclusion. The IRIX implementation of mutex locks is compatible with the kmutex_t lock type of SunOS, but optimized for use in SGI hardware systems. The mutex functions are summarized in Table 8-25.

Although allocation and deallocation functions are supplied, a mutex_t type is a small object that is normally allocated as a static variable or as a field of a structure. The MUTEX_INIT() operation prepares a statically-allocated mutex_t for use.

Once initialized, a mutex lock is used to gain exclusive use of the resource with which you have associated it. The mutex lock has the following important advantages over a basic lock:

The mutex lock implementation provides priority inheritance. When a low-priority process (or kernel thread) owns a mutex lock and a high-priority process or thread attempts to seize the lock and is blocked, the process holding the lock is temporarily given the higher priority of the blocked process. This hastens the time when the lock can be released, so that a low-priority process does not needlessly impede a higher-priority process.

In order to implement priority inheritance and retain high performance, the mutex lock is subject to the restriction that it must be unlocked by the same process or thread that locked it. It cannot be locked in one process or thread identity and unlocked in another.

You can use mutex locks to coordinate the use of global variables between upper-half entry points of a driver, and between the upper-half code and the interrupt handler. You should prefer a mutex lock to a basic lock in any case where the worst-case program path could hold the lock for a time of 100 microseconds or more.

Mutex locks become inefficient when there is high contention for the lock (that is, when the probability of having to wait is high), because when a process has to wait for a lock, a thread switch takes place. When there is high contention for a lock, it is usually better to use a basic lock, because waiting threads simply spin; they do not execute a context switch.

IRIX supports sleep lock functions that are compatible with SVR4. These functions are summarized in Table 8-26.

Although allocation and deallocation functions are supplied, a sleep_t type is a small object that is normally allocated as a static variable or as a field of a structure. The SLEEP_INIT() operation prepares a statically-allocated sleep_t for use. (In IRIX 6.2, a sleep_t is identical to a sema_t, but this situation could change in a future release.)

A sleep lock is similar to a mutex lock in that it is used for mutual exclusion between processes, and can be held across a function call that sleeps. A sleep lock does not have either the advantages or the restrictions of a mutex lock:

The basic time unit is the “tick.” Its value can differ between hardware platforms and between versions of IRIX. The drvhztousec() and drvusectohz() functions convert between ticks and microseconds in the current system. Use them in order to schedule a delay in a portable manner. (However, the timer function precision is the tick, not the microsecond.)

The “fast tick” is a fraction of a tick. Like the tick, the fast tick's value can differ between systems. Use fasthzto() to convert from microseconds to fast ticks.

Timer support is based on the idea of a “callback” function. You specify the following to dtimeout(), itimeout(), timeout() or fast_itimeout():

After a delay of at least the length requested, the function is called. The function is entered asynchronously. On a uniprocessor, it can interrupt execution of an upper-half routine. On a multiprocessor, it can execute concurrently with an upper-half routine or with an interrupt handler or a different timeout function. (Use locks or mutexes for mutual exclusion.)

The difference between itimeout() and timeout() is that the latter takes no argument values to be passed to the function when it is called. In order to get a repeated series of timer events, start a new timeout from the callback function.

The untimeout() and untimeout_func() functions cancel a pending timeout. In a loadable driver that has an pfxunload() entry point, cancel any pending timeouts before unloading.

The STREAMS_TIMOUT macro supplies similar timeout capability for a STREAMS driver (see “Special Considerations for Multiprocessing ” in Chapter 22).

In rare circumstances, a driver needs to pause briefly between two hardware operations. For example, the SGI support for external interrupts in the Challenge and Onyx computers sometimes needs to set a high output level, wait for a brief, precise interval, then set a low output level.

The drv_usecwait() function supports this type of very short, precisely-timed delay. It “spins” for a specified number of microseconds, then returns to the caller. The CPU does nothing else during this period, so clearly a delay of more than a few microseconds can interfere with other work. Furthermore, if interrupts are disabled during the wait, the response to another interrupt is delayed also—the delay contributes directly to the “latency” of interrupt handling.

The pfxstrategy() entry point is called directly from the filesystem or virtual memory management, or it can be called indirectly from a pfxread() or pfxwrite() entry point (see “Calling Entry Point strategy() From Entry Point read() or write()” in Chapter 7).

Typically the pfxstrategy() routine must interact with its interrupt handler. The pfxstrategy() routine can be designed in either of two ways, synchronous or asynchronous.

The synchronous pfxstrategy() routine initiates every I/O operation. Its interrupt handler is responsible only for detecting and signalling the completion of one I/O. The pfxstrategy() routine proceeds as follows:

When the interrupt handler is entered, the handler uses bioerror() if necessary, and biodone() to signal the completion of the I/O. Then it exits. The strategy code, which is waiting in the call to biowait(), regains control following the call to biodone(), and can use geterror() to check the results.

The asynchronous pfxstrategy() routine only initiates the first I/O operation of a series, and never waits. It proceeds as follows:

When the interrupt occurs, the handler proceeds as follows:

The sleep() and wakeup() function pair are the simplest, oldest, and least efficient of the general synchronization mechanisms. They are summarized in Table 8-31.

Used carefully, these functions are suitable for simple character device drivers. However, when you are writing new code or converting a driver to multiprocessing you should avoid them and use synchronization variables instead (see “Using Synchronization Variables”).

The basic concept is that the upper-layer routine calls sleep(n) in order to wait for an event that is keyed to an arbitrary address n. Typically n is a pointer to a data structure related to an I/O operation. The interrupt handler executes wakeup(n) to cause the sleeping process to resume execution.

The main reason to avoid sleep() is that, in a multiprocessor system, it is hard to ensure that sleeping always begins before wakeup() is called. The usual intended sequence of events is as follows:

In a multiprocessor-aware driver (one with D_MP in its pfxdevflag constant; see “Driver Flag Constant” in Chapter 7), there is a small chance that the interrupt can occur, calling wakeup(n), before the sleep(n) call has been completed. Because sleep() has not been called, the wakeup() is lost. When the sleep() call completes, the process sleeps forever. Synchronization variables are designed to handle this case.

Synchronization variables, a feature of UNIX SVR4, are supported by IRIX beginning with release 6.2. These functions are summarized in Table 8-32.

A synchronization variable is a memory object of type sv_t, representing the occurrence of an event. You can allocate objects of this type dynamically, or declare them as static variables or as fields of structures.

One or more processes may wait for an event using SV_WAIT(). An interrupt handler or timer callback function can signal the occurrence of an event using SV_SIGNAL (to wake up only one waiting process) or SV_BROADCAST (to wake up all of them).

SV_WAIT is specifically designed to handle the difficult case that arises when the driver needs to initiate an I/O operation and then sleep, and do these things in such a way that it always begins to sleep before the SV_SIGNAL can possibly be issued. The procedure is done as follows:

This process is sketched in Example 8-5.

lock_t seize_it;
sv_t wait_on_it;
initiator(...)
{
   int lock_cookie;
   for( as often as necessary )
   {
      lock_cookie = LOCK(&seize_it,PL_ZERO);
      [do something that causes a later interrupt]
      SV_WAIT(&wait_on_it, 0, &seize_it, lock_cookie);
      [interrupt has been handled]
   }
}
 
void handler(...)
{
   int lock_cookie = LOCK(&seize_it,PL_ZERO);
   [handle the interrupt]
   SV_SIGNAL(&wait_on_it);
   UNLOCK(&seize_it);
}

If it is necessary to use a semaphore as the lock, the header file sys/sema.h declares versions of SV_WAIT that accept a semaphore and a synchronization variable. The combination of a mutual exclusion object and a synchronization variable ensures that even in a multiprocessor, the interrupt handler cannot exit before the driver has entered a predictable wait state.

---

Tip: When a debugging kernel is used, you can display statistics about the use of a given synchronization variable. See “Including Lock Metering in the Kernel Image” in Chapter 10.

---

To use a semaphore for locking, initialize it to 1. (This reflects the idea that a process calling a locking function expects to continue.) When you require exclusive use of the associated resource, call psema(). Typically this finds a semaphore count of 1, reduces it to 0, and returns.

When you are finished with the resource, call vsema() to increment the semaphore count, and release any process that is blocked in a psema() call for the same semaphore.

For locking, a semaphore is comparable to a sleep lock. In some systems, the performance of semaphore operations may not be as good as the performance of a mutex lock. In other systems, mutex locks may be implemented using semaphores.

To use a semaphore for waiting, initialize it to 0. Then call psema(). Because the semaphore count is 0, the process waits. When the desired event occurs, typically in the interrupt handler, call vsema() to release the waiting process.

This synchronization method is as reliable as a synchronization variable, but it has slightly different behavior. When a synchronization variable is used correctly (see “Using Synchronization Variables”), if the interrupt handler is entered before the SV_WAIT call completes, the interrupt handler waits on a LOCK call.

When a semaphore is used, if the interrupt handler is entered before the psema() call completes, the vsema() operation is done immediately and the interrupt handler continues without waiting. The fact that vsema() was called is stored as a count within the semaphore, where psema() will find it. Because the semaphore can contain this state information, the interrupt handler does not have to be synchronized in time using a lock.

---

Note: In releases before IRIX 6.2, the vpsema() function was used in a way similar to synchronization variables are used: to release one semaphore and wait on another in an atomic operation. This function is no longer supported; replace it with synchronization variable.

---

IRIX uses interrupt threads to handle most of its physical interrupts. The section titled “Interrupt Entry Point and Handler” in Chapter 7 describes how to create a kernel thread and how to link it to a physical interrupt in one action. Some drivers perform background processing of events and queues that are not tied to particular physical interrupts. User-mode programs that do this are typically called daemons.

For systems running IRIX 6.5.17 and later, drivers can create kernel threads not associated with particular interrupts. These "system" threads can take all types of locks, block on events or resources, and do anything else that interrupt threads can do. For details on their creation and destruction, see the drv_thread_create(D3) and drv_thread_exit(D3) man pages .

Unlike interrupt handlers, most system threads should not return from their starting function until they are ready to destroy their thread. Most threads should use some form of loop, alternating between processing data and waiting for more data from user programs or from interrupt threads. The following example illustrates the creation and operation of a typical system thread:

...

#include <sys/cmn_err.h>
#include <sys/ddi.h>
...

void
example_system_thread(void * arg0,
                      void * arg1,
                      void * arg2,
                      void * arg3)
{
        /*
         * Loop processing events and sleeping waiting for more
         */
        while (1) {
              /*
               * Wait for more events to occur
               */

              /*
               * Do background work
               */
        }

        /*
         * If we need to exit this thread for some reason
         * we call the below.  This is equivalent to just
         * calling return() from the base function.
         */
        drv_thread_exit();
}

void
example_init(void)
{
        int error;
        void * myarg0, * myarg1;
...
        /*
         * Create a system thread to do background work
         */
         error = drv_thread_create("MyThread", 0, 0, 0,
                                   example_system_thread,
                                   myarg0, myarg1, NULL, NULL);

         if (error) {
            cmn_err(CE_WARN, "Creation of MyThread failed\n");
         }
...
}

When the irix.sm DEVICE_ADMIN INTR_TARGET directive is used to direct a physical interrupt, it also binds its interrupt handlers to the target CPU. In some situations, such as when running with the SGI Frame Rate Scheduler (FRS), it is desirable to put interrupt handler threads on CPUs in locations other than where their physical interrupts are directed. Systems running IRIX 6.5.16 or later can use the XThread Control Interface (XTCI) to control special behaviors such as this. Users can add XTHREAD entries in the /var/sysgen/system/irix.sm file. Kernel threads not given entries operate with default behavior. After irix.sm is modified, you should run lboot to reconfigure the system.

To preserve compatibility, in the event that conflicting entries are found, XTCI entries will defer to the legacy /var/sysgen/master.d/sgi interface. As in the master.d/sgi interface, system threads can be specified but they can later change their behavior; whereas interrupt threads must adhere throughout their lifetime.

Specific interface entries are of the following format:

XTHREAD: name[*] [BOOT] [FLOAT] [STACK s] [PRI p] [CPU m...n]

Entry descriptions are as follows:

XTHREAD:		Indicates that any line beginning with XTHREAD: controls kernel threads. All of the information must be on the same line.
name[*]		Indicates that any thread with a name equal to name is affected by the directives that follow it. If * follows, any thread whose name begins with name is affected.
BOOT		Indicates that the thread stays within the boot cpuset, if one exists.
FLOAT		Indicates that the thread will never be bound to a CPU.
STACK s		Specifies the thread stack size.
PRI p		Specifies the starting thread CPU scheduling priority.
CPU m...n		Specifies a list of CPUs on which to attempt to place the thread, if possible. Threads that cannot be placed on their CPU list will be considered FLOAT. This is comparable to the sysmp() MP_MUSTRUN command for user threads. You can list up to four processors. Note: At boot time the XTCI mechanism is enabled before IRIX enables its device drivers but after some of the core IRIX services are initialized. Therefore some kernel threads, such as the timeout threads, are not affected by XTCI entries for them.

The following examples illustrate the use of XTHREAD entries:

XTHREAD: ioc3* FLOAT

On SGI Origin series systems, this entry prevents all of the interrupt handler threads for the IOC3 hardware (including the mouse and keyboard handlers) from being bound to a CPU. This entry is useful for the previously described situation of routing the physical interrupt for the external interrupt (and thus, also the keyboard and mouse) to a CPU running the FRS. Because the FRS controls the CPU, it will not allow mouse and keyboard handlers to run. The FLOAT directive allows them to run on a different CPU.

XTHREAD: vme_intrd0 CPU 2

This example forces the kernel interrupt thread for level 0 VME interrupts to run on processor 2.