As discussed in “CPU Access to Device Registers” in Chapter 1, an I/O device is represented as an address, or range of addresses, in the address space of its bus. A kernel-level device driver has the ability to set up a mapping between an address on an I/O bus and an arbitrary location in the address space of a user-level process. When this has been done, the bus location appears to be a variable in memory. The program can assign values to it, or refer to it in expressions.

The PCI bus addresses managed by a device are not wired or jumpered into the board; they are established dynamically at the time the system attaches the device. The assigned bus addresses can vary from one day to the next, as devices are added to or removed from that PCI bus adapter. For this reason, you cannot program the bus addresses of a PCI device into software or into a configuration file.

In order to map bus addresses for a particular device, you must open the device special file that represents that device. You pass the file descriptor for the opened device to the mmap() function. If the device driver for the device supports memory mapping—mapping is an optional feature of a PCI device driver—the mapping is set up.

The PCI bus defines three address spaces: configuration space, I/O space, and memory space. It is up to the device driver which of the spaces it allows you to map. Some device drivers may set up a convention allowing you to map in different spaces.

Device special files for PCI devices are established in the /hw filesystem by the PCI device driver when the device is attached; see “Hardware Graph” in Chapter 2. These pathnames are dynamic. Typically, the system administrator also creates stable, predictable device special files in the /dev filesystem. The path to a specific device is determined by the device driver for that device.

The PCI bus adapter also creates a set of generic PCI device names for each PCI slot in the system. The names of these special files can be displayed by the following command:

find /hw -name pci  -print -exec ls -l {} \; 
/hw/module/1/slot/io1/xwidget/pci/0
total 0
crw-------    0 root     sys        0, 78 Aug 12 15:27 config
crw-------    0 root     sys        0, 79 Aug 12 15:27 default
crw-------    0 root     sys        0, 77 Aug 12 15:27 io
crw-------    0 root     sys        0, 75 Aug 12 15:27 mem
/hw/module/1/slot/io1/xwidget/pci/1
total 0
crw-------    0 root     sys        0, 85 Aug 12 15:27 config
crw-------    0 root     sys        0, 86 Aug 12 15:27 default
crw-------    0 root     sys        0, 84 Aug 12 15:27 io
crw-------    0 root     sys        0, 82 Aug 12 15:27 mem

The names are not leaf vertexes and cannot be opened. However, the names config, io, mem, and default are character special devices that can be opened from a process with the correct privilege. The names represent the following bus addresses:

---

Note: With pciba under IRIX 6.5 it is no longer possible to access config space directly by means of mmap() I/O—ioctl() calls must be used instead.

---

When you have successfully opened a pciba device special file, you use the file descriptor as the primary input parameter in a call to the mmap() system function.

This function is documented for all its many uses in the mmap(2) reference page. For purposes of mapping a PCI device into memory, the parameters should be as follows (using the names from the reference page):

The meaning of the off value depends on the PCI bus address space represented by the device special file, as indicated in Table 4-1.

The value returned by mmap() is the virtual address that corresponds to the starting PCI bus address. When the process accesses that address, the access is implemented by PIO data transfer to or from the PCI bus.

When the PCI bus adapter reports an addressing or access error, the error is reflected back to the device driver. This can take place long after the instruction that initiated the error transaction. For example, a PIO store to a memory-mapped PCI device can (in certain hardware architectures) pass through several layers of translation. An error could be detected several microseconds after the CPU store that initiated the write. By that time, the CPU could have executed hundreds more instructions.

When the pciba device driver is notified of a PCI Bus error, it looks up the identities of all user processes that had mapped the part of PCI address space where the error occurred. The driver then sends a SIGBUS signal to each such process. As a result of this policy, your process could receive a SIGBUS for an error it did not cause; and when your process did cause the error, the signal could arrive a long time after the erroneous transaction was initiated.

The code in X demonstrates how to dump the standard configuration space registers of a device in PCI slot 1 on an Origin200 (PCI slot 1 is XIO bus slot 5 on this system).

/*
 * Use pciba to dump the registers found 
 * using base address register 0.
 *
 * See pciba(7m).
 */
#include <sys/types.h>
#include <sys/mman.h>
#include <sys/fcntl.h>
#include <sys/prctl.h>
#include <unistd.h>
#include <stdio.h>
/*
 * Path assumes O2000/Onyx2 PCI shoebox installed
 * in first CPU module.
 */
#define MEMPATH "/hw/module/1/slot/io2/pci_xio/pci/2/base/0"
#define MEMSIZE (128)
extern int errno;
main(int argc, char *argv[])
{
        volatile u_int *word_addr;
        int     fd;
        char    *path;
        int     size, newline = 0;
        path = MEMPATH;
        size = MEMSIZE;
        fd = open(path, O_RDWR);
        if (fd < 0 ) {
                perror("open ../base/0 ");
                return errno;
        } else {
                printf("Successfully opened %s fd: %d\n", path, fd);
                printf("Trying mmap\n");
                word_addr = (unsigned int *)
                     mmap(0,size,PROT_READ|PROT_WRITE,MAP_SHARED,fd,0);
                if (word_addr == (unsigned int *)-1) {
                        perror("mmap");
                } else {
                        int     i;
                        volatile int    x;
                        printf("Dumping registers \n");
                        for (i = 0; i < 32; i++){
                                x = *(volatile int *)(word_addr + i) ;
                                if (newline == 0) {
                                        printf("0x%2.2x:", i*4);
                                }
                                printf(" 0x%8.8x", x);
                                if ((++newline%4) == 0){
                                        newline = 0;
                                        printf("\n");
                                }
                        }
                }
                close (fd);
        }
        exit(0);
} 

As discussed in “CPU Access to Device Registers” in Chapter 1, an I/O device is represented as an address or range of addresses in the address space of its bus. A kernel-level device driver has the ability to set up a mapping between the bus address of a device register and an arbitrary location in the address space of a user-level process. When this has been done, the device register appears to be a variable in memory—the program can assign values to it, or refer to it in expressions.

VECTOR: bustype=EISA module=if_ec3 ctlr=1
iospace=(EISAIO,0x1000,0x1000)
exprobe_space=(r,EISAIO, 0x1c80,4,0x6010d425,0xffffffff)

Using the mmap() Function

When you have successfully opened the device special file, you use the file descriptor as the primary input parameter in a call to the mmap() system function.

This function is documented for all its many uses in the mmap(2) reference page. For purposes of mapping EISA devices, the parameters should be as follows (using the names from the reference page):

addr	Should be NULL to permit the kernel to choose an address in user process space.
len	The length of the span of bus addresses, as documented in the iospace group in the VECTOR line.
prot	PROT_READ, or PROT_WRITE, or the logical sum of those names when the device is used for both input and output.
flags	MAP_SHARED, with the addition of MAP_PRIVATE if this mapping is not to be visible to child processes created with the sproc() function (see the sproc(2) reference page).
fd	The file descriptor from opening the device special file in /dev/eisa.
off	The starting bus address, as documented in the iospace group in the VECTOR line.

The value returned by mmap() is the virtual memory address that corresponds to the starting bus address. When the process accesses that address, the access is implemented by data transfer to the EISA bus.

---

Note: When programming EISA PIO, you must always be aware that EISA devices generally store 16-bit and 32-bit values in “small-endian” order, with the least-significant byte at the lowest address. This is opposite to the order used by the MIPS CPU under IRIX. If you simply assign to a C unsigned integer from a 32-bit EISA register, the value will appear to be byte-inverted.

---

The EISA bus adapter is a device on the GIO bus. The GIO bus runs at either 25 MHz or 33 MHz, depending on the system model. Each EISA device access takes multiple GIO cycles, as follows:

Table 4-2 summarizes best-case (no EISA wait states) data rates for reading and writing a 32-bit EISA device, based on these considerations.

Table 4-3 summarizes the best-case (no wait state) data rates for reading and writing a 16-bit EISA device.

As discussed in “CPU Access to Device Registers” in Chapter 1, an I/O device is represented as an address, or range of addresses, in the address space of its bus. A kernel-level device driver has the ability to set up a mapping between the bus address of a device register and a location in the address space of a user-level process. When this has been done, the device register appears to be a variable in memory. The program can assign values to it, or refer to it in expressions.

Opening a Device Special File

When you know the device addresses, you can open a device special file that represents the correct range of addresses. The device special files for VME mapping are found in the hardware graph at paths having the form:

/hw/module/mod/slot/ion/baseio/vme_xtown/pci/7/vmebus/usrvme/assm/width

The naming convention for these hwgraph paths is documented in the usrvme(7) reference page. Briefly, each path contains these variable elements:

mod	The Origin or Onyx2 module number.
n	The XIO slot number of the VME adapter.
ss	The address space, either 16, 24, or 32.
m	VME address modifier, s for supervisory or n for nonprivileged.
width	Data width to be used, for example d32; covered in later table.

Shorter names are also created in the form

/hw/vme/busnumber/usrvme/assm/width

---

Tip: In previous versions of IRIX, comparable device special files were defined in the /dev directory using names such as /dev/vme/vme0a16n and the like. If you have code that depends on these names—or if you prefer the shorter names in /dev—feel free to create compatible names in /dev in the form of symbolic links to the /hw.../usrvme names.

---

The data width that is designated in the pathname as width can be selected from the values shown in Table 4-4.

Table 4-4. Data Width Names in VME Special Device Names

Address Space in Pathname	Supported Widths in Pathname
a16n, a16s	d16, d32
a24n, a24s	d16, d32
a32n, a32s opened for PIO access	d8, d16, d32_single
a32n, a32s opened for DMA access	d8, d16, d32_single, d32_block, d64_single, d64_block

Opening a device for DMA use is described under “VME User-Level DMA ”.

---

Tip: You can display all the usrvme devices in the system using the find command in the /hw directory, as in

---

# find /hw -name /hw/vme/\*/usrvme/\*/\* -type c -print

Once a VME device has been mapped into memory, your program reads from the device by referencing the mapped address, and writes to the device by storing into the mapped address.

Typically you organize the mapped space using a data structure that describes the layout of registers. Two key points to note about the mapped space are:

On an Origin or Onyx2 system, a PIO read can take one or more microseconds to complete—a time in which the R10000 CPU can process many instructions from memory. The R10000 continues to execute instructions following the PIO load until it reaches an instruction that requires the value from that load. Then it stalls until the PIO data arrives from the device.

A PIO write is asynchronous at the hardware level. The CPU executes a register-store instruction that is complete as soon as the physical address and data have been placed on the system bus. The actual VME write operation on the VME bus can take 1 or more microseconds to complete. During that time the CPU can execute dozens or even hundreds more instructions from cache memory.

All DMA engine transfers are initiated by a special device driver. However, you do not access this driver through open/read/write system calls. Instead, you program it through a library of functions. The functions are documented in the vme_dma_engine(3) reference page. They are used in the following sequence:

In prior releases, user-level DMA was provided through a comparable library of functions with different names and calling sequences. That library of functions is supported in the current release (see a prior edition of this manual, and the udmalib(3) reference page if installed). The new library described here is recommended.